We’ve been doing some work behind the scenes to modernize our servers; as part of that, I have moved the scenery tools code to GitHub.
The GitHub public repositories are the official repositories for WED and the other scenery tools that live in the XPTools code base; I push my changes directly there.
using GitHub means much faster clone times (since the server we had the code on was getting long in the tooth), better public browsing, and you can do all of the normal GitHub things that you’d want, like forking, making pull requests, etc.
If you are using our CGIT browsing (at http://dev.x-plane.com/cgit/cgit.cgi/) please switch to GitHub; CGIT will be going away soon. Similarly, if you have the code checked out, you should change your remote’s URL to https://github.com/X-Plane/xptools.git.
I mentioned in a previous post that we are working with Ondrej on a next-generation Blender 2.7 OBJ exporter. I am excited for this work because I believe it will greatly improve the artist interface to X-Plane. Let’s use instancing as an example to explain the idea.
X-Plane 10 will sometimes draw scenery objects using hardware instancing. When an object is drawn with instancing, performance is a lot better than without it; the huge numbers of objects you can see in the autogen scenery are due to instancing. But the conditions for instancing are, to put it mildly, complicated.
- The object has to be attached to a DSF. Well, that’s not crazy.
- There’s a giant pile of things you’re not allowed to do in an instanced object – animation, material attributes, smoke puffs, bla bla bla. (The object has to be simple so that the GPU can draw a big pile of objects without the CPU intervening to handle things like smoke generation on a per-object basis.)
- Buuuuut…not every ATTRibute is a problem. You can use ATTR_hard in an instanced object, no problem! But don’t use ATTR_shiny_rat – that’ll kill instancing.
- But wait, there are these weirdo new attributes like GLOBAL_specular that seem to replace ATTR_shiny_rat that do work with instancing.
If you model, and this seems confusing, it’s because it is. You were never supposed to see this!
File Formats for Machines and Applications For Humans
My view is this: the file formats of X-Plane are meant to be seen by programmers, not artists. When was the last time you had to hand-edit an .acf? You just use Plane-Maker. You don’t hand-edit a PNG file or worry about its internal compression – you use PhotoShop.
OBJ should be the same way – you should export from a 3-d tool and never have to look inside the OBJ itself. This means the OBJ8 format can serve two purposes:
- Help X-Plane load models quickly and draw good looking content and
- Maintain compatibility so older scenery continues to load.
Note that “be really easy to understand in Notepad” is not on the list!
Blender the Transformer
Now here’s the trick: the rules for OBJ that a 3-d modeler can present can be different from the rules that the OBJ8 format itself has. And that is exactly what we are trying to do in the new Blender exporter.
Rather than expose global attributes and non-global attributes and all of the crazy complexity of OBJ8, our strategy with the new exporter is this:
- When you export a scenery object, you specify if you want the object to be instanced or not.
- If the object is not instanced, Blender creates the best OBJ it can for this case.
- If the object is instanced, Blender tries to create the best OBJ it can for this case.
- If the object is supposed to be instanced but you’ve done something that is not instancing-friendly, the exporter tells you, in terms you can understand.
So for example, if you make an animated radar dish that spins and attempt to export with instancing, the exporter can say “you cannot have animation in an instanced object.” Now you, the scenery designer, can make a decision: is it more important to have the radar dish draw really really fast, or is it more important that it spins?
This is the strategy we used for our modifications of Marginal’s 2.49 exporter* – the artist specifies instancing and gets a check that instancing requirements have met. This means instance-tagged objects will be fast. (It is possible to check instancing within the sim with DataRefEditor but it’s not easy, especially for large projects.)
Don’t Expose the Weird
This idea that Blender can transform your work, and thus hide weird X-Plane rules, is not specific to OBJ, not new to X-Plane, and not even specific to X-Plane. When you look at major 3-d game engines (e.g. for combat games and racing games) a ton of code goes into the “art asset pipeline”, transforming the artist’s original work into something already digested for the 3-d engine.
Another example of this is WED. DSFs have an annoying rule that polygons can’t cross DSF boundaries. There are also some even weirder exceptions for a few special cases, introduced in X-Plane 10.20.
WED knows all of these rules; you draw polygons wherever you want and set the export target for the version of X-Plane you want to fly in, and WED figures out the best way to export your work, chopping up polygons as needed.
Shedding Light on Things
Our current work on the Blender exporter focuses on WYSIWYG animation and the material model (which includes instancing support). There is one more area that I think will make a huge difference to authors that we haven’t started working on yet: lighting.
The lighting values for commands like LIGHT_PARAM are, to put it mildly, completely goofy. If you’ve ever tried to read this, you know that even when things are explained, they make no sense.
No author should ever have to write a LIGHT_PARAM. This is a classic case where the exporter should do the transformation for us. What I would like to get into Blender 2.7 is the ability to place a cone light in Blender and have the exporter write the spill light automatically, e.g. filling in the parameters from the position and shape of the light. You light your scene and you see the same thing in X-Plane.
For years we’ve been behind the curve on user experience when it comes to 3-d modeling. I think we’re starting to fix some of those problems in a major way, which should help make the power of X-Plane 10’s rendering engine quite a bit more accessible.
*As a side note, I have no idea what the current 2.49 script status is. We view the 2.7 scripts as the long term way forward, due to Blender 2.49’s not being maintained, so our effort on a truly user-friendly tool is going into 2.7. But we also maintain updates to 2.49 to support our internal team. X-Plane 10.50 will feature a few new OBJ commands, so I will try to post our branch of the 2.49 scripts (which do have this new support) for anyone whose aircraft is in 2.49.
In the long term the 2.7 scripts will feature a migration tool, but I expect we will have a useful 2.7 release (for 2.7 authors) without migration first, and then migration will come later.
X-Plane 10.42 is a small bug fix release, and it’s out now as a free update for all X-Plane 10 customers, Global Edition, Digital Download and Steam. We also updated the installer on Monday.
With 10.42 out, we’re looking to get onto 10.45 ASAP, which will include the latest set of airports from the X-Plane Scenery Gateway.
I’d like to see us release new airports from the gateway every eight weeks, as long as there are enough new updates (and so far that has definitely been the case). I’m posting that here, publicly so that when Julian and Robin are ready to release new airports and data and I’m going “um, well, maybe next week, I’m really busy and the kids threw up on me this morning” they can point to this post and go “eight weeks!”
X-Plane 10.42r1 is available for beta – the release notes are here. This is a very small bug fix patch that mostly aims to address problems we’ve found with digital download over the last few weeks. (We have already significantly upgraded the capacity of the digital download authorization servers, and a new installer release will finish this work next week.)
Even if you don’t use digital download, please do get the new build by running the installer, checking for updates, and checking “get betas”. Once some users have tested 10.42 and we release it, we can move on to 10.45 and more interesting things.
Since El Capitan came out, SASL has been crashing on quit when used in aircraft with customized sound. While I work on neither SASL nor El Capitan, both are (at least partly) open source, so I spent a few hours yesterday and located the bugs.
The good news is: this bug is fixed in SASL version 2.4. The bad news is: you’ll need to make sure the aircraft you are flying is using SASL 2.4 – and this applies to every aircraft that you have that uses SASL.
If you develop an aircraft using SASL, you can get the latest free version of SASL here. Note that this new version is apparently 64-bit only.
If you are using a freeware aircraft that is no longer supported, I think you could theoretically drop in the new SASL and see if it fixes things. If the aircraft is payware, the aircraft author might not be thrilled to have to support a “modified” aircraft and might prefer to do the upgrade in-house.
If you are a developer of SASL or you use OpenAL, you can read the gory technical details here.
Here’s a pop quiz: you download the release candidate for a new X-Plane patch – maybe it’s 10.41 – and you run your add-on. To your horror, you find that a dataref is missing – or maybe a command no longer works. Do you:
- Immediately rewrite your code to work with the release candidate and release it immediately, preferably before the beta is over or
- File a bug, clearly indicating what changed from the last shipping version. Laminar fixes the release candidate and by the time the patch ships, your add-on “just works”.
Pencils down! I have said this before, but I’ll say it again: the answer is choice number two – file a bug.
If you find a problem with your add-on during a beta where something used to work and now does not:
- Do file a bug. We cannot maintain compatibility if we do not know there is a problem.
- Do not change your add-on (and definitely do not release that change). Most likely that change is a mistake, and when we fix the bug, your original add-on would have worked but your modified add-on is now broken.
We try really, really, really hard to make sure that you don’t have to cut a new release of your add-on just because we are issuing a free update to X-Plane.
What If I Am Late?
What if you are using the new shipping version of the sim? You just got 10.41, but you didn’t try any betas, and you find your add-on is broken.
File a bug. We will still fix the bug, and issue a patch. The only difference here is that:
- Your users will be grumpy about the 1 or 2 week period during which the add-on was broken – that time could have been zero weeks if you participated in the beta and
- I will be grumpy that you reported the bug after beta instead of during the beta.
Free Passes
Even if we find a bug that requires a modification to your aircraft to fix, we still try to put a compatibility mode into X-Plane so that you don’t have to release your aircraft immediately. As I blogged before, shipping add-ons can get a free pass. In these cases we sometimes recommend you update your add-on when you can, but there is no requirement that you must update to match our new update. Update when you can, on your own schedule.
We did hit three bugs in X-Plane 10.40 for which you do have to update your aircraft to get the benefit of the bug fix:
- Light sizes for billboard lights.
- Fixing buggy fuel consumption at high altitude.
- Feathering props on low oil pressure.
In all of these cases, we couldn’t apply the bug fix correctly to all aircraft – the changes would have positive effects on some aircraft and negative effects on others. So you get a free pass to leave the aircraft as is until you next save it in Plane-Maker; at that time you can make sure that the results of each bug fix are working for your aircraft. More details here.
You may need your rant-proof goggles for this post.
In the last few years, we’ve seen the price of major operating system upgrades drop, down from $100-to-$150 to…$0. And free is a magic number – once you make something free, people install it without doing a lot of critical thinking about whether something is a good idea or not.*
This creates a software compatibility disaster for OS vendors, app developers and users.
- The OS vendors (Microsoft, Apple, etc.) try really hard to keep the minor patches to the OS from breaking compatibility. They need users to install updates aggressively to fight malware. So a break due to a minor patch is a rare thing.
- The OS vendors can’t keep legacy APIs going forever – at some point they need to change fundamental technology.** So they save the big changes for major versions.
- Major versions are now free – so users install a major update aggressively, taking incompatible changes that the OS vendors had to put somewhere, breaking their apps.
We first saw this with Yosemite, which caused us a bit of a fire drill, and now we’re seeing it with Windows 10, where I have just seen a pile of weird-o random behaviors that are specific and unique to Windows 10. At least some of them seem to be a clear case of OS compatibility rules changing at a major version.
So first, here’s my advice:
Don’t be the first pilot on the block to update the OS on your X-Plane machine.
Let someone else go first. These days new OS releases just don’t offer some new feature that is so totally awesome and amazing that you can’t wait a few weeks to see if the compatibility situation is good or bad.
In the long term, what we need is a new normal. Currently application developers depend on compatibility-breaking API changes happening at well-defined, infrequent, major inflection points, e.g. the full OS upgrade. But full OS upgrades are gone; Microsoft is saying that they’re just going to patch Windows 10 forever, and Apple puts out a new OS version so frequently that they’re not all that different from patches.
We could simply declare ourselves “not compatible” with future OS versions until we’ve had time to survey the damage, but that’s not a real solution – most users are going to update first and read the fine print later.
It may be that we have to follow OS betas much more closely. There’s definitely some irony here – I am always berating third party developers to retest their add-ons against X-Plane betas. (“We’ll fix the bug, just check your add-on!”) Third party developers have a good reason not to want to do this – they need to do real work on their new aircraft, not spend their time testing our builds. We have a similar view about OS betas – it’s a time-sink for us to keep re-testing X-Plane on the latest unstable crazy-town beta. (OS betas can be a pretty big mess, so isolating regression bugs that affect your product from general breakage is sometimes difficult.) But we’re in the same situation with Apple and Microsoft that third parties are with X-Plane: if you don’t report a problem, it may not get fixed.***
* This has had the effect of making free-to-play a near-requirement in the mobile market. If your mobile app category is even slightly competitive, then you are one of ten apps doing the same thing. A user will download and try every free app in the category before buying even one $0.99 app. So if your competitor is free-to-play and isn’t totally unusable and awful, you had best be free-to-play too!
** For example, Microsoft has been trying to evolve the rules about high-DPI devices and app resolution. This is a necessary step because monitors have become much larger and much higher DPI; keeping the old system forever would mean not actually supporting this technology.
*** Apple and Microsoft test third party products themselves to try to catch problems directly, and we now do the same thing with third party add-ons. But the amount of test coverage we can find is definitely not adequate to guarantee that your add-on is perfect coming out of beta. All we can hope to do is add another layer of defense.
When you capture an AVI movie in X-Plane, a few things can make the video capture look bad:
- X-Plane might not be able to compress and save the frames to disk as fast as they’re being recorded. In this case you’ll see a message on screen about dropped frames. (This isn’t a huge problem because the AVI compressor is multi-core, so it can usually keep up on modern machines.)
- X-Plane itself might not be able to hit the framerate you’ve selected in the AVI capture settings.
- If X-Plane’s framerate is fluctuating, the AVI movie can’t record that properly, because AVI files run at a fixed framerate.*
There is another way to make a movie, however, that will give you much better quality: record your replay.
When you record an AVI movie in replay, X-Plane slows down its playback of the replay as necessary to guarantee that it can render and compress each frame at the framerate you have selected for your AVI. If you’ve picked very high rendering settings, you’ll see X-Plane running in what appears to be slow motion while capturing an AVI in replay mode.
When you play back the AVI, motion will be in “normal time” and will be frame-perfect, even if you’ve picked a high framerate, large movie size, and heavy rendering settings.**
* This wasn’t a problem with the older QuickTime capture; QuickTime movies can have variable framerate.
** This mode was available in X-Plane 10.36 via a special button; in 10.40 it is the only way to record an AVI in replay mode. The change was made to simplify the process of making a high quality movie.
TL;DR – X-Plane won’t ask you if you want to reset your rendering settings when you crash unless you hold down the shift key.
Getting Locked Out
Some of X-Plane’s rendering settings take effect on restart, and this can lead to a nasty situation:
- You dial in a setting that takes affect on restart, like FSAA in non-HDR mode.
- You restart, and your graphics card can’t handle it; the driver crashes.
- Now you can’t start X-Plane without deleting your saved preferences, which will cost you your joystick setup and other prefs.
This was a particularly nasty problem in the early 32-bit days of X-Plane 10; rendering settings that take effect on restart could push you over the RAM limit, causing a crash the next time around.
Safe Mode
To work around this problem, we added “safe mode”. If X-Plane detects that it crashed the last time it was run, then on the next start, X-Plane gives you a chance to reset rendering settings. If you reset the settings, you might avoid a crash on startup. You’re not locked out.
Don’t Reset My Settings
Now that X-Plane is 64-bits, getting locked out is much less of a concern. So we have received a number of complaints from users who don’t want their rendering settings reset if the last run didn’t shut down properly. The previous crash might have been due to an errant add-on, and resetting X-Plane’s rendering settings won’t help. If you have a multi-machine setup, having the dialog box come up on the external visual machines means an annoying trip to the KVM switch.
So we’ve changed how safe mode works. Starting in X-Plane 10.40 beta 10, safe mode is an option if and only if you start the sim with the shift key held down.
This gives you access to safe mode when you need it (e.g. after repeated crashes) but skips the annoyance of having your settings reset (or having to get a keyboard for external visuals).
X-Plane 10.40 beta 5 is out – this fixes my breaking the zoom wheel, and has a few small loose ends tied up. We have a bunch of new stuff to get into beta 6, maybe this weekend.
Nvidia users may notice weird artifacts around the edges of the skyline in various rendering settings; my current speculation is that this is a bug in the NVidia shader compiler, but I am not sure. In the process of isolating the bug for NVidia, I discovered a work-around*, but we haven’t shipped it yet. I’ll update the score card when we know for sure what’s up.
We investigated a bunch of performance complaints since we fixed the prop disc slow-down and nothing has been reproducible. If you think you see a performance problem, please get two clean side-by-side demos (10.36 and 10.40b5) and then add your add-ons to both in lock-step to create a good test case. If you isolate a performance problem that way, then file a bug.
* From what I can tell, the shader compiler is optimizing the shader and sometimes producing illegal code; randomly moving things around can make the bug go away – it’s a “Doctor, it hurts when I do this” kind of thing, but if there’s something illegal in the shader that ships in beta 5, I don’t see it.
