X-plane 10.31 is now final; if you’ve run X-Plane 10.30 you probably already know that. 🙂
Steam Users: the Steam Edition of 10.31 for OS X was broken, so we rolled it back to 10.30. We’ll get 10.31 out on Steam as soon as we can fix the problem – it should be a few days at worst.
Since X-Plane 10.30 shipped, we have received a lot of feedback about the stability of X-Plane 10.30 – or rather, the lack thereof. We are getting more bug reports that the sim crashes, more tech support requests, and more discussion of crashing in the forums.
We don’t have a good way to characterize with hard data how much less stable 10.30 may be – unfortunately even the crash reports we gather don’t give us a statistically clear picture.* But with this much user reporting, stability is our first line of inquiry. 10.31 will come out shortly (and will fix a few issues) and we’ll keep investigating problems and pushing small patches (to fix stability issues without introducing new ones) until we’re back on solid ground. The rest of this post discusses some of what I’ve seen so far looking into stability issues.
Since X-Plane 10.10, we’ve had the ability to send crash reports** directly to Laminar Research. Here are a few suggestions:
Reproduction steps are gold to us – sometimes we’ll see a crash and the proximate cause is “hrm – the sim crashed drawing an OBJ”. Sadly this doesn’t tell us much about how to reproduce the crash – and clearly the sim does not crash every time you draw an OBJ; it wouldn’t start up.
Our crash reporter can only trace the origins of a crash through X-Plane code.*** Unfortunately, of the top 5 Windows 64-bit crashes in 10.30r2 (and this represents most of the crashing going on), 87% of the crashes originated in an OpenGL driver from AMD, NVidia or Intel. (This does not mean the driver is bad – it could mean that X-Plane sent the driver junk data. Or it could be the driver’s fault. We don’t know.)
Unfortunately that means that for the vast majority of 64-bit crashes, we have no idea what the sim was doing when it crashed – the trace from the driver will contain junk. All we know is that we were, in fact, in the driver at the time of death. (We also have the log.txt but often it just indicates the user was flying.)
This is a long term problem that we need to address as developers with our crash report technology; we have some ideas on how to fix it but they will take time to implement. Clearly we’ve hit a point where the low hanging fruit (crashes that affect a lot of users but are entirely in X-Plane code) are gone.
We do not have good statistics (yet) for what percentage of submitted crashes were on X-Plane installations heavily augmented with add-ons, but it is not uncommon to see a user with a custom aircraft running a custom plugin, an online flight plugin, tons of custom scenery packs, and general utility plugins.
This kind of customization is great! We wouldn’t have added the SDKs if we didn’t want people to use them. But when it comes to isolating a crash it does make things quite a bit more difficult.
My goal for X-Plane is to not have X-Plane crash due to bad third party add-on data, no matter how badly it is formed. To that end, when I find cases where bad data will crash the sim, I am adding code to protect, document, and shut down based on the error. (Some errors are recoverable, some are not, but the important thing is to give the author some feedback as to what is going on.)
In the long term I think we need to add better diagnostics to our crash reporting to show what add-ons are involved in a crash. If you have 1000 scenery packs, I can’t go download them all in an attempt to reproduce a crash.
In the short term, if you are seeing instability and you have third party add-ons installed:
Reproduction steps are gold to us! The combination of the add-on and simple steps to reproduce let us find and fix problems, even with third party add-ons. If you have a flight that crashes, please try to simplify it. Try going directly to the location that you were at where you crashed, starting with the engines on, etc.
While we’re talking about stability, I have to mention Yosemite: on OS X the vast majority of bug reports we’re getting that involve “weird stuff” and crashes involve Yosemite. We’re still trying to sort out the issues, but Yosemite is definitely the most disruptive OS X major version release we’ve had in a long time, and because it’s free, a lot of people downloaded it.
A major OS upgrade is when the OS vendor can make major changes, drop old technology, etc. On OS X it also means new OpenGL drivers (because the drivers are always packed with the OS), so it’s a potentially disruptive update.
Update: it turns out Yosemite also broke our automatic crash reporter, which would explain why none of the common Yosemite crashes show up in our top crash list. 🙁 Some users have sent me the Apple crash reports that appear (since our crash reporting doesn’t run) so that we can diagnose what’s going on.
10.31 should be out in a day or so (including Steam); as we continue to find and fix stability problems we’ll continue to issue small patches. We’re looking to do a 10.35-type release with small features and new airports; the current plan is that Julian and Tyler will do a public beta of the new airports (as a single separate scenery pack) so that people can try the new airports and spot problems before we get into beta.
* Crash reports don’t tell us how many times the sim was run overall; users with stable systems don’t send us any data, so an increasing crash report incidence would be meaningless – we don’t have data about increasing or decreasing use of the sim.
** Here we’re talking about true program crashes – the uncontrolled exit of the sim for unknown reasons. If you have a controlled failure, e.g. you load scenery and it’s missing objects and the sim refuses to run, you don’t get a chance to auto-report that. We also don’t view this as a stability issue – X-Plane can be very unforgiving about broken third party content, and we can have a separate argument about whether this is a necessary evil or terrible user experience (or both!), but there’s no diagnostic information to gather.
Update 2: Blog comments are now working again.
*** The problem is that for 64-bit there is no standard for back-tracing a stack without the debug symbols to decode the stack linkage; there is no standard link register. I’m guessing that the ABI designers thought they were helping the compiler guys optimize, but in practice it makes debugging other people’s DLLs miserable. Microsoft publishes debug symbols for every DLL they make ,which is a big help, but the driver writers do not.
Quick note: 10.31r2 is up on servers now. Please try it!! We think this is a keeper 10.31 – it has just a few more bug fixes. A list of fixes is here.
X-Plane 10.31 release candidate one is now live. To get it, you need to run the X-Plane installer and check “Check for new betas as well as updates” – 10.31 isn’t final so it won’t auto-notify you.
Please do get 10.31 Release Candidate 1 and try it. We’d like to go final with it shortly.
The release notes are here – that’s basically a list of the bugs fixed. Go read them if you want to know what is fixed! (If the release notes don’t say it is fixed, then there is no change from 10.30.)
Report bugs here.
Please do not report bugs in the comment section – a little bit of me dies inside every time you do.
It looks like we have some bug fixes that will go into a 10.32, but for 10.31 I wanted to get out smaller fixes that we could ship quickly. So hopefully 10.31r1 will be the only 10.31 patch and then we can get fixes in for 10.32 in another week or two.
Just a quick note: last night I fixed the bug in 1030 where the sky is corrupt when you use the instructor’s station and HDR together. This is the last of the ‘quick’ fixes for 1031, so I’m hoping to have 1031 cut over the weekend.
When I first saw pics of this bug I thought it might have been an old driver, but the number of users seeing it made it clear that it was a real thing; as it turns out, it was a subtle OpenGL screw-up by me. Frankly the amazing thing is that we’re not seeing it on every Windows driver. (From what I can tell, the AMD drivers “helpfully” work around my error, hiding the bug.)
We’ll post release notes with the beta with a complete bug fix list.
For about a year there has been a subtle bug in how X-Plane draws taxiways: if you build an S-curve shape out of a single bezier and it is almost perfectly symmetrical, X-Plane would go “nah, why bother” and replace it with a single straight line segment.*
So in 10.30 I fixed it, and the result was a bunch of broken airports with missing pavement. (YMML is high on this list!)
It turns out that these airports have authoring errors – typically a segment of pavement that should be straight instead being formed by two overlapping beziers. This is definitely wrong, but due to the bug, X-Plane would simplify the overlapping S curve into a single straight segment and the layout would work. Only now that X-Plane correctly renders the S curve does the taxiway fail (because taxiways may not have overlaps). So two wrongs may not make a right, but they do make a “hey, that looks okay, let’s ship it”.
Note that the overlaps depend on the rendering setting of X-Plane – a different S curve is formed at different rendering settings; the overlap that causes the taxiway to disappear may only appear at a particular rendering setting.
For 10.31 I am going to undo my bug fix. This doesn’t make me happy, but I think it is necessary:
This is too much uncertainty to solve ‘by hand’. So my plan is:
This isn’t going to be a quick process, but then it can’t be, because third parties have apt.dats shipping now that only work when X-Plane has the buggy taxiway code in place. So we need to ship WED and then give third parties enough time to go back and check their layouts and fix them if necessary.
I expect to get a 10.31 beta with the taxiway code changed back this week.
For WED validation, I have some test code to detect errors but it isn’t ready yet. The problem is that it’s not good enough to detect errors with overlapping beziers**; we have to consider two bezier curves near enough to each other that with the error in rendering introduced by WED’s rendering settings, we get an overlap. (So authors, better be safe than sorry in creating your pavement.)
If there’s a moral to this story, I think it’s this: when we (LR) don’t provide good tools for authors to validate that their work is correct, the resulting body of work will end up with subtle errors.
* X-Plane renders beziers by measuring how far the mid-point of the curve is from the average of the ends. As long as this distance is ‘too far’ and the iteration count isn’t too high, X-Plane divides the bezier in half and repeats the process. In this way beziers are broken into enough line segments to approximate the bezier within a minimum error limit. The rendering settings control the error limit.
The bug: if the curve was a ‘balanced’ S curve the mid point of the curve was the average of the end points and X-Plane went “great, no error” and stopped dividing.
** Which is already not an easy problem – the analytical solution for bezier intersection is a 9th order polynomial!
X-Plane 10.30 Lives!
Laminar Research, creators of the X-Plane flight simulator franchise, has released the latest update to X-Plane, version 10.30.
This latest release represents the largest and most significant X-Plane update to date. It not only contains numerous fixes and performance improvements, but also some substantial additions to the product itself. Foremost is the inclusion of the new GPS. The previous versions of X-Plane also included a GPS instrument but it could only do direct-to navigation and could not implement a GPS approach. With the recent introduction of GPS enabled approach systems at many airports, the new GPS in X-Plane will allow customers to practice actual GPS approaches using an instrument familiar to most general aviation pilots. The new GPS is a drop-in replacement for the old X-Plane GPS, so that with one click in Laminar’s PlaneMaker, old aircraft can be upgraded to the more IFR capable version. The X-Plane 10.30 default aircraft come with the new GPS already installed. The new GPS is based on the Garmin 430 and Garmin 530 and both variants are included.
The 10.30 update also includes major improvements to both the weather and scenery systems. The system for the placement of autogen scenery has been refined so buildings, trees, and other objects more accurately reflect their real-world counterparts.
With the recent release of World Editor 1.3 and the Airport Scenery Gateway, hundreds of new highly detailed airports have also been included in this latest and major update to X-Plane 10.
We’ll have a 10.31 bug fix patch, probably in about a week, with a few straggler bugs that didn’t make the cut. In the meantime, aircraft designers can start using the new GPS.
Once X-Plane 10.30 goes final*, we will also update X-Plane on Steam to version 10.30. The update for Steam users will be applied using Steam’s normal updating system, not the X-Pane updater. (Basically if you installed via Steam, you update via Steam; if you installed via Laminar Research’s installer, you update via our installer.)
How soon will it be available? I don’t know; this is the first patch we’ve done via Steam. We’ve done literally one hundred patches (including betas) for X-Plane 10, so we know how long it takes to cut a patch, upload it to the servers, etc. For Steam, we’re just learning this now. I am hoping it will only lag our installer by a few days at most, but Steam is new to us and partly out of our hands (in that it’s Steam’s servers doing the downloads).
* So far it looks like 10.30r2 will be final, but I will look at the submitted bugs tomorrow.
X-Plane 10.30r2 is out now. You should go get this RC and try it. I believe it is very likely to go final in the next week.
There are known bugs* in 1030r2 – the plan is to put 1030r2 out as final, then issue 10.31 in perhaps a week or two to mop up about a dozen bug fixes that didn’t make in.
X-Plane 10.30 has had an incredibly long beta period, so our goal is to get it out now and unblock third parties.
* I know, you are shocked, right??
It’s official:
August 28, 2014, Columbia, South Carolina. Laminar Research is pleased to announce the release of its Airport Scenery Gateway, as well as an updated and improved version of its airport scenery tool, World Editor 1.3. The Airport Scenery Gateway can be accessed on the internet at http://gateway.x-plane.com and the new and improved version of World Editor is available for download at http://developer.x-plane.com under Scenery Tools.
The Airport Scenery Gateway was created to facilitate the populating of X-Plane with 3D airport sceneries that are consistent in quality and appearance, error-checked, and will work with a standard installation of X-Plane 10. During the past several months Laminar Research has received hundreds of airports designed by World Editor users. World Editor is an airport scenery creation and editing tool for Laminar’s X-Plane 10 that allows the typical X-Plane user to create airport scenery.
By establishing an internet gateway for scenery submission, it is now possible with World Editor 1.3 for users to automatically submit airport scenery contributions directly to the Airport Scenery Gateway. X-Plane users can visit the Airport Scenery Gateway to search for airport scenery submissions. The scenery can then be immediately downloaded into X-Plane 10 before it appears in a scheduled X-Plane update.
Read more from this press release.
A few notes:
Finally, I think this has been discussed before, but…custom scenery authors:
Always put exclusion zones around your custom airports! Even if there are no 3-d buildings at an airport today, they may appear in the next update.
With the Gateway open, 3-d is going to appear quickly!