WorldEditor 1.4 is almost ready for beta, and among its new features (there are several big ones) one is very important to the X-Plane Airport Gateway:
- WED 1.4 can browse airports on the gateway and download scenery packs directly into the WED workspace.
“Direct download” is really important for a few reasons, some of which might not be obvious:
- It saves time. Getting a gateway airport, even if you just want to look at it, is much faster when you don’t have to download the scenery pack, unzip it, install it, then import it into WED. Once you use direct download, you’ll wonder how you lived without it.
- It prevents mistakes. We have seen airport submissions where a user clearly downloaded a scenery pack, imported only the apt.dat (but not the DSF files) and then uploaded it. We cannot take scenery packs like that, because they fundamentally remove the 3-d data from the airport. (A fundamental policy of the gateway is that if you upload a scenery pack and one already exists, yours can’t be worse than the existing one in some way. We have to always move forward.)
- It provides ancestry information. When you download and then upload directly from the Gateway in WED 1.4, WED provides the scenery pack ID of the original pack as the “parent” of the new pack. This means that when Julian goes to look at an upload, he can look at the “original” and more rapidly spot problems. If your pack is the same as the original except for taxi signs, he only needs to inspect taxi signs.
- It prevents data loss. DSF is a slightly lossy data format – that is, if you get your data back out of a DSF file, it won’t be exactly the same as what you put in. (It is like a JPEG image in that regard.)
More on that last point: DSF stands for Distribution Scenery Format – it was meant as a way to make final scenery packs; it was not meant as an interchange format for continuous editing. So users are constantly importing and exporting DSFs to do work, small rounding errors (“bit rot”) will creep into our 3-d, and features that were perfectly aligned might not be well aligned after 4 or 5 edits.
The internal format for scenery on the gateway is not binary DSF files, so doing round trips to the gateway has much less “bit rot” than importing scenery packs.
Finally, DSFs are tiles; if your airport spans a DSF boundary, all DSF features (polygons, fences, etc.) get split along the DSF boundaries.
So if you import a scenery pack that you downloaded, you’re getting the split version, which is harder to edit and work with.
The internal format of scenery packs on the gateway is not split, and thus you can edit the original in almost the same form as it was uploaded.
WorldEditor 1.4 beta should becoming in “weeks” (and hopefully not that many weeks); I’ll post here when we are ready for beta testers.
I’ve been fielding crash-related bug reports for X-Plane 10,30, and a lot of them are turning out to be caused by third party scenery packs.
Now there are two possibilities for why a third party scenery pack might crash X-Plane:
- There is a bug in X-Plane. In this case, I want to fix the bug!
- There is something wrong with the data in the scenery pack (that is, the data doesn’t meet X-Plane’s expectations). In this case, I want to change the result from a crash to some kind of clear error report that an author could use to fix the problem.
The Old Rules
During my work on the scenery system for X-Plane 8 and 9, my view was “garbage in, garbage out” – that is, if you feed junk data into the scenery system, the result would be unpredictable and could include a crash. In X-Plane 8, scenery load was completely synchronous – that is, your flight just stopped while we loaded files, so my idea at the time was that validation of scenery files should be done ahead of time, perhaps by the tools that write those files.
That logic (if it was even a good idea back then) doesn’t apply now. Loading is done entirely on other CPU cores and does not interrupt flight; we can take the CPU time to make sure incoming data is valid. And the cost of not validating the data is clear:
- The instability from badly formed data results in a sea of crash reports – if we have real serious sim bugs, it’s much harder to catch them in the noise.
- Because we don’t trap bad data, the end result crash can be in the OpenGL driver (due to cascading errors), and due to the way 64-bit crash detection works, we don’t get any trace of the contributing X-Plane code path.
- Authors who create scenery packs can’t tell if their pack is the source of crash, or how their data induced a crash.
My goal is to chase down crashes caused by third party scenery and both fix any bugs that emerge and tighten up error checking to detect the cause of crashes early on, in a way that is useful to authors.
The New Rules
Because X-Plane has not run validation over all scenery files in the past, there are probably scenery files in use now that contain data that I would consider illegal (either by the documented format, or just by my own view of how the file format was meant to work; I realize that second definition is quite useless to authors). But for illegal data that doesn’t crash the sim, I can’t just go in and change X-Plane to reject that data in the next patch. (“Surprise! Your scenery packs won’t load!” isn’t cool.)
So my thinking for validation is:
- Detected error conditions that would have caused a crash 100% of the time can be handled by any clear means possible, including quitting the sim with an error message. If the sim can handle the error and continue, that’s better, but even a controlled quit with diagnostics is an improvement over an uncontrolled crash.
- Detected error conditions that would have crashed the sim some of the time need to be logged but can’t quit the sim or cancel loading of the entire scenery pack; while a crash is serious, users were probably still enjoying the scenery pack (and merely being annoyed at how unstable their flying experience was). The user experience shouldn’t get worse.
- Detected error conditions that have only minor effects (e.g. out of range values that might cause rendering artifacts) whose legal values are well-documented can be logged, perhaps with a noisy error. The pack works, so quitting or refusing to load is not acceptable, but if the error is too quiet, the condition will never get fixed.
- Detected error conditions that have only minor effects whose legal values were never documented need to be logged unobtrusively; it’s not fair of me to penalize scenery packs for failing to read my mind.
On those first two points: one of the issues that complicates validation of scenery packs is that some errors are not discovered until mid-flight, because scenery is processed while you fly. This means that a large scenery pack might contain areas that are problematic to fly in; if the original author doesn’t cover every inch of scenery in a test flight, the author might not know about the problem.
The ATC system takes a more aggressive approach: it validates every single .atc file at load time even if it doesn’t need to use the data until later. (Virtual air traffic controllers are instantiated near the aircraft as the user flies, but the data in the .atc files that will be used to set up those controllers is checked at load time.)
This approach wasn’t too expensive performance-wise: the .atc files are small and the ATC system has to read them anyway to index them for later use. I am not sure how to best replicate this for scenery; consider Alpilotx’s HD scenery mesh – reading every single DSF in Europe at load time isn’t acceptable performance wise but without that, how can he know that his scenery doesn’t contain areas that could crash X-Plane?
Please Send Me Bogus Scenery Packs
Here’s where I need you to help: if you find X-Plane is crashing, and you find that removing a scenery pack fixes the crash, please report a bug, including the scenery pack (or where to get it) and where you had to fly/start up to induce the crash. Even if it is the fault of the pack (and it may not be – it may be an X-Plane bug!!!!) I still want to clamp down on the failure condition.
If you develop 3-d models for X-Plane using Blender 2.49 on OS X, do not upgrade to OS X 10.10 (Yosemite). Something changed in the OS that causes the python scripts to not load, which means you won’t be able to export.
(As a side note, I am of the opinion that if you use your computer for work, you should not install major OS upgrades without investigating first. If Apple is going to break support for something intentionally, it will be in a major version. Once you install the new OS, going back is hard. So why jump in so fast? These days I don’t believe that any one OS update is so much better than what was there before that it trumps keeping your productivity software working. But then, as Chris points out, I am older than dirt.)
Update: Chris found a way to make Blender 2.49 work with Yosemite. For reasons unclear to us, the Python 2.5 framework that comes with the OS in Yosemite is just a symlink to version 2.6 (rather than the real Python 2.5). If you delete the symlink and put in place the old 2.5 binary from an older OS, Blender starts working again. I’ll post instructions when we have something written up in readable form, but this is a tricky mod at best – you have to change admin-writeable files in your library.
Update #2: Hi guys, Chris here. Here’s an outline of the steps that you’ll need to take…and you’ll need to be an administrator of course since you’ll be running sudo. A basic disclaimer….you should have a full bootable and recent backup of your system anytime you mess with system files (even though this should be relatively straight forward). These instructions worked for us but you should follow them at your own risk.
- Go into your time machine archives or find a friend that has 10.8 or whatever version of OSX you used to be on, and obtain a copy of the Python 2.5 version framework at this path and put it somewhere accessible:
/System/Library/Frameworks/Python.framework/Versions/2.5
- Open Terminal
- Change directories by running
cd /System/Library/Frameworks/Python.framework/Versions/
- To be safe, type:
ls -al
to get a full directory listing. If you see 2.5 -> 2.6, you have a symlink from 2.5 to 2.6 which is what’s causing Blender to use the wrong Python version. IF YOU DON’T SEE THIS, YOU DON’T HAVE A PROBLEM!
- Delete the symlink by running
sudo rm -rf 2.5
- Copy the old version of 2.5 into this location (pay attention to the dot at the end!):
sudo cp -R some/path/to/the/good/2.5 .
- Now as good practice, let’s put the permissions the way they should be:
sudo chown -R root 2.5
sudo chgrp -R wheel 2.5
- As one of our artists discovered, if Step #7 fails for you because “wheel” is not a known group name, it might be a good idea to run Disk Utility and let it “Fix Permissions” on your drive.
Posted in Tools
by
Ben Supnik |
For about a year there has been a subtle bug in how X-Plane draws taxiways: if you build an S-curve shape out of a single bezier and it is almost perfectly symmetrical, X-Plane would go “nah, why bother” and replace it with a single straight line segment.*
So in 10.30 I fixed it, and the result was a bunch of broken airports with missing pavement. (YMML is high on this list!)
It turns out that these airports have authoring errors – typically a segment of pavement that should be straight instead being formed by two overlapping beziers. This is definitely wrong, but due to the bug, X-Plane would simplify the overlapping S curve into a single straight segment and the layout would work. Only now that X-Plane correctly renders the S curve does the taxiway fail (because taxiways may not have overlaps). So two wrongs may not make a right, but they do make a “hey, that looks okay, let’s ship it”.
Note that the overlaps depend on the rendering setting of X-Plane – a different S curve is formed at different rendering settings; the overlap that causes the taxiway to disappear may only appear at a particular rendering setting.
For 10.31 I am going to undo my bug fix. This doesn’t make me happy, but I think it is necessary:
- We have no idea how many airports have their taxiways broken by this bug.
- Authors have no easy way to detect this problem, other than re-testing every airport at every rendering setting.
- Even if an airport looks okay at all rendering settings, future rendering settings may cause the problem.
This is too much uncertainty to solve ‘by hand’. So my plan is:
- Undo the code change for 1031. YMML and friends comes back.
- Develop validation code in WED to detect this kind of authoring error.
- Ship that version of WED so new authoring work will be checked.
- Run the WED code on all airports and make a list of ones that need repair.
- Fix all of the known problems in the airport gateway.
- Redo the code change so X-Plane is correct.
This isn’t going to be a quick process, but then it can’t be, because third parties have apt.dats shipping now that only work when X-Plane has the buggy taxiway code in place. So we need to ship WED and then give third parties enough time to go back and check their layouts and fix them if necessary.
I expect to get a 10.31 beta with the taxiway code changed back this week.
For WED validation, I have some test code to detect errors but it isn’t ready yet. The problem is that it’s not good enough to detect errors with overlapping beziers**; we have to consider two bezier curves near enough to each other that with the error in rendering introduced by WED’s rendering settings, we get an overlap. (So authors, better be safe than sorry in creating your pavement.)
If there’s a moral to this story, I think it’s this: when we (LR) don’t provide good tools for authors to validate that their work is correct, the resulting body of work will end up with subtle errors.
* X-Plane renders beziers by measuring how far the mid-point of the curve is from the average of the ends. As long as this distance is ‘too far’ and the iteration count isn’t too high, X-Plane divides the bezier in half and repeats the process. In this way beziers are broken into enough line segments to approximate the bezier within a minimum error limit. The rendering settings control the error limit.
The bug: if the curve was a ‘balanced’ S curve the mid point of the curve was the average of the end points and X-Plane went “great, no error” and stopped dividing.
** Which is already not an easy problem – the analytical solution for bezier intersection is a 9th order polynomial!
I’ve marked WorldEditor 1.2.1 as final. If you are a WorldEditor user, please upgrade from 1.2 to 1.2.1. Besides fixing a bunch of bugs (including annoying usability bugs), it changes the format of the zip files for submissions to the global airports to a format that is a lot easier for us to handle on our end.
The next version of WED will be 1.3 and will include more usability features; we’re not sure what else will go in that bucket or when we’ll cut it off and kick it out the door.
We’re on release candidate three of WorldEditor 1.2.1. RC3 fixed export of 970 airport files to include frequencies and fixed a bug where you could end up with a two-point hole in a polygon (which would then cause the airport to fail validation).
Hopefully this is the last RC and we can call WED 1.2.1 done shortly.
I just built a release candidate of WorldEditor 1.2.1r2; if you work on airports, please try it and post a WED bug if something is really borked.
There are a few minor bug fixes, some usability tweaks, and two major ones:
- The export format for sending zip files to Robin is modified to make things easier in the collection process. For this reason, we’d like everyone submitting airports to go to WED 1.2.1.
- The library view now hides the resources marked private or deprecated in X-Plane 10.25 or earlier. So WED 1.2.1 and X-Plane 10.25 provide a much less cluttered library view. (WED 1.3 will have a nice improvement to filtering as well.)
For more details on what changed, all commits to WED 121 are visible here. (The commit notes are listed, and the yellow tags tell you the version where we then re-shipped the code.)
We are discussing internally how to handle a more permanent airport submission tracking system. My current thinking is that eventually we’ll build file transfer directly into WED – that is, you’ll say “share this airport” and WED will call up our servers directly and send the airport; no zip files and email. But that’s for the future.
X-Plane 10.25b2 should be “real soon” I think – I’m just waiting on one more change to cut the new beta.
Posted in Tools
by
Ben Supnik |
I just posted a beta of WED 1.2.1; hopefully it will only take one beta to get this quick “bug fix” patch done.
Some usability fixes:
- Locked and hidden items don’t cause clicks to select their parents.
- The interior of an airport boundary is not selectable – it never should have been.
- The marquee tool will never select the “world” entry on the hierarchy.
- The click hot spots of handles don’t shrink at low zoom – they were doing that before, which made clicking hard.
(Basically Chris tried to use WED and got really annoyed*, so we dug in and fixed some of the smaller annoyances. The ones we left (like how locking works) need a major rework and can’t go into a bug fix.)
1.2.1 also filters out the private and deprecated library items (once you have X-Plane 10.25, beta coming “real soon now”). This should make the library display a lot cleaner. Also, hidden airports won’t export or fail validation, which is what you would expect.
The only other changes are a bug fix for exporting polygons on a DSF edge, and a bunch of changes to help Robin collect global airports.
If you are using WED 1.2r4 to make airports, please try 1.2.1b1 this weekend, and file bugs here.
* Which was difficult to differentiate from his normal state.
Over the last few months we have been lucky to have Ted Greene working for us as a summer co-op student. You may have seen his commits if you look at the open source WED repository. Ted is heading back to school now, so this post both discusses a little bit of what he accomplished and serves as public recognition of his hard work.
Ted spent the summer working on ‘workflow’ items for WED – that is, things that make WED easier for authors to use. One of the main features he implemented was a new ortho-photo importer.
If you have a GeoTIFF you wanted to use in WED 1.2, the process goes something like this:
- Make an overlay image out of your TIFFs.
- Run the “make orthophoto command” which will create .pol files and place them in the project.
- Delete the original overlay image, which unfortunately looks almost exactly like the POLs in WED.
- Resize the TIFF to a power of 2.
- Grind the TIFF in XGrinder.
- Edit the .pol file in WordPad to use the DDS file.
- Edit the .pol file i WordPad to add LOAD_CENTER (if you remember) – good luck getting all of those numbers right!
- Export your scenery pack.
- Get some astonishing error in X-Plane, because the odds of getting all eight steps right is approximately 0%.
The new process with Ted’s code is much better:
- Import the TIFF files into WED as orthophotos.
- Hit control-B to build your scenery pack.
- Go get a coffee – WED is grinding your TIFFs to DDS for you, which is a time-consuming operation. Don’t worry, it won’t grind them again unless you change the source file.
- Open in X-Plane and enjoy your orthophotos – with correct load-center directives!
No hand-editing pol files, no chance for human error, and of the four steps, two you already do now and one involves beverages. What’s not to like? This screenshot shows the results of importing the TIFF files into WED and the resulting scenery pack.
This level of automation is how all of the scenery tools should work and it’s nice to finally reach a point in WED where we can add this level of convenience.
For The Nerds
WED is an open source project, and this next part only affects users trying to work with the WED source code. (If you are not a programmer, you may want to skip to the next section.) Ted’s finishing the new Orthophoto importer completes the removal of CGAL from WED.
CGAL is an open source library that is critical to the DSF generation code that makes the global scenery. CGAL is also a very complex, heavy-weight library that can be quite finicky about compilers, brings in a lot of dependencies, and is generally hard to work with on Windows.
By dropping CGAL for WED, it makes the WED code significantly more accessible. Ted created a complete MSVC 2010/2012 project for WED, so you can just download MSVC Express, open the project and hit ‘compile’ – all of the libraries are included in the repository in binary form.
(For Mac users, unfortunately we are still stuck on X-Code 3.2/10.6 – to move WED to the modern development tools we will have to purge old APIs that Apple has dropped – just like we did on Desktop.)
We are not dropping the makefile system we use on Linux, and it should even still work on Windows, but finally mingw is no longer a requirement to get started.
Removing CGAL from the project should make a Lion-compatible Mac build possible too at some point.
Upcoming Releases
At this point I expect to do a WED 1.2.1 bug-fix patch relatively soon with a few usability fixes. Ted’s work will go into WED 1.3; I don’t know what the release time frame for 1.3 will be.
Posted in Tools
by
Ben Supnik |
WorldEditor 1.2 is done and declared final!
Laminar Research, creators of the X-Plane flight simulator franchise, is pleased to announce the availability of WorldEditor for X-Plane (WED) version 1.2.
WorldEditor is an airport scenery creation and editing tool for Laminar’s X-Plane. WED is intuitive and easy-to-use, as it features drag-and-drop scenery creation in a graphical environment that is designed for the typical X-Plane user.
WorldEditor takes a graphical CAD-like approach to creating airports. All airports are made up from a collection of different items or entities of a specific type. For example; runways, taxiways, windsocks, signs, and buildings. A runway has length, width, surface type, lighting and taxiway signs for example. WED is organized to create and edit each of these different items on an individual basis, so a user can add an item, then edit it’s characteristics or attributes at will.
A key feature of WorldEditor is the ability for a user to create an airport scenery and then send his or her creation to Laminar’s airport scenery database service. Once checked by Laminar for acceptability it is then included in the airport data when X-Plane users receive automatic X-Plane updates from Laminar.
Robin is ready to accept submissions of airport layouts including ATC traffic flows and default airport object placements using the new “Export for Global Database” function. We are also working on tutorials and additional documentation beyond the WED user manual.
