Every now and then someone tries to set a taxiway in WED to “transparent”, and it pretty much never does what the author expects. Here’s a brief explanation of what’s going on.
Runways
“Transparent” is one of the many built-in surface types that runways can take on in X-Plane; more commonly you would pick asphalt, concrete, or grass. So what is a transparent runway?
The answer is: it is a runway with:
No texture. That means you see nothing where the runway is. (This is fast by the way; we are not drawing the runway with a 100% clear texture, we actually don’t even place the polygons.)
No physics. The runway does not change the physics from the underlying ground.
At this point a sane author is thinking: then what does a transparent runway actually do? Why have a no-op?
The answer is: user interface and lights.
A transparent runway is still a runway; and thus X-Plane can know “hey, there is a runway 3L at KXYZ airport.” X-Plane even knows where the runway is (since the transparent runway has ends and a width) and can thus start your aircraft ont hat runway.
A transparent runway has approach lights and all other types of runway lights. A few of the common approach light fixtures with “rabbit” strobes are incredibly annoying to build by hand (you can do it, but you basically need a plugin, a gajillion objects, and super-human patience).
So the transparent runway lets you do the graphics and physics with draped polygons and leave the hard things (user interface and lights) to us.
The primary thing to note: the physics are up to you too, and the expectation is that you’ll do the physics with the same tool you’ll use for the graphics. So if you put a draped .pol file down, you can set its surface type (with the SURFACE directive) to match the visuals of the texture you are using.
Taxiways
Taxiways follow the same logic, and thus they are really quite silly.
Physics and graphics are up to you – the taxiway does nothing.
There really isn’t anything else to a taxiway; it isn’t part of the UI, and you can place taxi lights directly using light/line strings in WED. You don’t actually need the taxiway polygon.
The fact that you can make a transparent taxiway in WED is actually a bug – the UI simply knows all surface types and does not have special code to say “hey, for a taxiway this is silly!”
What Transparent Taxiways Are Not
Transparent taxiways and runways are not a way to get the physics without the graphics. Instead, get the physics by putting a surface directive on your draped polygons.*
* There is one inefficiency here: if you have a huge draped orthophoto that covers a wide area, it will contain imagery that spans multiple surfaces: grass, concrete, etc.
Here is my suggestion: overlay a second polygon (with a repeating texture at very high res) with some kind of “grit” overlay. Place this only on the areas with concrete (or asphalt, depending on the kind of grit you use) and set the overlay’s surface type to match the overlay’s appearance. This way the polygons you must place for physics correctness at least add visual value too.
WorldEditor 1.4 is almost ready for beta, and among its new features (there are several big ones) one is very important to the X-Plane Airport Gateway:
WED 1.4 can browse airports on the gateway and download scenery packs directly into the WED workspace.
“Direct download” is really important for a few reasons, some of which might not be obvious:
It saves time. Getting a gateway airport, even if you just want to look at it, is much faster when you don’t have to download the scenery pack, unzip it, install it, then import it into WED. Once you use direct download, you’ll wonder how you lived without it.
It prevents mistakes. We have seen airport submissions where a user clearly downloaded a scenery pack, imported only the apt.dat (but not the DSF files) and then uploaded it. We cannot take scenery packs like that, because they fundamentally remove the 3-d data from the airport. (A fundamental policy of the gateway is that if you upload a scenery pack and one already exists, yours can’t be worse than the existing one in some way. We have to always move forward.)
It provides ancestry information. When you download and then upload directly from the Gateway in WED 1.4, WED provides the scenery pack ID of the original pack as the “parent” of the new pack. This means that when Julian goes to look at an upload, he can look at the “original” and more rapidly spot problems. If your pack is the same as the original except for taxi signs, he only needs to inspect taxi signs.
It prevents data loss. DSF is a slightly lossy data format – that is, if you get your data back out of a DSF file, it won’t be exactly the same as what you put in. (It is like a JPEG image in that regard.)
More on that last point: DSF stands for Distribution Scenery Format – it was meant as a way to make final scenery packs; it was not meant as an interchange format for continuous editing. So users are constantly importing and exporting DSFs to do work, small rounding errors (“bit rot”) will creep into our 3-d, and features that were perfectly aligned might not be well aligned after 4 or 5 edits.
The internal format for scenery on the gateway is not binary DSF files, so doing round trips to the gateway has much less “bit rot” than importing scenery packs.
Finally, DSFs are tiles; if your airport spans a DSF boundary, all DSF features (polygons, fences, etc.) get split along the DSF boundaries.
So if you import a scenery pack that you downloaded, you’re getting the split version, which is harder to edit and work with.
The internal format of scenery packs on the gateway is not split, and thus you can edit the original in almost the same form as it was uploaded.
WorldEditor 1.4 beta should becoming in “weeks” (and hopefully not that many weeks); I’ll post here when we are ready for beta testers.
I’ve been fielding crash-related bug reports for X-Plane 10,30, and a lot of them are turning out to be caused by third party scenery packs.
Now there are two possibilities for why a third party scenery pack might crash X-Plane:
There is a bug in X-Plane. In this case, I want to fix the bug!
There is something wrong with the data in the scenery pack (that is, the data doesn’t meet X-Plane’s expectations). In this case, I want to change the result from a crash to some kind of clear error report that an author could use to fix the problem.
The Old Rules
During my work on the scenery system for X-Plane 8 and 9, my view was “garbage in, garbage out” – that is, if you feed junk data into the scenery system, the result would be unpredictable and could include a crash. In X-Plane 8, scenery load was completely synchronous – that is, your flight just stopped while we loaded files, so my idea at the time was that validation of scenery files should be done ahead of time, perhaps by the tools that write those files.
That logic (if it was even a good idea back then) doesn’t apply now. Loading is done entirely on other CPU cores and does not interrupt flight; we can take the CPU time to make sure incoming data is valid. And the cost of not validating the data is clear:
The instability from badly formed data results in a sea of crash reports – if we have real serious sim bugs, it’s much harder to catch them in the noise.
Because we don’t trap bad data, the end result crash can be in the OpenGL driver (due to cascading errors), and due to the way 64-bit crash detection works, we don’t get any trace of the contributing X-Plane code path.
Authors who create scenery packs can’t tell if their pack is the source of crash, or how their data induced a crash.
My goal is to chase down crashes caused by third party scenery and both fix any bugs that emerge and tighten up error checking to detect the cause of crashes early on, in a way that is useful to authors.
The New Rules
Because X-Plane has not run validation over all scenery files in the past, there are probably scenery files in use now that contain data that I would consider illegal (either by the documented format, or just by my own view of how the file format was meant to work; I realize that second definition is quite useless to authors). But for illegal data that doesn’t crash the sim, I can’t just go in and change X-Plane to reject that data in the next patch. (“Surprise! Your scenery packs won’t load!” isn’t cool.)
So my thinking for validation is:
Detected error conditions that would have caused a crash 100% of the time can be handled by any clear means possible, including quitting the sim with an error message. If the sim can handle the error and continue, that’s better, but even a controlled quit with diagnostics is an improvement over an uncontrolled crash.
Detected error conditions that would have crashed the sim some of the time need to be logged but can’t quit the sim or cancel loading of the entire scenery pack; while a crash is serious, users were probably still enjoying the scenery pack (and merely being annoyed at how unstable their flying experience was). The user experience shouldn’t get worse.
Detected error conditions that have only minor effects (e.g. out of range values that might cause rendering artifacts) whose legal values are well-documented can be logged, perhaps with a noisy error. The pack works, so quitting or refusing to load is not acceptable, but if the error is too quiet, the condition will never get fixed.
Detected error conditions that have only minor effects whose legal values were never documented need to be logged unobtrusively; it’s not fair of me to penalize scenery packs for failing to read my mind.
On those first two points: one of the issues that complicates validation of scenery packs is that some errors are not discovered until mid-flight, because scenery is processed while you fly. This means that a large scenery pack might contain areas that are problematic to fly in; if the original author doesn’t cover every inch of scenery in a test flight, the author might not know about the problem.
The ATC system takes a more aggressive approach: it validates every single .atc file at load time even if it doesn’t need to use the data until later. (Virtual air traffic controllers are instantiated near the aircraft as the user flies, but the data in the .atc files that will be used to set up those controllers is checked at load time.)
This approach wasn’t too expensive performance-wise: the .atc files are small and the ATC system has to read them anyway to index them for later use. I am not sure how to best replicate this for scenery; consider Alpilotx’s HD scenery mesh – reading every single DSF in Europe at load time isn’t acceptable performance wise but without that, how can he know that his scenery doesn’t contain areas that could crash X-Plane?
Please Send Me Bogus Scenery Packs
Here’s where I need you to help: if you find X-Plane is crashing, and you find that removing a scenery pack fixes the crash, please report a bug, including the scenery pack (or where to get it) and where you had to fly/start up to induce the crash. Even if it is the fault of the pack (and it may not be – it may be an X-Plane bug!!!!) I still want to clamp down on the failure condition.
If you develop 3-d models for X-Plane using Blender 2.49 on OS X, do not upgrade to OS X 10.10 (Yosemite). Something changed in the OS that causes the python scripts to not load, which means you won’t be able to export.
(As a side note, I am of the opinion that if you use your computer for work, you should not install major OS upgrades without investigating first. If Apple is going to break support for something intentionally, it will be in a major version. Once you install the new OS, going back is hard. So why jump in so fast? These days I don’t believe that any one OS update is so much better than what was there before that it trumps keeping your productivity software working. But then, as Chris points out, I am older than dirt.)
Update: Chris found a way to make Blender 2.49 work with Yosemite. For reasons unclear to us, the Python 2.5 framework that comes with the OS in Yosemite is just a symlink to version 2.6 (rather than the real Python 2.5). If you delete the symlink and put in place the old 2.5 binary from an older OS, Blender starts working again. I’ll post instructions when we have something written up in readable form, but this is a tricky mod at best – you have to change admin-writeable files in your library.
Update #2: Hi guys, Chris here. Here’s an outline of the steps that you’ll need to take…and you’ll need to be an administrator of course since you’ll be running sudo. A basic disclaimer….you should have a full bootable and recent backup of your system anytime you mess with system files (even though this should be relatively straight forward). These instructions worked for us but you should follow them at your own risk.
Go into your time machine archives or find a friend that has 10.8 or whatever version of OSX you used to be on, and obtain a copy of the Python 2.5 version framework at this path and put it somewhere accessible:
cd /System/Library/Frameworks/Python.framework/Versions/
To be safe, type:
ls -al
to get a full directory listing. If you see 2.5 -> 2.6, you have a symlink from 2.5 to 2.6 which is what’s causing Blender to use the wrong Python version. IF YOU DON’T SEE THIS, YOU DON’T HAVE A PROBLEM!
Delete the symlink by running
sudo rm -rf 2.5
Copy the old version of 2.5 into this location (pay attention to the dot at the end!):
sudo cp -R some/path/to/the/good/2.5 .
Now as good practice, let’s put the permissions the way they should be:
sudo chown -R root 2.5
sudo chgrp -R wheel 2.5
As one of our artists discovered, if Step #7 fails for you because “wheel” is not a known group name, it might be a good idea to run Disk Utility and let it “Fix Permissions” on your drive.
For about a year there has been a subtle bug in how X-Plane draws taxiways: if you build an S-curve shape out of a single bezier and it is almost perfectly symmetrical, X-Plane would go “nah, why bother” and replace it with a single straight line segment.*
So in 10.30 I fixed it, and the result was a bunch of broken airports with missing pavement. (YMML is high on this list!)
It turns out that these airports have authoring errors – typically a segment of pavement that should be straight instead being formed by two overlapping beziers. This is definitely wrong, but due to the bug, X-Plane would simplify the overlapping S curve into a single straight segment and the layout would work. Only now that X-Plane correctly renders the S curve does the taxiway fail (because taxiways may not have overlaps). So two wrongs may not make a right, but they do make a “hey, that looks okay, let’s ship it”.
Note that the overlaps depend on the rendering setting of X-Plane – a different S curve is formed at different rendering settings; the overlap that causes the taxiway to disappear may only appear at a particular rendering setting.
For 10.31 I am going to undo my bug fix. This doesn’t make me happy, but I think it is necessary:
We have no idea how many airports have their taxiways broken by this bug.
Authors have no easy way to detect this problem, other than re-testing every airport at every rendering setting.
Even if an airport looks okay at all rendering settings, future rendering settings may cause the problem.
This is too much uncertainty to solve ‘by hand’. So my plan is:
Undo the code change for 1031. YMML and friends comes back.
Develop validation code in WED to detect this kind of authoring error.
Ship that version of WED so new authoring work will be checked.
Run the WED code on all airports and make a list of ones that need repair.
Fix all of the known problems in the airport gateway.
Redo the code change so X-Plane is correct.
This isn’t going to be a quick process, but then it can’t be, because third parties have apt.dats shipping now that only work when X-Plane has the buggy taxiway code in place. So we need to ship WED and then give third parties enough time to go back and check their layouts and fix them if necessary.
I expect to get a 10.31 beta with the taxiway code changed back this week.
For WED validation, I have some test code to detect errors but it isn’t ready yet. The problem is that it’s not good enough to detect errors with overlapping beziers**; we have to consider two bezier curves near enough to each other that with the error in rendering introduced by WED’s rendering settings, we get an overlap. (So authors, better be safe than sorry in creating your pavement.)
If there’s a moral to this story, I think it’s this: when we (LR) don’t provide good tools for authors to validate that their work is correct, the resulting body of work will end up with subtle errors.
* X-Plane renders beziers by measuring how far the mid-point of the curve is from the average of the ends. As long as this distance is ‘too far’ and the iteration count isn’t too high, X-Plane divides the bezier in half and repeats the process. In this way beziers are broken into enough line segments to approximate the bezier within a minimum error limit. The rendering settings control the error limit.
The bug: if the curve was a ‘balanced’ S curve the mid point of the curve was the average of the end points and X-Plane went “great, no error” and stopped dividing.
** Which is already not an easy problem – the analytical solution for bezier intersection is a 9th order polynomial!
August 28, 2014, Columbia, South Carolina. Laminar Research is pleased to announce the release of its Airport Scenery Gateway, as well as an updated and improved version of its airport scenery tool, World Editor 1.3. The Airport Scenery Gateway can be accessed on the internet at http://gateway.x-plane.com and the new and improved version of World Editor is available for download at http://developer.x-plane.com under Scenery Tools.
The Airport Scenery Gateway was created to facilitate the populating of X-Plane with 3D airport sceneries that are consistent in quality and appearance, error-checked, and will work with a standard installation of X-Plane 10. During the past several months Laminar Research has received hundreds of airports designed by World Editor users. World Editor is an airport scenery creation and editing tool for Laminar’s X-Plane 10 that allows the typical X-Plane user to create airport scenery.
By establishing an internet gateway for scenery submission, it is now possible with World Editor 1.3 for users to automatically submit airport scenery contributions directly to the Airport Scenery Gateway. X-Plane users can visit the Airport Scenery Gateway to search for airport scenery submissions. The scenery can then be immediately downloaded into X-Plane 10 before it appears in a scheduled X-Plane update.
If your airport is not already in X-Plane 10.30, please use the Gateway to share it via WED 1.3; do not send it directly to Robin. At this point Robin should only be getting navaid patches. You must use WED 1.3 to upload to the Gateway!
The airports that are already in 10.30 are uploaded by ‘WEDBot’, which is an account we used to transfer our existing collection into the Gateway. As users share data, we will start to get good attribution; I think this is important because we will know who to contact to sort out issues with airports.
If you could not upload an airport due to a missing ICAO, we are working on this now.
We are working on a Linux build of WED 1.3 and will post it as soon as we resolve the chaos of shipping a working app on multiple distributions. mroe has done great work to make Linux happen.
We will periodically ‘roll’ shared airports into X-Plane via point releases to the sim. I do not recommend collecting a huge pile of Gateway custom scenery packs; they will simply conflict with newer data from X-Plane when it comes out. The ability to download a pack directly from the Gateway is meant for authors who want to view the state of a pack and possibly modify it. The gateway is not meant to be a scenery file sharing site.
Finally, I think this has been discussed before, but…custom scenery authors:
Always put exclusion zones around your custom airports! Even if there are no 3-d buildings at an airport today, they may appear in the next update.
With the Gateway open, 3-d is going to appear quickly!
X-Plane 10.30 beta 8 is mirroring to our servers now; this beta contains a pile of recut DSFs, fixing some of the worst DSF bugs. I will post a complete list of recut DSFs in the release notes; what follows are just a few hilights from really prominent DSFs.
Terrain recut to match the complete airport.
The truckee River is back, connecting Pyramid Lake and Lake Tahoe
The Tennessee River is back.
The lakes near Edwards AFB are now Salt Flats. One is still wet – it wasn’t fixed in OSM when we pulled data.
Terrain recut to support new runway in Tokyo.
The Ottowa River is back, eh?
Botany Bay is now…a bay.
Terrain recut around new runway development.
Beaches underneath the Manhattan bridges fixed.
Lake Hood restored.
The mountains are fixed – the originals were affected by a bad DEM.
Terrain recut to reflect new airport area as they expand KORD.
Terrain recut with airport included. This airport is still a bit wild, but have you seen the real airfield?!?!
I’m not sure how long the mirror will take to finish; I’m hoping we’ll go live with the beta tonight or tomorrow.
The X-Plane updater will download scenery updates if (and only if) you have the base scenery installed. So if you have the entire world installed, you’ll get the entire set of updates (the DSFs themselves are about 500 MB); if you only have the US, for example, you’ll get some tiles but not others, and if you just have a demo, you’ll just a core file update.
If you install scenery later, simply run the updater again (the install-scenery function gives you a chance to run an update at the end of DVD scenery install) to get any fixed DSFs you might not have gotten at the time.
(Alpilotx had a good suggestion for the installer that I’d like to do in the future: an option to skip scenery updates. This would let users get a sim update immediately, and then fetch the scenery later if bandwidth is tight.)
Robin, Tyler and I have been going through the airports that have been sent in since X-Plane 10.25; the good news is that there are over 350 batches sent in!*
But – I had to throw thirteen of them overboard; they were using OpenSceneryX. For airports that are going to be sent to the gateway and become part of the default airport file for X-Plane, you can’t use third party libraries! Everything you use has to ship with X-Plane.
WED 1.3 has user interface features to help hide third party libraries, but in this case you’d think that having the objects in the opensceneryx/ folder would have been a hint.
It’s not a coincidence that this was the same thirteen entries not sent in the right format; WED 1.2.1 won’t do an export for the global database with third party libraries, and this will be true with WED 1.3 as well, where the export is uploaded directly to the gateway.
I have been working on visibility and fog-related bugs in X-Plane 10.30; two fixes will improve high-altitude long-visibility viewing:
10.25 has a bug where the fog color changes abruptly at the cut between DSF and planet rendering, particularly when atmospheric scattering is on. This is fixed in 10.30
The 64-bit version of X-Plane 10.30 will load 12 DSFs instead of 6, for longer-range drawing of detailed DSF terrain.
These comparison shots show 35k above KSEA looking at Mount Rainier with max visibility. (Note that at 35k feet, it doesn’t make a huge difference what ground visibility you pick – the planet-wide upper atmosphere model is almost entirely in control.)
X-Plane 10.25 32/64-bit
X-Plane 10.30 32-bit
X-Plane 10.30 64-bit
These pictures are taken at “very high res” – the most my Mac Pro can survive with an old GPU; you might get a little bit better definition on “extreme” res.
More DSFs
A few notes on loading more DSFs:
My current plan is that the 64-bit version will always load the 4×3 DSF box – if we have to make this a setting for compatibility we will, but I think it’s preferable to not add more rendering settings.
The 32-bit build will stay with a lower DSF count for memory reasons; generally speaking, I don’t think we will have any additional visibility improvements in the 32-bit build since we are almost always going to hit a memory wall. If you have a 32-bit OS, consider upgrading to 64-bits! (In particular, if you are running Windows XP, please upgrade to Windows 7 or 8 64-bit – there are no more service packs for Windows XP, so you’re just asking to get malware!)
In 10.30 the DSF loader can load up to 4 DSfs at a time (for 4×3) or 2 at a time (for 3×2). So if you have a multi-core machine, the load time should be better even though there are more DSFs being loaded, thanks to multi-core. (The limiting factor here is that adjacent DSFs can’t be loaded at the same time because their edges need to be matched.)
I am still hoping to address other low-visibility fog issues, but that code is not complete yet.
The Planet Could Look Better
X-Plane renders nearby terrain using DSFs, but it renders the very far terrain and entire planet using a single “planet” model, which is a textured sphere displaced by a normal/height map. As of X-Plane 10.25 (and 10.30) we are not including as much detail in the 3-d planet mesh as the data on disk contains.
This pair of pictures is 45k above LOWI; the second texture has the mesh spacing on the planet artificially increased from 3 km to 2 km. (The data on disk is at 1 km spacing, but my machine can’t cope with that.)
10.25, 10.30
Extended Planet Detail – Maybe for 10.40
You can see we pick up a little bit of definition in the far mountains. In the long term, I think we could ship a 500 meter planet mesh, which would make the far view in X-Plane as good as the close view used to be in X-Plane 7.
I’m not sure when we’ll ship a higher density far-planet, but I think it will have to be post-10.30.
If 1 km or 500m seems like bad resolution, do consider how far away the planet mesh is. With a 4×3 DSF box, the planet starts 100 km from the viewer. At 90 degrees FOV (an extreme case, but it makes the math easier) the screen is 200 km across at the DSF cut-over. With a 2 km planet grid, that means we will show 100 planet vertices left-to-right. At 1080p the planet triangles are thus at 20 pixel increments – not bad for a low res mesh.
Last week I spent a little bit of time looking at a bug: some ILS approach paths have tall buildings in the way. For an example, fly the ILS 22L into KBOS, and look to your right on short final. I hope the people living in those apartment buildings like watching airplanes!
A Library Bug
It looks like the bug is caused by a bad mapping of art assets in the library. The global scenery DSFs contain virtual paths for art assets, which are then mapped to real autogen art assets by the library. The virtual paths indicate the legal height ranges that the autogen can extend to, e.g. the library path
lib/g10/autogen/urban_high_solid_f_30x40_v0.agb
Indicates that this art asset fills a square block at least 30m deep with roads on all four sides, and the buildings should be between 32 and 40m tall for the tallest ones included. (The actual encoding of the library is partly handled by scripts we use to build the global scenery, and the encoding is byzantine enough that neither Alex nor I understand the entire thing.)
Well, if the virtual library path calls for a low building but the physical art asset contains tall buildings, you get skyscrapers in places you did not want them. And that is exactly the bug causing the ILS incursions.
I have an edit to the library paths for our next release (probably 10.30) that should fix this problem. The good news about this being a bug in the library (and not the DSFs) is that a single library fix will fix the problem everywhere.
For the Brave
(This is a hack to the sim…don’t try this if you aren’t willing to break your copy of X-Plane. If you try this and your sim dies, please don’t contact me or tech support!)
If you want to patch this bug yourself now in 10.25, simply locate the file
and replace it with this file: library.txt. Warning: because this is a mod to the sim itself, when you go to update, you will get a warning as to whether you want to overwrite the file. When the next beta or release comes out, do overwrite this file!
What About the Raw Data
I also wrote some code to attempt to ‘protect’ the ILS approach path in our DSF generator. Here’s a picture:
What you see here is a color-coded trace of height limits for autogen. Red are the strongest height restrictions, yellow less so. The height restrictions take into account not only the glide slope, but also the terrain; in the case of KSAN, the airport is at the bottom of a hill next to a mesa, so the ‘red’ (low flying planes) zone is quite large – even when a plane is on a 2 mile final, the plane is low to the ground because the ground is higher up.
The logic of the code is to let all ‘known’ tall buildings exist, but limit any generated tall buildings to follow the height restriction.
Here’s the thing: having coded it, I have not caught any cases where the autogen buildings were violating the height restrictions!
Buildings that we generate (without obstacle data) always have a maximum height that is a fraction of the FAA-spec’d tallest building. The idea is that if the FAA says there is a single 80m (roughly 20 story) building, then the nearby buildings are probably somewhat tall, but they’re not also 80m – if they were, the FAA data would include them. So we might generate some 40m buildings near the 80m building to create a plausible ‘neighborhood’.
From what I can tell, the fact that the generated buildings are so much lower than the FAA datapoints keeps them out of the approach path, and the actual cause of tall buildings has been the library path misalignment. (For example, in the KBOS 22L case, the block has an 8m height restriction and yet the buildings are either 50m or 70m tall, I can’t remember which.)
I think I am going to keep the ILS-protecting DSF generation code around – it’s useful to have approach paths noted, and it’s something we’ve been intending to do for quite a while. But the library fix is what will make a real difference.
