Something I’m seeing now that WED is in beta: airport layouts with the entire taxiway structure made from one really complex polygon.
I’m not really sure if this is a good idea. First the potential problems:
- I suspect that the creation of the taxiway layouts can get slow when the number of sides in the airport layout is really huge and there are holes. I don’t know this for a fact because we let the OpenGL libraries do the heavy lifting. Because this loading is done on a second CPU, it might not be noticeable to all users.
- The pavement can have only one texture direction per polygon, so multiple polygons may be necessary.
- Certainly in WED having a few large complex polygons slows down editing — if all else is equal, the tools work better with smaller polygons.
Now…overlapping pavement is generally bad (that is, there is a performance cost), but more sides are also expensive. More thoughts:
- The more square feet of overlap, the worse. So a small overlapping intersection is not so bad, but avoid layering a huge polygon on top of another huge polygon, which just strains the video card.
- Fewer segments are better. Consider two crossing taxiways…8 segments with overlap, but 12 by making a plus.
- But wait – the above example is misleading…if you need to change the light types so the blue taxiway lights don’t cross the intersection then you’ll need to add 4 more segments, so now it’s 12 and 12 – a wash. (In this case, having one big polygon is probably easier to manage.)
And for performance…
- Try going to your layout from far away and watch the last step of loading…if it starts to take a long time to “preload” things it means your layout might be a bit complex.
- Do not expect X-Plane to become faster at loading airport layouts…the limiting factors are proportional to complexity, so if you have a killer polygon now it’ll be pretty expensive later too.
One other note, from a conversation with Tom…WED splits vertices into a fixed number of segments (per zoom level) so splitting a bezier makes it smoother. X-Plane does not! X-Plane splits beziers based on the overall curvature, so adding more nodes without changing the shape has no effect.
So please do not try to use the split command to make X-Plane “smoother”. We’ll provide a rendering setting for this some day. The current value was chosen because anything smaller looks awful and you have to make it a lot bigger (read: a lot slower fps) to get an even marginal visual improvement.
In my previous post I mentioned that the scenery tools are separate code with a separate release schedule from X-Plane. This implies that the updates to the tool won’t be in sync with X-Plane. (The same thing is already true for the AC3D plugin and all of the wonderful third party tools that people have written.)
X-Plane’s scenery compatibility strategy is “backward but not forward”. In other words, if you open old scenery with a new X-Plane and it won’t open or looks different, that’s a bug (please tell us). You should never have to modify your OBJs or scenery if they are correctly made, and the definition of “correctly made” shouldn’t get more strict over time.*
Why no forward compatibility? Well, it’s possible to have syntactic forward compatibility (have X-Plane read a file it doesn’t understand because at the time of release the spec didn’t exist) but it’s not possible to have X-Plane display that file with any kind of sanity.
You could say “just leave out stuff you don’t understand”. Well, imagine if we had done that with apt.dat — imagine if X-Plane 840 tried to display an 850 apt.dat by simply skipping what it didn’t understand. Since taxiways are fully replaced with a new code, 850 layouts would have no taxiways at all if shown in 840. This is certainly unacceptable.
The only way around this would be to require all new features to have “fallback” content, e.g. have the file contain two copies of all taxiways. Since the structure of 850 and 810 taxiways are so different, this would basically require the author to do twice as much work…similar problems happen for animation commands, and missing datarefs.
So the content has to be older than the sim. What about tools?
Well, first, I don’t write the tools with forward compatibility. Since they are all free a user can easily get the newest tools to work (in a sane manner) with new content.
The tools do have to support old file formats though – we don’t usually post old copies of the tools (the AC3D plugin is an exception, since old versions work with old host copies of AC3D). Instead the tools are designed to support restricted file formats. For example, the AC3D plugin will export to OBJ7 or OBJ8 – authors targetting X-Plane 7 can export to OBJ7.
WED only exports apt.dat 850 files – this is by design from day 1, and it will not ever be an apt.dat 810 editor. (Use TaxiDraw if you need to do this.) However, as new X-Plane features emerge (and I am sure they will), WED will have options to target future X-Plane versions, or restrict the export to only features supported by older versions, perhaps with a warning if the WED document contains features that will be lost in an old export.
* One problem we have with this is: X-Plane isn’t a strict validator of scenery file formats, so it is possible to have mistakes in scenery that were always illegal by definition of the file format, but X-Plane doesn’t notice at first. When I detect a case like this, I try to make the problem a “warning” rather than have X-Plane quit, so that authors can get information on buggy files without users suffering too much. X-Plane will print only one warning per package per scenery load to keep things streamlined.
Some very advanced users have asked: can we change lights.txt. The answer is: please don’t.
Lights.txt is not a part of the “scenery SDK”, that is, it’s not a file whose format we will keep the same and allow you to modify. (The fact that it isn’t accessible via the library system is an indication of our intention NOT to make it part of the scenery system.)
The problem is basically this: lights.txt translates named lights into the inputs to our pixel shaders for the hardware-accelerated lights. That pixel shader is really new and likely to change a few times. If the shader changes, we might need new parameters not in lights.txt, requiring a fundamental format change.
For example: those who have poked in the named lights file have noticed that the hardware lights can either have directional or flashing properties. This is because they run on two different shaders, each taking only four input values. This was done a while ago, when we were using low level assembly language shaders. In the future we might merge the two shaders and have 8 params per light. This would give us more flexibility (directional flashing lights), more bus usage (pushing 8 params per light instead of 4) and fewer state changes (we have to change shaders right now).
My point is: we can’t predict what will happen, so we can’t safely expose these parameters. The best thing to do is: email me and request named light types. We can easily have hundreds of named-light types (see how many there already are just for airports!).
Named lights make our lives easier because it tells us WHAT to draw but not HOW to draw it. So when we put in the next evolution of the lights code, we can remap the named lights to look the best they can for the new technology, instead of worrying about how to map the old params to the new ones.
(I appreciate the input from the users who emailed me about this — it gives me more insight into what extensions to the scenery system would be useful.)
The livery system I described in my previous blog entry (proposed independently by several users) would imply a contract between the aircarft creator and livery creator. The way I look at it:
- The livery creator agrees to utilize the texture conventions established by the aircraft modeler.
- The aircraft modeler recognizes that changes in the fundamental texturing scheme of the airplane become much more “expensive” because they potentially invalidate a whole set of liveries.
This is an exceptional situation in X-Plane…we have avoided contracts regarding the layout of images in all previous cases. (Examples: you can’t use the library to replace an OBJ’s texture. You specify terrain via a text file that references the image file, not the image file itself, so information on how to use the texture isn’t part of the contract.)
But in the case of liveries, I think the exception is the inevitable outcome – a livery quite literally is a repainting of an aircraft, and the X-Plane community has been living with the limits of this kind of add-on for a while.
What this means is that it would be appropriate to override object textures in a livery system, but this feature is unlikely to appear anywhere else in the sim.
Any time two independently varying components need to interoperate, a contract is required.
Because X-Plane varies independently of third party add-ons (we release patches, you make new add-ons), any extensible part of X-Plane implies a contract. That contract basically says what legal things the add-on can do and how X-Plane will react.
We have to consider this “contract” with third party add-ons any time we modify the sim, and in some cases it means we can’t change things. How this applies to liveries will become clear later.
Libraries and PNGs
There was some discussion on X-Plane.org about whether it should be possible to share and/or override PNGs via the library system independent of their objects. I say “no” for this reason:
The library system connects independent third party add-ons, that is, components that vary separately. Therefore there needs to be a “contract” any time the library is used, between the package requesting something from the library, and the package fulfilling it.
My concern with PNGs is that a library PNG would have to have a fixed layout. But realistically the kinds of PNGs that people want to share get reorganized on a regular basis. In particular, people want to reference the default sceney PNGs. But where we have a contract, we are limited in what we can change, and reorganizing the default scenery is critical to our ability to grow our content. I agree with Aussie’s comments on x-plane.org that fixed-layout PNGs don’t add a lot of value to the library system.
File System Vs. Text Files
There are a number of design trends in all of X-Plane’s third party add-on systems. One is the use of the file system to specify modifications to the sim. For example:
- Put a “cockpit” folder in your aircraft, we’ll try it first.
- Some files in the aircraft folder must have certain names.
- Sometimes putting _LIT after a texture causes it to be used as a lighting map.
On the other hand, I’ve gone a very different route with the scenery system:
- Libraries are looked up via a text file, not via filename.
- In all scenery cases, the lit texture is specified in a text file, not by filename.
What’s the difference? Well, the file system way seems to be simpler for most users to understand. The text file mechanism is a lot more flexible. (Consider: we have versioning info in a text file, we can have each line in the text file clearly identify a feature, and so there is no risk of a file name being mistaken for a feature.)
Whether we use file names or text files to control extensions, doing so creates a contract, so we must ask: can we easily extend or safely modify the contract later? Can we express what we want using filenames (or text files)?
In the case of the scenery system, I think we need the full expressiveness of text files. Imagine if we wanted to provide a new kind of lighting that uses a different format of texture. Or a way to control at what time of day the lights get turned on? Or what if we want seasonal varying textures, and the rules for how seasons affect the light map aren’t simple? In all of these cases, a file name convention rapidly becomes unworkable.
On the other hand, the airplane system has done reasonably well with filenames. Based on what I see, the problems with airplane distribution mostly come from a lack of features (no livery support, no plugins built into airplanes), not file names as a convention running out of flexibility.
Sergio sent me a scenery package with the question: why don’t these two textures appear as the same color in X-Plane (they did in Photoshop). The answer is: gamma correction.
Some background (with much hand-waving): gamma refers to how bright the mid-tones of an image appear. (It’s a lot more complex than that, but that’s what Wikipedia is for!) Basically Macs and some other computers adjust the colors in an image to compensate for the deficiencies of CRTs, while PCs leave them alone. The result is that the same numeric color levels, when sent to Mac and PC hardware, result in brighter images on the Mac than the PC.
Since X-Plane is authored almost entirely on the Mac, the old complaint (around the time of X-Plane 6, with BMPs) was that X-Plane looked too dark on Windows. PNG addresses this issue: the gamma curve of the system an image was created on can be written into the PNG, allowing X-Plane to adjust the colors (making them brighter or darker) depending on what destination system we are running on.
Unfortunately, X-Plane isn’t too brilliant about this, in two ways, one of which isn’t our fault:
- X-Plane assumes the platform default gamma for both Mac and PC (that would be 1.8 for Mac and 2.2 for PC). If you read that background article, you know that this isn’t real clever of us. But it actually is better than doing nothing at all.
- Gamma in PNG files is optional; if we get a PNG file with no gamma information, we make the rather arbitrary assumption that it came from a Mac. Since X-Plane used to be authored on Macs, this seemed like a reasonable thing and in the case of no gamma information, we’re going to be wrong half of the time no matter what.
If you open up the default scenery PNG files in Preview, you can see the gamma inforation – 0.4545. This is 1/2.2, meaning the files are encoded to PC gamma standards. It turns out that one of the textures Sergio sent me had no gamma information, so X-Plane assumed Mac gamma (0.5555, or 1/1.8). Thus the brightness of these textures were being adjusted by different amounts.
My recommendation to authors is simple: make sure that all of your PNG files always have gamma values written into them. Otherwise there is a risk that the default gamma guess that X-Plane makes will not be the one you authored under, causing color shifts.
Ari asked a good question regarding sloped runways and the new apt.dat 850 format:
Am I understanding it right that airport taxiways, ramps and runways are from now on going to be merged into one big mesh, instead of bunch of rectangle pieces overlapping each other? If yes, will this finally allow us turning on sloped runways option in X-Plane without any of the current side effects?
This brings up some interesting questions. First the most basic answer:
- apt.dat 850 prrovides curves, irregular taxiway shapes, which will allow you to create complex taxiway shapes with only one piece of pavement, rather than many overlapping ones.
- X-Plane still honors the order of apt.dat 850 for drawing, so you can also overlap and get visually consistent results.
- We recommend using a smaller number of curved taxiways rather than many overlapping rectangular ones because X-Plane can handles this case more efficiently. It is not necessary to build the entire airport out of one taxiway though.
Now the second part of the second of this question is a little more complex, because the cause of bumps in the scenery changed.
Bumpy Runways in the Good Old Days
Back in X-Plane 806 there was a fundamental problem with the way we did sloped runways that made them virtually unusable: while the corners of each rectangular piece of pavement would sit directly on the terrain (no matter what the terrain’s slope), the area of the taixway was formed by a flat plane. This means that the middle of the taxiway might be above or below the terrain.
Now the real problem comes when we have two taxiways that overlap. Because they are only aligned to the terrain at their corners and not centers, there may be differences in their height when one taxiway’s corner hits another taxiway’s center (which happens a lot). As the airplane travels from one taxiway to another, the elevation of the ground changes instantly, inducing a major jolt to the suspension. At high speeds these damage the airplane’s suspension.
Bumpy Runways Now
In X-Plane 850, we break all runways and taxiways (new and old) into multiple pieces each tiem the terrain underneath them has an edge. The resulting taxiways are then aligned to the mesh at their corners. But since no taxiway center goes over a mesh corner, the taxiway “hugs” the mesh perfectly. And since all taxiways hug the mesh in the same way, there is never a height gap between taxiways.
It’s the Mesh, Stupid
So why do we still have bumps in X-Plane 850 if we so carefully make sure the taxiways exactly reflect the mesh height? Well, you’re effectively driving on the terrain, so any bumps are ones from the terrain. Simply put, even with the new system the usability of sloped runways is only as good as the underlying terrain.
Now our meshes come from SRTM data, which is radar data – it naturally has a certain level of noise and “speckle” which makes it pretty unusable for airports…airplanes are very sensitive to even small bumps during takeoff.
We attempt to “condition” the elevation data for airport use, smoothing out hills and bumps. Unfortunately our algorithm doesn’t always work right. The X-Plane 8 US scenery was way too bumpy to be usable. The 7-DVD set is better, but still makes bumpy airports in a few cases:
- If there was no airport in the apt.dat file at the time of scenery creation, no conditioning was applied, and the underlying terrain is probably inappropriate for draping.
- The scenery creator has a bug that causes airport flattening to fail when it’s very close to water. For example, the big bump in the runway at KLGA is due to a water-airport interaction.
- I think that flattening across DSF tiles can have problems too.
If you’ve been thinking — wow, the diagram for 850 has a lot more triangles (10 vs 4) than the one for 806, you are right. Fortunately, the number of triangles, all part of one taxiway, in an airport layout, doesn’t really affect frame-rate, since this is handled by the GPU.
But this is also a case where a few curved polygons can be much more efficient than several overlapping ones – when we cut up the taxiway based on the mesh, if there is ovelapping pavement, each overlapping taxiway must be cut, multiplying the effects of the mesh on triangle count.
It also turns out that in the real case this is somewhat moot: because X-Plane smooths the airports and then induces triangle borders around the edge of the airport. Since the interior area is so flat, it doesn’t require a lot of triangles, and therefore the trianglse inside an airport tend to be big, so the number of times we have to cut an actual layout is quite small.
Since before I’ve been involved in X-Plane, scenery has been broken into tiles. You know the naming scheme…+42-072.XXX is some kind of file for Boston. Perhaps it’s a .env or a .dsf, but what is unchanged is that the world is broken up into bite-sized pieces.
Tiling is very necessary for X-Plane…it allows the sim to rapidly load only the information we are interested in. The world is too big to go fishing for the relatively small amount of data that is loaded at one time.
The source data that is used to build global scenery is also tiled – if you look at the SRTM files, they often have names like N42W072. It’s no surprise that global data requires tiling to make it manageable.
When we produce global scenery, we work on a per-tile basis. We load the raw tiled data* into source XES files (one per tile), then process and convert that XES file into a DSF. By working one tile at a time, we limit how much RAM we need.
But if you used WorldMaker on an airport or city that spanned a tile boundary, you know how annoying tiling can be. With WorldMaker you couldn’t see both halves of an airport at once.
WED will not use tiles. For custom scenery packages, the total data is not so large that we have to tile. When you work on a scenery package in WED, you work on the entire package at once as a single seamless workspace. When you export your work, WED will split the data into tiles as needed. This will mean:
- You do not have to decide in advance where you are going to work. All scenery packages cover the entire planet, and DSFs are created only as needed.
- You will be able to work on more than one “tile” of area at a time, because there are no tiles.
- You will be able to work on airports that span tile boundaries seamlessly.
First, I just want to be clear: I am not announcing any future features for the scenery tools. I am not saying when they will be released, and I am not saying what they will do, because honestly I do not know. There have been too many cases when users have emailed me and said either “I thought you guys were going to do X” or “you guys promised you were going to do X”, so now I am officially paranoid.
So this blog post is not about what the future scenery tools will do – it is simply a discussion of the difference between editing source data and compiled DSFs.
When we make the global scenery, we start with a bunch of source data that roughly consists of: road maps, coastlines, elevation, landuse, climate data, airport locations, etc. When we build a DSF out of it, we “bake” these items together into a single file. During this baking process, our tools apply some “integration effects”. Here are a few of the more obvious examples:
- Terrain under airports is forced to be an airport grass texture, appropriate for local climate data.
- Roads are removed from airport areas.
- Intersections are computed for highways – that is, a highway and city street form an overpass, but two city streets become a real intersection.
- Generated Buildings are put around the roads, not under them.
- Buildings are oriented to “face” the slope of the ground they are under, based on their shape.
That’s not a complete list, but it gives you an idea of some of what goes into making a DSF. All of this information is precomputed and represented in the final DSF. But consider this last point: the DSF contains the actual orientation (north/south/east/west) of each building. It does not contain the slope underneath the building and it contains no information about which buildings need reorientation. In other words, the results of the process, not the inputs to the process, are present.
So consider what would happen if you could simply edit the data in the DSF:
- If you moved an airport, the airport grass would stay in its own location.
- If you moved an airport over a road, the road would still be there.
- If you changed a city street to a highway, it would not form an overpass.
- If you moved a road on top of a generated building, the building would remain in place.
- If you changed the ground elevation, buildings would not change their orientation to face the new slope.
If you changed the source data and re-ran the DSF building process, these effects would occur. But remember, we need elevation, land use, etc. to build a DSF from scratch, and the DSF itself doesn’t contain its source data.
So we have two possible strategies for editing DSFs:
- We could build a DSF “retouching” tool that let us make very small changes to the existing DSF without having any source data. None of these effects would “work” so authors would have to make very small changes and then hand-fix any problems that appeared.
- We could build a DSF “rebuilding” tool that let authors make new DSFs from source data. All of the effects would look good, but authors would have to get some of the source data. (We could post our source data, or provide links to places where it can be downloaded.)
Note that we can’t have our cake and eat it…we cannot get the “integration effects” listed above unless we go back to the source data. If I can stretch my cake metaphor to the breaking point, once we make our cake, we can’t easily remove the flour and add another egg – we need to start over with the raw ingredients.
Which strategy will the scenery tools use? I don’t know yet. I am focusing on airport and overlay editing, which sidestep this issue a bit (we can easily edit an apt.dat 850 or overlay from the final product). We may do a bit of both strategies – it depends on what users want and what we can code efficiently.
Why don’t the finished DSFs contain everything we need to edit them? The answer is size. The finished global scenery was about 56 GB of DSF. When I last checked, the raw data that forms the DSFs was at least 100 GB or more. So for each DVD we shipped of scenery, we’d have to ship two more DVDs of source data, for a 21 DVD set, most of which wouldn’t be useful to most users.
A user reported a bug: the 3-d buildings in all parts of the world outside the United States* are European-style, even in Australia, Canada, China, etc.
It takes two things to see 3-d objects in X-Plane:
- Location data in the DSF you are flying over.
- 3-d models (stored in OBJ files) that match the definitions called for by the location data.
Now we have location data for our “generic” objects (objects that are placed pseudo-randomly based on local topography and roads, to synthesize cities) for all urban areas that are covered by the 7-DVD global scenery set. But we only have two sets of artwork: a set of US-style buildings for use in the US, and a set of European-style buildings for the rest of the world.
So we had an unfortunate dilemma. Either we could have European style buildings everywhere, which looks funny, but at least it’s 3-d. Or we could use the European-style buildings only in Europe and have no 3-d at all everywhere else. Both options make some users unhappy and neither is very good. We chose to have the European buildings everywhere – my guess is that it generated slightly less complaints.
Well, the silver lining is: it’s actually really easy to remove the European-style builldings from the rest of the world (Canada, Australia, China, Russia, Africa, etc.). All you have to do is make a custom scenery pack that applies to the regions where you don’t want the buildings and maps them to a blank object. I have built such a custom scenery pack and posted it here. If you install this scenery pack, then Canada will be devoid of 3-d, but at least there won’t be mismatching architecture.
This begs the question: why not use the US buildings to populate some of those regions? The answer is that the US buildings are built to a different spec than the European ones, because the road grid data for the US is more detailed than the rest of the world. So amongst our current artwork, only the European buildings will work globally.
* When I refer to the United States, I really mean the continental United States. Sorry to everyone in the beautiful states of Hawaii and Alaska. This is just an artifact of how the data was imported – I could not get the Census data for Hawaii, Alaska, or the territories to import, so I reverted to the generic “global” data.
