There’s a slight performance win to be had by grouping taxiways by their surface type.
Now clearly if you have to have an “interlocked” pattern of asphalt on top of concrete, on top of asphalt, this isn’t an option.
But where you do have the flexibility to reorder, if you can group your work by surface type, X-Plane can sometimes cut down on the number of texture changes, which is good for framerate.
X-Plane will try to do this optimization for you, but X-Plane’s determination of “independent” taxiways (taxiways whose draw order can be swapped without a visual artifact) is a bit limited and can only catch simple cases.
For what it’s worth, interlocked patterns of surfaces were much more a problem with old X-Plane 6/7 type airport layouts, where the taxiways were sorted by size, and there could be hundreds of small pieces of pavement.
Some of the newer text file formats for art assets (.lin files, roads, etc.) require you to specify how wide the texture is.
Huh? Why does X-Plane need me to tell it how wide a texture is? Doesn’t it know?
Well, first, the answer is: no! When X-Plane builds the roads and draped lines, the texture that is needed may not actually be loaded. It may only be on the todo list of textures that need loading, and if you have orthophoto scenery, that list could be very, very long. There is no guarantee that the texture is loaded before the geometry gets built. If you have a multicore machine, both might happen at the same time!
So the first reason why the road and .lin files require you to declare the size of your texture is: to allow X-Plane to build the mesh on a separate core from the one that loads the texture, for faster loading. Isn’t multi-core fun?
There’s another reason, and it’s a little bit more subtle: you can change the size of your X-Plane textures. Imagine that you get a new graphics card and you decide to “upgrade” your scenery pack. So you double the size of all of your textures, and then go in to photoshop and add more detail.
But if X-Plane were to look at the size of the texture, then all of your .lin files will be wrong! For example, let’s say you have a .lin file with a line from pixels 0 to 16 across the texture. When you double the size of your texture, you’d have to go in and change this to be 0 to 32 pixels across the texture.
But here’s the trick: since X-Plane gets the size of the texture from you and not from the actual texture, you can just leave your .lin file the way it was – with the old texture size and the old coordinates. X-Plane will do the math out and generate correct texture mappings even though the texture size is double.
In other words, the important thing about texture coordinates is that they are self-consistent, not that they reflect the actual texture size. This is because internally all texture mapping is done in ratios.
(So why not just use ratios in the file format? Try entering the size of 10 or 12 lines as ratios and tell me whether you like it. The ability to enter the line coordinates in pixels is to make it easier to set up the line files from the image file.)
Periodically I see the question asked: why does the global scenery take up so many DVDs? Or more typically the question is: if it is so big, why is X missing, where X is the detail the user expected to see but did not find.
The size of X-Plane’s global scenery is due to two factors:
- Some aspects of the scenery are very detailed, and this takes up disk space.
- Some aspects of the scenery are computed ahead of time, not by X-Plane, and this takes up disk space.
Big Data
A few data sources make the scenery pretty big:
- The scenery is generated from 90m SRTM data for the entire world.
- US scenery includes the entire US road grid.
- Coastline data is fairly detailed in the US and for most oceanic coasts.
In other words, part of the size of the scenery comes from having reasonably accurate topography and coastlines everywhere.
Precomputation
The other source of file size is that X-Plane precomputes aspects of the scenery to reduce load on X-Plane while you fly. (We do this to improve frame rate.) In particular, the scenery uses a class of algorithms that are expensive to compute, and thus we compute them ahead of time and save the result (using up file space).
- Land class changes not on a regular grid, but along iregular polygonal boundaries following the shape of the terrain. This produces much more natural terrain transitions in the mountain, but requires more storage.
- “Auto-Gen” forests and buildings don’t follow a grid – rather, they fit into the irregular spaces made by vector roads and water. This fitting process is too slow to run inside X-Plane, so every auto-gen building and forest area must be pre-computed and saved.
Room To Grow
One interesting side effect of saving the end results of building locations (“pre-placement”, what X-Plane does) rather than the formula for how to place them (“auto-gen”, what FS X does) is that we can make the placement algorithm for buildings significantly smarter without any increase in file size or decrease in frame-rate. Having already eaten the cost of putting every building into the DSFs, we can make those locations better without hurting performance.
That algorithm to pre-place buildings also can have access to source data that isn’t available in the final DSF, allowing for effects that might not be possible in an auto-gen system.
I just finished fixing some bugs with WorldEditor and DSF precision. A WED user was seeing draped polygon tiles exporting with alignment error, and the culprit turned out to be DSF precision. This begs the question, how precise are the lat/lon coordinates in a DSF?
DSFs are based on 16-bit coordinates, for a precision of one part in 65536. Now before you go screaming that 16-bit is dreadful, here’s the key: those 16-bit coordinates are interpreted within an arbitrary sub-rectangle of your tile, called a “pool”. If we only had one pool covering the DSF we’d have 1.5 meter precision. But by having many smaller pools, we can have high resolution within a pool.
All of the current DSF writing tools from Laminar Research are based on DSFLib, and DSFLib will create a number of pools along a pair of offset grids. In practice what this means is that you specify a number of pools across the DSF.
In WorldEditor 1.1dp2 the DSF exporter uses 8 pools. Since a DSF is about 100 km across, each pool is 12,500 meters in size, and the internal resolution of each pool is 19 cm. This is pretty good but there are cases where you can see the limits of precision. The new code (which will be in preview 3 or beta 1) uses 32 pools, for 5 cm precision.
If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want.
WorldEditor’s internal representation is 64-bit floating point, which gives precision better than 1 mm.
I am trying to be disciplined and put documentation on the X-Plane wiki, and limit the blog to announcements, rants, and explanations of what’s going on inside X-Plane.
You can read about custom lights here. The short of it is that a custom light is a billboard on an object where you (the author) texture the billboard (with part of the object texture), pick the texture coordinates and color, and optionally run all of these parameters through a dataref* that can modify them.
For named lights, the light texture comes from a texture atlas that Sergio made a few years ago – it’s a nice grid 8×8 pretty lights.
So…why can’t you use it with custom lights? Why do custom lights use the object texture?
The answer is: future compatibility. Sergio and I are actually already working on a new texture atlas for the sim’s built-in lights. (This has been a back-burner project for a while … I have no idea when we’ll actually productize this currently experimental work.) What happens when we create a new texture atlas with all of the lights moved around and scrambled? If your object referenced that texture, the texture coordinates would be incorrect.
Thus, for the lights where you specify texture coordinates (custom lights) you use your own texture. For named lights (where the texture coordinate is generated by X-Plane) it’s safe to use ours.
A Dangerous Bug
I found a bug in 940 that’s been in the sim for a while now: given the right strange combination of named and custom lights in a row, the sim would accidentally use Sergio’s texture atlas rather than the object’s texture for custom lights.
This is a mistake, a bug, and it will be fixed in the next 941 release candidate. I certainly hope there aren’t any objects out there relying on this erroneous behavior, which violates the OBJ spec and is pretty dangerous from a future compatibility standpoint.
* Dataresfs are normally thought of as data we read, so the idea of using them to “process” data is a bit of a bastardization of the original abstraction. You can read about the dataref scheme in detail here.
On this week’s FSBreak, Holger Sandmann described MSFS’s scenery system as a “pie” made of layers. This made me think: well, in that case X-Plane must be a cake. It turns out X-Plane has been a cake for a while.
If this discussion seems tasty but confusing, let me clarify. The issue is: which parts of a scenery system are combined when the simulator runs, and which parts must be combined (“baked”) in advance.
With MSFS, you can separately install a new mesh, new land class data, new coastlines, and new orthophotos. In X-Plane, all four of those elements must be pre-combined into a single “base mesh”. In X-Plane, you have to bake those elements.
This means that you can’t make “just” an add-on mesh for X-Plane. You have to create an add-on that addresses elevation and land class and orthophotos and water. Third party authors are quite often not very happy about this – I don’t know how many times I’ve seen “can I replace just the elevation and not the water” posted in forums.
But the Orbx team brings up the exact reason why I thought (five years ago when designing DSF) that requiring all of the elements would be okay: often if you replace one part of the scenery and not another, the results are inconsistent. If you move a coastline but don’t adjust the mesh, you may have water climbing up a mountain. If you move a mountain but don’t adjust the landuse, you may have a farm at a 45 degree angle.
Cake and Pie
Not everything in X-Plane is pre-baked. As of X-Plane 940, you must pre-bake:
- Land class
- Wide-scale orthophotos
- Terrain Mesh
- Coastlines
You can separately replace:
- All art assets (E.g. change or add types of buildings, replace the look of a given land class for an area).
- All 3-d (forests, buildings, airports, roads*).
- Small area orthophotos
Orthophotos are on the list twice. This is because X-Plane has two orthophoto technologies. Draped polygons (.pol files) can be overlaid anywhere, but they degrade performance if used over too large of an area. They are really meant for airport surface areas and other “detail” work. Orthophotos terrain (.ter files) can be used over huge areas with no performance problem, but must be baked.
* Roads are sort of a special case in X-Plane 9: you can replace them with an overlay, but the replacement roads must be “shaped” to the underlying terrain mesh, which means they won’t work well if a custom mesh is installed. This limitation in the road code dates back to X-Plane 8, when most of us had only one CPU – pre-conforming the road mesh to the terrain shape saved load time.
I had a conversation with a third party developer the other day – he offered to sign a non-disclosure-agreement (NDA) and wanted to know: would custom scenery he made for X-Plane now keep working with the next major version?
Well, no NDA is necessary for that. The simple answer is: yes! (A more complete answer is: if you use the current file formats and not legacy formats from 9 years ago, then yes.) Here’s a quick review of how long the various scenery and modeling file formats have been supported:
- DSF: 5 years
- OBJ8: 5 years
- OBJ7: 7 years
- ENV: 9 years
- OBJ2: 9 years
With X-Plane 8 we rewrote a lot of the rendering engine – since then, rendering engine enhancements have been incremental, building on what we have. The next major version will be like that too: the new version will do more than X-Plane 9, but it’s not going to drop existing capabilities.
So if you model a building with an OBJ or model a terrain area with a DSF, I expect that it will work unmodified with the next major version.
Modeling Formats in Detail
X-Plane 9 supports 3 revisions of the OBJ file format (X-Plane’s modeling format):
- Version 800, which is the current version. Introduced with version 8, the OBJ800 format has been extended heavily, but the format was never changed, so original version 800 objects are not incompatible.
- Version 700, which was used with version 7.
- Version 2, whic hwas used for most of the X-Plane 6 run.
The next major version will clearly support version 800, and will probably support version 700 as well.
Do not make new objects in the version 700 format! This format is obsolete, supported only for legacy purposes, and is an inferior format.
We may drop support for version 2 objects – I haven’t seen user content with a version 2 object in a very long time. Version 2 objects date back to a time when every polygon was expensive, so content authored in version 2 is likely to look, well, ten years old.
If you do have version 2 content, you can use ObjConverter to convert it to version 800 format. (ObjConverter will also convert version 700 OBJs to version 800.)
Scenery Formats
DSF has been our scenery format for five years now, and will continue to be so. DSF has not had any format revisions – new features are supported by allowing DSF data to be tied to new art asset formats.
I do not know if we will support ENV in the next major version. Supporting ENV is relatively trivial in the code, but whenever there is a bug, we have to fix it in the legacy ENV code as well. ENV supports a 500m terrain mesh, which is completely obsolete by today’s standard.
Do not make new content in the ENV format! Like OBJ 2 and 700, it is a legacy format for backward compatibility.
Tom had a good question that others have asked me too: why not ignore missing scenery resources? Why does X-Plane act so cranky?
Ignorance Is Bliss
We have tried the other approach: ignore missing art assets. ENV based scenery in version 7 did not require custom objects to actually be available – missing objects were ignored.
When I was working on the ENV reader for version 8 (the ENV code needed to be retrofit into the new rendering engine) I found to my surprise that virtually every ENV-based custom scenery pack I looked at was missing at least a few of the OBJs that the ENV referenced! I don’t know how this happened – it seems that in the process of working on scenery, authors started to “lose” objects and simply never noticed.
Quality Control
When we developed DSF we had a chance at a clean slate: there were no DSFs in existence so we could set the rules for art assets any way we wanted. So we picked the harshest rule possible: any missing art asset was illegal and would cause X-Plane to refuse to load the scenery package, with no way to ignore the error. Why be this rude?
- Missing artwork failures are 100% reproducible – you don’t have to try your package more than once to see the problem. If you are missing an art asset, you will have the failure every single time you run.
- The error is found on load – you don’t have to fly over the art asset to discover that it is missing.
- Therefore if an author tests a scenery package even once, even in the most trivial way, he or she will discover the missing art asset.
- Once the error is fixed, it is fixed forever, so a scenery pack that passes this quality control measure in development will be just fine “in the wild”.
- This rule has been in place since 8.0 beta 1 for DSFs, so there are no legacy DSF files that would have this problem.
Libraries
There is one special case worth mentioning: a scenery pack might reference an art asset in another scenery pack, and that other scenery pack might not be installed. This is why the library file format allows for “export_backup”. (Read more here and here.) Export_backup is your scenery pack’s way of sayingg “only use this art asset if you can’t find it somewhere else. It lets you provide emergency art-work in the event the other library is not installed.
What should you use as an emergency backup art asset? It could be anything – a big floating question mark, an empty object, a poor approximation of the desired art asset. But my main point is that responsibility for location of art assets lies with the author of a pack – so if you make a scenery pack, be sure to provide backups for any libraries you use.
(If you use OpenSceneryX, the library comes with a “developer pack” – read more here. Basically they already built a “backup” library that you can put in your scenery pack to avoid nasty messages from X-plane when OSX isn’t installed.)
There are a few changes to 940 regarding airplane lights…I will try to get some permanent documentation on the Wiki, but here’s the basic ideas:
There is a new “type” of light in the OBJ8 format, called a parameterized light. A parameterized light is somewhere between a named light (totally as-is, can’t be modified, simple to use) and a custom light (totally complex, can do anything, requires a lot of work). In a parameterized light, you control just one or two aspects of the light.
Parameterized lights are aimed at airplanes, not scenery, because typically parameterized lights are customizable and slow.* The goal is to give airplane authors some flexibility without having to invent a huge number of named lights.
Consider, for example, landing lights. A landing light could vary based on what switch controls it (we have 16 now), how big it is (many authors have pointed out that one size does not fit all) or how wide it’s view angle should be. (Lights that are inset in a structure might not be easily viewable from the side.) With a parameterized light, we can provide one light definition with 3 parameters instead of a huge matrix of lights.
Generic lights are a new collection of 64 lights that can be used for any purpose, sort of like misc wings, misc bodies, and sliders. The main difference between a landing light and a generic light is that the landing light halo won’t show up on the runway when a misc light is turned on. They are meant to be used for logo lights, inlet lights, etc. A series of new named lights will “listen” to the generic switches.
(Tip: combine ATTR_light_level with generic lights to have a light turn on and your lit texture appear at the same time.)
Finally, there is now a plugin override for the beacons and strobes (and in the systems model there can be up to 4 separate sets of beacons and strobes flashing at different times). With parameterized lights you can make two sets of strobes and use a plugin to control when they flash.
The combination of these three things let an author create an airliner that models all of the various lights and their behaviors.
* Slow needs some qualifications here. There are two code paths for lights, the fast and slow path. The slow path IS pretty fast, just not as fast as the fast path. The fast path is expected to be able to draw at least 10,000 lights in a single frame on low-end hardware, while the slow path is expected to be able to draw at least 500 lights per frame on low-end hardware.
500 lights is a lot for one airplane, especially if you have to place them by hand. And most modern computers will easily do thousands of slow lights.
Basically slow lights are not appropriate for scenery objects in the library that might be placed a huge number of times: OBJs attached to roads (e.g. street lights), OBJs used for buildings, taxiway lights. The are plenty fast for airplanes. In the X-Plane world, slow doesn’t really means slow, just slower than something else.
Two warnings about normal maps:
-
Make sure that the RGB color underneath transparent sections does not turn black or white! Some image editing programs (in particular Photoshop) will lose the color beneath a transparent area.
With a normal map, this is very bad – black and white are not legitimate normal map colors, and the result will be bogus normal vectors under the non-shiny part. Normal maps affect more than just specular shiny hilights – the normal map affects all lighting, so having black or white under your transparent (non-shiny) parts is bad news.
To check whether this has happened, I recommend Graphics Converter, which will show you your alpha and RGB channels separately, exactly as they are in the file.
-
Make sure your RGB value are normalized. The “length” of the normal (as encoded in RGB) must come out to a distance of 1. This is virtually impossible to do using PhotoShop or an image-oriented program…I suggest you use a real plugin to PhotoShop or Blender to create normal maps that are correctly “normalized”.
It is also very possible that X-Plane’s gamma correction is distorting normal maps, but that’s one for me to fix.
