I’ve blogged in the past about ATTR_poly_os…it’s a tricky topic. ATTR_poly_os is
a feature of the OBJ file format designed to let authors fix z-buffer thrash problems. Unfortunately, the cause of z-buffer thrash is pretty complex. To make things worse, it turns out I never finished my intended documentation on the subject. (I’m an idiot!)
The fundamental problem I think is that what we have now (ATTR_poly_os and ATTR_layer_group) provide a mechanism to correctly fix z-buffer thrash, but they don’t in any way enforce good behavior over bad behavior. The two attributes are very flexible, and if used together, can do all sorts of bad things. The problem is that ATTR_poly_os was thought up years before the layer-group mechanism, and thus they don’t really reinforce each other.
So…here are a few simple rules to help with z-buffer thrash in X-Plane 860:
- Never use the names of objects or their order in the DSF to accomplish anything. X-Plane ignores both names and orders when processing your scenery.
- Do not move your polygons above the terrain to fix z-thrash. This won’t work.
- When possible, divide your objects into ones that are 100% on-the-ground (and thus may z-thrash) and ones that are 100% 3-d above the ground (and will not thrash). I realize that more objects means slower fps…so this applies best when you have many objects and can pick how you divide them up.
- Always use ATTR_poly_os for any polygons that lie along the ground. Use the smallest number you can to fix the thrash.
- If you have an object with ATTR_poly_os geometry and non-poly_os geometry, make sure the ATTR_poly_os geometry is first!
Those five rules should keep you out of trouble.
What about ATTR_layer_group? Well, secretly X-Plane 860 will change the layer group of an object that is ATTR_poly_os for you. So as long as your object contains only offset geometry (this is what I recommend in rule 3) it wll always be drawn before the rest of the objects, preventing artifacts.
You’ll need ATTR_layer_group if you want to put objects underneath runways, or underneath taxiways, for example.
I am working on more comprehensive documentation on the topic, and appreciate any feedback on stuff that I’ve written that’s unclear…the rules are complicated!
I can’t say enough good things about Jonathan Harris (Marginal) — his work on X-Plane has been fantastic, he is one of the most advanced third party scenery authors I know of, and when he sends us a bug report, it is usually so perfectly patched up that I’m looking at the bad line of code in minutes! (One of these days I’ll post one of his bug reports — he always isolates the bug in a simple package that makes it very clear, with no extra “stuff”.)
He emailed me a while ago with some questions about layer groups, and I saved them to rewrite into a blog entry. A little bit of background:
X-Plane 850 and 860 introduce the concept of “layer groups”, which provide a way to control the draw order of scenery to some extent. Objects naturally fall into a layer group based on their type (e.g. objects go into the “object” layer group by default, and runways always go into the runway layer group). However, some scenery elements let you customize their layer-group placement in two ways:
- By changing which layer group the element goes to entirely, or
- By providing a “bias number”, which indicates that within the catagory of scenery elements, this one must be drawn early or late.
Layer groups let you do a number of useful things:
- Make sure that polygons and objects are drawn under runways or over taxiways when needed.
- Make sure that runway markings with polygon offset are drawn before 3-d objects.
Simply changing the order of objects in the DSF is not a reliable way to control draw order! Layer groups are. You can read about layer groups in the OBJ8 spec.
With that in mind, some Q and A. (I will elaborate on my answers from what I originally sent Jonathan.)
J: I assume that objects and polygons within a single layer can be drawn in any order – ie there’s no defined drawing order between different scenery types.
B: You assume correctly! Within a layer group, X-plane is free to reorder to improve fps. So you cannot rely on the draw order of any two scenery elements without assuring that they are in different layer groups, either by using different group names, or different relative offsets.
J: But is there are defined drawing order between objects/polygons and apt.dat-generated scenery?
B: Yes because airport scenery goes into specific layer groups! (In fact, all scenery has a “default” layer it goes into, and they usually vary by type of scenery element.)
J: eg if I have an object or polygon with ATTR_layer_group runways 0 will it be guaranteed to be drawn after the runway?
B: You’re close. The runways go into the runways group so
ATTR_layer_group runways -1
will always be before runways and
ATTR_layer_group runways 1
will always be after.
ATTR_layer_group runways 0
is the default layer group for runways, so your object or polygon would be in the layer group with all of the other runways, and X-Plane would be free to change the order amongst your object and runways in any way that would optimize framerate.
J: If not, is there any other way to insert objects and/or polygons between the taxiways and the runways?
B: The numeric offset is provided for this. The “spacing” of the layer group numbers is such that you can have up to 5 groups before and after each “named” group. So anything from runways -5 to runways +5 is fair game. (In other words, you can separately control up to 5 different “layers” of elements with a well-defined order for any layer group name.)
Here’s one of those little details that probably isn’t in the scenery docs but needs to be: if you are making base DSF meshes and they use water (the special terrain type “water”) it must be on the bottom of the stack of terrains – it can’t be used as an overlay!
I just finished an example scenery package that shows how to place orthophotos using .polygon files. Check out the scenery website (look under tutorials) to download the package and also read the illustrations. (This is an experiment: this is more of an annotated scenery example tha a true step-by-step tutorial.)
The basic anatomy of an orthophoto-in-a-polygon is:
- A PNG file contains the orthophoto. (In the case of our example, ksbd_alpha.png.)
- A .pol file references the PNG file and defines the physical properties and layering information. (In the case of our example, ksbd.pol.)
- An overlay DSF references the .pol file. (In our case this is +34-118.dsf.)
- The DSF overlay contains exactly one usage of the .pol file, with a polygon parameter of 65535 (a flag to indicate that texture coordinates are in the DSF) and each vertex of the polygon contains texture ST coordinates.
This pattern is actually the same design pattern that is used everywhere in the X-Plane scenery system:
- PNG files are never used directly by DSFs, they are always referenced by some kind of “definition” file, like an OBJ, .pol, .ter. Sometimes the definition file has a lot of info (like an OBJ, which contains a 3-d model), and sometimes they just contain some basic attributes (like a .ter or .pol file, which contain info for the physics engine).
- The artwork definitions are always separate from the DSF, so that they can be reused easily in a lot of DSFs.
- The DSF says where the polygon goes and the .pol file says what it looks like, just like the DSF says where an object goes and the OBJ says what it looks like.
One decision I made early on in my work on the scenery system was to stop using “magic file names”. Prior to X-Plane 8, in a number of cases it was enough to add _LIT to the end of a texture file and a lighting mask would appear. In all modern file scenery format, there is a texture command (usually TEXTURE_LIT) to add this information.
Now this is probably more confusing to new users, and certainly a little bit confusing to anyone who was used to X-Plane 7. To some extent my goal is to have most scenery files be generated automatically – we’re still a long way away from that, so my decisions to prefer extensibility to ease-of-use in the low level formats cause problemes for now. (But try the new AC3D X-Plane plugin version 3.0 – with UI to edit all X-Plane properties, does it matter anymore whether OBJ8 files are easy to read? We now have a real UI for editing X-Plane specific properties.)
In the case of file formats my concern was extensibility. There are only so many “tricks” that we can possibly cram into a file name, and the more we try to make the file name do, the less simple it becomes. When textures can only be “lit”, tagging _LIT on the end is simple, and haivng to add TEXTURE_LIT can be annoying. But since then we’ve also added layering information and physics information. Imagine ksbd_LIT_AIRPORTS_-1_GRASS.png. Now we’re getting into the domain of confusing file names.
Take a look here at some of the things we’d like to someday do for textures…seasonal textures would really make the texture names crazy. (MSFS9 accomplishes this using four seasons, which keeps the names simple…but what if we want to customize the time range for seasons?)
And dataref-controlled textures? Well, you can’t encode a dataref in a file name – the / is reserved!
So all of these text files (.OBJ, .pol, .for, .ter, etc.) provide extensibility – they let us add new features to the way that DSF content is viewed without requiring cahnges to the DSF file format itself.
Since this blog post has gone off into a philosophical rant and the real info is in the tutorial, let me drift totally off subject by mentioning that while you’re looking at tutorials, take a look at Kriss’s tutorial on OBJ8 animation using the new AC3D plugin. If you’ve tried using the old system of animation (by typing cryptic goo into the AC3D object naes) you’ll find the new plugin will let you work a lot faster!
In my previous post I discussed airports using the apt.dat and polygons and when to use custom artwork in a DSF.
One problem with the X-Plane scenery system (and a topic for another blog) is that, because scenery is fully pre-processed*, introducing a new airport won’t put grass underneath the airport surface area. (Only airports that existed at the time the global scenery was created have grass underneath them.)
To make an airport there are two types of processing you need to do to the underlying area:
- Flatten it, if it has bumps. Since our mesh comes from SRTM, which is “first return” (meaning whatever the radar bounces off of first, that’s the height of the terrain, even if it’s a tree top), it often is quite bumpy.
- Change the underlying texture to something appropriate for an airport.
With X-Plane 8.60 (in RC as of this writing) you can do both! I’ve blogged about airport flattening (and why it isn’t very good) before; today I’ll try to describe how you can fix the base terrain.
You can cover the base mesh with a draped polygon in a DSF overlay. All you need to do is:
- Create a .pol file that references a PNG for the texture and describes the resolution it will be applied at (i.e. how many pixels per meter) and
- Use that .pol file in a DSF with a polygon in the overlay to describe where to put that texture down.
In X-Plane 8.60 we provide .pol files for you for all of the basic airport terrain textures. If you open the file Resources/default scenery/820 world terrain/library.txt you will find the library listing for the global scenery terrain. Toward the bottom you will see a section like this:
EXPORT lib/g8/pol/apt_sdry_cold.pol pol/apt_sdry_cold.pol
What this means is: the library path lib/g8/pol/apt_sdry_cold.pol is apped to the file pol/apt_sdry_cold.pol in this scenery package. You will see a fairly large number of .pol files on the left. Basically these represent the basic “environments” that an airport can exist in.
So to make an overlay DSF that has wet cold grass underneath it, make an overlay DSF with a .pol file using the path “lib/g8/pol/apt_wet_cold.pol”.
EDIT: Important — there are actually four .pol files in this library that do not work!! They are:
apt_desert.pol
apt_dry.pol
apt_grass.pol
apt_sdesert.pol
You will know they don’t work because your scenery will not load with them. Basically, they are accidental hold-overs from a previous naming scheme for airport terrain, but were never deleted. Note how they name only an amount of rainfall (except for apt_grass, which tells us nothing). The rest of the files are named things like apt_sdry_cool, which gives us both temperature and rainfall. This 2-dimensional climate grid (with rainfall and temperature) is how most of the X-Plane terrains are organized.
Why use the library system to do this? Three reasons: copywrite and VRAM, and compatibility.
One advantage of using our library is that it gives you a way to use our artwork without copying our artwork. Remember that you cannot copy the X-Plane PNG files into a custom package and then sell it – it’s a violation of X-Plane’s EULA. But you don’t need to! You can simply use the .pol file and X-Plane will load up the appropriate PNG files.
This is also a win for VRAM. If the wet grass texture is being used in the base terrain (for an airport that has wet grass underneath it from the global scenery) and in your overlay DSF (via the library .pol file) X-Plane will only load one copy of the texture, saving VRAM, speeding up load time, etc. If you copied the PNG file, besides being a EULA violation under some circumstances, you’d also be wasting VRAM. So the library system is more efficient!
Finally, by using our polygons, you ensure that your base grass will match the rest of the scenery system. If we update the look of our grass, because the library .pol files reference our artwork, you will see the same thing in your airport that we have in the default ones.
(Of course my warning from last week applies: if you use a wet airport grass polygon for something other than wet airport grass, then in the future our improvements on the artwork may make your scenery look strange. With these airport polygons we are aiming for “as much like an airport as possible”, not “as much like 850 used to look as possible”!!)
* By this I mean, all the decisions about how scenery will look are made ahead of time, so the look of scenery does not respond to the combining of multiple packages by changing the underlying packages – only by superimposing the new ones on top. Thus a new apt.dat layout does not change the underlying terrain from urban and forest to grass. This is a design trade-off; full pre-processing has both good and bad aspects.
Sergio and I were discussing the future of facades today. Facades are DSF polygons that are extruded into buildings by pushing up the polygon into a roof and texturing the roof and walls from a single texture using some simple formulas. The rules for facades are very simple – we originally thought them up for the purpose of creating buildings that precisely fit a city grid no matter what the block spacing.
The problem with facades is that the rules make very simple buildings – especially the roofs. So we want more powerful facades and the question becomes: how best to do this? There are two possibilities and I mention them here because they underscore what I think is the largest overarching design decision when creating a flight simulator scenery system.
1. We could make the facade builder in X-Plane smarter (write new rules).
2. We could write a stand-alone tool that converts the DSF polygons (plus a rule file) into OBJs that are then placed directly into the overlay.
This second strategy would turn the facade from a type of scenery element into a tool for making OBJs. We would be “pre-compiling” our facades.
Here are some of the considerations:
SPACE VS SPEED
Precompiling primitives is a space-vs-speed trade-off. For any given algorithm, it’s usually faster to pre-build the entities and load them from disk than to build them in the sim. But pre-building means more files on disk, meaning bigger downloads, more DVDs, etc. Building up buildings from facades inside X-Plane is actually a form of compression.
EASE OF DEVELOPMENT
This is one criteria where I think pre-compiling scenery is a clear win: it’s easier to build a stand-alone facade-to-OBJ converter than to implement it inside X-Plane. X-Plane is a relatively unfriendly place to build scenery algorithms because it’s busy being a flight sim.
STABILITY VS UPGRADABILITY
If we make OBJs out of facades using a tool, the objects will look the same even if we create a new version of the tool. This is good in that it means that custom scenery won’t change how it looks, but it is also a limitation because improvements to facade technology require rebuilding a lot of objects to take effect.
(Another aspect of this: if the objects made from the tool use a texture, that texture can’t have its shape changed even if we make new rules, because some objects will be on users machines that were built with the old rules.)
X-Plane mostly errs on the side of pre-building scenery, and this is the one area where we really take it on the chin for that decision: the inability to easily create a new composite look from changes to some parts of the scenery. (Airports that don’t cause grass to appear under them is an example of this.)
Perhaps some good questions for weighing these options are:
1. Would an author care whether the facade looks exactly the same in the future?
2. Are there enhancements coming along in this technological area that we would not want to miss out on for older scenery?
OTHER CONSIDERATIONS
Information loss: when we turn a facade into an object, the original polygon information is lost. Is it important that we ship the actual base polygons that make up buildings, or are the OBJs that represent them good enough?
Contextual Information: often we know more about a given situation when making scenery than when displaying it. Could we do better at building up a facade inside the sim (where we have context from other scenery packages), or when creating the sim (where we have a much larger dataset)?
In a previous post I discussed polygons, how they can be used, and a little bit about how they relate to X-Plane 850 and the new apt.dat system. I have been working on some demo scenery that will make this all clear, but the great is the enemy of the good, so rather than wait I’ll post more on this now and get the demos done as soon as I can.
There are essentially two ways to get at the new polygon code: via the apt.dat system or via an overlay DSF. When should you use apt.dat and when should you use an overlay DSF?
- If you are trying to model something that is directly in the apt.dat spec, use an apt.dat file. For example, use apt.dat if you are making blue taxiway lights.
- Use a custom overlay DSF if you are modeling outside an airport. (Do not make “fake” airports to use apt.dat features.)
- If you need a custom look not supported by apt.dat, use an overlay DSF – it’s the only way.
- Use a custom overlay DSF if you are modeling something that isn’t found in an airport, even if it looks similar. (For example, if you want blue lights to model some unique architecture in an airport, do not use apt.dat taxiway lights – your lights may look the same, but they are not the same!)
This last point is important: do not “abuse” the definitions of apt.dat files just because they look similar.
There are essentially two kinds of features in the scenery system:
- Features that do not change how they look, ever. For example, we do not change the way a textured triangle looks in an OBJ file.
- Features that are designed to model the real world. Over time, we change them to look more like the real world. For example, approach lights have changed a lot in 850 to look more like the real world ones.
This second type of feature is the one where I issue caution: if a feature in apt.dat or the sim is meant to emulate how the real world looks, we will change how it looks to improve rendering! This is why it is important not to use apt.dat feature for purposes other than they were intended for. It might be that in X-Plane 8.50 the blue taxi lights look just like some other feature you want to code. But in a future version, we might make them more realistic for an airport and they will look worse for your other application. By using apt.dat features only for their true purpose, you help us ensure that our changes to the base artwork make scenery better, not worse.
(This division of all scenery features between ones that are “stable” and ones that are “based on the real world” can also be seen in most parts of the sim. In particular, the flight model is designed with a “based on the real world” philosophy, a very controversial decision I’ll have to blog about some other time.)
Next: fixing airport terrain with polygons.
I realized when I saw the “view stats” of 0 that I haven’t posted in maybe two weeks! Nothing’s wrong, I’ve just been busy coding scenery tools since 860 went into RC. I’ll post something useful here soon!
Just a quick random note: the maximum number of vertices in a DSF polygon is 65,535 vertices. If you have a polygon with more sides, you’ll need to simplify or subdivide it.
I would also add that the speed with which forests are built up is related ot the polygon complexity (as well as the total trees made). The algorithm is pretty fast, but you may want to consider a simplification algorithm that removes sides, making the polygon smaller, if the total error is less than a few meters (or whatever your tree spacing is), nuke the side!
