This picture shows explicit layering control for objects, a proposed extension for X-Plane 850. In this case the blue square, red square, and yellow cube are three separate objects, all sharing a texture. The blue and red objects use polygon-offset (ATTR_poly_os) to avoid flicker. The blue and red objects also contain an experimental layer controll attribute (ATTR_layer_group. The red object has a layer number lower than airports, putting it under the runways, while the blue object contains a layer number higher than airports but lower than 3-d objects, guaranteeing that the yellow cube goes over it and not vice versa. Without these the load order would be random and every object would be on top of the runways.
The performance of the new X-Plane 8.50 lights using pixel shaders is looking good. However, old scenery that uses a lot of lights won’t benefit.
The problem is that the old lights were subject to ATTR_LOD limitations. Outside the last LOD the lights disappear; authors usually do some creative things with the LOD of the object to make the lights appear at the right distances.
But the new lights are drawn independently of the object LOD. This is good – it means that a low-LOD object (good for framerate) can still produce a light visible a long way away.
The problem is – these two things are not compatible. So we don’t use the new light technology for existing OBJ lights.
If you want to use the new lights in your scenery and you have a “point lights object” all you have to do is convert the object to OBJ8 format (if it isn’t already) and then substitute the old light commands for the new ones. If you have two LODs you may want to delete one, as it’s no longer necessary. Change the OBJs and leave the ENV or DSF alone and you should see a performance increase on hardware with pixel-shaders.
(Yes, you can use OBJ8s with ENVs. I don’t recommend it, but when X-Plane finds a custom object in an ENV it will read it no matter what version – 2, 700 or 800. So if you have a custom ENV package with orthophotos and point lights you can convert them.)
I’ve been asked by a few object authors to put a show/hide command in the OBJ8 format for more advanced animation. It turns out this is trickier than you might think. For example, consider this set of OBJ8 commands (based on a hypothetical ANIM_hide command):
ATTR_flat
TRIS 0 30
ANIM_begin
ANIM_hide sim/flightmodel/whatever
ANIM_smooth
TRIS 30 30
ANIM_end
TRIS 60 30
What’s tricky here is: the use of state changes inside the “hidden” part of the object. Should the triangles after the animation be smooth, flat, or change depending on the animation?
Internally X-Plane has an object optimizer; when objects are loaded we analyze it and attempt to reduce the number of total commands, so that object drawing is as fast as possible. I strongly recommend you optimize your objects heavily – if we optimize, we might make the object faster, but if you optimize you will make it faster. Our optimizer is aimed at speeding up internal commands that are generated by X-Plane as it translates the object from OBJ7 or OBJ8 to our internal memory structure.
The problem with using show/hide to optionally skip commands is that the optimizer can no longer remove state changes. Because we don’t know whether we’re flat or smooth (after a hide command) we can’t know whether the next flat or smooth attribute is needed or unnecessary. So if show/hide is truly conditional (meaning it can skip commands) then it actually can cause things to be slower because it prevents X-Plane from removing unnecessary state change.
(The case of smooth/flat might seem trivial and contrived; it is. But there are additional “hidden” attributes that are generated by X-Plane as it translates the OBJ into our in-memory internal format. So state change can come from places you might not expect. Thus it’s important that the optimizer do its best to remove state change that can’t be addressed in the OBJ itself.)
The alternative (and one that is more likely to get implemented) is show/hide as state. Under this model, the full object is completely evaluated, but the ANIM_hide command basically causes triangles to not appear. This is similar to the workaround that people are doing now: using ANIM_translate to move the unwanted geometry a very long way away (and hoping it is out of sight).
Show/hide as state makes a trade: all commands are executed always, so we run no faster by hiding, but since the execution of the object never changes its flow, it can be more heavily optimized.
I am currently leading toward the latter approach for this reason: currently objects are slow because the CPU must do the work of state change (ATTRibutes, etc.). But graphics cards are becoming more sophisticated; in the future it may be possible to run an object with state change as a single command for the graphics card, with no CPU intervention, making things faster. But the graphics card does not yet have the ability to handle conditional logic, so show/hide as conditional logic could stop us from fully utilizing the graphics card in the future.
(Some OpenGL geeks may be thinking now “but the latest graphics cards do have conditional logic”. This is true, but currently it is only usable in shaders; the cards do not yet have the ability to skip whole commands conditionally. I believe they will be able to handle static state change before they can handle conditional state change.)
Normally I try to avoid saying anything about future releases; I find that pre-announcing things just induces Murphy’s Law…something unexpected goes wrong, and people are disappointed when what we ship turns out to not be what we thought we shipped.
So at risk of getting myself into trouble, let me say this:
It is my goal to ship the first version of the graphical DSF overlay
editor concurrently with X-Plane 8.50.
That’s an aggressive goal, because X-Plane 8.50’s going to have a lot of features in it, so Austin and I are really busy right now. But hopefully by putting it out there, I can hold my own feet to the fire and get this important tool out to users!
A quick note on the AC3D plugin based on a conversation wtih Tom….
The AC3D X-Plane OBJ export plugin is not an optimizing plugin – it is meant for “finishing”.
The idea is that the plugin can edit an OBJ and preserve almost any strange sequence of attributes you come up with. If you’ve carefully ordered your polygons and used ATTR_no_blend and a bunch of other tricks to use translucency and other tricks, the plugin will not scramble your work, but will preserve it. This lets you put the finishing touches on your OBJ, hence a “finishing” plugin.
The other side of this is that in preserving your object, the plugin does not attempt to make any optimizations that would reorder your object. So if you “ask” the plugin to make an inefficient object by using lots of state changes, it will do so and your framerate will suffer.
Basicaslly there is a direct correspondence between the model in AC3D and the exporter OBJ file. So to use AC3D efficiently you must understand both what makes an OBJ fast (not using ATTRibutes among other things) and how you do these things in AC3D (not using flat shading at all, for example).
The best of both worlds would be of course to have a menu option to optimize a model, which would let you optionally change polygon order for speed. This is something I would like to do but I’m not sure when I’ll have time to code it.
850 is turning out similar to most feature-based X-Plane releases – like a tropical storm picking up speed features keep getting sucked in, creating more chaos in its core. I thought I would be working on airports and lighting only for 850, but there’s already a number of other things floating around in the release. The end result should be better visuals.
A quick note on usage of PNG: the formats of X-Plane and PNG images are not quite the same.
- To provide a non-alpha image in X-Plane, use an RGB or grayscale PNG.
- To provide an image with alpha in X-Plane, use an RGB or grayscale PNG with an alpha channel.
- To provide an alpha mask in X-Plane, use a grayscale PNG.
This last case is the strange one: as far as I can tell PNG does not allow an alpha-only image. So when X-Plane needs an alpha mask (a 1-channel image) it uses a grayscale (1-channel image).
Also please note tht you can use a color or grayscale PNG for a color texture, but internally X-Plane always operates in RGB, so you don’t save any VRAM by reducing your PNGs to grayscale. You might save some space on disk.
First the basics: you can use the ATTR_LOD command in an object to specify what distance range an object is seen from. This has two implications:
- You can have your object look different (and usually less complex) from farther away.
- Probably more important is – your object is not drawn from arbitrarily far away. All drawing is bounded!
This second aspect is actually the more important one…because at any given time most of the objects in the sim are too far away to draw due to LOD! But this begs the question: what happens if you don’t specify ATTR_LOD at all? Is your object drawn from hundreds of miles away?
The answer is: if you don’t specify ATTR_LOD X-Plane calculates an LOD for your entire object that starts at 0 meters and goes to a distance where we think no one would care if your object disappeared. The calculation is based on the bounding sphere of your object.
Our built-in algorithm usually works pretty well, but there are some cases where it gets fooled. One important case is: when you merge two small objects into one big object, the LOD distance goes up, because the total area that your object covers goes up!
What this means is: if you have a dozen small taxiway signs from XTaxiMaker, they’ll all disappear relatively quickly as you go away from the airport, protecting framerate (by keeping the number of drawn objects down). If you merge those into one big object, the performance of the sim changes in two ways:
- Framerate when the objects are drawn goes up – we now draw one object instead of five. The number of objects drawn is very important to framerate.
- The objects will be drawn more frequently, so overall framerates maybe lower.
The moral of the story is this: if you merge objects for performance, be sure to insert an ATTR_LOD statement that is apporpriate to the actual distance at which point the objects are invisible, not the total “radius” of the object! Taxi way signs are not needed after 1500 meters, even if they cover the whole airport. Similarly some packages have taxiway markings in an OBJ. These objects usually cover big areas but the lines themselves are so thin that they aren’t needed when far away.
(Also beware: LOD is calculated frmo the middle of the object, so if the object is huge, take that into account.)
As a final note, ATTR_LOD is “free”. X-Plane will calculate and check LOD distances on every object no matter what. If you don’t use ATTR_LOD, one is generated for you. So there is no harm in adding ATTR_LOD to your object. Your framerate will only go down if you pick an LOD that is larger than the sim would have picked for you.
EDIT FROM 2/19/07: during X-Plane 850 when I was developing the new lighting system (particularly used for airports, but also for the rest of scenery) I brought up the subject of “detachment”, a internal process by which the X-Plane rendering engine processes lights.
Simply put: detachment is virtually never a factor for scenery authors. Back when I wrote the article, it was possible for an author to make a few special optimizations if he or she knew about detachment. This is no longer true! The sim will correctly optimize all lighting cases as much as possible without any special help from the author.
So….the blog article in its original form follows and may be useful as an insight into how X-Plane works, but as an author, just remember: you don’t need to do anything – just make nice OBJs.
I think I’ve confused just about everybody in the last few days – especially Sergio and Austin – with work on the new lighting system. One reason I’m so scrambled: the internal representation of lights in X-Plane is very different from the commands you can put in an OBJ8 file. So there’s a whole set of terms and ideas that are important to how X-Plane works that I shouldn’t have mentioned to anyone making OBJ8s because they won’t even be part of the file format.
Then there’s detachment. I think I can explain it simply like this: detachment is when X-Plane changes the order that lights are drawn inside an OBJ and the LOD ranges in which they are drawn to improve framerate.
X-Plane 840 does not do detachment. X-Plane 850 will probably not do detachment for objects made only with the old lighting commands. But 850 will do detachment for objects that use new (to 850) lighting commands, and this will allow those objects to render very quickly even with textured lights.
This picture is actually from X-Plane 8.03 I believe. It’s a simple trick: edit the roads file to add street light objects along all roads. Make a street light object with a light command and then turn roads on and watch teh city at night appear.
The only problem is the frame-rate – this experiment brought 8.03 to a standstill.
The goal of detachment is to make textured lights in objects fast enough that this kind of scene can be rendered at reasonable framerates. I do not know if this will be possible, but detachment should certainly improve performance in cases like this.
In considering how we design new scenery features, we have to choose between a high level or a low level approach.
- In a high level approach, we describe the data abstractly as possible. “Taxiway light” is a pretty high level description.
- In a low level approach, we describe the data as specifically as possible. “Light whose RGB value is 0.1,0.1,0.9, with a halo, 0.4 meters above the ground, with texture coordinates…” is a very low level description.
In favor of a high level approach is performance and ease-of-use. For some catagories of usage, it’s easier to work with high level primitives because the description is less tedious. It’s easier to plot taxiway lights than to describe one. High level primitives also make performance easier: it gives X-Plane the freedom to draw taxiway lights in the fastest technique possible.
Low level approaches offer flexibility. Authors can make up lights we’ve never heard of before. It also means less code; there are perhaps a few dozen distinct types of lights in the airport area, all of which must be programmed, but code to handle a generic light of a given color can be written once.
The danger of a low level approach is that we could allow a scenery construction that is difficult to support in the future, and then have to choose between sacrificing performance and breaking backward compatibility. Neither option is much fun, which gives me pause in considering such design decisions.
At this particular instant for lights I am looking at a two-tiered approach:
- Simple mode lights will provide only the most basic lighting “tricks” (flashing or turning on at a certain time of day) and will be very fast. Because these lights are not very flexible, X-Plane has a lot of room to optimize how they work.
- Complex mode lights will do almost anything but be relatively slow (meaning the same speed as textured lights now). This will let us do the really tricky lights like airport beacons, which are complex but fortunately don’t have to be fast because they are relatively rare.
So to answer the general question “why don’t you let us customize XXX”, the answer is often: if we let you do whatever you want with the scenery itself, then we can’t do whatever we want with the rendering engine, and that could mean slower performance on future hardware. The design decisions are a compromise of flexibility and performance.
The new scenery tool will work with scenery packages. Here are some details:
The new scenery tool will use a “working scenery package.” This is a normal X-Plane scenery package (preferably in a custom scenery folder in an x-system folder) with an extra folder that contains the “in progress” materials – your working master copies of scenery that will be saved later as DSF files.
This way the scenery tool can easily find all of your objects (using the library system) and display them as you edit, and X-Plane can show the objects in the package. When you finish editing a scenery file, you export and the scenrey tool saves a DSF in the Earth nav data folder.
The idea is to provide a single folder containing all of the resources (in progress and finalized) for your scenery project in one place. When we were working on global scenery last year, we had several folders that could be in different places, and the result was total configuration chaos. With the package system, all of your work is together – easily backed up, shared with other authors, and kept separate from other versions or works in progress.
This picture shows explicit layering control for objects, a proposed extension for X-Plane 850. In this case the blue square, red square, and yellow cube are three separate objects, all sharing a texture. The blue and red objects use polygon-offset (ATTR_poly_os) to avoid flicker. The blue and red objects also contain an experimental layer controll attribute (ATTR_layer_group. The red object has a layer number lower than airports, putting it under the runways, while the blue object contains a layer number higher than airports but lower than 3-d objects, guaranteeing that the yellow cube goes over it and not vice versa. Without these the load order would be random and every object would be on top of the runways.