Before I post anything to my blog saying what might happen, standard disclaimers:
- This blog represents my rambling about the directions I am considering for X-Plane’s rendering engine.
- This blog is not a promise or commitment of any kind to deliver any particular feature.
- If I say I am looking at doing feature X, and feature X does not materialize, either in the near or far future, or, like, ever, consider this to be one big fat “I told you so.”
With that in mind, I think the direction for lighting in version 9 is to introduce per-pixel lighting.
I don’t know what other set of features we’ll get with per-pixel lighting, but I am reviewing normal maps, specular maps, and the material attributes. Per pixel lighting will mean smooth, round, shiny looking surfaces without using a huge number of triangles.
Now there are two sets of hardware that will not be able to support per-pixel lighting:
- Cards without pixel shaders. (GeForce 2,3,4, Radeon 7000-9200.) You might know your card does not have pixel shaders because the pixel shader check box is not available in the rendering settings.
- Cards with first generation shaders. (This is the GeForce FX series and the Radeon 9500-9800 and X300-X600.) These cards can actually perform per-pixel lighting, but they are so slow that per-pixel lighting will bring them below minimum frame-rate.
So unfortunately, there will be an authoring decision: add more triangles so that per-vertex lighting looks good, or use fewer triangles and rely on per-pixel lighting. The decision will depend on what hardware you want to target at what performance level. (For what it’s worth, hardware that cannot support per-pixel lighting usually isn’t very powerful, so there is something to be said for not having a lot of triangles on these lower end machines.)
X-Plane 8 provides a useful baseline for rendering technology:
- It is finished and unchanging.
- Its use of shaders is very minimal, so even lower-end hardware can show the “X-plane 8 model” of lighting.
- X-Plane 8 rendering is completely supported in X-Plane 9. (That is, turn off shaders, and OBJs should look the same in X-Plane 8 and 9.)
So what do we have, and is it any good? Well, we have:
- Per-vertex lighting. Lighting is calculated per vertex, and interpolated between vertices.
- Very limited materials. Basically you can use attributes to set emissive lighting (so your day texture stays bright when back-lit, like taxiway signs) and shininess (to induce white specular hilites). The shininess ratio isn’t very flexible, but it does match what the built-in ACF shiny property does.
- Very fast vertex output within a batch.
I looked at some nice third party planes before writing this up, and one thing became clear: X-Plane can output a lot of vertices in an object if they are batched, and authors are using this aggressively. The advantage of just using a lot of vertices is: curved surfaces look round, the errors that are induced by per-vertex lighting are less ugly, and the object looks the same everywhere (because this path isn’t dependent on having pixel shaders).
The big weakness of the current situation is that you have to burn a lot of vertices to get close to per-pixel lighting, particularly for very shiny surfaces. I saw at least one plane (I do not recall who authored it) that just had more triangles in the engine nacelles than you could imagine. They look beautiful even in X-Plane 8 – great specular hilites. But that eats into your vertex budget pretty severely – it’s not a technique that you could use for every static airplane on a tarmac at LAX.
The triangle is at the heart of 3-d modeling – but before we discuss what might become of the triangle, we need terminology.
- Per-vertex lighting. This means that the brightness of the model (a function of the sun and camera position, etc.) is calculated for each vertex in the model, and then crudely interpolated between the vertices to light the pixels.
- Per-pixel lighting. This means that the brightness of the model (a function of the sun and camera position, etc.) is calculated for every pixel on the screen separately.
- Tessellation. This is the process of splitting a triangle into a number of smaller triangles, increasing the number of vertices in a model.
- Specular lighting. The specular lighting component is an extra amount of brightness that you get when the angle from the sun to the model to your eye is very small. (That is, if the model was a mirror and you could see the sun by looking at a certain location, then that location would have a bright “specular hilite”.)
- Normal map. A normal map is a texture that describes the way light bounces off a surface. This is one way to do “bump mapping”. This tutorial shows a pretty good example of how normal maps work. (The earth orbit textures in version 9 use normal maps to create “bumpy” mountains when pixel shaders are in use.)
- Specular map. A specular map is a texture that describes how strong the specular component of the lighting model appears for a given textured location. Here’s another tutorial that explains it.
- Environment Map. An environment map is a texture that represents the world around an object, used to simulate reflections. Here’s another blender tutorial that explains it better than I. (The reflective water in X-Plane 9 is effectively using a dynamic environment map created by taking a picture of part of the sim’s world every frame,)
- Material attributes. These are OBJ attributes that change the lighting model. For example, ATTR_shiny_rat changes the lighting model so that specular hilites appear.
- Batch. A batch is a single set of triangles sent to the graphics card without any change of mode. Basically every TRIS command in an OBJ becomes a batch; submitting a batch requires the CPU, but submitting a bigger batch (more triangles) does not require more CPU.
That’s enough vocabulary to describe just about everything that is happening now, will be happening in the future, as well as some pie-in-the-sky stuff. 🙂
The choice of panels (2-d panel vs. 3-d panel) for your cockpit and the choice of OBJ commands (ATTR_cockpit vs. ATTR_cockpit_region) both affect how your 3-d cockpit looks. Since these two techniques can both be varied, there are a lot of combinations, and 920RC2 does not have the right behavior. (RC3 will fix this I think.)
2-d vs. 3-d Panel
The 3-d panel is a new flat panel whose purpose is to provide the image for ATTR_cockpit or ATTR_cockpit region. Building a new panel for 3-d has a few advantages:
- The instruments can be packed together – no need for windows or other texture-wasting elements. This can help reduce panel size — panel size is expensive when using ATTR_cockpit_texture.
- The 3-d panel can be smaller than the 2-d panel; having a huge panel feed the 3-d object is slow.
- Instruments that are drawn with perspective in the 2-d panel can be redrawn orthographically, which is more useful for texturing real 3-d overhead panels.
Because the 3-d panel is meant only to be used as part of a 3-d cockpit object, spot lights and flood lights are not available, nor is a night-lit alternative. Why not?
- Such customized 2-d lighting would not match the rest of the 3-d cockpit visually.
- We will eventually have a more global lighting solution.
Basically I don’t want to provide features that will clash with the future implementation and eat framerate! The 3-d panel is aimed at next-generation content.
ATTR_cockpit vs. ATTR_cockpit_region
ATTR_cockpit_region provides a new alternate panel texturing path that gets rid of legacy behavior for improved performance and image quality.
- ATTR_cockpit_region requires the region be a power of 2, which saves VRAM. (If your panel is 1280×1024, then ATTR_cockpit rounds it to 2048×1024. Yuck!)
- ATTR_cockpit_region grabs the lit and unlit elements of the panel separately, and can thus provide lighting that is consistent with the rest of OBJ.
- ATTR_cockpit_region does not preserve transparency (which isn’t a good way to model a 3-d cockpit performance wise) – removing the alpha feature improves framerate and saves VRAM.
- ATTR_cockpit_region lets you pick out parts of a panel to texture only what you need.
This last point is less important now that we have 3-d panels (ATTR_cockpit_region came first) – it was meant to let you pick out a small subset of a large size 2-d panel, skipping windows. But if, for example, you need more than 1024×1024 pixels of panel texture, two cockpit regions are better than one 2048×1024 – some graphics cards hit a performance cliff when a cockpit or region exceeds 1024×1024.
Expected Behaviors:
(Under all situations, the instrument brightness rheostats should be preserved correctly.)
ATTR_cockpit + 2-d panel:
- The 3-d cockpit should look exactly like the 2-d cockpit.
- The 2-d panel is used as source.
- Panel transparency is preserved.
- Spot/flood lighting effects are available and work.
- Flood color is the forward flood color.
- The panel texture and object texture may not look the same under some lighting conditions.
ATTR_cockpit + 3-d panel:
- The 3-d panel is used as source.
- Transparency is preserved.
- Spot lights are not available, but flood flights work.
- Flood color is the side flood color.
- The panel texture and object texture may not look the same under some lighting conditions.
ATTR_cockpit_region + 2-d panel:
- The 2-d panel is used as source.
- Transparency is not available.
- Spot and flood lights are not available.
- Panel and object texture colors should match under all lighting conditions.
ATTR_cockpit_region + 3-d panel:
- The 3-d panel is used as source.
- Transparency is not available.
- Spot and flood lights are not available.
- Panel and object texture colors should match under all lighting conditions.
The Future
Basically both the 3-d panel and ATTR_cockpit_region are aimed at next-generation cockpits – they both strip legacy features to provide a clean platform for real 3-d cockpits. The expectation is:
- Global lighting will be applied to all 3-d geometry – panel texture and object texture. Non-emissive lighting (spot lights, flood lights) will apply to everything.
- Windows will be built using geometry, not alpha.
- The panel texture can be minimized by packing a 3-d panel and using regions. Manipulators let you provide interaction to regular object geometry.
With X-Plane 920 RC1, the user can now control how smooth taxiway curves look. More smoothing looks better, but can slow frame-rate.
Below are four pictures of KSBD (which has good, sparse vertices) at the four rendering settings.
I was reading this blog post on the Microsoft Office binary file formats…to quote:
A normal programmer would conclude that Office’s binary file formats:
- are deliberately obfuscated
- are the product of a demented Borg mind
- were created by insanely bad programmers
- and are impossible to read or create correctly.
Hrm – deliberately obfuscated, a product of a demented borg mind, created by an insanely bad programmer, and impossible to read or write correctly…he must be talking about DSF!
But then Joel discusses why the formats are the way they are…tell me some of these don’t apply to DSF:
- They were designed to be fast on very old computers. (Sort of true for DSF – load time on low-end hardware was a DSF design goal.)
- They were designed to use libraries. (True – DSF is a lot less weird when you consider the layered structure, and relatively easy to work with via DSFLib.)
- They were not designed with interoperability in mind. (I’ll get to this later.)
- They have to reflect all the complexity of the applications.
In fact, the only reason Joel sites for byzantine Office formats that doesn’t apply to DSF is:
- They have to reflect the history of the applications.
In our case DSF is a new invention of X-Plane 8, so you don’t see a lot of past history artifacts in it.
The File Format as API vs. Data Container
File formats can serve two fundamentally different purposes:
- A file format can specify a data container, used to save information for use later. DSF fits this category: the purpose of the DSF is to hold scenery data in a compact and fast-to-read format for X-Plane. (The other goal of DSF is to be structured so that we don’t have to change it all the time.)
- A file format can be an interchange standard between two programs, similar to a programming API. DSF2Text text files are like that.
You pay for every design goal with more engineering time – if DSF had to be an interchange standard and a high performance data container, it would be a lot harder to design and probably not do either job terribly well.
X-Plane Owns Its Own File Formats
One of the decisions I made early on in my work with the X-Plane scenery system was to not try to use open standards for most parts of the scenery system. (OBJ8, DSF, these are X-Plane specific inventions.) Does this make me a closed-source fascist? Perhaps, but the intent was not to lock people out of X-Plane.
Instead the goal was to separate the problems of interchange and data storage in the places where the two would come into conflict. What became immediately clear was that we couldn’t meet all of our goals for performance, compatibility, and interoperability for a scenery file format with one giant scheme.
So instead we have DSF, whose goal is to be a cryptic but fast final step in the scenery pipeline, and we have DSF2Text and MeshTool, which take simpler text file formats, meant for interchange, and create DSFs.
The intended user of a MeshTool or DSF2Text text file is another program that is outputting its data format. Thus by forming a number of small links in a chain, no one file format is meant to be the “super-format” that has to do everything for everyone.
I am in Boston visiting family for the 4th of July, so there are a few compatibility bugs that I have seen but probably won’t be able to fix for a few days. Austin is still cranking out betas. So if new betas come out and these are not fixed, don’t panic – they are definitely on my todo list.
- Sceneries converted via FS2XPlane crash on load. It looks like this is due to a bug in the new threaded texture loader – I think you can work around this by turning off “load scenery in the background” in the rendering settings (but then loads will be slow). I have found the area where the bug occurs but haven’t isolated it.
- A user submitted a plane to us that crashes Plane-Maker on open – the panel code gets confused. I haven’t isolated it yet. If you have planes you’ve made, save the pre-920 versions!
- I think Benedikt’s x737 plugin should start working again in beta 2.
I have
blogged about correct airport layouts before, but let me bring the point up again, because it is so important:
- You must create structurally correct layouts (that is, vertices connecting to vertices, not lines) in order to get good rendering in X-Plane. Just because the preview looks okay in WED doesn’t mean your layout is correctly formed!
I wrote some documentation on the scenery site that describes the problems in more detail, with pictures.
(I’ve been trying to create more permanent documentation – it is tempting to simply blog the issues because it’s so easy to throw a blog post up, but after 110 blog posts in 2007, the scenery site is still very thin on the documentation front.)
Do Not Add Vertices To Make Smoother Curves
I really can’t stress this enough: please do not go adding extra vertices in your layout to make bezier curves look smoother in X-Plane. Why is this such a bad idea? Let me count the ways!
- X-Plane will fight you all the way! X-Plane adds vertices to curves (to make them smoother) when it detects large errors. If you have a lot of small curves, the errors are inherently smaller and X-Plane will add fewer points. So the first vertices you add to your curve do almost nothing. You have to add a huge number of vertices to get a marginal improvement in your curve. In the meantime…
- X-Plane will provide variable-quality curve rendering in the future! Curve detail should be a user-controlled setting. X-Plane has to run on a wide range of hardware; any time we can let the user pick rendering quality, this is a win, because it helps bridge the gap between the user who just bought a brand new Core 2 Extreme system with GeForce 9800 and the user trying to keep X-Plane 9 running on his G4 laptop which can’t be upgraded. When you add vertices, you take the decision about rendering quality out of the hands of the user, and force high quality on a user who may not be able to handle it. Adding vertices forces a decision of lower framerate on some users.
- Adding vertices bloats the size of apt.dat. This is not a huge factor for custom scenery, but is a factor for the default apt.dat. Robin received a big pile of new airport layouts, and that’s great. But one risk is that the total size of user submitted data could get out of control. For new layouts made with WED, vertices represent a big chunk of the data. If you are increasing your vertices by a factor of 5x or 6x to improve tessolation, you are bloating the apt.dat file.
- Manually adding vertices to smooth curves lowers the level of abstraction in the apt.dat file. Any time we can have a high level abstract representation of scenery, X-Plane has the freedom to improve rendering in the future. If your layout is made up of a large number of small curves (instead of a small number of large curves), X-Plane cannot tell that those small pieces make up some larger structure; in the future it may not be able to render those layouts as nicely as ones that are made with fewer control points.
In summary, please use the smallest number of vertices to create your layouts. (But always add vertices to ensure that your T junctions are correct!)
I seem to be in a philosophical mood these days with my blog posts…thought for the day: the human mind easily goes from the specific to the general. Our brains are generalizing machines, pattern matchers finding the rule in the noise.
My preference in creating new scenery-system features is to make them very limited, and my reasoning is: our brains don’t go backward very well. We do not go from the general to the specific.
Now you might think: when making a scenery-system addition, the best thing would be to have a general feature, more useful because it can be used everywhere. But I say: the most important thing is to fully understand the feature – otherwise the feature comes out buggy.
(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)
Since the human brain doesn’t go from general to specific well, it is hard to start with a rule (“let’s allow feature X in all parts of the scenery system”) and comprehensively derive all of the implications; it is human nature to be surprised later by some unintended side-effects.
It is always easier to extend a feature later to its natural full implications than to declare certain uses illegal later, after authors of planned or started trying to use the feature in that way. If the generalization of the feature makes sense, extending it is often quite painless.
Texture Paging – Scope For Now
Texture paging is the ability for X-Plane to raise and lower the resolution of scenery textures dynamically as you fly. This means more VRAM used for nearby things and less for far away things. In practical terms, this reduces VRAM used by orthophotos by down-sampling the far-away textures, making larger orthophoto scenery packages possible. As you fly, the sim reloads some textures at higher resolutions and some at lower. The cost of the features is the load time while you fly, which burns up some extra CPU cores.
It is my hope that we will productize some very simple texture paging in the next major patch of X-Plane 9 (that would be 920, not 902). But the usage will be pretty specific:
- Texture paging will only be available for .ter and .pol textures (we can extend to other scenery types later if it makes sense).
- Texture paging will require changing the .ter and .pol files (X-Plane will not automatically analyze your scenery to see what can be paged.)
- Texture paging will not be available for ENV scenery.
- If you share textures and texture page, the results will probably be really bad and cause chaos. Be sure to use only one .ter or .pol file (and reference that text file only once in the your DSF definitions section) if you want sane paging. We can extend paging to shared textures in the future, but for now orthophotos are the intended target.
I am also deferring work on dataref-driven textures; we’ll get there eventually, and the infrastructure from the pager will make it easier. But dataref-driven textures really need to be available in a lot more places – it’s a bigger, more complex feature* and I can’t keep adding scope to 920.
Make New Meshes!
While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route. RealScenery sent me their new “State of Washington” package to use as test material; I was pleasantly surprised at the high framerate. Part of that comes from them using base meshes and not overlays.
Overlays cause the sim to draw the scenery twice (first the old scenery, then your overlay), burning a lot of pixel shader and fill power. Base meshes simply replace the old mesh which is at least twice as efficient.
(I’m just going to keep beating the dead horse of base meshes because I believe that the sooner everyone moves toward base meshes, the more bang for our hardware buck everyone gets.)
* In particular, remember that texture paging happens on threads. But datarefs can come from plugins that are not threaded! Insert anarchy here…
I don’t know how much of a problem this is yet, or how much of a mess it’s going to make of people’s scenery. Here’s the background:
- ASCII defines 128 character values, mostly letters like A-Z. With ASCII, you can write English and that’s just about it.
- The byte that ASCII is stored in on all modern computers can store 256 values.
- Clever people got the idea to put some more letters in the other 128 values to create characters like é and å.
- People defined different “codepages” that have different sets of charcters in those “upper 128” slots. So one code page might be good for French, another for Russian.
- Modern software uses unicode characters, which have a lot more than 256 values, and can thus hold all sorts of characters in one string.
Code pages were around for a while, but they’re not a good idea. The problem with code pages is that the same numeric values are used for different letters. The result is that a correctly written Russian document, when converted to a different code page, looks like gibberish. And if you want one document with both French and Russian, well, one code page doesn’t do you much good.
Now X-Plane’s handling of non-ASCII characters is pretty poor in version 9.00 (and all previous versions). It will draw ASCII and take keyboard input from ASCII but not much else. If you hit the é key on your foreign keyboard, probably nothing will work.
But it turns out there is one way to use foreign characters in X-Plane – I just discovered it tonight. If you use Windows and your system’s codepage* is set for a foreign language, you can use those foreign language characters in an OBJ file to name a file on disk with the same name. In other words, you can have textures named été.png and it will work.
Sort of. If you then change your system to work in Russian (which changes the code page) your texture will stop working. The reason things stop working is that the file system uses unicode; that is, the OS knows that été requires a Latin character set that’s French friendly, but X-Plane is using Russian since the system’s set that way. The result is that the file system has no way name the file in Russian and we fail to load the texture.
So using the “high 128” characters from your system’s code page to make non-ASCII characters is a bad idea because your scenery won’t work on other people’s computers.
But it’s going to get worse in the future. X-Plane is going to start using UTF8 in a lot of places. UTF8 encodes unicode into one byte characters by using more than one byte for non-ASCII characters, but as a result it uses the “high 128” character codes for very different things. été.png in UTF8 comes out quite different.
I’m not sure how we’ll handle this yet (use UTF8 in the scenery system or have some kind of backward compatibility). But for now I can only advise one things: use ASCII only for your file names. In fact, a good guideline for filenames for the scenery system is to use only numbers, letters, and the underscore.
