I am going to take Marginal’s suggestion: in the future, ATTR_no_depth will be mapped to ATTR_poly_os 1, and ATTR_depth will be mapped to ATTR_poly_os 0. As far as I can tell, historically the ability to turn off all depth buffering was a misguided attempt to do the kind of things that ATTR_poly_os is meant for.
This implementation will hopefully help any content that is (for some reason) still using ATTR_depth and ATTR_no_depth…modern OBJ generators like the Blender plugin and the AC3D plugin never use this old attribute.
Almost two years ago, I posted this blog entry, pointing out that some legacy OBJ commands are, well, evil.
My inclusion of ATTR_poly_os in that blog post was a little strange – when used correctly, ATTR_poly_os is the right way to overlay decals on the ground, and it is fully supported. (The message is just, I suppose, that if you need a huge number for your poly_os, something is really wrong.)
Now ignoring ATTR_poly_os, you might notice that the AC3D plugin exports almost none of those dubious attributes. But what about flat shading? Flat shading isn’t necessary, but you were allowed to use it.
So I came to my senses and modified the AC3D exporter – the exporter will now never use ATTR_shade_flat – instead it uses smooth shading and per-vertex normals, avoiding an unnecessary attribute no matter what you do in AC3D. 99.99% of the time this will yield fastest framerate?
Why wasn’t the plugin always like this? I thought it was, and only discovered today that I hadn’t finished this optimization.
If you look at a lot of the text file formats we often have something like this:
TEXTURE
TEXTURE_LIT
TEXTURE_NOWRAP
TEXTURE_NOWRAP_LIT
etc. It’s a bit of a disasteer. The problem is that the command is encoding two separate ideas:
- What the texture is used for (primary texture vs. lit vs. composite vs. mask)
- How the texture is loaded (with wrapped vs clamped edges)
What got me looking at this was a test Sergio did with some ground lighting. He had texture compression on and his nice lighting and halo textures were absolutely trashed, because texture compression isn’t very nice to alpha masks.
Internally we deal with this by marking textures as “not to be compressed” — this is why the clouds don’t look ugly. I thought, “why don’t I make this available in OBJ and pol files…that’s not very hard”. But do we really want this?
TEXTURE_LIT_NOCOMP
TEXTURE_LIT_NOCOMP_NOWRAP
So I’m thinking we may need a syntax that separates the “what” (what slot in the scenery the texture is used for) from its settings. Something like this:
TEXTURE2 -nocomp -wrap my_truck.png
TEXTURE2_LIT -comp -nowrap my_truck_LIT.png
The “flags” would affect how the texture is loaded (the two obvious ones are wrapping/clamp controls, and compression-inhibition) and the command name would say what the texture is used for.
I am also looking at adding normal maps to objects – this would be a third texture attached to the object (so you could have a base texture, normal map, and lit texture). The advantage of this scheme is we’d need only three commands with a pile of flags.
Anyway, just something I’m thinking about.
(One thing I haven’t worked out is how this would interface with dataref-driven textures, which would require yet more file format gunk.)
This goes into the bucket of “weird X-Plane behavior”: X-Plane will try both PNG and BMP file extensions when opening images, no matter what is in your file. How we got to this state is, at best, confusing.
Originally, most X-Plane image files did not contain a suffix. So an ENV file contains “grass” and X-Plane would change that to grass.bmp.
Then we added PNG support. X-Plane would try grass.png and then grass.bmp. In this case, not having the extensions turned out to be handy — authors could simply bulk-convert their images and go on with life.
With most new scenery system files, the extensions are a lot more rigid:
- The extension appears in the referencing file.
- The sim only tries that extension.
- If the format doesn’t match the extension, it’s an error.
So if you want a DSF file to reference a facade, it’s buildings.fac and if that .fac file is actually a forest file, it’s an error. The sim won’t try to decide which is more correct, the header of the file or the extension, it will just go “you’re nuts” and bail out.
But (for historical reasons) images are an exception. Keeping with the “any extension goes” theme for images, X-Plane will actually try PNG and then BMP versions of your file. The extension has to match the format…that is if you call your bmp foo.png X-Plane won’t load it at all.
We have PNG as our primary image format and BMP for backward compatibility. But it’s imaginable that we could have DDS and PNG both as primary formats — PNG for images that need lossless fidelity and DDS for images where compression is acceptable. In such an event, X-Plane’s tendency to try every extension means authors can bulk-convert from PNG to DDS (making their packages load faster) and go home happy.
Austin and I have this debate over and over when we find a case where third party content doesn’t work right with X-Plane (but did in the past due to X-Plane’s permissiveness):
- If we code X-Plane to work around the problem, users get to enjoy the scenery, and authors will never fix the problem.
- If we code X-Plane to abort, users get grumpy, and authors fix the problem real fast.
What to do? A few versions ago I implemented the scenery warning system (the spanking system) to try to get the best of both worlds. I just discovered another problem that illustrates how this works well.
Before WED some authors made apt.dat 850 layouts by hand using a text editor, and sometimes they forgot to include a closing code on their rings and chains. I can hardly blame them, writing an apt.dat file by hand is very difficult, and I am impressed (and grateful) that they did this.
The apt.dat 850 spec is clear:
- Pavement chunks (line code 110) must comply with the following:
- Chunks MUST terminate in a segment point of type 113 or 114.
- Linear features (line code 120) must comply with the following:
- Linear features must terminate in a point of type 113, 114, 115 or 116.
- Linear features do not need to be “closed”.
- Airport boundaries (line code 130) must comply with the following:
- Boundaries MUST terminate in a segment point of type 113 or 114.
- Boundaries must be a closed loop and cannot contain holes. As with pavement chunks, the airport boundary cannot overlap itself.
So, you must have one of the closing codes at the end of a boundary, taxiway, or line. But X-Plane 860 doesn’t check this and simply assumes a “closed ring” if no code is provided. With X-Plane not telling authors, the authors never caught this mistake.
So now that I know about this (and now that X-Plane needs the end codes to be right, due to other bug fixes) it’s time for the spanking system. In future versions of X-Plane:
- Custom scenery packages with no ending code on a polygon block will automatically be adjusted to be “ring”-type on load-in, for compatibility with X-Plane 860.
- A detailed message will be printed per polygon in the log.txt file.
- A single alert box will be shown to the user saying “there is a problem with the custom scenery pack XXX”.
The idea is for the dialog box to walk a fine line between being unobtrusive enough for users and annoying enough for authors.
Back in X-Plane 7 the default scenery location was the “Earth nav data” folder of the Resources folder. Custom scenery went into packages in the “Custom Scenery” folder, just like now.
In X-Plane 8 we introduced the concept of a “default scenery” folder in the resources folder. The default scenery folder acts exactly like the custom scenery folder, in that it holds scenery “packages”, but it has two special properties:
- Packages in this folder have lower priority than any custom scenery package.
- It is in the resources folder and is therefore the turf of Laminar Research and our updater — the Custom Scenery folder is for third parties.
Okay that’s not really a special property, more of a convention. But the main idea here was to remove special cases from the scenery system…in X-Plane 8, all scenery comes in packages, no matter what it is and who ships it.
Scenery comes in packages, period. Repeat it like a mantra.
The original X-Plane 8.0 still installed legacy ENV files into the Resources/Earth nav data folder (but installed DSFs into a package). X-Plane 8.20 (when we shipped full DSF scenery) installs everything into packages, as will all future versions of X-Plane.
X-Plane 8 still looks in “Resources/Earth nav data” as a last resort. But this is going away! Future versions of X-Plane will only look for real scenery packages in the “Custom Scenery” and “Default Scenery” folders.
What does this mean to you? Almost certainly nothing…but…
- If you are an author and your scenery needs to be installed into the resources/Earth nav data folder, fer cryin’ out loud, please put your scenery into a custom scenery package. It takes about 5 seconds and will make life easier for you and your users.
- If you are a user and you have been moving files into the the resources/Earth nav data folder, please don’t do this! Learn how scenery packages work and put whatever you’ve been moving into a custom scenery pack. You’ll find things much easier to deal with.
In my previous post I described a bug where, when we have a split vertex in an apt.dat layout, X-Plane would draw taxiway lights all over the place. Fortunately, there is a workaround.
Remember that a split vertex is represented as two or three colocated vertices with one or two zero-length segments connecting them. The bug is that zero-length segments cause lights and lines to go haywire.
The fix is therefore simple:
- Programs that write apt.dat files can simply set the attributes on the zero-length segment to none.
- Future versions of X-Plane (with a real bug fix) can join taxiway lines even when there is a zero-length “break” in the attributes.
This picture shows the results – a split vertex with no attributes on the zero-length segment.
The next WED beta will automatically export split vertices this way – there will be no need to patch X-Plane or change your WED layouts.
In my previous blog post I defined a split vertex in an airport layout and described how they can be simulated in an apt.dat pile using multiple points.
Now that WED is in public beta and people can easily make split beziers, many have noticed the “split bezier” bug in X-Plane 860:
This is a vertex that is split…on the left you can see what it should look like – on the right you can see what it does look like. There are two problems going on:
- The taxiway lights have gone crazy at the split vertex. (This is what everyone sees.)
- The taxiway line is a bit jumbled at the split vertex too.
It should be noted that this bug will can also happen for any vertex (even unsplit) in rare occaisions due to interactions with the mesh. Same symptom, same buggy code, different cause.
The good news is:
- There is a workaround that will make the lights look correct and the lines acceptable in X-Plane 8.60.
- The workaround will be implemented inside WED – no need to change anything.
- When X-Plane is fixed, it will make the lines correct too.
In my next post I’ll describe the fix and what the results look like.
(This blog entry explains the background of split beziers – the next parts will explain the bugs that they cause and the workarounds.)
In apt.dat terminology, a split vertex is any vertex of a polygon where the control handles on either side of the vertex are not exact mirror images. (When there is only one control handle it is therefore by definition split!)
You need a split bezier any time you want to:
- Have a sharp corner between two curves and control the tangents of the curves or
- Have a sharp corner between a truly straight segment and control the curve of the next segment.
Split Beziers and apt.adt
Now here’s the rub: the apt.dat format does not allow for split beziers – each curved point has only one control handle – the other is calculated by x-plane by mirroring…thus no vertex can ever be split.
(This is due to a total lack of brains on my part when working on the apt.dat format, which is quite embarrassing considering how long I spent thinking about it.)
The Hack
There is a way to simulate a split bezier: if you use zero-length segments (that is, multiple points on top of each other), you can create a shape that works as if it is split.
In its simplest, a split bezier can be created by using 3 vertices.
- The first vertex uses the control handle of one side.
- The second vertex is not curved.
- The third vertex uses the control handle of the other side.
Why does this work? Well, a bezier curve between a curved point and a straight point has zero length if the two points are on top of each other. So what we’ve done is inserted two zero-length segments. The result of this mess is that the control handles on either “side” of this cluster of points can be different!
Is the second point really necessary! Yes! The reason is this: if we simply had the first and third point (two bezier points with different control handles), X-Plane would draw a loop from the first to the second. Remember: two colocated points with ONE control handle form a zero-length curve, but two colocated points with TWO control handles form a loop.
(To see this for yourself, just draw some examples in WED or photoshop. 🙂
Line Continuity
There is one more wrinkle we have to add to the puzzle in order to understand how this works, and what the pitfalls are: line continuity.
A bezier path (taxiway edge, linear segments, etc.) is made up of one or more bezier curves. Each curve has zero or more attributes.
When X-Plane draws the actual taxi lines and lights, it looks for continuous adjacent bezier curves with the same attribute and makes sure the linkage between those attributes is correct.
(This linkage is computed separately for each type of property. So if you have taxiway lines on two segments and lights on one, the taxiway lines will still link!)
The picture above shows a correct vs. an incorrect link. When dealing with unsplit beziers and non-curved points, linkage is pretty much automatic, it just works.
But there is a pitfall to our above hack for split vertices: we have three points on top of each other. They must all have the same attributes in order for linkage to work. A “break” in the continuity of the line for one of the zero-length segments still counts as a break in linkage. The picture on the right was produced by creating a split bezier and removing the double-yellow-line attribute from the second of three vertices.
In my next post I’ll explain the bugs that this causes in X-Plane 8.60.
Every now and then someone sends us this:
http://www.xtremesystems.org/forums/showthread.php?t=117500
The question is of course, why doesn’t X-Plane look like that yet?
Now there’s a lot of reasons why flight simulators don’t look like first person shooters…you can definitely optimize any game content for a specific viewpoint — X-Plane’s lack of constraints on the camera position (you can put the camera quite literally ANYWHERE on the Earth at any time of day, atmospheric condition, and orientation) means that there are going to be views that don’t look so hot. And the global scope of X-Plane means that we have to focus on quantity to a certain extent over quality. (If we made KSBD look totally awesome and didn’t ship any state but California in global scenery, where would we be.)
I’ve been working lately on pixel shaders and lighting…when you look at those shots, the total integration of a number of great lighting effects is responsible for a lot of the look. But…I think there’s a more fundamental issue that pixel shaders and carefully made content wouldn’t address.
Simply put, X-Plane’s LOD system isn’t scalable enough.
In order to get images that look that good you need to put a huge amount of detail up close near the camera, where the user can see it, but not put that detail in the background. (Imagine if every tree in those scenes was done in the detail of the foreground – the polygon count would be unmanageable.)
But X-Plane’s scenery SDK (the interface by which scenery content is specified to X-Plane, in other words, “the file formats”) lacks really strong LOD capabilities.
- The terrain mesh is fixed – you can use LOD to eliminate overlay details, but you can’t actually simplify terrain.
- The cost of LOD in objects is high enough to prohibit really gradual LOD. No morphing is provided.
- Textures are mipmapped but loading is not variable, so our VRAM budget doesn’t benefit from LOD and locality of textures in X-Plane space.
- Generated geometry (roads, trees, etc.) don’t have any LOD except “eliminate whole feature”.
- There is no far view of 3-d clouds.
- Airport layouts are tessolated at only one complexity.
I could go on and on…the bottom line is, X-Plane’s rendering model is very static.
Why did we do that? Well, it seemed like a good idea at the time. In particular, recalculating LOD is very expensive on the CPU and at the time we had only one core. Recalculating LOD would cost more in lost fps than it would benefit in offloading the GPU. So we went for static meshes that we could blast out to the card “real fast”.
What you might not know about those screenshots is what kind of hardware it’s running on:
http://www.gameklip.com/v/1606/
Yep…four cores, and overclocked by over a ghz. (I can only speculate that shortly after the clip ends the machine caught on fire. 🙂 In a multicore environment we can apply additional CPU resources to dynamically rebuild the environment to increase the “LOD range” (difference between the near and far view).
In fact, we already started to do that! In X-Plane 830 we modified the sim to build 3-d content (roads, forests, etc.) on a second core while flying, instantiating only the close ones. This saved RAM and improved the overall performance of the sim, and it increases our LOD range.
(Even if something isn’t drawn, it has a cost just to exist – by saying that things that are really far away don’t even exist we improve performance.)
As you can see from the above list, there’s still a lot to do on the LOD front. But the scenery system is continually growing – new features for the various primitives and improvements to the engine will let us continue to improve sim efficiency.
