Tom had a good question that others have asked me too: why not ignore missing scenery resources? Why does X-Plane act so cranky?
Ignorance Is Bliss
We have tried the other approach: ignore missing art assets. ENV based scenery in version 7 did not require custom objects to actually be available – missing objects were ignored.
When I was working on the ENV reader for version 8 (the ENV code needed to be retrofit into the new rendering engine) I found to my surprise that virtually every ENV-based custom scenery pack I looked at was missing at least a few of the OBJs that the ENV referenced! I don’t know how this happened – it seems that in the process of working on scenery, authors started to “lose” objects and simply never noticed.
Quality Control
When we developed DSF we had a chance at a clean slate: there were no DSFs in existence so we could set the rules for art assets any way we wanted. So we picked the harshest rule possible: any missing art asset was illegal and would cause X-Plane to refuse to load the scenery package, with no way to ignore the error. Why be this rude?
- Missing artwork failures are 100% reproducible – you don’t have to try your package more than once to see the problem. If you are missing an art asset, you will have the failure every single time you run.
- The error is found on load – you don’t have to fly over the art asset to discover that it is missing.
- Therefore if an author tests a scenery package even once, even in the most trivial way, he or she will discover the missing art asset.
- Once the error is fixed, it is fixed forever, so a scenery pack that passes this quality control measure in development will be just fine “in the wild”.
- This rule has been in place since 8.0 beta 1 for DSFs, so there are no legacy DSF files that would have this problem.
Libraries
There is one special case worth mentioning: a scenery pack might reference an art asset in another scenery pack, and that other scenery pack might not be installed. This is why the library file format allows for “export_backup”. (Read more here and here.) Export_backup is your scenery pack’s way of sayingg “only use this art asset if you can’t find it somewhere else. It lets you provide emergency art-work in the event the other library is not installed.
What should you use as an emergency backup art asset? It could be anything – a big floating question mark, an empty object, a poor approximation of the desired art asset. But my main point is that responsibility for location of art assets lies with the author of a pack – so if you make a scenery pack, be sure to provide backups for any libraries you use.
(If you use OpenSceneryX, the library comes with a “developer pack” – read more here. Basically they already built a “backup” library that you can put in your scenery pack to avoid nasty messages from X-plane when OSX isn’t installed.)
No posts for two weeks…sorry, I’ve been head’s down with next-gen tech. It’s a little bit too early to blog about this stuff.
I was able to fix a few MeshTool bugs, but I have more problem reports, so I might be able to do a MeshTool beta 3 in a few days if things go smoothly.
940 is final – there might be a 941 maintenance release – we’ll know in a few weeks.
Order independent transparency (OIT): first, the shiny robot.
So first, what’s so special about this? Well, if you’ve ever worked with a lot of translucency in X-Plane, you know that it doesn’t work very well – you invariably get some surfaces that disappear at some camera angles.
The problem is that current GL rendering requires translucent surfaces to be drawn from farthest to nearest, and who is far and who is near changes as the camera changes. There are lots of tricks for trying to get the draw order mostly right, but in the end it’s somewhere between a huge pain in the ass and impossible.
What’s cool about the robot data is that the graphics card is drawing the transparency even if it is not drawn from back to front, which means the app can just shovel piles of translucent triangles into the card and let the hardware sort it out (literally).
X-Plane is currently riddled with transparency-order bugs, and the only thing we can do is burn a pile of CPU and add a ton of complexity to solve some of them partly. That proposition doesn’t make me happy.
So I am keeping an eye on hardware-accelerated OIT – it’s a case where a GPU feature would make it easier for modelers to create great looking content.
This feature is not in the WED developer preview (because I just coded it) but: WED 1.1 will feature “click-to-split” for edges. With WED 1.1 you can option-click the edge of a polygon or line feature (but not a must-be-straight entity like a runway) to insert and drag a split point.
This will hopefully be a lot easier than the current, convoluted, WED 1.0 technique of selecting the two surrounding vertices and picking “split”, then repositioning the vertex.
Hackers: this features is not yet checked into the tree, so … building from source won’t help you. It’ll be available some time in the next week.
I realize as I write this that I am going to get some comments mocking the fact that X-Plane 940 is on RC, um…13. I don’t decide the milestones for X-Plane, nor do I decide the version numbers. If you want to discuss why X-Plane is 940 (and not 930 or 950) and why it goes beta and RC when it does, email Austin. What follows is all about the scenery tools, not X-Plane.
With that out of the way: I have released a WorldEditor 1.1 developer preview. So I wanted to explain in a little bit more detail what the difference is between a developer preview and a beta. Here is an approximation of the standard definitions of “milestones” – they are what I use for WED.
- Development: not all features are coded, no guarantees about bugs.
- Alpha: all features are coded, no bug is so severe that you can’t at least try a feature. (For example, if WED crashed on startup, it would not be alpha, because you could not test saving files.)
- Beta: all requirements of alpha, also no bugs that cause program crash or data loss.
- Release: no open bugs.
This all applies to known bugs. Beta software may crash and cause data loss – it’s just that we wouldn’t have put it out as beta knowing that this happened.
WED 1.1 is still in phase (1) – development, and the build I posted is a developer preview – a cut of whatever code I had laying around. So: I can’t promise it isn’t going to trash your data or crash! Be even more cautious with the developer preview than you would with a normal beta. You don’t want to run a five hour session without saving your work, and you want to be backing up your work often – the “save” command might trash your entire project.
Why do a developer preview if it’s still so buggy? Some users who know how to compile WED from its source code are already using WED 1.1 and they seem to be enjoying it. So far it appears not to be lethally broken. Given that and the fact that most of the uncoded WED 1.1 features are usability and edge-cases, it seemed like the developer preview could be useful for getting earlier feedback.
One last note: the manual is not updated at all, nor is there any documentation on the new features. Let me be clear: no tech support or help is provided what-so-ever. Do not email me, or X-Plane tech support with “how do I use WED 1.1” questions. If you cannot figure out how to use WED 1.1 on your own, don’t use the developer preview.
Now before you come at me with pitch-forks, I don’t hate photo-realistic scenery, and X-Plane’s performance with orthophoto-based scenery is very, very good. But…the term “Photo-Realistic”…it makes me crazy. Here’s why:
First, photo-realistic scenery links the use of photographs in scenery to realism in its very name, and I don’t buy it.
Yes, some photo-based scenery packages are realistic looking, by today’s standard of flight simulation. Some are not. Just look at any old photo-realistic package to see what I mean…realistic is a relative term, defined by how much fidelity we expect, and that expectation has steadily gone up. Even with a modern package, a photo-based scenery pack might not be realistic if the photos are not used well.
(For example, is a package that uses orthophotos on the mesh but provides no 3-d in a city still considered realistic now? What kind of review would such a package get?)
Nor do photos have a monopoly on realism. They can look nice when well used, but I would put Sergio’s custom panel work up against any photo-based panel. (Sergio does not manipulate photos for his panel, he constructs them from scratching. He has thousands of photos for reference, but the pixels you see are not originally from any photo.)
Second, the term photo-realistic (in the scenery world) is most commonly applied to scenery that applies orthophotos to the terrain mesh in a non-repeating way. But orthophoto base meshes don’t have a monopoly on the use of photographs, which can be used to form land-class textures or to texture objects.
Okay, so “hate” is a strong word. But I feel some frustration whenever I see scenery discussed in terms of “photorealism”.
The rule is very simple: if you want to put a marking on the ground, your results will be much better if you use a .pol (draped polygon) or .lin (draped line) than if you use an OBJ. In fact, an OBJ is probably the worst way to put markings on the ground.
Requirements for Good Looking Markings
In order for markings to look good you need to have three things happen:
- You need to make sure your markings are at the exact same level as the ground itself.
- You need to use polygon offset to tell X-Plane to tell the video card that these are “coplanar” (at the same height) triangles that must be managed in a special way.
- You must guarantee that the draw order is: the stuff under your markings, your markings, then the 3-d buildings and other 3-d stuff.
With these requirements we can compare .pol/.lin files to OBJs:
- .pol and .lin files always “drape” to the ground, so they always meet rule 1 perfectly. With an OBJ you can set the height of your OBJ to 0 but on sloped terrain this won’t be correct.
- .lin and .pol files are always “polygon offset” automatically, so they always meet rule 2 perfectly. With an OBJ you need to use ATTR_poly_os.
- ATTR_layer_group (or LAYER_GROUP) tell the OBJ or pol/lin in what order to draw, so you can set this correctly in all cases. The default values if you don’t specify a layer group are more appropriate to the task of markings on the ground when you use a pol/lin – that is, by default they meet rule 3 perfectly. By comparison, an OBJ may not be in the right draw order.
So we can see from these 3 rules that you will always get the rules right just by using a pol or lin, but you have to be very careful and may still not get the rules right when using an OBJ.
Performance? Think .pol
When you have a large number of small markings, the performance of a .pol is going to be significantly superior to the performance of an OBJ. For example, imagine an airport where you want to draw on-pavement signs for all taxiways and runways. With an OBJ, to get the height right, you’d need to use a large number of small objects. (With one large object, it will be nearly impossible to make the OBJ markings be on the ground when far from the object center.)
When you use a large number of .pol markings that share one common texture and all have the same .pol file, X-Plane merges them behind the scene into one huge object-like pile of triangles. The cost of drawing all of those polygons will be similar to the cost of drawing just one object! That’s a huge speed win.
Floating Objects Are Wrong
What I see most commonly in scenery packs that are sent to me and have thrashing problems are OBJs that have a height other than 0. This is simply the wrong way to create overlaid geometry in X-Plane, and it will produce artifacts in a wide variety of situations. At best, the “floating” objects will cause airplanes driving over the marking to look like they “sink in”. At worst, the offset you pick will be too small for the video card’s resolution and you’ll get thrash anyway.
There is no one right vertical offset for all scenarios, and even if there was, it would still look ugly! See the above rules for what an OBJ really has to do.
How Do I Get a .pol into my scenery.
Well that is the $10,000 question. I must admit that I don’t know what Overlay Editor’s capabilities are in this regard. WED 1.1 will be able to add draped polygons, including texture coordinate editing of the polygons. I’ve been working on the texture coordinate editor in WED this weekend and am hoping to get some kind of WED 1.1 preview built this week.
With WED 1.1 the process is fairly simple:
- Create your texture.
- Create a single .pol text file that names the texture, so you can use it.
- In WED 1.1 you can select the .pol from the list of resources for your scenery pack, and then use the “polygon create tool” to simply draw the draped polygons into place.
- Once the draped polygons are created, you can select the polygon and open the “texture coordinate editor” tab to edit the way the texture is applied to the polygon.
My hope is that this process will be easier than creating markings using a 3-d editor – you can still edit the texture coordinates, but you can do so directly in WED.
I was revisiting Bézier curves today. First I should say that the current scheme for handling Bézier curves in apt.dat and DSF is horribly convoluted – I am looking to replace the syntax with something less weird for the next set of features that use them. But I was also looking at the problem of generating curves from high-density line data.
Did you know that a Bézier curve can have any number of control points? This picture illustrates:
Each curve has one more control point than the last. Zero control points is of course a line segment, and two control points is a “cubic” Bézier, which is what we have now. (Note that having one control point in WED does not give you the one-point quadric Bézier you see here. WED gives you a cubic Bézier with one control point on top of an end point.)
But it also show: having tons of control points are not particularly useful for modeling a shape in detail – a string of connected cubic Bézier is probably just as good. And the high-order Bézier doesn’t really trace out the control points, so it’s not a good way to convert line segments into a curve.
There are a few changes to 940 regarding airplane lights…I will try to get some permanent documentation on the Wiki, but here’s the basic ideas:
There is a new “type” of light in the OBJ8 format, called a parameterized light. A parameterized light is somewhere between a named light (totally as-is, can’t be modified, simple to use) and a custom light (totally complex, can do anything, requires a lot of work). In a parameterized light, you control just one or two aspects of the light.
Parameterized lights are aimed at airplanes, not scenery, because typically parameterized lights are customizable and slow.* The goal is to give airplane authors some flexibility without having to invent a huge number of named lights.
Consider, for example, landing lights. A landing light could vary based on what switch controls it (we have 16 now), how big it is (many authors have pointed out that one size does not fit all) or how wide it’s view angle should be. (Lights that are inset in a structure might not be easily viewable from the side.) With a parameterized light, we can provide one light definition with 3 parameters instead of a huge matrix of lights.
Generic lights are a new collection of 64 lights that can be used for any purpose, sort of like misc wings, misc bodies, and sliders. The main difference between a landing light and a generic light is that the landing light halo won’t show up on the runway when a misc light is turned on. They are meant to be used for logo lights, inlet lights, etc. A series of new named lights will “listen” to the generic switches.
(Tip: combine ATTR_light_level with generic lights to have a light turn on and your lit texture appear at the same time.)
Finally, there is now a plugin override for the beacons and strobes (and in the systems model there can be up to 4 separate sets of beacons and strobes flashing at different times). With parameterized lights you can make two sets of strobes and use a plugin to control when they flash.
The combination of these three things let an author create an airliner that models all of the various lights and their behaviors.
* Slow needs some qualifications here. There are two code paths for lights, the fast and slow path. The slow path IS pretty fast, just not as fast as the fast path. The fast path is expected to be able to draw at least 10,000 lights in a single frame on low-end hardware, while the slow path is expected to be able to draw at least 500 lights per frame on low-end hardware.
500 lights is a lot for one airplane, especially if you have to place them by hand. And most modern computers will easily do thousands of slow lights.
Basically slow lights are not appropriate for scenery objects in the library that might be placed a huge number of times: OBJs attached to roads (e.g. street lights), OBJs used for buildings, taxiway lights. The are plenty fast for airplanes. In the X-Plane world, slow doesn’t really means slow, just slower than something else.
Beta users, see the bottom of this post for how scalability turns into possible bugs.
In computer science, a program or architecture is scalable if it doesn’t totally vomit up a lung as its constituent parts become bigger. For a cleaner definition, see Wikipedia, source of all internet knowledge.
An ant is not scalable – if you made an ant 100 times larger in every dimension, its tiny legs would break under its new weight. (An ant is not scalable because its weight grows faster than its structural strength. Thus elephants are not built like ants.) Geeks: scalability is to computer science as marginal cost is to economics.
Before X-Plane 940, the apt.dat file was distinctly not scalable. The entire file was loaded into memory; as users created more and more taxiway lines and signs and details, we simply used more and more memory. This approach isn’t very scalable because authors have the potential to grow the apt.dat file faster than our system requirements can increase.
X-Plane 940 fixes this by not loading the entire apt.dat file into memory. Instead, only essential airport information is loaded into memory, along with a note as to where in the file the airport lives. Whenever an airport actually has to be built into a 3-d mesh while you fly, we go back to the apt. file and load the rest of the data for the one airport we are building, use it, and throw it out. Since 3-d airport meshes are built on a second core, the cost of loading one airport off disk is pretty harmless.
The problem with this fix is that it introduced a new scalability problem. Consider:
- Meshes in 940 are built on as many CPU cores as you have – some users have 8!
- Each CPU core could be working on a different airport, depending on how many are nearby.
- Each airport has to load up the apt.dat file to get the extra airport data.
This means that at times on an 8 core machine we could easily have allocated 50 * 8 = 400 MB of memory just to temporarily hold 8 copies of the apt.dat file.*
This is of course completely silly – there’s no reason to load the whole apt.dat file to get one airport, and the fix that is going into beta 8 lets the airport loader surgically grab just one airport. Thus we will be scalable again, because adding more cores won’t cause memory usage to go up.
Beta Users: Please keep an eye out for X-Plane running out of memory – if it starts to do so in the next beta it means that some part of this code change munged memory management. We’re running stress tests on the sim now, but touching the low level memory nd file handling code late in beta isn’t something I like to do.
* While loading 8 copies of the apt.dat file is wasteful of memory, it is not slow; X-Plane uses memory mapped file I/O, so reading a small part of a large file is very fast – just not very virtual-memory efficient.
