Periodically I see the question asked: why does the global scenery take up so many DVDs? Or more typically the question is: if it is so big, why is X missing, where X is the detail the user expected to see but did not find.
The size of X-Plane’s global scenery is due to two factors:
- Some aspects of the scenery are very detailed, and this takes up disk space.
- Some aspects of the scenery are computed ahead of time, not by X-Plane, and this takes up disk space.
Big Data
A few data sources make the scenery pretty big:
- The scenery is generated from 90m SRTM data for the entire world.
- US scenery includes the entire US road grid.
- Coastline data is fairly detailed in the US and for most oceanic coasts.
In other words, part of the size of the scenery comes from having reasonably accurate topography and coastlines everywhere.
Precomputation
The other source of file size is that X-Plane precomputes aspects of the scenery to reduce load on X-Plane while you fly. (We do this to improve frame rate.) In particular, the scenery uses a class of algorithms that are expensive to compute, and thus we compute them ahead of time and save the result (using up file space).
- Land class changes not on a regular grid, but along iregular polygonal boundaries following the shape of the terrain. This produces much more natural terrain transitions in the mountain, but requires more storage.
- “Auto-Gen” forests and buildings don’t follow a grid – rather, they fit into the irregular spaces made by vector roads and water. This fitting process is too slow to run inside X-Plane, so every auto-gen building and forest area must be pre-computed and saved.
Room To Grow
One interesting side effect of saving the end results of building locations (“pre-placement”, what X-Plane does) rather than the formula for how to place them (“auto-gen”, what FS X does) is that we can make the placement algorithm for buildings significantly smarter without any increase in file size or decrease in frame-rate. Having already eaten the cost of putting every building into the DSFs, we can make those locations better without hurting performance.
That algorithm to pre-place buildings also can have access to source data that isn’t available in the final DSF, allowing for effects that might not be possible in an auto-gen system.
I have blogged in the past regarding the rendering settings in X-Plane, but this seems to come up periodically, so here we go again. Invariably someone asks the question: “what computer do I have to buy to run X-Plane with all of the sliders set to maximum?”
I now have an answer, in the form of a question: “How hungry do you have to be to clean your plate at an all-you-can-eat buffet?”
There is no amount of hungry that will ever be enough to eat all of the food at an all you can eat buffet – you can always ask for more. And when it comes to rendering settings and global scenery, X-Plane is (whenever possible) the same way. You can always set more traffic, more birds, more objects, more FSAA.
Now the all-you-can-eat buffet doesn’t have infinite amounts of food in the building – just enough that they know that they won’t run out. And X-Plane is the same way. There is a maximum if you set everything all the way up, but we try to make sure that no one is going to hit a point where they want more eye candy but they’ve maxed out the settings. Eat all you want, we’ve got more.
Why on earth would we set up X-Plane like this? The answer is choice.
If you go to an all you can eat buffet, you can fill up on nothing but potatos, or you can have five pieces of chicken. It’s up to you. X-Plane is the same way – you decide if you want objects to be visible farther away or more densely. Would you rather have roads or trees? Birds or high frame-rate? You decide!
Not everyone’s appetite is the same, and not everyone’s taste is the same. This is very true when it comes to flight simulation. There are huge variations in hardware capability, target framerate (some users don’t mind 20 fps, some demand 80 fps) and in what part of the visual experience people care about most (objects vs. FSAA vs. visibility distance, etc).
Given such a heterogeneous environment, the only way to meet the needs of a wide group of users is to present choice, and make sure that we have enough of everything.
So when you go to set the rendering settings, don’t think that setting objects to anything less than maximum is like only eating half the steak you bought at a steak-house. Rather, the rendering settings are like picking which food from the buffet makes it to your plate. You choose how much you want based on what you can consume, and you pick and choose what is most desirable to you. And like an all you can eat buffet, don’t eat too much – the results won’t be pleasant!
A few days ago Austin made the recently cut X-Plane 9.45 “final” – that is, it is now the version you get when you update or grab a demo. This (hopefully) ends a sequence of 940 patches that represented a mix of fixing last-minute bugs and breaking and then fixing the throttles for a few add-on airplanes.
As always, if your add-on worked in an older version 9 but is broken in 945, please let us know. I believe the compatibility situation with 9.45 is pretty good though.
Will there be a 9.46? I don’t know, but I think the answer is: “probably”. I found a driver bug (occurs only on OS X) we can work around a few days after 9.45 was cut. We maintain a list of fixes, and when it starts to add up, we’ll cut a new patch to address them. Normally a driver bug would get a patch immediately, but from what we can tell, this one is very rare, so we’re not going to fire-drill and cut a new patch 1 day after 9.45 went final.
What kinds of bug fixes make it into these “bug fix patches”? To give an example, I received a report that the “clipping” checkbox on instruments is not preserved when you export an instrument as a text file. That’s the kind of thing we’ll fix in a patch, but we won’t cut a new patch immediately for.
I just finished fixing some bugs with WorldEditor and DSF precision. A WED user was seeing draped polygon tiles exporting with alignment error, and the culprit turned out to be DSF precision. This begs the question, how precise are the lat/lon coordinates in a DSF?
DSFs are based on 16-bit coordinates, for a precision of one part in 65536. Now before you go screaming that 16-bit is dreadful, here’s the key: those 16-bit coordinates are interpreted within an arbitrary sub-rectangle of your tile, called a “pool”. If we only had one pool covering the DSF we’d have 1.5 meter precision. But by having many smaller pools, we can have high resolution within a pool.
All of the current DSF writing tools from Laminar Research are based on DSFLib, and DSFLib will create a number of pools along a pair of offset grids. In practice what this means is that you specify a number of pools across the DSF.
In WorldEditor 1.1dp2 the DSF exporter uses 8 pools. Since a DSF is about 100 km across, each pool is 12,500 meters in size, and the internal resolution of each pool is 19 cm. This is pretty good but there are cases where you can see the limits of precision. The new code (which will be in preview 3 or beta 1) uses 32 pools, for 5 cm precision.
If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want.
WorldEditor’s internal representation is 64-bit floating point, which gives precision better than 1 mm.
I have blogged about this before, but I keep seeing this issue come up in the forums, so I want to go over it again. If you want to effectively tune X-Plane to trade off maximum visual quality with desired framerate, you must first reduce your rendering settings all the way to the bottom, then work your way up. Let me explain with an analogy.
I have fallen off my motorcycle and hit my head, skinned my knee, and broken my arm; the bone is sticking right out of my skin! Ouch! So I go to the hospital and the Doctor examines me. Here is how the conversation goes:
Dr: How does your arm feel?
Me: My arm hurts so much! OUCH!
Dr: And how does your leg feel?
Me: My arm is burning and stinging! Please make it stop!
Dr: Do you feel dizzy or light headed?
Me: Please fix my arm!!!!!!!!
Clearly with a bone sticking through my skin, there is no point in doing a physical examination. My arm hurts so much that I can’t tell the Doctor whether I have any other aches or pain. I feel one thing: the arm.
What does this have to do with framerate? Well, framerate is just like pain. The low framerate you see is caused by only the one worst problem with your setup. If your graphics card is a little bit overloaded, you are a hair short on VRAM, and your CPU is absolutely being killed, then the low framerate you see is totally because of the CPU. The weakest link decides your framerate. And like the Doctor, if we go trying to deal with the VRAM problem, we will see no change because it’s the CPU that hurts the most.
This is what I see over and over: a user is running X-Plane, his framerate is bad, and he has turned some but not all of the settings down. At this point the user is usually pretty grumpy – his visuals now look bad and his framerate is poor.
The problem is that the user hasn’t turned down the setting that really matters. This is why the first thing you need to do in order to tune framerate is to turn everything down, so that you are running with framerate at least as high as what you want for your target value. Then you can turn settings up one at a time and watch which one lowers framerate.
(Don’t worry, you’re not going to stay at the lowest settings. The key is just to always be turning settings up, not down.)
Here are some of the settings I see that need to be turned down but often are not.
- Full screen anti-aliasing. (FSAA) Always turn FSAA down to none. FSAA will kill fps on any graphics card that is fill rate limited.
- Pixel shader effects – every one of these should be turned off to start. And when you do start them, try them one at a time and have the water reflections off – work your way up in small steps. The gap from shaders without volumetric fog, shadows, reflections and per pixel lighting to shaders with all of these effects is huge!
- Turn objects all the way down to default, or even off. If your CPU is the problem, objects could be a factor.
- Leave texture compression on in your “rock bottom” settings. Texture compression improves fps and lowers visual quality, so having the check box be on is the minimal setting.
- Don’t run at a huge screen res or FOV. Run 1024×768 and 50 degrees FOV.
What happens if you turn everything down and you still see 19 fps? Now it’s time to investigate your video drivers and control panel settings. If your setup is even remotely better than minimum hardware, it should scream when all of those settings are turned down. If you still see low fps, check drivers, check control panel settings. There are a lot of control panel settings, for example, that will completely kill framerate.
In MeshTool 2.0, you can specify how wet orthophotos are handled. There are three possibilities:
- The orthophoto has no alpha-based water. The alpha channel will be ignored.
- The orthophoto has alpha-based water. Draw water under the alpha, but for physics, make the triangles act “solid”.
- The orthophoto has alpha-based water. Draw water under teh alpha, and for physics make the triangles act “wet”.
The reason for 2 and 3 is that the X-Plane physics engine doesn’t look at your alpha channel – wet/dry polygons are decided on a per triangle basis. (The typical work-around is to use the “mask” feature in MeshTool to make some parts of the orthophoto be physics-wet and some physics-solid. This is described in the MeshTool README.)
Whenever possible: don’t use alpha-based water at all. It is certainly easy to set all of your orthophotos to alpha-water + physics-solid, but there are three costs to this:
- You eat a lot of fill rate. X-Plane manages alpha=water by drawing the water underneath the entire orthophoto, then painting over it with the orthophoto. This is fill-rate expensive. If you know there is no alpha, tell MeshTool, so it can avoid creating that “under-layer” of water.
- If the terrain is very mountainous, you may get Z-buffer artifacts from the layering, particularly for thin, spikey mountains (which probably aren’t wet anyway).
- The reflection engine tries to figure out the “surface level” of the water, but it doesn’t understand the alpha channel on top of the water. So all of that water “under” your mountain or hill is going to throw the reflection engine into hysterics.
Limiting the use of water under your orthophotos fixes all three problems.
Surely I’m not the only one with that song embedded in some deep part of my psyche, right? One thing we seem to have down with 940 is counting upward…945 is in testing now.
There are two things driving the perpetual 940 bug fix releases:
- Third parties are using the sim heavily, so the bug fix releases have been vehicles for fixing a number of small bugs that are only reproducible with custom add-ons. (The fix for night lighting and orthophotos is an example of this.)
- We’ve played a real game of whac-a-mole with the throttles. (The bug is that X-Plane will lock the two throttles together. I believe that the new 945 release candidate finally fixes this.) Having done a bit of a 3 stooges routine on this bug, we’ve tightened up our internal procedures a bit.
As always, if your third party add-on broke with 940, please file a bug report! The only case of this I know of is the throttles bug – 945 should fix the BK-117, the Mu, and any other plane suffering from throttle-glue.
Will there be a 946? There very well might be, and I sincerely hope it will not be because 945 missed its target. But if, two months from now, we’ve collected another half-dozen fixes for third party authoring, I don’t see why we wouldn’t release them.
Could The Name Beta Be Any More Confusing?
The version number gets bumped when we have a release and it still needs patching. Of course, some bugs get reported after we ship, and some address issues not even under consideration during release. Had we not had any fire drills, we would probably still be on 942 or 943 right now, because we’ve had third party reports with very specialized authoring cases after we finalized the build.
Here’s what makes things even more confusing: that check box “Check for new betas as well as updates” in the installer. At any one time we may have two versions of X-Plane availalbe for download:
- The official version (currently 944) for everyone.
- Whatever the latest in-test build is (currently 945).
(You can even see which versions are current here.) What’s confusing is that while this check box says “get betas”, in truth it gets any build that is “in staging”. At this instant that I blog, X-Plane 9.45 final is in staging. That is, we believe 9.45 fixes our two known bugs, we think it’s done, it’s a release candidate, we think it is the next American Idol^H^H^H^H^H^H^H^HX-Plane. But until we certify it, you still have to check “get betas” to download it.
What we probably need to do is re-label the check-box in some way; the “staged” approach where every build is available for test first is, I think, a valuable tool to let third party authors confirm that we fixed their bugs before the entire world gets the app.
A few quick notes on the various betas floating around:
-
X-Plane: with the 9.4x series we are trying to fix small one-off bugs and improve stability. The BK-117’s throttles will be fixed in the next beta, and we fixed a crash with a third party orthophoto scenery pack.
This goes for any beta, but: if you have an add-on that used to work on an older build (especially 940) and it does not work, please report this immediately! There are no intentional add-on compatibility breaks in this patch.
-
WorldEditor: apparently developer preview 2 doesn’t contain the latest texture coordinate editor (TCE) code, so I will try to cut a preview 3 soon. The texture coordinate editor is a new small editor that lets you specify the texture placement on draped polygons. This facility lets you create custom markings. (The current TCE in preview 2 apparently doesn’t have snap to grid and a few other useful features.)
What’s holding us back from a real beta? The problem is that the DSF export of draped polygons cannot split bezier curved polygons (especially with texture coordinates) across the boundaries of DSF tiles. Since I don’t have an algorithm implementation for this yet, I’m not sure how soon I will be able to fix it. For now, don’t try to export a DSF polygon that spans DSF tiles.
-
MeshTool 2: beta 4 seems to be a keeper. I hope to cut an RC some time in the next week. (You can see the results of MeshTool 2.0 in these FranceVFR preview pics.)
This is my 500th post. I put off posting it all week because I wanted to post something lofty and clever. But in the end, the great is the enemy of the good – if I wait until I have a really good post, it could be weeks before I have time to write a 6000 word treatise on the relationship between quantum physics, shaders, and the price of crude oil.
The decision to publish less now or more later always comes up in software release planning – once the resource budget for a project is fixed* the only choice is ship sooner with less features or later with more features.
With both X-Plane and WorldEditor we often choose “ship now with less” for a simple reason: we are going to ship with more later, but if we ship now with less as well, people get some benefit in the meantime. WED is a perfect example: the first version could only edit airports, and shipped almost 18 months ago. Had we waited until we had overlay editing and airports, we would have had a more impressive release, but authors would have had no editing at all for 18 months. Why force the people who want to edit airports to wait 18 months for overlay features?
(An assumption in this is that the cost of actually doing a release is fairly low. Obviously we don’t want to do a new release every single week!)
There is a contrary force that might argue against frequent releases: once we change a feature to make it better, users are surprised if we don’t make it perfect. Users assume that if we fix some bugs in a feature but not all bugs, that it’s because we didn’t know about the bugs we didn’t fix. (The truth is usually that we had limited resources.) This produces a very strange situation where users are sometimes happier with a product that is less featureful/more deficient/more buggy because a small improvement in real functionality introduces an expectation of a large improvement.
A second behavioral phenomenon amplifies this: in my experience users consider new bugs to be significantly “buggier” (for lack of a term) than bugs that have been around for a while. This is perfectly understandable: humans are very adaptable and we get used to a bug over time to the point where we may not consider it as “bad” as when we first saw it. Trade the old bug for a new one, and we have to become re-acclimated.
These two behaviors argue (particularly when bugs and limited functionality are involved) to make a small number of large changes that move an aspect of the program from one ‘stable’ configuration to the next.
* If you think that more resources can break this trade-off between features and a quick release date, I strongly recommend “The Mythical Man Month“. The short version: 9 women can’t have a baby in 1 month – if you want a quicker release you have to do less.
I’ve been very busy with X-Plane feature development…when the blog is quiet, usually something interesting is getting coded. (Actually this weekend, I’ve just been sick in bed, so…hrm…when the blog gets quiet, perhaps I am infected.)
Just a quick note: if you see any kind of spam or vandalism on the wikis, please let me know immediately. The wiki engine we use (mediaWiki) has some fairly advanced anti-attack capabilities, but they need to be brought in when a problem occurs. Over the last two weeks I’ve cleaned out repeated attacks on the SDK site and one attack on the main site, and it appears that the software installed now is working.
But if more attacks come in, the next step is filting, e.g. programming the wiki to automatically reject posts based on typical off-topic keywords like certain ED drugs, etc. So if you see a post like this, please let me know – I’ll kill the post and set up the anti-spam filter.
