One of the comments on the suburban preview screenshots I’ve heard in a bunch of places is “I hope it’s not going to be that dark” or “could you guys add more light”?
At this point, light levels in our previews are not reflective (no pun intended) of how the sim will really look. The reason is simple: the light model is not fully debugged (who am I kidding — it’s not even remotely debugged) and it only takes one light model bug to completely throw off the light levels. So I think light levels are going to be a case where the sim’s lighting looks a bit funny right up until the last bug is swatted – it’s just the nature of those kinds of bugs.
To give you an idea of how much change there is in lighting from version 9 to 10, here’s a short laundry list of sim changes that affect lighting:
-
Dynamic Exposure. X-Plane 10 reduces the effect of emissive (_LIT) textures base on the brightness of the sun. In X-Plane 9, emissive textures have the same impact regardless of time of day; thus a lighting effect that looks good at night will look too strong during the day. (In real life, you eyes would adjust for the sun and the artificial light would seem less bright.)
-
Linear Light Mode. This gets confusing fast, but basically, our eyes perceive light in a non-linear way; we are more sensitive to low light levels than bright light levels. Computer graphics mimick this behavior; the result is that most computer lighting models are physically incorrect in some circumstances. Using the non-linear eye-based behavior has been the norm for while because it was cheaper hardware-wise, but these days it is possible to do physically correct linear lighting. We are adding this wherever we can; the correctness varies with rendering settings since physically correct lighting is more expensive GPU-wise and we don’t want to hurt fps for low-end users.
-
Deferred Rendering. X-Plane 10 has two rendering modes: a forward renderer (which is a lot like X-Plane 9) and a deferred renderer that supports global illumination and an HDR rendering space.* This creates a certain amount of chaos because the code for forward and deferred rendering are separate, and they seem to develop separate, unrelated lighting bugs.
-
Global Illumination. X-Plane 10 supports global illumination in deferred rendering mode, which means that thousands of lights can light up any part of the scenery system. Thsi means that (for the first time) an object may be lit by dozens of light sources at once. It turns out that the linear light model is a lot more important when we have more than one light source. (In fact, Alex and I realized that we needed a linear light model when looking at highways lit by streetlamps.)
-
New Light Billboards. X-Plane, like most flight simulators, uses billboards (textured squares that face the camera) to draw the light effects near a light source, like glare and bloom. The shape, textures, and equations for the light billboards are heavily revised in version 10.
-
Clouds. The weather system is being rebuilt, including new shaders for cloud puffs. Since cloud puffs aren’t like solid buildings or airplanes, they have their own shaders with their own light characteristics. We are also experimenting with increased ground visibility, which affects fog.
If there’s a take-away point, it’s this: lighting isn’t just one piece of code in X-Plane – it’s the sum and interaction of a large number of features, all of which are being heavily worked over. Only when all of the features work correctly and work together in harmony will we have what appears to be sane lighting.
* Some users may confuse HDR, which just means an image with increased dynamic range for light levels, with the more common effects that games ship once they have an HDR render: bloom and tone mapping. Bloom is when bright light sources “blow out” and splat light around nearby areas; tone mapping is a technique to visualize that high dynamic range on a normal monitor – often it is used to simulate your eyes adjusting to variable light levels.
I do not think we will ship bloom in version 10.0; I experimented with it and found it had almost no value. First, there are very few scenes in a flight simulator where bloom is that useful; it seems to be a lot more useful for interior rendering, like you’d see in a first person shooter. Second, X-Plane already has a number of bloom-like effects, including halos around lights via billboards, sun glare, etc. With most of the important cases already covered by ad-hoc effects, my early experiments with bloom weren’t very promising. We may revisit bloom later, but I don’t think it’s as important as other effects for now.
Similarly, I don’t think we will have dynamic tone mapping because we will have an overall dynamic exposure control running all of the time. Again, the value of tone mapping is more obvious with first person shooters, where you can go from interior to exterior and you want the world to be ‘overpoweringly bright’ for a while. By comparison, pilots do their best to preserve their night vision, and the interior of an airplane is designed to match that; instruments auto-calibrate their brightness to the overall light levels, making tone mapping less important.
A few days ago Tyler posted some pictures of the “auto-gen”* for X-Plane 10’s global scenery.
First, a few notes on hardware and performance: Propsman took these pictures on a new iMac, so that’s a core i5 and a Radeon HD 5000 series GPU – that is, a pretty decent system. Hardware technology continues to advance rapidly (especially on the GPU front), so there’s a big difference between a ‘decent’ system bought today and even a hard core system from two years ago.
I don’t know what your performance will be like. I do know that the system is performing well enough so far in its not-really-all-that-optimized form that we think we can ship it, and more importantly I know that we can turn the level of detail down in a number of ways to lighten the load as needed.
The most expensive feature you see here is the real-time 3-d global shadows. Heavy shadowing combined with heavy 3-d does add up and hit the system hard, but I think we’ll be able to have intermediate shadow settings that should be more affordable.
X-Plane 10 will use hardware instancing if your GPU is capable of it, and it makes a big difference in the amount of 3-d you can show.
X-Plane 10 is also quite a bit more fill-rate intensive than X-Plane 9; if your GPU is having fill-rate problems with version 9, some version 10 features will be out of reach. In the past, X-Plane has been light on fill-rate, so we’ve had users running with cut down cards (like a GeForce 8400) without realizing that their card isn’t that fast.
Some users have asked about architecture and localization. I expect we will not ship out of the box with multiple local regions; however, the library system allows us (or any third party) to provide new artwork sets for local, architecturally reasonable buildings.
Finally, it might be a bit difficult to see in these pictures (because they are focused in on the detail), but the 3-d buildings you see here work with the real-world roads. In the past, we’ve had a clash between the buildings and roads vs. the terrain texture. This is a problem we are solving for X-Plane 10.
* Auto-gen, meaning bulk buildings that populate the world in urban areas…whether it’s really auto or gen or anything like autogen in the past is a complex discussion that will have to wait.
This summer Tyler migrated the X-Plane scenery SDK documentation to X-Plane’s main wiki. I just put in a series of redirects, so that the old URLs for the top level pages will point to the various wiki sub-categories. The wiki contains the most recent information now, as well as up-to-date download links, etc.
At some point I will try to replace the individual scenery documents (part of the library.php script) with redirects to the appropriate wiki pages; until then I will leave the old site in place.
Just an FYI: when it rains it pours. Normally betas increase load on our set of update servers. To compound this, one of them is suffering a midlife crisis^H^H^H^H^H^H^H^Hhard drive failure.* We’re working on it now; hopefully it will be resolved in the next 24 hours.
EDIT: the update server is back up – our host not only swapped out the drive, but the whole box. We’ll have to take it down one more time in the future, but for the most part I think we’re out of the woods.
Another note on servers: Chris has restructured X-Plane for Android to separately download the art assets from our servers, rather than contain all art assets in the actual download. What he found after several painful weeks was that the Android store is not yet reliable for large apps. While the official app size limit is 50 MB, many phones have problems with their configuration that cause downloads to fail. When the user buys our app and the download fails, they get angry at us. (X-Plane may have been, until it was restructured, one of the largest Android game APKs. The other games with large amounts of 3-d content were already doing separate downloads.)
We originally wanted to build a monolithic app (everything in the APK) because we thought that this would provide the simplest, easiest configuration to maintain, and thus hassle-free installation for our users. You get the APK, you install it, you fly! Unfortunately, the Android Market isn’t reliable for such a large download, so we had to re-evaluate.
The new system downloads only the core app from the Android Market and then pulls the art assets from one of our servers. So far this appears to be an improvement. If/when Google provides an integrated solution, we will probably switch back to it to simplify the process again (right now we have two points of failure: the Android Market and our server farm, which, per the above notes, sometimes does fail). But for now, we’ll host the apps and try to give people the best download experience we can.
Finally, I will try to roll out at least a beta of new installers some time this week. The new installer simultaneously downloads from multiple servers, with a more efficient HTTP implementation; this should hopefully result in better download times and also lower server load per demo.
* Chris pointed out: most normal humans don’t know what this ^H^H^H^H is about…it’s nerd-speak for the delete key, e.g. to undo a text. ^H is control-H, which you may find works just like the delete key. Yes, I’m a huge nerd.
First, Happy New Year! As is typical, I’ve been quiet on the blog because things have been insanely busy here at work. Just to give you an idea of the insanity:
- There will be a 9.63 relatively soon – the bug driving this is some Linux distros not finding the DVD. But we’ll get a few new datarefs in there too.
- We have new updaters and installers to get tested, again addressing Linux DVD issues, but also with updated web download code that should give a nice speed boost.
- Chris has been working hard on Android. X-Plane for Android is pretty much the biggest APK anyone has tried to ship, and as a result we’ve hit a number of problems with the market that we are going to work around.
- All that’s just the side show; X-Plane 10 development is of course the meat and potatoes.
Now, about visibility. X-Plane 9 restricts ground visibility to 25 nm (about 46 km) in an attempt to prevent you from seeing off the edge of scenery tiles. Many users have expressed (some more persistently than I would have liked) an interest in longer range visibility. Austin recently posted a note to X-Plane.org discussing level of detail and distance management in the new weather system, and users immediately picked up on his mention of 100 nm visibility. Here’s what we’re thinking; all of this is subject to change as we keep working on the product.
First, visibility: you can come up with a formula for the distance to the horizon based on height above a sphere: d = sqrt((r+h)^2 – r^2) where r is the radius of the planet and h is the height above the planet. Since the Earth is roughly 6 million meters in radius, we get a visibility to the horizon of:
100 meters: 34.6 km
500 meters: 77.4 km
1000 meters: 109.5 km
10000 meters: 346.5 km
Clearly a little bit of altitude lets you see a long way.
But there’s more to it than that: X-Plane has always changed the visible distance with altitude. The 25 nm limit applies to surface observations (which is what you get from a METAR). As you move up into orbit, that distance is scaled out to the horizon distance, so that you can see the whole planet from orbit. That scaling can reveal the edge of DSFs, which are blended into the planet when volumetric fog is enabled.
So here is what I think we really need to do:
-
We do need a larger ‘surface level’ maximum visibility, so that distant features are visible from the ground.
-
We need a scaling from ground to upper atmospheric visibility that gives us more visibility sooner; one of the problems with version 9 is that the increase of visibility is slow, which gives mid-elevations a hazy look.
-
In the long term, we need to load more DSFs, probably twelve instead of six. X-Plane 10 already has some improvements in how scenery shift is done, but my guess is that we can’t productize this until we have a 64-bit build (since more DSFs chew more memory), so I expect this to happen in a patch.
-
We need to add elevation displacement to the whole-Earth planet render, so that the blend between DSFs and the planet don’t have huge height gaps at high-elevation locations. I am hoping we’ll have this in 10.0, but it is not coded yet. (Usually we recut the planet textures last, since they are cut off of the DSFs.)
-
We need to improve the quality of haze, fog and atmospherics. In real life, atmospheric scattering reduces the contrast of far away terrain. I believe that correct scattering could make a huge difference in the quality of the transition from DSF to planet, the required tex res (we need less if we scatter more), and generally it would be a big contribution to the realism of the image.
I’m not sure how much of this we’ll get into 10.0; I have a prototype of Sean O’Neil’s atmospheric scattering shader from GPU Gems 2 running in the sim, but I don’t think it’s shippable. I do hope we’ll get at least some scattering in place, with improvements in patches.
That’s a road map, at least. If there’s a take-away point, it’s this: increasing visibility is complex and involves a lot of parts of the sim and there are still significant parts that need work. So I really don’t know if we’ll hit some kind of hitch or problem that requires us to back off visibility.
Austin’s comments about 100 nm visibility reflect what the slider in the sim happens to be set to now. It’s also a design goal of the new weather system – that is, we want the new weather system to handle significantly larger distances (and have better scalability) than the old one did.
It shipped! X-Plane Mobile is now available for Android phones – look in the Android market under “X-Plane 9”.
Edit: Chris sent me this QR Code – scan it to go to the store listing.
Edit: if you either cannot see X-Plane in the Android market or you cannot download it, please first look here for trouble-shooting tips, then contact customer support (info at x-plane dot com). Please do not use the comments section of this blog for customer support; if you need help we will need to contact you one-to-one.
As some have noticed on the org and on FaceBook, Randy mentioned that we may be able to ship X-Plane Mobile for Android. Some users were quite befuddled to learn that we were aiming to ship X-Plane Mobile for Android so soon when X-Plane 10 is delayed. Here’s the full story.
Chris, the third and most recent addition to the X-Plane programming team, began a port of X-Plane Mobile to Android a while ago; this was the second port of X-Plane Mobile after our port to Palm WebOS. He was able to accomplish most of the port fairly quickly; hence the video floating around the web of X-Plane on a Nexus One back in May.
Unfortunately we ran into some issues that stopped ship; it looks like Google may have them fixed shortly, hence our hope of finally shipping the app. So while Chris has spent a little bit of time recently working on the last few Android issues, our hope is to release a product that we already put development time into a while ago.
If you have a DirectX 10 or 11 class video card (that is, a GeForce 8nnn or newer or a Radeon HD card) and you’re on a Mac, consider updating to OS X 10.6.x if you’re still on OS X 10.5.8.
10.6 has performance enhancements in the video drivers that I suspect will benefit X-Plane 9 users, but it will really matter for X-Plane 10. We need OS X 10.6 to expose some of the OpenGL extensions that these cards have. Thus 10.6 will get you faster frame-rate, more realistic lighting, and more efficient VRAM use.
(If you have an older card, I don’t know if you’ll get any benefit, although I doubt you’ll see a performance loss.)
X-Plane 9 allows you to categorize objects as being on the plane’s outside, inside, or glass. X-Plane depends on these flags being right for a few things:
- The draw order of the airplane is determined by the object types – glass is drawn last to avoid translucency artifacts.
- Interior light from the plane is only spilled on the “inside” objects.
- Glass objects are excluded from shadow calculations to avoid having opaque windows in the airplane shadow.
It is important that you use these flags as intended; X-Plane 10 depends on this information as well, and X-Plane 10’s global spill and global shadowing algorithms are more sensitive to incorrect categorization of objects than X-Plane 9’s forward renderer.
In particular, you should have glass for the airplane windows in an attached object tagged as type ‘glass’; do not attach your glass to the cockpit object, which cannot be categorized as glass. If you have an old plane with glass in the cockpit, consider cutting the object in half in a 3-d editor and attaching the glass separately.
(You should also use our prop disc animation, rather than use an OBJ for prop discs; the OBJ format doesn’t contain the z-buffer tricks necessary to make the prop look right.)
Sometimes these posts get off topic, sometimes in the direction of the art of computer programming, sometimes in the nature of the industry, and sometimes with pictures of the pets. This post is going to go off a bit into the subject of project management.
Randy and Tyler posted what was becoming clear (by the lack of an already existing beta): our estimated release date for X-Plane 10 was incorrect. Software project delays are pretty common, and often when a third party add-on is delayed, the community jumps to speculate about “what’s going on” inside the project and tries to infer whether the delay is an indication of serious problems.
I’d like to try to reframe the issue of delays in terms of an analogy. You ask me: how fast can you run a mile? I tell you “4 minutes and 15 seconds”. I then run a mile and you time me. My time: 6 minutes, 10 seconds.* What can we learn from this episode? I think we can learn two things:
- For a computer programmer, I am surprisingly fast – a six minute mile isn’t to be sneezed at when you spend your days sitting on your ass in front of a monitor drinking coffee.
- My ability to predict my own speed is not very good. I was pretty naive to think I could run a 4 minute mile – that’s what world class athletes run. My estimate was off by a fairly big error margin.
One thing we should not conclude is that because my mile time was 2 minutes slower than estimated, that I am a slow runner. The estimate sets up an expectation, but if the estimate is wrong, it’s not a useful metric of efficiency.
The same applies to X-Plane; we missed our original projected ship date because the estimation of when we would be done was not a very good estimate. This isn’t good for a few reasons:
- It creates uncertainty for third parties as to when a platform will change.
- It makes it difficult for marketing to properly plan a roll-out.
- It makes it difficult to balance the value of more features vs. an earlier release date (since we don’t know how much “time” we are trading for “features” if the time estimates are wrong).
But the delay is not at all a black mark for our team – on the contrary, they’re working their asses off and creating some really great work.
When looking at a project that will be delayed (because the original schedule was wrong) there’s a few things you can do:
- Add more people. This is quite often the wrong thing to do – please read the Mythical Man Month to understand why. Once your team is the right size, adding more warm bodies usually makes schedule delays worse and hurts efficiency.
- Remove features. This is the only real way to bring in a ship date.
- Move the date back.
When Austin and I were working on X-Plane 8, we hit a similar scheduling problem – what we had set out to do was going to take a lot longer than we thought. (Like X-Plane 10, we had just doubled the team size and begun a project that involved massive rewrites, which made it hard to ship until the work was fully complete. Sound familiar?) The difference? With X-Plane 8 we had contracted to ship with an external distributor for Thanksgiving, so we had to go for item 2 – we cut scope. What we cut was the world – that is, we shipped new global scenery only for the US, and the existing ENVs for the rest of the world. We also had to ship the artwork we had on hand, despite being unhappy with its quality. We didn’t finish the rest of the world and graphics we were happy with for another 11 months.
Option 2, cutting scope is painful and hard. Sometimes it is the right thing to do. In the case of X-Plane, however, we have the luxury to move the date back. With that in mind, we’re trying as hard as we can to keep feature-creep minimal and finish what we’ve already bit off, so we can get the release done and out the door.
* My mile time is not 6 minutes, 10 seconds…I would be astounded, and quite possibly in the ER if I could run that fast for any sustained amount of time.
