Traditionally, a pilot’s priorities are: aviate, navigate, communicate.
But that might not be true for X-Plane for the iPhone.
It’s real! And it pretty much is X-Plane – there really are OBJs and DSFs in there, as well as an ACF model, all tuned for the iPhone.
In the next few posts I’ll blog a little bit about the impact of doing an iPhone port on scenery development. The iPhone is an embedded device; if you go digging for system specs you’ll see that it’s a very different beast from the desktop. The porting process really helped me understand the problems of the rendering engine a lot better, and some of the techniques we developed for the iPhone are proving useful for desktop machines as well.
Yesterday I went off on a rant about how the OpenGL 3.0 spec doesn’t matter a lot because OpenGL grows via a la carte extension. And I mentioned that this creates a dilemma for anyone designing a rendering engine that has to run on a wide range of hardware, with very different underlying capabilities.
Back in the old days of X-Plane 6, there wasn’t a lot of variety in hardware. Some cards were faster than others, but the only real difference in capability was how many texture units you had. X-Plane’s job was simple…
- First we have a runway texture.
- Still got a second texure unit? Great! Skid marks!
- Still got a third texture unit? Great! Landing light!
- Got another one? Etc…
Other than texture stacking, there wasn’t much to do.
Since then the rendering engine has become a lot more complex, as have OpenGL cards. To try to keep the combinations down, I tried to use a “bucketing” strategy for X-Plane 9. The idea of bucketing is to group cards into major buckets based on whole sets of common functionality, so that we only have to test a few configurations (the “low end” bucket and “high end” bucket), rather than having to test every single card for its own unique set of features.
The obvious bucketing choice was pixel shaders – given a card with shaders and a few other features, we can render all of the new effects. A card without shaders basically gets X-Plane 8.
So what went wrong? Driver compatibility, that’s what. Ideally we don’t want to allow every single underlying rendering engine feature to be turned on or off individually because the combinations are uncontrollable.* But in practice, being able to turn features on and off is necessary to trouble-shoot drivers that don’t always do what we expect them to.
With the GeForce 8800 and Radeon HD cards, there is a potential third bucket for DirectX-10 capable cards, capable of significantly more advanced pixel shading effects. But time will tell whether we can actually make a bucket, or we have to look at each feature individually. My suspicion is that even if we organize the shipping features into buckets, we’ll have to support a lot of combinations under the hood just to trouble-shoot our own application.
*Example: cross a standard vs. oversized panel with the presence or absence of advanced framebuffer blending, crossed with whether render-to-texture works. That’s 8 possible ways just to render the 3-d cockpit panel texture. Cross that with panel regions and 3-d cockpits and the new panel spotlighting and you have 64 configurations. Ouch!
A few people have asked me about OpenGL 3.0 – and if you read some of the news coverage of the OpenGL community, you’d think the sky was falling. In particular, a bunch of OpenGL developers posted their unhappiness that the spec had prioritized compatibility over new features. Here’s my take on OpenGL 3.0:
First, major revisions to the OpenGL specification simply don’t matter that much. OpenGL grows by extensions – that is, incremental a la carte additions to what OpenGL can do. Eventually the more important ones become part of a new spec. But the extensions almost always come before the spec. So what really matters for OpenGL is: are extensions coming out quickly enough to support new hardware to its fullest capacity? Are the extensions cross-vendor so that applications don’t have to code to specific cards? Is the real implementation of high quality?
So how are we doing with extensions? My answer would be: “okay”. When the GeForce 8800 first came out, the OpenGL extensions that provide DirectX 10-like functionality were NVidia-specific. Since then, it has become clear that all of this functionality will make it into cross-platform extensions, the core spec, or some of each. But for early adopters there was a difficult point where there was no guarantee that ATI and NVidia’s DirectX 10 features would be accessible through the same extensions.
(This was not as much of an issue for DX9-like features, e.g. the first generation of truly programmable cards. NVidia had a bunch of proprietary additional extensions designed to make the GeForce FX series less slow, but the basic cross-platform shader interface was available everywhere.)
Of more concern to me is the quality of OpenGL implementations – and for what it’s worth, I have not found cases where a missing API is standing between me and the hardware. A number of developers have posted concern that OpenGL drivers are made too complex (and thus too unreliable or slow or expensive to maintain) because the OpenGL spec has too many old features. I have to leave that to the driver writers themselves to decide! But when we profile X-Plane, we either see a driver that’s very fast, or a driver that’s being slow on a modern code path, in a way that is simply buggy.
Finally, I may be biased by the particular application I work on, but new APIs that replace the old ones don’t do me a lot of good unless they get me better performance. X-Plane runs on a wide range of hardware; we can’t drop cards that don’t support the latest and greatest APIs. So let’s imagine that OpenGL 3.0 contained some of the features whose absence generated such fury. Now if I want to take advantage of these features, I need to code that part of the rendering engine twice: once with the new implementation and once with the old implementation. If that doesn’t get me better speed, I don’t want the extra code and complexity and wider matrix of cases to debug and test.
In particular, the dilemma for anyone designing a renderer on top of modern OpenGL cards is: how to create an implementation that is efficient on hardware whose capabilities is so different. I’ll comment on that more in my next post. But for the purposes of OpenGL 3.0: I’m not in a position to drop support for old implementations of the GL, so it doesn’t bother me at all that the spec doesn’t drop support either.
The real test for OpenGL is not when a major revision is published; it is when the next generation of hardware comes out.
An author asked me some questions that I think are so important that I’ll blog the answers:
- The new texture paging system (LOAD_CENTER) works for both terrain textures (.ter files) and draped polygons (.pol files). You do not have to use draped polygons to get texture paging – you can use paging in a base mesh!
- Orthophoto terrain via a (.ter) file is by far the preferred method for orthophoto sceneries – it is a vastly better option than draped polygons. Draped polygons are horribly wasteful of hardware resources, and should really only be used for tiny areas, e.g. airport surface areas. If you are using even a moderate amount of orthophotos, make a base mesh!
- MeshTool is the future of photo scenery, and will continue to be the way to make high performance orthophoto meshes for X-Plane.
The future of MeshTool is bug fixes, a richer syntax, and some day maybe a UI front-end.
I just received a series of reports today that certain converted scenery will cause X-Plane to crash with a “bad alloc” error. Basically, this couldn’t have hit us at a worse time. The final 920 was cut a week ago. We physically can’t recut; Austin is on the road, and I am knee deep in it. But there is a possible work-around, and there will be a patch. Here’s the whole situation.
What is a Bad Alloc?
A bad alloc error is an error that comes up when X-Plane runs out of memory. This can happen for two reasons:
- We have run out of address space – that is, there is no more virtual memory left, or
- We have run out of page file/physical memory – that is, we can’t back that virtual memory.
The first case is by far the most common – you’d only hit the second if you are on Windows with a fixed-size (but small) page file. (Hint: if you have a fixed size page file, make it big!)
X-Plane can run out of memory for many reasons – everything that runs in the sim uses memory, and the amount used depends on what area you are in, what rendering settings you pick, and what third party add-ons you use. While I’d like to someday reach a point when the sim tells you gracefully that it’s out of memory, it will always be a fact of life that at some point (hopefully an absurdly high one) the amount of stuff you’ve asked X-Plane to do will exceed how much memory you have.
(If you are thinking 64 bits, well, that will just change the problem from a crash to a grinding halt when we run out of physical memory.)
We see bad allocs when there are too many third party add-ons installed (XSquawkBox is a particular pig because it loads every CSL on startup), too complex scenery, and it can also be caused by drivers not efficiently using memory. (This is particularly a problem on Vista RTM.)
The Bug
When X-Plane creates a curved airport taxiway, it allocates a temporary memory buffer to hold the intermediate product of the pavement. The size of that buffer depends on the complexity of the curve it is processing and a constant, based on the maximum curve smoothness.
In 920 I provided an option to crank up the curve smoothness in X-Plane. In the process, I increased that constant factor by 4x, which causes X-Plane to hit its memory ceiling on layouts that used to be acceptable. You’ll see this problem more often on:
- Bigger, more complex layouts.
- Configurations that were already chewing up a lot of memory.
- Machines with less address space (Windows without /3GB, older Mac OS X operating systems.)
What really suckered us about this bug was that it comes in a form that looks almost the same as a driver issue we’ve seen with ATI drivers on Windows — we’ve seen strange forms of memory exhaustion on ATI when shifting scenery with high rendering settings. So we didn’t realize that this was something new until G5 users reported the bug (making us realize it wasn’t a driver thing).
What To Do
The bad news is that we can’t do an RC5 – we’re out of time. But – there will be a patch – relatively soon. This bug is on the short list for a patch to fix 920.
In the meantime, there is actually a work-around. By coincidence, some of the internal rendering engine constants are viewable via the “private dataref” system — basically a series of datarefs in the sim/private/… domain that I use for on-the-fly debugging. The dataref that matters here is:
sim/private/airport/recurse_depth
If you load up DataRef Editor you’ll see it has a value of 12 . That’s too high. Changing it to 10 will allow otherwise problematic airports to load.
I will try to post a plugin in the next 10 days that sets this dataref to 10 on startup, effectively patching the problem. This will also limit the maximum smoothness of curves – but my guess is that if you see the crash (not all users do) then you can’t run on the max airport curve setting anyway.
Of course the next patch will contain a real solution: a more efficient memory allocation scheme!
Some users reported during 920 beta that X-Plane would sometimes not detect its DVD – a condition that would come and go. Tonight I figured out what is happening.
- In order to validate the DVD, X-Plane decompresses part of its contents into the preferences folder. Why preferences? There is no good reason – it’s historic.
- X-Plane will create a preferences folder if there is not one. But it does not do that until you quit.
- The X-Plane installer will not make directories unless they contain files.
So put these three things together: on the first run of a new install, there is no preferences file, so the DVD check fails since the directory that will contain some temporary files is missing. Run a second time, and the directory is there and the DVD check succeeds.
The next patch of the sim will fix this, but in the meantime, if you delete your preferences, leave the empty directory in place!
Two random and unrelated notes:
First, RC4 is going out as is, despite the engine modeling changes being incomplete. Basically we now have a more sane approach to the engines themselves, but no FADEC control. FADECs are on the short list for the next update. Sometimes we just run out of time – not every release can have everything.
Second, a note on autopilot customization – I am party to a fair number of questions about whether the plugin system can be used to make subtle changes to the autopilot logic. The answer is of course: no. If you really want something different for an autopilot, you’d have to replace the entire “top-half” set of logic and drive the flight directors yourself – in this situation you are responsible for:
- All modes and mode changes based on conditions.
- The actual selected flight envelope to achieve the desired AP setting.
But you are not responsible for driving the trim and yoke, which are done by you setting the flight director.
Why can’t you just override one specific behavior? It’s an issue of infrastructure.
Fundamentally, the autopilot only does a few certain tricks. If it were capable of doing customized behaviors, you’d already see it, in the form of a dataref or (more likely) a Plane-Maker setting. Basically there is no generality to the autopilot that we secretly have inside the code but don’t expose.
Will there be a more general autopilot someday? Maybe – I don’t know, I don’t work on that code. But the plugin system has always aimed to make it possible to do anything, but not necessarily easy. In particular, the plugin system doesn’t aim to make your development easier by recycling the simulator itself as a convenient library of lego bricks. In the end of the day, X-Plane is an application, not a library. If it were a library, that would be lots of fun for third parties, but it is not.
The choice of panels (2-d panel vs. 3-d panel) for your cockpit and the choice of OBJ commands (ATTR_cockpit vs. ATTR_cockpit_region) both affect how your 3-d cockpit looks. Since these two techniques can both be varied, there are a lot of combinations, and 920RC2 does not have the right behavior. (RC3 will fix this I think.)
2-d vs. 3-d Panel
The 3-d panel is a new flat panel whose purpose is to provide the image for ATTR_cockpit or ATTR_cockpit region. Building a new panel for 3-d has a few advantages:
- The instruments can be packed together – no need for windows or other texture-wasting elements. This can help reduce panel size — panel size is expensive when using ATTR_cockpit_texture.
- The 3-d panel can be smaller than the 2-d panel; having a huge panel feed the 3-d object is slow.
- Instruments that are drawn with perspective in the 2-d panel can be redrawn orthographically, which is more useful for texturing real 3-d overhead panels.
Because the 3-d panel is meant only to be used as part of a 3-d cockpit object, spot lights and flood lights are not available, nor is a night-lit alternative. Why not?
- Such customized 2-d lighting would not match the rest of the 3-d cockpit visually.
- We will eventually have a more global lighting solution.
Basically I don’t want to provide features that will clash with the future implementation and eat framerate! The 3-d panel is aimed at next-generation content.
ATTR_cockpit vs. ATTR_cockpit_region
ATTR_cockpit_region provides a new alternate panel texturing path that gets rid of legacy behavior for improved performance and image quality.
- ATTR_cockpit_region requires the region be a power of 2, which saves VRAM. (If your panel is 1280×1024, then ATTR_cockpit rounds it to 2048×1024. Yuck!)
- ATTR_cockpit_region grabs the lit and unlit elements of the panel separately, and can thus provide lighting that is consistent with the rest of OBJ.
- ATTR_cockpit_region does not preserve transparency (which isn’t a good way to model a 3-d cockpit performance wise) – removing the alpha feature improves framerate and saves VRAM.
- ATTR_cockpit_region lets you pick out parts of a panel to texture only what you need.
This last point is less important now that we have 3-d panels (ATTR_cockpit_region came first) – it was meant to let you pick out a small subset of a large size 2-d panel, skipping windows. But if, for example, you need more than 1024×1024 pixels of panel texture, two cockpit regions are better than one 2048×1024 – some graphics cards hit a performance cliff when a cockpit or region exceeds 1024×1024.
Expected Behaviors:
(Under all situations, the instrument brightness rheostats should be preserved correctly.)
ATTR_cockpit + 2-d panel:
- The 3-d cockpit should look exactly like the 2-d cockpit.
- The 2-d panel is used as source.
- Panel transparency is preserved.
- Spot/flood lighting effects are available and work.
- Flood color is the forward flood color.
- The panel texture and object texture may not look the same under some lighting conditions.
ATTR_cockpit + 3-d panel:
- The 3-d panel is used as source.
- Transparency is preserved.
- Spot lights are not available, but flood flights work.
- Flood color is the side flood color.
- The panel texture and object texture may not look the same under some lighting conditions.
ATTR_cockpit_region + 2-d panel:
- The 2-d panel is used as source.
- Transparency is not available.
- Spot and flood lights are not available.
- Panel and object texture colors should match under all lighting conditions.
ATTR_cockpit_region + 3-d panel:
- The 3-d panel is used as source.
- Transparency is not available.
- Spot and flood lights are not available.
- Panel and object texture colors should match under all lighting conditions.
The Future
Basically both the 3-d panel and ATTR_cockpit_region are aimed at next-generation cockpits – they both strip legacy features to provide a clean platform for real 3-d cockpits. The expectation is:
- Global lighting will be applied to all 3-d geometry – panel texture and object texture. Non-emissive lighting (spot lights, flood lights) will apply to everything.
- Windows will be built using geometry, not alpha.
- The panel texture can be minimized by packing a 3-d panel and using regions. Manipulators let you provide interaction to regular object geometry.
I don’t like to delete/reject people’s comments, but I do not want new users to find this blog, see tech support requests, and add their own, only to have them sit unanswered for, well, ever.
If you bought X-Plane from Laminar Research, the tech support contact info can be found here:
http://www.x-plane.com/contact.html
If you bought X-Plane in a store, the distributor will have their own tech support contact info on the box.
From this point on, I am going to reject requests for tech support that come in the comment box. If you need help with X-Plane, use the email or phone number found on the contact page above!
Posted in News
by
Ben Supnik |
I believe I am getting close to a possible solution for the dreaded “Framebuffer Incomplete” errors – these error messages pop up when X-Plane starts, and you end up quitting.
If you meet these criteria, please contact me:
- You have an ATI card that has shown this error in the past.
- You can put on the latest Catalyst drivers. (I know a lot of you have put on older drivers to work around this.)
- You can run X-Plane 920 RC2.
If you’re in this crew, please email me at my XSquawkBox email address!
The rub is: despite having four machines with ATI cards, I never see this error. So I need to send you a build to get close to a fix!!! Let’s swat this bug for real!
