X-Plane 902 has an instrument limit of 400 instruments for the 2-d panel and 391 instruments for the 3-d panel. 920 will remove these limits, allowing you to build very complex panels via generic instruments.
(An altitude indicator might be built out of 3 or 4 generic instruments, so 400 instruments is a lot for pre-built but not for generic-instrument-based panels.)
The
Manipulator RFC has been rewritten…the new one proposal is less complex.
There are a lot of other panel and airplane-related changes coming in 920 too…
I spent part of the morning tracking down installer issues. Here are some things I have learned:
-
If you have Windows Vista and User Access Control, there’s a good chance that the installer may get caught on permissions problems. I don’t fully understand what’s going on, but I don’t think it’s a coincidence that it is US GraphSim customers who see this.
Basically those DVDs, which went to press a little bit early, default to installing X-Plane on the C drive, which is not a generally accessible area. UAC then makes some guesses and allows some but not all operations (or something equally weird) and the user gets stuck.
I am still investigating, but I think the fix is to install to your home folder.
-
Some users have bad DVDs. It happens, and I think it happens more now that we’re dual layer. There’s also a variance in the sensitivity of drives – some users send us back damaged disks and they look bad but we can still read them.
Tech support will tell you this, but if you can’t install, one test is to simply copy the entire DVD to your hard drive; if the copy fails mid-way with “I/O error” or some other message about damaged media, call up tech support, they’ll swap you a new disk.
We try to find duplication facilities that do high quality work, but bad disks do happen. Tech support should be able to make this right for you.
- I’ve gotten a wide variety of weird reports on Linux – there are a lot of distros out there, so potentially a lot of versions of the OS code base. For Linux users I strongly recommend the X-Plane Wiki, where we are trying to accumulate all of the fringe cases.
If you are developing an add-on for X-Plane, you should always check the log file (Log.txt) after running. (Remember, the sim must quit before the log file is completely written to disk.)
There are three times you should check the log:
- Any time your content doesn’t look the way you expect. (Perhaps the error is described in more detail in the log file.
- Any time there is an error (especially the “a problem with the package X” message, which is always accompanied by details in the log).
- Right before posting your work as a last check.
Posting errors to the log gives us a way to provide verbose feedback to authors when the sim hits a problem without making the user experience too horrible; minor errors are logged and major errors are mentioned to the user only once per package and logged.
The flip side of this is that if you are working on content, you need to seek out these error log messages; the alternative is to have the sim quit every time something goes wrong.
In a previous post I said that our tech support guys will trouble-shoot the most likely problems first (based on what we see in our entire user base) – they’re playing the odds.
Well, a lot of our users do too. Over and over and over I see the recommendation “delete your preferences” as a cure for a wide variety of strange symptoms. And a lot of the time deleting preferences works.
I fear that deleting preferences has become a bit of a
Cargo Cult, that is, a ritual induced to fix the mysterious beast that is X-Plane without consideration to why X-Plane is broken. If the fix works, the previous problem is ignored.
Now here’s the thing: preferences files are relatively small and easy to read! And they’re really easy to save.
So next time you have a problem and consider deleting the preferences, simply move them outside your Resources/preferences folder to the desktop and restart.
If the problem goes away, you can then delete the newly generated (clean) preferences and put the old funky ones back.
If you then truly find a situation where one preferences file causes the problem, you can look at what’s actually different and file a real bug. (Unix nerds: most of the preferences files are text and can be “diffed”.)
At this point, almost every option in the preferences file has a user interface item, so if the preferences file causes the sim to run poorly, there should be a setting that has been changed that can be identified. Screenshots of the other airplanes, weather and rendering settings before and after the prefs might provide another quick way to compare what has changed. Control-period will take screenshots when dialog boxes are shown.
(Remember that the effect of preferences on framerate varies a lot with hardware. There may be some preferences that slow fps a lot but do not make an obvious change in what you see “out the window”. By comparing two rendering settings screenshots you might find something subtle that changed.)
I seem to be in a philosophical mood these days with my blog posts…thought for the day: the human mind easily goes from the specific to the general. Our brains are generalizing machines, pattern matchers finding the rule in the noise.
My preference in creating new scenery-system features is to make them very limited, and my reasoning is: our brains don’t go backward very well. We do not go from the general to the specific.
Now you might think: when making a scenery-system addition, the best thing would be to have a general feature, more useful because it can be used everywhere. But I say: the most important thing is to fully understand the feature – otherwise the feature comes out buggy.
(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)
Since the human brain doesn’t go from general to specific well, it is hard to start with a rule (“let’s allow feature X in all parts of the scenery system”) and comprehensively derive all of the implications; it is human nature to be surprised later by some unintended side-effects.
It is always easier to extend a feature later to its natural full implications than to declare certain uses illegal later, after authors of planned or started trying to use the feature in that way. If the generalization of the feature makes sense, extending it is often quite painless.
Texture Paging – Scope For Now
Texture paging is the ability for X-Plane to raise and lower the resolution of scenery textures dynamically as you fly. This means more VRAM used for nearby things and less for far away things. In practical terms, this reduces VRAM used by orthophotos by down-sampling the far-away textures, making larger orthophoto scenery packages possible. As you fly, the sim reloads some textures at higher resolutions and some at lower. The cost of the features is the load time while you fly, which burns up some extra CPU cores.
It is my hope that we will productize some very simple texture paging in the next major patch of X-Plane 9 (that would be 920, not 902). But the usage will be pretty specific:
- Texture paging will only be available for .ter and .pol textures (we can extend to other scenery types later if it makes sense).
- Texture paging will require changing the .ter and .pol files (X-Plane will not automatically analyze your scenery to see what can be paged.)
- Texture paging will not be available for ENV scenery.
- If you share textures and texture page, the results will probably be really bad and cause chaos. Be sure to use only one .ter or .pol file (and reference that text file only once in the your DSF definitions section) if you want sane paging. We can extend paging to shared textures in the future, but for now orthophotos are the intended target.
I am also deferring work on dataref-driven textures; we’ll get there eventually, and the infrastructure from the pager will make it easier. But dataref-driven textures really need to be available in a lot more places – it’s a bigger, more complex feature* and I can’t keep adding scope to 920.
Make New Meshes!
While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route. RealScenery sent me their new “State of Washington” package to use as test material; I was pleasantly surprised at the high framerate. Part of that comes from them using base meshes and not overlays.
Overlays cause the sim to draw the scenery twice (first the old scenery, then your overlay), burning a lot of pixel shader and fill power. Base meshes simply replace the old mesh which is at least twice as efficient.
(I’m just going to keep beating the dead horse of base meshes because I believe that the sooner everyone moves toward base meshes, the more bang for our hardware buck everyone gets.)
* In particular, remember that texture paging happens on threads. But datarefs can come from plugins that are not threaded! Insert anarchy here…
I’ve been reading Fooled by Randomness (highly recommended – it will either change your life or you won’t finish it because Taleb’s writing style annoys you) – and it has me thinking about the nature of certainty in software development. Consider two approaches to uncertainty and how they are completely at odds with each other.
No Weird Behavior
The “No Weird Behavior” approach goes like this: you never want a harmless behavior inside your code that you don’t understand. The reason is that if you don’t understand the behavior, you don’t really know that it’s harmless!
In fact this isn’t just a theoretical problem – I only truly started to believe in “No Weird Behavior” after fixing several very hard to find, hard to reproduce crash bugs, only to discover (once the analysis was done) that the broken code also produced a very frequent, less harmful behavior. Had I taken the time to investigate the “weird behavior” first (despite it not being of high importance) it would have led me to a ticking time bomb.
No Weird Behavior implies that a bug isn’t fixed until you know why it’s fixed, and that randomly changing code until the behavior goes away is absolutely unacceptable. This shouldn’t be surprising; if a bridge was
swaying would you accept an engineer who fixed it by randomly tightening and loosening screws until it stopped?
Wait And See
I get a lot of email with bug reports, questions, and reports of strange sim behavior. These emails have some unfortunate statistical properties:
- A good number of them are resolved by the user who emailed within a day or two.
- A certain percentage are simply never resolved. (Often I email a question that would point to a clear diagnosis and the user never writes back. I can only speculate that the user answered the question, found the problem, fixed it, and promptly forgot about our emails.)
- A certain percentage are solved by the user, who tells me what the problem was, and it was something I would never, ever be able to help them with. (Things like “I have this program called WickedMP3Player – it turns out if I set its visualizer setting to ‘Rainbow’ then X-Plane stops crashing” when I’ve never even heard of the WickedMP3Player program to begin with.)
- There is a high correlation between bug reports reported by a very small number of users and a resolution from the above categories, or a resolution by the user changing his or her local configuration.
So playing the odds, the right thing to do when presented by a third party with weird behavior is to wait and see who else reports it; the frequency of reports gives us information about the likely resolution.
(And for those of you who think our tech support are being lame when they ask if you’ve updated your drivers, they are playing the odds to the hilt – they ask you about drivers first because updating drivers fixes an absurdly huge number of the tech support calls we get.)
Getting It Wrong
So we have motivation to investigate everything (no weird behavior), motivation to ignore everything (wait and see) and a rule of thumb that the frequency of reports can help us pick which strategy is best. Of course, sometimes the rule of thumb is completely wrong.
One user reported a crash using the current web updater (version 2.04) – I had not seen this crash anywhere and it was inside the operating system UI code, so I assumed it was a local problem, e.g. some kind of extension or add-on that caused the OS problems.
The problem, it turns out, is simply low frequency – as the incorrect code made it into 902b1, I got a few reports from more users and realized that this wasn’t a local problem, it was weird behavior! (The bug will be fixed in the 205 installer and 902b2, both of which will be out in June.)
Consider this: if you make a measurement of a known quantity with a dubious measuring device, you learn more about the measuring device than the object you are measuring. (If you have a ruler and you don’t know the units, you could determine them by measuring yourself.)
In a number of cases, we have seen serious “should-happen-all-the-time” crash bugs that get reported by very few users. (Once we know the actual root cause of the bug, we can deduce logically whether the bug should happen to all users with the configuration or be intermittent.) We can then look back at the actual number of reports to make a judgement call on how much testing is really happening.
For example, we can make some guesses about how quickly a new Macintosh have saturated the X-Plane user base when there is a hardware-specific serious bug in just that machine.
We had this with the iMacs (where the runway lights were broken) and we could watch the machines sell – slowly at first, but then quite quickly. (BTW, I think that 10.5.3 fixes this and anti-aliasing problems.) We can even see who runs with anti-aliasing when there is a setting-specific bug (a lot of you do)!
In the end, I think the right approach to balancing “no weird behavior” and “wait and see” is to remember a truth about uncertainty that is very hard for humans to grasp:
The most likely outcome of an uncertain situation is not guaranteed to happen – in fact a lot of the time it won’t.*
So we can play the rule of thumb and wait and see, but we always have to keep one eye toward the improbable, which is still possible!
* Blatant blog cross-promotion…the point of my
big rant here was essentially the same…it’s easy to expect the most likely outcome, but the unlikely outcome will happen some of the time.
Since X-Plane 9 went final I’ve been going in about 5 different directions with the code, and as a result my posts have diverged pretty far from my charter within the company, namely scenery. Here’s some of what I’m looking at on the scenery front:
Texture Paging
Texture paging is a new feature where X-Plane can change the resolution of orthophotos while you fly. The big limitation of orthophoto-scenery right now is that X-Plane loads every orthophoto at the maximum user-defined resolution when the underlying DSF/ENV is loaded. This can cause X-Plane to run out of virtual address space and crash. With texture paging, only the nearby textures are loaded at high resolution.
Far away textures may be low res, but you’ll never notice because they are far away (and thus you’re seeing them over only a few pixels on screen anyway.
The cost of this technique is that textures are being loaded over and over and over. But this cost is made less painful by two features:
- DDS texture loads are very fast and cheap, especially at very low resolutions.*
- With the next major patch, texture loading will be entirely on a second core (if you have one) and can even span multiple cores.
This feature will require modification of scenery packs in that you’ll have to add some “paging” information to your .ter files; I will try to get a preliminary spec posted soon. Because you only have to modify text files, it should be possible to create “paging patches” to existing DSF orthophoto sceneries that are fairly small.
I do not know if paging will be available for .pol files. My guess is yes, but I reiterate that using .pol files for large areas of orthophotos is a bad idea! Make a new mesh!
Improved Multi-Core Performance
This is going to be an on-going process through the entire v9 run and beyond, but basically with every change to our rendering engine (and we make some serious modifications with almost every patch to keep it modern and competitive) we try to move toward using the new hardware (meaning machines with 2-4 cores) more efficiently. Some of this will be in 920, although my guess is we’ll need at least one more patch after 920 to really see the improvement.
Tools
It’s going to be a little bit before I can put more time into the various scenery tools. My top priority is to keep up with user feedback and work on MeshTool. Hopefully we’ll also declare WED final soon. But for now, since I am working on cockpit and airplane modeling features, my next set of work will probably be for the airplane authors.
Shaders
I do want to put additional shader work into v9. I realize I am at the edge of provoking a bunch of rants in the comments section, so let me save you some time:
“Hey Ben, stop working on eye candy and create some more tools. I don’t want a shader feature that makes my 1.2 ghz G4 with a GeForce 2 MX run any slower. You should finish the tools so they do exactly what I want!”
Okay, I admit, that was totally unfair…there is a lot of truth in the complaints about shaders vs. tools.
- I really do try to keep an eye on system requirements, particularly once we shipped. I’m going to try to prioritize shader features that improve existing rendering techniques, rather than introduce new rendering techniques, so we don’t seriously lower fps during the version run. But also bear in mind that shaders can be turned off, and there are users who have GeForce 9s and such.
- Tools are very important, hence the effort to get MeshTool out. But tools without engine work aren’t very useful either; most of the engine work we do is needed to keep performance up so that new scenery (made with those new tools) don’t bury the sim. For example, MeshTool without texture paging would be a dead-end…you could easily make a MeshTool scenery that you can’t fly.
So in planning what goes out when, I look for clumps of features and tools that can go together to make some sense: WED to use apt.dat 850, texture paging to go with MeshTool. It wouldn’t make sense to defer texture paging to make the next tool while MeshTool is waiting for engine enhancements.
* A DDS already has all of the smaller-version pre-compressed textures in the file. So loading a DDS at low res involves loading a small amount of data from disk to memory and sending it to the graphics card.
By comparison, a PNG file only contains the maximum size, so to load a PNG at low res, we load the largest size, shrink it on the fly, then apply DDS compression.
For those who posted comments, sorry it took so long to moderate them – for some reason my spam filter decided that notifications of comments are, well, spam, so I just found them now. I should have known people would have jumped into a Vista-bashing thread. 🙂
There is an X-Plane 9.02 beta 1 posted – like 901 we’ve been pretty quiet about this, but you can get it by enabling “get betas” and running the X-Plane updater. Please give it a try. Like 901 it is a small change for the purpose of localization, but it actually has an interesting feature pair:
- True-type fonts and
- Unicode-aware.
This is part of some rework we did to provide better language support. So…you should be able to run X-Plane no matter what weird characters* are in your folder names, name your airplanes funny things, and see diacritical marks. 902 uses a font that provides all of the Latin and Greek/Cyrillic code pages.
Also I have heard reports of improvements based on drivers:
- nVidia has 175.16 drivers out and they apparently address “stuttering” issues. The stuttering issue has been on my list to investigate because it happens under Windows but not Linux. If you have stuttering performance on high-end NV hardware, particularly with forests and Windows, please try 175.16 and let me know how it goes.
- ATI has released Catalyst 8-5. Catalyst 8-3 and 8-4 were causing “incomplete framebuffer” errors for some users, but I was unable to reproduce it (after spending a good day trying to jam Windows XP onto an iMac already crammed with Windows and Linux….yet another episode of a Tale of Three Operating Sytstems). Anyway, at least one user reported the issue as fixed in Cat 8-5, so if you are having problems, please try the new driver set.
As always, bugs in the X-Plane beta should go to our bug report form, on the X-Plane contacts page.
* You might accuse me of being American-centric in decrying diacritical and greek letters weird – but the truth is I am computer-centric…anything that is not in the original ASCII set is weird. 🙂
I’ll warn you in advance: this is going to start off topic and go way off topic. “Catching up” with the changes to Mac OS, Windows and Linux has me thinking about the nature of technology. I feel a little bit guilty about this post because it’s going to turn into a rant about Vista, and ranting about Vista is like shooting fish in a barrel. On the other hand, having used Vista, well, I have a lot of rant to give.
One of the most important things to understand about technology (and computers are no exception) is that changes in the scale of the technology change the very nature of the technology. That is, as you make computers faster and cheaper, at some point the sum of all of those small improvements changes the fundamental nature of the beast. We’ve seen this as the computer transformed from main frame to desktop (which is really just a change in cost and size), finding an entirely new audience, and now again as the computer changes from what we know of now as a computer to cell phones, MP3 players, and other small, mobile devices.
“Commodification” is what happens when, as things get better, cheaper, faster, etc., consumers stop caring about the marginal improvement. Back in the days of Windows 95 and 386’s, there were ways you could improve the operating system and hardware in substantial ways; a doubling in processor speed and a rewrite of the operating system got you protected memory, which meant less data loss.
A few years ago we reached the point where desktop hardware became commodified. For the average user, 1.8 vs 2.2 ghz makes no difference at all. It’s a question of how quickly your computer can wait for keystrokes and data from the internet. (Answer: even a lowly Celleron is light-years faster than the I/O devices it typically has to talk to. Even if you’re the last kid in your class at Harvard, you’re going to be bored discussing politics with a bunch of four-year-olds.) At that point things became very difficult for major vendors like IBM (sold out), HP and Compaq (merged), Gateway (bought out of it’s misery), etc. The price of a desktop plummeted from over $1000 to less than $400.
I believe we’ve reached the point where operating systems have become a commodity as well;
- Every major operating system has all of the features of a “real” operating system – that is, protected memory, virtual memory, plug & play driver support, etc.
- The performance for normal applications is just about the same; there are some specific variations that matter in the server market, but for all practical purposes the operating system is not in the way, and the machine is much faster than users need anyway.
- Every major operating system has a similarly designed GUI experience that, once you get used to the quirks of where the close box is, is just about the same, more or less. (Mac users – keep your pants on. 🙂
And this is why life is not so good for Microsoft. In a non-commodified market, you can charge a premium for incremental improvements over the competition. That’s a game Microsoft can play – they have a lot of capital to invest in changing their operating system, as long as they are rewarded with a lot of cash for doing it. (And normally they are – about six billion dollars for a major OS revision, I’m told.)
The problem is that operating systems are now a commodity. Simply put, users don’t need a new operating system. There are no big ticket features missing from OS X 10.4, Windows XP, or Linux 2.6. This makes Microsoft’s business model fundamentally vulnerable to Linux for the first time. If the name of the game is:
- Keep costs down, as low as possible.
- Incrementally improve quality very slowly without ever causing the pain of a major OS upgrade.
That’s a game Linux, with their army of distributed bug fixers and free source code, is going to win.
When I looked at Windows XP and Ubuntu 6.06 I was afraid that Linux wouldn’t make traction into the desktop market. I blamed the adoption of X11, the KDE/GNOME schism, and the Linux communities’ being made up of Shell nerds for the tolerable desktop experience.
But look where we are now: Vista is a vehicle for bloat. Combine “we make money by shipping major features” and “there are no more major features to ship” and you get Vista…an attempt to change a lot of things when you should have left things alone.*
By comparison, Ubuntu pretty much just works – you put the live CD on your machine, it asks you some questions and installs…it knows about more hardware, has less bugs, more drivers, and a better user experience. In a commodified operating-system space, the only thing to do is try to avoid a bad user experience – if you can’t offer a really juicy carrot to users, try to avoid hitting them with a stick.
And it is in this environment that the Mac is actually gaining market share. Apple’s business model has always been at odds with the industry. Complete vertical integration meant higher costs, lack of market share, and out-of-date technology – back when having more for less meant something, that was a real weakness, and explains why the Mac never dominated in market share.
But what a difference a decade makes! Hardware is now commodified (and Apple is integrated at the system-building level, leveraging cheap third party parts like they always should have). Operating systems are commodified. But on the one frontier left, quality of user experience, Apple’s vertical integration gives it an immense advantage.
The question is: why does an operating system “just work”?
- Vista: it doesn’t. There are too many systems and not enough testers and engineers trying to solve the problem.
- Linux: massive distributed engineering. For any given hardware system, eventually a Linux nerd will integrate it. Anyone can solve the problem of poor user experience.
- Apple: they have it easy. With only half a dozen machines in production (and maybe another two dozen legacy configurations to support) they have a much smaller configuration space to worry about than anyone else.
I don’t kow what Microsoft’s future is, but it can’t be very good. At some point they are going to have to transition from a “major revision for cash” to an “incremental tuning” approach to operating systems. As long as they have market share, they still get the “Windows tax” – that is, their OEM pricing from major vendors on every new computer that is built. It’s going to be harder and harder to convince the entire world to make a major jump (see how well XP to Vista went). In this situation, they’d be better off with a more solid operating system. It’s unfortunate that they’re going to have to try to sustain market share with Vista.
Their best-case scenario is that they eventually get Vista back to an XP-quality experience, in which case all they’ve done is spend a huge amount of R&D; money and pissed off a lot of customers to maintain the status quo.
* I have mixed opinions on Vista’s video-driver-model change. But that’s a different post.
Austin posted another State-of-the-Union yesterday, and he was good about only mentioning things that are fairly close to completion (e.g. the panel texture optimizations) as well as things that are general (e.g. threading optimizations). After a long beta, when 9.0 went final I sort of went crazy and started a whole series of new projects at the same time; here are some things that I have in progress:
- Major rewrites to the texture management code for both better orthophoto handling and better threaded performance.
- Moving the sim to unicode and true-type fonts, with a new system for translating the application’s strings. (I’ll post more on this soon – thanks to those who have already volunteered to work on translations!)
- Working on new shader technology to take better advantage of DirectX-10-class hardware.
- A big pile of airplane features to complement what was added to 9.0.
That’s a bit much for the next patch, so it’s likely that only some of these things will actually make it into the next patch, and I’m not sure what you’ll see. A lot of sim work goes in as a series of small independent pieces; only the last parts of a feature are user-visible.
For example, the first part of texture work was simply rearranging how the code was structured to make new things possible. Change to the user experience: none. The second part changed threaded handling of textures, which at least shows up as performance improvements. But both the stage for new features later.
So even if a lot of the above doesn’t make it into the next major patch, a lot of ground work is going in, setting us up for features later.
