I don’t usually blog about plugin issues, but this one falls into limbo, somewhere between plugins, aircraft design and OBJ animation. This is written for plugin developers; if you don’t write plugins, you can tune out.
Plugin Drawing
Plugin drawing is documented in a few tech notes – I strongly recommend reading them all.
The basic idea is this: to draw in a plugin, you register a callback. X-Plane tells you “it’s time to draw” and you draw.
The first rule of drawing callbacks is: you only draw. You don’t do anything else.
The second rule of drawing callbacks is: you only draw. You don’t do anything else.
There are a number of good reasons for this.
- You don’t know how many times, or in what order the callbacks will be called! You might be called four times per frame or none. So doing flight model calculations in a draw callback is a bad idea.
- Drawing has to be fast. In the drawing code we are trying to stuff the GPU to the gills with work to do. Drawing time is not a good time to go off and do other expensive calculations.When we look at the interaction of the CPU and GPU, we know that the flight model takes some pure CPU time, and we can improve efficiency by queuing an expensive OpenGL operation before we hit that CPU-only phase. If your plugin is doing its real calculations during draw time, our pipelining gets thrown off.
- We’re continually increasing the parallelism of the sim via threads to take advantage of multiple cores. But plugins are single threaded by definition. This means that plugin interaction points will (in the future sim) halt parallel operation and slow overall performance. When we design parallel operation, we can try to optimize our design to avoid this “halt” case, but if you are doing something strange (like flight calculations in a draw loop) you’re more likely to fall off the “fast path”.
Surprise
Now here’s the surprising thing: your dataref callback is actually a drawing callback, sort of!
Object datarefs are called back during object drawing. Therefore they have all of the above problems that drawing callbacks have. Here is my recommendation:
If you create a custom dataref that is used by an OBJ for animation, return a cached value from a variable in the dataref, and update that variable from the flight loop.
This avoids the risk of your “systems modeling” code being called more than once per frame, etc.
When you update X-Plane, it scans about 10,000 files before it runs the update. This scan checks every single byte of every file. As a result, it’s a little bit slow – it takes a few minutes.
With the new installer, when you add and remove scenery, we do the same scan on the installed scenery. This scan is very slow – 78 GB is a lot of data!
When the next installer comes out (2.05, which will be cut after 920 goes final), the add-remove scenery function will do a quick scan to see what files exist. This is a lot faster – even with full scenery the scan takes only 10-20 seconds on a slower computer.
I can’t pull the same trick for updates – we need to detect any possible defect in a file during update to ensure that the install is correct!
We get a number of questions about whether X-Plane takes advantage of 64-bit CPUs. The short answer is: “no, not yet”, but here’s the details.
Some of the people who ask are Linux users who haven’t been able to set up 32-bit compatibility libraries for their 64-bit Linux distribution. To me, this is a portability request, e.g. can you make X-Plane work on yet another operating system. I don’t view this as a high priority because you can run X-Plane on a 64-bit operating system using 32-bit compatibility libraries for a number of distros. Linux comes in a million flavors and we can’t support them all.
Other users ask about 64-bit because they have 64-bit CPUs (and possibly paid more for a 64-bit operating system) and want to know when they get some benefit for their money. But…is 64-bits actually useful?
A 64-bit CPU is one that can deal with larger integral numbers – as a result it can access more memory than 4 GB, and it can do math with very large integer numbers more efficiently. 64-bit CPUs also have some architectural differences that are theoretically beneficial.
Is It Faster?
Joshua tried building an experimental 64-bit version of X-Plane for Linux when he first got his 64-bit AMD box and he found…
…no performance benefit at all.
Doh! This actually isn’t surprising.
- X-Plane doesn’t use very large integral number math. X-Plane uses a lot of floating point math.
- The biggest time-sinks for the CPU in X-Plane are talking to the driver and sorting through piles of scenery. The later task requires a lot of logic and memory access — neither of which work any better on a 64-bit CPU.
So the short answer is: you’re not going to get a fps boost from 64 bit. Sorry.
Memory, Memory, Memory
Now 64-bits is useful because a 64-bit app (on a 64-bit OS and 64-bit CPU) can access more than 3 GB of RAM. We’re reaching the point where a lot of users have 4 GB of RAM and a machine that could be using all of it. While I’ve been dragging my feat on the move, someday we will support 64-bits and X-Plane will be able to use more memory.
But in the meantime, I’ve been doing everything I can to reduce the amount of memory X-Plane uses. This is because optimizing memory usage benefits all users, while 64-bits only helps users with a 64-bit OS and a 64-bit CPU.
I believe that we’ll reach a point where 64-bits is useful for users who want to use a ton of add-ons simultaneously. But the out-of-box X-Plane configuration needs to work with much less memory than 4 GB.
It’s that time of the year again…I’ll be out of the office next week, and more importantly, away from all cell and email contact.
Austin and Randy will be at EAA – stop by and say hi if you’re there. Once Austin gets back, the last few betas should resume – we’re close to an RC on 920 but not quite there.
We’re about to do something strange: any time now, Austin should announce 920 beta 1.
But what happened to the 902 beta program?
The answer is simple: it turns out we don’t need it.
Both 901 and 902 were tiny releases whose sole purpose was to add new languages to X-Plane. (901 added French/German, 902 Russian). The idea was to get our foreign distributors a localized version of the sim faster than we could do the next major patch.
Well, 920 is here, and the Russian port is still being debugged, so 920 will contain Russian and also a huge host of new features. There will never be a 902 final; simply update your beta to 920 when it’s out – any tiny changes in 902 are baked into 920.
Don’t Panic
As always, my advice for beta 1 is: don’t panic. Remember that we are a small company, and thus we have a small number of computers between all of our employees. The first few betas usually involve one or two serious bugs (the sim explodes and sets your computer on fire, etc.) that only happen on the one hardware configuration none of us have.
If you are a user, consider waiting for a later beta, or upgrade a copy of the sim.
If you create third party add-ons, please get a copy of the beta early and start filing bugs! It is much easier for us to fix compatibility bugs when we get the reports early.
Multi-Lingual
A number of users have offered to help translate X-Plane. With 920 we finally have the “new” localization infrastructure in place; I will finally be able to get users started on this process.
The moral of the story is: just because I dismiss an idea by email doesn’t mean it won’t get coded. In particular, one user requested a generic instrument that could display text from datarefs. While my initial reaction was “no way”, I ended up changing my mind. The text instrument will be available in 920 – it displays a string as long as the input dataref is of type “data bytes”. The first intended use is to let you build an FMC interface out of generic instruments – we’ll make datarefs that provide the display strings.
My initial negative reaction was due to the nature of the request: I get a lot of requests for “can you implement feature X in subsystem Y?” One of the biggest questions about any feature is: where does it go within the maze of SDKs that X-Plane supports? So if you specifically want it in a particular place, the answer is more likely to be no.
In particular, most people want feature X in the subsystem they are already using. So 3-d modelers want a feature in the OBJ format, airplane editors want an ACF feature, panel creators want a new instrument, and off we go.
I can’t put a feature into the sim without considering compatibility, and the cost of compatibility goes way up if a feature is in the wrong place in the code.
Put It In a Plugin
A lot of the time my response is “a plugin can do that.” Now I know that that’s a blow-off answer; plugins make all things possible, but they don’t make them easy. Not everyone is a programmer. At the same time, plugins often anticipate sim features by years: consider that the first planes to demonstrate key frames, 3-d cockpits, and per-vertex normals used plugins, with the sim developing the feature later.
My decision to include a text instrument is based on the idea that it makes sense to partition which “SDK” a feature goes with based on the nature of the activity. As of this writing, custom systems modeling has to be done in a plugin – no other interface provides this. But panels can be built almost entirely using the built-in panel editor; with 920 generic instruments provide a huge amount of flexibility.
So the purpose of the text instrument is to keep things this way – imagine that an artist is building a panel (using generic instruments) around the datarefs written by a programmer (using a plugin to do systems modeling). Without the text instrument, the programmer has to write some graphics code (for the first time) and there’s one part of the panel the artist can’t manipulate. This is a case where since the artist could already do 95% of the job, it makes sense to expand generic instruments so they can do it all.
(That’s not entirely true – there will always be custom displays so complex that they can only be programmed. But the 920 generic instruments make most things straight forward.)
As a final note to the rant: we are not going to provide any systems modeling capability as part of generic instruments. Generic instruments are how you visualize the simulation, not how you configure it.
To give a trivial example of why I don’t want systems modeling code in the generic instruments: it would be unfortunate if the pressurization model changed because the 2-d panel scrolled such that the pressurization instruments were no longer on screen!
I commissioned
PropsMan to make me a test bed for some of the new 9.20 airplane features. Clearly he has never worked in a furniture store — this is what he sent me!
In his defense, it’s very comfy, flies surprisingly well, and is a great comprehensive test of a huge pile of advanced aircraft features. Yes, you can close and open the laptop with the mouse in the 3-d cockpit.
(One of the new features in 920 is “
no-op manipulators” – that is, the ability to make 3-d geometry “eat” clicks before they get to the panel, without having to use panel texture as the consuming surface. So when the laptop is closed, you cannot click on the buttons that are exposed when it is open.)
My real question is: how did he know about Laminar’s new secret test vehicle?
I spent part of the morning tracking down installer issues. Here are some things I have learned:
-
If you have Windows Vista and User Access Control, there’s a good chance that the installer may get caught on permissions problems. I don’t fully understand what’s going on, but I don’t think it’s a coincidence that it is US GraphSim customers who see this.
Basically those DVDs, which went to press a little bit early, default to installing X-Plane on the C drive, which is not a generally accessible area. UAC then makes some guesses and allows some but not all operations (or something equally weird) and the user gets stuck.
I am still investigating, but I think the fix is to install to your home folder.
-
Some users have bad DVDs. It happens, and I think it happens more now that we’re dual layer. There’s also a variance in the sensitivity of drives – some users send us back damaged disks and they look bad but we can still read them.
Tech support will tell you this, but if you can’t install, one test is to simply copy the entire DVD to your hard drive; if the copy fails mid-way with “I/O error” or some other message about damaged media, call up tech support, they’ll swap you a new disk.
We try to find duplication facilities that do high quality work, but bad disks do happen. Tech support should be able to make this right for you.
- I’ve gotten a wide variety of weird reports on Linux – there are a lot of distros out there, so potentially a lot of versions of the OS code base. For Linux users I strongly recommend the X-Plane Wiki, where we are trying to accumulate all of the fringe cases.
I’ve been reading Fooled by Randomness (highly recommended – it will either change your life or you won’t finish it because Taleb’s writing style annoys you) – and it has me thinking about the nature of certainty in software development. Consider two approaches to uncertainty and how they are completely at odds with each other.
No Weird Behavior
The “No Weird Behavior” approach goes like this: you never want a harmless behavior inside your code that you don’t understand. The reason is that if you don’t understand the behavior, you don’t really know that it’s harmless!
In fact this isn’t just a theoretical problem – I only truly started to believe in “No Weird Behavior” after fixing several very hard to find, hard to reproduce crash bugs, only to discover (once the analysis was done) that the broken code also produced a very frequent, less harmful behavior. Had I taken the time to investigate the “weird behavior” first (despite it not being of high importance) it would have led me to a ticking time bomb.
No Weird Behavior implies that a bug isn’t fixed until you know why it’s fixed, and that randomly changing code until the behavior goes away is absolutely unacceptable. This shouldn’t be surprising; if a bridge was
swaying would you accept an engineer who fixed it by randomly tightening and loosening screws until it stopped?
Wait And See
I get a lot of email with bug reports, questions, and reports of strange sim behavior. These emails have some unfortunate statistical properties:
- A good number of them are resolved by the user who emailed within a day or two.
- A certain percentage are simply never resolved. (Often I email a question that would point to a clear diagnosis and the user never writes back. I can only speculate that the user answered the question, found the problem, fixed it, and promptly forgot about our emails.)
- A certain percentage are solved by the user, who tells me what the problem was, and it was something I would never, ever be able to help them with. (Things like “I have this program called WickedMP3Player – it turns out if I set its visualizer setting to ‘Rainbow’ then X-Plane stops crashing” when I’ve never even heard of the WickedMP3Player program to begin with.)
- There is a high correlation between bug reports reported by a very small number of users and a resolution from the above categories, or a resolution by the user changing his or her local configuration.
So playing the odds, the right thing to do when presented by a third party with weird behavior is to wait and see who else reports it; the frequency of reports gives us information about the likely resolution.
(And for those of you who think our tech support are being lame when they ask if you’ve updated your drivers, they are playing the odds to the hilt – they ask you about drivers first because updating drivers fixes an absurdly huge number of the tech support calls we get.)
Getting It Wrong
So we have motivation to investigate everything (no weird behavior), motivation to ignore everything (wait and see) and a rule of thumb that the frequency of reports can help us pick which strategy is best. Of course, sometimes the rule of thumb is completely wrong.
One user reported a crash using the current web updater (version 2.04) – I had not seen this crash anywhere and it was inside the operating system UI code, so I assumed it was a local problem, e.g. some kind of extension or add-on that caused the OS problems.
The problem, it turns out, is simply low frequency – as the incorrect code made it into 902b1, I got a few reports from more users and realized that this wasn’t a local problem, it was weird behavior! (The bug will be fixed in the 205 installer and 902b2, both of which will be out in June.)
Consider this: if you make a measurement of a known quantity with a dubious measuring device, you learn more about the measuring device than the object you are measuring. (If you have a ruler and you don’t know the units, you could determine them by measuring yourself.)
In a number of cases, we have seen serious “should-happen-all-the-time” crash bugs that get reported by very few users. (Once we know the actual root cause of the bug, we can deduce logically whether the bug should happen to all users with the configuration or be intermittent.) We can then look back at the actual number of reports to make a judgement call on how much testing is really happening.
For example, we can make some guesses about how quickly a new Macintosh have saturated the X-Plane user base when there is a hardware-specific serious bug in just that machine.
We had this with the iMacs (where the runway lights were broken) and we could watch the machines sell – slowly at first, but then quite quickly. (BTW, I think that 10.5.3 fixes this and anti-aliasing problems.) We can even see who runs with anti-aliasing when there is a setting-specific bug (a lot of you do)!
In the end, I think the right approach to balancing “no weird behavior” and “wait and see” is to remember a truth about uncertainty that is very hard for humans to grasp:
The most likely outcome of an uncertain situation is not guaranteed to happen – in fact a lot of the time it won’t.*
So we can play the rule of thumb and wait and see, but we always have to keep one eye toward the improbable, which is still possible!
* Blatant blog cross-promotion…the point of my
big rant here was essentially the same…it’s easy to expect the most likely outcome, but the unlikely outcome will happen some of the time.
For those who posted comments, sorry it took so long to moderate them – for some reason my spam filter decided that notifications of comments are, well, spam, so I just found them now. I should have known people would have jumped into a Vista-bashing thread. 🙂
There is an X-Plane 9.02 beta 1 posted – like 901 we’ve been pretty quiet about this, but you can get it by enabling “get betas” and running the X-Plane updater. Please give it a try. Like 901 it is a small change for the purpose of localization, but it actually has an interesting feature pair:
- True-type fonts and
- Unicode-aware.
This is part of some rework we did to provide better language support. So…you should be able to run X-Plane no matter what weird characters* are in your folder names, name your airplanes funny things, and see diacritical marks. 902 uses a font that provides all of the Latin and Greek/Cyrillic code pages.
Also I have heard reports of improvements based on drivers:
- nVidia has 175.16 drivers out and they apparently address “stuttering” issues. The stuttering issue has been on my list to investigate because it happens under Windows but not Linux. If you have stuttering performance on high-end NV hardware, particularly with forests and Windows, please try 175.16 and let me know how it goes.
- ATI has released Catalyst 8-5. Catalyst 8-3 and 8-4 were causing “incomplete framebuffer” errors for some users, but I was unable to reproduce it (after spending a good day trying to jam Windows XP onto an iMac already crammed with Windows and Linux….yet another episode of a Tale of Three Operating Sytstems). Anyway, at least one user reported the issue as fixed in Cat 8-5, so if you are having problems, please try the new driver set.
As always, bugs in the X-Plane beta should go to our bug report form, on the X-Plane contacts page.
* You might accuse me of being American-centric in decrying diacritical and greek letters weird – but the truth is I am computer-centric…anything that is not in the original ASCII set is weird. 🙂
