You may have seen a crpytic message like “async task jammed” or “thread timed out” with X-plane 8.20 or 8.30, after which the sim quits. Here’s what’s going on and what we’re trying to do to fix this.
X-Plane offloads part of the scenery load process to a second processor. (If you don’t have a second CPU, the first one does the work, but the operating system switches between the two tasks automatically.) For example, X-Plane “outsources” the work of turning a road (which is just a line segment in a scenery file) into real 3-d roads using the second CPU.
X-plane puts a request for the roads out to the second CPU, and then every frame it checks to see if it’s done. If the roads are processed, it can then send them to the video card and go on to the next thing to be done. If the roads are not finished – no biggie – it just checks again on the next frame.
Now here’s where the trouble begins: sometimes X-Plane needs the outsourced work to be done immediately. For example, when we load new scenery we have to recycle the memory from the old scenery. This requires the work of building the roads on the old scenery to be done! When this happens, X-Plane waits (up to 2 minutes) for the background work to finish.
If after 2 minutes the work is not done, X-Plane gives up and puts up a cryptic message like “async task jammed” or “thread timed out”. (In 830 beta 11 the message should be less cryptic.) I must admit: I do not understand how this condition happens; it has never happened on my machine or Austin’s.
Here’s what we’re doing to alleviate the problem: in X-Plane 830 you can turn off the “load scenery in the background” option. When this happens, rather than outsource jobs to the second CPU, X-Plane will do the work immediately. This will have two effects:
- There will be slight pauses as you fly while x-plane stops and builds roads, etc.
- There is no way that we can get stuck waiting for the second CPU, because it won’t be used.
Besides turning off this option, you can reduce the amount of stuttering (either with background loading on or off) by decreasing the amount of roads and objects. For example, on my G5, the sim flies pretty smoothly on “default” settings with background loading on or off. But on “insane” settings the sim is still smooth with background loading on and stutters visciously with background loading off.
I hope to get the background code to be more reliable in 840, but in the meantime you should be able to fly (etiher with or without background loading) in 830 with reasonable results if you set your rendering settings appropriately.
“Scenery load is now twice as fast!” Where do Austin and I come up with these claims? Here’s a little bit of info on on our performance tuning process.
When we work on performance, we always measure a baseline metric and then look at this metric while we work on the code. The test situation must be the same every time, so usually we use a standard plane position (like on a known runway) for framerate, or a known scenery box (for scenery loading).
Here’s the statistics for X-plane scenery load before and after tuning the DSF loader:
Before optimize
DSF load time: 2297036 for file +33-119.dsf
DSF load time: 4614647 for file +33-118.dsf
DSF load time: 4251205 for file +33-117.dsf
DSF load time: 5137940 for file 34-119.dsf
DSF load time: 5038098 for file +34-118.dsf
DSF load time: 3893140 for file +34-117.dsf
After Optimize
DSF load time: 1634270 for file +33-119.dsf
DSF load time: 2156187 for file +33-118.dsf
DSF load time: 1882647 for file +33-117.dsf
DSF load time: 2418666 for file +34-119.dsf
DSF load time: 2318480 for file +34-118.dsf
DSF load time: 1819997 for file +34-117.dsf
These numbers are not normaly visible; a special sim option available to us causes it to print the load time per DSF. The nubmers are in microseconds and shown per DSF.
So on immediate inspection you can see improvements of between 140 and 205%. But to really understand what this means you have to consider all of the fine print:
These stats are with objects and roads turned off – this is with scenery on minimal settings in both cases. With more 3-d stuff turned on, the overall loadtime might be better or worse; these statistics don’t tell us anything about that.
These tests were done on the same 6 DSFs in both cases, but other DSFs might have different improvements. You can see that even in the six DSFs here the amount of improvement varies a lot!
These tests were done on my dual 1.8 ghz G5. This is a fast machine for a Mac (but probably only average compared to modern desktop PCs). The speed-up could vary on other machines that don’t have similar performance characteristics, since scenery loading depends on the speed of the CPU, memory bandwidth, I/O performance, etc.
This is all a long-winded way of saying “your mileage may vary”. When Austin and I post performance numbers, they are calculated in strict test conditions with all variables controlled. But since these statistics represent one datapoint, they are not universal changes, just indicators that our code changes are going in the right direction.
(Could it be that an optimization makes a difference on our machines but not yours? Yes – this is always a risk! We try to run on different machines and we listen to user feedback on performance to catch this case.)
Thank you to everyone who has reported the crash bug in beta-8. I found the crash this morning thanks to some helpful bug reports (technical explanation: boneheaded code on my part) and the fix should be in the next beta.
Also it looks like QuickTime 7.0.4 isn’t compatible with the AC3D x-plane export plugin. This makes no sense to me…I spent over an hour remotely debugging on a user’s machine (thanks Peter!) and came to the conclusion that something is borked deep down inside a library. Somewhere. Maebe. I’ll try to get this resolved soon but since I do not have an OS X 10.4 machine, it may take a little bit. In the meantime it is possible to go back to QuickTime 7.0.1.
Here are a few notes on animation:
The animation commands for OBJ8 take two changes (a start and end rotation or translation) and two dataref values and map them. For example,
ANIM_rotate 0 1 0 -40 70 0 1 sim/flightmodel/whatever
would rotate the following geometry around the Y axis from -40 to 70 degrees as the dataref goes from 0 to 1. Intermediate values between 0 and 1 are interpolated. (For example, 0.5 maps to 15 degrees.) But what happens if the dataref exceeds the bounds specified?
The answer is: the result is extrapolated. If this dataref goe sto 2.0, the rotation will go to 180 degrees. (How I came up with that is left as an exercise for the reader.)
You can take advantage of this to animate rotating things like radar dishes and beacons using the sim/time/total_running_time_sec dataref. This dataref is the number of seconds x-plane has been running. For examlpe:
ANIM_rotate 0 1 0 0 360 0 5 sim/time/total_running_time_sec
This will cause the following geometry to continuously rotate, making a full rotation every 5 seconds. But why does the animation loop? Well, let’s take a look in detail:
When the time is 0 the rotation is 0. 5 seconds later the rotation has made it up to 360, one full turn. Then what? At 6 seconds the rotation will now be 420 degrees. But 420 degrees is the same as 60 degrees; you can’t tell whether the object has made one, two or a hundred full revolutions before turning the extra sixty degrees. So the rotation is actually becoming a huge number but the wrap-around cannot be seen, making the appearance of continuous rotation.
Rotations always occur around the origin. But what if we want to rotate around another point in our object? We must use the ANIM_translate command to move the rotation to where we want. For example:
ANIM_trans 3 4 5 3 4 5 0 0 no_ref
ANIM_rotate 0 -1 0 -90 90 -1 1 jetway/circle1
ANIM_trans -3 -4 -5 -3 -4 -5 0 0 no_ref
This will rotate the geometry around the Y axis from -90 to 90 degrees as the circle goes from -1 to 1. But the rotation happens around the point 3,4,5.
Note that the animation-translate commands have the same coordinates for both translations and no dataref. This is a static translation – it is alawys the same and its sole purpose is to move the rotation to be around a useful point. It is always followed by an opposite translation.
(It is possible to omit the second translation and simply build your animated sub-part with 0,0,0 = rotation point instead of 0,0,0 = the object’s location in x-plane. But this is an advanced optimization.)
An example:
A rotating radar dish
And from the end of the OBJ:
LIGHTS 0 2
TRIS 0 1536
ANIM_begin
ANIM_trans 20.226 22.468 -0.039 20.226 22.468 -0.039 0.0 1.0 no_ref
ANIM_rotate 0 -1 0 0 360 0 6 sim/time/total_running_time_sec
TRIS 1536 1116
ANIM_end
One of the secrets of X-Plane development revealed…Nala supervises my work very carefully to be sure I not introducing bugs into the rendering engine. In the event you get threading errors in the next 830 beta, you now know who to blame!
Scenery loading has changed a lot in 8.30. Before 8.30 we loaded 6 scenery tiles (DSFs or ENVs) and completely built them up into memory before we let you fly. We then loaded any new texures while flying. The result is a possible hit on framerate right after a scenery load for textures, and a very long pause while the scenery area gets built.
In 8.30 we now read the full DSF and build the terrain, then start the sim running. We then build the local 3-d “clutter” (objects and roads) while you fly, but we only build up the ones near your plane. As you fly, the ones behind you are destroyed and new ones in front of you are built.
This means a few things:
– Shorter scenery pauses, because we have to do less before we can start the sim.
– Lower framerates right after a load because the sim is doing extra work while you fly to finish the load.
– Lower framerates occaisionally while you fly as a few more roads are built and objects destroyed. This is pretty minimal though because these items come into range gradually.
– Lower memory usage, since only the roads nearby are fully built, not all of them.
This strategy works out particularly well for people with dual-processor or dual-core machines; some of the work to do this “clutter-building” is done on the second CPU while you fly, so the framerate hit is a lot less noticable. Basically we are taking work that was done by the main CPU while you waited and doing it on the second CPU while you fly.
One problem still in 830: when you place the plane, we do not pre-build the roads (but we DO pre-load the textures). If we did not preload the textures, you would be placed in a gray area and see the textures come in. So hopefully in a later beta I will preload the roads if you place the plane using the airport-picking dialog box. This should help provide good framerates as soon as your plane is placed.
I have a new program called “Env2Overlay” that converts the custom obojects in an ENV file to overlay DSFs. This makes it very easy to convert custom airports to version 8.
However one thing I’ve found is that a lot of ENV scenery contains references to missing objects. The ENV loader historically has worked even with objects missing. The DSF loader will refuse to load scenery if objects are missing.
For now I am going to leave the DSF loader in “unforgiving” mode so that missing objects are an error. I think this is more useful to authors: better to clean up the scenery package while converting it rather than convert it to DSF and continue to have missing objects; I think this indicates a packaging problem or other lost content!
I am pleased to say that I have overlays working in X-Plane 8.30. Overlays are DSFs that contain only objects and are loaded on top of the existing terrain. Overlays allow you to add objects to the global scenery to customize airports, etc.
The overlay system will become more powerful over time, but it should be available shortly when X-Plane 830 goes beta.
DISCLAIMER: the overlay DSF listed below may change in format during beta! This is only illustrated to show the relative brevity of overlays compared to full DSFs. (Users who have turned full DSFs to text know how big they can be.)
To give you an idea of what an overlay DSF looks like:
A
800
DSF2TEXT
PROPERTY sim/require_object 1/0
PROPERTY sim/overlay 1
PROPERTY sim/west -71
PROPERTY sim/east -70
PROPERTY sim/north 43
PROPERTY sim/south 42
PROPERTY sim/planet earth
OBJECT_DEF KBOS_TAXI:15R33L.obj
OBJECT_DEF KBOS_TAXI:279.obj
OBJECT_DEF KBOS_TAXI:C.obj
OBJECT_DEF KBOS_TAXI:D.obj
OBJECT 0 -70.999146 42.359913 76.000000
OBJECT 0 -70.997780 42.359650 76.000000
OBJECT 0 -70.997566 42.358742 75.000000
OBJECT 0 -70.996155 42.358467 75.000000
OBJECT 0 -70.993088 42.355259 75.000000
OBJECT 1 -70.999634 42.358353 136.000000
OBJECT 1 -70.998306 42.358078 0.000000
OBJECT 1 -70.996956 42.358261 135.000000
OBJECT 1 -70.996773 42.358913 135.000000
OBJECT 2 -70.999100 42.358513 254.000000
OBJECT 2 -70.994141 42.355080 136.000000
OBJECT 3 -70.992317 42.361107 76.000000
OBJECT 3 -70.998398 42.359409 314.000000
Tools to make overlays coming soon!
I just wanted to say thank you to everyone who has emailed in the last few days – it’s been very uplifting! I’m sorry I haven’t been able to answer everyone individually – but I do appreciate the kind words!
I’ve spent the last week performance tuning X-Plane 830. Performance tuning is a rewarding change of pace from regular global scenery development, because the gratification is instant. Run the sim, get a framerate. Twaek code, run the sim again, get a higher framerate. The gains are immediate and quantifiable.
By comparison, work on global scenery is an exercise in deferred gratification and unpleasant surprises. I write a lot of the code before I have artwork; the results don’t look good until later when the real images are dropped in.
This is a picture of X-Plane 8 beaches with testing textures; Sergio made a beach texture with a descriptive word on each segment and colors indicating their uses. In the simulator, I can verify that the texture placement is correct, the textures line up correctly, they’re not cut off, etc. When people post screenshots of the new global scenery I am usually surprised by what it looks like, because I spent most of the development time looking at images like the one on the left, not the final product.
The unpleasant surprises come from the size of the world: given how big and varied the world is (and how unreliable our global data can be at times), there will always be a location where the particular local data causes our algorithm to make bogus results. A few users have reported vertical walls of water; a classic global data varation problem. Given all of the combinations of shapes of water bodies overlayed on elevation data, our flattening algorithm doesn’t always get it right. Since the water body and elevation data may come from different sources, it’s possible that the water body data is saying there is a lake where the elevation data is saying there is a mountain!
Anyway, enough babbling; Happy Holidays to everyone! Soon we will have a New Year and a new beta of X-Plane – in the meantime, enjoy the time with your families and fly safe!
X-Plane offloads part of the scenery load process to a second processor. (If you don’t have a second CPU, the first one does the work, but the operating system switches between the two tasks automatically.) For example, X-Plane “outsources” the work of turning a road (which is just a line segment in a scenery file) into real 3-d roads using the second CPU.
