I’ve posted a number of times on the transition to 64-bit, we have release notes and a FAQ, as well as a number of tech notes for plugin developers, but in this post I want to cover how I see the community moving over to 64-bit.
64-bit is a fundamentally confusing issue – it’s one of those things that should be totally transparent to computer users, but due to the nature of code and plugins, 64-bit is not transparent. So more explanation will hopefully at least not confuse the issue even more. Maybe.
Is a Transition to 64-bit Even Necessary? Does it Have to be Now?
The short answer is yes. 32-bit applications simply cannot take advantage of more than 4 GB of physical memory (among other limitations). Every year CPUs get faster, people get more cores, GPUs get faster, GPUs ship with more VRAM, and every now and then we even get a faster PCIe bus. During this time hard drive, SSD and RAM prices continue to fall. You can bring your computer up to 8 GB of memory (Crucial memory – good stuff!) for $34. But what’s the point if the app you are feeding is 32-bit?
In other words, the march of hardware is unrelenting, and if we want X-Plane to be able to use the full power of our computers, we have to go to 64-bit.
How soon do we need to go to 64-bit? Mac users who like add-ons have been crammed up against that 32-bit ceiling for a while now, while Win 7/64 users mostly still have some headroom. But the real question is: could we finish out a major version run in 32-bit? Could we wait years to go 64-bit? I think the answer is no. Even if we didn’t care about the RAM limit on Mac and Linux* 64-bit will become a Windows issue too; it’s only a matter of time, and I suspect not that much time.
The Phases of Adoption
This is how I see the migration from 32-bits to a 32/64-bit world going:
- Beta. That’s where we are now. In this phase, 64-bit technology is unproven even in the sim itself. Plugin developers are hopefully trying the 64-bit betas and looking for major critical bugs, but 64-bit isn’t useful for actually flying. (Well, the sim is beta, and I never recommend flying on the beta.) I don’t expect anyone to ship a 64-bit add-on during this phase.
- Crossover. Once X-Plane 10.20 goes final and we have a stable 64-bit build that third party developers can actually test against, we’ll have a phase where some add-ons come out in 32/64-bit formats and others are not yet ported. The length of this phase will be the difference between the fast and slow add-on developers; different plugins will take different time to port, have different levels of complexity, etc. During this phase, users will have to pick 32-bit for compatibility or 64-bit for memory use.
- Acceptance. Eventually we’ll arrive at a point where so many add-ons have been made 64-bit compatible that the norm and expectation is that all add-ons are 64-bit, and the community will mostly have to abandon add-ons that are not migrated forward.
Note that “acceptance” is not only after 10.20 goes final but there is an entire phase between 10.20 going final and being able to “just run all add-ons in 64 bit”. This is because it is going to take time for people to move their plugins forward.
32-bit Still Works!
I cannot emphasize this enough: there is a 32-bit build of X-Plane in the 10.20 betas, and it should just work! The switch from 32 to 64 bits is not a binary transition where we pull the rug out from under everyone in one swoosh and everyone scrambles. It is the beginning of a migration that can happen over time, while 32 and 64-bit apps areboth available to everyone.
* I have heard people tell us that we don’t care about the Mac just when it is surging, that we should drop everything but Windows, and that Linux is, in fact, the wave of the future. We’re going to hedge our bets and keep supporting all three. Having done the work to be cross-platform it doesn’t make sense to drop multi-OS support.
I probably should have said this earlier.
Please do not release 64-bit add-ons.
If you want to beta test your add-on in 64-bits, great! If you have gotten 64 bits working, that’s totally cool!
But, please don’t ship 64-bit code. The problem is: we are still in beta and there are still open issues with 64-bit plugins that have not been resolved. If we make changes to the plugin system during beta to make 64-bits work right and your add-on is broken, you won’t be able to react to it if it is already released.
So wait – make sure to test against the RC. Like new features and scenery file formats, while the new features is in beta, it is unstable and we’re not going to try to ‘protect’ you from change. Once we go final, things will be set in stone but we are not there yet.
(In beta 3 we changed our address space layout on Mac…that should give you an idea of how new this stuff is.)
Please file a bug if:
- You have no sound with the default aircraft in X-Plane 10.20 beta 3 (either 32 or 64 bit).
- You do have sound on 10.11.
- You are on Windows.
If you meet all 3 conditions, please file a new bug even if you filed this before; beta 3’s log.txt includes better diagnostics to try to trace this; we need to know if changes we made fixed this problem or if it’s still out there. (Like most configuration bugs, our machines here in the lab do have sound.)
Please do not file sound bugs against 3rd party aircraft in 64-bit mode; most of the time it’s the plugin not loading.
X-Plane 10.20 beta 3 is out. This build went out without new art assets. We have a pile of new art assets waiting to go out the door: autogen from Alex, more terrain textures from Albert, an airplane upgrade from Javier, and fixes to airplanes and aircraft from Tom.
But I held them back and kept them out of this release. Why am I such a bastard?
The answer is: two complex low level changes went into this beta, and I want to capture at least one beta’s worth of crash report data to check for destabilization. The new art assets may use more memory (especially if users crank them up to see what’s going on); if we rolled everything into one beta, we wouldn’t be able to tell art-induced crashes from code-change induced crashes. So we’ll roll things out in steps. As I have said before, the purpose of betas is not to have an awesome flying experience, it’s to find bugs!
Bear in mind that not all of the bugs reported are fixed; we cut beta 3 to get these two specific code changes fixed; more bug fixes are coming soon. Please see the release notes for what is fixed. please do not re-report bugs unless they are listed as fixed but still show up as broken on your machine.
I’m hoping to get a beta 4 out over the weekend with art assets if beta 3 goes smoothly.
So What Did You Do?
So what are the two bug fixes that are major enough to warrant their own beta?
- A change in the load order between X-Plane’s sound code and OpenAL. I realized that the old way we load sound (e.g. in 10.10) makes it virtually impossible for a globally installed plugin (one in Resources/plugins) to share OpenAL, even if it follows all of the rules set out in the OpenAL plugin tech-note. This change fixes that, apparently fixes conflicts between Gizmo and SASL, and hopefully should let plugins do the right thing. Plugin developers, see here for more info. (The original load order assumed that OpenAL would be more robust to two copies being opened; it now appears that this is not the case and OpenAL can be quite picky at times.)
- The OS X 64-bit memory layout has been modified to be compatible with LuaJIT 2.0, the Lua engine inside most Lua-based plugins (e.g. Gizmo, SASL, FlyLua, etc). This does not entirely fix the porting problems developers have been seeing with LuaJIT, but it at least makes the plugins work under light memory conditions. (These plugins would fail 100% of the time in beta 2.) Here’s the technote; I am in contact with most of the plugin developers using Lua, but if you use Lua and haven’t heard from me, email ben at x-plane dot com.
These are not the only bug fixes in beta 3, but they are the two that are weird enough that we want ‘clean’ bug data, without additional art assets to cloud our view.
So: hopefully these two changes will go smoothly and we’ll get on to beta 4 and art assets soon. The artists work their butts off to create this stuff and we want to roll it out!
Will Lua Ever Work With 64-Bit OS X?
A few developers who depend on SASL and Gizmo have asked me whether Lua will work with 64-bit X-Plane on OS X. The only answer that I can give is that I do not know, but I am optimistic that we will engineer a solution. I consider making Lua work to be a very high priority.
Lua is in almost every commercial third party airplane we’ve seen in the last few years; the market has spoken and third party developers have said “we like Lua.” So I think it’s worth doing some serious engineering to keep Lua working. Making Lua work with 64-bit OS X is currently my top priority bug.
This post follows up on the case for DDS and DDS in v10 and answers some basic questions.
DDS and Texture Compression are Not the Same!
This is the first and most important point: the decision to compress textures and the decision to use DDS are not the same!
Texture compression is decided by the user. If the user clicks “compress textures” (and nearly all users should do this) then 99% of the textures in your add-onwill be compressed. You can’t avoid this. The user has spoken and said “sorry, I don’t have 3 GB of VRAM, we need to tighten things up.”
DDS is a pre-compressed format; DDS says “well, if you are going to compress, I have already done the compression for you with DDSTool, so it will look better. See the pictures in the posts above for why this is so important!
I cannot emphasize this enough: most users are going to have to use compression to save VRAM, so you should ship DDS files so that the compressed images look good. You can ship DDS and PNG if you want to ensure that the uncompressed case looks good too, but I think that users running uncompressed is going to be rare.
Does DDS Improve Framerate?
No! See the above post: the user may be running compressed textures regardless of whether you ship a DDS, so when you ship a DDS you may see no VRAM savings and no FPS change. All you will get are compressed textures that look better.
Does Compression Improve Framerate?
Sometimes. Texture compression will improve framerate under two conditions:
- The texture unit bandwidth on the GPU is exhausted. This is very rare now – I haven’t seen this happen since the Radeon 9600 XT.
- The user is out of VRAM. This is quite common.
But recall how free VRAM affects framerate:
- When the user runs out of VRAM, the fps loss is often catastrophic. The user might see huge jerking, big pauses in framerate especially as the camera moves, and performance will be unusable.
- When the user has extra unused VRAM, there is no benefit – framerate does not go up indefinitely by freeing up VRAM.
So most users will simply turn down texture resolution until they can fly. Thus the real effect of texture compression is to let users turn their overall texture resolution back up again one notch, improving the overall look of the sim.
Is There Any Way to Make DDS Look Less Bad?
Sometimes DDS is relatively harmless to a texture, and sometimes it just absolutely destroys it. I don’t have a good answer to this, except: be sure to run the latest DDSTool/XGrinder (see the “command line tools” option on this site) when targeting X-Plane 10, and never set your gamma to anything other than 2.2. I do hope to someday look at improving DDS grinding capabilities.
X-Plane 10.11 is now final, so you will receive an update notification even if you have not participated in the beta program.
Starting with 10.11, I am stripping the ‘r’ designation off the end of the build; the complete build number is still visible in the log file and properties, but what you see on your startup screen will just read 10.11.
I do not know if we will cut an X-Plane 10.12 before we get into a 10.20 public beta of 64-bit; I have seen a number of bug reports for 10.10/10.11 that need investigation, but I don’t know if the problems are due to bugs, running out of memory, rendering settings cranked too high, etc.
The main goal of 10.20 is to get us to 64 bit; we’ll also ship better anti-aliasing and autogen in that build. We aren’t sure what will come in 10.30, but at this point more performance tuning and stabilization is likely.
A while ago, I wrote up a summary of the issues with OSM and water. I still do not know a time-frame or plan for OSM recutting, but I can say it will be after getting 64-bit done, and once 64-bit is done, it will be high priority. If the water you care about is not in the OSM map itself, a recut won’t help until you or someone else in the OSM community updates the map.
(For Americans who are frustrated about water that was in X-Plane 9 but is not in X-Plane 10: X-Plane 9 used US Government data. There is work in the US mapping community to get the NHD data into OSM; I suggest participating in this effort. You can import and check a single water shed and thus bring a local region up to date, and the changes are ‘forever’ since they are in the OSM data set themselves.)
I’ve seen a few bug reports complaining about ‘flicker’ with HDR enabled. It took me a few tries to understand that the users were not actually reporting Z-Thrash (which is what I think of when someone says ‘flicker’, but were actually reporting temporal anti-aliasing of anisotropic meshes like roofs and roads.
Ants are alienating an icy tropical metro what now?!?
Sorry, graphics programmers have lots of big words for things that aren’t actually that complicated. (Seriously, we call a daytime texture an “albedo”. Who is Mr. Bedo anyway??) But basically the issue is this:
- We have something that appears long and thin on the screen, like the roof of a far off building (wide, but not tall, in pixels) or a road (again, wide, but not tall – a road might be 20 pixels wide but only 1 pixel tall on screen). Anisotropic just means different lengths in different dimensions, more or less.
- The road or roof will be rendered in a stair-step manner, as the graphics card tries to approximate a diagonal with pixels.
- As the camera moves, which pixels form the stair-step will change every frame, causing parts of the road or roof to flicker into and out of existence on a per frame basis.
Going for a Swim
In the old days, this effect used to be called ‘swimming’. A diagonal line would appear to ‘swim’ as the stair-step pattern changed as the camera changed. The swimming was annoying, but if you had a lame graphics card, you could live with it.
The problem is that in X-Plane 10, a lot of the meshes we draw are a lot smaller. As we build more 3-d detail and improve the engine to draw more detail on screen, the result is a lot of really small things. In X-Plane 9 we could draw 5-10k objects; now we can draw over 75k objects. That means that individual objects on screen may be 1/10th of their size (since there are more of them).
So instead of having big objects with big triangles that ‘swim’, we have tiny triangles that flicker out of existence entirely.
Anti-Aliasing 101
One reason I haven’t blogged about this before is because there are a ton of different full-screen anti-aliasing technologies out there and the prospect of explaining them was daunting. Fortunately Matt Pettineo did an awesome job with this post. Go read it; I’ll wait here.
The main idea is that full screen anti-aliasing draws everything bigger and then down-sizes it to get softer edges. Diagonals don’t look stair-stepped, and a tiny roof won’t flicker into and out of existence because relative to the larger size that it was drawn, the roof is no longer tiny. In other words, 4x MSAA makes everything 4x less tiny from a perspective of a triangle being ‘too small to not flicker’.
The second reason why I am getting bug reports about flicker (besides a larger number of smaller triangles) in v10 is that HDR mode doesn’t use conventional MSAA. For various technical reasons, MSAA works poorly with a deferred renderer, and HDR is a deferred renderer. So like many games today, X-Plane’s problem is to anti-alias without letting the hardware do it. If you’re used to 16x MSAA from your graphics card, HDR with no FSAA is a rude surprise.
Current Option Number One: FXAA
FXAA is an anti-aliasing shader written by Timothy Lottes at NVidia. FXAA is typical of a whole category of anti-aliasing solutions in that it is a post-processing solution – that is, it takes an aliased, jagged image and attempts to smooth out the image after the fact. (MLAA is also in this category.)
FXAA has a few things going for it that are good:
- It’s very fast. The cost of enabling FXAA is very low compared to other anti-aliasing algorithms.
- It doesn’t consume any extra memory or VRAM.
- It produces smooth diagonal lines, more so than lower-levels of FSAA.
It does, however, have one major down-side: because it doesn’t actually draw the scene at a higher resolution, any mesh that is so small that it is flickering is still small, and thus it will still flicker. On any given frame, the roof will have no jagged edges, but the roof may simply not exist in some frames. If the roof isn’t drawn at all, FXAA can’t fix it in a post-process.
So FXAA is fast and cheap and makes still images look nice, but it can’t deal with temporal artifacts, that is, flicker between frames.
Current Option Number Two: SSAA 4X
4x SSAA simply means we draw the entire world at double the resolution in either dimension, and then down-size it later. Jagged edges become blurred in the down-size and thus aliasing is reduced. (Nerd note: when technical papers talk about OGSSAA, they mean ordered grid super-sampled anti-aliasing, which just means the image is bigger. 🙂
The up-side to SSAA is that it reduces flicker. Because the drawn image is bigger, very small elements won’t flicker (since they are bigger when drawn).
The down-side is the cost: 4x SSAA is the same as doubling your screen res in both dimensions. And if you’ve experimented with monitor resolutions, you know that once you are GPU bound, doubling the resolution in both dimensions uses 4x the VRAM and cuts your framerate to a quarter of what it was.
So the big problem with 4x SSAA is cost. Since we’ve improved HDR performance in 10.10r3 I’ve seen more users reporting the use of 4x SSAA. But it’s not cheap.
Newer, Better Options
I have two new tricks for HDR FSAA that I’m hoping to roll into 10.20. (They’re new to X-plane; I am sure someone else has coded these things in other games before.)
First: FXAA and SSAA can be used at the same time in the engine, for better quality than either can provide on their own. SSAA does better at fixing temporal artifacts like flicker (because it makes things ‘4x bigger’ relative to the size at which they flicker) but FXAA does better at making diagonals ‘less jagged’. Now you can have both. (10.20 features a newer version of FXAA.)
Second: I realized that our aliasing is anisotropic (there’s that word again) meaning it’s not the same in both directions. X-Plane’s worst aliasing comes from long thin horizontal screen elements like roads, and roof tops. Therefore having more anti-aliasing vertically than horizontally is a win.
So rather than just have SSAA 4x (which is twice as big horizontally as vertically) we can now do 2x (only vertical) and 8x (2x horizontal, 4x vertical). This provides a range of options; 2x SSAA will be affordable to some users who can’t run 4x SSAA at decent framerates. 8x SSAA will provide anti-flicker that should be as similar to non-HDR with 16x MSAA for urban scenes, for those who have a big new graphics card.
I posted a set of technical test pictures here.
What about TXAA?
NVidia has announced a new anti-aliasing technology, called TXAA. At this point there isn’t enough technical information published about it for me to comment meaningfully on it. My understanding is that TXAA is not yet available for OpenGL based games.
I can say that in the future we will continue to try to adopt the best anti-aliasing technology we can find, and the problem X-Plane faces (anti-aliasing for a deferred renderer) is not at all unique to X-Plane, so it is likely that there will be good solutions forthcoming. Far smarter minds are working on this problem at ATI and NVidia as we speak.
Yesterday I was able to reproduce and fix a bug that several people reported against X-Plane 10.10; the bug is random, which explains why, when I tried to follow their reproduction steps, I did not get the same results. The last step in the bug report could have been “get lucky”.
The bug is: on Windows, sometimes the sim would output a ton of LOD warnings for objects (including default objects that ship with the sim) and then framerate would be reduced. Running a heavy framerate test on my PC I saw fps reduced from 25 to 15 fps when the failure happened.
The bug was 8bytes of uninitialized data deep in the OBJ engine; the impact depended on the random contents.
I will cut X-Plane 10.11r3 soon with a fix to this.
Reminder: please do not post bug reports to the comment section. This includes reports of “I saw this bug too or “I saw this, but on OS X” or “it works fine for me” or “why haven’t you guys fixed XXX.” Please report bugs via the bug reporter.
X-Plane 10.11 release candidate 1 is now available online – run the installer, and update with the “get betas” button checked to get it. This is a small bug fix build to get new Japanese strings out; most of these bug fixes were coded during 10.10’s release candidate period but held to keep the sim stable for DVD pressings. It’s a small update.
EDIT: Please stand by, the upload did not work right – it should be available later tonight.
EDIT 2: Okay – now we are live.
We may do one more small bug fix release (e.g. a 10.12) before we get to 10.20, which will introduce 64-bit and have a longer beta period. 64-bit porting is going well and I am happy with the progress so far.
What About WED?
A number of WED users have reported a really nasty bug: while mousing around, WED would put up a fatal error dialog box and quit, with work lost. This bug sucks, and what was worse, no one could find a good procedure to reproduce it. The steps appeared to be “do some work, get a lot of changes ready, click, lose work.”
That is, until now; the other day Chris round the reproduction case. I now have a fix, so I will try to cut a WED 1.2 beta 2 in the next week.
First, for the plugin authors: I am hoping to start a 10.20 beta for 64-bit in weeks, not months. I don’t know how many weeks it will be – there’s a huge potential for variation, but if you maintain a plugin, be aware that you’ll be able to actually test your plugin against a 64-bit X-Plane this year.
At this point we have launched X-Plane, 64-bit-style on all three operating systems. That doesn’t make it beta-ready but we have a heartbeat.
X-Plane 10.10 is final, and we may cut a small bug-fix patch (10.11) before we go into 64-bit. We have a handful of lower-priority bug fixes that we kept out of 10.10 for stability that we need to release at some point; the plan hasn’t been finalized.
Mac and 64-Bit: Not That Fast
So once I had a 64-bit X-Plane running on my machine, I did the obvious thing: crank the settings through the roof and see what happens. And I can now report the results:
Very low framerate.
The problem is: my machine was very close to maxed out with 32-bits. When I was able to crank the settings beyond where I could before, I simply overloaded it. Too many objects, too many textures, too many vertices, too much stuff. It’s a 2008 Mac Pro with a 4870 – not a spring chicken.
I mention this now because:
- The overwhelming majority of users telling us they want 64 bits are Mac users. On 64-bit Windows OSes, X-Plane has significantly more address space headroom, so you have to push the sim a lot farther to run out of memory.
- Macs just aren’t that fast. You either have a laptop (highly constrained by the need to be power-efficient) or an iMac (power constraints and no update for over a year) or a Mac Pro (with the best graphics card two generations old and no real CPU update in a while).
In other words, my Mac may be older and slower, but the very fastest ones aren’t that fast. There is no Mac equivalent right now to an Ivy Bridge i5 or i7 with a GeForce 680.
So while you may be hitting address space limits and crashing on your Mac right now, you may not have that much hardware budget left over, and it may be a short trip from 64-bits to finding you’ve simply maxed out your hardware.
It’s All About the Watts
One last thought on Macs falling behind Windows gaming machines: while this used to be a function of technology it’s really become a race with only two factors: watts and time.
- The older a model gets, the farther behind the curve it is. So the Mac Pro is really behind due to being age constrained; if they update it with a current-gen desktop GPU and an Ivy-Bridge based Xeon CPU it won’t be cheap, but it will be competitive. For desktops the big issue is one of cost: you can get the latest mobo at any time for Windows and a game machine is significantly less than a Mac Pro.
- Watts: how much GPU power you get is a function of the power budget of the card, and both the iMac and laptops are constrained relative to desktop machines. The new Retina-Book MacBook Pros are nice, new, top of the line laptops, but they are also using the GeForce 650M, a decision to trade off some GPU power for battery life, heat dissipation, etc. There will inevitably be an AlienWare laptop that ways 12 lbs, burns through its battery in five minutes, but ships with a bigger mobile GPU for better performance. I’d rather use the lighter laptop, but my concern is traveling with my work, not flying.
My point here is: these two factors (revision time for models and power use profile) are unlikely to change any time soon – they are fundamental to Apple’s business model. So Mac users, on average you are never going to have the same performance options as your Windows brethren.
