TL;DR version: we’re not dropping any operating systems; we may add one or more new APIs while continuing to support OpenGL in the future. The new APIs are similar to each other in design.
About two minutes after WWDC announced that Metal (Apple’s new low-level high performance 3-d graphics API) is coming to OS X, I received a bunch of questions about whether we would support it, and what this means for X-Plane. Now that I’ve had time to watch the WWDC Metal videos (and read the Direct3D 12 preliminary documentation and the Mantle programming guide and watch the Vulkan introductory talk) I’ll try to clarify a little bit about Metal, Mantle, Vulkan, Direct3D 12, and what we might do with them someday.
So Many APIs, So Little Time
Recently there has been an explosion of new graphics APIs. Here’s the list. It’s long, so get coffee first:
- Metal is Apple’s new 3-d graphics API; it shipped on iOS last year and will run on the next major release of OS X. Metal’s API is actually objective C, which means you’re almost certainly not going to see it on a non-Apple platform. Metal runs on iOS devices with an A7 chip or newer; I don’t know what GPUs OS X 10.11 will support.
- Mantle is AMD’s low level 3-d graphics API. I don’t know what kind of weird state it’s in; drivers have been available for a while and there are shipping games using it, but it’s still a beta or something. Mantle is AMD and Windows specific (but could be ported in theory), and requires a GCN architecture GPU or newer (e.g. a Radeon HD 7nnnn or the new RX290s.)
- Direct3D 12 is the next version of Direct3D, coming out with Windows 10. I think it’s in beta right now. It is Windows only and I think will run on a mix of DirectX 11 and 12 hardware. (This isn’t that different from Mantle.) I think Direct3D 12 will be available for the X-Box One in the future, but I didn’t investigate consoles carefully.
- Vulkan is a proposed new cross-platform open low level API from Khronos, the group that manages the OpenGL spec. Vulkan’s rough design goals were announced a few months ago, in a joint announcement that featured developers from AMD, Intel, NVidia, Qualcomm, ARM, and Imagination – that is, almost the entire set of GPU vendors from the mobile and desktop space. AMD is apparently letting Khronos use Mantle as a starting point, and most of the sample code shown as Vulkan looks like Mantle code with the prefixes renamed from gr to vk. Vulkan would potentially run on Windows, Linux, and Android – it is the only candidate next-gen API for Android. So far Apple has said nothing about Vulkan support that I am aware of. Vulkan is thus simultaneously both the logical successor to OpenGL and the least developed of the four APIs. (The other three are either shipping or in beta.)
It should be noted that we care about all of the operating systems mentioned above:
- We are working on the Android port of X-Plane 10 Mobile now, so we care about iOS and Android for the mobile space.
- On desktop, Windows is our majority platform – last I checked we were at least 65% Windows, and this number has been growing. Windows is also the most heavily used OS where a user can build a true “gaming” machine (that is, a machine that uses a lot of watts to generate a lot of compute power), so having good graphics on Windows really matters.
- We are not dropping OS X – it’s still a significant platform, and as long as people want to fly X-Plane on OS X, we’ll let them. Mac hardware doesn’t really qualify as gaming hardware – every current Mac is a power-constrained by the standards of core gamers, but the machines are also not slow either – Apple makes viable machines for flight simulation.
- Linux Desktop support is always the question mark – by sales volume it is a platform we could drop, so we evaluate it each time we ship a new major version basis. But with the Linux port already done, help from Linux users on platform-specific code, and Linux running the same drivers as Windows, it’s not that hard to keep it working. If we can run the same next-gen API on Windows and Linux, that would be great. We don’t have the resources to do a port to a new graphics API only for Linux.
(In other words, whatever OS you care about, we’re not looking to drop it.)
How Am I Different?
The next question to answer is: how are these new APIs better than OpenGL? How are they different from OpenGL? How are they different from each other?
All four new APIs aim to provide higher graphics performance and fewer driver bugs, and they all do so with a few basic architectural changes that make them different from OpenGL:
- Threading, synchronization and queuing are all explicitly handled by applications with new APIs, rather than having OpenGL do these tasks implicitly without app intervention. This makes it possible to write 3-d code that leverages multicore, and also makes performance more predictable. (With OpenGL, synchronization is done for you, sometimes not efficiently.)
- Expensive operations are explicitly identified, so apps can perform them ahead of time or in background threads. This lets an app programmer know that the app’s rendering loop doesn’t contain any expensive (or sometimes-expensive) operations. (With OpenGL, expensive work sometimes just has to happen in the middle of rendering.)
- The new APIs let applications manage memory with more control than OpenGL. Here Metal differs from the other three APIs: Metal’s memory management is really “a better OpenGL” in terms of design (the driver still does a lot of work, but the rules for memory management are very clear),while Mantle, Vulkan and D3D12 all give applications control at a much lower level – basically with these three, apps manage graphics memory entirely by themselves.
A key observation: these four new APIs are all a lot more like each other than any of them are like OpenGL. If you ask me to write a rendering engine that supports any two of OpenGL, Mantle, Vulkan, Metal, or D3D12, my job is pretty easy until you add OpenGL into the mix. This is because OpenGL is missing fundamental operations possible in all of these new APIs, e.g. preparing a GPU workload on two threads in parallel.
Supporting Multiple APIs
In the future, I think it is possible that X-Plane could support more than one 3-D API. This is not an uncommon thing in the game engine world – lots of game engines support multiple back-end APIs; pretty much any title available for Playstation, Windows and X-Box is already doing this, as are many games ported to the mobile space. Typically the game engine abstracts the rendering API behind some kind of general purpose drawing layer.
To a limited extent, X-Plane already does this, and has been doing it for a while: we support three different rendering paths within the sim (HDR, no-HDR, and no-HDR with volumetric fog disabled). X-Plane 9 supported three paths as well (fixed function, shaders, and the in-development HDR mode, which was not accessible to users, but was in process for at least the last two years of X-Plane 9). Game engines often have to do this when the difference in capability between their high and low-end hardware is extreme.
We’re Not Going to Drop OpenGL
If we ever adopt a next-generation API, I do not think we would drop OpenGL support. All of the new APIs have something in else in common: they don’t run on old hardware. If we want to capture higher performance on DirectX 11+ hardware before we are prepared to drop anything before it, we’ll need to support OpenGL as well.
- In the mobile space, the iPhone 5 is very capable but can’t run Metal.
- In the Android world it may be that users have modern hardware and old drivers incapable of running Vulkan (due to carriers not updating their OS).
- We support Intel GPUs because they are very often the only GPU a user trying X-Plane for the first time has access to, and some of these may not be able to run modern APIs. (For example, a Sandy Bridge i5 is still a nice CPU with a usable embedded GPU, but it’s not a DX11 part.)
Which API Will You Choose?
The short answer is: it’s too soon to tell, and it might not just be one API. The real cost will be supporting any graphics API other than OpenGL. Our preference is for as few APIs as possible covering as many operating systems as possible, so if Vulkan ends up having wide platform coverage, that’ll be great. But if it turns out that we need to go to a few places to get high performance, we may have to swallow that pill.
It’ll Be Done Tuesday, Right?
This isn’t something that I’m going to start coding tomorrow. Vulkan isn’t even released as a spec, OS X 10.11 and Windows 10 haven’t shipped, and there are plenty of things I can do now with the existing OpenGL rendering engine to make X-Plane faster for everybody.
The process of supporting a new API will probably also start with a cleanup exercise designed to limit how much code is talking to OpenGL directly. So this is a long term architectural change in the sim, not something we bang out in a few weeks.
What About Plugins?
Plugin developers may be wondering: how is my plugin going to work if you move to another graphics API? Right now, plugins use OpenGL directly.
I think the answer is that an X-Plane running on a next-generation graphics API may have to also be running OpenGL, just for plugins; the app would have to create an OpenGL environment that looks believable for plugins to render into, then composite the results with X-Plane’s work. This implies having some kind of interop facility as part of the 3-D API. (This almost certainly exists on Metal via IOSurface; I don’t know about the other APIs.) It’s too soon to know what interoperability options will look like between OpenGL and next-gen APIs – that’s something where the devil is in the details.
An OpenGL sandbox for plugins is actually what we planned to do to support a Core-only OpenGL profile on OS X; going forward we may decide to target Metal instead of Core OpenGL; if we’re going to go to the work of sandboxing plugins in an OS X specific way, we might as well get the fastest API possible.
Aaaaah! Finally. The API discussion.
People have been pushing along for using Direct3D on Windows for years.
Now next-gen API’s and presumably the death of OpenGL (unlikely to happen for the lifetime of X-Plane 11 though – yes 11 not 10), “pushed” you guys to look around. (I am sure you DID look around all this time, but didn’t even discuss this in public ever before)
It’s also the matter that the favorite platform of LR is Apple (we all know it), but money mainly comes from Windows (and possibly also Android over IOS).
Anyway… good to know you have the API matter in your books and something that will more actively come with X-Plane 11.
If you ask me (which you don’t), I see Vulkan in the future. Mantle will silently die (it’s something from AMD after all :P), Direct3D 12 will go on because “Microsoft can” (you get to be able to make such decision when you are #1 in the marker) and Metal will go on because Apple will push for it for as long as they can. (Apple is like Sony in many things – remember the push for betamax, minidisks, memory sticks etc. – also remember what happened to all these)
In the end I see Vulkan though.
But it matters what API will NVidia support more “natively” in their drivers for their retail cards (unlike Quadros that need to be strong in OpenGL).
(read: What is happening with Direct3D for years)
In any case gooooood that you opened this discussion finally.
but money mainly comes from Windows (and possibly also Android over IOS). Pure speculation. Ever heard of piracy? It is obvious that it is way more prevalent on the poor man’s operating systems.
Apple is like Sony in many things – remember the push for betamax, minidisks, memory sticks etc. – also remember what happened to all these You happen to know which company was first to introduce their computers with USB, ethernet and a CD burner when PCs didn’t have more than a parallel and a COM port? And who later abandoned the Floppy Drive and the DVD drive when they became obsolete while the PC industry was or still is sticking to it?
But I shouldn’t feed a troll who slipped through moderation…
Yes. It is -possible- that the difference in indicated active users between Mac and Windows is entirely due to piracy. But honestly, it’s pretty unlikely – the move in the hardware split to 2:1 windows matches every other internal indication we have in terms of the platform split. Realistically, we have to assume that we’re getting significantly more revenue from Windows than OSX.
Would it help development speed and progress if you guys ever went Windows only (on desktop)?
Mac users (I’m one), can always do Bootcamp for their gaming needs – I find gaming performance better in Windows usually anyhow.
Not really.
Look at it this way: if we switched to Windows only it would speed up X-Plane development indefinitely, because it would immediately stop Austin from doing any work. Austin couldn’t even find the start button on a Windows, thus he would immediately stop changing X-Plane and breaking stuff all the time. Imagine the resources freed up if Austin just stopped changing things 🙂
I moved to X-Plane on a Windows box, but recently completely change my environment to Linux except my 3D workstation (software limit).. But the one major reason was the support for X-Plane on Linux. This was a key factor and since then I am just loving life. X-Plane runs smooth on my Ubuntu Mate box and I could not dream on moving back, even though I will be getting a free Windows 10 package. So if Linux is dropped in distant future, I sure would need to go back to Windows, so fingers crossed on that one.
I for one do hope X-Plane will incorporate new APIs, especially if they would allow new features like the “ever so discussed” tessellation and multi-core GPU’s. But there must be other features out there that could benefit these new APIs also, new functions that would open new possibilities in X-Plane like Seasons and Weather/Water dynamic?
Thank you Ben for highlighting this subject, I for one find them very interesting and I enjoy every post.
Windows user here, and I really believe in the Vulkan project… one (higher-performance) API for Windows + Linux + Android… now that is cool! xD
Now, is there really no hope anymore for Apple to implement Vulkan when the spec gets released? (in the end, they are part of the Khronos group too), then it would make things a lot easier for you! (and, in return, better for us the users, ofc)
That’ll be up to Apple, and impossible for us to know right now.
Curious about things to come:
http://developer.amd.com/community/events/amd-at-gdc-europe-2015/
i dont think changing APIs will help any thing there is some thing deeper in the sim that is a bigger issue
look at GPU usage when flying and youll see it bounce around… the issue is graphics engine is waiting on SOME THING be for rendering the next frame
and there is still issue with things like LOD(or lack there of) and 2d sprites hitting the frame rate harder then they should
lack of more threading in the sim engine it self is also an issue
If you think that the lack of more threading in the sim engine itself is an issue…
…and the engine is built on top of OpenGL…
…and OpenGL places severe restrictions on how you can split work (that references OpenGL across threads), both in terms of API rules and actual implementation issues in the drivers
…and all four new APIs specifically allow an engine to be multicore
Then I don’t know how you can possibly come to the conclusion that changing APIs isn’t part of the solution.
there are TONS of OpenGL games that are use more then one thread (most of the them really)
i really dont think just changing the API is going to help any thing
sim really needs a top to bottom rewrite for modern hardware
64bit only cut lose any thing thats not at lest Shader model 3.0 or better
even keeping in OpenGL there are some thing that could be added like megatexturing and better streaming of scenery data so that it doesnt crash the frame rate
https://en.wikipedia.org/wiki/MegaTexture
but the fact i can not get a solid 60fps at 1920×1200 with out nearly turning every thing off is silly at this point
This is a “I haven’t seen your code but I know you need to rewrite all of it” post – that is, I don’t think it’s something we can really debate. Two general comments:
1. We are not going to rewrite the sim top to bottom. Joel Spolsky wrote a pretty good explanation of why this kind of thing is not a good idea – it says everything I could hope to say, only better.
http://www.joelonsoftware.com/articles/fog0000000069.html
I can only add that I have –lived– this – I have been in a development team (in my time before LR) that tried to rewrite everything top to bottom and ship product, and Joel is spot on.
2. Lots of games ARE using OpenGL from multiple threads, and they are all taking a hit on performance, which is why every next-generation 3-d API is using a different threading model than OpenGL. The threading model is one of the things in OpenGL that is so broken that glNext (e.g. Vulkan) made an incompatible break.
If you read any of the modern, recent advise from Nvidia and AMD, they both say the same exact thing: “Use OpenGL from only one thread.”
Here is why they are saying this: the OpenGL threading rules and object model are written in such a way that the driver has to take a shared lock on some or all of the context for most OpenGL calls. For example:
– Flushed changes to a GL object from another thread are guaranteed to be visible after a bind in the rendering thread. (This means that “bind” is a synchronizing operation.) Annoyingly, bind is also necessary for every change of state in every draw call, meaning -every- bit of state change on every draw call has to take threading and synchronization overhead. That’s why the draw call CPU overhead in the new APIs are so much lower – they explicitly do no synchronization and trust the app not to be stupid, and it makes the code path a lot leaner.
– All OpenGL object classes can be mutable, and they can be mutated from any thread. This means all mutations need to take a lock to protect against mutations in another thread (since mutating an object is not allowed to crash the host app under any circumstances in the GL spec). This combines with…
– All OpenGL object classes are mutated only when bound (if DSA isn’t available) and even with DSA, the ‘namespace’ of objects is a table lookup in the GL context. Binding also looks up that namespace, and object creation mutates it. That means there’s going to be some kind of global lock per object class (e.g. buffer object, texture) that has to be taken every time we bind (lock acquire on draw) to protect against async loader code.
In other words, GL is thread-safe, but it’s not fast when it’s thread safe, because the object and threading model isn’t designed for a multi-core world. The overhead is high enough that the driver guys say “don’t do it”, and in fact, they add a special case in the driver to bypass locking overhead when they know the context is unshared (and therefore synchronization is unnecessary).
3. I agree there are things that can be done now to make X-Plane’s rendering better without a next-gen API. (These things can even be done without a top-down rewrite! 🙂 That’s why we continue to update and modernize the engine.
I just want to be clear here: the next-gen APIs don’t exist because OpenGL is ‘no longer shiny’ – they exist because OpenGL’s abstraction doesn’t fit the hardware and it’s threading model is way too expensive for modern use.
have you looked in to things like MegaTextures at all?
seems like it would do wonders for X-plane 1 massive texture for each 1×1 deg block
Yes, but I don’t think sparse virtual texturing (SVT, of which MegaTexture is I guess related) is very useful for X-Plane because X-Plane isn’t based on a single giant high-res roaming “base terrain texture” (where texture size is a limitation).
X-Plane’s terrain falls into three categories:
– Land class, where repeating the texture with careful rules and heuristics lets us get higher res with less VRAM – that’s a win in terms of real usage of VRAM compared to SVT.
– Various bits of overlay texture that run along vector features with alpha – the logic here is about the same.
– Orthophotos – at the scale we draw orthophotos, the cost of using a set of textures is not at all harming our performance, so SVT isn’t needed.
SVT fits a particular type of tool chain for a particular type of game well, but X-Plane is not one of those games.
Well,
My guess (humble educated one) is that we are not going to see Vulkan on Apple platform.
Especially on iOS where they keep a tight control on how many RAM an app is allowed to use. This is I think directly in line with there profit margin policy. No need to have large amount of RAM and increased fabrication costs if they keep a strict policy on apps RAM management…
Sure we have 2 GB on newer Apple tablets I think and surely on next iPhones but that is still half the memory on many cheaper Android devices now.
So I am not seeing Vukan coming to iOS devices, but that’s just my guess on a reason why that wouldn’t be the case…
On OSX why not, amounts of RAM are much bigger and the profits realised on laptops and workstations is way lower than those on mobile devices. Also, a full fledged modern operaring system like Mavericks, Yosemite or El Capitan (what a strange naming idea) needs more RAM, so…
But even then, why putting Metal on OSX instead of waiting for the availability of Vulkan if not for pushing forward the use of the former?
The main reason is that with Metal as for everything else in Apple ecosystem: they have full control of it.
But maybe my understanding is wrong.
Have a nice day!
David.
I like the user important that the X-Plane 10 there was a real leap forward in graphics, using the latest drivers and other programs neobizatelno Open GL, which developers have stuck, the main thing that in the X-Plane 10 was the highest chart, and the engine first all I want is to have implemented for Windows. Especially in late July release of Windows 10.
Metal for Mac is great news for us who work solely on a Mac and also enjoy X-Plane. In my view the principle issue preventing the platform from operating on par with Windows/Linux has been software, not hardware. Now with the Metal for Mac announcement we Mac users will finally have the potential to fully leverage our hardware. It would be great if we didn’t have to wait for seemingly complex architectural changes in XPlane to make use of this new feature. Unless of course your changes and the Metal for Mac release coincide.
Best Regards
It’s not just software, it’s hardware too. There is no Mac that’s equivalent to putting together a -desktop- PC with a serious power supply. At this point every Mac in the line-up has some kind of nod to being low power, and that means the total amount of GPU you can get is limited.
the total amount of GPU you can get is limitedTrue. But are Macs so far away from the GPU power of an average PC running X-Plane? Maybe it’s time for a hardware survey!
First, the total amount of GPU you can get _is_ limited. Compare the GeForce 780M – that’s the best GPU you could get in the last gen hardware if you maxed out the iMac (and you payed a LOT for that iMac) vs the GTX 780 Ti that you could put in a desktop PC. I have not priced this stuff out, but the current “maxed out” iMac (which is the only way to get the top GPU) starts at $2500; a premium current-gen GPU is typically $600. Anyone who has built a PC will tell you that they can get a ridiculously powerful box for the remaining money. (To match the iMac, they will need to buy a nice high quality monitor…but then they don’t have to re-buy it each time they upgrade.)
On to the numbers:
Core configuration (shaders, texture units, ROPs)
780 Ti 2880:224:48
780 M 1536:128:32
So we’re looking at almost 2x the shader cores and at least 25% more ROPs.
Clocking and watts (core, shader, memory bandwidth, watts):
780M: 823/823/5000 MT/sec, 100 watts
780Ti:876/928/7000 MT/sec, 250 watts
So clearly the Ti is just slurping power – that’s why Apple doesn’t put them in anything they build. You can run a 250 watt GPU in a PC, not a problem.
But let’s look at the final numbers: how does this change in core config, watts and speed turn into real performance?
Gflops/second (a measure of pure compute power:
780M: 2369
780Ti: 5046
Pixel fill rate (gigapixels/second)
780M: 24
780Ti: 42
In other words, the high end desktop GPU has over 2x the compute power and almost 2x the fill rate.
So yes – the Mac is pretty far away — a factor of roughly 2x. (For what it’s worth, this is a LOT better than it used to be – a few generations ago the gap could be as much as 4x for an iMAc — albeit that was back when Mac pros could run real desktop GPUs. Now that the Mac Pro is a low power device, it’s good that the gap from top-end mobile part (in the iMac) to a big desktop card is only about 2x.
What about the averages? My experience with users so far is actually worse for Mac: you really have to burn your wallet to get that top-end iMac. Lots of Mac users are using laptops (much worse performance), older generation iMacs, and non-top-end iMacs. Windows users have not-the-top-end cards sometimes, but they can put a one-gen-off card or one-notch down card in for somewhere between $200 and $300 and stay up to date. The Mac user hast to ride their GPU down until it’s time for a new machine.
Only a hardware survey will tell, but I would be shocked if every Mac user is maxing out their iMacs and updating them every 2 years, while PC users are consistently getting bottom-end cards. So far nothing we’ve seen indicates that.
Ben,
Of course the hardware is not as good. There’s no argument there. The Apple ecosystem is what it is. However with the release of Metal for Mac many hope to see equal performance, independent of OS. So for me there’s no point in discussing hardware, that can’t be changed/fixed by LR. But we can leverage these new software technologies to improve the experience on the Mac platform. And, as a customer, it would be great to see it implemented. If not, I’ll continue to use what I have, no hard feelings. Just a little disappointing 😉
Perhaps you should think about creating a high-level rendering engine which is API independent like some game engine developers have. Make it illegal for plugins and your own code to call directly into OpenGL. Force everything must go through your rendering engine. The engine can be configured to use OpenGL, DirectX, Metal, etc. as required by the platform.
Speaking of Linux support, I’m a Linux/Mac OS X developer. I’ve been thinking about building a high-end Linux box just for running sims like X-Plane and KSP, so I’m definitely hoping you guys won’t drop Linux support. Having written software for Windows years ago (and therefore having been forced to use Windows), I don’t care to repeat the experience.
Such a high level API is exactly what we would do! But that’s good for us, not good for plugins – existing plugins are linked -directly- against OpenGL and just going “you’re all broken” isn’t a great solution.
One can certainly build a proper gaming rig using Linux as OS 🙂 In 2016 I will upgrade my Linux gaming rig with what ever AMD comes with as a new generation GPU. My hope then is for X-Plane to allow using the open source AMD drivers.