X-Plane SDK Roadmap | User Interface and Avionics

Time for an upgrade. Ben Supnik discloses the future of X-Plane Panel Rendering and UI SDKs, including some long-awaited requests!

For the last few months, I have been working on our road map for user interface and avionics. This effort is now far enough along that I can share what it looks like.

We have two goals with this roadmap:

1. Improve performance by allowing plugins to create user interface and avionics without OpenGL. OpenGL drawing requires expensive bridging and is impossible to make multi-core, so I see it as a roadblock to faster FPS. We want something that can be faster.
2. Make it easier to create user-interface and avionics. OpenGL is high-level for a graphics API but low-level as an avionics API; plugin developers often have to add libraries on top of it, like ImGui , or use slower alternatives like Cairo.

The roadmap addresses both of these, based on three bits of tech:

1. XPLM Panel Graphics
2. Native ImGui Integration Support
3. Browser-Based Windows

The rest of this post describes these three things and how they fit together.

Panel Graphics: an X-Plane Native Drawing API

Panel graphics is a new set of APIs in the plugin SDK that supports high-level drawing into an avionic or XPLM window. The AP supports:

• Vector graphics (outlined and filled shapes) with anti-aliasing, line width, and stipple, e.g. the things you need to draw avionics.
• True-type font rendering (rendered as MSDFs, so they look good at multiple scales and rotations).
• Texture atlasing – you give us some PNG files and we batch them together and draw textured meshes for you. This is the technology underneath Plane-Maker’s generic instruments.
• Scissors, stencil clipping and 2-D transforms to make complex avionics.
• Hot zones, e.g. the ability to recover touch-screen clicks and act on them.
• The ability to save bits of panel graphics and re-run them for efficiency.

Panel graphics meet our goals because:

1. It is implemented directly by X-Plane using Vulkan, so we have a fast, potentially thread-safe path to the hardware.
2. A bunch of problems are taken care of for you. We will load and render the true-type font into our shaders; you just name your file. We will load your PNG files, atlas them together, DMA them to VRAM, etc.

While panel graphics is not a general-purpose 3D drawing API, it should be better suited to avionics than OpenGL was.

Panel graphics will be an opt-in option for both windows and avionics.

Aside: Why Roll Your Own?

It’s a question you have to ask: why would we create a graphics API when there are already so many out there?

The short answer is: we looked at the other options, and none of them met our needs. We looked at Cairo, Skia, GLES, NanoVG, ImGui and we considered exposing Vulkan directly (but came to our senses in about 15 µsec). We have looked at a lot of stuff.

The biggest problem was integration: to meet our performance needs, we needed a graphics API where we could really get fine-grained control over how it interacted with the rest of our rendering engine. This, for example, was why we rejected Skia – we looked at how it rendered and concluded it would likely have similar integration pain as Zink. (They’re both great technologies – but they’re not meant to be “thin middleware for games” the way ImGui is).

Other APIs had the opposite problem: integration would be great, but they wouldn’t be useful to our users. For example, ImGui contains a “graphics API” in there somewhere, but it’s not the library you want if you want to make avionics. The libraries small enough to integrate well often didn’t have the functionality, and the really big APIs that could do everything were hard to integrate.

Having made it a significant way through implementation, I have no regrets; it’s not trivial to make a full-featured API, but we have most of the tools we need inside X-Plane, and straight implementation is much less hairy work than, for example, debugging Zink.

Native ImGui Support

Dear ImGui GitHub: https://github.com/ocornut/imgui

ImGui has become a de facto way to do UI in X-Plane plugins, for obvious reasons: it’s fully featured, easy to use, and is about 1000x better than the old “widgets” API I created (and then abandoned) in 2004.

To support ImGui natively, the panel graphics API (which can be used in a window as well as avionics) has an API designed to consume and draw ImGui draw lists directly. To upgrade an ImGui plugin to draw natively, a plugin developer would need to:

1. Set the drawing type for their XPLMWindow from OpenGL to Panel Graphics.
2. Replace the OpenGL draw callback that calls glVertexPointer, etc., with two XPLM panel graphics calls to draw.

My expectation is that the change will make the plugin smaller, and we can post sample code showing how to do this.

Aside: Why Not Just Put ImGui Into the SDK

We looked at compiling ImGui directly into the X-Plane SDK – heck, we had it working internally for a while.

The main drawback for this approach is that ImGui really, really, really wants to be a C++ API, and shipping C bindings for it would mean plugins couldn’t use any custom widgets in their ImGui code (because the custom widgets are almost always meant to be compiled against the C++ API).

By leaving ImGui inside plugins and making the actual drawing native, we are leaving more complexity in plugins, but we’re making adoption, versioning and upgrading easier while hitting the thing we care about: performance.

Browser-Based Windows

The third way to make window content will be to use a browser: plugins can specify the contents are a browser than set a URL for the window. We use CEF (which we ship with) to render. Browser-based windows will have some limited communications ability to send JavaScript messages to the browser and receive plugin callbacks from it for integration.

This is one area where I am not sure of the full extent of what we will offer because while we have this working, we have not worked through all of the security implications. One thing I can say is: a plugin window browser is not an appropriate secure context; every key stroke the user can enter will have gone through X-Plane’s event loop, so maybe don’t use a browser-based plugin window to access your crypto wallet.

Aside: What About Your Own UI Framework?

We have our own custom, fully built native-drawing-based UI framework inside X-Plane. (It’s name, “GUI”, is not my most original work.) GUI was born to support the Plane-Maker panel editor, then grew to support the current X-Plane mobile UI, and then grew again to support the X-Plane 11 user interface rewrite.

For a while, we were considering providing a wrapper around our UI, which would provide a look matched to X-Plane’s native UI and native drawing. But there are a few things that would make this not that great:

• The interface isn’t that mature; we add features as we need them (which would be API-disruptive).
• The interface is wide so it would be a lot to wrap (locking us into compatibility issues).
• There isn’t a widely developed ecosystem to support UI using our tools, the way there is for ImGui and browsers.

In fact, my own coworkers complain regularly about developing using our own internal UI (“why the hell is GUI doing this? This sucks, ima rewrite everything using HTML”), which doesn’t bode well as a developer-friendly framework.

We could still potentially add our own native framework as yet another option, but at this point, we are not pursuing it at all; my guess is that ImGui , panel graphics, and browser-based windows will be a good foundation for the future.

Timeframes and Adoption

I do not know what the time frame is for shipping the panel graphics API. What I do expect is that we will have some kind of not-yet-released developer preview for our developer lobby by this summer at the latest. So there are a few situations where, if you are an add-on dev, you might care:

1. If you are starting or designing a new project that uses avionics or has UI, you should consider whether you should be using panel graphics for the avionics, panel graphics, ImGui, or a browser for the UI.
2. If you have an ImGui -based plugin, you should plan to swap over to panel graphics for drawing when possible.
3. If you support an existing product (particularly middle-ware), you should consider whether any of your APIs can be moved over to panel graphics for better performance.

ImGUI is currently scheduled for 12.5.0 (However may be subject to change.)

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

View all posts by Ben Supnik →