Category: Development

Heavy Metal

TL;DR version: we’re not dropping any operating systems; we may add one or more new APIs while continuing to support OpenGL in the future. The new APIs are similar to each other in design.

About two minutes after WWDC announced that Metal (Apple’s new low-level high performance 3-d graphics API) is coming to OS X, I received a bunch of questions about whether we would support it, and what this means for X-Plane. Now that I’ve had time to watch the WWDC Metal videos (and read the Direct3D 12 preliminary documentation and the Mantle programming guide and watch the Vulkan introductory talk) I’ll try to clarify a little bit about Metal, Mantle, Vulkan, Direct3D 12, and what we might do with them someday.

So Many APIs, So Little Time

Recently there has been an explosion of new graphics APIs. Here’s the list. It’s long, so get coffee first:

  • Metal is Apple’s new 3-d graphics API; it shipped on iOS last year and will run on the next major release of OS X. Metal’s API is actually objective C, which means you’re almost certainly not going to see it on a non-Apple platform. Metal runs on iOS devices with an A7 chip or newer; I don’t know what GPUs OS X 10.11 will support.
  • Mantle is AMD’s low level 3-d graphics API. I don’t know what kind of weird state it’s in; drivers have been available for a while and there are shipping games using it, but it’s still a beta or something. Mantle is AMD and Windows specific (but could be ported in theory), and requires a GCN architecture GPU or newer (e.g. a Radeon HD 7nnnn or the new RX290s.)
  • Direct3D 12 is the next version of Direct3D, coming out with Windows 10. I think it’s in beta right now. It is Windows only and I think will run on a mix of DirectX 11 and 12 hardware. (This isn’t that different from Mantle.) I think Direct3D 12 will be available for the X-Box One in the future, but I didn’t investigate consoles carefully.
  • Vulkan is a proposed new cross-platform open low level API from Khronos, the group that manages the OpenGL spec. Vulkan’s rough design goals were announced a few months ago, in a joint announcement that featured developers from AMD, Intel, NVidia, Qualcomm, ARM, and Imagination – that is, almost the entire set of GPU vendors from the mobile and desktop space.  AMD is apparently letting Khronos use Mantle as a starting point, and most of the sample code shown as Vulkan looks like Mantle code with the prefixes renamed from gr to vk. Vulkan would potentially run on Windows, Linux, and Android – it is the only candidate next-gen API for Android. So far Apple has said nothing about Vulkan support that I am aware of. Vulkan is thus simultaneously both the logical successor to OpenGL and the least developed of the four APIs. (The other three are either shipping or in beta.)

It should be noted that we care about all of the operating systems mentioned above:

  • We are working on the Android port of X-Plane 10 Mobile now, so we care about iOS and Android for the mobile space.
  • On desktop, Windows is our majority platform – last I checked we were at least 65% Windows, and this number has been growing.  Windows is also the most heavily used OS where a user can build a true “gaming” machine (that is, a machine that uses a lot of watts to generate a lot of compute power), so having good graphics on Windows really matters.
  • We are not dropping OS X – it’s still a significant platform, and as long as people want to fly X-Plane on OS X, we’ll let them. Mac hardware doesn’t really qualify as gaming hardware – every current Mac is a power-constrained by the standards of core gamers, but the machines are also not slow either – Apple makes viable machines for flight simulation.
  • Linux Desktop support is always the question mark – by sales volume it is a platform we could drop, so we evaluate it each time we ship a new major version basis.  But with the Linux port already done, help from Linux users on platform-specific code, and Linux running the same drivers as Windows, it’s not that hard to keep it working. If we can run the same next-gen API on Windows and Linux, that would be great. We don’t have the resources to do a port to a new graphics API only for Linux.

(In other words, whatever OS you care about, we’re not looking to drop it.)

How Am I Different?

The next question to answer is: how are these new APIs better than OpenGL?  How are they different from OpenGL?  How are they different from each other?

All four new APIs aim to provide higher graphics performance and fewer driver bugs, and they all do so with a few basic architectural changes that make them different from OpenGL:

  • Threading, synchronization and queuing are all explicitly handled by applications with new APIs, rather than having OpenGL do these tasks implicitly without app intervention. This makes it possible to write 3-d code that leverages multicore, and also makes performance more predictable.  (With OpenGL, synchronization is done for you, sometimes not efficiently.)
  • Expensive operations are explicitly identified, so apps can perform them ahead of time or in background threads. This lets an app programmer know that the app’s rendering loop doesn’t contain any expensive (or sometimes-expensive) operations. (With OpenGL, expensive work sometimes just has to happen in the middle of rendering.)
  • The new APIs let applications manage memory with more control than OpenGL. Here Metal differs from the other three APIs: Metal’s memory management is really “a better OpenGL” in terms of design (the driver still does a lot of work, but the rules for memory management are very clear),while Mantle, Vulkan and D3D12 all give applications control at a much lower level – basically with these three, apps manage graphics memory entirely by themselves.

A key observation: these four new APIs are all a lot more like each other than any of them are like OpenGL. If you ask me to write a rendering engine that supports any two of OpenGL, Mantle, Vulkan, Metal, or D3D12, my job is pretty easy until you add OpenGL into the mix. This is because OpenGL is missing fundamental operations possible in all of these new APIs, e.g. preparing a GPU workload on two threads in parallel.

Supporting Multiple APIs

In the future, I think it is possible that X-Plane could support more than one 3-D API. This is not an uncommon thing in the game engine world – lots of game engines support multiple back-end APIs; pretty much any title available for Playstation, Windows and X-Box is already doing this, as are many games ported to the mobile space. Typically the game engine abstracts the rendering API behind some kind of general purpose drawing layer.

To a limited extent, X-Plane already does this, and has been doing it for a while: we support three different rendering paths within the sim (HDR, no-HDR, and no-HDR with volumetric fog disabled). X-Plane 9 supported three paths as well (fixed function, shaders, and the in-development HDR mode, which was not accessible to users, but was in process for at least the last two years of X-Plane 9). Game engines often have to do this when the difference in capability between their high and low-end hardware is extreme.

We’re Not Going to Drop OpenGL

If we ever adopt a next-generation API, I do not think we would drop OpenGL support. All of the new APIs have something in else in common: they don’t run on old hardware. If we want to capture higher performance on DirectX 11+ hardware before we are prepared to drop anything before it, we’ll need to support OpenGL as well.

  • In the mobile space, the iPhone 5 is very capable but can’t run Metal.
  • In the Android world it may be that users have modern hardware and old drivers incapable of running Vulkan (due to carriers not updating their OS).
  • We support Intel GPUs because they are very often the only GPU a user trying X-Plane for the first time has access to, and some of these may not be able to run modern APIs. (For example, a Sandy Bridge i5 is still a nice CPU with a usable embedded GPU, but it’s not a DX11 part.)

Which API Will You Choose?

The short answer is: it’s too soon to tell, and it might not just be one API. The real cost will be supporting any graphics API other than OpenGL. Our preference is for as few APIs as possible covering as many operating systems as possible, so if Vulkan ends up having wide platform coverage, that’ll be great. But if it turns out that we need to go to a few places to get high performance, we may have to swallow that pill.

It’ll Be Done Tuesday, Right?

This isn’t something that I’m going to start coding tomorrow. Vulkan isn’t even released as a spec, OS X 10.11 and Windows 10 haven’t shipped, and there are plenty of things I can do now with the existing OpenGL rendering engine to make X-Plane faster for everybody.

The process of supporting a new API will probably also start with a cleanup exercise designed to limit how much code is talking to OpenGL directly. So this is a long term architectural change in the sim, not something we bang out in a few weeks.

What About Plugins?

Plugin developers may be wondering: how is my plugin going to work if you move to another graphics API? Right now, plugins use OpenGL directly.

I think the answer is that an X-Plane running on a next-generation graphics API may have to also be running OpenGL, just for plugins; the app would have to create an OpenGL environment that looks believable for plugins to render into, then composite the results with X-Plane’s work. This implies having some kind of interop facility as part of the 3-D API. (This almost certainly exists on Metal via IOSurface; I don’t know about the other APIs.) It’s too soon to know what interoperability options will look like between OpenGL and next-gen APIs – that’s something where the devil is in the details.

An OpenGL sandbox for plugins is actually what we planned to do to support a Core-only OpenGL profile on OS X; going forward we may decide to target Metal instead of Core OpenGL; if we’re going to go to the work of sandboxing plugins in an OS X specific way, we might as well get the fastest API possible.

 

Posted in Development by | 26 Comments

When You File a Bug, Be Specific!

X-Plane 10.40 beta 3 should be out if things go well; I’ll post more on the state of the beta when it’s released. In the meantime, it’s been five years since I’ve ranted about this, so:

When you file a bug report, always be specific!

This is not a good bug report:

Radio tuning does not work in any aircraft.

The problem here is the word “any”. Whomever wrote this bug summary tried a particular aircraft in X-Plane (or maybe even two or three), found a bug, and then assumed that the bug applied to all aircraft. Whatever actual aircraft were tested are lost from the bug report.

Here’s what happens when we get a bug report like this:

  1. We read the bug report and find it doesn’t tell us which aircraft had the problem.
  2. Our tester picks one, say, the C-172.
  3. Murphy’s Law kicks in and the C-172 turns out to be an aircraft that does not show the bug.
  4. We email the user back with “cannot reproduce – what aircraft did you use.”

In other words, whomever is triaging this bug has now spent time testing and emailing to get information that should have been in the bug originally. This lengthens the time from when the bug is filed to when we can fix it.

When you file a bug, you know what aircraft you tested – tell us what it is!  If you tested two or three, list all three.  You don’t need to do any more testing work – you just need to include a little bit more information. Our tester and programmers will then go look at the exact aircraft you specified, and we’ll find the problem.

(If you are reading the developer blog, you probably know all of this – I’m really writing this up so we can link to it when sending requests for more information to users.)

Posted in Development by | 21 Comments

Android Status Update – Testers Wanted!

We’ve been quiet on the Android front for X-Plane 10 Mobile but typically around here, quiet is a good thing because it means we’re busy!

We’ve put in a lot of effort and we’re at a point now where we’re going to begin testing the product. More on that in a minute…

First, I’m excited to say that for the first time, our iOS and our Android products will be the same! In the past, Android came around much later than iOS did and the Android operating system and the devices running it were quite a bit behind Apple’s so we had to make our Android product different. This is no longer the case. Google offers all of the same amenities that we’re currently using on iOS. We will have leaderboards, achievements and multiplayer. The product itself will be effectively identical. It’s the same X-Plane Engine, flight model, scenery packages, aircraft etc. There will be no teaser screenshots because…well….it looks exactly the same as all of the iOS screenshots that we’ve already been posting.

Our goal, once the Android version is released, is to keep the iOS and Android products simultaneously up to date. If a new feature is added to one platform, it will be available on the other platform almost immediately (store approval times permitting).

Testing, testing…1,2,3

We’re going to begin testing relatively soon. I have a few odds and ends to tie up before we’ll be ready to go but i’d say in the next week or so we’ll begin our first round. If you want to be considered to be a tester, you can click on the link and submit a request. There’s no guarantee that you’ll get in. We can’t play favorites.

This testing will be slightly different from the norm. Typically we let users Beta test our products to find some bugs early and help us stabilize. With Android however, our goal is to release an early Alpha version to a small number of users and then slowly increase the size of the test group until we’re confident that it’s running properly on a wide range of devices. Then we will expand the group even more and begin Beta testing per usual.

Why are we doing things differently? With iOS, we had one operating system version to worry about (iOS 8) and twelve different devices (every supported iPhone and iPad) running one brand of GPU to test against. Between Ben, Tyler and myself, we essentially own one of everything. On Android, we have three (4.1.X, 4.4.X and 5.X) operating system versions, four major GPU manufacturers and (wait for it….) over six thousand different devices. Needless to say, we do not own one of everything! We need to ease into the testing to find driver issues and other device-specific problems early which is why we’re doing the Alpha test.

We’re mainly focused on testing the Android-specific pieces because 90% of the code is the same as what’s running on iOS which has already been thoroughly tested.

When will it be released?!

As soon as it’s ready! Awwww (sad face), I know you hate that answer…but it’s true. Our goal is to release it as soon as testing shows that it’s stable! This will probably be in a number of weeks, not days but also not likely months.

Posted in Android, Development, Mobile Devices by | 29 Comments

Windows and Scenery Tools

Note: This post is a boring discussion of the state of the scenery tools. If you use MeshTool or you work on the scenery tools code – or would like to modify MeshTool’s code – read this post!  If you just want to know the status of the Oculus Rift or whether we’ve changed X-Planes’ fogging model, a little bit of your soul will die if you read this post.

Last weekend I discovered that MeshTool 3.0 seemed to “just work” for OS X; when I mentioned this on the blog I received a number of “I’d like a copy” emails. When I further emailed with those authors, I found that they all wanted a Windows build.

Unfortunately the Windows build of the scenery tools was in a state of disarray.  The Windows build was until recently based on running mingw and trying to make Windows look like Linux. This has a few problems:

  • Windows is not Linux.
  • Mingw is often a lower-tier priority (or non-priority) for the developers of the libs the scenery tools use.
  • It’s not easy to get a fully functional mingw environment to begin with.

A while ago Ted moved WorldEditor to MSVC.  This was a pretty big win: debugging actual problems with WED is about a billion times easier in MSVC (which has one of the best visual debuggers, um, ever) than trying to use gdb via mingw on Windows.  It also sets WED up to be worked on by actual real developers; if you are a programmer and Windows is your preferred platform, MSVC is mandatory; gcc/mingw/gdb is a foreign country.

So in response to MeshTool for Windows being borked due to mingw/Windows issues, I horse-traded with Chris: I’d do a bunch of his dev tasks if he’d port MeshTool to MSVC.

Will he succeed? I don’t want to say yes until it’s a done thing, but as of yesterday he did have MeshTool running, only to discover that libtiff was missing AdobeDeflate support. But I think his efforts do look promising.

The Plan

So here’s my plan for scenery tools:

  • MSVC will become the supported compiler for all scenery tools: WED, MeshTool, and the various command line tools (DSFTool, DDSTool, etc.).
  • The scenery tools will come with an MSVC .sln/.proj set to support them.
  • We will let mingw support die.  (If you want to make mingw work, send me patches, I’ll take them! But no one involved now has the expertise to keep mingw working, and it is already significantly broken.)
  • Gcc/Makefiles will be the supported platform for Linux, working on modern distros.
  • X-Code 3 will remain the supported compiler on OS X until we have time to do a Yosemite upgrade.  (A Yosemite upgrade is badly needed, but it’s also just a ton of work; I have not had time to even start on this.)
  • “RenderFarm” (the internal DSF tool we use for global scenery) will be the only non-MSVC-ported tool.

For libraries, Mac and Linux will continue to use the “libs” archive that exists now, which is a series of tarballs and a makefile that creates a static library pack for WED/Meshtool to use.  Libs is a GIT submodule.

For libraries, Windows will use a different sub-module that contains source materials and already-compiled Windows binaries for all needed libraries.  Windows users will simply have to get the submodule from GIT and they can immediately compile WED.  (This is already how WED on MSVC works, but we’ll form a submodule so that the main repo doesn’t get huge.)

For libraries, when we get to Yosemite, we may move OS X to a precompiled-binary strategy; we’ll have to see what the state of libraries is.

32-vs-64-bit

Right now the Linux scenery tools are compiled to the native ABI of the machine they are built on; binary builds are built on Ubuntu 12.04 LTS 64-bit, so those are 64-bit builds.

Windows is currently a 32-bit build; I believe Chris is working on both 32 and 64-bit support.

Mac build are limited to 32-bit due to the use of legacy APIs.

The long term direction of the scenery tools is 64-bit only!  At some point the “next” version of the scenery tools may be 64-bit only builds, and this may happen within the X-Plane 10 version run.  The scenery tools are free and not part of X-Plane itself; we don’t consider ourselves to be under any obligation to maintain 32-bit support indefinitely.

For the scenery tools, 64-bit support is a win because it allows the tools to access relatively huge source data sets.  There are global DSFs that currently can only be built on Linux because they require a 64-bit RenderFarm to load all road data without crashing.

Whither Betas?

There’s some restructuring to the various projects’ we’re doing as a result of all of this; I’m going to hold off WorldEditor beta until we’re done.  I’m hoping this will end in “days” – that’s all of the time Chris and I allocated for it. Therefore hopefully:

  • WED 1.4 public beta 1 next week. (I think I said that last week – I forgot to schedule in time to get the flu.)
  • MeshTool 3.0 public beta 1 next week too.  We’ve run a private beta on Mac/Linux and confirmed that MeshTool 3.0 basically works.  We’ll do a Windows build publicly.

We should also have a 10.35 release candidate Real Soon Now™.

Posted in Development, Scenery, Tools by | 9 Comments

An Update on Our Quest for the One Bug Base to Rule Them All

Ben posted a few weeks ago about our quest for “One Bug Base to Rule Them All“—our goal of cleaning up the fact that we had no fewer than five bug bases running, each for different products, with no communication between them.

This situation is understandably confusing—bugs that dealt with the Airport Scenery Gateway wound up in the WED database, bugs filed against WED wound up in the X-Plane database, and so on.

I’m happy to report that we’ve taken the first steps toward unification: today, we have just one place to report issues with:

  • any of the scenery tools (WED, MeshTool, the AC3D plugin, etc.),
  • airports and scenery packs in the Gateway or in X-Plane, and
  • the Scenery Gateway itself.

All you need to do is:

  1. Create a free Scenery Gateway account and log in.
  2. Click the “Report a Problem” link in the navigation at the top of the page.
  3. Use the tabs to select the type of bug you’re reporting: does it deal with an airport/scenery pack, the Gateway site, or the scenery tools?
  4. Fill in the reporting form with as much detail as you can, and hit Report Issue.

In the future, we’ll move to a similar system for the plugin system and X-Plane itself. At that point, we’ll have one page (probably on the main X-Plane site, rather than the Gateway) with a tab for all the things you might need to file a bug for.

Posted in Development, News by | 4 Comments

One Bug Base To Rule Them All

It’s very simple: if you want to file a scenery bug, it goes to the bug report form if it is actually a problem with X-Plane or the implementation of X-Plane’s core libraries.  If the functionality actually needs to go into WED it goes into the public scenery tools code base, but if it’s a problem with the existing airport data, it goes into the X-Plane Airport Gateway bug database. Plugins have their own bug database. Simple, right?

Given the above, I can’t really be annoyed when authors, developers and users file their bugs in the wrong place. We have five bug databases running now, some public, some private, using three different bug database packages on at least three different servers, implemented in three different back-end languages.

The good news is: Tyler is creating the one bug database to end all bug databases.

Now, if you fly X-Plane and you don’t develop add-ons and you don’t create airports at the airport gateway, I won’t fault you if you don’t care at all. The rest of this post is going to be an (even more) boring discussion of bug databases. Go look at pictures on airliners.net; I won’t fault you for not finishing this post.

For the three of you still here: basically we are creating one unified bug database. The bug database is mostly private (to meet our internal development needs) with a front-end with variable access for authors to contribute. Bugs on some products will remain totally public (e.g. scenery tools), some bugs will private, and there may even be bugs where you can only see what you filed, a la Apple’s “Radar” bug filing system.

Gateway Airport Bugs

This all started when Tyler built a bug report system around the X-Plane airport gateway; it lets you use your airport gateway identity to report bugs on gateway airports. Thus a front-end to a real bug database was born. The front-end lets you have a single user log-in for both uploading airports and reporting bugs.

Scenery Tools Bugs

Tyler has now begun the next step: merging the scenery tools bug database into the new bug database. Thus the scenery tools bug database is temporarily closed to new submissions.  All existing bugs will be transferred, but if you don’t have a gateway login you’ll need one to file future bugs.

The scenery tools bugs will remain fully public; the code repository is open source, so the bug base should be as well.

A small number of users have direct access to the scenery tools code repository, e.g. they contributed enough that we gave them their own set of keys. Those users will get direct bug base access as well; however, I think in the long term the front end will include all of the functionality that almost all users will need.

I’m hoping that the migration will be complete within a week or two, and be ready for WorldEditor 1.4 public beta.

Plugin Bugs

The X-plane plugin system has its own bug base; if the scenery tools bug database merge goes well, we’ll merge plugins next.  The plugin bug base is a public bug base, like scenery tools, and will probably remain that way.

X-Plane Bugs

I don’t know what we are going to do with X-Plane bugs.  The current bug report form does not go directly into our bug base, and I continue to say that that is a good thing: way too high of a percentage of the “bug reports” we get on X-Plane itself look like this:

Help!  I just bought X-Plane 10 and it does not work; when I fly I do not see anything!  Please help!

Steps to reproduce: Fly the sim!

If you have experience in software quality assurance, you can understand why I don’t want “bugs” like that piling up directly in the bug database; most of the bug reports have to get forwarded to customer support.

 

Posted in Development, News by | 14 Comments

10.32 Is Out, WED and 10.35 Soon

X-Plane 10.32 is now final – you’ll get an auto-update message when you run X-Plane.

Coming soon:

  • 10.35 – we still have code to put the finishing touches on, but the plan is to release more Gateway airports in 10.35.  This will not be the last gateway release (obviously 🙂 so if your airport was approved too late to make the cut, we’ll try to get the next release out soon.  (Our goal is to keep gateway airports flowing rapidly so that everyone can get the full benefit of everyone else’s work easily.) I am hoping for public beta within a week.
  • WED 1.4 – will include GeoJpeg2000 image support, improved orthophoto overlay creation, and importing directly from the Gateway.  Again, I am hoping for public beta within  week.

If you would like to know the status on the Oculus Rift, I suggest you comment on this post (even though it is off topic), any other post that is still open for editing, and every additional post we make (no matter how off topic) until Philipp posts an update; while he has not responded yet, he did assure me that he will make a statement if he gets at least 275 off-topic comments on the Rift.*

* This last paragraph is completely false, just another case of me being snarky and poorly behaved on the interwebs.

The real number is 374 posts. 😉

Posted in Development, News by | 60 Comments

Editing Gateway Airports (and Bit-Rot)

WorldEditor 1.4 is almost ready for beta, and among its new features (there are several big ones) one is very important to the X-Plane Airport Gateway:

  • WED 1.4 can browse airports on the gateway and download scenery packs directly into the WED workspace.

“Direct download” is really important for a few reasons, some of which might not be obvious:

  1. It saves time. Getting a gateway airport, even if you just want to look at it, is much faster when you don’t have to download the scenery pack, unzip it, install it, then import it into WED. Once you use direct download, you’ll wonder how you lived without it.
  2. It prevents mistakes. We have seen airport submissions where a user clearly downloaded a scenery pack, imported only the apt.dat (but not the DSF files) and then uploaded it. We cannot take scenery packs like that, because they fundamentally remove the 3-d data from the airport.  (A fundamental policy of the gateway is that if you upload a scenery pack and one already exists, yours can’t be worse than the existing one in some way.  We have to always move forward.)
  3. It provides ancestry information.  When you download and then upload directly from the Gateway in WED 1.4, WED provides the scenery pack ID of the original pack as the “parent” of the new pack.  This means that when Julian goes to look at an upload, he can look at the “original” and more rapidly spot problems.  If your pack is the same as the original except for taxi signs, he only needs to inspect taxi signs.
  4. It prevents data loss.  DSF is a slightly lossy data format – that is, if you get your data back out of a DSF file, it won’t be exactly the same as what you put in. (It is like a JPEG image in that regard.)

More on that last point: DSF stands for Distribution Scenery Format – it was meant as a way to make final scenery packs; it was not meant as an interchange format for continuous editing.  So users are constantly importing and exporting DSFs to do work, small rounding errors (“bit rot”) will creep into our 3-d, and features that were perfectly aligned might not be well aligned after 4 or 5 edits.

The internal format for scenery on the gateway is not binary DSF files, so doing round trips to the gateway has much less “bit rot” than importing scenery packs.

Finally, DSFs are tiles; if your airport spans a DSF boundary, all DSF features (polygons, fences, etc.) get split along the DSF boundaries.

So if you import a scenery pack that you downloaded, you’re getting the split version, which is harder to edit and work with.

The internal format of scenery packs on the gateway is not split, and thus you can edit the original in almost the same form as it was uploaded.

WorldEditor 1.4 beta should becoming in “weeks” (and hopefully not that many weeks); I’ll post here when we are ready for beta testers.

Posted in Development, File Formats, Scenery, Tools by | 25 Comments

Try X-Plane 10.32r1

X-Plane 10.32r1 is now available to beta test – to get it, run the updater and click “get new betas”.  Release notes here.

10.32r1 is another small patch aimed at fixing critical bugs. Hopefully soon (E.g. in a week or two) we’ll start a beta of X-Plane 10.35, which will contain new airports from the X-Plane Airport Gateway, as well as smaller feature enhancements that people have asked for.

Right now it looks like fixes for Yosemite will go into X-Plane 10.35 and improved DSF loading and longer DSF visibility will go into X-Plane 10.40 if the code is solid enough.

Posted in Development, News by | 63 Comments

Mac Plugin Developers – You Should Be Using Native Paths!

TL;DR version: if your plugin runs on OS X, you you should be setting the capability “XPLM_USE_NATIVE_PATHS” to 1, like this:

XPLMEnableFeature("XPLM_USE_NATIVE_PATHS",1);

This sets your plugin to use Posix-style file paths on OS X.

I am going through X-Plane looking for APIs that Apple has deprecated and replacing them.  Aliases to custom scenery on other hard drives stopped working in Yosemite because we were using the Alias Manager to resolve the aliases, and the API is deprecated; in Yosemite Apple actually made it stop working.  So now I’m looking to see what other deprecation problems we might be sitting on.*

One thing I noticed in my search is that kCFURLHFSPathStyle is marked deprecated in OS X 10.9.  I don’t know when it will actually stop working, but we’re not supposed to be using it.

And here’s the thing: the only use case we have for it is incredibly silly: if your plugin doesn’t tell us that you can support Posix paths, we’ll convert to HFS paths so that you can then (in your plugin) convert back to Posix paths.  In this use case, both the XPLM and your plugin are using a deprecated API to temporarily convert a file path to a silly format, and then back again.

Why Are Unix Paths Opt-In

The original XPLM API dates back to X-Plane 6 and ran on the classic Mac OS under the HFS file system.  In this environment, all file system paths were HFS paths, e.g. Volume:directory:then:filename.

For a while, X-Plane could run under OS 9 and OS X using the Carbon APIs and CFM file formats; in this environment, the SDK continued to provide HFS file paths to all plugins at all times.

When we introduced the ability for plugins to use the underlying Posix file paths (which makes life much easier for the plugin developer, since Posix paths are what the OS really wants) we had to make it opt-in; a plugin tells us it wants this new thing by setting a new feature.  Plugins that don’t opt in are assumed to be old and are assumed to expect the old convention.

HFS Paths Are Now Obsolete

Here’s the thing: at this point Apple has changed ABIs twice and changed CPU architectures; they have also changed executable formats.  Simply put, no plugin code that runs in X-Plane 9 or 10 can possibly be using HFS file paths directly, because all running plugins are only capable of running on OS X.

But because it was possible to write a plugin that worked both ways (by opting in only when X-Plane was ready) there are still plugins that will run in HFS mode if and only if X-Plane can’t support Posix (e.g. if they are running on X-Plane 9.)

So in order to fully dispose of HFS paths, we need your plugin to start opting in to Posix paths.  Doing so is really easy – it generally involves adding the one line above and deleting your HFS code (which is fun).

When Can You Use Posix Paths

Posix paths in the XPLM are available to plugins starting with X-Plane 10.0.  If your plugin requires X-Plane 10 or the XPLM 2.1, posix paths are always available.

The XPLMEnableFeature APIs are available to plugins with the XPLM API 2.0, starting with X-Plane 9.0.  So if your plugin only runs on X-Plane 9 and 10, you can attempt to set this (because the API is always available).

* Because our minimum OS is OS X 10.6 for X-Plane 10, X-Code normally doesn’t tell us about most deprecations.  To find all of the issues, I temporarily set our minimum to OS X 10.10 just to see the warning list.

Posted in Development, Plugins by | 5 Comments