The CEF post generated a lot of controversy, and rather than replying directly in the comments (and doing a bunch of copy & paste), I thought I’d collect my responses to the major objections here.
Objection: Nobody wants to browse the web in X-Plane!
Yes and no. I agree that, unless you’re in VR, a full-fledged web browser is totally useless. Nobody wants to be browsing cat pictures while flying. But CEF is really not designed to power a traditional web browser.
Consider this list of a bunch of other applications that nobody wants a web browser in:
Spotify
Adobe Acrobat
Evernote
The Steam client
The Atom text editor
The GitHub desktop client
WhatsApp
What do all those applications have in common? They’re built on CEF, or the closely related Electron project.
My point is this: the primary use case for CEF is not to build a full-fledged browser—it’s to support things like HTML5 user interfaces, which massively lower the bar for creating nice, easy-to-use plugins. It could also support things like displaying PDF charts or other electronic flight bag features—things that are theoretically possible without a webview, but which have such a high barrier to entry that nobody wants to tackle them.
We are always looking for ways to both
get more people involved in creating X-Plane add-ons, and
give existing plugin developers better tools so that they can create things they couldn’t have before.
Doing so makes the X-Plane community stronger, and supports the creation of add-ons that might otherwise not have existed.
Objection: This is a waste of time. What you should be working on is [some other feature].
As I mentioned previously, CEF is being used in X-Plane right now to enable in-app upgrades. This is obviously the most boring non-feature we could ever ship, and I understand why power users (people who might have bought X-Plane 11 on day 1!) are irritated.
The reality is: some features we ship for end-users*, and some we ship to “pay the bills.” Paying the bills is boring, but important: having X-Plane be financially successful ensures that we can continue improving the sim (ahem, in ways that end users actually care about!) for decades to come. In this particular case, we have reason to believe the current purchasing process is convoluted enough that it’s losing us sales. In-app upgrades will increase sales by some (small) percentage, so even though it’s not a direct improvement to the sim for existing users, it’s funding future development.
One more thing worth mentioning on the topic of “I wish you’d do x instead” is that we have a handful of full-time developers now, with people specializing in very different areas, so we tend to have a lot of features in progress at any one time. It’s certainly not the case that, say, Sidney stopped working on improving performance while I was off integrating CEF. 🙂
Objection: I don’t want X-Plane plugins using webviews!
The reason we’re having this discussion is that plugin developers are already using CEF—and they’re doing so in ways that are incompatible with other developers’ implementations.
It’s certainly an option for us to bury our heads in the sand and ignore this—we could force users to choose between, say, installing a Flight Factor plane and installing a fancy new web-based plugin (or using future X-Plane core features). Ultimately, though, this hurts both the end users who would have otherwise chosen “all of the above,” and it hurts the add-on makers who lose sales because users were forced to choose.
We can’t stop add-on makers from using webviews, but we can make sure that doing so doesn’t lead to a compatibility nightmare for end users.
Objection: CEF represents a massive security hole!
This is absolutely a concern for us. Webviews in X-Plane present two major attack surfaces:
The possibility of running untrusted code (e.g., a nefarious site exploits a browser vulnerability to execute harmful code on your machine)
The possibility of sending data somewhere you didn’t want it to go (e.g., a nefarious site masquerades as your bank’s web site, and sends your login info to bad guys)
We’re still in the “exploration” phase here, trying to understand plugin developers’ needs, but there are a lot of ways we could mitigate these threats. A few possibilities, listed from most extreme to least:
Don’t allow remote connections at all; plugin-created webviews could only load local assets (HTML, JS, etc.) that were bundled with the plugin
Disable JavaScript entirely
Require plugins to provide a whitelist of URLs that are deemed safe to send & receive data from
Disable JavaScript loaded from external (or non-whitelisted) domains
Allow user preferences to control the above policies (or even disable webviews entirely)
Objection: CEF will slow the sim down. This is bloatware!
That’s for measurement to decide! In our tests, having CEF displaying static pages while flying didn’t slow down the sim by any amount we could measure. And since CEF doesn’t run on the main X-Plane thread, the OS can schedule CPU-intensive things like page loads around X-Plane’s work. There is definitely some memory overhead involved (a few hundred megabytes), but considering the existing footprint of X-Plane, it’s not a crazy amount.
And, like everything in X-Plane, if you want to tune it to get the last drop of performance, you’d be free to avoid using features & add-ons that depend on webviews.
Objection: Don’t use CEF—You should use WebKit, Gecko, Servo, etc. instead.
I did a lot of research and test implementations (on Windows, I went so far as to use Trident, the underlying engine to Internet Explorer!), and the only webview framework I found that meets X-Plane’s requirements was CEF.
* Aside: What does 11.20 include for end-users? A few highlights from the release notes:
XPLM_MSG_EXITING_VR—Note that when moving back to the 2-D monitor, you’ll also want to reposition the window—if you just change your positioning mode, the window will be offscreen.
I’ve updated the VR sample plugin to take advantage of all the new stuff here, minus the widget API—really, once you enable native widget windows, your widget window becomes “just another XPLM window” as far as the display APIs are concerned.
X-Plane 11.20b5 came out this week – notes here. Beta 5 fixed the rest of our open rendering engine/material bugs, including artifacts on orthophotos. If you are a scenery or aircraft developer, please check your aircraft or scenery against 11.20 beta 5 and make sure there are no rendering changes compared to 11.10. If it looks different than 11.10 in a bad way, we don’t know about it and it’s not going to be fixed if you don’t say something.
Over the last year we’ve been working quite a bit internally to automate testing. We still do a lot of hand-testing (that is, Jennifer flies the sim and then tells us we’re bozos) but we also have extensive infrastructure to script the sim. So we’re looking to automate the process of checking third party scenery for rendering artifacts, so we can catch more rendering bugs more often.
I’ll post once 11.20 is final about how to get your custom scenery into our test. My hope is that with automation, we can catch bugs faster and you (the third party author) can have your add-on tested without having to do the work yourself.
I’m OOTO next week but if things go well, we’ll have a release candidate when I get back.
In the X-Plane 11.20b1, we’re shipping a web browser for the first time. We’re using the Chromium Embedded Framework (CEF)—essentially the same guts as Chrome, wrapped inside X-Plane.
For the time being, it’s being used in one place only*: to support in-app upgrades, so that if you have the demo, you can buy the full version of X-Plane without having to go to the web site. The web view is seamless—you can’t tell by looking at the app that it’s not just part of the native X-Plane user interface.
While its present use is quite limited, if all goes well, we’d like to expand its use elsewhere in the sim—in The Glorious Future™, we could potentially use it to load online charts and the like.
But, there’s a hitch!
There’s not a good way to load two copies of CEF at a time. And, since some plugins (like the ones used in the new Flight Factor A320 Ultimate) depend on a version of CEF provided by a (global) plugin, we have to disable X-Plane’s web browser functionality in the presence of that plugin. This isn’t a big deal right now—after all, if you’ve installed payware, you probably already own the sim anyway—but it will be a shame if, in The Glorious Future™, you have to choose between your payware and core X-Plane functionality.
The situation is even worse than you might expect: CEF absolutely doesn’t support multiple initialization calls in the lifetime of an app—you can’t initialize it, shut it down, then re-initialize it. That means it’s not even possible for plugin authors to be good citizens and “relinquish” CEF to X-Plane, such that you could at least use X-Plane’s browser whenever you’re not using an add-on that depends on it. It’s all or nothing! In fact, you’ll have to uninstall the CEF plugin before X-Plane can use a web view itself.
There are two possible fixes here:
At some point in the future, we provide access to CEF via the plugin SDK. If we did, it would have to be to the C API only—no fancy C++ wrappers for you. Speaking as someone with the memory of writing for the C API fresh in his mind, let me say: this isn’t a whole lot of fun, and it’s rife with the potential for memory leaks. Moreover, this would break any existing addons that depend on CEF—they would be forced to migrate to the SDK version of the API.
We could (theoretically) compile an X-Plane-specific version of CEF that would not conflict with the version plugins are using. This would require renaming all the Objective-C symbols on Mac, and renaming the DLL on Windows. I’ve not investigated this for feasibility, but it’s certainly theoretically possible. This would allow X-Plane’s version of CEF to coexist with (one) plugin-provided copy, albeit at double the CPU and memory cost.
Having us expose CEF via the SDK is not an unequivocal win. CEF is notoriously incompatible between versions—you can more or less guarantee that even minor updates will break compatibility of the API somewhere. That means X-Plane would be stuck at a fixed version of CEF for at least the lifetime of a major version to prevent breaking plugins. X-Plane 11.20 uses CEF release 3202; it would be at least the next major version before we updated CEF to a newer version. There are two major downsides to this:
When X-Plane updates to that newer version, it would break plugins compiled against old versions of the SDK that depend on CEF. That’s a lot more aggressive than our normal deprecation policy, and it makes CEF plugins a potential maintenance headache for plugin devs.
If your add-on really, really needs features from the latest and greatest CEF release, you’re out of luck—wait a couple years (!!) and maybe we’ll update.
So, here’s what I’d like to hear from plugin devs:
Are you currently, or are you planning to use CEF in the future?
Would you be willing to use the C version of the CEF API only?
Would you be able to accept the limitations and potential headaches (outlined above) of X-Plane-determined CEF versions and compatibility?
If you don’t mind telling me a bit about your intended use cases, that’d be very helpful as well!
EDITED TO ADD: There’s a third option here that I didn’t consider: X-Plane could provide a “wrapper” around CEF that offers “just” a browser surface, and simple interfaces like the ability to change the URL programmatically. This has the advantage that we could update CEF regularly without breaking the API (though there’s always the risk that a new version of Chromium would change how it handles your HTML + CSS + JS). The disadvantages are twofold:
a) If a plugin developer really, really needs the full power of the CEF API, they’re out of luck (or we’re back to square one with respect to having to disable core X-Plane functionality in order to support the plugin).
b) New XPLM APIs are a massive tax on our development time. Every time we need to make a change, we have to go test a dozen plugins… then go through an extensive beta… then inevitably hear about a show-stopping bug we introduced three hours after a release goes final. 😀 In contrast, providing a (major-version-stable), transparent copy of CEF costs us very little dev time; time that would otherwise be spent maintaining the XPLM API can be spent on, like, major features.
* Aside: This is actually how we test a lot of new tech in X-Plane: we find a single, relatively out-of-the-way place to make use of it, ship it in a major version update, and wait to see if it blows up. Betas catch a lot of bugs, but not all, so this is a way of hedging our bets when it comes to code that hasn’t been battle-hardened. This is how we worked the bugs out of the user interface framework (Plane Maker’s panel editor was the testbed), the X-Plane 11 particle system (behind the scenes, it was used for some very minor effects in X-Plane 10.51), and more.
After much bug fixing, X-Plane 11.20 beta 4 is out – here are the fixes. This beta fixes Linux performance, a few material rendering bugs for custom aircraft, and a bunch of X-Plane SDK bugs for VR.
And…we have docs on the vrconfig.txt file format. The VR config file is used to add VR-specific data to an aircraft, including:
Hot spots for seats (E.g. where does the pilot sit).
Extra data for manipulators.
Parameters for the yoke in VR.
Please do not ship your add-ons based on 11.20 VR tech – wait for us to go final. All file formats are subject to change during beta!
We are reaching the end of beta, for both WED and X-Plane. We’ll see, based on incoming bugs, whether we need a beta 5 before RC; hopefully WED 1.7.0r3 is a keeper.
We’ve been working on 11.20 beta 4 for a while now – the goal for beta 4 is to fix all of the SDK/authoring bugs and have docs for the new VR file formats. We’re making good progress; beta 4 should be this week, barring an unforeseen bugtastrophy. (Linux users: perf should be fixed with beta 4.)
We received a number of bug reports that beta 3 was showing large numbers of _vrconfig errors with third party aircraft. So this seems like a good time to re-iterate our policy on file format stability and betas.
Don’t ever ship an add-on that uses an undocumented file format. You can’t guess what the fine print is from example aircraft.
File formats are subject to change during beta, and are not stable until we go final. Don’t ship your add-ons against beta file formats.
In our case, the _vrconfig.txt file format changed in beta 3 and will change again in beta 4; as Chris wrote docs, he found issues that he is fixing.
Our priority during beta is not stability of your add-on, it is getting the file format right, so we’re not stuck with a buggy file format for the rest of time.
Once we go final, we’ll put compatibility code in to support the final version shipped in X-Plane when needed.
Similar logic applies to the SDK – it’s fantastic that people are using the latest SDK to test their plugins in VR, but don’t hit the “ship it” button until we’re final – it is always possible that the SDK will change in a way that breaks compatibility during the beta.
X-Plane 11 has had several fantastic updates to the library of lego-brick airport objects that ship with X-Plane. We’ve had new European-style objects, elements for small airports, and the terminal and jetway kits let you build really great looking airports that fit the exact shape of the real-world airport you are modeling.
In that context, I’m pleased to show you a sneak preview of the next set of lego-brick objects for gateway airport authors.
New Static Airplanes
New Static Helicopters
Emergency Vehicles
Luggage Tugs
Jet Blast Screens
You can download a preview if the new lego bricks here.
Release notes detail the bug fixes here. We have another set of bug fixes that will go into beta 4 next week.
We’ve received a number of bug reports from plugin developers trying to use the new plugin APIs. We’ll have another beta next week to address the rest of the open issues, including the ability to (finally) use widget-based UI in VR.
Beta 3 fixes the MFD brightness going crazy in the G1000 C172, and fixes shadow casting on transparent-alpha-textured objects, but we still have an open bug with panel brightness in a Carenado aircraft that looks like a low level rendering bug. 11.20 has a lot of change to the rendering engine to support VR, performance and our migration to Vulkan and Metal, so authors: please be on the lookout for changes to how your models render compared to X-Plane 11.11.
If beta 3 isn’t found to be a smoldering wreckage of poorly thought out C++ over the next day, we’ll post a Steam version of the beta.
It’s like beta 1, but without the missing art asset. 🙂
We’re working on beta 3 now, which will have some fixes for rendering bugs. A few notes:
This is not going to be a typical eight-week beta. Please go check your add-ons this week.
The biggest area of code change for VR that affected other parts of X-Plane was the rendering engine. I’ve fixed 3 rendering bugs where I broke something while restructuring the renderer for VR, and none were surprising. Please check your aircraft or scenery pack for visual defects and let us know!
