As some of you know, it was my plan to end-of-life the AC3D exporter for X-Plane; the decision was based on having limited resources to develop exporters.
The good news is: Andy Colebourne of Inivis has taken over development of the plugin, and has released the latest version of it – you can find it here. I am linking our download page to his forum link for easy access.
The AC3D plugin shares OBJ import/export code with WED and the rest of the Laminar Research C++ tools; my goal is to not break Andy’s work when working on WED. We use this code not only for WED object preview and the AC3D plugin, but for internal tools that pack up art for the iPhone version of X-Plane, so hopefully there will be some good leverage between the AC3D plugin and other Laminar scenery tools.
The first beta for World Editor 1.5 is now available to download.
This version features numerous bug fixes, along with major improvements to make editing airports easier and faster by providing more visual clues. It’s also the first 64-bit version of WED!
Some highlights of this version include:
You can see the full list of bug fixes, improvements and new features in the README.WorldEditor file found in your downloaded WED folder.
Please try the latest version as soon as you can and let us know if you find any bugs by filing a report on the Scenery Tools tab of the Gateway (not the desktop bug reporter for once!).
That’s the new taxi sign editor in WED.
Signs in the hierarchy are shown as a rendered preview of what the sign will look like.
When you click on it (as if it was text) you get a WYSIWYG editor with two “text” fields where signs can be directly typed, and a palette where signs can be added by clicking.
X-Plane 10.45 fixed one of the big “ecosystem” problems with the global airports by excluding airport objects by ICAO code. This makes the global airports much less likely to conflict with custom scenery, even if that custom scenery lacks exclusion zones.
For 10.50, I am looking at another change to how we manage airports that should help gateway authors: per airport control of flattening.
As of X-Plane 10.45, airport flattening is a user setting; a user can pick to have all airports flat, or all airports follow the terrain contours.
This is a rather silly setup. One size does not fit all for airport flattening, and the author of the airport is more likely to know how the airport will look with/without flattening than a user who is flying to that airport. It certainly isn’t efficient to have everyone in the X-Plane community set flattening individually without authors being able to set up their airport the way they want.
From what I’ve seen, there are two legitimate uses of airport flattening.
- To fix bugs in the underlying terrain, e.g. if the DSF is screwed up, then the airport may need to flatten it to make the airport usable at all. We want this to be the exception, not the rule. (X-Plane 10 is more buggy than past X-Plane releases WRT bumpy runways, so this may sound funny right now, but overall we aim to have users be able to fly with non-flat runways.)
- To accommodate highly customized airports where the 3-d models depend on a flat surface.
In X-Plane 10.50, it should be possible (using a new version of WED) to mark an airport as “always flatten”. The expected usage is:
- Users leave “runways follow terrain contours” on, all of the time.
- Authors mark individual airports as needing flattening, e.g. to fix bugs or accommodate custom scenery.
There is one use case that isn’t handled well by this: if an airport needs different flattening based on different base meshes, there is no way to tag that in the airport. But I view this as a fairly difficult problem to solve with existing technology – we would need a 2-d grid matching every custom version of an airport against every version of the mesh ever to exist.
Fixing the Mesh
Please note that this feature is not a replacement for being able to customize mesh elevation at a local airport from an overlay. “Mesh patching” is what we want to be able to do in the long term, but for X-Plane’s engine, it also means a lot of complicated internal changes to how the rendering engine works; customizing flattening is something I can ship now in 10.50 that at least helps.
Flattening is also not a replacement for fixing bugs in the base meshes themselves; one trend I have observed over my decade+ working on X-Plane (!) is that the accuracy of source data keeps getting better. Ten years ago it would be silly to say “how about if everyone just uses the real world values for their scenery and it will just work”. At this point that’s not actually a pipe dream, it is often completely manageable. So my hope is that someday we can reach a point where the terrain is just accurate and everyone uses it.
As of right now I have this code working in X-Plane but I do not have a build of WED that supports it. We are still in the middle of working on 10.50 apt.dat features, so I’m hoping to post something on that soon.
Ondrej has posted a public thread about the new version of the Blender 2.7 exporter here.
The 3.3 release is the first release where we (Laminar Research) have worked closely with Ondrej to build what we hope will be one of the future cornerstones of modeling for X-Plane.
If you are currently using Blender 2.49 or AC3D, I expect that 2.73 will provide the best way forward for modeling in X-Plane.
New Stuff
The new release has a few major features, all aiming to create a new modern work-flow:
- All animation bugs are fixed. (At least,we think – if you find one, please report it ASAP!) This means animation is WYSIWYG. Armatures are supported for animation as well as animated data block containers.
- The exporter understands all modern OBJ features, including ones scheduled for release in X-Plane 10.50.
- Instancing is handled via a single option with exporter validation, so you don’t have to know how instancing works to get instancing.
- The material rules are validated, and materials are found automatically; you can simply model as you want and Blender will do its best to export it or tell you if there is a problem.
- Specular and Normal maps are ‘merged’ together from two separate sources. This lets you set specularity and normal maps in separate material channels in Blender for a more WYSIWYG approach. It also means no more messing with Photoshop to combine these layers.
The goal of many of these is to hide the idiosyncrasies of X-Plane’s modeling format and provide a cleaner, more artist-friendly view of modeling.
Regarding instancing: the model we have adopted is the one we used internally on the 2.49 exporter: you (the artist) tag an export as either “instanced” or not.
- If instancing is on, the exporter writes only data to the OBJ that will be hardware-instanced by X-Plane. If you do something that cannot be instanced, like animation, you get an export error telling you what you have to remove.
- If instancing is off, the exporter writes the object normally.
The win here is that you don’t have to know the (complicated) instancing rules; set instancing for the scenery objects you need to make fast in bulk (e.g. a luggage cart, a house, something that will be used many times in a small area) and you’ll get optimal output.
Optimization – Coming Soon
The goal of the 3.3 release was to set up an environment where authors could work using the new work flow. We have not finished optimizing the OBJ output.
If you are using the existing 2.7 exporter for airplane work, the output should be similar in performance. If you are using the 2.49 exporter, the new output is (like the current 2.7 exporter) not as well optimized.
In a future update, we will tighten up the generated OBJ code; this should not affect anyone other than producing better OBJs.
Compatibility
The new exporter should be fully work-flow compatible with the previous Blender 2.7 exporter; if you find your existing project does not work, please file a bug!
Our plan is to create a migration tool for Blender 2.49 projects; this will forward-convert the datarefs on annotations, manipulators, and object properties from 2.49 to 2.73 format. This lets authors using 2.49 move forward to 2.73 and have a migration path for existing content. (This work is not started yet, but is planned – we have our own aircraft we need to keep working on.)
MH1212, developer of the Prefab Airports for X-Plane, requested this feature, and it looks like we are going to be able to sneak it into X-Plane 10.45. (If we hit bugs, it might get pushed out to 10.50, but so far things look okay.) The feature is: the ability to exclude objects by airport ID without using exclusion zones.
Right now when a custom scenery pack replaces an airport (via apt.dat), the old apt.dat is completely ignored. But the DSF-based overlay objects, facades, etc. are included; the custom scenery pack has to use exclusion zones to kill them off.
With this extension, the DSF-based overlay objects in a scenery pack can act as if they are in the apt.dat file, disappearing when the apt.dat airport is replaced. This means that when you replace an airport (via apt.dat file) not only do the runways go away, but so do the overlay elements.
Here’s the win: we can export our global airports from the X-Plane Scenery Gateway this way, and custom scenery will remove the overlay elements automatically just by replacing the apt.dat, even if no exclusion zones are present.
Here are some details:
- The scheme works if the underlying airport is correctly marked as having its objects and facades “inside” the airport. So unlike exclusion zones, this scheme works if the underlying airport is modified, not the overlaying airport.
- This is a new scheme – no existing scenery already does this; scenery must be re-exported to gain access to this functionality.
- The functionality requires the latest version of X-Plane, but is harmless to old X-Planes – the DSFs will still load.
- Exclusion zones still work and are still recommended; if you are making custom scenery and you are on top of autogen or an old scenery pack that is not modified using this new scheme, only an exclusion zone will save you.
There are two big advantages of this scheme:
- We (Laminar Research) re-export our collection of thousands of airports on a regular basis, so we can tag the entire set of global airports using this new scheme and have them ready for by-airport-ID exclusion very quickly. So this scheme will start to help conflicts immediately.
- The scheme doesn’t require modifying the overlying scenery at all. There are freeware airports that are effectively orphaned – the author cannot be found and the license doesn’t allow the community to modify them*. Since these airports cannot be legally redistributed with exclusion zones, this technique will help.
Once X-Plane has this extension and the global airports are re-exported using it, global airports will fully disappear when any custom scenery pack replaces the airport by apt.dat, even if the custom scenery pack doesn’t have good exclusion zones.
This functionality will be available to third parties in WED 1.5 when it goes beta. In WED 1.5, if an overlay element is in the folder for an airport, it will be excluded when that airport is excluded. If an overlay element is ‘loose’ in the outer level hierarchy, it will not be excluded by airport (but will be affected by other pack’s exclusion zones).
Since gateway airports already have the objects “in” the airport folder, they are already authored to make this scheme work.
If you create your own DSFs using a low level tool like DSF2Text, the DSFLib source code, or something else crazy, I have posted the proposed schema here. That’s a technical link for people tinkering with the DSF format itself, but if you’re in that category, please do ping me to get early test materials. (The new code is also on GitHub.)
Two warnings to custom scenery authors: if you are creating a custom airport scenery pack, especially payware, please read these very carefully:
- This is not an invitation to stop using exclusion zones. There are plenty of scenery packs in the wild that do not have their objects tagged by airport, as well as autogen and all sorts of junk that can be under your payware. If your airport doesn’t have an exclusion zone and it conflicts with another pack, it is your fault. Go add exclusion zones, like I told you to do years ago.
- If a scenery pack from X-Plane Airport Gateway is removed by your pack (by airport ID) then our exclusion zones are removed too. This means that if trees on the runway have been removed by an X-Plane Airport Gateway airport, you will no longer get those exclusion zones for free. You must exclude those trees yourself! (If you put exclusion zones in from point 1 this is a non-issue.) Test your airport with global airports enabled and disabled to make sure your pack is good.
* As a side note I consider it a real problem that airports get uploaded and shared with the community under licenses that don’t allow for abandon-ware to be maintained. It’s clearly the right of the author to use any license you want, but as a community I hope we can encourage freeware authors to use a permissive open license.
WorldEditor 1.4.1 is out – it’s a very small patch to 1.4.0 that increases the strictness of validation to catch a number of errors we found in the last export from the X-Plane Scenery Gateway.
Please upgrade! WED 1.4.1 will become the mandatory version to upload to the X-Plane Scenery Gateway in a few days if we don’t find any major bugs.
64-Bit WED
Last week I completed most of the work to modernize WED and the scenery tools for modern OS X/X-Code, which finally lets us build 64-bit Mac scenery tools. (This also lets anyone compile WED without having to find a Mac from the dark ages.)
WorldEditor 1.4.1 will be the last 32-bit build of WorldEditor. The next update (WED 1.5) will be 64-bit on all three platforms: Mac, Windows and Linux. This transition will also raise WED’s minimum operating system for OS X to OS X 10.7 (Lion).
Having WED be 64-bit on all platforms should solve problems that advanced scenery authors are seeing where WED runs out of memory when editing a custom scenery pack with a lot of textures or reference imagery.
WED 1.5 Features
I am not sure of the feature list for WED 1.5 yet; 1.4.1 has been kept intentionally small to get a quick patch out. (The longer we have a WED that does not properly validate airports, the more weird stuff ends up in the scenery gateway.) Two features are very likely for WED 1.5:
- 64-bit on all operating systems.
- Support for apt.dat file format extensions to coincide with X-Plane 10.50.
My plan is to have WED 1.5 ready for beta before X-Plane 10.50 is ready for beta, so we can test X-Plane’s new capabilities with WED.
We have internal notes on the apt.dat 1050 extensions; I’ll get an RFC posted in the next few days.
We’ve been doing some work behind the scenes to modernize our servers; as part of that, I have moved the scenery tools code to GitHub.
The GitHub public repositories are the official repositories for WED and the other scenery tools that live in the XPTools code base; I push my changes directly there.
using GitHub means much faster clone times (since the server we had the code on was getting long in the tooth), better public browsing, and you can do all of the normal GitHub things that you’d want, like forking, making pull requests, etc.
If you are using our CGIT browsing (at http://dev.x-plane.com/cgit/cgit.cgi/) please switch to GitHub; CGIT will be going away soon. Similarly, if you have the code checked out, you should change your remote’s URL to https://github.com/X-Plane/xptools.git.
I mentioned in a previous post that we are working with Ondrej on a next-generation Blender 2.7 OBJ exporter. I am excited for this work because I believe it will greatly improve the artist interface to X-Plane. Let’s use instancing as an example to explain the idea.
X-Plane 10 will sometimes draw scenery objects using hardware instancing. When an object is drawn with instancing, performance is a lot better than without it; the huge numbers of objects you can see in the autogen scenery are due to instancing. But the conditions for instancing are, to put it mildly, complicated.
- The object has to be attached to a DSF. Well, that’s not crazy.
- There’s a giant pile of things you’re not allowed to do in an instanced object – animation, material attributes, smoke puffs, bla bla bla. (The object has to be simple so that the GPU can draw a big pile of objects without the CPU intervening to handle things like smoke generation on a per-object basis.)
- Buuuuut…not every ATTRibute is a problem. You can use ATTR_hard in an instanced object, no problem! But don’t use ATTR_shiny_rat – that’ll kill instancing.
- But wait, there are these weirdo new attributes like GLOBAL_specular that seem to replace ATTR_shiny_rat that do work with instancing.
If you model, and this seems confusing, it’s because it is. You were never supposed to see this!
File Formats for Machines and Applications For Humans
My view is this: the file formats of X-Plane are meant to be seen by programmers, not artists. When was the last time you had to hand-edit an .acf? You just use Plane-Maker. You don’t hand-edit a PNG file or worry about its internal compression – you use PhotoShop.
OBJ should be the same way – you should export from a 3-d tool and never have to look inside the OBJ itself. This means the OBJ8 format can serve two purposes:
- Help X-Plane load models quickly and draw good looking content and
- Maintain compatibility so older scenery continues to load.
Note that “be really easy to understand in Notepad” is not on the list!
Blender the Transformer
Now here’s the trick: the rules for OBJ that a 3-d modeler can present can be different from the rules that the OBJ8 format itself has. And that is exactly what we are trying to do in the new Blender exporter.
Rather than expose global attributes and non-global attributes and all of the crazy complexity of OBJ8, our strategy with the new exporter is this:
- When you export a scenery object, you specify if you want the object to be instanced or not.
- If the object is not instanced, Blender creates the best OBJ it can for this case.
- If the object is instanced, Blender tries to create the best OBJ it can for this case.
- If the object is supposed to be instanced but you’ve done something that is not instancing-friendly, the exporter tells you, in terms you can understand.
So for example, if you make an animated radar dish that spins and attempt to export with instancing, the exporter can say “you cannot have animation in an instanced object.” Now you, the scenery designer, can make a decision: is it more important to have the radar dish draw really really fast, or is it more important that it spins?
This is the strategy we used for our modifications of Marginal’s 2.49 exporter* – the artist specifies instancing and gets a check that instancing requirements have met. This means instance-tagged objects will be fast. (It is possible to check instancing within the sim with DataRefEditor but it’s not easy, especially for large projects.)
Don’t Expose the Weird
This idea that Blender can transform your work, and thus hide weird X-Plane rules, is not specific to OBJ, not new to X-Plane, and not even specific to X-Plane. When you look at major 3-d game engines (e.g. for combat games and racing games) a ton of code goes into the “art asset pipeline”, transforming the artist’s original work into something already digested for the 3-d engine.
Another example of this is WED. DSFs have an annoying rule that polygons can’t cross DSF boundaries. There are also some even weirder exceptions for a few special cases, introduced in X-Plane 10.20.
WED knows all of these rules; you draw polygons wherever you want and set the export target for the version of X-Plane you want to fly in, and WED figures out the best way to export your work, chopping up polygons as needed.
Shedding Light on Things
Our current work on the Blender exporter focuses on WYSIWYG animation and the material model (which includes instancing support). There is one more area that I think will make a huge difference to authors that we haven’t started working on yet: lighting.
The lighting values for commands like LIGHT_PARAM are, to put it mildly, completely goofy. If you’ve ever tried to read this, you know that even when things are explained, they make no sense.
No author should ever have to write a LIGHT_PARAM. This is a classic case where the exporter should do the transformation for us. What I would like to get into Blender 2.7 is the ability to place a cone light in Blender and have the exporter write the spill light automatically, e.g. filling in the parameters from the position and shape of the light. You light your scene and you see the same thing in X-Plane.
For years we’ve been behind the curve on user experience when it comes to 3-d modeling. I think we’re starting to fix some of those problems in a major way, which should help make the power of X-Plane 10’s rendering engine quite a bit more accessible.
*As a side note, I have no idea what the current 2.49 script status is. We view the 2.7 scripts as the long term way forward, due to Blender 2.49’s not being maintained, so our effort on a truly user-friendly tool is going into 2.7. But we also maintain updates to 2.49 to support our internal team. X-Plane 10.50 will feature a few new OBJ commands, so I will try to post our branch of the 2.49 scripts (which do have this new support) for anyone whose aircraft is in 2.49.
In the long term the 2.7 scripts will feature a migration tool, but I expect we will have a useful 2.7 release (for 2.7 authors) without migration first, and then migration will come later.
Update: the comments form is fixed – sorry about that!
I’ve been working a bit on scenery tools this week. Here’s a road map of some of what’s coming up.
WED and the X-Plane Scenery Gateway
We are working on a release of the latest scenery gateway airports for X-Plane 10.45. Like all releases, we’ve found problems with airports that WED does not detect. So I will try to release a new version of WED with stricter validation soon.
Airport Parking Spots
Austin is working on new code to place static aircraft at unused ramp parking spots. I’l describe this in more detail in another post, but here are some key points:
- As an airport author, you simply place ramp starts, not static aircraft.
- X-Plane will feature some new static aircraft in the update.
- Third parties can further add to the static aircraft and have them be used via the library system.
- There will be a revision to the apt.dat format that stores more data per ramp spot and a WED update to support this new data.
- Since there may be scenery now with static aircraft on top of ramp starts*, we will auto-remove static aircraft from ramp starts in the gateway export as a temporary measure until authors can resolve the situation and post the fixes to the gateway.
- We are not removing the old static aircraft from the library, but we will be hiding them from WED’s library so that authors use ramp starts instead of placing them by hand.
Static aircraft in parking spots will ship next year, not this year, but is planned as a v10 update.
Scenery Tools on OS X
I made significant progress this week porting WED for OS X to 64-bit and modern Mac APIs. This effort basically meant rewriting the OS-level UI code in WED to be new AppKit based calls instead of the old Carbon API.
What this means is that developers will be able to build all of the scenery tools on a modern Mac running Yosemite or El Capitan with X-Code 7. Since Chris has already updated the projects to compiled WED in MSVC, a developer can build all of the common scenery tools from the most widely used open source IDEs.
It also brings us closer to a 64-bit WED on Mac, which is desperately needed since OS X provides the least amount of usable address space to 32-bit apps. I am not sure of the 64-bit status of the Windows WED build; it may need more work on the libraries.
Will It Blend
This might be the most significant scenery tools development: we’ve been working with Ondrej (der-On on Git-Hub) on new features for the Blender 2.7 exporter. This will include:
- Much better animation support. Complete WYSIWYG animation of data blocks and bones, with no work-arounds needed. Bone animations match Blender 2.49 so you can move your projects forward.
- Modernized material support including instancing for a really clean work-flow that takes advantage of X-Plane 10’s features.
- Updates for the latest OBJ features.
- Our plan is to create a migration script to bring 2.49 projects into 2.7, converting the special tags used for an X-Plane 2.49 export into 2.7.
Many of our artists still use 2.49, but there’s no question that 2.49’s days are numbered. It’s a question of when it stops working on OS X, not if. With the migration step, we can move our projects forward without artists having to redo work.
Like the previous 2.7 exporter, this one is open source and will be free, and should be able to export existing 2.7-type projects with no modifications – we are trying to maintain full backward compatibility.
* This situation is slightly silly – if the user picks the ramp start to fly, the user will be on top of a static aircraft.
