Before I can describe the way we are planning on handling Jetways in X-Plane 10, I’ll have to describe some of the rendering technology that goes into the lego bricks; the airports are built on a bunch of new version 10 features.
- Global lighting. The airport objects come with lights that cast halos in all of the expected places. Sometimes the light is attached to a building, but there are also tand-alone “light pole” lights. This simplifies placement and arrangement because you don’t need to worry about matching LIT textures on the ground to your objects. You simply place lights and the resulting lights do what they should. If things are too dark, just place more lights.
- Draped geometry in OBJs. In version 10, an OBJ can have draped geometry that sits on sloped ground perfectly. Tom uses this for parking spot markings, etc. To place a parking spot, it’s one click in WED to place the object and you’re done.
- Improved OBJ performance. On machines with new video cards, we use instancing to draw OBJs if they are simple enough (e.g. no animation, no material attributes). This means that the cost of very small, simple objects is much lower than in version 9. Thus you can place a lot of clutter on the apron and it shouldn’t hurt performance too much.
- Attached objects on facades. A .fac facade definition in version 10 can have attached objects, the same way the roads do in version 9. In version 9 we used attached objects to add pillars to road bridges; in version 10 Tom can attach a light pole object to a facade and it will automatically be placed in alignment with the facade section that it matches.
- Custom facade wall selection. in X-Plane 10, you can pick which art asset wall definitions are associated with a given facade. For example, when you place the fence facade in WED, you can pick which section has the gate. When you place a terminal, you can pick which sections have windows and which do not. (The wall selection is made via a popup in WED – WED reads the wall names out of the .fac file.)
- Autogen Point scenes (.agp files). An autogen point scene is a draped footprint texture which is annotated with a mix of vegetation (defined by a .for file), objects, and even facades. Tom builds the .agp in Blender, forming a “mini-scene”. You place an .agp just like an OBJ in WED – you specify its location and heading – point and click. (The file is called autogen point because it is located at a single point in the DSF.)
- The ground tile from an AGP can use a special shader that adds various amounts of grit and other high-res textures; a control texture with primary colors painted into it specifies where the various high-res texture effects take effect.
Autogen point scenes allow Tom to build a building that comes with extra objects (parked cars in the parking lot, a mailbox), facades (a fence around the object) and a draped polygon all in one click. Putting it all together, you can place an autogen scene in one click and get a facade with exact wall types, lots of objects (that run quickly) with instancing, and global lighting shining on the entire situation.
All of this rendering tech is also completely available to third parties – you can make your own .agp art assets or your own facade types with custom walls, etc. Everything that the airport lego brick library uses will be available via text files in your custom scenery pack.
There were a bunch of questions in the comments about the airport lego brick system. We’ve talked about it a bit, but we’ve never really described the project in one place. Tom sent out some pictures of the system as he was testing it, so I’ll repost them and explain what’s going on.
First, the basic idea: for the last few versions, X-Plane 9 has shipped with default airport layouts (built off of the apt.dat file) but no buildings. The apt.dat file is an open source data file managed by Robin Peel; Robin integrates multiple public data sources and users submit their own improvements and changes. The result is high quality airport layouts despite the lack of a good free global data source.
When we were first looking at new features we wanted to put into X-Plane 10, airport buildings were a high priority; we didn’t want the airports to be empty. Thus airport lego bricks were born. The idea is:
- Our art team builds a series of useful art components for airports (“lego bricks”), e.g. terminal buildings, hangers, light fixtures, trucks, etc.
- LR seeds part of the world with some initial buildings. I don’t yet know how many buildings we will put down. When an airport layout is automatically generated by Robin (just a runway and taxiway) we can easily put down buildings, but for a more complex layout a human may have to look at the layout and say “that’s where the hangers go.”
- WorldEditor 1.2 will have features to edit these layouts. (Tom is using WED 1.2 now – in fact, the source is posted. It’s my hope that we can get WED 1.1 out ASAP once X-Plane 10 is available.)
- LR or Robin or someone collects the placement data for these structures and integrates user submissions, as well as layouts that LR builds internally.
- The new data from all sources goes into X-Plane updates so that users get new layouts quickly.
The lego bricks come in forms you already know of: OBJs and facades, as well as a new “autogen point” file which I will explain in another post. Here is a picture of WED 1.1 with a simple airport layout that Tom built.
The green polygons are forests; the gray polygons are facades. In X-Plane 10 and WED 1.2 you can pick the wall types for each wall of a facade, so the walls are now color coded. (When you select the facade, a popup menu has names like “wall with jetway” or “glass wall”.) Placements of OBJs and autogen have real previews from the top down.
It is my hope that building these kinds of scenes will be easier than building the underlying pavement, because most of the airport elements are drag & drop objects, e.g. you just point and click; the facades are simply traced as outlines. To build a layout using these parts you won’t need to know how to use a 3-d editor, hack an OBJ file, texture-map a 3-d mesh, and you don’t have to do your own texturing work.
But what do the results look like?
ff
Not bad for point and click. The airport lego bricks leverage the rendering engine enhancements in version 10 to provide a lot of detail without having to build a custom scenery pack:
- Since X-Plane 10 has global illumination, light fixtures (either attached to a premade object or in one of the dedicated light posts) simply shine down on whatever is nearby. Put an airplane under a light post and the results look correct – no render-baking required.
- The version 10 facade engine is greatly enhanced; that terminal with jetways sticking out is a v10 facade, and it is in full 3-d.
- X-Plane 10 features new shaders that add extra detail to surfaces – that’s where the grit is coming from in the driveway in the third picture.*
- If the user enables shadows, 3-d casts shadows on everything else. Put a car next to a building, and it will be in shadow, but only at certain times of day.
We have not yet built the submission editing and review process, but at this point the plan is to collect and redistribute overlay DSFs – they are the logical container choice for a bunch of OBJ placements. We are not planning a special .dat format for these buildings.
A few other questions that have come up:
How will ATC data be handled?
Traffic flow information (e.g. where the planes taxi) sits inside the apt.dat file and will be collected as part of the regular apt.dat process. Non-airport ATC data will be collected separately; more on that some other time.
Can this be used for custom cities?
No. The scope is strictly airports. We can collect placements for airports because:
- Airports start out truly empty – the DSF generator clears them out. So there are no conflicts with autogen. For cities, we’d have to “coordinate” with the default buildings and roads, which gets complicated fast.
- The number of “things” in an airport is small – maybe hundreds if you really get into detail, compared to hundreds of thousands for a city. So it is practical to completely build an airport.
- Because one person can build an airport, we only have to worry about conflicts between submissions – we don’t have to worry about merging several parts of a city.
- The set of art assets we need for an airport is limited, so we can provide a reasonably complete library.
- Airports are really important to a flight simulator, so it’s worth all of the effort on everyone’s part to do this.
Generally speaking, I would like to increase user involvement in all parts of the scenery process. I get a lot of emails where people ask “how can I change X” or “can I fix this and send it to you”. The scenery system needs to address both highly skilled authors making complex custom payware packages and casual authors who want to do a freeware pack or just contribute to the overall quality of scenery. So we’ll be looking more at how we handle cities and OSM data in the future. For now we have our hands full just getting OSM water and roads into the sim.
Can we use other objects, custom or from OpenSceneryX?
No. The default layouts need to be built entirely from art assets that ship with the sim. We think OpenSceneryX is great, but the goal of the airport lego bricks is to solve a small, specific problem in a self-contained way, so that people can download the sim and immediately see buildings.
How does this relate to custom scenery?
The lego bricks definitely do not make custom scenery unnecessary – they provide a general set of elements for airports, but I think users will continue to want partially or fully customized airports.
A good analogy is the default instruments: because X-Plane ships with over 900 premade instruments, a user can rapidly build a decent looking 2-d panel with no photoshop or dataref work. But to make a panel that truly looks like the real plane (or to work in 3-d) custom work is needed – it’s more effort but the results are better. Airports will be similar: generic layouts will look good, but custom work will look better.
(Could a payware custom scenery pack include generic elements? I don’t know. In the airplane world, if a payware airplane contains default instruments it tends to get poor reviews. But I don’t see why a custom airport couldn’t use our lego bricks for some of the smaller elements like light poles and baggage trucks, if the main buildings were fully customized.)
* It is possible to make grit effects in X-Plane 9 in the same manner that it is done in MSFS: place a translucent overlay polygon with a high-frequency repeating texture on top of the surface you want to make “dirty. Having this capability in a shader in X-Plane 10 improves performance – we can draw one layer of “stuff” and have all of the effects go down at once.
WED 1.1 beta 4 is now posted – see the Scenery Tools wiki for download links. Thanks to Matthias for the patches to remember the hierarchy’s open/close state – I suspect a lot of users will appreciate this detailed touch. Thanks also to the handful of users who tested some early versions of this beta – there are a bunch of big changes in beta 4 that needed a few tries to get right. (If you find bugs in WED or any of the scenery tools, please report them here.)
This will hopefully be the last beta for WED 1.1, setting us up to move on to WED 1.2 with X-Plane 10 editing features; beta 4 contains three areas of improvement that had needed attention.
Correct Preview Order
WED used to preview scenery based on the order of your work in the hierarchy. This isn’t so good because X-Plane has its own draw order based on layer groups. The next beta will do a much better job of showing the final draw order that X-Plane will use, so you aren’t surprised when you view your exported scenery.
Saving Work When the Undo System Blows Up
There has been one data loss bug in WED that has persisted since version 1.0: if the undo system blows up, you get a nasty error (“WorldEditor has hit an error due to a bug. Please report the following to Ben…”) and no chance to save.
The next beta will automatically save your work in a special file (earth.crash.xml) that you can use to get your work back in the event that the undo system blows up.
Why does WED crash like this at all? The problem is that the undo system saves incremental deltas as you edit your project, to save memory (so we can have a gajillion levels of undo). If the book keeping gets screwed up, we are hopelessly borked because with missing deltas we can’t ever get the project back to a state that makes sense. Once we lose deltas, we’re lost. The error dialog box pops up when it does so that we can know that the problem is with the undo system – otherwise we’d just have an inexplicable crash with no idea what went wrong.
When last I checked, there were only one or two users still seeing undo problems – I’ve added additional logging, so if you do see problems please file bugs!
Changing the File Format
WED 1.0 uses a database as its file format. While this seemed like a good idea at the time, it has since proven to be a performance and maintenance problem. The next beta of WED will use an XML file format.
WED 1.1 will still read the old database format (including the format used in previous betas), so you can open your old projects. Projects saved in the new beta will not open in the older betas, but this is no surprise.
For the nerds: when I first started working on WED, I thought that it might be useful for WED files to be databases so that bulk operations could be performed on large quantities of data. Since then I’ve realized that WED isn’t a GIS or spatial database app – rather it’s a modeling tool. The new XML file format is basically a direct output of its internal editing-centric format.
Edit: the original post listed the version as 2.0 – the new WED beta is 1.1, MeshTool is 2.0, and my brain doesn’t work in this heat.
More scenery tools: I have posted a new beta for WorldEditor 1.1:
Like previous betas, this build mostly fixes bugs around import and export, which was the area that caused the most trouble. Please file WED bugs via the scenery tools bug base; I don’t often get to work on WED, but when I do, I can kill off a bunch of bugs at once if there are good reports. Thanks to all of the users who have submitted good bug reports with example scenery packs to help reproduce the problems.
Future WED Versions
Simultaneous to bug fixing on the WED beta, we are adding new features to support X-Plane 10 ATC: WED will be the tool you use to edit taxiway routing diagrams and airport layout flow information for an airport. (This means that you’ll have your physical pavement, your routings, and your custom scenery all in one environment.)
These features are disabled in the WED 1.1 betas, but once version 10 is released and WED 1.1 is final, we should be able to rapidly go to beta on the new features; since we use WED to build our test layouts now, it should be in reasonably good shape.
In the long term, I’d like to see WED provide a visual front-end for MeshTool*, but I won’t let this hold up a release of an ATC-ready WED; so far having “future features” floating around in the code (disabled for release) hasn’t caused any bug reports or stability problems for the existing 1.1 beta candidates.
WED also needs a file format change; my original decision to use an sqlite3 database was the wrong one; when the file format change happens, there will be full backward compatibility for reading old projects, so you’ll be able to bring your old projects forward just by opening them; you won’t have to reimport the DSFs and you won’t lose your groups and other WED-specific information.
* There’s no reason why WED has to be the only front-end for MeshTool, and it may not even be the first one released. One option might be to make a MeshTool export script for qGIS.
First, two quick notes on comments: all comments are moderated, so if you post a comment and it doesn’t appear, please don’t repost. Sometimes Chris and I are both out for a few days, and the comments sit there. Without moderation, WordPress-based blogs get the hell spammed out of them. Chris has also turned off commenting on very old posts. We’ve had a few cases of people digging out old threads to try to jump in to a discussion after the fact, but a lot of what goes onto the blog is time-sensitive – by the time a month is up, a newer post covers the topic in more detail.
Now on to tools: I have posted new release candidates for MeshTool, version 2.0 release candidate 6:
MeshTool has been in a “perpetual release candidate” state for a while. I think this is partly because we don’t eat our own dog food when it comes to MeshTool – the tool shares a lot of code with our default DSF generator but they’re not the same thing. Not surprisingly, the MeshTool code that is specific to MeshTool and not used for global scenery is where the bugs crop up.
So my thanks to those MeshTool users who have patiently sent me bug reports and reproduction cases – I’m hoping that this release candidate is final.
What’s Next for MeshTool
I think MeshTool will go in a few different directions over the next year:
- The scenery tools code is “branched”, with two forks. The older fork targets X-Plane 9, and the newer one targets X-Plane 10. I suspect we’ll eventually have to have two MeshTool versions, 2.xx for X-Plane 9 scenery, and 3.xx for X-Plane 10 scenery.*
- One logical next direction for MeshTool is to add “processing” features. For example, we have code to roughly flatten an airport area – useful when using SRTM data (where concrete runways make false radar returns that turn into tiny hills). This code could be made available from MeshTool.
- The other logical direction is to put a real user interface in front of MeshTool. Just like WED and OE provides graphical ways to place objects (where-as DSF2Text is low level), it would be easier to work with a mesh scenery if you could see your work, and have MeshTool run automatically in the background.
I don’t know what the time frame for future MeshTool work will be; future MeshTool work isn’t necessary to ship X-Plane 10 so it may have to wait a while.
* This is really not ideal, and it is not the way that most other compatibility will work. Wherever possible, I’m trying to make the tools simply work for both X-Plane 9 and 10 in a single tool. It’s less code for me, and more useful for you (because when we need two tools for two sim versions, inevitably one of the two tool versions will be less feature-filled than the other).
X-Farm has been canceled. Sorry.
However, there are some new MeshTool builds. Release candidate three fixes a crash on quit on Windows and fixes the SHAPEFILE_MASK command. Builds are here:
Speaking of Mesh scenery: Oahu’s out!
One user asked me about slow performance with an overlay scenery he created using GMaps. There are three things you need for fast orthophotos in X-Plane, and unfortunately his scenery is missing one.
Use DDS
I’ve written before about DDS. They key points here for orthophoto performance are:
- Since DDS is already compressed, the CPU has to do less work preparing the texture if it is going to be compressed.
- Since DDS is already pre-minified (meaning the smaller versions of the texture are already computed), if you are not running at ‘extreme’ resolution, the sim can simply load a lower res version. With png, X-Plane must load the full size version and scale it down on the CPU.
- Since DDS is already pre-minified, the sim doesn’t have to compute those minified versions of the texture on the CPU.
All of these things lead to much faster texture load times with DDS.
Use LOAD_CENTER
LOAD_CENTER is a directive that can be put in a .pol or .ter file to tell X-Plane at what location the texture needs to be at maximum resolution; X-Plane will reduce the resolution of the texture as you fly away from that point. LOAD_CENTER is important for orthophotos for a few reasons:
- It saves VRAM, since textures that are far away won’t be loaded at full resolution.
- When combined with DDS, it improves load time. Since some textures are loaded far away, they can be loaded at lower resolution, which (per above) is quick for a DDS file – less data, less load time.
Note that LOAD_CENTER causes X-Plane to reload the textures while you fly, so it requires at least one extra core to work well. It’s really important to use DDS with LOAD_CENTER; otherwise that reload time can get expensive.
Use a base mesh, not draped .pol overlays.
This is probably the most important thing: if you want to cover a lot of area with orthophotos, you need to rebuild the base mesh using .ter files, not cover it with .pol files. There are a few problems with using .pol files:
- Draped polygons are only ‘built’ for areas relatively near the airplane. So even under ideal circumstances, they are going to disappear in the far view. This way of using them (only when near the airplane) is a memory savings based on their intended use: for small surface areas like airports.
- Similarly, since the draped polygons are being built and destroyed as you fly, the amount of extra CPU work while flying is quite a bit higher with .pol files than with a base mesh (which only has to page the actual terrain). So a computer that might be fine paging .ter files can get behind in its work for .pol files. (Authors often use .pol files because they are easy to work with – specify a rectangle and X-Plane does the cutting and slicing…well, that work is happening while you fly, burning up CPU power that could be used to page the orthophotos.)
- Since .pol files cover the base mesh, you pay for your mesh twice – once when X-Plane draws the base mesh and once when it covers over it with polygons. This means twice the VRAM used to draw a frame and twice the fill rate.
If you want high performance orthophotos over an area any larger than an airport or down-town, please use .ter files!
One difference between X-Plane 10 development and previous development efforts is that, because we have a fairly large art team (by LR standards), we’ve been developing a lot of the tools for version 10 scenery/airplane development during the main development cycle. In the past, we’ve gotten the sim out, then added the tools later; I think the delay will be a lot shorter this time around, since many of the tools more or less work.
For WED, we have some of the new 10.0-related features controlled by #define; they’ll remain off for WED 1.1 (which basically makes WED a DSF overlay editor for X-Plane 9.0) and then go into a WED 1.2 release. I think the timing will be tight – that is, we’ll get 1.1 finished during or right after X-Plane 10 beta, and then have a development preview of WED 1.2 almost immediately, by exposing the new features we are using internally.
What will be in the next crop of WED features for X-Plane 10:
- ATC taxiway layout editing for the new X-Plane 10 ATC system.
- A few more DSF tricks, like forests that render trees on the polygonal outline instead of the interior.
There are also some 1.2 features that we’ll get eventually that are in rough or partial shape right now:
- Autogen placement (works pretty badly right now).
- Road grid editing (still highly experimental).
WED File Formats
This brings us to WED’s file format. If you’re not a coder, suffice it to say nothing’s going to break…the rest is technical.
WED 1.0 uses a sqlite3 database to persist earth.wed files. This seemed like a good idea at the time – I thought it might be useful to do database-like queries against WED files. Unfortunately, it’s not a good idea.
- WED can’t run against the database, it loads it all into RAM, so the database is wasted.
- Using sqlite3 slows down development time and makes versioning files more complicated.
- The performance of the code that uses sqlite3 is poor.
I really do like sqlite3 as an embedded database, but my use of it is just not well thought out.
So I am planning to replace the sqlite3 database with some other file format. I am planning on doing this before finalizing WED 1.1, to get the change over with before I go any further with the database format.
WED 1.1 will continue to read sqlite3 databases for the purpose of opening legacy projects (both 1.0 and older 1.1 beta projects) but will only save in the new file format.
The new format isn’t yet decided, but the strong contender is a simple XML format that persists each object in the WED hierarchy. This would give us something structured, readable enough to debug, and there are lots of third party tools to work with XML.
If you area third party developer and have any strong opinions on WED’s internal file format, shout at me by email.
This summer Tyler migrated the X-Plane scenery SDK documentation to X-Plane’s main wiki. I just put in a series of redirects, so that the old URLs for the top level pages will point to the various wiki sub-categories. The wiki contains the most recent information now, as well as up-to-date download links, etc.
At some point I will try to replace the individual scenery documents (part of the library.php script) with redirects to the appropriate wiki pages; until then I will leave the old site in place.
I posted a quick tip on how to create fence-like facades in WED. Basically WED doesn’t handle fences and other non-closed polygons very well, but you can work-around this. A future version will address this more completely.
This is my thinking on the WED roadmap:
-
WED 1.1 will relatively ship soon, without cosmetic and usability improvements. At this point the basic bugs (import/export) are fixed, so best to get out a finished, stable build so that people can at least edit overlays.
-
A future version will provide editing ATC layout information for X-Plane 10. This shouldn’t be too hard to get working, as we already have this in WED now so that we can develop test data for the new ATC system.
-
A future version will provide usability enhancements (e.g. previews of facades, etc.). I don’t have a ton of code done for this yet, but it’s important for everyone using WED for pretty much any purpose.
-
A future version will provide road editing since X-Plane 10 can drape roads.
I don’t know what order those 3 feature will come in, but they will all happen relatively soon after version 10 ships I think.
