Sandy and I are working on a major revision to the plugin SDK (all the old stuff will work, we’re just adding new things) that should be available in X-Plane 9 soon. The new “2.0” SDK APIs include some new functionality for working on scenery.
- Plugins can find the height of the ground at a given location, which is necessary to draw in the 3-d world in a realistic way (e.g. vehicles that drive on the ground in a sloped airport environment).
- Plugins can load and draw OBJ files using X-Plane’s built-in facilities. I’ve posted OBJ drawing code in the past, but this makes things even easier.
- Plugins can lookup virtual paths in the library to find artwork from scenery packages.
This makes a number of scenery-system concepts available to plugins.
I’ve been resisting OBJ-drawing support in the SDK for a while, but a few things changed my mind:
- We’ve moved as much drawing in X-Plane to OBJs as possible and it’s been a big win. A lot of the dynamic elements are OBJs, they’re used in scenery and cockpits and airplanes. Using OBJ files means our artists (who are not programmers) can customize just about every aspect of the sim. So by providing a file format with a rich tool chain to plugins, hopefully we are helping third parties streamline content development.
- With pixel shaders, X-Plane’s 3-d drawing environment has become complex and hard for third parties to safely augment. By encoding drawing at a higher level via pre-built OBJs (which can be animated via plugin-driven datarefs) we can insulate plugins from drawing-environment changes.
If you have content that worked in 864 but does not work in 9, please file a bug! Please do not assume we intentionally changed the file formats to break compatibility…I would rather get the bug report so we can fix X-Plane than have you change your work.
For every author who changes his or her work in response to a bug (an accidental breaking of compatibility), we’ll have dozens of users who find their add-ons not working but not knowing how to fix it themselves.
With that in mind, beta 16 will have two fixes:
- Some panels were showing the default instrument backgrounds over custom panels. This will also be fixed in the next beta.
- Lit customized overlay parts were not showing. This will be fixed in the next beta.
So if you see those bugs, please check your plane against the next beta when it comes out. If you find other problems, please file a bug!
I’ve spent perhaps the last two weeks almost entirely looking at driver-related graphics bugs. I should note that these are often not driver bugs, and the issue of whether a bug is in the sim or a driver is a lot stickier than you’d imagine, but I’ll blog about that later.
In the meantime, where do we stand on scenery development tools? Here’s a rough summary.
General Strategy
The scenery tools are not targeted to X-Plane 8 or 9 – they’ll export content for both (but if you use new V9-only features, your content will be v9 dependent).
There’s been one structural change in v9 that’s worth mentioning: version 9 allows road networks in DSF overlays. This cleans up the scenery model a lot…roads are more like objects and forests than like the base mesh, so now all of the 3-d elements can be in an overlay. This is particularly useful because you might want to adjust the road and objects nearby all at once.
Given that we have no road editing tools right now, I think there is one case where the tools will be v9-centric: I do not think I will ever provide a road-editing tools for DSF base meshes with roads. I do think I will add road editing to the list of overlay features that will eventually end up in WED.
This means that if you want to customize roads, you’ll have to do it in version 9. This is an inevitable outcome of the scenery system structure…replacing the entire base mesh to edit the roads is not a great idea.
WED
WED is in beta 4, and I think it will go final as soon as I have time to make it official. At this point most of the open bugs I get are usability issues and enhancement requests, not real bugs. We’ll get more usability features into WED in the future.
Please don’t send me WED bugs now (or attach them as blog comments). I’ll put out a formal last-call for WED bugs when I have time to dig in.
The future plan for WED is to add DSF overlay editing with all legal overlay types (which is now everything except beaches and base mesh).
AC3D
The AC3D export plugin already supports new v9 OBJ features – key frames and panel regions. These tools are available now!
Tools
The “tools” are the set of other scenery apps: DSF2Text, ObjConverter and ObjView. We’ll be adding two new tools soon:
- DDSTool is a PNG->DDS converter. There are already other programs (many free) for both platforms that can do this conversion, but we wanted something easy and drag & drop when we were doing the in-house conversion, so I coded DDSTool (based on the Squish library). We’ll make it available to authors.
- MeshTool is a base-mesh generation program. We’re still working out the details on this, but basically it will generate DSF base meshes from raw DEM data and polygonal coastlines.
I am restructuring the tools to be entirely command-line-based (except for ObjView) with a single “drag & drop” UI that wraps all of the tools. So if you use the mouse, you’ll use the one drag & drop application to do all your DSF2Text converions, 3DS->OBJ conversions, PNG->DDS conversion, etc. This is mainly to cut down the number of apps I have to build, but it should also declutter workflow a bit.
MeshTool
I’ll post more on MeshTool once we have more details, but here are the main points:
- MeshTool will not have a UI. It will take text files and process them. It is designed to provide a building block on which people can write more advanced tools. In other words, MeshTool gives other programmers our mesh algorithms just like DSF2Text gives other programmers our DSF-packing algorithms.
- MeshTool will not be an editor. It will create NEW meshes from RAW DATA, not from previous meshes. There are a lot of reasons why I do not want the mesh-creation workflow to be “take the global scenery and hack it”. So MeshTool lets you build your own.
- This means you will need your own elevation and coastline data for MeshTool! You can’t use what shipped in the global scenery! However, since the data that ships the global scenery is freely downloadable and widely available, I don’t think this is a problem.
- MeshTool will be able to “burn in” orthophotos into a mesh. This is probably its most important feature. Right now the only way to make orthophoto-based scenery is to use overlay polygons, but the performance of .pol files with a lot of scenery is really poor under version 9. MeshTool will let people build photo-sceneries that take full advantage of the framerate that X-Plane can deliver!
MeshTool should address some of the common issues I’m hearing with orthophoto sceneries (slow performance with a ton of .pol files, no ability to fix the coastlines or remove underlying base terrain, no ability to fix elevation).
MeshTool will create new default landuse terrain where there are no orthophotos or water. This is important because the elevation data can be customized. If you take an existing DSF and somehow edit the elevation points (for example, use DSF2Text and just start editing) and raise a grassy field, you’ll have grass going up a cliff. But if you pass the raw elevation data to MeshTool, it will build a rocky cliff over steep elevation, creating more plausible scenery.
MeshTool should be in the post 9.0 final time-frame, as will all the other tool reposts.
Posted in Tools
by
Ben Supnik |
I’ve posted the first beta of the AC3D X-Plane export plugin version 3.1 on the scenery web site. This plugin contains support for key frames and panel regions, two new version 9 features.
Generally, the scenery tool code is not part of the X-Plane code. There aren’t version 9 and version 8 scenery tools. There will always be one set of scenery tools that let you optionally use the v9 features or not.
An example is the AC3D plugin. Everyone can use the new plugin (and a few small features apply to X-Plane 8 too, like a less sluggish animation UI when editing large objects). If you use key frames, your object will only work with v9 – if you don’t, it will work with v8.
More specifically on key frames: a traditional v8 animated object is really just a key framed object with only two key-frames. So the UI has changed very little – basically where you only had two key frames (points of correspondence between the dataref and animation) in the old plugin, you can now use the “add” button to add more.
I will try to add features to the scenery tools to warn when v9 features are being used, so that if you want your scenery to run on v8 and v9, you can check for v9-only feature usage.
For this blog entry I will try to show how to put together taxiway segments to avoid WED problems.
Here we have a 3-way junction, that is, one taxiway ending with another. First, what not to do:
First we have an overlapped solution. This is not good because overlap is wasteful of graphics card power. We also have the problem that the edge markings will overlap. So in the second and third picture, we “cut away” the horizontal taxiway markings by adding vertices. (In picture two we have overlap, in picture 3 we try to end the taxiway right at the intersection.)
The problem with technique two is: we still have overlap (and imprecise markings). The problem with picure three is that we can’t guarantee that the vertices of the vertical taxiway are exactly “on top of” the horizontal taxiway edge. So you may get something in X-Plane that looks more like this:
(Gap exagerated for clarity.) This is a “slivering” problem. The solution is one that is both efficient and nice looking:
In this situation, I have inserted two vertices in the edge of the horizontal taxiway and then used “snap to vertex” to assure that the corners of the vertical taxiway are exactly the same as the “cut points” in the horizontal one. This gives me a connection that will look clean in x-plane with no cracks or slivering and no overlap. My lines also end at just the right places.
(Hint: once you do this, to move all of the overlapped vertices at once, select them all with the marquee tool and drag.)
Another issue I see sometimes with WED layouts is how to draw the “fillets”, or curved taxiways that touch runways. Because the lines are not as smooth in x-plane as WED, often users are surprised when the approximations of the bezier curves cause an overlap and theere is no pavement.
Note that we’re stuck – we can’t use the technique described above because we can’t insert vertices into the runway. So we have to overlap. Less overlap is better – the overlap here is exaggerated to make things clear.
In this curve, we have a control handle only on the bottom vertex. It will overlap as soon as the control handle reaches the other side of the top line. By pulling the control handle down, we restore the curve.
Here we have two control handles – now it is a question of whether the top control handle crosses the top of the taxiway.
Here is one solution – at the expense of more overlap we can simply give ourselves some breathing room with our bezier curve. However, my preference would be to try to keep the control handles below the taxiway top by a wide margin.
I don’t actually know all of the best ways to make an apt.dat file. The format is new and we haven’t had a ton of time to performance test it. But here are some thoughts:
- One thing I know is…it is best to describe the shape you are using with the fewest number of nodes possible. In other words, let bezier curves do their thing, don’t add a ton of vertices to them. And please don’t add vertices to try to make the curves smoother. The best thing is to leave the layout simple and let X-Plane render it. In the future we will allow the user to set the level of smoothness based on his or her hardware. Adding more nodes will hurt quality – we will be able to pick better nodes at runtime than you can in the apt.dat file.
- Avoid overlapping large areas of pavement. Overlap is bad – it hurts framerate by requiring the graphics card to draw the same pixels over and over. As X-Plane supports more advanced shaders for nicer lighting, this cost will be a lot heavier on users.
- Ignoring overlap, the question of whether to have several smaller chunks of pavement vs. one large one is a difficult question. I would suggest moderation – try not to make too many individual pavement elements, but don’t make the entire element one giant pavement either. You may have to experiment to find the best framerate.
I posted previously in my blog that we use airport boundaries to flatten airports when we make the DSFs, as well as place airport grass. Let me clarify this a bit: there are two stages of flattening that we do.
- The flattening in the DSF is a “soft” flattening – we do not completely flatten the area, but rather we try to remove high frequency spikes in the elevation data, so that the airport maintains its overall shape without being unusable. This flattening process is very precise in that it only happens inside the airport area. Because we decide the mesh when making the DSF, we can shape the mesh to flatten only the airport area.
- If flattening in the sim is enabled, we then do a “hard” flatten of an area including the airport and some surrounding area. This flatten is truly flat and destroys any topography for the airport and a lot of surrounding terrain. Because the mesh is already defined, X-Plane is limited in its choices of where it can choose to flatten or not flatten.
Now that first soft flattening is what we use the airport boundary for. In X-Plane 800 the US DSF flattening worked very poorly. For the 820 flattening it worked well for most cases, but had some very specific bugs. (Typically the confluence of water, airport, and/or a DSF tile in close proximity would cause the three different processors for water, airport, and boundaries to come into conflict.)
I am going to try to further improve the flattening algorithm for the v9 render, with the hope of fixing some of the buggy cases from the 820 render. I am sure there will still be some broken cases – with 14,000+ DSF tiles, one of them is bound to be weird…but my hope is that with each re-render we can get closer to a render where users can run with sloped runways enabled and enjoy realistic non-flat runways.
I don’t like it when overlay scenery depends on the underlying base mesh…I’ve been telling authors for a while now on the lists, “don’t assume the base mesh is the global scenery”, and while this actually has been a safe assumption for version 8, I suspect that once we get v9 out the door I’ll have some time to write mesh-editing tools, at which point the user will have a choice of base meshes.
Roads are currently built in MSL space (that is, absolute elevation on the planet) and thus don’t change when the base mesh changes. In a previous post I discussed the idea of having them be AGL based (draped) so that they would “hug” whatever base mesh you have.
First I should say: this isn’t coming any time soon and won’t be in 9.0. But also, upon closer examination, there are a number of ugly issues just below the surface. Consider all of the things that the global-scenery generator does with roads:
- Road type is selected not only from source road data, but from surrounding conditions, some of which are not obviously determined by the final DSF.
- Road bridging is determined based on nearby roads, water, gradient constraints, and available bridge templates.
- Road type is further influenced by bridging considerations.
- Roads are smoothed to try to build the most even access, “hiding” rapid changes in underlying terrain from a high-speed roadway.
- Powerline tower locations are picked from terrain peak to peak with constrained maximum lengths. (The individual segments also controls the span of wires, so introducing new nodes in the draping process might be undesirable.)
The problem therefore with a draped road is that the decision to bridge (both how high and what kind) are dependent on the underlying terrain.
The eventual solution would be to do all of the road processing in X-Plane while draping – but this implies that the author of a custom road grid doesn’t actually have control over how the roads come out. One advantage of the current system is that total control over the roads is provided in the DSF – given appropriate tools, a modeler could very carefully hand-craft the roads with a lot of detail.
X-Plane 9 will feature key-framed animation for the rotation and translation animations (but not the show-hide animations). Basically:
- You can have more than two key frames.
- Interpolation between them is linear.
- Extrapolation off the ends is linear (use two equal-value key frames to clamp).
Key frames will be available for all OBJs: scenery, cockpit and aircraft parts. I will try to post a spec soon – the implementation is very similar to the RFC I posted a while ago.
(If you read the .tcl code in the AC3D plugin, you may have guessed that this was coming.)
Trigger
Now I know somebody is going to ask me: “will this allow me to make an animation X start when event Y happens in the sim?” The answer is: no! Key frames allow you to map the relationship between an input dataref and your geometry in any way you want. But if the input dataref is not changing, the output position cannot change either.
Example: you can use key frames to make complex things happen as the gear deploys. Gear deployment is a ratio that starts at 0 and slowly changes to 1 as the gear deploys (over the cycle time as set in PlaneMaker). So you can use this to make the wheels move forward first, then down.
But you cannot use key frames in the cockpit to make the gear handle move slowly. The gear handle dataref is really a switch – it instantly changes from 0 to 1. If you set up a nice key framed animation, we will instantly jump from the animation position at 0 to the animation position at 1.
In other words, whether or not you have key frames, the timing of an animation comes from datarefs, and datarefs alone. I am not planning any extensions to the OBJ system any time soon that will let you trigger animations from events. In other words, you can’t synthesize timed sequences from an OBJ. (The purpose of OBJs is to visualize things, not model things.)
If you want to create timing data, write a plugin. If you don’t know how to code, I have seen at least one plugin that lets authors “invent” simple datarefs using a text file.
No Spare Datarefs
And while I’m going off on a tangent about authoring and datarefs, we will not be adding any “spare” datarefs. Again, datarefs should be made with plugins, and non-programmers should use a plugin that creates datarefs from text files. We will not be creating any “unused” datarefs in the sim.
Why not? Think of the integration problems. Even if we make 100, what happens if Joe uses “spare dataref 93” in the cockpit of his great 747 and Fred uses “spare dataref 93” for the windmills turning in his scenery pack? Splat.
The naming convention of datarefs start with the company or author who created it – or “sim” for X-Plane itself. This is intentional – it assures that two independent authors won’t use the same dataref name and create conflicts between add-ons!
When I posted about roads a few days ago, I didn’t realize that people had such, um, “strong” feelings about them. At risk of throwing fuel on the fire, here’s the overall state of the road system in X-Plane:
1. Architecture
For all of the problems with the road system, I think the architecture makes sense: DSFs contain directed networks of roads, graphic resources describe how to draw them as scenery, and cars are generated on that road network dynamically. I don’t see the system fundamentally changing – rather, we’ll have extensions to the rendering definitions, more advanced car behavior, and other additions within the existing framework.
(I think this is true for most v8 scenery technology – we have a lot of room to grow without a huge rewrite.)
2. Default Road Data
This is the weakest part of the road system – the data in Europe comes from horribly old sources (VMAP0) and even the US data (from TIGER) doesn’t contain accurate enough flow information to really draw a nice detailed road grid. I don’t see this getting a lot better any time soon…but I think there are three ways this could get better:
- We’ll keep improving how we process the data in the default scenery. (But to some extent, junk data going in is going to produce poor results no matter what.)
- If we can ship overlay roads in v9, users will be able to customize road data. This would be the source of the best road data possible, at least for the relatively small* area where users customize roads.
- Some one (us or a third party) could build an add-on pack that uses commercial navigation-quality roads. I do not think we will ever have this in the sim itself – the pricing of commercial data makes this impossible. (This is not about LR being cheap – this is about not losing money on every sale of X-Plane.)
3. Road Rendering
I believe that while road rendering leaves a lot to be improved right now, it is appropriate to the level of data we have. That is, we don’t have nicely rendered intersections, but with such weird intersection data in the global scenery, even nicely rendered intersections would look absurd.
Once we have better road data (via better generation, or more likely, user-modified data) we’ll incrementally provide improvements in the way roads look, probably including extensions to the .net format that defines their look.
4. Car Brains
Right now the AI for the generated traffic is very simple, and the way the cars drive is crude. But here again I think we need to crawl before we can walk – without nicely rendered intersections (from well-formed data), there’s not a lot of point in working on this.
Summary
The road system will grow incrementally, first by allowing users to insert customized data, then by providing tools to make that data, and then by improving the way the sim draws and drives on this data. It will be an incremental process, and when it comes to roads, version 9.0 will have some evolutionary steps but nothing revolutionary.
* I think in terms of the whole world (14,000 DSFs), so user-contributed data represents a small fraction of the total scenery system. On the other hand, human beings live in clustered areas, so the numerically small number of contributions we receive will have an outsized benefit by being in “important” areas.
