Sandy and I are working on a major revision to the plugin SDK (all the old stuff will work, we’re just adding new things) that should be available in X-Plane 9 soon. The new “2.0” SDK APIs include some new functionality for working on scenery.
- Plugins can find the height of the ground at a given location, which is necessary to draw in the 3-d world in a realistic way (e.g. vehicles that drive on the ground in a sloped airport environment).
- Plugins can load and draw OBJ files using X-Plane’s built-in facilities. I’ve posted OBJ drawing code in the past, but this makes things even easier.
- Plugins can lookup virtual paths in the library to find artwork from scenery packages.
This makes a number of scenery-system concepts available to plugins.
I’ve been resisting OBJ-drawing support in the SDK for a while, but a few things changed my mind:
- We’ve moved as much drawing in X-Plane to OBJs as possible and it’s been a big win. A lot of the dynamic elements are OBJs, they’re used in scenery and cockpits and airplanes. Using OBJ files means our artists (who are not programmers) can customize just about every aspect of the sim. So by providing a file format with a rich tool chain to plugins, hopefully we are helping third parties streamline content development.
- With pixel shaders, X-Plane’s 3-d drawing environment has become complex and hard for third parties to safely augment. By encoding drawing at a higher level via pre-built OBJs (which can be animated via plugin-driven datarefs) we can insulate plugins from drawing-environment changes.
I’ll try to get it fixed soon….it’s a bit frustrating, because it’s the second time in the last week that I made a change to X-Plane to try to improve performance, tested it on my hardware, then discovered in-field that it helps some machines but screws up a lot of others.
The fog was a screw-up though…it’s broken on the 9600XT and I have one of those.
(I’m not entirely sure what the minimum graphics card for volumetric fog will turn out to be…right now we let you use it no matter how slow it makes the computer, but generally it’s been sort of a performance problem…it’s just a very expensive algorithm that needs some kind of restructuring.)
If you have content that worked in 864 but does not work in 9, please file a bug! Please do not assume we intentionally changed the file formats to break compatibility…I would rather get the bug report so we can fix X-Plane than have you change your work.
For every author who changes his or her work in response to a bug (an accidental breaking of compatibility), we’ll have dozens of users who find their add-ons not working but not knowing how to fix it themselves.
With that in mind, beta 16 will have two fixes:
- Some panels were showing the default instrument backgrounds over custom panels. This will also be fixed in the next beta.
- Lit customized overlay parts were not showing. This will be fixed in the next beta.
So if you see those bugs, please check your plane against the next beta when it comes out. If you find other problems, please file a bug!
X-Plane 8 didn’t care much whether you used ATTR_cockpit in scenery objects or other strange places. It would simply show the cockpit panel texture, and if it hadn’t been updated, you might see an old one, and if it had never been used, maybe you’d see the random (but colorful) contents of memory. Similarly if you could get close enough to another airplane to look in the window, you’d see your own panel, since there is only one panel texture (for the user’s airplane) in the entire scenery system.
This is a bigger problem in X-Plane 9.
- Because the panel texture can be expensive for big panels, we are a lot more aggressive about not setting up the panel texture if we can avoid it. This means that sometimes the texture doesn’t exist at all. This is why in beta 14 you get an error if you do a formation flight having only been in “w” (forward 2-d) view…the panel texture doesn’t yet exist, but the exterior view of the Cessna tow plane uses it.
- With panel regions there can be up to four panel textures, so you can see the potential for anarchy.
- Panel textures aren’t even the same size any more, causing the wrong-panel-in-AI-plane problem to look even weirder than before.
So in beta 15, the panel texture is replaced with a dummy white texture for:
- Any cockpit object for an AI plane.
- Any scenery objects that are illegally using the panel texture.
This prevents crashes and other nasty stuff. If you want to make the panel be visible in your AI plane, consider using LOD to make a non-panel-texture “fake” cockpit image (at a very small res) at farther LODs. My guess is that in normal usage of the sim you’d really have to do something dangerous to get close enough to see the hack.
We did discuss live panels for all planes (for all of about 3 seconds), but the live panel texture in 3-d is so expensive that it’d be prohibitive to most users for even one AI airplane, let alone 20!
If you create plugins, airplanes, or scenery for X-Plane and haven’t tried your add-ons with X-Plane, please do so soon!! It’s much easier for us to fix backward-compatibility problems while we’re still in beta. Beta 14 introduced some bugs (that should be fixed in beta 15 real soon) but I think we’re reaching the point where you can do compatibility testing.
We’re working on a public Linux beta – see here.
We’ve been preaching “one big texture, not lots of little textures” for a while now, and generally speaking, packing a lot of art into one big texture makes life eaiser for X-Plane, because it can draw more triangles at once before it has to tell the card to change what it’s doing. Inside the company we call this the “crayon rule“.
Now the total set of geometry and textures that X-Plane needs to use for one frame is the “working set” – you can think of it as the crayons that you keep out of the box because you need them all the time. And as I said before, if the working set becomes too big, your framerate dies.
Now with large panels we’re seeing a new phenomenon, one of the first cases where the crayon rule might not be true. The reason is due to working set.
When you make an airplane with a large panel in version 9, you can either use ATTR_cockpit, which lets you use the entire panel as a texture, or you can use ATTR_cockpit_region, which will let you use several parts of the panel. Each ATTR_cockpit_region is a texture change, so that’s more crayons. And yet ATTR_cockpit_region is usually faster.
The reason is two-fold:
- You can often use cockpit regions that don’t cover the entire cockpit texture. Large panels are rounded up to 2048 if the are larger than 1024 in any dimension, so the “wasted space” in a 1600×1600 panel is actually quite huge. If you can get away with some smaller regions, your total panel texture area is smaller because there isn’t wasted space due to this rounding, and you can also skip things like Windows. Prepping the panel texure takes time, and it’s done once for lit and once for non-it elements, so it adds up!
- It turns out there are two categories of textures that contribute to the working set: static texures and dynamic ones, and their impact on VRAM is very different. Dynamic textures are much more expensive. The panel texture is dynamic and it’s uncompressed, so it really costs a fortune. (32 MB of VRAM for 1600×1600. That’s not a lot for a static texture but for a dynamic one that’ll kill you.)
Here’s the details on dynamic vs static textures: the OpenGL driver keeps a backup copy of a texture in main memory, so that if it has to purge VRAM (to make room for more stuff) it still has the texture. As it “swaps” textures, the process is to simply send textures as needed from main memory to VRAM. No big deal.
But with a dynamic texture, the texture has been modified in VRAM! So the copy in system memory is old and stale. The graphics card thus must send the texure back to main memory, consuming twice as much bus bandwidth as normal. (To free 16 MB of VRAM and refill it takes 32 MB of transfer, 16 MB to copy the old texture back to system RAM and another 16 to send the new textures to VRAM.) On non-PCIe cards, this back-transfer might be at 1/8th the speed of the transfer to the card, so this is even worse on AGP cards.
Thus the driver does its best to not throw out dynamic textures. And this is why the panel texture is so expensive. That P180 will cause X-Plane to make two 16-MB dynamic texures, and those textures will cause 32 MB of VRAM to basically be off the table. That’s less space for the other textures to swap in and out of. This kind of “permanent allocation” makes the VRAM budget tighter for all other drawing operations.
Given the right combination of large panels, large res, pixel shader effects (which make more dynamic textures), clouds, and FSAA, you can easily get even a 256 MB card to a state where the free space into which static textures are shuffled becomes horribly small, and the framerate just dies.
So the moral of the story is: yes, it can be worth 4 crayons (using panel regions) to avoid the huge cost of dynamic textures from large panels.
As to static textures (regular DDS files) that are 2048×2048 – the jury is still out but my guess is they don’t represent a huge performance problem. As one user pointed out to me, they’re only 2 MB when compressed (maybe more with alpha) so they’re not insanely huge, and they can be swapped out.
Lori and I are about to leave for a New Years Eve ski trip, but before I shut down the laptop for the last time in 2007 I wanted to say: Happy New Years to everyone in the X-Plane community. I had a lot of fun working on X-Plane in 07 and hopefully the sim brought you enjoyment too. I think 2008 is going to be very exciting – version 9 plants the seeds for a lot of interesting new possibilities during the version run.
When I get back I’ll post a bit on panel regions, for which I have the first performance numbers, as well as some of the strange effects FSAA has on the sim. We should have a progress report on Linux soon too.
See you next year!
In X-Plane 8 the global scenery was installed in the “default scenery” folder, which normally contains most of the built-in artwork to make X-Plane (and the global scenery) work.
In the X-Plane 9 beta it is installed in the custom scenery folder. But the sim recognizes it by name and makes sure custom scenery is higher priority than global scenery. This is imperfect but works. In the future we may do something different. Survival tips:
- The log file tells the priority of all scenery packages.
- Don’t rename or move the global scenery – leave it where it was installed.
- Custom scenery will always trump global scenery.
In X-Plane 9, road networks can be in DSF overlays. And in X-Plane 8 or 9, road types can be customized using a road.net file.
Within the road definition file, each road type can contain:
- “Segments” (textured quads that make the roads)
- Objects (placed along the road)
- Wires (for powerlines)
- Car lanes (that vehicles drive along)
The interesting thing is: none of these types are mandatory. So you can use any of these features without the others.
“Cars” are just objects too. So one way to make a vehicle drive around an airport would be:
- Create a road definition file with a new road type. That road type only has a car lane which references your vehicles OBJ model.
- Make an overlay DSF with this new road type, drawing the path the vehicle OBJ will randomly take.
- Package the overlay, .net file, and OBJ all together and you’ve got randomly generated vehicles moving around.
As Marginal pointed out on the .org forums, roads are currently not draped, so using them in overlays is a total PITA. This is something I hope to address in the 9.1 time-frame…draped roads are an important feature, but a little bit too invasive for the v9 betas.
I’ve spent perhaps the last two weeks almost entirely looking at driver-related graphics bugs. I should note that these are often not driver bugs, and the issue of whether a bug is in the sim or a driver is a lot stickier than you’d imagine, but I’ll blog about that later.
In the meantime, where do we stand on scenery development tools? Here’s a rough summary.
General Strategy
The scenery tools are not targeted to X-Plane 8 or 9 – they’ll export content for both (but if you use new V9-only features, your content will be v9 dependent).
There’s been one structural change in v9 that’s worth mentioning: version 9 allows road networks in DSF overlays. This cleans up the scenery model a lot…roads are more like objects and forests than like the base mesh, so now all of the 3-d elements can be in an overlay. This is particularly useful because you might want to adjust the road and objects nearby all at once.
Given that we have no road editing tools right now, I think there is one case where the tools will be v9-centric: I do not think I will ever provide a road-editing tools for DSF base meshes with roads. I do think I will add road editing to the list of overlay features that will eventually end up in WED.
This means that if you want to customize roads, you’ll have to do it in version 9. This is an inevitable outcome of the scenery system structure…replacing the entire base mesh to edit the roads is not a great idea.
WED
WED is in beta 4, and I think it will go final as soon as I have time to make it official. At this point most of the open bugs I get are usability issues and enhancement requests, not real bugs. We’ll get more usability features into WED in the future.
Please don’t send me WED bugs now (or attach them as blog comments). I’ll put out a formal last-call for WED bugs when I have time to dig in.
The future plan for WED is to add DSF overlay editing with all legal overlay types (which is now everything except beaches and base mesh).
AC3D
The AC3D export plugin already supports new v9 OBJ features – key frames and panel regions. These tools are available now!
Tools
The “tools” are the set of other scenery apps: DSF2Text, ObjConverter and ObjView. We’ll be adding two new tools soon:
- DDSTool is a PNG->DDS converter. There are already other programs (many free) for both platforms that can do this conversion, but we wanted something easy and drag & drop when we were doing the in-house conversion, so I coded DDSTool (based on the Squish library). We’ll make it available to authors.
- MeshTool is a base-mesh generation program. We’re still working out the details on this, but basically it will generate DSF base meshes from raw DEM data and polygonal coastlines.
I am restructuring the tools to be entirely command-line-based (except for ObjView) with a single “drag & drop” UI that wraps all of the tools. So if you use the mouse, you’ll use the one drag & drop application to do all your DSF2Text converions, 3DS->OBJ conversions, PNG->DDS conversion, etc. This is mainly to cut down the number of apps I have to build, but it should also declutter workflow a bit.
MeshTool
I’ll post more on MeshTool once we have more details, but here are the main points:
- MeshTool will not have a UI. It will take text files and process them. It is designed to provide a building block on which people can write more advanced tools. In other words, MeshTool gives other programmers our mesh algorithms just like DSF2Text gives other programmers our DSF-packing algorithms.
- MeshTool will not be an editor. It will create NEW meshes from RAW DATA, not from previous meshes. There are a lot of reasons why I do not want the mesh-creation workflow to be “take the global scenery and hack it”. So MeshTool lets you build your own.
- This means you will need your own elevation and coastline data for MeshTool! You can’t use what shipped in the global scenery! However, since the data that ships the global scenery is freely downloadable and widely available, I don’t think this is a problem.
- MeshTool will be able to “burn in” orthophotos into a mesh. This is probably its most important feature. Right now the only way to make orthophoto-based scenery is to use overlay polygons, but the performance of .pol files with a lot of scenery is really poor under version 9. MeshTool will let people build photo-sceneries that take full advantage of the framerate that X-Plane can deliver!
MeshTool should address some of the common issues I’m hearing with orthophoto sceneries (slow performance with a ton of .pol files, no ability to fix the coastlines or remove underlying base terrain, no ability to fix elevation).
MeshTool will create new default landuse terrain where there are no orthophotos or water. This is important because the elevation data can be customized. If you take an existing DSF and somehow edit the elevation points (for example, use DSF2Text and just start editing) and raise a grassy field, you’ll have grass going up a cliff. But if you pass the raw elevation data to MeshTool, it will build a rocky cliff over steep elevation, creating more plausible scenery.
MeshTool should be in the post 9.0 final time-frame, as will all the other tool reposts.
Posted in Tools
by
Ben Supnik |
