X-Plane 940 now supports normal maps on OBJ models (both scenery and airplane). I’ll get more formal docs up once the rest of my office is moved and unpacked but here’s the details for now:
The normal maps are in “blender” format see here. The alpha channel is optional; if it is present, it serves to modulate the level* of specularity. Opaque means full specularity, transparent means none. You can use this feature to make some parts of an object shiny and some dull on a per-pixel level.
Shininess level is modulated by both ATTR_shiny_rat and the alpha channel, so you need ATTR_shiny_rat 1.0 and an opaque alpha channel (or no alpha channel) to see full specularity.
Normal maps are only available for objects and only appear if pixel shaders are on and per-pixel lighting is enabled.
Normal maps should be PNG format, not DDS – they will not be texture compressed because S3TC compression tends to kill them. (There are some modern formats for normal map compression supported by the newer cards but we don’t use them yet.)
* Specular level: most serious 3-d programs let you control both the specular exponent, which controls how “tight” the specular hilights are, and the specular level, which controls how bright they are. X-Plane only lets you control specular level; if specular hilights exist, they are always as the maximum exponent for the sharpest specular hilights.
I receive bug reports periodically relating to how X-Plane and the scenery tools handle badly formed scenery files. Here’s the rules:
- It is a bug in the scenery tools if they create illegal scenery data. They should at least flag the condition that is leading to an illegal file and refuse to proceed.
- It is a bug in the scenery tools if they crash when trying to import illegal scenery data. Crashing can mean data loss, which is bad.
- X-Plane’s behavior with regards to illegal scenery data is undefined! This means it is not a bug for X-Plane to show a certain behavior with bad input data.
- There is no guarantee that X-Plane’s response to illegal scenery data will remain constant over multiple patches, or even multiple executions of the sim.
This third point has some dangerous consequences:
- X-Plane might handle illegal data in a way that an author views as useful; this “useful” side effect might go away in a future version.
- X-Plane might crash in response to illegal data…sometimes.
- There is no guarantee that X-Plane will provide useful diagnostics.
The reason for this scheme is that X-Plane is under pressure to get scenery loaded quickly, so at some point, there is a limit to how much validation it can do.
With that in mind, I do try to validate scenery when a problem is very common and the validation is not very expensive, or when there are serious data quality problems. But as we move to having the tools do more validation, we can have validation where we need it: immediately, for the authors working on the scenery.
I have received several requests for a transparent runway with a physical surface type. That request is just strange enough that we need to look back and ask, “how did we get here?”
High Level and Low Level Modeling
The “new” airport system, implemented in X-Plane 850 (with a new apt.dat spec to go with it) is based on a set of lower level drawing primitives, all of which are available via DSF. In other words, if Sergio and I can create an effect to implement the apt.dat spec, you can make this effect directly with your own art assets using a DSF overlay. This relieves pressure on the apt.dat spec to become a kitchen sink of tiny details.
The goal of apt.dat is to make a visually pleasing general rendering of airport data. DSF overlays provide a modeling facility.
Little Tricks
It turns out there are two things the apt.dat file “does” with the rendering engine that you can’t do in an overlay DSF:
-
The apt.dat file registers runways in the airport dialog box (for starting flights, positioning the airplane, etc.).
-
When the apt.dat reader places OBJs to form approach lights, it can offset their “timing base”, which is why the rabbit flashes in sequence.
(If you were to place a sequence of approach lights with rabbits in an overlay, every single light would flash at the same time because the DSF overlay format does not have a way to adjust the object’s internal timing parameter.)
The solution: the transparent runway. The idea of the transparent runway is to create with the apt.dat file the two aspects of a runway that you can’t build with a DSF overlay: the approach lights and the entry in the global airport dialog box. Transparent runways leave the drawing and surface up to you.
My thinking at the time was that the actual runway visuals and physics would be implemented together via either draped polygons or a hard OBJ.
Orthophotos and Bumps
So why do authors want a transparent but hard runway? The answer is orthophotos. With paged orthophotos, it is now possible to simply put down orthophotos for the entire airport surface area (whether as overlays or a base mesh) at some high resolution (our runways are 10 cm per pixel – I’m not sure if the whole airport area can be done at that resolution) and not have any special overlays for the runways. The transparent + hard runway would change the surface type.
I’m not sure if this is a good idea, but I’m pretty sure that this feature belongs in overlay DSFs and not the apt.dat file.
- Such a technique (varying hard surfaces independent of a larger image) is useful for more than just airports (and certainly more than just runways).
- The technique is unnecessary unless a DSF overlay is in use.
- Unlike nearly all of the rest of the apt.dat file, such an abstraction (invisible but bumpy) is much more a modeling technique and less a description of a real world runway.
I’m not sure we would even want the runway outline to be the source of hard data. If there are significant paved areas outside the runway then a few larger hard surface polygons might be more useful.
For most of its beta run, X-Plane 930 beta 14 didn’t handle engine power limiting very well*. Here’s the short version of the saga.
- Real planes sometimes have systems to limit total power output, because the power output of the engine (whether torque or internal temperature) can exceed safe operating limits at sea level).
- With X-Plane 9.00 you could set a critical altitude for an airplane – below this altitude, X-Plane would limit the power output of the jet. The idea is (roughly) to simulate these limiters by derating the engine’s power output below this “critical altitude”.
- This feature was really only meant for reciprocating engines – when Austin discovered in 9.20 that people were using this for turbines (understandable, given that there was no alternative) he simply disabled the feature for turbines. That wasn’t so good.
- To resolve the situation a little bit more cleanly, X-Plane 9.30 has a setting per airplane called “FADEC – automatically keep engines from exceeding max allowable power or thrust” that, when checked, gives you version 9.00 style behavior.
Now this was mostly good except for one problem: the betas would default this setting to “off” when loading an old plane. Since version 9 always acted as if the “FADEC”** was on, this meant that old planes would need to be edited.
Finally with beta 14 we switched things: beta 14 and newer default old planes to have the FADEC checkbox on, so that planes match their old behavior – you can turn the check-box off if you don’t want this behavior.
There is one final hitch: if you already went in and edited your airplanes for the earlier betas, you will find that they are now set wrong. You will have to reset the check-box. If you go back to the original, unedited, 920-compatible airplane you will find they “just work”.
I mention all of this to make two points:
- File formats for new features are subject to change during beta. In this case, the file format for the new FADEC check-box (introduced in 930) changed at beta 14. The OBJ syntax for dynamic lighting changed during beta too. Don’t do “bulk” work (e.g. change a large number of planes in the same way ) on your fleet based on betas – you might have to redo that work again! Wait until the sim goes final. That’s when the file formats are locked up.
- Don’t work around bugs in the sim. I have seen so many forum posts where there is a trivial bug in the sim (e.g. the sim is just screwed up in a simple way) and authors go in and update scenery to work around the bug. File a bug, then wait! If you work around the bug, we can’t fix the bug, and if we don’t fix the bug, then the bug just bites other users.
* Disclaimer: I don’t do systems, I don’t know anything about airplanes, so this whole discussion will be heavily simplified. The point of this post is not to get into a discussion of FADECS – in fact, don’t even bother to post about FADECs, I’m not going to approve them. If you want to talk about FADECs and engine modeling, email Austin. The point of this post is one about file formats and compatibility.
** FADEC isn’t a very good name for this feature – the feature generically limits power, without specifying a mechanism. My understanding is that some airplanes have mechanical limters, like a pressure valve on a turbo. Some planes have no limiters at all…ask a pilot “can you cook the engine by pushing the throttle too far” he will say “I’m not going to be the one to find out.”
In designing interfaces for building planes, writing plugins, etc. one of the main design questions that keeps coming up is: to copy or to reference? Should authors simply refer to an art asset, piece of data, or code in order to utilize it, or should the author copy a snapshot into the custom add-on. There isn’t one right answer. Here are the main considerations.
Performance and Efficiency
One of the obvious considerations is efficiency: in some cases we might be able to provide better performance when an art asset is referred from a common source.
For example, in some cases X-Plane will consolidate VRAM use based on actual files, so a library object is loaded once no matter how many packages use it, but is loaded many times if a package copies it.
(In other cases X-Plane will actually merge multiple copies of a resource – referencing is only a win in some cases.)
An indirect consideration: if an art asset is provided by Laminar Research and is used by reference, then a new update can provide a new, better optimized art asset – see below.
Dependencies and Contracts
When someone uses an art asset, algorithm, etc. by reference, it creates an implicit contract by the provider of the asset by reference to not change the properties of the asset. By comparison, when the asset is copied, the contract is only to support the format that the asset is encoded in.
This is the main reason why I am often against providing new assets by reference, whether it is a new dataref, texture, etc. Often I will simply send a user a snippet of code, rather than making X-Plane’s version available via a dataref. The idea is that copying does not create a new interface (and thus a new “contract”) between X-Plane and the add-on.
Copyright and Legal Issues
For historical reasons, the US legal system describes the privileges of intellectual property owners by regulating the act of copying. (To say that this is a bit quaint in the digital age doesn’t even scratch the surface, but that’s a rant for another post.) The result of this particular regulation of copying (but not of referencing) is that the decision to provide an asset by copy vs. reference has legal implications. If the author does not want to go through licensing, referencing may be the only option.
I’m not sure if these will make it into X-Plane 9.30 (we’re trying to close down features, but it doesn’t do much good to hold off features that help people make airplanes) but…while I was in Italy I created a few more manipulator types.
The set of manipulators let you change the value of a dataref directly with a mouse-click – the various flavors control how the dataref is changed.
These manipulators are the natural corollary to the command manipulator, which runs a command on a click.
Why have both? Commands are good, but they don’t cover 100% of sim functionality (just like datarefs don’t cover 100% of sim functionality). By having both, it will be possible to control switches and buttons that are best accessed by a dataref change.
For the basics on commands vs. datarefs, see
here.
The generic instruments already write to both commands (via the trigger instrument) and datarefs (via the rotary instrument) – these new manipulators provide the same functionality for 3-d cockpits.
It seemed like opposite-day at Laminar Research…Austin saying that an error shouldn’t quit the sim and me saying the error was never okay, ever. Well, I relented: with X-Plane 930 beta 8, if your scenery pack is missing objects, you can still fly. Instead you get a single error message like this:
That is the “non-fatal” error dialog box – you will see it only once for each scenery pack with a problem for each time you run the sim. It means that at least one thing is wrong with your scenery pack, but you need to look at Log.txt to see what’s wrong. For example, you might see this in the Log.txt file:
Failed to find resource 'KSBD_example.obj' at 'Custom
Scenery/KSBD Demo Area/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj' at 'Custom
Scenery/KSBD Demo Area/custom objects/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj' at
'Resources/objects/KSBD_example.obj'
Failed to find resource 'KSBD_example.obj'
at 'Resources/KSBD_example.obj'
***Error with scenery file "Custom Scenery/KSBD
Demo Area/Earth nav data/+30-120/+34-118.dsf"
(/Volumes/RAID/code/design/HLutils/Files/io_dsf.cpp: 503.)
Unable to locate object: KSBD_example.obj
In this case, X-Plane couldn’t find the object KSBD_example.obj – the sim is also listing all of the places it looked. Note that only the first location is a good location – the other 3 are legacy search paths that date all the way back to version 6. It is likely that in the next major version we will trim down our search paths significantly.
A few comments on this whole situation:
-
Authors, do not ignore error messages like the dialog box above – every one of them indicates a condition serious enough that we think you should fix it. Non-fatal errors like these may crash future versions of the sim, or your content may simply stop working.
If you file a bug against a future version of the sim saying your scenery pack used to work and is now broken, and we find that the old scenery pack had errors, we’re not going to fix the bug – we’re going to laugh maniacally and dance around you in a circle while singing “told you so”.
Okay – we’re very unlikely to do that – but if you have errors in your scenery pack, you’re doing something wrong and you need to fix it – treatment of illegal data is not stable between versions of the sim!!
-
I was never very sympathetic to this whole bug report because X-Plane has never accepted a DSF with missing objects – this has been a fatal* error since X-Plane 8.0 when DSF was introduced. So I simply don’t understand why there are any scenery packs floating around with objects missing. Why would you place an object if you don’t want to see it? The whole issue strikes me as a total failure to check quality by authors, since even running your pack once would reveal this kind of problem every time!
-
The motivation to make missing objects illegal comes from version 7 and ENVs. When looking at ENV scenery, I found that a large number of ENV scenery packs were missing at least some of their objects (an error that was silently ignored in ENV). It seemed like we were hiding an error and the result was authors not noticing simple mistakes that might “lose” an object (e.g. renaming an OBJ file).
Hence the “harsh” policy for DSFs – it was in response to a real problem with existing scenery!
- You don’t need to have missing objects just because you use library objects from another scenery pack (that might not be around). Use the EXPORT_BACKUP command in your library and a single blank OBJ as a place-holder for the objects you want from a library that might be missing. OpenSceneryX provides a stub library that authors can include so that their scenery will load without errors even if the OpenSceneryX library is not installed.
Anyway, Austin was right to make the error non-fatal. Besides being a little bit nicer for users who don’t know (or care) why their pack is gone) it lets authors get a list of all missing objects with only one run of the program.
* Fatal? In computer terms, a fatal error is one that makes the program quit, e.g. the error is fatal because it kills the program.
I have said this before, but now it’s finally true: new file specifications are subject to change in the middle of beta!
In particular ATTR_light_level has changed slightly from beta 7 to beta 8. If you are using this feature in your objects, you will need to update your objects.
A new ac3d beta will be posted later today that supports the updated syntax.
You can read about the syntax here.
I get asked about the maximum mesh density in X-Plane a lot.
First, I must note that X-Plane’s mesh is adaptive – the triangles are not arranged in a grid, but rather they’re arranged to maximize the quality of the mesh with minimal triangle count, while preserving the outlines of water bodies and airports. These pictures show a dynamic mesh fit to a section of the grand canyon (both with and without the shaded DEM for reference):
Second, the actual maximum mesh density is almost certainly limited by system resource constraints. The maximum spatial resolution in a DSF is about 0.6 millionths of a foot. You’re going to exceed the maximum number of vertices, or much more likely, the maximum amount of available memory in X-Plane long before you have a grid at that resolution.
(That doesn’t meant that resolution isn’t usable – remember, the idea of the adaptive mesh is to have very high densities in very improtant places, but not everywhere.)
But in practice this is all moot if you don’t code your DSFs by hand – the question that really matters is: what is the highest density mesh that you can use with the current scenery tools.
Until recently, MeshTool was limited to a 1201×1201 (90 meter) input DEM. The input DEM is a simple grid, and the finished DEM is never higher resolution than the input. So until now, it would have been just about impossible to make a really high res mesh, even if X-Plane could handle it.
The most recent changes to MeshTool fix this – MeshTool can now handle a 10801 x 10801 (10 meter) input DEM, which X-Plane should be okay with as well.
This new version will hopefully go into beta relatively soon.
What is the difference between a dataref and a command? They serve different purposes in X-Plane, but it’s easy to get them confused, especially because the names can look so similar. If you only take one thing away from this comparison, it should be:
- Datarefs are information.
- Commands are actions.
Datarefs
A dataref is a single bit of published information. For example, the user’s indicated airspeed, as seen by the pilot, is a dataref, stored in:
sim/cockpit2/gauges/indicators/airspeed_kts_pilot
Datarefs have names that do not change. Datarefs made available by X-Plane start with sim/ while datarefs made available by plugins start with another prefix. Datarefs have been in X-Plane since the release of the plugin system in version 6.70.
You can always read a dataref, but sometimes you can change it. Trying to change a dataref usually has one of three actions:
- If the dataref is not writable at all, nothing happens.
- If the dataref is writable, it will change.
- Sometimes a dataref may be writable, but only after changing some other sim configuration. For example, you can only “write” to the control surface deflection datarefs after setting the control surface override dataref to 1. (If you don’t set this override, X-Plane will constantly write its own ideas of the control surface positions to the control surface datarefs and your changes will be lost.)
You can read and write datarefs:
Commands
A command is an action that the sim can take on your behalf. For example, the command
sim/autopilot/altitude_arm
arms the autopilot for altitude hold.
Like datarefs, commands have permanent names, starting with sim/ for X-Plane or other prefixes for plugins. Commands have been available in X-Plane since version 9.0.
You can always actuate a command, but there is no guarantee that it will do anything. For example, the engine starter command won’t start the engine if the plane has electrical starters and the battery is dead.
You can use commands by:
Plugins
Plugins can add both new datarefs and new commands to the sim. Plugins can also change the behavior of all built-in sim commands, and can change the information in some datarefs.
Where Do I Find Datarefs And Commands
X-Plane’s default commands and datarefs are listed in the text files Commands.txt and Datarefs.txt in the Resources/plugins folder. (Note: providing the command list is new to X-Plane 930.) The dataref list is also available on the X-Plane SDK Wiki.
Up next: when should I use a command and when should I use a dataref?
