Two warnings about normal maps:
-
Make sure that the RGB color underneath transparent sections does not turn black or white! Some image editing programs (in particular Photoshop) will lose the color beneath a transparent area.
With a normal map, this is very bad – black and white are not legitimate normal map colors, and the result will be bogus normal vectors under the non-shiny part. Normal maps affect more than just specular shiny hilights – the normal map affects all lighting, so having black or white under your transparent (non-shiny) parts is bad news.
To check whether this has happened, I recommend Graphics Converter, which will show you your alpha and RGB channels separately, exactly as they are in the file.
-
Make sure your RGB value are normalized. The “length” of the normal (as encoded in RGB) must come out to a distance of 1. This is virtually impossible to do using PhotoShop or an image-oriented program…I suggest you use a real plugin to PhotoShop or Blender to create normal maps that are correctly “normalized”.
It is also very possible that X-Plane’s gamma correction is distorting normal maps, but that’s one for me to fix.
I’m seeing a number of bug reports where weird artifacts are showing up in 940…missing pieces of runway, flickering triangles…all sorts of good stuff!
I believe that this is due to some kind of bug relating to threading, X-Plane and the video drivers. I won’t say whose fault it is because I really don’t know. I do know that the bug appears to not happen on OS X. (But this could simply be because the threads time out differently on OS X.)
The changes to the rendering engine for 940 from 930 are substantial and aggressive – it’ll take us a little bit to fix these things.
When you wonder how come programs don’t use all 8 of your cores yet, well…this is why…multi-core programming is complex, tricky, tedious to debug, and often involves substantial changes from the original code.
X-Plane 940 now supports normal maps on OBJ models (both scenery and airplane). I’ll get more formal docs up once the rest of my office is moved and unpacked but here’s the details for now:
The normal maps are in “blender” format see here. The alpha channel is optional; if it is present, it serves to modulate the level* of specularity. Opaque means full specularity, transparent means none. You can use this feature to make some parts of an object shiny and some dull on a per-pixel level.
Shininess level is modulated by both ATTR_shiny_rat and the alpha channel, so you need ATTR_shiny_rat 1.0 and an opaque alpha channel (or no alpha channel) to see full specularity.
Normal maps are only available for objects and only appear if pixel shaders are on and per-pixel lighting is enabled.
Normal maps should be PNG format, not DDS – they will not be texture compressed because S3TC compression tends to kill them. (There are some modern formats for normal map compression supported by the newer cards but we don’t use them yet.)
* Specular level: most serious 3-d programs let you control both the specular exponent, which controls how “tight” the specular hilights are, and the specular level, which controls how bright they are. X-Plane only lets you control specular level; if specular hilights exist, they are always as the maximum exponent for the sharpest specular hilights.
X-Plane 940 beta 1 is out…it will take a little bit to get the release notes and docs on the website completely up-to-date. We’re still dealing with loose ends from our migration to the new web site, and most of my office is packed up for a move to the Boston area. I’ll try to get docs up as fast as I can given the chaos flying about.
Given the interest multi-core stirred up in previous notes, I will mention one change to 940: with this build we’ve added yet more multi-core to X-Plane.
In 931, X-Plane will use as many cores as you have to load textures, but only one to build “3-d scenery” (a loose category for the work we do when we make airport taxiways and lines, build forests, and extrude roads).
In 940, this “3-d scenery” is also done on as many cores as you have. This should speed up load times a bit, particularly under very heavy tree settings, and hopefully keep the forest engine running faster for users with more cores.
It also sets us up for long-term growth; X-Plane’s visual quality is sometimes limited by the time to build 3-d meshes…being able to do this work on many cores means we can use higher quality algorithms.
Consider for example the roads: my original “road extruder” (the code that converts a vector road into a 3-d model, called an extruder because it builds a 3-d road from a cross section like one of those play-dough toys) made beautiful intersections with stop-lines and cross walks and lots of other great stuff.
It was also really slow. And at the time the sim wouldn’t fly at all while roads were extruded, so speed was of primary concern. So I replaced it with the much dumber extruder you see today, where intersections are basically ignored.
Now that we have 3-d scenery build on multiple cores, we can begin to provide rendering options that take more CPU time but produce higher quality results. The trees and airport layouts already do this (in that they take more time and produce slower, more detailed, higher triangle count sceneery at higher rendering setting for the same input DSF ad apt.dat file). With more cores, we can continue this strategy with roads and other parts of the sim without worrying about overloading the one core that was doing this work.
Of course just because we can use 8 cores doesn’t mean we do…you won’t see 8 cores maxed out very often, particularly if you have simple scenery and a very fast machine.
I have set up a public bug database for the scenery tools. The reason is that I don’t have much continuous time to work on the scenery tools – it’s very stop and go and will be for a while. The bug base lets you create a permanent record of a problem that I can’t lose track of in the heat of whatever is on fire today.
WED is almost ready for a beta, but I am just completely swamped with work right now…for WED 1.0 I worked full time on fixing WED bugs for a significant time when WED went beta; this time around I won’t have that kind of time – at least not this year.
So…we’ll see how the bug base works out. My hope is that I can post WED, put it down, and pick up work on it intermittently with the bug base providing a record of what remains to be done.
Regarding other tools, there will be at least one more MeshTool beta with an improved shapefile processing algorithm that will handle broken shapefiles better. The ac3d plugin has some bugs filed against it but it’s possible that they’d be deferred past the 3.2 plugin.
I have been working on OpenStreetMap data for X-Plane this week. Use of OSM for global scenery is going to be a bit different from projects like X-VFR or other specific custom OSM-based scenery.
The issue at hand is accuracy vs. plausibility…
- Accuracy: how much error is there between what exists in the real world and what exists in the scenery. Is that road in the right location? Is it the right type of road?
- Plausibility: does the scenery as a whole look reasonable? Is that road on land or is it in the water? Is that river running up a mountain?
The global scenery needs to prefer plausibility over accuracy. Because we can’t check and manually fix errors in the source data for the entire world, and because we don’t cut the global scenery very often, it is important that the global scenery err on the side of reduced accuracy (remember, the global scenery isn’t that accurate in the first place) rather than plausibility problems that will clearly be ugly and distracting.
The implication for OSM-based global scenery: not everything in OSM is going to show up in the global scenery. This would be true anyway simply due to the need to keep the global scenery compact. (I trust that OSM will grow to the point where it can source scenery larger than we can ship for the entire world.) But the global scenery generator may need to err on the side of not including data that might have plausibility problems.
Fortunately it is possible to build custom scenery from OSM as well. I don’t see OSM-based global scenery as replacing efforts like X-VFR and others; rather custom scenery will always be able to use more OSM data , checking the data for accuracy, rather than reducing the data to maintain plausibility.
One technical note: I am working on an extention to the road .net file format that would allow road networks to be draped over terrain. This would allow overlay packages to add/replace road grids without having to know the shape of the base scenery mesh, and make it easier to both create custom road networks and to create the tools that manipulate them.
I receive bug reports periodically relating to how X-Plane and the scenery tools handle badly formed scenery files. Here’s the rules:
- It is a bug in the scenery tools if they create illegal scenery data. They should at least flag the condition that is leading to an illegal file and refuse to proceed.
- It is a bug in the scenery tools if they crash when trying to import illegal scenery data. Crashing can mean data loss, which is bad.
- X-Plane’s behavior with regards to illegal scenery data is undefined! This means it is not a bug for X-Plane to show a certain behavior with bad input data.
- There is no guarantee that X-Plane’s response to illegal scenery data will remain constant over multiple patches, or even multiple executions of the sim.
This third point has some dangerous consequences:
- X-Plane might handle illegal data in a way that an author views as useful; this “useful” side effect might go away in a future version.
- X-Plane might crash in response to illegal data…sometimes.
- There is no guarantee that X-Plane will provide useful diagnostics.
The reason for this scheme is that X-Plane is under pressure to get scenery loaded quickly, so at some point, there is a limit to how much validation it can do.
With that in mind, I do try to validate scenery when a problem is very common and the validation is not very expensive, or when there are serious data quality problems. But as we move to having the tools do more validation, we can have validation where we need it: immediately, for the authors working on the scenery.
To answer the most basic questions:
- This is a base mesh orthophoto scenery I made with MeshTool, as a test.
- The source DEM is 10m NED, the source imagery is 1mpp DOQQ, down-scaled to 3mpp. I gave MeshTool a point budget of 500,000 mesh points per DSF tile, and it used them all.
- This version of MeshTool (2.0 beta 2) should be out in the next 3 days.
- That’s X-Plane 930RC4. The framerate really is around 100 fps.
- There are no dynamic real-time shadows. Rather, the orthophotos have the shadows “baked” in because they’re photos.
- There are artifacts at the joins of the orthophotos because I spent time fixing projection errors.
Clearly we need more than 25 nm visibility in some cases!
I have received several requests for a transparent runway with a physical surface type. That request is just strange enough that we need to look back and ask, “how did we get here?”
High Level and Low Level Modeling
The “new” airport system, implemented in X-Plane 850 (with a new apt.dat spec to go with it) is based on a set of lower level drawing primitives, all of which are available via DSF. In other words, if Sergio and I can create an effect to implement the apt.dat spec, you can make this effect directly with your own art assets using a DSF overlay. This relieves pressure on the apt.dat spec to become a kitchen sink of tiny details.
The goal of apt.dat is to make a visually pleasing general rendering of airport data. DSF overlays provide a modeling facility.
Little Tricks
It turns out there are two things the apt.dat file “does” with the rendering engine that you can’t do in an overlay DSF:
-
The apt.dat file registers runways in the airport dialog box (for starting flights, positioning the airplane, etc.).
-
When the apt.dat reader places OBJs to form approach lights, it can offset their “timing base”, which is why the rabbit flashes in sequence.
(If you were to place a sequence of approach lights with rabbits in an overlay, every single light would flash at the same time because the DSF overlay format does not have a way to adjust the object’s internal timing parameter.)
The solution: the transparent runway. The idea of the transparent runway is to create with the apt.dat file the two aspects of a runway that you can’t build with a DSF overlay: the approach lights and the entry in the global airport dialog box. Transparent runways leave the drawing and surface up to you.
My thinking at the time was that the actual runway visuals and physics would be implemented together via either draped polygons or a hard OBJ.
Orthophotos and Bumps
So why do authors want a transparent but hard runway? The answer is orthophotos. With paged orthophotos, it is now possible to simply put down orthophotos for the entire airport surface area (whether as overlays or a base mesh) at some high resolution (our runways are 10 cm per pixel – I’m not sure if the whole airport area can be done at that resolution) and not have any special overlays for the runways. The transparent + hard runway would change the surface type.
I’m not sure if this is a good idea, but I’m pretty sure that this feature belongs in overlay DSFs and not the apt.dat file.
- Such a technique (varying hard surfaces independent of a larger image) is useful for more than just airports (and certainly more than just runways).
- The technique is unnecessary unless a DSF overlay is in use.
- Unlike nearly all of the rest of the apt.dat file, such an abstraction (invisible but bumpy) is much more a modeling technique and less a description of a real world runway.
I’m not sure we would even want the runway outline to be the source of hard data. If there are significant paved areas outside the runway then a few larger hard surface polygons might be more useful.
For most of its beta run, X-Plane 930 beta 14 didn’t handle engine power limiting very well*. Here’s the short version of the saga.
- Real planes sometimes have systems to limit total power output, because the power output of the engine (whether torque or internal temperature) can exceed safe operating limits at sea level).
- With X-Plane 9.00 you could set a critical altitude for an airplane – below this altitude, X-Plane would limit the power output of the jet. The idea is (roughly) to simulate these limiters by derating the engine’s power output below this “critical altitude”.
- This feature was really only meant for reciprocating engines – when Austin discovered in 9.20 that people were using this for turbines (understandable, given that there was no alternative) he simply disabled the feature for turbines. That wasn’t so good.
- To resolve the situation a little bit more cleanly, X-Plane 9.30 has a setting per airplane called “FADEC – automatically keep engines from exceeding max allowable power or thrust” that, when checked, gives you version 9.00 style behavior.
Now this was mostly good except for one problem: the betas would default this setting to “off” when loading an old plane. Since version 9 always acted as if the “FADEC”** was on, this meant that old planes would need to be edited.
Finally with beta 14 we switched things: beta 14 and newer default old planes to have the FADEC checkbox on, so that planes match their old behavior – you can turn the check-box off if you don’t want this behavior.
There is one final hitch: if you already went in and edited your airplanes for the earlier betas, you will find that they are now set wrong. You will have to reset the check-box. If you go back to the original, unedited, 920-compatible airplane you will find they “just work”.
I mention all of this to make two points:
- File formats for new features are subject to change during beta. In this case, the file format for the new FADEC check-box (introduced in 930) changed at beta 14. The OBJ syntax for dynamic lighting changed during beta too. Don’t do “bulk” work (e.g. change a large number of planes in the same way ) on your fleet based on betas – you might have to redo that work again! Wait until the sim goes final. That’s when the file formats are locked up.
- Don’t work around bugs in the sim. I have seen so many forum posts where there is a trivial bug in the sim (e.g. the sim is just screwed up in a simple way) and authors go in and update scenery to work around the bug. File a bug, then wait! If you work around the bug, we can’t fix the bug, and if we don’t fix the bug, then the bug just bites other users.
* Disclaimer: I don’t do systems, I don’t know anything about airplanes, so this whole discussion will be heavily simplified. The point of this post is not to get into a discussion of FADECS – in fact, don’t even bother to post about FADECs, I’m not going to approve them. If you want to talk about FADECs and engine modeling, email Austin. The point of this post is one about file formats and compatibility.
** FADEC isn’t a very good name for this feature – the feature generically limits power, without specifying a mechanism. My understanding is that some airplanes have mechanical limters, like a pressure valve on a turbo. Some planes have no limiters at all…ask a pilot “can you cook the engine by pushing the throttle too far” he will say “I’m not going to be the one to find out.”
