I posted a quick tip on how to create fence-like facades in WED. Basically WED doesn’t handle fences and other non-closed polygons very well, but you can work-around this. A future version will address this more completely.
This is my thinking on the WED roadmap:
-
WED 1.1 will relatively ship soon, without cosmetic and usability improvements. At this point the basic bugs (import/export) are fixed, so best to get out a finished, stable build so that people can at least edit overlays.
-
A future version will provide editing ATC layout information for X-Plane 10. This shouldn’t be too hard to get working, as we already have this in WED now so that we can develop test data for the new ATC system.
-
A future version will provide usability enhancements (e.g. previews of facades, etc.). I don’t have a ton of code done for this yet, but it’s important for everyone using WED for pretty much any purpose.
-
A future version will provide road editing since X-Plane 10 can drape roads.
I don’t know what order those 3 feature will come in, but they will all happen relatively soon after version 10 ships I think.
On my way back from South Carolina a few weeks ago I had some time in the airport to fix the WorldEditor 1.1 export bugs. All of the reports pointed to a single cluster of bugs that are hopefully now fixed in beta 2, and today I finally had time to get the betas posted.
I also cut a new build of MeshTool (2.0RC2) while I was doing builds; this fixes a bug when using orthophotos with varying physics.
So first: you can get the new WED 1.1 beta 2 and MeshTool 2.0 RC 2 on the tools page on the wiki.
The wiki? Yeah. Tyler has been helping me migrate the scenery tools to here. At this point I believe that all of the content on scenery.x-plane.com is now duplicated on the wiki (which also has additional articles).
WED Bugs
If you should find additional WED bugs, after you get over your initial surprise, please file bugs in the scenery tools bugbase. Please do not email me bug reports. WED has to take second priority to version 10 work, so I don’t have time to process bug reports now, and I will lose them. The bug base keeps your bug safe, keeps a record of what happened to it, and can take attachments.
Please do provide the minimal materials to reproduce the bug; simple packages with an earth.wed file are great. Thanks to all of the WED users who filed bugs with repro cases – this made it very easy to retest the export code.
My Polygon is Poly-Gone
It is illegal to have DSF polygons (or airport polygons) cross themselves; finally with beta 2, WED actually makes this case fairly easy to detect.
- If a polygon cannot be exported (because it is self-intersecting), an error message will indicate that some polygons were skipped, and those polygons are selected.
- If you pick “Error-Check Polygons” from the Edit menu, then for every polygon that has a self-intersection, an OBJ is added at the precise point of the self-intersection. Simply select and zoom in on those OBJs – they act as marker pins to show the problem. Delete the OBJs once you have untangled your polygon.
(I did experiment with error checking on the fly, e.g. simply showing red dots in the intersection points as you drag and resize the polygon, but the math is too slow. I am using CGAL to check bezier polygon intersections, and their algorithm is absolutely rock solid, but it can take up to a quarter of a second to recheck the polygon, which is too slow for interactive editing.)
UPDATE: beta 2 hangs when processing beziers. What is very odd is that this happens on the clean release code but not the working copy of WED I fixed the bugs in. Hence, when I checked all of the bug reports, they all passed. I have reopened all bezier-related bugs. I have not yet located the build environment differences causing the problem.
UPDATE 2: the hang on bezier processing was due to a bad compile configuration for a library WED uses, and was Mac specific. Having fixed this, I have recompiled and reposted WED beta 2 for Mac. If you already grabbed WED, re-download it, and make sure you get the version dated September 24th. You can tell you have the right one because the “compiled on” date in the about box will say “Sep 24 2010 19:34:09”.
For quite a while now, I have been advocating in favor of DDS compression. I am pretty damned obstinate, but eventually if enough people yell at me, I get a clue. I have come to appreciate that there are some cases where DDS compression is not a net win; this blog post explains when it happens and what we might do in X-Plane 10 to work around this.
DDS – The Good, The Bad, the Ugly
DDS is a file format that contains image data pre-mipmapped (that is, the smaller versions of the image that the video driver needs are included) in a format that may or may not be compressed. DDS is virtually always used with a compressed image format (like DXT1 or DXT5). This has three positive effects for X-Plane:
- Because the image is already compressed, we save CPU time when loading the texture that would be spent compressing while X-Plane is running.
- Because small versions of the image (the “mipmap pyramid”) is already in the file, we save time down-sizing the image with the CPU, another win for load time.
- Because the image is compressed ahead of time, it can be compressed with a slow high quality compressor rather than a fast low quality compressor, so relative to other compressed images we get an image quality improvement.
The bad is that the DDS file does not contain the original uncompressed file. If the user unchecks “compress textures to save VRAM”, DDS files remain compressed. If the image file contains details that don’t compress well, they’re going to get splatted and stay splatted.
What If VRAM Grew On Trees?
My original heavy arguments for DDS were based on the idea that VRAM is a limited commodity; if we don’t compress textures, the user runs out of VRAM faster and has to go down a level of resolution…and once that happens, everything starts to look ugly.
But what if the user has 1 GB of VRAM? At this point, we’ve limited the maximum quality the user can see because we don’t have the original uncompressed image anymore, only the DDS/DXT version. This can be frustrating to authors who spent a lot of time on their textures.
If you ship PNGs with your airplane or scenery, turning off texture compression will reveal this beautiful, uncompressed image, but now when texture compression is on, the compression will be done by the video driver, and that will look extra ugly.
The Best Of Both Worlds
This is my thinking for version 10. (These are just musings, we haven’t coded this yet.) Currently DDS are preferred to PNG files. We could relax the load rules in version 10 to prefer PNG over DDS when texture compression is off and DDS over PNG when it is on. This would allow authors to ship both PNGs and DDS files and have the right one be picked for the scenario: the pre-compressed one when texture compression is on and the uncompressed one when compression is off.
There’s a real difference between a useless beta (e.g. what we have with WED now) and a useful one. While the purpose of a beta program is to fix bugs, a beta can be useful if the features users must have work. I don’t think I will have time to finalize WED 2.0 this year, but I do think I will be able to fix the really nasty bugs, the ones that make it useless.
To that end I have fixed a number of crash-on-export bugs. To the users who filed complete crash-on-export bug reports: thank you! It was quite an education to see what was in some of these projects polygon-structure-wise.
If you filed a WED export bug, I may have requested feedback if you didn’t include the WED file. Basically I would like the minimal materials to fully reproduce the bug by opening the package and picking “Export to DSF”. For many packs this is just the Earth.wed file, although sometimes textures or text files may be necessary.
A number of the crash bugs are caused by degenerate polygons. For example, if your polygon crosses itself in an X shape, not only is this illegal in X-Plane, but it also drives the exporter crazy. With the next beta, WED will select polygons that failed to export to help you locate such problems.
I realized I have slightly better test shots of global illumination than the ones that got out a week ago. These are from only a day or two after the last shots.
This is the Cirrus again, with landing lights and strobes; you can see that all of the airplane’s lights contribute dynamically to the lighting on the fuselage and doors as they move. (Yes, that is heat blur on the engine; the heat blur still needs a lot of tuning.)
This shows airplane-mounted lights interacting with custom scenery. In this case the standard Cirrus (with global lights attached) casts both strobe and multiple landing light spill on LOWI. One of the powerful results of global illumination is that we get correct lighting interaction between diverse content, including third party content.
Finally, this shows an airport beacon lighting a plane and vice versa. The global lights on the airport beacon are inside an animation group, making them “sweep” the airplane, which can in turn animate the airport beacon. With global illumination, there are no rules about who lights who.
Thanks to my foolish use of unprotected directories, we have basically announced that X-Plane 10 will feature global illumination. Here is some basic information on global illumination.
What Is Global Illumination?
Global illumination is the ability of any part of an airplane or scenery system to cast light on any other part of the scenery system or airplane. In X-Plane 9, the only lights in the sim that ever actually cast cast light anywhere else are:
- The sun.
- The airplane’s landing light. (Even if your plane has many landing light billboards, there is only one spill effect.)
- Three 3-d lights in the 3-d cockpit.
This list was kept short due to the high cost per pixel of each light on all rendering.
When X-Plane 10’s global illumination is enabled, a “spill” light attached to any OBJ can shine light on anything near it. Since any OBJ can have a spill light, this means we can have light sources on airplanes, scenery, cars, whatever you want. The spill effects any 3-d scenery nearby, even from another scenery pack.
This kind of still effect can be simulated in X-Plane 9 by careful use of LIT textures. However, real global illumination works between art assets created by separate authors. You can drive your custom airplane up to a custom airport and the landing and logo lights on the airplane will cast light on the terminal; the apron lights from the terminal will cast light on the airplane.
Furthermore, global illumination is fully dynamic – as objects animate or move, the lighting effects are correctly applied in 3-d. This makes effects possible that cannot easily be created using LIT textures.
Requirements for Global Illumination
Like most new rendering tricks in version 10, global illumination will be a rendering option that can be optionally enabled by users who have a video card meeting hardware requirements. In the case of global illumination, that requirement is a DirectX-10 generation video card, e.g. any Radeon HD , nVidia GeForce 8000 or 9000 series, and “100” series (100,200,300,400 series).
For authors: global illumination is applied using named and parametrized lights on your OBJ. Anywhere you can attach a light billboard, you can attach a spill effect as well, with some tuning constants for how wide you want the light, etc.
It will be possible to create two versions of your LIT textures, one to be used when global illumination is enabled, and one when it is disabled. Thus if you are baking lighting into your textures with a 3-d modeling program, you can simply re-bake the lit texture with some lights disabled and add 3-d lights to your model. The result is an airplane with real 3-d lighting where possible, and a close approximation via baking otherwise.
Global illumination can be added to a model incrementally; existing art content will work normally with global illumination enabled or disabled, so authors can choose to add a few light spill effects or add a large number, as time permits.
The Cost of Global Illumination
Global illumination isn’t going to be free. The main cost is an increase in VRAM use and fill-rate. The cost of global illumination is mostly a one-time cost to put X-Plane into a new rendering mode. (Graphics nerds: global illumination is implemented via deferred rendering.) The incremental cost of lights isn’t that high, although a scene with a lot of lights will have impact.
My expectation is that users with new, highly capable high-end graphics cards will be able to run global illumination easily, but will lose some of the other benefits of fill rate. (For example, running at 2560 x 1024 + 4x FSAA is a lot more painful with global illumination than without.)
Global illumination also introduces two artifacts, both of which I am trying to minimize as best as I can. These artifacts are a function of deferred rendering – all games that use deferred rendering have to address these problems:
- The lighting calculations are shared between multiple translucent surfaces, which can create some strange effects. For example, if a translucent window is in shadow, the scenery behind the window will appear to be in shadow too.
- Traditional full-screen anti-aliasing is not available with deferred rendering. We should be able to offer a simulation of 4x FSAA as well as some kind of cheaper FSAA-approximation, but the cost will be quite a bit higher in fill rate than the 16x-style CSAA available now.
(Hardware-based FSAA can make a number of optimizations like CSAA to optimize throughput; this is how such high multiples as 16x are possible. Since our implementation is similar to “super sampling” and costs a real 4x in performance, 4x will be the highest setting.)
Why Global Illumination
Of the new X-Plane 10 rendering engine features (and there are a fair number of them), global illumination is certainly the one that has the most impact on the structure of the rendering engine. With global illumination, X-Plane effectively has two separate modes (“forward” rendering, which is the only mode X-Plane 9 has, and “deferred” rendering, which produces global illumination).
One of the reasons to get global illumination done earlier than other features was that implementing global illumination required rewriting or modifying nearly every piece of low level rendering code. Now that the work is in place, we can safely add new features and test them in both modes.
Global illumination also meets two requirements:
-
Sergio has long observed the central importance of lighting and shadows in the look of X-Plane; at some point more polygons and better textures still look synthetic without a realistic illumination model. Global illumination makes a more realistic lighting model possible at night. Airports represent an environment that can hopefully take advantage of such capabilities in a big way.
-
As hardware becomes more powerful, authors have to do more work to create content that takes full advantage of the rendering engine. We are reaching a point where artist’s time is going to be a limiting factor as well as hardware and engine capabilities. Global illumination thus kills two birds with one stone: it makes the rendering engine’s output look better, but it also makes the whole scene look better with less work by the artist.
(For example, when baking lighting into a model, the author must plan the model’s texture UV map to guarantee unique texture space for all spill effects. When lighting effects are dynamic, the author can simply texture so the model looks good without worrying about baking requirements.)
There have been a few posts about X-Plane 10 in a few random places; here is a summary of all of the version 10 material that’s been previewed so far, and some notes on what is actually being shown.
First we had Paul’s Oshkosh 2010 video:
http://www.youtube.com/watch?v=yCoDPNvOMP0?fs=1
I think this has been made clear already, but:
- The base simulator shown here is not X-Plane 10, it is X-Plane 9.
- Many of the airplanes shown here will be released for X-Plane 10.
- This is not showing the new X-Plane weather system or global lighting.
- Some of the content shown here are third party add-ons, available today for X-Plane 9.
The main purpose of this video was to show X-Plane off at Oshkosh; at the time we didn’t have X-Plane 10 in a state where we could do an exclusive version 10 preview.
Then I accidentally leaked two test videos of global illumination. This was strictly accidental: I was looking for a cheap way to post a large video for Austin and Propsman late at night and didn’t think anyone would sift through 191 zip files to find two obscurely named videos. I was wrong, and someone found them on the org. I appreciate that participants in the ensuing discussion withheld judgment; these were early test videos and don’t represent the final feature in any useful way. They do, however show off some of what global lighting will mean.
- This is the Cirrus Jet with landing lights implemented via global illumination. We get two distinct landing lights that cast specular hilights on the fuselage. As the door animates, it opens “into the light”.
- This is the Avanti Piaggio with strobes and beacons implemented via global illumination. The strobes cast light both on the fuselage and on the runway below the plane.
This second movie is typical of the kinds of tests I do: the beacon lighting is just awful – a gross huge red light splatted on the plane for test purposes. When I get the rendering engine code working I usually hand the feature off to Propsman or Sergio to tune the textures and art constants. In this case, the videos are pre-tuning.
Austin has posted three screen-shots of X-Plane 10-related content:
-
This is Javier’s new shuttle, which I believe will ship in version 10. I believe this shot may have been taken in X-Plane 9. So this is not the new weather system.
Some of these screenshots and Paul’s video were shot in X-Plane 9. By the time the new airplanes are finished, they will not be usable in version 9 – they will be version 10 only.
-
Propsman has done work on the lighting system. It can be subtle to see what’s going on here because the old runway lights looked pretty good too, but most of these billboard lights are actually rebuilt.
-
This night shot shows global illumination in the scenery system. The glow on the highway pavement is not rendered; it comes from the 3-d lamps along the side of the road. Similarly, the car headlights spill light on the pavement and each other as they drive. (Note how the highway lines are visible in the headlight spill even when there is no streetlight.)
One difficult problem with rendering a lit highway at night is that the lighting from street lamps on a highway tend to spill light on the surrounding terrain, an effect that is impossible to create with a LIT texture. If you look at the right side of the main highway at the bottom of the picture, you’ll see that the street light is casting light on the grass to the right of the highway too.
I think that that’s all we have posted. At least, it’s all I am aware of. I will go into some of the details of global illumination in another post.
A quick thought on procedural vs. algorithmic scenery: there was some discussion on the X-Plane dev list about procedural terrain; the Outerra screen shots have stirred up interest.
There is a fundamental difference between what you see with Outerra and what we do with our global scenery:
- Typically procedural scenery is based on the idea of ‘amplifying data’ – that is, making a little bit of data look more interesting by applying a recursive algorithm to “generate” detail.
- Algorithmic scenery involves taking a large amount of unique input data and translating it; the detail comes from not losing information in the source data.
The key difference is whether the resulting scenery comes from a process that “creates” information through a ‘procedure’ or “translates” information.
Our scenery process does a bit of both, but we are more in the algorithmic camp than procedural camp; the global scenery is produced from hundreds of GB of input data, and we are constantly looking for better input data to create more interesting and accurate output scenery.
In fact, I would say that we are becoming more algorithmic and less procedural. In version 9, the urban roads in non-US cities are “procedural” – an algorithm generates them from the terrain data, an algorithm, and some random noise. For version 10, we are importing OSM.
One thing I have noticed in the work on version 10 global scenery is that we’ve finally crossed a line. With version 9, the question was ‘what is the best data we can get’. With version 9, the question is ‘how much information can we keep’; we’ve reached a point where the resolution of our input data is so much higher than what can go on DVD, that it’s a question of filtering down, not synthesizing up the resolution.
I have updated some of the facade documentation on the wiki with new performance tips for using facades.
A few quick notes:
- Your facades must be counter-clockwise (when viewed from above). Do not repeat the first point; X-Plane will “close” your building for you. (A four sided building should have four points, not five.)
- If you turn off two-sided facade drawing and your walls look wrong (and your roof disappears) your facade is wound in the wrong direction.
The performance tips go into a fair amount of detail about saving memory. Most of X-Plane’s rendering fall into two categories:
- Shared meshes (objects), where the geometry of the object is saved once and used lots of times. Objects usually hurt frame-rate by consuming CPU time, because for each drawing of the object, we have to do some setup to draw that shared geometry in a different location. (Version ten should feature some major improvements in object efficiency.)
- Non-shared meshes (everything else), where every single “instance” of a tree, facade, forest, etc. is uniquely constructed in memory. Non-shared messages are very fast (because we can submit a huge pile of non-shared messages to the video card in one shot) but they consume a lot of memory (because we pay for RAM per building/tree, not just once). Typically non-shared meshes are limited by virtual address space, not by framerate.
Facades are non-shared meshes, so the performance tips focus on how to limit the amount of RAM needed to represent your facade.
The short answer is: to save memory.
The cars and replay seem to be a case of damned-if-we-do, damned-if-we-don’t. If we don’t stop the cars and reverse them in replay, we get piles of bug reports. If we do try to replay the traffic, we get bug reports too.
The current implementation is a bit strange: when you replay traffic, the cars will go back a bit in time, but at some point they will just stop and refuse to reverse any more. What’s going on?
The answer is that the cars have the memory of a goldfish. They simply don’t remember where they came from. Each car knows what “link” it is on, and about when it got onto that link and how fast it is going. (A link is a single straight piece of road.) So when we go into replay, we can easily move the cars along their links as time goes forward and backward.
But when we reach a time earlier than when the car entered the current link, the car has know idea how it got there, so it is forced to stop.
This is a simple case of not wanting to burn four tons of memory on a feature that is mainly visual. To replay the cars, we would have to accumulate a history of every link a car has been on as it drives. For 20,000 cars and a sim that’s been running a while, that’s a lot of memory to burn just in case you happen to hit the replay button.
In fact it gets worse. The cars are kept in a data structure that tells us who needs to make a driving decision and when.* This structure is optimized for the cars moving forward in time. We’d have to make and maintain an entire second copy of this structure to move the cars backward; again burning CPU and memory while you fly just in case you might hit a replay.
So instead we just provide replay on the current link.
* Programming nerds: the cars are in a priority queue by time to next navigation decision. I consider this to be very clever.
