This thread on X-Plane.org sparked off quite the discussion. Now a lot of this is a discussion of when LR will have an overlay editor – there are a few overlay editing functions that Jonathan Harris’ excellent OverlayEditor apparently does not yet support, sparking this discussion.
(I am not saying that LR should rely on Jonathan to do an overlay editor. But I am saying that the complaints I hear about a lack of overlay editing go down when Jonathan’s overlay editor does everything that the file formats can do.)
But another part of the discussion focused on the problem of mesh editing. In particular, the basic terrain in a DSF is a fully baked output of a complex process that starts with higher level GIS input data. In other words, we start with a raster DEM, polygon coastline, apt.dat file, vector roads, and a bunch of config files and hit “bake” and a DSF comes out the other side, with a lot of processing.
This is very different than FS X, which integrates its data sources on the fly. Why did we choose a precomputed route for scenery? It has some pros and cons. (In understanding how we made these decisions, think back to what scenery was like with X-Plane 7 and ENVs and single-core machines.)
Performance
The main benefits of preprocessing scenery are performance related. When you process scenery data into the final scenery while flying, that computer power takes away from the rendering engine, thus cutting down fps. At some point you have a zero-sum game between how much cost there is to loading scenery and how complex the scenery integration can be; you have to pick very simple scenery integration algorithms to keep fps up.
(This is less of an issue as more cores become available, but is still a factor.)
When pre-processing, we can use algorithms that take minutes per DSF without affecting framerate.
Similarly, there might be scenery processing algorithms that improve fps by optimizing the output triangles – but do we have time to run these algorithms during load? With preprocessing we have all the time in the world because it happens once before the DVDs are burned.
Preprocessing also breaks a similar zero sum game between scenery data size and quality; the source data we use to make the scenery is a lot bigger than the 78 GB of DSFs we cut; if we had to ship the source data, we’d have to cut down the source data quality to hit our DVD limitations. With be-baking we could use 500 GB of source data without penalty.
Format Flexibility and Stability
The second set of benefits to preprocessing are flexibility benefits. (Consider the file format churn of the ENV days.)
– With a preprocessed scenery file, what the author creates is what the user sees – X-Plane does not go in and perform subjective integrations on the scenery later that might change how it looks in a negative way.
- There is no need to revise the scenery file formats to introduce new data sets, because new data sets and old are all processed to the same final DSF container format.
- A wide variety of mesh generation techniques can be employed because the mesh generation is not built into X-Plane. This is a flexibility that I don’t think anyone has really utilized.
- Changes of behavior in the scenery generation toolset can never affect existing scenery because that scenery is already preprocessed; this help compatibility of old file formats.
Integration Issues
There are some real limitations to a pre-processed format, and they are virtually all in the bucket of “integration issues” – that is, combining separate third party add-ons to improve scenery. In particular, in any case where we preprocess two data sources, we lose the opportunity for third parties to provide new scenery to replace one of those data sources and not the other.
Airport is the achilles heal where this hurts us most; while airport layouts are overlays and can be added separately to the scenery system, the elevation of the base mesh below the airport needs to be preprocessed. This is something I am still investigating – a tolerable fix that other shave proposed is to allow an overlay scenery pack to flatten a specific region regardless of the user setting (so an author can be assured of a flat base to work from).
Preprocessing does fundamentally limit the types of third party add-ons that can be done; with version 9 and overlay roads, we are getting closer to letting road add-ons be overlays (see this post).
It appears to me that integration isn’t the primary complaint about the scenery system (the primary complaint is lack of tools) but we’ll have to see once we have mesh editing tools (mesh recreation tools really) whether preprocessing still limits certain kinds of scenery.
Note that a lack of tools or a lack of tool capability is not an inherent limitation of pre-processed scenery. We have an incomplete tool set because I have not written the code for a complete tool set, not because it cannot be done.
(The complexity of writing base mesh editing tools is a function of the complexity of a vector-based base mesh – this is also not related to pre-processing per se.)
Tools
In the end, I think the question of tools is not directly tied to the question of pre-processing. Whether we have scenery that is processed by X-Plane or a preprocessing tool, we have the same issues:
- Good tools require an investment in coding user interface.
- The code to convert source data which users might want to edit (like a polygon that defines a lake) to data the simulator might want to use (like a list of 78,231 triangles) has to be written.
I don’t think either option (pre-processing or in-simulator processing) reduces the amount of work to be done to create a good toolset.
As a final thought, using scenery file formats that are “easier to edit” (e.g. a file format that contains a polygon for water rather than triangles) doesn’t make the total code for scenery tools + simulator any easier; it just moves the task of “processing” the scenery from the tools to the simulator itself.
