In my previous post I drew an analogy between a scenery system with its file formats and a turtle within its shell. We are limited by DSF, so we are making a new file format for base meshes so we and all add-on developers can expand the scope of our data and make better scenery for X-Plane.
The really big change we are making to base meshes is to go from a vector-centric to a raster-centric format. Let’s break that down and define what that means.
Vectors are fancy computer-graphics talk for lines defined by their mathematical end-points. (Pro tip: if you want to be a graphics expert, you just need the right big words. Try putting the word anisotropic in front of everything, people will think you just came from SIGGRAPH!) DSF started as an entirely-vector format:
All 3-d clutter is defined by lat/lon locations, so we have the vector outline of polygons, autogen blocks, etc.
The base mesh is pre-triangulated, so most base-mesh features are defined by the corners of the triangles forming lines (e.g between land and water).
This isn’t the only thing DSF does – we added raster capabilities and there is e.g. raster sound and season data in X-Plane 12, but DSF is fundamentally about vector data – saying where the edges of things go exactly.
This was great for a while, but now that we have more and more vector data (complex coastlines, complex road grids, complex building footprints) the DSFs are getting too big and slow for X-Plane.
Raster data is any data stored in a 2-d grid. This includes images (which in turn includes orthophotos) but it also includes 2-d height maps (DEMs), and the 2-d raster data we include in DSF now (e.g. sound raster data, season raster data etc). Any time we store numbers that mean something in a 2-d array, we have raster data.
Raster data has several advantages over vectors:
Raster data is what the GPU wants to consume.
Raster data has really good LOD characteristics for close detail with long view distances.
We can put more interesting and dense information into a raster tile without it getting bigger.
Twenty years ago, when I first worked on DSF, computers didn’t have the capacity to use lots of raster data – this was back when 8 MB of VRAM was “a lot”. But now we no longer need to depend on vectors for space savings.
Raster tiles are raster data broken into smaller tiles that get pieced together. Raster tiles have become the standard way to view GIS data – if you’ve used Apple Maps or Google Earth or OpenStreetMap or any of the map layers in WED, you’ve used raster tiles.
Raster tiles have a bunch of advantages too:
They have really great LOD/VRAM usage properties.
They can be loaded incrementally.
They provide an easy way to vary resolution and let authors skip providing data that they don’t need to provide. (E.g. “forest” raster data over the ocean? Just don’t provide any tiles!)
So our plan for the next-generation base mesh is “all raster tiles, all the time” – we’d like to have elevation data, land/water data, vegetation location data, as well as material colors all in raster tile form. This would get us much better LOD/streaming characteristics but also provide a very simple way for custom scenery packs to override specific parts of the mesh at variable resolution with full control.
Raster tiles are not the same thing as orthophotos. A raster tile is any data contained in a 2-d array, not just image data cut into squares (e.g. orthophotos). So while a raster-tile system may make it easier to build orthophoto scenery, it does not mean that the scenery can only be orthophotos.
At the X-Plane Developer Conference in Montreal this year I gave a presentation sharing my thinking on our next-gen scenery system. This has created a lot of interest but also a lot of confusion. So in these next two blog posts I want to start by clarifying two fundamental ideas about scenery.
Here’s the key point for the first one:
A new scenery format is not the same as new scenery.
This can be confusing because we haven’t changed either our scenery file formats or our scenery in quite a while, and often the two change together. Let’s break this down.
A scenery file format is the way we represent scenery in our simulator. It consists of several things:
A file format specification, describing how scenery data must be encoded.
A file parser inside X-Plane that can read those files.
A renderer inside X-Plane that can render those files.
Tools that help people encode those files.
My first work for Laminar Research (two decades ago) was building all of those things: I invented DSF files, wrote the DSF reader inside X-Plane (DSFLib), worked on the rendering engine, and created the tools to write the files (DSFTool).
When we talk about just the scenery, this is the final rendered files that people fly with. Remember when we shipped 60GB worth of content in 12.0.9? That was new scenery rendered out with all the latest and greatest data.
X-Plane ships with the “global scenery” – a set of about 18,000 DSFs that ensure land everywhere from 60S to 75N. But this is not the only scenery out there – there’s TrueEarth, HD base meshes, SimHeaven, and scenery made with Ortho4XP.
Lots of people can make scenery, often in many different ways (using land class, orthophotos, autogen, etc.) but all of that scenery must be in X-Plane’s scenery file format, e.g DSF.
The scenery is the turtle and the scenery file format is the shell. The scenery can only be as complex as there is capacity in that file format (shell).
So the first part of my talk was a tour of how we have outgrown DSF, and pointed out that there are some things that DSF can’t do. For example, several add-on makers want to stream custom scenery, but DSF makes that basically impossible. DSF also isn’t meant for really high detail vector data, so we’ve been having trouble using all of the latest OSM imports.
The second talk discussed our plans for a replacement to the base mesh file format, which is based on raster tiles. This part of the talk said nothing about what kind of scenery we (Laminar Research) would make, which raised a lot of questions.
But now that you understand the the turtle and the shell, you have a lens to understand what we’re saying. This wasn’t an announcement of next-scenery, only an announcement of a bigger shell that will make that next-gen scenery possible.
So the next-gen scenery format is all about potential. The scenery file format limits what is possible for all scenery (both what is built into the sim and add-ons), so we want to raise those limits quite a bit in the next-gen base-mesh format.
The way we are doing that is by moving the base mesh from a vector-centric approach to a raster-centric post; I’ll break that down in another post.
This is a beta so make backups of your work before using!
New features!
Custom Spill Lights
Custom Spill Lights are now implemented (#312)! To use, make a light datablock with the XPlane2Blender light type as Custom Spill. Relevant properties are:
Property
Function
Light Color
RGB color for the light
Light Rotation
Spot light direction
Light Spot Size
The width of spot light
Size
Size parameter (in meters)?
Dataref
Dataref that controls the light
Custom Spill lights can make dataref driven custom omni-directional billboards and directional spot lights. As with Automatic Lights, the goal is What You See Is What You Get.
In this photo the green light is a Custom Spill, where sim/graphics/animation/lights/traffic_light changes the color between green, yellow, and red! The white lights have different widths, all made without doing any math by hand.
Cockpit Device
X-Plane has pre-made high quality GPS devices that are easily accessible to the artist. Thanks to (#481) using one is as simple as making a mesh, and giving it a material with a Cockpit Device set. Pick the device, at least 1 electrical bus (matching Plane Maker), the lighting channel, and if the screen’s brightness should auto-adjust. You’ll get a fully functional GPS device just like that! You can have multiple devices in the same OBJ, as well as use of the panel texture.
The Cessna’s cockpit provides 2 great examples of using cockpit devices.
Cockpit Features UI
Given the changes and additions to our Cockpit Features, the UI has been changed slightly. On the material Properties tab use the “Cockpit Features” to find “Cockpit Regions” and “Cockpit Device”. The updater should adjust the setting for you if you were using regions.
Minor features
(#426) Shadow Blend’s Blend Ratio can finally be set. Thanks @kbrandwijk! (#548) Non-Exporting Lights are back in. They’re intended as “work lights” and will never ever show up in the OBJ! Simply set the X-Plane Light Type to “Non-Exporting” and use freely.
Important fixes
Light Level can now be used multiple times in the same .obj without tricks or getting lucky!
Thanks to everyone who downloaded alpha.1! I’m glad that this series has been so successful and I can wait to see what you make with the new light features! As always make backups!
This alpha contains changes to the updater. Make backups before using!
Global Material Settings -> OBJ Settings
This new version contains one of the most requested fixes (#357, #599) for XPlane2Blender: Normal Metalness, Blend Glass, and Global Tint have been moved out of the Material Properties Tab and into OBJ Settings! The annoying “All Materials in an obj must have the same Normal Metalness/Blend Glass Value” error is gone!
The updater tries its best to guess if you wanted Normal Metalness, Blend Glass, or Global Tint, however it isn’t perfect. If you have to manually correct more than 3 of your OBJ settings, please tell me.
These new settings can be found under the Textures section of the OBJ Settings.
Emissive Panel Texture Only and Panel Mode Selector
#595 Also known as ATTR_cockpit_lit_only, this directive has actually been in X-Plane since 11.10, but now is accessible in XPlane2Blender! It makes a panel only use the emissive “Lit” texture – a great speed boost if that is all you need. This is the perfect feature for computer displays.
For Aircraft or Cockpit export types, look in the Cockpit section for “Panel Mode”. This changes the meaning of “Part of Cockpit Panel” for the whole OBJ. Setting it to “Emissive Panel Texture Only” mode activates the feature. You cannot mix panel modes.
Cockpit Regions
People who use Cockpit Regions will have to manually change “Panel Mode” from Default to Regions to see the UI and have regions export again. A one time fix. A future updater will do it for you. Until then I hope it isn’t too many OBJs to change.
Updater Version History Synchronization
#471 The updater now synchronizes its last version across every scene and new updater functions will not use data cleaning to protect against accidental or purposeful updater re-runs. Simply put this means a safer to use XPlane2Blender (but still make backups). Since this is new updater code, however, it had to be put in an alpha. People with multiple scenes should be especially on the look out for problems. I feel very confident about it however.
With testers and users like the ones XPlane2Blender has we’ll be getting to v4.1.0-rc.1 much much much much faster this time! For the users who requested moving Normal Metalness and Blend Glass, thank you for your patience. I know now just how annoying that error message was! I’ll certainly keep this experience in mind for future decisions and don’t worry, “make it the same across every material” is not going to be chosen again without an extremely important reason!
We did it folks! Long in development (mostly because I kept adding features, not because of many disastrous releases) we can call this complete! A huge thanks to our dedicated volunteers who beta tested, send bugs, e-mails, and trusted their projects and livelihoods to us. We hope to live up to that trust everyday. Again thank you. Also thanks to a few people who contributed code (Premek Truska and @kbrandwijk)!
This release notes page serves as an overview of important new features and changes since Blender 2.79. Since the manual has not kept pace with development, you’ll find documentation on the new features in past release notes. This is being worked on.
Don’t be surprised if you see the words “Exportable Collection/Object”. That name was later changed to “Root Collection/Object”. Read more about that change here.
New Features
Blender 2.80-83 upgrade!
Blender 2.80-83 is supported! XPlane2Blender in Blender 2.90 has been tested by some with positive results but is not officially supported yet.
The UI is almost entirely the same. The replacement for the Layers Mode feature, “Root Collections”, is almost identical in nature (but also much more powerful!). A few unclear property names and poor tooltips have been reworked, but are still in the same place. Unlike Blender 2.80’s large UI change, we have changed much at all.
I have yet to meet an artist who, after learning modern Blender’s UI, wishes they could go back. Have no fear in upgrading!
Root Collections and Root Objects
Blender 2.79’s 3D View’s layer system has been removed and replaced with Blender 2.80’s Collections. To adapt, we removed “Layers Mode” and replaced it with “Root Collections”. It is nearly the same as Layers but is much more flexible. The updater preserves Layers Mode projects and the UI for this workflow is still in the Scene Properties Tab. For old tutorials you can, in general, replace the word “layer” with “a collection”. Read more about the feature here. “Exportable Collection” and “Root Collection” refer to the same concept, don’t get confused by the old name.
You can use Root Collections and Root Objects in the same scene, no need to pick only one workflow
Automatic lights
Automatic Lights makes lighting in XPlane2Blender easy! Automatic lights are aimed for you without the need for tricks, any parameters are filled in automatically or with easy to use Blender properties, and the exporter and UI prevents many types of lighting mistakes. Pick your X-Plane light name and What You See Is What You’ll Get! Read more here.
Knowing what lights to use and how is now easier. The “Create lights.txt Summary” button makes a text block with light names available and what parameters an Automatic Light will be setting in an easy to use format. See the “Feature Controls” table in beta.2’s release notes for the translation between parameter name and Blender control.
Don’t worry, Named and Param light types haven’t been changed at all.
Though it shouldn’t be necessary to read lights.txt anymore, it has been cleaned up quite a bit!
Split Animations
“Split Animations” offers a new way of sharing animations between objects and even .obj files. An excerpt from the manual explains some uses: Split Animations allows you to share animations that are outside the current root collection. *Its purpose is to reduce the manual labor of keeping these shared animations in sync. Some examples uses of this feature include
Synchronizing the motion of a landing gear and wheel in separate OBJs
Synchronizing the movement of complex wings flex and their lights
Synchronizing the movement of the cockpit OBJ’s door handle manipulator and a door OBJ’s door mesh
Having groups of manipulators follow the animation of a parent element of the cockpit
Many useful cases I’m sure no one else has dreamed of yet!
The feature is activated by very specific uses of Collections and parenting instead of checkboxes and buttons, so read the tutorial on how to take advantage of this powerful new feature.
Override LODs
The Levels of Detail feature in XPlane2Blender has been expanded to reduce the amount of clicking required and also to accommodate the removal of Blender 2.79’s 3D View layer system. How Root Objects projects use LODs had to be changed. See details and backwards compatibility instructions in alpha.5’s notes.
Optimizations
Thanks to Premek Truska and @kbrandwijk the Vertex Deduplication algorithm (aka using the Optimize checkbox) is set) takes almost no time at all, even for large .objs
A one time keyframe gathering pass makes Objects with many keyframes export twice as fast as before
Highly optimized string formatting increase the performance of the exporter across its whole write and logging operation All optimizations work automatically, no work is needed to enjoy the benefits! A full export of the BD-5J on my computer now takes 14 seconds over 2.79’s 30. Other people report similar or better results.
Removed
Layers Mode removed due to Blender’s choice (but Collections are far better anyways so this isn’t a big loss).
Blender removed Texture Slots so we removed autodetection and certain texture validations. For now the root’s texture file path string properties must be used.
2.79 support. Additionally v3.5.1-beta.2 is cancelled (not that anyone ever asked for it of course). Future versions of the converter will consider Blender 2.79 a pit stop before Blender 2.80.
Include In Export, Export Mesh In Layers for being redundant and broken
Again, thank you so much to everyone who uses this addon and sends screenshots and bug reports and e-mails and forum posts and stories of all the great things you’re making. X-Plane wouldn’t be X-Plane without talented people like you making the world real with Blender and XPlane2Blender!
By fixing #571 and #569 aircraft with complex animations should receive seconds off their export time! It basically marks the end of the “can squeeze more out of Python” quest which is good (and a bit sad).
Drag Rotate Bug Fix
Fix for #578. For those struggle with trying to make things like trim wheels that rotate multiple times around (going from -1440 to 1440), this beta fixes it. Please let me know if this fix broke other manipulators.
A tiny UI fix
#570 is fixed for anyone who was bothered by this. Yes, even the small bugs matter to me I swear!
Once confirmed that these have gotten the fix, we’re at rc.1!
As always, remember that this is a beta, so make back ups!
We’re Going Back To Saying ‘Root Objects’
#544 – it appears the “Exportable Collection/Object” naming scheme was confusing. We’re going back to 2.79 phrasing “Root Collection/Object” in the UI, error messages, bug reports, feature requests, e-mails, everything. If I make a mistake please correct me quickly!
Old Name
New Name
Exportable Roots
Roots
Exportable Collection
Root Collection
Non-Exportable Collection
Organizing Collection
Exportable Object
Root Object
Non-Exportable Object
Object
Automatic Lights
Our bright and shiny new fun feature is here! Automatic Lights (also called WYSIWYG lights) replaces Named and Param lights. It fills out parameters and aims directional lights for you in the most efficient way possible! No more doing math by hand, no more reading the lights.txt file, no more being scared to experiment! The UI guides you and prevents mistakes! Automatic lights are the new XPlane2Blender default light. Download this example to see many different uses of this feature!
Feature Controls
We’ve tried to make this as intuitive as possible, but just in case you’re not sure what fills in what parameter, see this table:
Blender Property Used
PARAM NAME
Light’s color picker
R, G, B
XPlane2Blender’s Index property
INDEX
XPlane2Blender’s Light Size property
SIZE
Spot Light’s Rotation (in any mode)
DX, DY, DZ
“Omni” if light is a POINT else Spot Shape > Size
WIDTH, DIR_MAG
XPlane2Blender’s Flash Frequency property
FREQ
XPlane2Blender’s Phase Offset property
PHASE
Any other parameter found in lights.txt can’t be changed in Blender.
Limitations
Some very very old lights are not compatible.
This is currently an opt-in feature only. An updater for old named and param lights would be difficult and limited and likely won’t happen.
A dataref search window style feature feature is planned, but currently to get a list of lights and their parameters you’ll need to click “Create lights.txt Summary” and look at the internal text block “lights.txt Summary
The lights.txt included is the same content as X-Plane’s lights.txt, with slightly different formatting. Your .obj will be completely correct in X-Plane, but X-Plane’s old and currently shipping lights.txt is not compatible with XPlane2Blender. This will hopefully be corrected soon.
Please send me pictures and stories of what you’re trying now that the light system is more easy! I can’t wait to see more sparkling lights across the skies and airports of X-Plane!
As always, make backups! Well folks, we are finally feature complete, very stable, and as bug free as we can be at the moment! This update was for some small bugs and to wait and see if there would be feedback on alpha.6.
Features
When attempting to export a project with no exportable collections or objects you’ll get an error asking if you forgot to check any Exportable Collection|Object checkboxs, Hopefully this helps beginners find those checkboxes and get started.
Bugs Fixed
#504 Where, when the collection algorithm starts in the middle of the tree, walks up, and re-enters the exportable collection it started in, it now respects the visibility and ensures the XPlaneBone is re-used. For an example of this weird case see parent_out_of_collection_snake_out.test.blend
#514, #536, #538 which all related to making the Hidden or Visible feature work as intended – not including what shouldn’t be included. Hidden Lights with the Default, Strobe, Traffic, or Flashing lights no longer include the VLIGHT table entry. ATTR_manip_ is not included across split animation OBJ boundaries, Objects hidden by other means than their or their parent’s visiblility are still counted as hidden (“True WYSIWYG”)
Sorry for keeping XPlane2Blender in Alpha for so long. I develop everything with the knowledge that the artist community will jump on the latest and greatest as soon as it is out there and strive for the greatest possible quality at all times. In the future I think we’ll not use the label as much. It caused a lot of concern about whether or not it should be used, even when it was quite ready.
XPlane2Blender is developed using Blender 2.80 because some Linux Distros are slow to update Blender, and some people are slow to update Blender versions. If you are using Blender 2.82 it should work, but, tell me how it goes. I haven’t heard anything bad yet.
As always, please report any bugs you find and thank you for using XPlane2Blender!
As always, make backups! This includes making backups before saving a 2.79 file in 2.80. There is no going back from that!
Override LODs feature and backwards compatibility
A quick LODs recap
This release includes the last must-have feature for 2.80 – LODs. For those who haven’t used LODs before, it stands for (Levels of Detail).
It is a way of defining what to draw when the camera is a certain
distance away from an OBJ. In XPlane2Blender we call those ranges
“buckets” and put Blender Objects (meshes, lights, armatures, empties)
“inside them”. For instance, suppose an OBJ of a hanger with 2 defined
ranges (buckets) 0 to 200 and 200 to 400. It has two meshes
“HangerDetailed” (bucket 1) and “HangerLowPoly” (bucket 2).
When the camera is close (0 to 199 meters), only the “HangerDetailed”
mesh will be drawn. When it is far (200 to 399), only “HangerLowPoly”
will be drawn. At 400 meters and above, nothing will be drawn. This is
very useful for increasing the performance by not drawing what is too
far away for the pilot to see.
There are two styles of Level Of Detail: Additive (where every bucket
starts at 0) and Selective (where the end of one bucket and start of
the next are equal (see example above)). Use one or the other depending
on what you need to draw when; don’t mix between them.
LODs Mode in XPlane2Blender
Using LODs in XPlane2Blender is very easy: Define the ranges
(buckets) in the OBJ settings, and tell Blender Objects which buckets
they should go in.
“LODs Mode” is considered On when you have specified a number of LOD buckets in the OBJ settings.
This means that:
LOD validations are on (see later section)
Objects must be placed into defined buckets or they will not be
drawn. This is where the new “Override LODs” checkbox on Blender Objects
comes into play.
[Picture]
The LODs feature works just like 2.79’s Layers Mode LODs, with a new
time saving feature and some very small backwards compatibility notes.
Override LODs
In the Object Properties Tab of the Properties Editor, you’ll see a new “Override LODs” checkbox.
When checked, the familiar 4 LOD bucket choice checkboxes will appear.
Specify which buckets an Object will go into. All children below it
will also go into the same buckets. Children who also override LODs will
pass their new choices on instead.
In this picture HangerDetailed has its “Override LODs” checkbox checked,
and has been placed into bucket 1. Its child, WindowDetails, will
automatically be placed into bucket 1 too.
An Object’s LOD choices will only be used if “Override LODs” is checked.
The idea is that just a few objects in the Blender Hierarchy will
need to override the LOD choices, and old projects can be quickly
reorganized to take advantage of this new feature. Inheritance will
greatly reduce data entry.
Backwards Compatibility
There are only a few differences from 2.79 to be aware of
Backwards Compatibility Concern
Resolution
In 2.79 “No buckets chosen = Write to every LOD“. In 2.80 “No buckets chosen = Object not written“
Use the new feature to quickly specify which objects should be in all buckets
2.79 Layers Mode projects did not have an “Override LODs” checkbox, LOD choices hidden
Your old choices are remembered. Use the new Override LODs feature to quickly get the OBJ correct again
2.79 Root Objects Mode used layers 1-4 to put Objects into buckets
Quickly restore your project like this: Make an Empty for each layer
used, override the Empty’s LOD buckets, and re-parent the objects so
that all Objects are in their former buckets. If an object was used on
more layers, make more empties to organize these cases
LOD Validations
For mistakes like not starting the first bucket at 0, having gaps or
overlaps between ranges, and etc, an error will be emitted in the log.
Other Bug Fixes
#518 – When Export Type is “Cockpit” LOD buckets is now shown.
#512
– “Deck” (a rarely used Material property which controls how planes
collide with this surface and the area beneath it) is now renamed to
“Only Slightly Thick” implying that this collide-able surface is “Only
Slightly Thick” as opposed so something that “extends the earth up to
that surface”. Imagine a bridge that could not be flown under because
under it is “an invisible mound of rock and world surface” vs a bridge
that can be flown under because “its deck is only
slightly thick”. Get it? Well, maybe the new name will make more sense
to everyone without needing to know this example.
#511 – People will now be able to use a Particle System File again
#276, #525
– Some UI cleanup. “Scenery Properties” will no longer be shown when
Export Type is Aircraft or Cockpit, “Autodetect Textures” is no longer
drawn because it doesn’t do anything and confuses people on how to use
an essential feature: specifying texture paths. The value isn’t removed
and one day we’ll have that feature back in some form.
Many thanks to all the people who e-mailed me and helped me design
the new LOD feature. This is an alpha feature so if the overall response
is “this isn’t doing what we need” it can be ripped out, but, I think
it will meet expectations. Also, especially tell me if it difficult to
take a 2.79 project with LODs and make it work again. The goal is that
it an artist should be able to do it in about 5 minutes without the use
of an updater for most projects.
I’m so glad to be working on a project with users who are very
willing to send in good bug reports and great constructive feedback and
criticism! Very refreshing! As always, if something goes wrong write a
bug report (preferable) or e-mail me!
Over the past month people have reported to me all sorts of weird things:
OBJ geometry moving with every export
Blender Objects moving after every export
Animations spiraling out of control
Even simple renaming Blender Objects and restructuring of the Blender hierarchy might trigger a bad export.
And the only cure was to apply all transformations or restart Blender!
Thanks to your bug reports and e-mails I was able to track down the source: all these problems were facets of the same bug!
To make a long story short: I have a fix for XPlane2Blender, but
other Blender exporters and users may not be so lucky. I’ve filed a bug
with Blender itself, but that is nothing to worry about for X-Plane
artists.
Please continue to send me your bug reports! I do read every one of them, even if I don’t get back to you right away.
