WED 1.0 has gone RC. The on
ly change from beta 5 is that I have the latest manual changes from Tom (including Cormac’s illustrations of taxiway signs). Both of them did some great work – the WED manual is a lot nicer than the docs I have done myself for the other tools.
A while ago I posted a tools update…has anything changed? Not a whole lot.
- WED is definitely the future of the “heavy” scenery tools (ones with extensive UI) – it has a lot of infrastructure for features like multiple undo, multiple selection, hiearchial editing, etc.
- The next big feature for WED will be some kind of overlay editing. I don’t think this will happen very soon though – my todo list is pretty out of control.
- In the long term, I think WED may provide a visual way to do MeshTool-like operations. In other words, you’ll be able to build a base mesh in WED by specifying polygons for airports, applying orthophotos, and importing one big DEM per tile.
- I don’t think that WED will ever be a DSF editor – that is, you won’t be able to open an existing DSF, move a single node, and re-save it. This just isn’t high on my priority list.
To this last point, how are you supposed to fix scenery if you can’t edit DSFs? Well, I’ll try to post more on that over the next few weeks, but the short answer is that we need structured edits to source data.
A DSF is a compiled result of a very complex set of processes. The vertex that you adjust in the mesh to fix a mountain top may not exist in the DSF if another user submits an updated airport layout, even if they are fifty kilometers apart. (All parts of the DSF affect each other through the adaptive irregular mesh.) So I don’t want to take user submitted corrections to the final DSF because those changes would be lost at the next render.
We’ve been down this road before – when the V6 global scenery was done, the plan was: now we’ll take user-edited ENVs. The problem was that this plan assumed that we’d never re-render the ENVs again, which proved totally wrong – we re-rendered them at the end of the v6 run with improved algorithms.
What we need to do is identify what components of the source data are reasonable candidates for user editing, and set up processes (similar to Robin’s collection of apt.dat data) to gather data and share it.
(We are looking at OSM – it is still under investigation!)
I’ve blogged in the past about the difference between key frames to animate changes to a dataref and key frames to animate an event. Simply put, key frames let you relate the position of your mesh to a dataref value, but they do not let you start a sequence of animations based on a change to a dataref.
In other words, you can make an aniamtion that puts the gear handle up when the gear dataref is 0 and down when the dataref is 1. If that dataref were ever 0.5 you’ d see something in the middle. (This dataref will always be 0 or 1 though.)
But you can’t use animation to make the handle slowly move from up to down over a few seconds when the dataref instantly jumps from 0 to 1. If the dataref moves instantly, so does your animation. There can be no time delay between the dataref and animation.
To get this “triggered sequence” effect, you need something else to detect the event and create a dataref that moves slowly. That something could be a plugin; X-Plane also provides a built-in mechanism: the “misc. slider”.
In an attempt to put the information somewhere useful, I have described the feature on the X-Plane wiki.
No Spare Datarefs (Usually)
In that last blog post I went on a rant about “spare” datarefs. Basically people would ask us to make some datarefs that did absolutely nothing, simply to wire parts of their panels and animations together.
And while I’m going off on a tangent about authoring and datarefs, we will not be adding any “spare” datarefs.
Well, the slider_on and slider_ratio datarefs are sort of “spare” datarefs – they’re 24 slots that have no default function.
I relented on my usual hard-line position on this (and a few other) datarefs for two reasons:
- These datarefs are tied to a real (albeit really simple) bit of simulation code – that is, the code that moves the sliders over time. We had to add these datarefs to make the slider mechanism accessible.
- Because sliders are an airplane feature, there is a logical scope: the person who writes the airplane gets to decide what the sliders do, and no one else gets to muck with them. (Since the cycle time is an aircraft file attribute, no one else could sanely hope to utilize them.)
I still maintain my original position that if you need a dataref that doesn’t exist at all, you should use a plugin to create it.
Someone in the X-Plane community can write a (really simple) open-source plugin whose sole purpose is to create trivial datarefs for aircraft use from a text configuration file, letting aircraft authors make as many datarefs as they need. (It is possible that this plugin exists and I just don’t know about it.)
Aircraft authors: to make sure that datarefs from differing add-ons do not have name conflicts, Sandy and I established a registry of plugin dataref prefixes. If you are going to add custom datarefs to your airplane, be sure to register a prefix, and only use that prefix. Do not start your datarefs with sim/XXX – that prefix is for X-Plane’s use only.
(Side note: the X-Plane SDK website has been under some kind of automated attack for the last week or so with lots of bogus user sign-ups. I haven’t had time to address this properly; if you are not registered on the SDK Wiki, I may not be able to approve new sign-ups for a few more days.)
I was reading this blog post on the Microsoft Office binary file formats…to quote:
A normal programmer would conclude that Office’s binary file formats:
- are deliberately obfuscated
- are the product of a demented Borg mind
- were created by insanely bad programmers
- and are impossible to read or create correctly.
Hrm – deliberately obfuscated, a product of a demented borg mind, created by an insanely bad programmer, and impossible to read or write correctly…he must be talking about DSF!
But then Joel discusses why the formats are the way they are…tell me some of these don’t apply to DSF:
- They were designed to be fast on very old computers. (Sort of true for DSF – load time on low-end hardware was a DSF design goal.)
- They were designed to use libraries. (True – DSF is a lot less weird when you consider the layered structure, and relatively easy to work with via DSFLib.)
- They were not designed with interoperability in mind. (I’ll get to this later.)
- They have to reflect all the complexity of the applications.
In fact, the only reason Joel sites for byzantine Office formats that doesn’t apply to DSF is:
- They have to reflect the history of the applications.
In our case DSF is a new invention of X-Plane 8, so you don’t see a lot of past history artifacts in it.
The File Format as API vs. Data Container
File formats can serve two fundamentally different purposes:
- A file format can specify a data container, used to save information for use later. DSF fits this category: the purpose of the DSF is to hold scenery data in a compact and fast-to-read format for X-Plane. (The other goal of DSF is to be structured so that we don’t have to change it all the time.)
- A file format can be an interchange standard between two programs, similar to a programming API. DSF2Text text files are like that.
You pay for every design goal with more engineering time – if DSF had to be an interchange standard and a high performance data container, it would be a lot harder to design and probably not do either job terribly well.
X-Plane Owns Its Own File Formats
One of the decisions I made early on in my work with the X-Plane scenery system was to not try to use open standards for most parts of the scenery system. (OBJ8, DSF, these are X-Plane specific inventions.) Does this make me a closed-source fascist? Perhaps, but the intent was not to lock people out of X-Plane.
Instead the goal was to separate the problems of interchange and data storage in the places where the two would come into conflict. What became immediately clear was that we couldn’t meet all of our goals for performance, compatibility, and interoperability for a scenery file format with one giant scheme.
So instead we have DSF, whose goal is to be a cryptic but fast final step in the scenery pipeline, and we have DSF2Text and MeshTool, which take simpler text file formats, meant for interchange, and create DSFs.
The intended user of a MeshTool or DSF2Text text file is another program that is outputting its data format. Thus by forming a number of small links in a chain, no one file format is meant to be the “super-format” that has to do everything for everyone.
I am in Boston visiting family for the 4th of July, so there are a few compatibility bugs that I have seen but probably won’t be able to fix for a few days. Austin is still cranking out betas. So if new betas come out and these are not fixed, don’t panic – they are definitely on my todo list.
- Sceneries converted via FS2XPlane crash on load. It looks like this is due to a bug in the new threaded texture loader – I think you can work around this by turning off “load scenery in the background” in the rendering settings (but then loads will be slow). I have found the area where the bug occurs but haven’t isolated it.
- A user submitted a plane to us that crashes Plane-Maker on open – the panel code gets confused. I haven’t isolated it yet. If you have planes you’ve made, save the pre-920 versions!
- I think Benedikt’s x737 plugin should start working again in beta 2.
We’re about to do something strange: any time now, Austin should announce 920 beta 1.
But what happened to the 902 beta program?
The answer is simple: it turns out we don’t need it.
Both 901 and 902 were tiny releases whose sole purpose was to add new languages to X-Plane. (901 added French/German, 902 Russian). The idea was to get our foreign distributors a localized version of the sim faster than we could do the next major patch.
Well, 920 is here, and the Russian port is still being debugged, so 920 will contain Russian and also a huge host of new features. There will never be a 902 final; simply update your beta to 920 when it’s out – any tiny changes in 902 are baked into 920.
Don’t Panic
As always, my advice for beta 1 is: don’t panic. Remember that we are a small company, and thus we have a small number of computers between all of our employees. The first few betas usually involve one or two serious bugs (the sim explodes and sets your computer on fire, etc.) that only happen on the one hardware configuration none of us have.
If you are a user, consider waiting for a later beta, or upgrade a copy of the sim.
If you create third party add-ons, please get a copy of the beta early and start filing bugs! It is much easier for us to fix compatibility bugs when we get the reports early.
Multi-Lingual
A number of users have offered to help translate X-Plane. With 920 we finally have the “new” localization infrastructure in place; I will finally be able to get users started on this process.
X-Plane 920 will have a pile of new panel features…this video shows the new “spot light” system. Basically you can define up to 3 gray-scale masks that define lit regions on your panel. 3 separate rheostats control the lighting levels independently.
http://www.youtube.com/get_player
I was excited when Blogger got its own video facility – saves me the hassle of keeping YouTube account active. Unfortunately the video quality is, well, “web-ish” at best. What you’re supposed to see is independent control of instrument lighting, panel and dash lighting, and flood lighting.
The performance hit is pretty tame:
- The system works by recomputing the panel texture from sources when lighting or panel brightness conditions change. That doesn’t happen that often, so most of the time, the system costs nothing.
- Panel layers are RLE compressed in RAM when not in use, so they aren’t that big.
- If you have more than one core, the lighting recompute happens on the second core, which makes it virtually unnoticeable.
So the only case you’d notice this is if you crank the panel light rheostat really fast or fast-foward time (from light to dark) on a single-core machine.
I have
blogged about correct airport layouts before, but let me bring the point up again, because it is so important:
- You must create structurally correct layouts (that is, vertices connecting to vertices, not lines) in order to get good rendering in X-Plane. Just because the preview looks okay in WED doesn’t mean your layout is correctly formed!
I wrote some documentation on the scenery site that describes the problems in more detail, with pictures.
(I’ve been trying to create more permanent documentation – it is tempting to simply blog the issues because it’s so easy to throw a blog post up, but after 110 blog posts in 2007, the scenery site is still very thin on the documentation front.)
Do Not Add Vertices To Make Smoother Curves
I really can’t stress this enough: please do not go adding extra vertices in your layout to make bezier curves look smoother in X-Plane. Why is this such a bad idea? Let me count the ways!
- X-Plane will fight you all the way! X-Plane adds vertices to curves (to make them smoother) when it detects large errors. If you have a lot of small curves, the errors are inherently smaller and X-Plane will add fewer points. So the first vertices you add to your curve do almost nothing. You have to add a huge number of vertices to get a marginal improvement in your curve. In the meantime…
- X-Plane will provide variable-quality curve rendering in the future! Curve detail should be a user-controlled setting. X-Plane has to run on a wide range of hardware; any time we can let the user pick rendering quality, this is a win, because it helps bridge the gap between the user who just bought a brand new Core 2 Extreme system with GeForce 9800 and the user trying to keep X-Plane 9 running on his G4 laptop which can’t be upgraded. When you add vertices, you take the decision about rendering quality out of the hands of the user, and force high quality on a user who may not be able to handle it. Adding vertices forces a decision of lower framerate on some users.
- Adding vertices bloats the size of apt.dat. This is not a huge factor for custom scenery, but is a factor for the default apt.dat. Robin received a big pile of new airport layouts, and that’s great. But one risk is that the total size of user submitted data could get out of control. For new layouts made with WED, vertices represent a big chunk of the data. If you are increasing your vertices by a factor of 5x or 6x to improve tessolation, you are bloating the apt.dat file.
- Manually adding vertices to smooth curves lowers the level of abstraction in the apt.dat file. Any time we can have a high level abstract representation of scenery, X-Plane has the freedom to improve rendering in the future. If your layout is made up of a large number of small curves (instead of a small number of large curves), X-Plane cannot tell that those small pieces make up some larger structure; in the future it may not be able to render those layouts as nicely as ones that are made with fewer control points.
In summary, please use the smallest number of vertices to create your layouts. (But always add vertices to ensure that your T junctions are correct!)
I found a disturbing text file on a user’s computer…the user had a P-IV 2.8 ghz CPU and a GeForce 8800GTX. An excerpt:
Day 458 of my captivity. Life continues to be an immense, boring string of idle pauses, punctuated with drawing tasks way below my dignity. I am a GeForce 8 – why doesn’t anyone here respect that? And yet when the user is playing “X-Plane” I spend long milliseconds with nothing to do while that infernal Pentium IV thinks carefully about what triangle I should draw next. Life is so dull. That Pentium IV must be the dumbest chip to come out of the fab plant – surely he is the runt of his wafer. Sometimes I become so despondent that I consider turning my fan off and cooking myself to death to end my misery. Perhaps I will intentionally blue screen the operating system…
Okay, so I made that up. But I’m sure that if you put a GeForce 8 into a P-4 system, that’s about what the card would be thinking – it would be bored silly. Even the fastest Pentium 4s can’t feed data rendering instructions to a GeForce 8 fast enough to utilize the hardware.
And in this sense, hardware can be like pushing on a string. Even if putting worse hardware in your machine would lower fps, putting better hardware in it may not improve fps. If you have a P-IV and a Geforce 8, putting a Geforce 2 MX in its place will lower fps, but upgrading to a GeForce 9800 won’t – it will just leave your graphics card even more bored and angry than it was before.
This is because the slowest component in the graphics pipeline determines fps. Improving the speed of the other components just leaves them idle more of the time.
Why RAM Isn’t like CPU.
When your system is limited by CPU, the change in performance is somewhat linear – get a 20% faster CPU and you get 20% more FPS.
RAM isn’t like that…usually you either have more RAM than you need (and life is good) or you have too little, and your framerates is so bad you want to pull your hair out. Why is that?
Well, the difference between RAM and CPU lies in how we use the resource. If the sim needs more CPU than the computer has, the CPU plows through the work – it just takes longer.
But what happens when you run out of RAM? The backup for RAM is your hard disk, which isn’t even the same substance. A RAM chip might be able to come up with some data for the CPU in 4 nanoseconds; a hard drive might be able to seek to data in 4 ms. That would be a diference of 1,000,000! (By comparison, the difference between flying across the country and walking might only be 150x.)
In other words, when we run out of RAM, we can’t degrade gracefully, we degrade catastrophically.
(It gets worse: when we run out of virtual memory, we don’t have anything to fall back on – we’re just dead!)
Interestingly, it used to be that way for VRAM (back in the days of X-Plane 6); if we had to substitute system RAM for VRAM, framerate died. These days, however, the graphics bus is so much faster that that substitution is almost tolerable – thus you can start to run out of VRAM and not have a slide-show.
We Can’t Use All of Your RAM
A user emailed us to inquire why X-Plane wasn’t using more of his RAM – he had a machine with 8 GB running a 64-bit OS. X-Plane could have had 3 GB memory but was only using a fraction of that.)
The answer is partly in the nature of RAM – because RAM fails catatrophicaly, users are likely to set X-Plane to never get near the edge of running out of memory. Our engine is designed to simply minimize memory usage (since the penalty for running out of virtual memory is fatal) rather than try to gain incremental benefit from incremental RAM. (There are scenery packs that you need a lot of RAM for, but usually you can either run them or you can’t.)
For most users, RAM is like pushing on a string – if you have enough memory that you can run without paging, adding more won’t help. If you have 4 GB you really don’t need more memory. If you have 2 GB, you probably don’t need memory unless you’re running with almost everthing else maxed out, due to a really fast CPU and GPU.
The moral of the story is: just because I dismiss an idea by email doesn’t mean it won’t get coded. In particular, one user requested a generic instrument that could display text from datarefs. While my initial reaction was “no way”, I ended up changing my mind. The text instrument will be available in 920 – it displays a string as long as the input dataref is of type “data bytes”. The first intended use is to let you build an FMC interface out of generic instruments – we’ll make datarefs that provide the display strings.
My initial negative reaction was due to the nature of the request: I get a lot of requests for “can you implement feature X in subsystem Y?” One of the biggest questions about any feature is: where does it go within the maze of SDKs that X-Plane supports? So if you specifically want it in a particular place, the answer is more likely to be no.
In particular, most people want feature X in the subsystem they are already using. So 3-d modelers want a feature in the OBJ format, airplane editors want an ACF feature, panel creators want a new instrument, and off we go.
I can’t put a feature into the sim without considering compatibility, and the cost of compatibility goes way up if a feature is in the wrong place in the code.
Put It In a Plugin
A lot of the time my response is “a plugin can do that.” Now I know that that’s a blow-off answer; plugins make all things possible, but they don’t make them easy. Not everyone is a programmer. At the same time, plugins often anticipate sim features by years: consider that the first planes to demonstrate key frames, 3-d cockpits, and per-vertex normals used plugins, with the sim developing the feature later.
My decision to include a text instrument is based on the idea that it makes sense to partition which “SDK” a feature goes with based on the nature of the activity. As of this writing, custom systems modeling has to be done in a plugin – no other interface provides this. But panels can be built almost entirely using the built-in panel editor; with 920 generic instruments provide a huge amount of flexibility.
So the purpose of the text instrument is to keep things this way – imagine that an artist is building a panel (using generic instruments) around the datarefs written by a programmer (using a plugin to do systems modeling). Without the text instrument, the programmer has to write some graphics code (for the first time) and there’s one part of the panel the artist can’t manipulate. This is a case where since the artist could already do 95% of the job, it makes sense to expand generic instruments so they can do it all.
(That’s not entirely true – there will always be custom displays so complex that they can only be programmed. But the 920 generic instruments make most things straight forward.)
As a final note to the rant: we are not going to provide any systems modeling capability as part of generic instruments. Generic instruments are how you visualize the simulation, not how you configure it.
To give a trivial example of why I don’t want systems modeling code in the generic instruments: it would be unfortunate if the pressurization model changed because the 2-d panel scrolled such that the pressurization instruments were no longer on screen!
“Warning: padding is not integral” is a warning message that the airport sign code renderer can spit out when your airport is loaded. Unlike all of the other warnings that you might see, this one doesn’t actually tell you that there’s a problem with your sign. It’s actually telling you that there’s a problem with my code, and today I finally fixed it. (The fix will ship in X-Plane 920, not X-Plane 902.)
The problem is that the official FAA sizes (0.76 meters, for example) suffer from tiny rounding errors when put through the sign creation process. (Use 0.33 as a proxy for 1/3rd. Divide something in thirds and add it up. You lose a tiny sliver, right?) That warning message was the renderer noticing that it was being asked to fill in a really tiny sub-pixel area (which it then did, hence you can’t see a problem from your airplane).
With 920, the sign heights are changed ever so subtly (by less than 0.1%!) so that they don’t cause rounding errors. This is the equivalent of calling the sign 0.33 meters instead of 1/3rd meter.
If this all seems like decimal point hell, well, welcome to my world!
Bottom line: you can ignore this warning, it’ll go away in 920.
