This post will pretty much only be of interest to Linux users – everyone else, talk amongst yourselves. Topic of discussion: X-Plane.com is neither a plane nor a communist. (Sorry, work on the road processing code has drained me of any remaining sanity.)
Linux History
The brief history of X-Plane for Linux is that Joshua Wise ported X-Plane to Linux a while ago because he felt like it (and being too young to drive at the time, apparently had some free time). The Linux port was never a commercial effort. – Laminar Research didn’t say “we’ll make a ton of money by porting to Linux, so let’s hire someone”. It just sort of happened. (Well, I wasn’t in the room so I don’t know exactly what was said.)
Linux has steadily represented about 5% of our user base from that point on, and this gets to the “commercial problem” of Linux: even with the work we do to try to keep platform specific code in X-Plane to a minimum, Linux sales don’t justify the development cost to maintain the platform.
Yet there still is a Linux version of X-Plane. We keep renewing the Linux port for “one more version” because while the Linux community doesn’t buy a ton of copies of X-Plane, Linux users do contribute in an outsized way by providing code back to the scenery tools and other development efforts. In other words, the Linux community doesn’t contribute dollars, but they contribute a lot of code.
Code Contributions
This came up during the X-Plane 9 and 10 Linux discussions. Our primary request to Linux users has been “please support each other”. There are too many Linux distributions for us to provide support; our dedicated support staff does not even have Linux expertise, so Linux support immediately hits engineering.
While a number of Linux users have really done a great job of helping other users out, another issue came up. One of the ways we try to minimize Linux-specific costs is to keep the OS-specific code very basic and rudimentary. In the past we’ve taken patches from users to fix OS specific bugs, but this is an awkward process at best.
Some Linux users asked if we could find a way to release source to the OS-specific parts of X-Plane so that they could improve the Linux-specific code (and thus the platform-specific experience) themselves. Right now this is a very awkward process: I have to tell the user “give me a snippet that does X” and then hope that I can get the snippet integrated and tested on my own.
One of the suggestions that came up at the time was to separate out some of the code that abstracts the OS/hardware so that Linux users could patch it more easily.
A Hard Abstraction
Separating out the platform specific code is a good idea, but it wasn’t one we were going to drop everything to pursue – if we did, the end result would be no improvement to X-Plane after some number of weeks of heavy development. Instead, we are slowly moving the code in this direction as we work on it.
The first code to get a work-over is the joystick code; Chris has been refactoring the joystick code for a few weeks now. 10.10 won’t show you any new behavior; it runs the existing UI on top of new low level code. Once we are sure this works, Chris can implement a bunch of new joystick UI features on top of the new low level code quite easily.
The new joystick code separates out the OS-specific joystick interface code from the rest of X-Plane. In the long term, this will make it practical for us to share the hardware abstraction code with users as source, take patches, and even allow developers to swap it out on their local development machines via a shared object. We’re not there yet with this level of flexibility, but the new code makes these things possible.
You Can Keep Your Hat On
There is one immediate win to the new code that will be apparent to Linux users in X-Plane 10.10: the new code (which is based on input.h instead of joystick.h) recognizes hat switches as a set of buttons, rather than a pair of axes; this is almost certainly better from a UI configuration standpoint.
I fear one of the main points of my last post was perhaps lost in the excited discussion of how to cope with conflicting crowd-sourced airport submissions. The engineering principle is:
Fix real bugs, not hypothetical bugs. Solve real problems, not hypothetical problems.
By this I mean: it’s invaluable to wait until you actually have evidence of a problem before you design the solution. Without a real data point showing what the bug is, the solution might not actually fit the problem as it actually happens. In a program as complex as X-Plane, particularly when third party content is thrown in, it’s hard to predict in advance what is going to go wrong.
In the previous case (coping with conflicts) the key here is that I want to see a few conflicts before I choose a plan. I see proposals for a revision system, public logs, a registery, a voting system…any one of these ideas might be the right solution, but we don’t know yet because we don’t have two people submitting buildings.
The same principle applies to another issue that was brought up a while ago:
Won’t all of the default airports look the same since they all use the same library?
First, to be clear: we’re not trying to make the default library airports look like the real world airports, we’re not trying to duplicate custom scenery, and if people submit buildings for all 20,000 airports in X-Plane there’s no way we’re going to have 20,000 control tower libraries in the library.
But there is a valid issue: what if people heavily use the assets? Won’t we start to get a “sameness” either between two airports or within an airport as we use the library more and more?
This brings up the second half of “fix real bugs”:
Identify possible solutions, but don’t use them until necessary.
I am not worried about collisions of airport because there are a huge number of possible solutions, many of which are easy to implement and have been proven to work well in the real world in other problems. In other words, the problem of merging crowd-sourced data is a solved problem, and we can use the right existing solution when we have enough data to pick that solution.
The same thing applies to repetition in the library: we have ways of fixing repetition. I want to develop a few airports and see what looks repetitive before asking our artists to spend their time making more stuff* but we at least know how we can fix the problem when the time comes.
Options to Deal With Repetition
So what are the safety valves here? How can we deal with repetition in the library? There are a few ways:
- Every object in the library can have multiple real OBJ models for each “virtual” object. X-Plane picks randomly among the choices, so we can always add more real objects. We already do this for cars. (There are something like a dozen cars and trucks backing up one or two library entities.)
- Some of the library elements are .agp (autogen-point) files – these files are mini-scenes made up of other objects, and the objects referred to within the scene are also subject to library variation. So we can have a mini-scene where some of the decorations are constantly changing.
- The facade engine contains some fairly flexible rules for picking which segments of walls go where, so there’s a lot of potential to break up the order of facade walls.
- We can add more types of facade walls to a given facade without hurting X-Plane’s performance.
All of these changes can be put into an X-Plane update.
In summary, there are a lot of ways we can make the library more diverse and less repetitive, and as we see the library get used (and certain elements become over-used) we can add more variation to keep things looking fresh.
* For example, it wouldn’t make sense for me to ask Tom to make 50 variants of the administration building if almost no one places it in any airport. We need to see what kinds of assets get used the most and then make decisions.
I have updated the WorldEditor page to link to the new builds of WED 1.2 beta 1. This is the first beta of WED 1.2, which we showed at the X-Plane developer’s conference in SC a week ago. A few hilites:
- Finally, a real airport import dialog box – when you import a dialog box, WED will let you pick a subset of airports to grab. So now you can open the entire apt.dat file from Robin and just pick the airport you want.
- There is now a library preview pane for objects, autogen scenes (.AGP files) and draped polygons.
- WED 1.2 supports all of the new v10 overlay editing extensions, e.g. forests as tree lines, facade wall-type control, etc.
Tyler and I are working on a 1.2 manual but it’s still chaotic enough that I think it isn’t even ready as a beta document.
One complaint we hear a lot from tech support is that the 747 knobs are hard to control in the 3-d cockpit. Javier and I did some investigation into this; this post describes what we found, what we are changing, and what why I don’t think the scroll wheel probably shouldn’t be used to affect the 3-d cockpit.
Hard To Drag
The fundamental problem is that it’s hard to control the autopilot knobs in the 747 by dragging with the mouse. Large drags make only a small change in the knob, so it takes forever to dial in an autopilot altitude. You’d think the solution is simple: change the scale for dragging on the knobs, right? Well, not quite.
It turns out that the “sensitivity” of the knobs to dragging is a function of the way you turn your head in the cockpit. Sit in the default 3-d position, turn your head 30 degrees to the right and drag and the knobs turn quickly. But look straight ahead, slide to your right, and drag and they are very slow.
The problem is one of perspective, and this is where it gets interesting for authors. The drag axis manipulator (which lets you make the 3-d cockpit respond to dragging the mouse) measures its drag in meters. But the distance on the screen that the drag distance takes up (in meters) depends on where the camera is placed and at what angle it is turned. This can lead to some very strange effects: in some views, a 500 pixel drag moves the altimeter only a few hundred feet, while in other views, 500 pixels moves tens of thousands of feet.
Screen Space Dragging
For real physical parts like a lever (a part that moves as you drag it), dragging in meters makes perfect sense; it lets authors match their animations to their manipulations.
But for a drag that doesn’t have a real-world correlation (e.g. you drag on the knob and the knob spins but it doesn’t move) having the camera angle affect the drag distance results in panels that can be used only from certain viewpoints.
To fix this, we are introducing a new “pixel” drag axis – unlike the current drag axis, the distance over which the user can drag is specified in pixels, so that the “sensitivity” to the mouse is the same no matter what view angle the user has. I will post details on this when we go beta.
The Mouse Wheel
While I was working on pixel drag axis, I looked at using the mouse wheel to turn knobs, something our users asked for. And while the prototype seemed ‘clever’, after some arguments with Chris I came to a bit of an inescapable conclusion: the mouse wheel for changing parts of the panel is the wrong tool for the job.
The problem with the wheel is that in the rest of the universe it is use to manipulate view information. This is true in X-Plane 10.05 as well (and it works well), but things get quite tricky once the mouse wheel is added in. Some of the problems:
- Making the mouse wheel zoom and manipulate (e.g. if you are over a knob it manipulates the knob, otherwise it zooms) risks surprising results. A user who wants to zoom might accidentally “bump” a cockpit knob, something that’s pretty frightening to a real pilot.
- We looked at requiring once of the buttons to be held down while mouse-wheeling, but that’s not a gesture you see anywhere else in the universe – effectively one of the two uses of the wheel is “buried” and we might as well only use the other. Furthermore, if we are going to require a click, the user might as well just drag on the knob itself.
- If we have to pick one or the other (zoom or manipulate), zoom is by far the most consistent thing, the thing that fits with the host OS.
- If we make the option a preference (e.g optional mouse-wheel on knobs) so few users will enable it that authors won’t be consistent in adding support to their cockpits, and the system will never get momentum. (We can’t just add “mouse wheel automatically” because the sim doesn’t know how much one click of the wheel should change a given dataref.)
We tossed the mouse wheel idea around, but in the end we concluded that the wheel should be a view/zoom/scroll function, not a data change function – we couldn’t find any example apps that used the wheel to change the contents of the screen. In the end, authors need to make clicking work well, and we need to provide manipulators (like the screen-space drag manipulator) to make that possible.
One of our servers is down for maintenance. This means that the following are temporarily offline:
- Scenery tools downloads.
- GIT repos (and the cgit web interface).
The installer-updater is still running – it runs off of a different server. I just fixed a number of broken links to the installer on the main website.
I don’t have an ETA on when the server will be back up, but if it’s not today I will stab myself repeatedly with something sharp.
Just a quick tip on tuning X-Plane’s rendering performance: if you see your fps start off strong and then drop over time to a crawl, with low GPU, turn down the cars.
Here’s the problem: cars are spawned over time. The way the car engine works is that X-Plane comes up with a budget for cars based on rendering settings, and adds a few more every frame as long as it’s under budget; if it is over budget it simply doesn’t replace cars that are killed by being too far from your plane or reaching a dead end.
It can easily take several minutes for car traffic to build up.
And the thing about the car traffic is: it’s incredibly CPU intensive. This is something that I am working on and it may get better for future patches, but for now just consider this: a reasonable setting for cars (for your computer) should be based on the framerate after several minutes, not the one you instantly get when there are virtually no cars!
X-Plane 10.05r1 is now available – click “get betas” and run the installer’s update function to get it. This is a really tiny patch; we needed to fix language on the 32-bit Windows warning, and we took the opportunity to fix hard cockpit walls and the carrier landing while we were at it.
Everything we’ve discussed on this blog for the upcoming beta is coming – but for a future patch in a few weeks. That will be a major patch with a multi-week beta period and real enhancements to the sim. We’re already deep into development on this work; the 10.05 micro-patch is just a quick build to fix a few tiny bugs.
I expect us to go final with 10.05 in a day or two – there are only two code changes, so it should be quick to find any possible problems.
It looks like, for the next major patch, we’ll be able to get a few features in that authors have asked for:
- Exclusion of scenery elements from shadowing for pretty much all scenery elements.
- Use of “conditionals” (see the OBJ8 spec or previous post) to use different textures when HDR is on or off for facades, ter and pol files. (This is available for OBJs since 10.0 – this is an extension to other scenery formats.)
- Texture size limit bumped up to 4096 x 4096 max. (The max panel size will remain 2048 x 2048.)
The one other authoring request that we heard loud and clear in Mallorca but will probably not get to in the next major patch is an option for seasonal textures in third party add-ons. It’s a slightly more complex change and will have to come in a later patch. (But we have moved the priority of this feature way up in response to developer feedback.)
I finally finished up an update to the OBJ8 specification, as well as the forest specification – see here for the documents. These specifications are mostly of interest only to developers who are working on scenery exporters.
The OBJ specification is very thick – here are some of the hilights of what’s new.
Global Lighting
In X-Plane 10, you create global spill lights by attaching a light to an object. (Thus, spill can come from any object-bearing part of the sim – an airplane, custom scenery, etc.)
One way to do this is with the existing named and parameterized lights – these features existed in version 9, but in version 10 there are some spill lights added to the light list.
What may be of more interest to custom authors is the new LIGHT_SPILL_CUSTOM command. This lets you build a completely customized spill light. You control its size, color, shape and direction. You can even optionally run the light through a dataref, giving a plugin custom control of the light.
Draping
In X-Plane 10, an object can contain geometry that is “draped” on the ground – that is, X-plane will subdivide, bend and modify part of your object mesh so it sits perfectly on the ground even if the ground is sloped. ATTR_draped makes this happen.
This feature is a much better alternative to using ATTR_poly_os to make marks on the ground. Draping produces objects with better performance, and the geometry always sits on the ground with no artifacts.
As an added bonus, you can optionally use a second, different texture for the draped part of your object from your regular object. (Internally the draped geometry actually becomes something like a .pol file – this is why it can have its own texture.)
Draped geometry makes it much easier to make airport markings. If you want a parking spot, simply draw it on a quad, make it draped, and drop it into place with WED. Tom uses this heavily in our airport library.
Draped geometry also makes it easier to have ground markings that match the object they come with. For example, if you want a house and the house comes with a driveway texture, you can make the driveway texture a draped quad and when you place the object, the draped driveway is always in the right place.
Global Attributes and Instancing
In X-Plane 10, it is possible to set a few key attributes (blending and shininess, among others) globally for the entire OBJ, rather than using an ATTR_ to change part of the object.
These global attributes make hardware instancing possible. In hardware instancing, X-Plane draws many objects with a single instruction to the CPU. In order for this to happen, X-Plane must be able to draw the entire object without needing CPU intervention mid-object. This means an instanced object has to be free of animation, attributes, and a few other features.
The global attributes let you set things like shininess and still have a single-call draw object, ready for instancing. Alex uses these heavily in the urban autogen, and it really helps performance.
My fear is that global attributes are going to be a source of confusion for authors. When should you use them? How do you add them? This is my thinking:
- Modeling program exporters should allow an author to identify an object as “for instancing” or not.
- Authors should check “for instancing” for any object that is heavily repeated. (A car or a single tree or a static airplane, for example.)
- The modeling program can then try to prefer global attributes for instancing objects but not for regular ones, which should come very close to optimal behavior.
Conditionals
Conditionals are simple logic statements that include or ignore parts of an art file based on the rendering settings. In particular, they let you change an OBJ based on whether HDR is on or shadows are on. For example:
IF GLOBAL_LIGHTING
TEXTURE_LIT my_airplane_hdr_LIT.png
ELSE
TEXTURE_LIT my_airplane_LIT.png
ENDIF
In this example, which LIT texture the OBJ uses will depend on HDR.
Because the conditionals can be used anywhere in the OBJ, you can change any aspect of the OBJ to customize for HDR. You can replace a texture, remove lights, add more geometry, etc.
I don’t know how heavily people will use conditionals, but they give authors the option to make one file tuned for both HDR and non-HDR, shadows and non-shadows.
I think the two most common uses of conditionals will be:
- Providing alternative LIT textures when HDR is on or off. Note that only one texture is ever loaded (when the HDR rendering setting is changed, X-Plane unloads one and reloads the other) so this does not increase VRAM.
- Removing drop shadows that are baked into a model when shadows are on.
That second case would look like:
IF NOT GLOBAL_SHADOWS
ATTR_draped
# This is the shadow geometry
TRIS 300 6
ATTR_no_draped
ENDIF
When global shadowing is turned on, the entire set of draped geometry disappears, removing baked vs. real shaodw conflicts.
X-Plane 10.04 went final yesterday – as always there’s one bug that gets by you. In our case, “solid” cockpits (cockpits that constrain the camera so you can’t move through walls) are inoperative.
At this point it looks like we will do a tiny micro-patch to make 10.05 that fixes this and puts some new strings into the app. Honestly if we didn’t have string changes, I’d probably wait for the next real patch for the cockpit fix, but I believe that it is a low risk change if we must cut the app anyway. The main goal of 10.05 is to get the language just right in some of the dialog boxes for the next set of DVD masters.
The amount of change from 10.04 to 10.05 will be very tiny – comparable to 10.04 rc2 to 10.04 rc3 – literally one or two lines of code changed total.
In the meantime we are working in parallel on the next big patch.
Work on 64 bits is underway – I do not believe it will be available in the next big patch, and we have not specified any date (soon or far away). Simply put, I am staying out of the business of release dates for work like this. Our internal estimates have huge margins of error, due to the many unknowns of the process, and I don’t think we make anyone happy by saying when the feature will be available and then being wrong.
Who Will Use 64 Bits
When X-Plane or the installer checks for updates, it sends an identifier (called a “user agent”) to the server. This is a standard part of HTTP and all web browsers and web-communicating programs do this; our servers (we use Apache, like most of the universe) keep a log of the “user agents” calling in, which can tell you what web browsers and other programs are requesting web pages.
Here is an example of two user agent strings from a real access to our web page:
Mozilla/5.0 (iPad; CPU OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2
As you can see, Safari and Firefox are both identifying what version they are and what operating system they are running on. This means the website owner can tell how many people are viewing with older browsers or tablets/mobile phones, and optimize the layout of the site accordingly.
Here’s what X-Plane looks like when it calls up our web server:
X-Plane 10.04r3-3-IBM6.1_64
For as long as we have had auto-check-for-update, X-Plane has include the operating system in its user agent string, just like web browsers do. This means that we can tell the approximate operating system split by looking at the user agent strings on the server.
With X-Plane 10.04 we have moved to the more complete string you see above – in particular, it includes whether the version of Windows is a 32 or 64 bit edition. (The above string comes from Windows 7 64-bit. Why Windows 7 is versioned 6.1 is beyond me. The extra “3” is an indication that the user has a full global edition of the sim – this is a paying user running on Windows 7 64-bit.
So I can tell you that at least for users who have updated to 10.04r3:
- The platform split for full copies of X-Plane remains about 63% Windows, 32% Mac, 5% Linux.
- Among full-copy Windows users, about 85% are on a 64 bit Os – that is a surprise to me, but a good one – it means a lot of users will be able to utilize the 64-bit port. (Among demo users it’s about 75% 64-bit.)
Uh, What Do You Guys Do With This Information?
Not much. We didn’t actually set out to do a hardware survey – rather we put a user agent string into the installer when we first wrote it, because it’s required by the protocol. Apache logs user agents by default, so it was only after running the installer a while that we realized that we could “mine” the log for platform information.
We do not:
- Try to correlate these user agent logs with customer information in any way.
- Archive the logs permanently – as anyone who admins a web server knows, the logs get very big, so the server keeps a few weeks of records, then throws them out. Our only change from the default Apache config is to cut down the retention time because our server gets a lot of traffic.
- Send any personally identifying information in the user agent string or otherwise. The server logs incoming IP addresses, but this is not part of our installer – all web servers know the IP address of the incoming requests; logging them at least temporarily is necessary to be able to identify the source of a web attack should one occur.
