One of the major initiatives of X-Plane 10 was to add buildings to our airports – terminals, hangers, warehouses, control towers, etc. In X-plane 8 and 9 we had detailed taxiway line markings, lights, and signs, but most airports were devoid of structures. The plan we came up with went something like this:
- Create a library of “lego brick” airport buildings. The library provides visually high quality, high performance art-work with moderate variety and ships with the sim.
- Extend WorldEditor to edit DSF overlays as well as airports.
- Extend the global airport database that Robin Peel maintains to include buildings and not just apt.dat data, so that we can collect, share, and redistribute the buildings.
- Release airport building layouts (created by users, contributed to the database, using the lego brick buildings) as part of the sim.
In other words, take the process we have been doing all along for taxiways, and extend it to buildings. (Some would call this “crowdsourcing” the airports, but Robin has been collecting user-created airport layouts for us since long before such terms existed.)
So where are we on the project? As of now, it is more or less completed!
- X-Plane 10 shipped with the airport library.
- WED 1.2 provides the tools necessary to create airports using those lego bricks.
- We have a provisional process to collect those layouts.
- The first 250+ airports are now included in X-Plane 10.25 beta 1.
Next Steps
So what do we do next? We are working on a few separate initiatives.
- Documentation, Instructions, and Tutorials. We are working on videos, tutorials, etc. to help explain the process. Some of the docs are done now – for example, Marty and Nick just finished their 13-part WED tutorial, which is a great introduction.
- Automation and infrastructure. We’re looking into what we can do to provide a more automated submission process, real-time viewing of what has been submitted, etc. We haven’t made any decisions yet; we’re evaluating options.
- More library variety and WED features. We wanted to wait on library extensions until we could see how people were using the existing elements; it makes sense to add more variety in popular categories. WED 1.3 has additional usability features to make navigating the library easier.
How to Participate Now
We’ll have better documents on “how to participate” in the near future, but for now if you want to create your own airport and share it with the community, the basic process is:
- Get WorldEditor 1.2.0 (or 1.2.1 beta).
- Watch the tutorial on how to create an airport.
- Use “Export for Global Airports” to export the airport – you’ll get a zip file.
- Email that zip file to robin at x-plane.com for inclusion in the sim.
I just made the X-Plane 10.10 beta live only to realize that: some of the airplanes are missing. We are reorganizing where the fleet is stored, and the installer didn’t kit them properly. So:
- If you ran the update and are now missing some aircraft, you can update back to beta 9 to get them, or wait until tomorrow.
- A new beta will be posted tomorrow with the aircraft all present, just in their new locations.
- Beta 9 is “live” again until I can get things repacked – should be less than 18 hours.
Sorry about the confusion!
Edit: And…it looks like there’s something weird going on between our software that creates the patches and Code Signing (which we’re just starting to do for Mountain Lion users). So…it might take a little longer than expected to get this beta kitted.
Having such a small development team, it’s very difficult for us to test ALL conditions in which X-Plane is being used. Each of our offices are littered with joysticks, laptops, desktops, loose video cards, hard drives and various other hardware. We do our best to always be making steps in the right direction but it’s a losing battle to keep our testing internal. A prime example is the recent Joystick debacle. I rewrote the entire Joystick subsystem on 3 platforms and tested it on all 3 platforms on all joystick hardware I own, and Ben and Austin did the same. We worked out all known bugs and then shipped it…and immediately we found hardware that caused crashes 100% of the time. Once those were fixed, other devices were still broken…some devices didn’t even meet USB compliance and we had to work around that issue as well. I can’t possibly own every USB Joystick device on the market so how on earth do we get the product to be as stable as everyone wants it to be?
The answer is simple…we rely on ambitious and courageous users to take the plunge and help us test it. Being a Beta tester is a bit of a thankless job. Sure you get an early look at the new goodies but it can be very frustrating since often, you can’t enjoy them without crashes, artifacts and other annoying bugs. Most users are happy to provide their feedback via the bug form, and now with the new automatic crash reporting system, we’re finding and fixing crash bugs in record time. A prime example is the Joystick crash. Within minutes of releasing 10.10Beta 1, I was aware of the crash and had all of the information that I needed to solve it. And within 24 hours, we had a new build that took care of it. But that’s only useful for crashes, it doesn’t help us learn about visual artifacts or other problems. That’s where we need YOU.
After reading through the forums, it’s become clear to me that many people are just not exactly sure what it means to Beta test software so I’ve come up with my own list.
- Keep two copies of X-Plane. If you care to ever fly X-Plane for enjoyment, ALWAYS keep two copies of the simulator on your hardware. One on a stable release that you use for enjoyment, the other on the latest beta for test purposes only. Your sanity will thank you later.
- Don’t use the beta for recreation time. If you’re only in the mood to fly and have fun that night, stay away from the beta copy unless you don’t mind your flight getting cut short. Too many users expect to fly for 6 straight hours and become enraged when the sim crashes on short final. That’s Murphy’s law!
- Always stay on the LATEST beta. Users often say “The latest beta broke XYZ, how do i go back to the previous beta?”. The answer is you don’t and you shouldn’t. That defeats the whole purpose of beta testing! During a beta process, the code is often changing very rapidly. If we’re on Beta 6 and you’re still submitting bugs on Beta 2, you’re wasting your own time and you’re not helping the product. It’s important you keep up with development.
- Blame us, not your system. If it worked in Beta 3 but it’s broken in Beta 4, do NOT tear your machine to pieces trying to figure out the cause of the problem. Report it and let US figure it out. Time after time, I see users change their drivers, install service packs, uninstall and reinstall X-Plane, update their OS etc trying to solve the bug…when it’s probably OUR bug…not your computer’s.
- Beware of the placebo effect! _EVERYTHING_ affects frame rate. Time of day, clouds, location, aircraft, view angle, view direction, addons, plugins etc etc. Sometimes we’ll release a simple patch that does nothing but fix one bug that was unrelated to the rendering pipeline. I’ll poke my nose in the forums to get some feedback and see 10 users who all of a sudden claim to see some massive fps increase in performance and 10 users who see a massive decrease in performance. I role my eyes at the claims because it’s not a controlled experiment. For example, if you’re the type of person who flies with real world weather enabled, flying one day with clear skies may get you 60fps while the next day you may see 40fps with overcast. Heck, even departing from a different runway than the previous day can result in different frame rates. Running with the command line fps_test IS a controlled experiment. THAT’s the only way to be sure you’re seeing a change in performance. Don’t trust your instincts, gather data in controlled experiments.
- Always read the release notes (http://wiki.x-plane.com/beta) for each new version. They’ll tell you what bugs are supposed to be fixed as well as what new features have been added. If you see a bug that’s claimed to have been fixed yet it’s still happening to you, that’s when you write a new bug report. If we didn’t claim to fix it, it’s probably not fixed in that particular beta.
- We need a Log.txt! When submitting a bug, ALWAYS ATTACH the Log.txt from THAT sim run. Even if you don’t think the Log.txt is applicable, attach it anyway. Tell us in as much detail as you can what you were doing at the time.
- We already know about your crash. Because of the new automatic crash reporter, there’s no need to submit a manual bug report for crashes! Let the system do its thing. If you remembered some details after the fact that you want to tell us, then go ahead and file a report. Of course, if the auto-crash reporter didn’t load up for whatever reason, that’s a good time to file a manual report.
- Tell us who you are and what you know. We really appreciate users who enter their email and comments into the crash report form. It gives us a way to contact you to get more information. This was invaluable to me when attempting to fix the joystick issues earlier in 10.10. A handful of really helpful users made the research a breeze.
- Be objective, not emotional. No one hates bugs more than we do but writing a rant instead of a bug report may make you feel better, but it makes me grumpy. And when I get grumpy, I attend seminars…and when I attend seminars, I feel like a winner. And when I feel like a winner, I go to Vegas…and when i go to Vegas, I lose everything. And when I lose everything, I sell my hair to a wig shop….Don’t make me sell my hair to a wig shop.
- Stop staring at the fps counter! It seems that many users are obsessed with their fps counter. “It went up this beta. It went down this beta. It stayed the same this beta.” Yes, the framerate is ONE useful metric to determine the software’s performance but it’s not the ONLY one and often it’s not the right one. Turn it off and fly! Enjoy the simulator. Sometimes there’ll be a bug in one beta that increases fps as a side effect because the sim is no longer drawing what it’s supposed to. Suddenly, some users are thrilled to see a 20% boost in performance and perhaps they don’t notice that half of the usual streets aren’t being drawn any longer. Ben fixes the streets and in the next Beta, all of a sudden they lose their 20% performance gain and are outraged that we “broke things again.”
I have spent almost the entire last week looking at ATI performance on Windows in depth; this post summarizes some of my findings and what we are now working on. Everything on this post applies to ATI hardware on Windows; the ATI Mac driver is a completely separate chunk of code.
Forgive me for quoting an old post, but:
As with all driver-related problems, I must point out: I do not know if it’s our fault or ATI’s, and the party at fault may not be the party to fix it. Apps work around driver bugs and drivers work around illegal app behavior all the time…that’s just how it goes. Philosophically, the OpenGL spec doesn’t require any particular API calls to be fast, so you can’t really call a performance bug a bug at all. It’s ATI’s job to make the card really fast and my job to speculate which subset of OpenGL will cause the card to be its fastest.
This proved to be true – most of the ATI performance problems on Windows involve us leaning heavily on an API that isn’t as fast as we thought it was, but that’s not really a bug, it’s just the particular performance of one driver we run on. The solution is to use another driver path.
Cloud Performance
I’m going to try to keep this post non-technical; if you are an OpenGL nerd you can read more than you ever wanted to know here.
With 100,000 cloud puffs (this is a typical number for one broken layer, looking at an area of thicker clouds) we were seeing a total cost of about 9 ms to draw the clouds if we weren’t GPU bound, compared to about 2 ms on NVidia hardware and the same machine.
Is 7 ms a big delta? Well, that depends on context. For a game developer, 7 ms is a huge number. At 15 fps, saving 7 ms gets you to 16.7 fps, but at 30 fps it takes you up to 37 ms. That’s one of the crazy things about framerate – because it is the inverse of how long things take, you get bigger changes when the sim is running faster. For this reason I prefer to think in milliseconds. If we can get a frame out in 20 ms we’re doing really good; if it starts to take more than 50 ms, we’re in real trouble. You can think of 50 ms as a budget, and 7 ms is 15% of the budget – a number you can’t ignore.
The ATI guys pointed us to a better way to push the cloud data through to the card, and the results are better – about 3 ms for the same test case. That should make things a bit better for real use of the sim, and should get clouds out of the “oh sh-t” category.
Now there is one bit of fine print. Above I said “if we weren’t GPU bound”. I put the sim through some contortions to measure just the cost of the geometry of clouds, because that’s where ATI and NV cards were acting very differently. But for almost anyone, clouds eat a lot of fill rate. That fill rate cost is worse if you crank the rendering setting, run HDR, run HDR + 4xSSAA, have a huge monitor, or have a cheaper, lower compute-power card. So if you were CPU bound, this change will help, but if you don’t have enough GPU power, you’re just going to be blocked on something else.
(A good way to tell if you are fill rate bound: make the window bigger and smaller. If a smaller window is faster, it’s GPU fill rate; if they’re the same speed it’s your CPU or possibly the bus.)
At this point I expect to integrate the new cloud code for ATI Windows into the next major patch.
Performance Minus Clouds
I took some comprehensive measurements of framerate in CPU-bound conditions and found that with the “penalty” for the existing clouds subtracted out of the numbers, my machine was about 5% faster with NV hardware than ATI hardware. That may represent some overall difference in driver efficiency, or some other less important hardware path that needs tuning. But the main thing I would say is: 5% isn’t that much – we get bigger changes of performance in routine whole-sim optimization and they don’t affect all hardware in the same way. I have a number of todo items still on my performance list, so overall performance will need to be revisited in the future.
The Cars
The other code path in the sim that’s specifically slower on ATI cards is the cars, and when I looked there, what I found was sloppy code on my part; that sloppy code affects the ATI/Windows case disproportionately, but the code is just slow on pretty much any hardware/OS combination. Propsman also pointed me at a number of boneheaded things going on with the cars, and I am working to fix them all for the next major patch.
So my advice for now is to keep the car settings low; it’s clear that they are very CPU expensive and it’s something I am working on.
Fill Rate
One of the problems with poor CPU performance in a driver is that you never get to see what the actual hardware can do if the driver can’t “get out of the way” CPU-wise, and with clouds having a CPU penalty, it was impossible to see what the Radeon 7970 could really do compared to a GTX 580. Nothing else creates that much fill rate use on my single 1920 x 1200 monitor.*
I was able to synthesize a super-high fill-rate condition by enabling HDR, 4x SSAA, full screen, in the 747 internal view. This setup pushes an astonishing number of pixels (something that I am looking to optimize inside X-Plane). I set the 747 up at KSEA at night so that I was filling a huge amount of screen with a large number of flood lights. This causes the deferred renderer to fill in a ton of pixels.
In this “no cloud killer fill” configuration, I was able to see the 7970 finally pull away from the 580 (a card from a previous generation). The 7970 was able to pull 13.4 fps compared to 10.6 fps, a 26% improvement. Surprisingly, my 6950, which is not a top-end card (it was cheaper than the 6970 that was meant to compete with the 580) was able to pull 10.2 fps – only 4% slower for a significantly lower price.
In all cases, this test generated a lot of heat. The exhaust vent on the 7970 felt like a hair dryer and the 580 reached an internal temperature of 89C.
CPU Still Matters
One last thing to note: despite our efforts to push more work to the GPU, it’s still really easily to have X-Plane be CPU limited; the heavy GPU features (large format, more anti-aliasing, HDR) aren’t necessarily that exciting until after you’ve used up a bunch of CPU (cranking autogen, etc). For older CPUs, CPU is still a big factor in X-Plane. One user has an older Phenom CPU; it benches 25-40% slower than the i5 in published tests, and the user’s framerate tests with the 7950 were 30% slower than mine. This wasn’t due to the slightly lower GPU kit, it’s all in the CPU.
The executive summary is something like this:
- We are specifically optimizing the cloud path for ATI/Windows, which should close the biggest performance gap.
- We still have a bunch of performance optimizations to put in that affect all platforms.
- Over time, I expect this to make ATI very competitive, and to allow everyone to run with “more stuff”.
- Even with tuning, you can max out your CPU so careful tuning of rendering settings really matters, especially with older hardware.
* As X-Plane becomes more fill-rate efficient it has become harder for me to really max out high-end cards. It looks like I may have to simply purchase a bigger monitor to generate the kind of loads that many users routinely fly with.
Working on the installer is neither exciting nor glamorous work, but it is inescapably important. If the installer doesn’t do its job right, nothing else in X-Plane matters, because you can’t fly the sim until you have the sim. This is particularly true since we push out regular patches during the life of the product.
X-Plane 10.04 will complete it’s beta cycle in the next week, and this update includes a new installer with a fairly major change: the DVD installer (which also adds and removes scenery) and the net updater are being merged. (The demo installer remains separate.) The rest of this post explains what’s going on and why.
The updater and DVD installer were originally separated a few years ago so that each update/install product could do exactly one thing. We kept getting tech support calls where users tried to update and instead installed a demo copy. By having each tool do one task and only one task, it helped users to do that one task without error.
Fast forward a few years. People have faster broadband, we have more server bandwidth, and we’re using OpenStreetMap for our scenery; scenery updates are going to be part of the product. (We have already patched 5 DSFs that had major defects when originally produced.)
So now when you go to add or remove scenery, running a net update may be a necessary step to finish the scenery. That is, you need to get new scenery from you DVD, and then grab any newer patches off the web. By merging the updater and installer, we can build an installer that provides updates immediately, eliminating a second step. (This also means that when you first install the sim, we can update to latest before you run.)
The new installer/updater will also remove the “repair installation” option (which is now replaced with “update installation). The old repair option restores an installation back to the DVD version, which is almost always older than the net version, and thus undoes bug fixes (which is hardly a “repair” at all). A net update should be more useful.
Update from the about box is still supported – when you update from the about box, the sim downloads the latest installer/updater and runs it, skipping the main menu and going directly into an update.
The new installer can be found here for Mac, Win, and Linux. The demo installer is here. Links to the updater now simply grab the installer.
Last week I received a number of emails about the technical details of v10 scenery. My short answer has always been “please wait for the docs” – that is, I am trying to spend more time writing docs quickly for everyone and less time writing one-off emails. I was hoping to be further along in the doc process, but at least I can say I had a little bit of time to work on WED last week.
So I apologize to everyone who is stuck waiting, but we are getting closer to a useful package of “stuff” (that is, tools, source code and docs). I think you’ll need both tools and source to take advantage of X-Plane 10.
