This may have happened to you: you import the latest approved airport scenery pack from the X-Plane Airport Gateway and modify it. When you go to export your improvements back to the gateway, WED says your work is invalid and has a problem — but not in the changes you made!
Huh? If this was the approved airport on the Gateway, how can it also be invalid in WED? How did that other author upload the airport before?
You Get a Free Pass
What do we do if we find that scenery and airports have bad data that is causing bugs? What if we can’t just fix the scenery pack for you? The policy we’ve been using is: “you get a free pass until your next scheduled maintenance; then you have to fix the bug.”
The X-Plane Scenery Gateway is a good example of this. Sometimes we find new categories of authoring mistakes – I write improvements to WED’s scenery validation function to catch these authoring errors. These are errors like:
- Typos in taxiway signs – the old syntax makes a sign with incorrect letters.
- Overlapping ATC routes – with the overlapping routes the AI aircraft do not taxi correctly.
In other words, these are situations where the WED scenery pack, if left alone, is causing clearly bad things to happen inside X-Plane. This isn’t a case of “old style or new style”, it’s a case of “broken or fixed.”
So what we do is we set WED to reject these scenery packs on future uploads, but we do not delete the airports from X-Plane itself. In other words, the airports with serious mistakes get a free pass until the next time an author does scheduled maintenance.
Then when an author is working in WED and is going to update the airport anyway, the author must fix the problems we have found.
No One Likes a Fire Drill
This strategy is a compromise between two extremes:
- If we force you to fix your airport right now (e.g. “fix the airport or we remove it from X-Plane”), that’s not fun, that’s a fire drill. And having been exposed to plenty of fire drills myself dealing with the new OS X 10.11 and Windows 10 releases, I’m sympathetic to authors not wanting to have to stop everything to deal with an issue ASAP.
- If we never require that these kinds of problems be fixed, they simply won’t be fixed. The overwhelming evidence that I have seen from working on authoring tools for X-Plane for over a decade is that some authors won’t fix problems unless the tools force them to do it.**
So requiring the change when the author does maintenance but not requiring the existing shipped scenery pack be modified strikes me as the best possible compromise: we still get quick responses to serious bugs, but we don’t force anyone to drop everything.
* I do realize that some authors are incredibly diligent about getting their scenery to be correct even if the tools don’t force them to do so! But the goal here is to have all scenery be correct; it only takes one broken scenery pack in the hundreds a user installs to crash X-Plane.
I’m declaring WorldEditor 1.4 to be final. From now on, you must use WED 1.4 to upload to the X-Plane Scenery Gateway. If you have not upgraded yet, please do; you can download it here!
I cut a new release candidate for WorldEditor: WED 1.4r2 – please update to this latest build. (1.4r1 had a crash-on-export under a relatively rare configuration of lines in a scenery pack.)
Posted in Tools
by
Ben Supnik |
It’s come to my attention that the Scenery Gateway has a serious shortcoming:
There’s currently no good way to get feedback from other users.
This is a shame, because when you upload scenery to a site like X-Plane.org, you get to hear from real people using your scenery—they might tell you how much they appreciate your work, or suggest ways you could improve it.
There’s a tradeoff here: because all airports on the Gateway are periodically exported to X-Plane, users don’t have to seek out your scenery in order to enjoy it; they get it automatically.
That means your work has a much wider impact—it benefits hundreds or even thousands of times as many users compared to posting to a download site. But, it also means people who use your scenery probably don’t know your name.
So, if you’re a Gateway artist, I’d like to hear your thoughts on a couple things:
- How do you feel about the current situation? Do you want a way to get feedback (and kudos) on your Gateway submissions?
- If so, what would be the best way(s) to give that feedback? A few possibilities I can think of include:
- A “like” button for your scenery pack or the airport as a whole
- Pros: Easy to understand, zero friction for people leaving feedback (means more people are likely to use it)
- Cons: Impersonal (compared to text-based comments like “I love this scenery!”), doesn’t solve the problem of hearing from X-Plane users who never visit the Gateway
- The ability for other users to leave comments on your scenery pack
- Pros: Very personal, the “standard” way to get feedback on other download sites
- Cons: Higher friction (fewer people will use this compared to a “like” button, for instance), doesn’t solve the problem of hearing from X-Plane users who never visit the Gateway
- Automatic estimates of how many X-Plane users see your scenery pack each month/year—for instance, you submit a scenery pack for KBOS, and you see that x,000 people fly there each month
- Pros: Gets “feedback” (such as it is) from users who never visit the Gateway, gives a very clear indication of how many users you impact
- Cons: Impersonal
- Something else entirely?
Disclaimer: As with any discussion of future features, I can’t promise we’ll implement any of the above… but I can tell you we’ll consider it.
So, drop a comment below and let me know what you think!
UPDATE: As of July 11, we now have a basic discussion system on the Gateway which integrates with the existing bug reports. Give it a try and let me know what you think!
I just posted WorldEditor 1.4 – “WED”, as we seem to call it most of the time – release candidate one for Mac, Windows and Linux – see the WorldEditor page for download links, report bugs on the gateway bug reporter.
Besides fixing a few bugs, this build has a new certificate so uploading/downloading from the gateway will work.
WED has a validate function that finds authoring problems in your scenery packs; two notes on this function:
First, I have been trying to improve the validation every time an issue comes up with a scenery pack. Therefore WED 1.4’s validation is quite a bit more strict than 1.3’s validation.
Once WED 1.4 goes final, it will become the required version to upload to the gateway. This ensures that we have the strongest validation, and saves Julian from having to hand check authoring issues.
This means that when you go to make a modification to your existing work, you might not be able to resubmit it until you fix existing problems. Your project didn’t change, the validation just became tighter.
Second, I have added the ATC layout error checks to the validation step. These checks (select crossing routes, select doubled nodes, select degenerate route lines) have been available since WED 1.2, but you now have to have a ‘clean’ layout to build your project. I’m hoping that this stricter validation will help fix layout bugs; when the layout is wrong, the AI aircraft’s taxi routes become even weirder than normal.
Posted in Tools
by
Ben Supnik |
Update: WorldEditor 1.3.2 is out now and has new certificates to work with the gateway; get it here!
I screwed up: WorldEditor 1.3.1 contains a certificate that allows it to authenticate that the X-Plane Scenery Gateway is who it says it is before WED transmits your user name and password during an airport upload. And this certificate expires in about two hours.
Last night we cut a new build (1.3.2) with a new certificate with a much longer time range, but Tyler said that for some reason the new certificate did not work. So it’s most likely that we’re going to run out of time before we get a new WED build posted. Here’s what this means:
- You will not be able to upload airports to the gateway with WED 1.3.1.
- Once WED 1.3.2 is available, you will be able to upload airports using WED 1.3.2.
- Every other function of WED will keep working.
- The Gateway’s public web page will keep working.
I’ll post an update here when we can get WED 1.3.2 “live” – unfortunately it will probably be more than two hours. I’m hoping to have this solved by the end of the day.
I will also cut a new WED 1.4.0 beta with the latest bug fixes and an updated certificate. That should be available tonight.
As a side note, I think that everything that is “must fix” for WED 1.4 is fixed, so this will be a WED 1.4 release candidate. We are deferring jpeg-2000 support out of WED 1.4 entirely so we can ship it.
Last night I posted new versions of WorldEditor, MeshTool, and our command line tools. Follow the “tools” menu on this page to find them – developer.x-plane.com is the official home of our Laminar Research’s scenery tools. (We’re migrating scenery.x-plane.com and wiki.x-plane.com to just be on this site.)
Please remember that the scenery tools bug share the gateway bug reporter; please search the gateway bug reporter before you report a new bug – maybe your bug has already been reported. (You do not need to log in to Jira to search!)
WED 1.4 Beta 2
I have packaged a README with WorldEditor, and including release notes on all fixes since beta 2; bugs filed to the gateway reporter are also updated. Unfortunately I do not have release notes dating back to 1.0.
At this point there are still a number of Linux UI bugs, and Geo-JPEG 2000 support is still out. At this rate I expect to not ship Geo-JPEG 2000 support in WED 1.4 at all.
MeshTool 3.0 Beta 1
This is the first public beta of MeshTool 3 – so far there aren’t new features compared to MeshTool 2; the main change is that MT3 builds X-Plane 10 style DSFs using X-Plane 10 land classes. It is therefore appropriate for add-ons that target X-Plane 10 only.
I have linked to the config and land class files for both versions of MeshTool on the download page; it is important you use the right land class and config files for your project. (Upgrading MeshTool without replacing the config file or land class data won’t work.)
Scenery Tools 15-3
This is a recut of the command line tools. Not much has changed.
- ObjConverter is no longer included; right now we don’t have a compiling version for Windows, and frankly I can’t think of a good reason to use this tool ever.* If ObjConverter was, for some reason, part of your workflow, you can still download it from the 12-2 tools release; email me and I can also explain how to use a different, better option to convert your objects.
- DDSTool now defaults to sRGB gamma on input files. Both the old and new version read gamma information from your source PNG file, but if the PNG file is not tagged properly (and it’s very easy to have tagging go wrong, particularly when Photoshop** is involved) you would get classic Mac gamma in the old version. This is basically never what you want; the new recommended work-flow is to always work in sRGB, use the new DDSTool, and you’ll always get the right results.
* ObjConverter tried to convert 3DS and DXF files directly to OBJ. But since there is no standard encoding of animations, materials, and other X-Plane specific data in 3DS, the converter could only copy the mesh and texture map. This made it appropriate only as a way to get from one 3-d program to another.
But even this use is not a good way to move your data, because it strips out animation and meta-data. My suggestion is that you export directly from 3DS using an export script, or open the source 3DS or DXF file in a modeler that has native X-Plane support like Blender or ac3d.
X-Plane’s OBJ format is not meant as a way to move your models between programs; it is meant only as a final destination format for shipping your scenery.
** The problem with photoshop isn’t that it writes the PNG files incorrectly, rather the problem is that Photoshop is way too smart; it writes a full color profile when libpng only understands a few simple constructs like sRGB and gamma. So what I was seeing was libpng not understand the sRGB color profile from Photoshop because the encoding was too complex.
Defaulting to sRGB is a bit of a band-aid, but it also is what everyone should use all of the time.
WorldEditor 1.4 public beta 1is available now. First, the links:
- Download from the WorldEditor page.
- Report bugs on the gateway – the scenery tools have their own tab.
- If you have reported bugs against WED in the past and the bug says “please retry in WED 1.4” or “fixed in WED 1.4”, please go re-check the bug now!
The online user’s manual is up-to-date; pick WED Manual from the help menu to see it and read about the new features.
I’ll try to write some release notes up later but there isn’t a procedure in place for WED for that right now. Some major features:
Download from the X-Plane Scenery Gateway
I think the most important feature of the new build is the ability to directly download an airport from the scenery gateway. This feature is intended for authors and editors who want to modify and re-upload the scenery; in this case direct download has a number of advantages:
- It’s a lot quicker and easier.
- Better data quality: there’s a lot less data precision loss in the direct download because the format used is not binary DSF; overlay elements spanning DSF tiles will not be split when you get the airport directly.
- Version tracking: when you download from the airport, WED knows the scenery ID you downloaded from and sets up a history chain when you re-upload. This sets us up to more easily track changes and understand what are major airport changes vs. minor editing changes.
I think direct download is going to be especially good for bug-swatting. If an airport had one small problem, it used to be that most of the work in fixing it was the import and export; this is now totally automated, so you can just download, edit, re-upload.
Orthophoto Creation
WED 1.4 builds orthophoto draped polygons for you. In this workflow you:
- Import source imagery, e.g. a TIFF or PNG. If it’s a GeoTIFF, WED places it for you. (GeoTIFF placement is fixed in WED 1.4.)
- WED converts the image format to DDS when you build the scenery pack.
- WED makes the draped .pol file for you, and puts a correct LOAD_CENTER directive in place to get paging.
It’s a much quicker and simpler work-flow than the old scheme from WED 1.1 (which was basically a hack I put in for Sergio to get the LOWI demo area built for X-Plane 9).
If you want to make your own .pol files you can still use .pol files directly – WED works either way.
GeoJPEG-2000 Got Kicked Out
I turned off GeoJPEG-2000 support in this beta; our testing indicated that .jp2 was super-unstable and unreliable. I’m not sure whether .jp2 will make it into this release or whether we’ll even keep the feature, but one thing is clear: it’s holding up an otherwise solid beta. There’s no reason why anyone should have to deal with broken GeoTIFF location, crashes on certain library scenery objects, or having to manually download from the Gateway for longer than necessary.
I still need to do more investigation into the crashes we’ve seen but so far the signs don’t look good – there are multiple indicators that point in the direction of .jp2 not being ready for prime-time.
This is another “Request for Comments” post – please discuss the proposal in the comments; if you comment asking about the OccRift your comment will be piped to /dev/null.
There’s one aspect of the library system that acts as a sharp unprotected corner, poking users on a regular basis: some scenery packs require other scenery packs to function. For example, many freeware airport scenery packs require OpenSceneryX. When the library pack is not available, X-Plane will not load the custom pack because it is missing art assets.* Users report this to us as a bug surprisingly often.
In my view, the big problem here is that a user has no way of knowing from X-Plane’s diagnostic message what library they should have installed. The diagnostic message isn’t useful because X-Plane doesn’t know either. All X-Plane knows is that there was a library path, no one is providing art for it, and therefore life isn’t worth living.
The Proposal: Library Pack Dependencies
My proposal goes like this:
- A library scenery pack can declare its “official name”, e.g. “opensceneryx” or “proaddons/trees” or what-ever. Like plugin names, the goal is to pick a reasonably verbose name tied to your identity to avoid conflicts. This directive would go in your library.txt file.
- A scenery pack that needs that library declares a need for the library by the same name.
- When X-Plane tries to load an art asset from the scenery pack and it is missing, if at least one dependent library is not present in the system, then the error message changes to something like
The scenery pack “KLAS Las Vegas” could not be loaded because the library “OpenSceneryX” is not installed.
The log.txt file would contain complete details, but hopefully it would be clear(er) to an end-user what has gone wrong: OpenSceneryX is missng, and thus KLAS Las Vegas cannot be used.
How Is The Link Made
In order for this proposal to work, scenery packs that require library X have to contain a directive stating so. Therefore this proposal is not a cure-all for existing load problems. It would help in the long term as new scenery packs and libraries are created with these directives.
How would the link be made? WorldEditor (or other scenery editing tools) would automatically write the dependency into the scenery pack by looking at the dependencies in place on the author’s machine and copying them into the scenery pack when the user picks “Build”. Thus as long as the libraries had correct “naming” annotations (this does require library authors to update) then new packs made with WED would contain the right dependencies automatically.
Nasty Details
A few nasty details:
- Library packs would need to contain both their “permanent” name and some kind of “human readable name” for error reporting.
- The dependency statement in custom scenery packs would list the permanent names of needed libraries and copies of the human-readable names; if we need “librutrees” and it is missing, we don’t know that it’s real name is “Russian Trees 2.0” unless this has been copied at build time.
- Dependencies would also need an integer version number. This allows us to declare the case where the library is installed, but it is too old.
X-Plane’s built-in libraries would not contain dependency names because they are always available.
Dependency names for scenery packs would be written as DSF properties; there is no guarantee or need for the non-library scenery pack to have a library.txt file.
Open Issue: if a scenery pack declares a dependency and it is missing, should it be allowed to load if all of the art assets are present? This is the more permissive use case, but it implies something fairly strange is happening on the end user’s machine. Permissiveness might be desirable during the transition into using dependency names.
Will It Work
The library system has (for quite a few years now) allowed “place-holder” objects to be declared in a scenery pack that act as fall-backs for missing objects. The use case goes like this:
- OpenSceneryX provides a “fallback” pack that is dumped directly into the scenery pack with blank objects for every library path.
- If the end user has OpenSceneryX installed, they see the real art.
- Otherwise they see nothing – the fallbacks are blank, and no error is generated.
It seems clear from the number of users who report a missing OpenSceneryX object as a “bug” that this is not working. Authors who use OpenSceneryX are not bothering to copy the “fallback” pack. This might not even be a bug – maybe the authors don’t want their work being viewed without OpenSceneryX installed. My guess, however, is that the authors just don’t know that the fallback pack exists. Since the authors have OpenSceneryX installed, they have no need for the fallback and can’t even tell if it is working properly.
My hope is that the library dependency scheme can be more successful in the long run because it requires no action by individual scenery authors, as long as a small number of library maintainers update their libraries. The work of annotating scenery packs is automatic.
* Please do not try to convince me that what X-Plane should do is ignore this problem and proceed. With the RFC proposed above, we could do something less drastic, like not loading the scenery pack if the library isn’t present. But I am strongly against “load what we can and keep going”.
If X-Plane treats errors in authorship as acceptable results, then authors trying to get actual work done will have to do a lot more work to detect human mistakes in their own authorship. We need a bug to be a bug.
Note: This post is a boring discussion of the state of the scenery tools. If you use MeshTool or you work on the scenery tools code – or would like to modify MeshTool’s code – read this post! If you just want to know the status of the Oculus Rift or whether we’ve changed X-Planes’ fogging model, a little bit of your soul will die if you read this post.
Last weekend I discovered that MeshTool 3.0 seemed to “just work” for OS X; when I mentioned this on the blog I received a number of “I’d like a copy” emails. When I further emailed with those authors, I found that they all wanted a Windows build.
Unfortunately the Windows build of the scenery tools was in a state of disarray. The Windows build was until recently based on running mingw and trying to make Windows look like Linux. This has a few problems:
- Windows is not Linux.
- Mingw is often a lower-tier priority (or non-priority) for the developers of the libs the scenery tools use.
- It’s not easy to get a fully functional mingw environment to begin with.
A while ago Ted moved WorldEditor to MSVC. This was a pretty big win: debugging actual problems with WED is about a billion times easier in MSVC (which has one of the best visual debuggers, um, ever) than trying to use gdb via mingw on Windows. It also sets WED up to be worked on by actual real developers; if you are a programmer and Windows is your preferred platform, MSVC is mandatory; gcc/mingw/gdb is a foreign country.
So in response to MeshTool for Windows being borked due to mingw/Windows issues, I horse-traded with Chris: I’d do a bunch of his dev tasks if he’d port MeshTool to MSVC.
Will he succeed? I don’t want to say yes until it’s a done thing, but as of yesterday he did have MeshTool running, only to discover that libtiff was missing AdobeDeflate support. But I think his efforts do look promising.
The Plan
So here’s my plan for scenery tools:
- MSVC will become the supported compiler for all scenery tools: WED, MeshTool, and the various command line tools (DSFTool, DDSTool, etc.).
- The scenery tools will come with an MSVC .sln/.proj set to support them.
- We will let mingw support die. (If you want to make mingw work, send me patches, I’ll take them! But no one involved now has the expertise to keep mingw working, and it is already significantly broken.)
- Gcc/Makefiles will be the supported platform for Linux, working on modern distros.
- X-Code 3 will remain the supported compiler on OS X until we have time to do a Yosemite upgrade. (A Yosemite upgrade is badly needed, but it’s also just a ton of work; I have not had time to even start on this.)
- “RenderFarm” (the internal DSF tool we use for global scenery) will be the only non-MSVC-ported tool.
For libraries, Mac and Linux will continue to use the “libs” archive that exists now, which is a series of tarballs and a makefile that creates a static library pack for WED/Meshtool to use. Libs is a GIT submodule.
For libraries, Windows will use a different sub-module that contains source materials and already-compiled Windows binaries for all needed libraries. Windows users will simply have to get the submodule from GIT and they can immediately compile WED. (This is already how WED on MSVC works, but we’ll form a submodule so that the main repo doesn’t get huge.)
For libraries, when we get to Yosemite, we may move OS X to a precompiled-binary strategy; we’ll have to see what the state of libraries is.
32-vs-64-bit
Right now the Linux scenery tools are compiled to the native ABI of the machine they are built on; binary builds are built on Ubuntu 12.04 LTS 64-bit, so those are 64-bit builds.
Windows is currently a 32-bit build; I believe Chris is working on both 32 and 64-bit support.
Mac build are limited to 32-bit due to the use of legacy APIs.
The long term direction of the scenery tools is 64-bit only! At some point the “next” version of the scenery tools may be 64-bit only builds, and this may happen within the X-Plane 10 version run. The scenery tools are free and not part of X-Plane itself; we don’t consider ourselves to be under any obligation to maintain 32-bit support indefinitely.
For the scenery tools, 64-bit support is a win because it allows the tools to access relatively huge source data sets. There are global DSFs that currently can only be built on Linux because they require a 64-bit RenderFarm to load all road data without crashing.
Whither Betas?
There’s some restructuring to the various projects’ we’re doing as a result of all of this; I’m going to hold off WorldEditor beta until we’re done. I’m hoping this will end in “days” – that’s all of the time Chris and I allocated for it. Therefore hopefully:
- WED 1.4 public beta 1 next week. (I think I said that last week – I forgot to schedule in time to get the flu.)
- MeshTool 3.0 public beta 1 next week too. We’ve run a private beta on Mac/Linux and confirmed that MeshTool 3.0 basically works. We’ll do a Windows build publicly.
We should also have a 10.35 release candidate Real Soon Now™.
