Note: beta 5 came out before the long weekend and then I managed to not publish the announcement. Sorry about that! Since the original writing of this it has become clear that only some of the no-texture objects were fixed, and the throttle reversers are borked. Beta 6 should be out in a day or two addressing both.
Update: just to clarify, since this post kind of freaked people out: We are not closing off the scenery system. We are not changing or hiding any file formats. We are not removing any scenery tools. If you’re using a supported part of the scenery system now, you can keep doing what you are doing.
X-Plane 10.50 beta 5 is here – see the release notes for a complete list of bug fixes. There are some big bug fixes here including a few big ones:
- Fix for fuel consumption (which was tied to the number of physics cycles you had set).
- Fix for backward control surfaces (which was intermittent based on what aircraft you had loaded before).
- Black textures and crashes loading airplanes are now fixed. (This case was getting hit only when the right combination of untextured objects was used.)
- Fixed some crashes loading scenery.
On this last bug: it looks like the w2xp “net” sceneries are responsible for virtually all of the crash on scenery load bugs I’ve seen in beta 5. The problem is that w2xp doesn’t pre-process and sanitize raw OSM data before sending it into the sim; the result is that a small number of weird OSM roads are causing a cascade of ever-weirder data in the rendering engine, eventually resulting in a crash.
I’ve gone to the extraordinary step of programming X-Plane to attempt to clean up the road data on DSF load. You’ll see a log warning with w2xp net scenery when this happens telling how many road chains had to be deleted. Even with this, I can’t guarantee that there isn’t some OSM data out there, when dumped directly into the simulator, that won’t crash.
I’m not sure what the long term fix to this is, but it did make me question our scenery strategy. We’ve always tried to keep the scenery system open, and I think it’s fantastic to see third party developers doing things that we’ve never done with X-Plane. But I also don’t know how we can guarantee X-plane’s stability when this kind of unmetered low level access to the rendering engine is provided.
Hi Ben
I’ve been testing b5 from it’s release and i can only congratulate you everything seems to be working fine except the thrust reverses for me.
thank you a million and keep up your great work.
Great Ben! a lot of working i see. Thank you.
When you expect to publish the new WED version?
Thank you!
“I’m not sure what the long term fix to this is, but it did make me question our scenery strategy.”
Hi Ben, you got me seriously worried ! Does that mean you question the possibility to compile/decompile the dsf with XPtools ? Wouldn’t it be better to give tools and/or solid documentation to ease “sanitation” ? And be very clear on what is “legal” and “illegal” in the scenery engine in the first place ? I’m sure scenery tools creators don’t do such mistakes on purpose or by laziness, but because they lack the proper knowledge and tools.
Pascal
Thanks for the update! And waiting for WED!)))
The way of improvement X-Plane with taking into account works of users is unique, but very complicated. Today we can’t imagine the user’s sceneries without “w2xp” and so on. I’m ready for long waiting for better times. Thanks.
Perhaps re-utilize that same code that is sanitizing on load into some kind of tool that does a recursive scan through the custom scenery folder? If there’s a way to determine in sim that the crash was likely caused by scenery, this scan could be suggested during the next startup so that hopefully it results in a bug report to the scenery developer and not Laminar. Scenery developers would (in theory?) also use it as a sanity check on what they’re putting out into the wild.
I don’t know that you necessarily *can* guarantee stability with the amount of customization you allow, but locking it down feels like throwing the baby out with the bath water. Reading between the lines here, the problem isn’t so much that people can make the simulator unstable; it’s that they have the ability to do it with poorly-designed addons which aren’t easily singled out for being the culprit, which leads to them blaming XP10 for being unstable and support requests Laminar shouldn’t have to deal with.
I was very exciting to simulate flights since I began to use X-Life from JAR Design. Airport went alive with traffic in the air and on the ground and my flights were simulated under ATC. Now X-Plane is coming out also with AI aircraft traffic and new ATC.
I see here very strong parallel of great developments that that happen at the same time.
My question to you Ben is, if this two developments will complement in future each other or run in parallel?
BTW I heard from JAR about being in contact with you and I really hope for further great results ….
W2XP simply puts whatever data you give it into the sim, so if the road data in OSM is naff, then it will be given as it was to X-Plane. The only preprocessing W2XP does on road data is to try and fix some bridges and doubled up nodes. Roads are just one of many problems I’ve seen in OSM data. This area has always been buggy, it even says so in the manual 😉
W2XP has up until now not needed any sort of database. This has been a plus for users but mostly a huge headache, as preprocessing large amounts of data and trying to clean it up without a database can be a real pain. My own unreleased version I use for my pro sceneries uses a PostGIS backend, and all the data (often mixes of several sources I get out is sanitised and formatted as needed.
I do however sanitise facades and forests as after a lot of experimentation and bugs from users, I found out what X-Plane would allow. It was painful decompiling DSF files when a crash occurred, commenting bits out until it worked and isolating the entry (These files are huge). Often the crash would give no indication of what the problem was and I think if there was a debug mode in XP10 which would spew out more info to the log as well as better error messages, this would hugely help from my point of view.
With buildings and forests, W2XP will tell the user in the console which OSM object is buggy and they then have a chance to fix it. I don’t know however what is considered a valid road, so if you could provide a validation rule list of some sort, I can then exclude or sanitise the roads and let the user know of the problematic road ID in OSM to fix. I’m also sure the documentation would be very useful for anyone else developing scenery.
Tony @ W2xp
Tony is up against several problems here:
1. When X-plane crashes, _X-plane_ doesn’t know what went wrong. The crash is at the end of a cascading chain of problems, with the original issue often being lost in the fray.
2. The guidelines of “what is a terrible idea” are NOT well specified. This was less of a problem before OSM, which is a lot like feeding a random number generator into X-Plane.
3. What the sim can cope with depends on the art assets as well as limits within the sim. E.g. “no zero length segments in roads” is not only sort of obvious and totally undocumented, but it is also easy to enforce programatically. “Don’t make a road so short that two intersections, one on each side overlap” requires you to know a ton about the actual art assets.
Tony, I can put up some validation rules for DSFs…I can’t guarantee they will be complete, but they’d be a good start for validating this kind of scenery. My previous policy (for several versions of v10) has been to try to convert every crash into a diagnosed, clear error in X-plane by catching it earlier. The roads are the one case where that isn’t feasible, because you can get a crash by “over-stressing” the art assets by making silly shapes that are otherwise legitimate syntactically.
Thanks Ben
With point no. 3. I thought I actually did throw out 1 or 0 node roads, I produce a line geometry from the points for doing some intersection tests and there is actually some basic validation which will refuse to create single-noded roads. Roads with two identical nodes I can easily check for (and thought I did). If you have seen these then let me know. In most cases, they are very easy fixes.
Regarding silly shapes, I have seen facades with heights in kilometers (reaching into space), and facades as big as an entire town because it was tagged wrong. There are lots and lots of silly but real examples that I do filter out, but generally I leave it up to the user of the tool to sanitise their own data as they can pretty much control what is generated and with what by editing the config files which makes it somewhat challenging to do validation when the user can change what they like. If there is a list of rules for DSFs then I can simply add this as a pre-compile check, and in most cases it should hit the worst of it and I can also tell the user what to fix in OSM
Hi Tony,
Simheaven is definitely shipping DSFs with zero-length segments within points. One of the difficulties here is that you can’t guarantee which -version- of your tools a user uses and ships. 🙁 Those DSFs will be with us forever no matter what we do.
10.50 actually does at least spit out some stats so that you can confirm whether your filtering works as expected.
I have written down some basic data validation rules here.
http://developer.x-plane.com/?article=dsf-usage-in-x-plane
I’ll try to add to it as I think of stuff; if anything there seems blatantly wrong (e.g. “hey it’s really useful to make X” please) let me know.
Hi Ben,
Let me add a question to this topic: what tricks is RF applying to the OSM road data to make it “less problematic”? I mean, I have never encountered a crash with HD Mesh Scenery or UHD Mesh Scenery even though they use a lot of that “road randomness” from OSM too. Or is it just pure luck, that we maybe filtered out (maybe not even intentionally) the worst offenders (with our list of included attributes)? Or maybe some of the simplifications in the RF scripts removed the offenders? What ever it is … it might be the right pointer for Tony to look at potential “bad data” constellations.
And yes Tony, going with PostGIS is a good idea (which also gives you a lot of flexibility and power) … even if it narrows the user base of the tool (on the other hand, you really only get more pro users with better understanding of complex issues 🙂 )
Hi Andras,
RenderFarm is -heavily- processing the road data. This is both because we’re aiming a lot of computational firepower at it (we use an arbitrary-precision geometry library to clean up junk data) but also (and perhaps more importantly) because the philosophical approach is different.
Because RF aims to create the “global scenery”, which is already going to be down-sampled since it is heavily space constrained, there is an emphasis on making the data sane and plausible at the expense of accuracy. Thus we don’t feel bad deleting roads, moving roads, merging roads, you name it…our output is more like sausage than steak. 🙂
(Mostly in that you don’t want to look at how it is made. 😉
Note that RF a segment-to-bezier-curve conversion that intentionally loses precision to simplify and reduce the size of the data while producing curves. So there isn’t necessarily a 1:1 relationship between input and output data at all!
Speaking of more info to the log.txt, while it’s monsterously convenient to have everything under one roof, the complexity of X-Plane is making this a very cumbersome diagnostics tool. Perhaps it would help if we had more specialized logs. One for scenery, one for plugins, one for operating system information, and one for X-Plane core activity. This is a seed thought, not something that could remotely be implemented easily.
One question: have there been behind the scenes updates that are not being reflected in the SDK? Lots of awesome stuff going on in X-Plane, Ben. I feel your pain! More awesome always means more hair pulling and extra coffee consumption. With or with out the high test additives. 😉 Love that you got the fuel issue wrangled for 10.50. So far, stable here, but my flying has been simplistic at best.
Hi Steve,
The intention of the I/DSF syntax was to make the log -filterable- by streams of interest. I’d like to improve the categorization over time to make this more useful.
We’re going to keep the log as one log because one of it’s primary use is support – our goal is to get a LOT of info from one VERY confused user with as little human error as possible, so “send us this one file” is better than “send us these 5 files”.
cheers
ben
As Engineer Scotty would say the more sophisticated we make the plumbing the easier it is to plug up the drain LOL!
Keep up the work guys and don’t pullout to much hair I love this sim wish all you developers a sincere good luck!
Hi, Ben!
And thanks for all your work.You might want to set up on KLGA Runway 13 and see, in the distance, in Queens–which is home to a LOT of 15-20 storey apartment complexes, but only ONE skyscraper–a cluster of huge skyscrapers dead-ahead. Me-thinks your autogen algorithm needs work. Or at least exclusions, when used with, say, Drzezwiecki’s NYC XP.
Best,
Marshall
why was the hide yoke option removed from the baron?
We didn’t remove it, we moved it. It used to be a “fake switch” on the panel – it is now a real first-class sim command. Our goal was to have the hide yoke be a _UI_ function since real life airplanes don’t have hardware switches to hide the yoke. The UI command can be used by any airplane to hide the yoke so that users get a consistent interface; we’ll be using it in our airplanes more often.
ok so how do i hide it now?
sim/operation/toggle_yoke
Hi Ben
Is there any plans you are looking at improving “lights draw distance” as well? I remember you said this comes from the old 32 bit days memory limitations but you could try an attempt in the future. I see this question coming and coming on the forums and i am particularly interested myself too
regards
“I’m not sure what the long term fix to this is, but it did make me question our scenery strategy. We’ve always tried to keep the scenery system open, and I think it’s fantastic to see third party developers doing things that we’ve never done with X-Plane. But I also don’t know how we can guarantee X-plane’s stability when this kind of unmetered low level access to the rendering engine is provided.” ——-This is a scary statement. Please elaborate for us simple folk who are relying on being 3rd party add-on devs for our family’s sake, ah! 🙁 Please Ben! I don’t want to work at the zoo shoveling antelope & monkey dung for the rest of my life…I could have dev for FSX or P3d but I see a very bright future here with X-plane & it’s community, especially because of you! 😉 **special handshake**
Well, let me be clear: I am absolutely _not_ proposing that X-Plane become a closed platform!!!!!
As an example of the kind of decision that I have to think about (I’ll write more on this later), there are two ways to support content creation:
1. Publish the file formats and let people make stuff for the product.
2. Publish content creation tools and let people use them.
We do both; if you think of content going through a pipeline, the question is: how much of _LR’s_ pipeline does content have to go through to get into the sim. Right now the answer is “potentially none” – in that you can write your own OBJs by code or even your own DSFs (gasp) because the file formats are published.
The up-side of the open format is that -anything- is possible; this is the most open and most flexible we can be and we are never a bottleneck.
The down-side of the open format is that it allows for a lot of our pipeline to be bypassed, and in that event, validation code may be skipped.
By comparison, for the X-Plane Scenery Gateway, that pipeline is heavily specified – content must come from WED, full stop, and it goes through WED’s validation layer. This means that we have a tool to improve quality: any time something goes wrong, we add validation -in WED- to catch the problem.
We have no such strategy for DSFs – for DSFs we have to catch them in sim, at load time, while the user waits.
Ben, I’m currently working on high quality scenery production, at the scale of a few dsf at time. I’m using techniques I’ve been working on for years. I build my own (valid) dsf from scratch, in text format (not the binary format). If this possibility disappears in the future, I can stop right now all the same and find a new job. Sorry for being slightly dramatic, but I’ve invested a lot of time and money into this project. I can give more details in private.
All the best,
Pascal
Just some precisions, I work mostly on texture, not roads or objects. I work with GIS databases and assign arbitrary ter files to the base mesh triangles. I also add overlay triangles where needed. These are the only operations I use, but they require access to the dsf.
Using DSF2Text is _not_ going to disappear.
OK, thanks !
Pascal
Ben, I’m working with the Beta’s, and noticed that when developing taxiways now with WED, the old “Don’t send me co-located points” message still pops up, but now it has LAT and LONG coordinates listed. They aren’t detailed enough for me to find my mistakes. Could the coordinates be carried out a few more decimal points? The info it gives, by example, is
Point:8
LAT 32.36
LONG -95.40
Thanks, Matt
Please file a bug and include the full Log.txt; this may be unrelated to your layout. If you validate that there are no colocated points in the ATC layouts it should not be possible to get this message due to your layout. I think there is a fix for a related problem in beta 6, but you’ll need to check the notes once beta 6 comes out.
Hey Ben,home sick this weekend. Is Beta 6 delayed? Was hoping to use thrust reverse avain :). Thanks
Hi, Ben! Congrats on all the new features and changes! I was excited to hear that the radio volumes can now be controlled individually using a generic rheostat to control each radio data ref, but I can’t find any new data refs in the data refs.txt file. I am also excited to implement scroll wheel support for knobs and such – I see hints of those new commands in the Baron cockpit.OBJ, but is there any documentation that describes the syntax of the new manipulators?
Thanks!
P.S. I am really enjoying the new auto-gen! Hey, there really are tall buildings in my City 🙂
OBJ8 spec isn’t updated for the manips yet but if you look at the Blender 2.7 GITHub project, there is a working spec.
Looks like we might be missing the datarefs? I’ll investigate.