Next generation trees and OpenGL

64 comments on “Next generation trees and OpenGL”

1. OK, nice trees are nice, BUT: I’d prefer to have “tree contact” (with the airplane) being modeled in a realistic way, maybe like this:
   If you hit the treetop with the gear, the plane should rotate “nose down”.
   If you hit a tree with the left wingtip, the plane should rotate left.
   …you get the idea…
   OK, not just for trees, but also for builings.
   I consider this basic realism/simulation feature much more important than good-looking trees.

   1. At least in the current implementation, there is no collision with trees. The problem with making everything in the world a hard object is that it requires us to eat quite a bit of RAM as well as processing time. For starters, all meshes would have to live in RAM as well now, since reading anything back from the GPU is insanely slow. Additionally we then have to also walk the whole scene graph during physics update to find potential colliders, and once we have those, we would additionally have to also apply all animations to the mesh in order to get a fine collision detection.

      Unfortunately for now, this just isn’t really feasible with the amount of things that can be in the world. Especially for trees of which we can have thousands close by to the aircraft. The system is really designed to make it fast to render without the CPU having to know much about it. For collision detection the CPU would have to be in on the joke about trees, negating all the GPU benefits.

      1. Tree and scenery collision is a curious thing indeed inside flight sims.

         As I recall FSX implemented it (rather poorly by modern standards) once upon a time – I ended up turning it off because it felt like simplified and overdone ‘hitboxes’ rather than collision.
         I think most of us will happily wait for the tech to catch up instead of risking that sort of ordeal again 😉

         Also I appreciate the technical explanation here and the reasoning behind the decisions.

         Keep up the good work. I’m looking forward to the day we get to see all this new stuff live in sim.

      2. Hi Sid,
         wouldn’t it be possible to simplify things a bit? Can collision detection be tasked to the GPU as well, using something like what Steve Wilson proposed with the LOD? It does not have to be real time connected to the physics engine on the CPU. What I mean is that crash simulation is not really needed and is just enough if collision event is returned only to be used as a warning to the user. That way the slow GPU-CPU reading back will not matter as this will be one time event. Than there can be a popup to the user “You have just crashed” and option to start new flight, replay or continue as if it never happened. No one will care if this happens at the same frame as the event occurred or 100 later.
         I have no idea if this is possible, I am just thinking out loud because this is a feature I want to have even not 100% simulated as Ulrich proposed.

         1. I’d assume there is also a fair percentage of the population who , like Austin puts it, knows what happens when you hit a tree. I understand that for bush flying this may be an interesting concept to have. Why not a compromise:
            – trees around airfields say on the glide path are part of the mesh and interact with the model?

            1. Hi Pedro,
               I think Austin misses a point – it is not about knowing what happens, it is about knowing when it happens. I personally love bush flying and landing at though spots. I am working on a agricultural/firefighter plane for some time and if you watch some vids on yt you’ll see a lot of near misses. That is just how those guys fly, clearing the trees at the end of the field at the last possible moment for 10s or 100s of passes in a row and avoiding power lines, wind turbines and all kinds of obstacles. If you hit a tree or anything head on of course you will know it, but if you just touched it with your gears or vertical stabilizer out of your field of view you’ll know only if you watch a replay. And as I said it is not important to model the interaction, only to detect the event.
               Not that this feature is critical, but it will be nice to have if possible.

         2. With Vulkan/Metal, you can read back data from the GPU with a frame’s delay without risk of slowing anything down. Should be sufficient to solve the problem, with some trickery.

            However, my guess is that it’s probably not worth the quite large amount of work it takes to set that up, weighted against other priorities of the team.

      3. There are optimization techniques to reduce the workload for general collision detection for non-moving objects, such as subdividing a large space using sparse matrices and only comparing against objects which are in a localized space.

      4. Collision detection with trees is very important for realism. Enable below 50ft agl and near an airport, and use simple (invisible) polygons for collision detection. Let the scenery maker define these polygons.

         1. Realism?
            In reality I never ever would like to test what happens hitting a tree.

            1. I am talking about the realism of knowing _when_ you collided with something. You are talking about the post-impact realism.

               I could care less about the post-impact, just give me a red screen to communicate “you crashed”. Post-collision realism is for video games, not for flight simulators.

      5. I know where the trees are — I worry about missing the power lines strung across canyons and valleys.

      6. The new trees looks great!!
         Never mind colliding with those though..
         Still any places like FLAT ROOF TOPS should really be hard objects by default for helicopters.

   2. If you’re hitting the trees you’re flying the plane wrong! 🙂

      1. Also when you stall, misuse the autopilot, hit the ground, etc.
         A flight simulator is not a scenery viewing tool, but a learning tool: It should detect any errors you make.
         Somehow many years ago more simple simulators could detect that. For example Flight Unlimited fid not just “make boom (an explosion)”, but you could get a bent gear, wing, propeller, etc. For example when you hit a tree while landing. For X-Planers: If you haven’t tried, try landing at Ranger’s Creek in the Seattle area…

2. Great stuff, Sid. Thanks for the insight. A vegitation engine? I’ve seen those before. They run on cannabis. 😉 Naturally.

   We’ve recently seen some experimentation on X-Plane.org with collision modeling and trees – and I was wondering what thoughts you may have. The notion – which is likely in error according to an unimpeachable source – was to add a very close-in LOD for a modest bit of “hard” surfaces. The idea is to provide consequences for errant flying – not flying into things deliberately. There are some annoying trees right in the normal descent path into (the fictional) NZMH on the southern island of New Zealand. I’m invariably brushing into these since I land a bit on the hot side, considering my bush bird of choice. I would love to have the challenge of a must miss scenario.

   1. As a permanent resident in Canada, I can assure you that I too run on cannabis. Sorry west coast, that smoke is all me.

      1. Have to admit, that reply just stumped me with a good come-back!! As a permanent resident in AZ, USA, I could say the same thing as of early this year, but I have yet to visit one of the many happy new establishments. Have to wonder what the effects could be on code!

         Oh…. oops… weren’t we talking about collisions?

      2. That smoke has made its way here to northwest Alberta. I think you may need to slow down a bit.

         1. But then I would have to accept the fact that there are two out of control wildfires burning just a few kilometres from where I live and that would seriously impact my mental well being.

3. Really promising features ahead. September and FSExpo can’t come soon enough to see more.

   At risk of asking too soon about this, and I don’t want to spoil anything, but since grass is part of vegetation, is Volumetric/3D grass expected to be added also? Is a really good addition for airports, but also for brush landings

   1. Yes me to on this Sid… Volumetric/3D grass would be a huge significant step forward!

      1. Honestly I don’t really care about 3D grass, but, 3D clouds is now a must. This is now going to be a show stopper I think, especially since MSFS 2020 and DCS 2.7 … They can’t do without it.

         But… That is just my humble opinion, we will see. Oh, and 3D water as well (they could implement the Proland ocean plugin for example, it’s free to use if I’m not mistaken), though clouds and general weather modeling is more important.

         While MSFS is doing it impressively, I don’t like their thunderstorm lightning depiction for example. Not a big deal, but there is room for improvement. They could out do MSFS if they want and have the time and money to do so. Still I’m curious to see what XP 12 is going to be…

4. So, for clarity – x-plane next gen is dropping the end-user option of an open-gl renderer (as you’ve always said), but plug-in devs still have access to your open-gl compatibility layer like they do today. Just checkin’. ?

   Whats your rough gut-feel on vr framerate loss with these trees?
   I expect the lod ranges will be hidden from the ui, but id like the idea of an ‘airports only’ option for vr users with less capable hardware. – wait wait wait, scrap that, here I am trying to solve problems before they’re even found! Looking forward to xp-next.

   1. Yes, you got it right about OpenGL

      1. Why not drop GL support completely, even the compat layer/bridge, for next gen?

         1. You mean besides that it would kill basically every plugin and add-on aircraft dead?

         2. The open-gl plug-in architecture will be replaced in the future when Laminar has a Vulkan/Metal based plug-in architecture. Only Laminar knows when it will be released (but I guess they don’t currently know when, since the idea is still on blueprint or on draft code).

            1. Just to make this clear: We have absolutely no plans to expose Vulkan and/or Metal to third parties for their drawing needs. We also have no plans to remove the OpenGL support for third parties either.

5. This is great news, thank you!

   The X-Plane experience is so nice in many ways, and most importantly on physics and flight model. Graphic updates, rendering and lighting are indeed very welcome, needed to make the step into the future.
   One particular feature I have always missed though, is audible ambience from the scenery and given surroundings. As a sound engineer I come from a strange religion based on “hearing is believing”.. There is a major benefit from immersion by sound. Soon you have 3D trees. Maybe they will also be able to move or sway.. Now how about hearing wind buildup based on METAR and seeing those trees sway. Or the waves of the ocean when near. Or urban/city ambience, airport sounds, birds chirping at a bush-airfield etc.
   That really breathes life into a simulator. And would perhaps do so WITHOUT the expense of framerates as its handled mainly by your audio card.
   That would be something 🙂 (sorry for de-railing this fine post!)

   1. So, first of all, I have to absolutely agree with you. I think game developers tend to really lose focus on sound a lot, even though it adds so much to the feel of things.

      Unfortunately I can’t give you anything concrete about what things are planned for environmental sounds. But what I can say is that Daniela is working on sounds that aren’t just airplane sounds.

      1. What I’d like to hear is (for example) the sound of air when the gear is close to touchdown (think of instrument failure). I don’t care much about grass, trees, birds or bees.

6. Marvelous,
   Now i can find an empty spot to land my airplane, and do some barbeque and have picnic walk around the tree, with hope not to confront with air-inflated bears that stair at me.
   All in all, Great Job, we are eager to see X-Plane leap edition in a year time with all next-gen goodies and maximum capitalization on new hardware/software 3d acceleration stack.
   i believe it can be achievable to end up with forza-like smooth cornered wet and dry roads and 3d vegetation and urban recreation that stretches in an open world…
   Not to forget mirage effects on tarmac.

7. Great explanation, Sidney, as always.
   Question about the comment on “solid trees ” Or “tree contact”.
   I totally understand the point, this is not a gta game, it is a flight simulator.
   Also I understand the impact it could have in term of gpu ressources.

   But perhaps.could it be feasible to generate those kind of options only in the case of bush airports (Canada, Alaska, etc) and trees immediately surrounding a runway?

   Last question: the video and comments from Austin refer to trees. Is there something about 3d grass?

   1. Happy to be pointed out wrong here, but I’m like 70% sure tree collisions would be a hit to the CPU, not the GPU. You essentially have a few problems to deal with

      – How do the CPU and GPU communicate to each other efficiently about trees.

      – How to ensure the flight model takes priority so it is not dragged down by any visual features

      – How do you actually define a tree boundary in the sim. I saw quoted somewhere that the demo of XPNG trees was 300k. But trees take place in many different shapes and are animated. So you would need a pretty tight hitbox simulation. Even if you masked the collidable area to a couple hundred of those trees, it would still have a big impact with tighter hitboxes. Sure you could slap a singular cube hitbox, but then you risk the chance of striking “nothing” in midair. Make em too small and you’re still clipping branches, then there’s no point

      – How do hitboxes also work for legacy forest definitions.

      – In a practical world… why do you even need this collisions at all? We know what the end-scenario irl, you die. From a user-experience, it probably does not provide any positive encouragement to enable tree collisions if all you have to do is restart your mission.

      It’s a cool idea, but I just don’t see what the imperative need for tree collisions is, and can see why it might not be top of Laminar’s priority list. Although I can think of a few things that should *coughmesheditingcough*.

      1. To be honest and clear: if my little LOD notion has any validity, it’s good to confirm that validity, and that’s all I’m seeking for myself and other interested parties at this point. That’s what one enterprising fellow is doing over on the .Org with a whole bunch of experimentation, and with some interesting results. If the notion is valid, then LR doesn’t have to do a thing. If it’s not, I do not think that we need to prod for it. There are many much more tasty and important fish to fry for LR at the moment.

         So, in sum, I don’t think there’s any need to interrupt Ben’s, Sidney’s and Michael’s very important efforts — except to seek a deeper understanding and *re-confirmation* of how the use of LOD parameters may help prevent CPU load when it comes to collision detection.

         If all else fails, the few trees or other obstacles that “get in the way” can be paired with an invisible “hard object” on a case by case basis if this is really important.

         The key thing is that collision detection already exists. It’s how we use it, and how the scenery uses it, that may bear scrutiny and independent development.

         1. I’m a little confused where the notion of LODs interacting with the physics engine is coming from. LODs are used to speed up rendering, by using less complex geometry for objects further in the distance so that the GPU doesn’t have to process a ton of small triangles that don’t contribute to any detail. Having LODs in the scene does nothing for the CPU side of things, since it still needs to discover all of the same objects regardless. Collision checks are performed against the highest LOD mesh, because why would you want to check against an object that doesn’t physically represent what you want to check against?

            Checking for collision costs CPU time, I don’t know really understand why people are trying to argue against that. The CPU will have to transform the mesh just like the GPU to compute its final pose and then do a ray/triangle intersection check for potentially every single triangle. On top of that, all of the geometry data needs to be additionally held in RAM so the CPU can access it in the first place.

            1. I do not think people are arguing against the reality of the CPU cost of collision checking, Sid. In fact, that’s what I was trying to prevent with the idea of mixing LOD with collidable tris. I know what LOD is for – it saves rendering visually useless information at distance.

               The germ of the idea was that tris with ATTR_hard applied are also useless at distance, theoretically.

               It comes down to two possible code organization possibilities – which comes first, LOD or collision?

               If “LOD filtering” is applied before the CPU does collision testing, it won’t “see” the hard tris.

               If “LOD filtering” is applied *after* collision testing, as Michael Minnhaar has declared, then this whole line of discussion is moot – at least as far as using LOD to reduce the CPU load of collision testing. We could not use LOD in this way.

               Have I made myself any clearer? It was just a momentary inspiration, but it’s kind of gained a bit of a life of its own.

               It’s an efficiency thing – rolling with Austin’s current development paradigm – trying to reduce load, not increase it, while still gaining what some would find a valuable feature, and while not making a single change to X-Plane as it stands right now nor any changes to the next generation version either.

            2. Physics always comes first, since that drives what needs to be rendered. The aircraft tends to drag the virtual camera with it, and the position of the virtual camera drives LOD selection. So we always resolve the physics first, then figure out what needs to be rendered.

            3. Thanks Sid, that’s a solid second source verification. Thanks for sparing the keystrokes. 😉

8. well i like the new threes only this opengl i dont like its not more supported for the next gen xplane in xplane 11 i dont use vulcan because its unstable i am using 1080ti and if i use vulkan i get crashes or after every flight i need to restart the sim i cant do two legs

9. Will the two bugs related to AMD GPU users be eliminated in the “Next Generation” Xplane? Since AMD has been asked to resolve the issue and not done so to date. The flickering continues. Will the usage of OpenGl from other plugins still cause this issue? Sorry I am not a coder so have to ask. Thanks!

   1. Unfortunately this is something AMD has to fix in their driver, I have tried a lot of things to work around it and unfortunately nothing works. I am under the impression that RDNA based GPUs don’t have this issue, although I do also fully understand that new GPUs do not grow on trees. AMD has gotten plenty of repro applications with source code from us, as well as numerous X-Plane builds with various debug options. So far they have been unable to reproduce it in their lab, despite numerous attempts at getting this fixed.

      I fully understand just how incredibly frustrating that is. Additionally because AMD so heavily benefits from Vulkan in the first place. Unfortunately this puts is between a rock and a hard place. On the one hand I hate to take away options from users, on the other it just isn’t feasible for us to simply stop iterating on the product because an IHV can’t fix their drivers.

      This is with regards to the flickering anyways, what’s the second bug? Is it the broken GUI rendering that some users experienced with plugins? If so, I spoke to the AMD driver team in June and they promised it would be fixed with a recent driver (I believe 21.7.2 ships with the fix, but this is of the top of my head). Although this is obviously frustrating as well, since this represents the second time a driver update broke the plugin interop.

      So, errr, overall, all I can say is that I very much sympathize with you on this. We have some very longterm plans to move away from the IHV OpenGL driver for our plugin bridge needs. Unfortunately that isn’t super easy either, since most projects that run OpenGL on top of Vulkan target newer versions of OpenGL, while most plugins use a very old version of OpenGL. The one project that I am aware of that could potentially be used is Zink, a Mesa backend that runs OpenGL on top of Vulkan. Sadly that one isn’t really meant to run as a cute library integrated into an application, since it’s really more of an actual graphics driver. The long story short, this is all incredibly complicated and also incredibly frustrating.

      1. Thank you for responding. Happy to sign an NDA and test the ” Next Generation” with my AMD 6900XT GPU…. LOL :). I do have 21.7.2 and all previous versions dating back to 12/2020. It happens with certain aircraft with certain weather add on products. It does not happen with standard XP weather. It is frustrating because I use XP11 for IRL proficiency simulation. Hopefully, a fix is implemented in the future.

         1. Hold up… You got an RDNA2 card AND you see the flickering? Mind if I ping you privately at the email address you used to comment with?

            1. Yes sir! I have an RDNA2 6900XT and an AMD 5900X. Feel free to email me. Happy to help! Thanks.

            2. Same here! 6900XT, constant flickering with certain planes & xEnviro…

      2. Hi Sidney also, are you aware of the checkboard distortion with VR+Vulkan issue?, Is this only AMD side? From what I know I can easy play with Vulkan in “flat screen” on any OS, but when I start the VR mode, the screen becomes totally noisy T.T

10. RDNA2 – at the moment – runs best in Linux in OpenGL (3x faster than OpenGL in W10 – as long as FPS test goes). Vulkan is currently INOP outside stock planes.
    MESA-team is developing Zink really fast. No word about fixing OpenGL-Vulkan bridge X-Plane uses in MESA.
    AMDVLK + OGLPRO (combination nobody uses) was promised to be fixed and sholud be in similar shape to W10 driver.
    Please use Zink!

    1. Using Zink isn’t just a small change. Zink is a proper Mesa backend, requiring the whole Mesa frontend to do anything useful, which we can’t just grab and just kinda jam into X-Plane. I would love to ditch the IHV based Vulkan/GL interop code and replace it with Zink, it’s actually ideal for that. But that’ll require quite a bit of work on Zink and Mesa first, which we just don’t have time for right now.

       1. I’m in the same boat regarding Linux with AMD (RDNA2) on RADV or AMDVLK and quite frankly, the outlook of ever seeing a driver-side fix for the OGL-Vulkan synchronization in RADV or AMDVLK is nothing but bleak at the moment because it’s such a niche issue.
          It would be nice if you could offer some additional technical insight on the thread in the respective issue tracker about what exactly the OGL/VLK bridge does in terms of extension(?) calls as this might make it easier for the driver devs to pinpoint where things go awry.

          Otherwise, my only hope is that there’s Vulkan support in XPNG’s SDK and that third party plugin devs are going to use it.

          P.S:
          Regardless of that and unless you have some qualified in-house candidates by now, I herewith volunteer for compatiblity testing XPNG on Linux with AMD and RADV/AMDVLK. Hint, hint, nudge, nudge. 😉

11. Until I saw Austin’s talk, I thought you guys had most of this locked down but were being stingy with screenshots and stuff.. but it turns out all of this is in very active development!

    To that end, and in regard to the trees first, The way it works now is say, there is a dirt/grass texture, and then trees over that, so a mountain 20 miles away is basically a dirt/grass mountain because the trees don’t render because of the LOD. Any thought to having like a low res “filled in” texture for far away, that will cross fade with the trees? Maybe combined with some sort of bump map or tessellation so far away mountains don’t look like 4 polygons… but don’t hit the cpu hard? I know it seems nitpicky, but we spend a lot of time looking at the high contrast area right at the horizon. I think I’d refer to this as sort of an “orthophoto like, inverse LOD kind of thing” – not literally ortho photos, but something that tries to maintain the same angular sharpness from the camera.. like close in textures should have grass and dirt, but far away mountain texture should have like… trees and major features… maybe at a far distance, the autogen can be rendered isometrically from above into a texture, and that can be used for far off stuff… sort of like an on-the-fly ortho photo kinda thing. To avoid having that huge, you could like… render color to like 512×512 tiles, but the street lights could be 4096×4096 b/w and just make the white channel be “this is a sodium white light” or something.

    I feel like there are ways to sort of lower the fidelity where it doesn’t need it, and trade it for the few places its really needed, besides just geometry LOD.

    I guess I have just thinking about this because I notice sometimes like… if I am at KLGA and want to see Manhattan, I need to see every freaking building in queens, and it probably takes a lot of work to render that grocery store at the corner of Astoria blvd, and x-plane goes “ok, guys… get rid of the Manhattan skyline, this grocery store is more important!” and thats… kinda weird. Maybe I am looking for a LOD based not on distance, but on importance. Like there is a middle distance where it can just be super low detail, as long as close by is sharp, and far away and huge is sharp. Or far away and *high contrast* is sharp (skyscrapers, lights). Or a lod based on speed and importance. Like in a Cessna, up close and very far away makes sense, but in an airliner, in internal views, the clouds and the horizon matter the most, followed by skyscrapers and generic city textures…

    This is all said with a tremendous amount of love and appreciation. I just feel like at the moment, the rendering engine seems to render a gnats butt at 1km with high priority, but honking huge things at a distance as an afterthought.

    The other answer, which we had in x-plane 10 and I think with 3rd party tools is possible in 11 is to just set the LOD to render everything and use an overclocked computer in a bath of mineral oil 😛 – not super realistic, I know.

    1. I think what you want can be achieved by using building HEIGHT and maybe also VOLUME to prioritize what to draw, not just distance.

12. I understand – thank you!
    I have RX6800 and don´t see a reason to install W10 just now. According to GravityMark(pure GPU benchmark) performance advantage in Vulkan is only 9%. This time – AMD outperforms NVIDIA in OpenGL (Mesa 21.2-devel vs NVIDIA 465.31). One wonders what is going on with W10 OGL performance??? It will flicker. No way it can be synced properly.
    We are well served in Linux now but may get stranded without OpenGL – if VLK/OGL bug persists.
    Best regards,
    S

13. Great stuff, can’t wait to see all next gen stuff come together in a final teaser. : )

    Out of curiosity: How will the lowest LOD tree billboards work when viewed from (high) above?
    You wrote there will be a billboard always facing the camera. I assume it will be a side view of the tree, rotating only on the y-axis (up-down). Will there be another non-rotating billboard parallel to the ground for the view from top?

    Thanks for the great work and giving us these insights!

    1. This is an interesting question! It goes to whether the LOD distance should be/could be/or *is* the slant line “absolute” distance to an object when said object is part of the world depiction, or to simply be the 2D “lat/lon” distance to an object. In an advanced simulation, is there any use for different kinds of LOD?

14. This looks amazing! I have been siming, for a long time, since mfs 2004, XP11 is the best user experience and the most realistic simulator ever made! What I didn’t understand, is this going to be a part of some future update? Also, I read that some 3d vegetation has already existed in current version… I am using 11.55 ver, but I can see only 2d default vegetation. Does that mean to make any modification to some files to be able to see 3d trees?
    Thank you so much for your great work and all effort you put in this simulator. It is great to see that your intention is to make it even better in the future.

15. One question regarding OpenGL compatibility in plugins and Metal.
    Will METAL allow OpenGL rendering from plugins using the draw callback or is this still an open issue that won’t be solved any time soon (meaning, it is what it is).

    Thanks

    1. I’m a bit confused, Metal already has support for draw callbacks. The only one unsupported is the modern 3D drawing phase that Vulkan has, due to limitations in how Metal and GL interop works. If you are asking about the modern 3D drawing phase, then no, Metal will not gain that one. But at the moment, there also isn’t a clear future for the modern 3D drawing phase under Vulkan either, given how the scene is rendered now.

16. “The billboards are just flat tree quads that always face the camera”
    So that also improves on XP11’s ‘trees are a thin x when viewed from above’, right?
    If so, that’s great. (and answers JJJ’s question I think)

    Johns question above – about using an even smaller forest LOD – a single (draped) polygon for the furthest away forests. Sounds like a reasonable idea to extend distant visual accuracy??

17. When will you guys fix the ATC and have AI traffic?

    1. Instead of commentating on something completely off-topic to what you’re asking for, here’s a video where such topics are already in discussion. (including AI traffic where Jim Kier of Traffic Global fame is working on it).

       https://youtu.be/wBY4X6kQ0KY

18. This is truly amazing!

    I would like to ask, if there will be a feature where you can reduce the draw distance through the settings, so less trees will be drawn. Possibly even changing to 2d trees after a certain distance?

    1. Yes – the tree LOD is managed dynamically by the GPU, so we can tune the parameters to provide different settings, including possibly a low-end “all 2-d” mode for users who want to get by with older hardware.

Comments are closed.