Physics and Magic

Perhaps I am too optimistic, but: rendering solid, opaque objects that look realistic in real-time (read: for games and rendering engines) is rapidly becoming a solved problem. I look at the bleeding edge of the top-tier shooters as a preview of the direction for the field of real-time rendering*, and for the last year, physically based rendering (PBR) has been front and center.

I’ll write more about PBR in another post, but the short version is that, in PBR, the equations for light interacting with materials in a rendering engine are modeled more closely on real-world physical equations; the goal is something that is very accurate to real world optics, and produces realistic results over its entire range of inputs.

PBR is made possible by increased computing power on the GPU.  (And here when I say computing power I really do mean computing, or ALU power; PBR requires more math per pixel to calculate for a wider range of optical effects, often with more expensive but better approximations of the underlying optics.)  We now have big enough GPUs to do “real physics” (well, real-ish) when it comes to light and solid objects.  This is a huge step forward when you consider that a decade ago, calculating lighting equations per-pixel was marginal and had to be done in the wrong color space for speed.**

So for solid objects like airplane fuselages, runways, buildings, baggage trucks, etc., we have physics.

Then there are clouds, and the air, and rain and fog.  For those we have magic.

By magic, I do not mean something truly magical, something outside the realm of what is possible with computer graphics.  Rather, I mean the definition a professional magician like Penn Jilette might use: trickery.  To quote Penn***:

The only secret in magic, there’s only one, and that is, the secret must be ugly.  You cannot have a beautiful secret…So, in magic, what you want is an idea that is not beautiful…If I have to say, he’s lying about that, and there’s gaffer’s tape over behind there, and, ah, they’re not actually telling you the exact truth here, and it gets so…you don’t get an a-ha.

But there’s another side to magic: to make the trick look magical, the magician has to practice – the trick is a craft, and it can be finely honed.  That’s why, before Teller does the Red Ball trick, Penn has to tell you how it’s done; you can’t appreciate the workmanship if you don’t know how much workmanship went in.  The secret of the trick may be ugly, but the craftsmanship can be beautiful.****

Clouds, fog, atmospheric scattering – in X-Plane (and pretty much all rendering engines), these are not physics, they are magic, meaning they are tricks.  The sky is a hemisphere around your head, like in The Truman Show.  The color of the clouds are just painted onto textures with photoshop.

It’s trickery, but I feel no shame in these things, because the real physics are approximately a gajllion times more expensive than we can possibly afford on today’s hardware.  A single square meter of ground (on an overcast day) might be lit by 10^18 photons per second – a lot more when the sun comes out, and each of those photons can take a different path through the atmosphere, bouncing off molecules multiple times before arriving at a surface.  The actual optics of clouds and the atmosphere is incredibly complex.

The hard work and craft of the trickery is in making all of the tricks work together to make a unified effect, and this is where some of my time has been spent over the last few days.  It is frustrating work because there isn’t a right answer (the way there might be in the programming of physics equations); instead it is a series of tricks and compromises that try to make an illusion as convincing as possible given limited resources.

Whenever I describe the code behind atmospheric scattering, clouds and fog to Chris, he is always horrified by the complexity, by the layers of tricks.  But such complexity matches Teller’s the definition of a functional magic trick:

Make the secret a lot more trouble than the trick seems worth. You will be fooled by a trick if it involves more time, money and practice than you (or any other sane onlooker) would be willing to invest.

If clouds and scattering weren’t so much trouble, they wouldn’t be magic.

https://www.youtube.com/watch?v=FFq-fn-nSZI

* The field of real-time rendering includes computer programs that produce graphics at high framerates, like games and CAD programs.  This contrasts it with the movie industry, where they can spend two years with a giant render farm to produce 90 minutes of final output.

** In 2004, we had the GeForce FX series and the Radeon 9700.  The 9700 was insanely powerful for its time, but it only had a 24-bit internal floating point path, a 96 instruction shader limit, etc.; you could do per-pixel calculations but burning shader instructions to work in linear space wasn’t on anyone’s mind.  The Geforce FX series was significantly less powerful.  This chapter wasn’t published until 2007 – 3 years later.

*** The link is to a Radio Lab story about a man who tries to understand how his grandparents performed a mentalism act on the radio – it’s a good listen!

**** In Fool Us, many times Penn and Teller knows exactly how the trick was done, but still praise the magician for their presentation – because they know how the trick was done, they know how hard it is to do it well.  Within the show, they seem to have a preference for tricks that are not easy and require skill to perform.

About Ben Supnik

Ben is a software engineer who works on X-Plane; he spends most of his days drinking coffee and swearing at the computer -- sometimes at the same time.

View all posts by Ben Supnik →