A while ago, Austin was on FSBreak, and I wrote this post as commentary on his interview. The main point I meant to make was this: from what I have heard from other engineers and seen in my own experience, most software companies prefer to develop new features on top of implementations that are known to have architectural problems. At LR, we fix the implementation architectural problem first, and that has been a net win for us.
Now that’s basically a statement of my opinion on software engineering – in hindsight it probably belongs on my programming blog and not here. Unless you develop X-Plane plugins, you’re not a programmer; I will try to constrain future scenery blog posts to things that non-programming X-Plane users will notice. If you are a plugin developer, you might want to look at the “Hacks of Life” posts tagged with OpenGL.
Anyway, back to the story…the responses to that blog post were all well thought out comments on X-Plane’s quality control process. At the time my immediate reaction was: that’s totally off topic – I’m commenting on architecture and they’re talking about QA. I do think the authors made fair points.
But in hindsight, I think that there’s a deeper issue: one of verifiability. Simply put, my statement that we (LR) rewrite stale implementations is nearly impossible to verify without source code access, something that you can’t get for X-Plane. So from the perspective of anyone outside the company, my original statement is not falsifiable (it cannot be proven false) and thus rather useless as a statement of fact. Even though I claim that we make rapid progress on features by keeping implementations clean, you as a user don’t care how we develop our features – clean architecture, more developers, or the use of time travel and voodoo dolls, it’s a bit moot.
Thus the comments were off topic, but also they were moving away from an unverifiable topic and toward one that users can measure, namely the quality of X-Plane’s betas.
There’s a fair amount of marketing that gets put out in the tech and games industry. It’s a slippery slope from giving a new, real technology a whiz-bang name (e.g. HyperZ is a real technology, and it is good for your frame-rates) to using techno-babble to make the bad seem good (e.g. HyperMemory just means that your video card lacks VRAM and is going to be slow). When new products come out, the feature list is parroted, but it’s not always clear whether the new features turn into real benefits.
So what I’m going to try to do with the scenery blog is: try to keep the blog limited to verifiable, measurable aspects of X-Plane. If we ship X-Plane with “psychoactive rendering*”, I’ll try to explain what the heck that is and why you’d want it, and how you might notice that it’s working.
* X-Plane does not have psycho-active rendering, except possibly when I make a mistake in the shaders and everything turns purple.
It's always valuable to put marketing where marketing is … or should be.
Thank you for your honesty,
and huge congrats for your blog and your work at LR.
Keep on !
Oli
I have a great deal of experience working with code that has incredibly tangled architecture. Reminded one of a typical extended Appalachian backwoods residence. A shack with as many added on shacks as is needed by the family, kind of like the way cells divide and grow.
That said, I think it an outstanding feature of LR development that keeping the architecture clean is a top priority. Such an approach makes it easy to add coherent new features, and hopefully to debug.
What I think happened, Ben, was that the prose used to describe this was a bit carefree. Given that the beta for 9.3 was going through long teething pains, truly odd behaviours et al at the time, it was natural to blame architectural janitoring for this. Add to the mix Austin's rather ill-advised admission that he never tests his Windows releases, and one touches a match to gasoline fumes. Hence the OT diatribes on LR's internal QA.
Bottom line, though, is you guys are awesome, approachable, reasonable and eminently creative. The brilliance that is in X-Plane is comparable to the sort of thing one sees in high end graphics and development packages, yet Graphsim only allows you to charge $29. Ridiculous, from a fairness standpoint. Insanely good luck for the new user.
I agree with Oli…congrats on all you do, and thanks huge for the continued, inspired development of X-Plane.
Steve
Hi Steve,
One or two things:
1. Pricing – it's not GraphSim, it's the big-box retailers – it comes all the way from them! Our distro gets squeezed just like us.
2. Testing windows – Austin _does_ test windows…what he does not do is "smoke test" each build after the final compile. (Since 98% of real dev is done on OS X now, OS X is naturally the "last tested").
Same no-smoke-test goes for Linux…the reason we don't smoke test per build is that the number of per-OS per-build failures that get by _and_ would have been caught in a smoke test is really, really low.
(Sometimes they do get by and we feel like idiots of course. 🙂
I'm not advocating our Q/A process in any way for anyone else or even us. But I can say that when I look at our x-platform bug stats, it doesn't make me say "smoke tests would be a good use of time."
One reason why I think a smoke test wouldn't catch much is that when we do platform-specific work, we do platform-specific testing. But X-Plane is as much x-platform as we can get it to be, to cut dev times down.
3. We untangle architecture of _implementation_ but often not of _interface_. For example, the panel SDK is a nightmare, because it supports so many historical weird combinations of functionality. So far we've dealt with this by suffering, rather than by accelerating the deprecation schedule for those SDK features.
(When people complain that X-Plane breaks compatibility, I think to the time spent making the panel code keep working and fume. 🙂
So basically if the top interface is clean, we can clean out the code…but when we set up an interface that's crazy to start with, we try and try to make the implementation sane but there's only so much you can do.
At least on the graphics side of things I see, the panel code is the only interface that's really out of control, at least that I can think of right now.
Thanks, Ben. Lots of good thought fodder there.
Sorry to go a bit OT, but as to the panel code – branch it. A simple set of radio buttons at the top: Legacy configuration, 9.4 and beyond configuration. One box is ticked and we get the current gobbledegook. Tick the box for 9.4 and up, we get the new slick, clean, well organized command setup interface that leaves the deprecated stuff in the dust. Might be something for X-Plane 10. Heck, you probably thought of this too, at some point!
Steve – branching it isn't a bad idea, it's normally in our tool chain, and it may yet happen.
The panel code has so far been resistant to branching because of it's scope. The panel system has multiple pathways at _many_ different levels in a layered design. To truly branch cleanly we'd have to:
– Duplicate the entire panel "stack" (it's a lot of code). Some of those layers may actually be other subsystems in X-Plane .. I'd have to dig in and I'd be petrified as to what I find.
– Then recognize when we are "fast path". This part wouldn't be so bad because we can do the eval at panel-load time.
It is an interesting idea though – code is cheap…could the whole thing just be "cloned off"….