Blog

ATC changes in 12.06 and beyond

Jim Keir here – today I’m going to talk about what’s been going on with ATC for the last six months or so, and what’s planned for the near future.
You’d be forgiven for thinking that the answer to the first is “not much”, simply because there’s been very little visible in the release notes for ATC, but that’s definitely not the case. Normal work on bugs and small improvements has been going on as usual, and 12.06 contained quite a backlog of these which I’ll go into more detail about below. Normal progress has been slowed, however, by time spent on adding one huge new feature. More on that later.

First though, a quick peek behind the curtain. This is a very high-level list of the changes that went into 12.06, now with added explanations! I’ve removed about half of it because they were maintenance, corrections to previous changes and other non-visible things, and edited some of the rest to protect the idiot innocent. Many of these changes, especially the ones starting with “Merge branch”, would be made up of many changes themselves and could cover days to weeks of work; others might be a change to a single letter.

New in 12.06

• Fix incorrect voice phrase for requested altitude change
• AI had a stupid max speed under “AI flies” at key points on the ground which was never reset
• Fix unwanted transmission possible during spawn of AI on arrival.
  This simply clogged up the airwaves for the first minute or two of your session.
• Fix for descent time calculations
• Merge branch ‘fix_XPD-13780_Pushback_Messages’
• Merge branch ‘fix_XPD-13783_Pushback_Dialog’
  These two should make the text messages and buttons in the pushback dialog clearer and more consistent with the current state. Note that you can also request pushback by radio, from ground or tower.
• Modify altitude monitoring to try and remove inappropriate warnings
  This is a long-standing bug with repeated altitude correction calls from the controller, when you were already at the right altitude. Since this change I finally got a log from a bug report that let me nail the main cause, and a final (?) fix should hopefully be included in 12.07.
• Allow editing of ATC flight info if required information is missing, even if a flight is under way.
• Take two for the “AI not calling downwind” bug.
  AI may not always have called downwind. They still called final and landed correctly, this was just a missing radio message.
• Merge branch ‘fix_tropo_jumps’
  People flying airliners found that their altitude was suddenly changing during cruise, sometimes by hundreds of feet, making their autopilot want to take a bit of a lie down in a dark room. This turned out to be two subtle bugs in creating the atmosphere’s vertical temperature profile, where it was possible to get a sudden change instead of a smooth gradient. It depended on your precise location and altitude, and having real weather which had specific temperatures at very specific altitudes.
• Latest airport names and combined voice changes from several branches.
• Merge branch ‘fix_XPD-13873_Wake_Sep_On_Runways’
  ATC will now delay your departure clearance until the wake from the previous aircraft, whether arriving or departing, has reduced to a safe level for your aircraft.
• Merge branch ‘fix_XPD-13842_Departing_AI’
  The calculation for the time you would take to line up was just plain broken; sometimes it was too long, sometimes too short. This affects AI as well, of course, so this one fix should prevent you being told to line up with an A330 on short final, and also reduce AI go-arounds. It also now takes account of how far the hold-short line is from the start of the usable runway, and whether there are any sharp corners.
  Why wasn’t this seen before? Because the estimate wasn’t always way too long or way too short. Sometimes it could be just right, and the airports that habitually get used for both manual and automated testing by total coincidence were just right. This one also only was able to be found and fixed after someone sent a logfile with full details.
• Merge branch ‘feat_XPD-12784_multi_channel_atc’
  This one’s a biggie. I’ll talk about it separately.
• Merge branch ‘feat_ATC_AI_TCAS_Awareness’
  ATC is now aware of TCAS aircraft – read “plugin-controlled AI” – when checking your line-up time against other aircraft using the runway to work out whether to give you departure clearance or not. This means that third-party AI aircraft systems should play a little more nicely with ATC, with no changes required to the plugins.
• Contact nag was broken entirely.
  Fixes the “Are you with me” if you are handed off to a new controller, change frequency but forget to check in with them.
• More AI-flies changes for specific situations. Alter the test to account for fake-flow changes.
  A lot of testing is done using AI aircraft, and the “AI flies” mode for the main aircraft. Getting that transition from human to AI done and having the AI pick up an aircraft mid-flight in any state whatsoever, is not simple! This also shows one of the automated tests being changed because it correctly failed as a result of another change.
• Fake calm flow was only using one runway even if multiple valid runways existed.
• Allow “Missed Approach” earlier during the approach.
• Speech for airport names not being generated if the apt.dat has no country code set for that airport
  Airport designers, please use correct ISO country codes! “USA”, “U.S.”, “America”, “Amercia”, “U S A”, “US of A” and so on.
• Fix several problems with missed approaches
• Add option to file and send plan to FMS, and to transfer plan from FMS
  Simple quality-of-life change for the “File a flightplan” tab in the ATC dialog. If you’ve set your FMS, maybe from a file downloaded from an online planner, you can now transfer your route to ATC in a single click. Likewise, if you’ve asked the ATC dialog to plan a route, you can send that to your FMS. Of course, you’re free to program your route into the FMS manually as before.
• AI did not avoid user if parked on the end of the runway with no ATC interaction
  If you start a flight on the runway ATC is aware of you, you will be ‘owned’ by tower and cleared for departure. The AI, however, didn’t get the memo until you made your first radio call. Oops.
• Further fix for OnRunwayStartMegaHack – was dependent on immediate radio access
  On-runway starts are an absolute bloody nightmare to handle! In this situation, a bug with the 737 where the radios would be unavailable for a frame or two after the sim started totally broke the AI.
• XPD-13764: Approach clearance cancellation was not being spoken
  You’d be given go-around vectors, but not told that your approach had been cancelled, or why.
• Synthetic flows now align to the longest runway instead of using NSEW
  When an airport definition has no runway flows defined, the sim has to invent them. Previously five flows would be created, one each for the cardinal directions and one for calm conditions. This change has the fake flows offset by the direction of the longest runway, which would usually be aligned to the prevailing wind. Hopefully this will reduce some of the valid “You’re using what runway, now?” questions – but not as effectively as airport designers actually adding flows to their airports!
• XPD-13788: Approach cancelled if perfectly on straight-in approach a long distance away
• Another runway-start fix for switching to AI
  I hate runway starts.
• Route planner will consider great-circle sections
  If you ask the ATC dialog to auto-route between two airports, it was previously only considering airways or nav-to-nav. Turns out there are no defined airways across the north Atlantic, and flights from KBOS to EGLL were flying halfway down Brazil before crossing the ocean… Now it will try to pick this up and use a DCT section if following airways looks silly…
• XPD-10555: AI can fly great-circle routes without continual corrections
  … which means the AI need to be able to actually fly them. Normally AI pop in and out of existence quite quickly and this wasn’t that noticeable – right up to the point where you ask the AI to fly a long route. Now they fly the great-circle route correctly instead of picking a straight-line heading to the destination and jink back onto the correct course every five minutes or so. This had fun consequences for trans-polar routes too; send an AI near the poles and they would just fly around it forever.
• XPD-13768: Route generation can create routes which go past the destination, and can choose the wrong fix where multiple with the same name exist.
  Two fixes here. First, if you asked ATC to auto-route to a destination, it could be that the best fix to head for was actually beyond the destination, which makes no sense as an actual route. That was discovered at the same time as finding that in some places, two different fixes with the same name exist within a few tens of miles of each other. That feels like a bug in reality to me, but there you go.
• XPD-13767: Crash when filing from an airport with no radio facilities
  Good ol’ fashioned crash bug. For 12.06, requesting IFR clearance on the ground from an untowered airport is disabled because the system was never intended to handle it.
• XPD-13762: Global region now uses mbar instead of inhg for pressure.
• XPD-13759: Adjust descent rates used to predict descent timing
  Please, folks, don’t use the “Request altitude change” over and over to manage your descent! Either wait for ATC to call it, or specifically request descent. Either way, this change pushed the decision for ATC to initiate descent out a bit.

Now for the big change – ‘feat_XPD-12784_multi_channel_atc’. This one’s been queued for quite a long time. Until now, radio frequency use was handled very simply. All controllers would both transmit and receive on all of their allocated frequencies at the same time, and you’d always be told to use the lowest frequency in their list. For small airport towers they probably only have one frequency, but for approach and regional controllers they may have more than one. In fact, some especially large controllers have dozens.

Also previously (i.e. X-Plane 11), everyone being on the same frequency really didn’t matter. AI was used less, ATC was definitely used less, and also had a lot less to say. In X-Plane 12, though, that’s changed. If you’re using both AI and ATC you’ve almost certainly found that near larger airports or even just flying cross-country legs with a controller covering a large area, you just can’t get a word in edgeways. This leads to missed turns, inability to make time-sensitive requests, avoidable go-arounds and other assorted havoc – and that was with human radio time prioritised over the AI. 

It was even worse for the AI because they are totally dependent on ATC instructions, so even short delays can put them into a situation where they suddenly need a burst of revector instructions – which increase radio load further. You can see where this is going.

With this change, a controller will actively manage frequency allocation. Instead of everyone always transmitting on a single frequency and the control facility simul-casting on all frequencies, aircraft will be allocated to other frequencies based on both anticipated and actual load. This means that, while you might still end up waiting for a burst of radio traffic to go away, you should be able to speak without too much of a delay. You might also, of course, be asked to change to a different frequency yourself (or, in betas 1 and 2, continually given this call… my bad, fixed for b3).

To be clear, this isn’t a full model of controller sectors, we just don’t have that data available. If you think of it as ‘sectors-lite’, where there are aircraft around you which are being directed by the same control facility but different staff within that facility on different frequencies, you won’t be far wrong. The short version, I hope, is that one of the key frustrations with the ATC system, a valid frustration which I saw people saying simply stopped them using ATC at all, should be gone.

And for the future? Normally of course we don’t go into too much detail about what’s planned unless it’s a hot topic, simply because shit happens and things that are planned, even if the code’s done, don’t always make it in to the next release. Or the one after that. Just this once though, I think it’s worth a little glimpse of what’s going on, just to show that outward signs of (in)activity can be misleading!

First there’s another list of fixes and improvements to come. As with the 12.06 list, here’s some of what I’m hoping will be in a very near-future release. Also as with 12.06, this is cut down and edited a little.

Coming in 12.07 (ish)

• Ensure we issue altimeter setting for aircraft that start in the VFR/IFR approach position (i.e. 3NM/10NM from the runway, airborne).
• XPD-14246: Fix discrepancy between flight model and ATC ISA altitude
  This is part two of the “stop whining about my altitude” bug.
• Prevent flow changes while any aircraft is on zone 2 or higher, instead of just on final.
  You should be less likely to be revectored for a different approach if you’re already approaching.
• Merge branch ‘feat_Untowered_Flightplan’
  This adds the ability to file an IFR flightplan with a regional controller, while on the ground at an untowered airport, including timed departure.
• XPD-14239: Radio calls used correct indicated altitude even if altimeter setting was wrong
  When you called to check in with a controller, the altitude spoken by the pilot if you were below the transition layer was the correct one even if you’d not set your altimeter correctly.
• Any airport with procedures can’t be a FISO.
  Airports with some kind of radio support can be full ATC or FISO. This was new for X-Plane 12 so, while this can be set per-airport in WED, most don’t have the information set about which control type they are so the sim has to guess. This simply says that if an airport has SIDs or STARs, it’s going to fully controlled.
• Fix Austin’s AI switching control modes at intermediate hold-short points
  AI might behave differently after crossing hold points while enroute to or from the active runway.
• Fix rotating the aircraft using the compass in the map in pause mode
  OK, I admit it, this one’s for me. I use this constantly, when testing. Previously if you used the map to change your aircraft’s heading in pause mode, the heading reset if you then changed anything else. Now the heading sticks.
• XPD-14193: Request diversion mentions the previous destination instead of the new one.
  Duh-level typo ☹
• Uncertain of Position wasn’t always available when it should have been.
• XPD-14192: Ensure the AI closes *all* aircraft doors, not just the ones it would have opened itself, when service completed.
  Yet another switch-to-AI oddity, when an AI would taxi and depart with doors open if a human opened them and the AI wouldn’t have.
• XPD-14190: Don’t continually regenerate fake flows except at load time – if the flows are unusable, this never stops.
• XPD-14189: LSO left/right commands were reversed
• Disable most ATC functions in external-visuals machines.
  Only the main computer handles ATC, so why waste cycles?
• Additional condition for allowing on-runway flightplan editing.
• Add “Departure clearance cancelled”
  If you’re on the runway with takeoff clearance and then request taxi to parking – forgot your lunch maybe – you’ll have that clearance explicitly revoked.
• Make com[12]_[rt]x datarefs work on aircraft with no Garmin unit
• Fix altitudes in ATC check-in calls
• Potential fix for being unable to contact ATC at all after cancelling services
  If you cancelled IFR in-flight then, depending on the exact state when you did it and what you do next, it was possible to end up in a situation where you couldn’t call any controller at all.
• Change some ATC calls to allow use as VFR when filed but not cleared.
  Related to the previous bug. If you filed a flightplan, the only valid action would then be to activate it by requesting clearance. If you didn’t make this request, and took off VFR/uncontrolled, some calls were blocked.
• Fix double call for “Cleared to Land” for VFR flights to controlled airport
• Allow request for a different parking spot when one is already allocated. Add new “TaxiIn” state.
  You can now request a specific parking spot even after ATC has allocated a different one.
• Expand log for inappropriate radio frequency.
  Finally, this is an example of one of the “no visible change” bugs that I’ve removed from the lists. There is a very rare problem detected in internal testing, where the system flags that an AI is on the wrong frequency. Right now there’s no clear cause, so this just adds more logging around the warning – which doesn’t appear in public builds – to help find and fix this.

Hopefully this has given an idea of some of the impact of the changes that are already in place, and what those terse bullet-points in the official release notes mean. It might also give an indication of the graceful, swan-like visible progress (hah!) compared to the furious paddling that’s normally hidden below the waterline, and I think it’s fair to say the same for all of the developers and designers, not just me working on ATC.

Looking a little further ahead

Oh yeah, nearly forgot. SIDs and STARs. If the lists of fixes and new features above look a bit sparser than you’d hope, now that Ben’s officially let the cat out of the bag I can say that this is why. SID/STAR support is written – past tense – and about to hit a lengthy period of polishing and internal testing. The key word here being “lengthy” – don’t expect to see these in 12.07, 12.08 or twelve point whatever I listed last plus one – but SID/STAR support is on the way.

Altitude Management: A Tale of Ups and Downs in the ATC System

Deciding what altitude a particular flight segment needs shouldn’t be tricky, right? You take off from somewhere near ground level, you climb to whatever you chose for cruise altitude, you start to descend when you get near the destination and eventually end up on the ground again. What could possibly go wrong?

Why, I’m glad you asked! Make yourselves comfortable, this might take some time.

A bug report arrived from someone saying they’d been vectored into a hillside on approach. That’s something that has had a lot of attention given to it for X-Plane 12, so I jumped on it and reproduced the flight. Happily the person reporting had included the correct logfile (not everyone does, we get loads where the log shows the sim sitting at the main menu!) and given the flightplan details; nothing complicated, a simple direct flight between two airports about 35 miles apart, cruising at 6000 feet.

I set it up and let the AI fly. Takeoff was fine, and then it was told to climb to 8000 feet – above the requested cruise altitude. Checking the map, okay, there was a mountain ridge in the way so that makes sense. Then the problems started. Descend to 7000, then descend to 5500, then climb to 7000, then an endless loop of climb/descend to what looked like random choices of 5000, 6000, 7000 and 8000 feet.

All of the “normal” – and a lot of the unusual – situations for approach and landing are covered in automated tests and they’d last passed the previous day, so I knew that this wasn’t a common problem but it would still be damned annoying if it happened to you.

Digging in a bit, the first thing I found was a good old-fashioned bug. Before X-Plane 12, the ATC system was pretty much exclusively written for airliners and an old bit of code had a problem where, if you were below a normal cruising altitude for an airliner, you might be told to climb slightly before starting the descent. This was kicking in because, when the AI was handed off from tower to the next level controller, it was already close enough to the destination airfield to be seen as an incoming arrival.

If you think about it, this completely breaks the simple altitude model already. The Cessna 172 climbs slowly, so it was approaching the whole “descend, approach, land” sequence from below. X-Plane 12 is wise to this though, so it shouldn’t have been a problem.

So, we’ve got one inappropriate climb instruction sorted. Re-test.

As expected, much the same as before; total indecision on the part of the controller. The next problem soon comes to light: terrain avoidance.

On the face of it, terrain avoidance should also be simple: If hill then climb. Most of the time, it is. Where that falls down is where you’re deliberately trying to get close to the ground like, say, during approach and landing.

If you’re familiar with aviation charts, you’ll know the acronym MSA – Minimum Safe Altitude. Basically the chart is divided into large squares and, in each square, there’s an MSA given. If you fly through that square, you’ll be guaranteed not to hit the ground if you stay above that altitude. It’s simple, it works, and X-Plane does pretty much exactly the same. The terrain elevation data is reduced so that a single pixel gives the MSA for whatever area that pixel covers, and this makes it relatively quick and easy to check.

This flight had a couple of steep ridges going from sea-level to about 6,200 feet which ended just inside one MSA square, and the flight was going just along the top of it at about 90 degrees to the ridges. A few hundred meters difference in location when the terrain samples were taken were enough to change the MSA between about 50 feet and 6,200 feet.

Checking a little to either side of the route can make this problem more or less go away, and that’s done in some places, but it can’t be used during approach because you’re actively trying to get close to the ground. If you went wide on the terrain checking, you’d never be allowed to land at any airport in a valley or just generally mountainous terrain.

So, back to the problem. The terrain avoidance systems were getting two very different readings depending on both exact timing and position because they were just clipping the edge of the MSA squares and one of those squares covered high ridges which finished at sea level just inside the square’s boundary. One moment it would think you had 5,000 feet of clear air beneath you, the next it was asking the poor, confused AI to climb to avoid terrain.

This problem can’t really be fixed as such, but it can be improved by reducing the sample distance and using full-resolution elevation data instead of the MSA-square summary. The problem with that is that it’s more expensive in CPU terms so it can’t be done all the time. Think of a flight from London to Delhi; checking that for every 30 meters along the route is going to take a long time, and not all of the high-res data is even in memory at once. However, high-res data is already used in some cases, notably for short legs close to the MSA or for known approach legs, but this was happening on a longer cross-country leg which happened to be the one before the actual approach started. Still, easy enough to adjust for that so let’s do that and see what happens.

The next flights were much, much better. Yay! Go for a celebratory coffee, but something nags at the back of my mind.

Coffee over, check the exact routes on the map (anyone can do this – it’s on the Developer menu as “Toggle Air Traffic Paths”). Yep. The wind had changed – I had real weather enabled – and the cross-country leg was starting a couple of miles further north, which meant that all the nonsense with clipping corners in the terrain avoidance code had simply stopped being an issue. 

Righto.

Set manual weather, re-test, and… well, it’s better, but still bouncing between two different altitudes even though the MSA’s now more or less stable.

If you remember, this is all happening during approach. When these segments are created, the terrain clearance for each one is checked and the overall approach profile is modified so that you’re not told to descend to the normal altitude, then climb again for terrain avoidance for a later part of the approach. For this flight, the destination airport was on the plains near the coast but surrounded by high mountains to the south, so this system was kicking in and keeping the approach altitudes much higher than they would normally be. However, there’s one more waypoint involved; this is created somewhere between you and the airport and used as a kind of gateway between cruise and approach. The segment after this waypoint was also being terrain-checked, but if any terrain-related adjustments were made to the subsequent approach segments, they weren’t being pushed backwards onto this pre-approach leg. Normally, this wouldn’t happen and even if the approach were modified to clear terrain, it would have to be a very unusual, large adjustment to kick the approach legs higher than the pre-approach waypoint. 

Yep, you guessed it.

Still, at least that’s a simple one to fix. Do that, re-test.

This fix is good and the next set of altitudes is… oddly low. So low, in fact, that the “oh shit” mode of the terrain avoidance kicks in. This ignores the route entirely and just uses the plane’s recent movements to predict a near-future location and checks that. This means that the MSA is being ignored somewhere, and there’s now a missing climb instruction.

More testing and this turns out to be caused by a check, during approach, which tries to avoid unnecessary climb instructions. If you’re already at low altitude and call in to an airport to land – i.e. a typical small-plane flight – you don’t want to be told to climb thousands of feet only to descend back to pretty much the altitude you’re already at. So, the controller tries to be nice and lowers parts of the approach if you’re already underneath it. That’s all fine, but… not… if… the approach has been raised to avoid terrain!

Fix that, re-test.

All is well for the first half of the flight. I know, from watching the code as it runs, that from here on in the MSA should be in the region of 5,200 to 7,200 feet depending on the exact timing. High resolution elevation data is being used now, the two mountain ridges are steep from left to right and fairly narrow back to front in relation to the flight direction, so the MSA can vary quite quickly. The expected climb instruction comes in: “Climb to FL080”. Wait; eight thousand?

This was a flight-level though. Checking the transition altitude (the point where it stops using “feet” for altitude and starts using “flight level” and a specific set of rules for choosing them) for the approach controller shows that it’s set at 6000 feet. Yet again, tiny changes in both timing and position were affecting the MSA, but this time it was because the high-resolution elevation data was being used. For the low end, a simple “add some wiggle room and round up to the next 500 feet” was coming up with either 5500 or 6000 feet. At the high end, it was being rounded up into flight-level territory and, because of the direction of flight, punted upwards to either FL070 or FL080. This was only happening because the requested altitude in the plan was exactly on the transition altitude!

Okay, so this one’s probably confusing, but not technically wrong – certainly within the limits of the system.

So the original, simple “Let the AI fly a route and see if it starts tunnelling” check has now taken three days to properly resolve, between investigation, code changes, many re-tests, and then validating the changes on a larger scale using the existing automated tests. On the plus side, it’s been an absolute torture-test for a number of systems which worked just fine in most circumstances, and has brought out problems that were able to be either fixed or at least explained. On the down-side… yeesh!

For a normal flight, none of this would have been a problem. You’d be way above the terrain, descending onto the approach from above. This flight was short, so the slow-climbing Cessna 172 was still on climbout when the approach vectors were issued for altitudes higher than it was currently at. In between the two airports there were ridges which just clipped terrain-altitude data squares so MSA checks returned altitudes which varied significantly with tiny variations in both position and timing. The destination airport had no flows defined, so X-Plane was auto-generating them, but this doesn’t look at terrain so thanks to the airport being in a steep valley, the MSA of the approach vectors unusually remained at 4,500 feet all the way in to final for an airport close to sea-level. Finally, the planned altitude was exactly the controller’s transition altitude, magnifying small differences which would normally be rounded away.

If either airport had been a few miles north or south, or further apart; if the ridges had stopped a few miles further south; if the coast had been a few miles north or south; if the aircraft in use had been able to climb to cruise altitude faster; if the plan had requested an altitude which was above the controller’s transition altitude; if the terrain MSAs hadn’t also straddled the transition altitude; if the airport had defined flows which prevented landings from the south; if the wind had been different. If any of these things had not been exactly as they were, little or none of this would have happened.

So, my – er – thanks to the person that filed this bug report! Completely by accident, you sent several systems beyond what they were able to do and, by filing the report and actually attaching the log from the same flight, allowed me to dig into the system and get rid of a good number of rough edges. It’s also given a fantastic example of “Pssh, how hard can it be?” for the obvious assumption of “well, you climb, then you descend, job done”.

Plus a couple of sleepless nights, but hey.

Oh, the “vectored into a hill” problem? Couldn’t reproduce it.