FAA NOTAM System Outage | Hacker News Reader

FAA NOTAM System Outage

fly.faa.gov

422 pointschucksmash217 comments

Fun stuff about NOTAMs: https://fixingnotams.org/wp-content/uploads/2019/11/Field-Gu...

Fun but deadly. They’re not wrong about the absolute avalanche of irrelevant data that will be used to show you “you should have known” - hence the proliferation of “interpretation” services.

But it’s up to airline how they present NOTAMS to the pilot in the briefing package. They can (and do) use geolocation present in some (many) NOTAMs to filter them out. For example do not show the NOTAM just because the flight path intersects with the airspace. They can do even better - consider not only location but also altitude range, the best flight planning systems can consider also estimated time of arrival of the plane near the location of the NOTAM and filter out those which are not yet valid or those which are expired (the validity range usually present in each message). This makes the problem of including the NOTAM a 4-dimensional - geographical location 2 dimensions, altitude 1 dimension and a time. Not the easiest to solve, but solvable. Of course for every dimension margins can be applied to accommodate re-routes or departure delays, or ATC commands. There is no written in stone standard of how to present NOTAMS. Newer may be displayed at the beginning, etc Not to say that they can (and do) separate airfield and airspace messages to segment the large set onto meaningful subsets. I mean there ways to make sure important information is not lost and less important available if needed.

Airlines aren't the only things doing "air missions" - there is software that filters out much of it, but it IS quite archaic (similar to how the "official weather" notifications are set out).

Look at the Ukraine one for a pretty scary example.

Nit: it's just "Ukraine" not "the Ukraine." See https://en.wikipedia.org/wiki/Name_of_Ukraine

Try replacing it in the sentence - it doesn't work, "the" is modifying "one" not Ukraine.

I suppose it could have been written "the Ukrainian one" but the the would still be there.

Whoops - yup, I jumped the gun there.

Double nit: this was not a misuse the deprecated term "The Ukraine" but the correct use of 'the' attaching to "Ukrainian one" referring to the NOTAM basically telling everyone to remain clear of Ukrainian airspace.

Triple nit: “airspaces” - there are 5 FIRs and 1 UIR in Ukraine https://uksatse.ua/index.php?s=330da740cf9126adde95bad0face2...

nit RFI: What is it about "Ukraine" (phonetically maybe?) that makes me, a westerner in the US, want to put a "the" before it every time I reference it as a country instead of just, you know, using the name of the country?

I don't go around calling other proper nouns "the" (mostly). I don't call my son "Hey there, the Bob, how was school today?" (his name is not bob). But if I had named my son Ukraine then I feel like I'd change that sentence to: "Hey! The Ukraine, how was school today?"

Why is this a thing? I have a formal background in applied/computational linguistics and a bit of trivial searching doesn't really reveal why this is a general tendency. Hurt's my brain meat.

Here's some that you want to "the": Alps, Matterhorn, US Navy, Rock.

Depending on your age and general personal history, in the past it was very common to see references and use of "the Ukraine" to talk about the region/area as well as the country (that mostly covers that geographical area). It could be remnants of your recollections popping up subconsciously.

In this case it really was meant to be projected as similar to "Have you seen the Bob document yet?"

Maybe because it's similar to "the UK" (United Kingdom).

“The Yukon”

In German (maybe some other languages too?) it's common to use "the" to refer to a 3rd person, so if Bob's sister Charlotte came home from school but not Bob, she could say "The Bob went to the mall".

Seems like it's the same way in Finnish, Finnish F1 driver Mika Häkkinen referred to Michael Schumacher as "The Michael".

Whoops - yup, I jumped the gun there.

The official weather notifications (METARs/TAFs) can be displayed in more modern ways, but many pilots like the brief coded weather messages because they can be easily skimmed for an entire route of flight.

In my mind, AIRMETs were a bigger problem - they had limitations in terms of how they were geographically defined, time range, and lack of flexibility (e.g. differentiating turbulence vs LLWS vs strong surface winds for AIRMET Tango) - but that's been fixed with the newer G-AIRMET feed[1], which is based around BUFR[2] instead of plain text.

The catch being you can't easily convey G-AIRMETs via voice or in an non-interactive environment, so we still keep the legacy text-based AIRMET feed around in parallel.

[1]: https://www.aviationweather.gov/gairmet/help

[2]: https://en.wikipedia.org/wiki/BUFR

100%. As a GA pilot who mostly flies under IFR, the NOTAM system is borderline useless to me.

For every warning about an approach being altered or a runway closed, there are 100s of "birds invof airport" or "unlit crane 50' AGL 5 miles from the field" NOTAMs. And even for the relevant NOTAMs, Foreflight will highlight them for approaches etc and ATC won't send you to a closed runway or taxiway.

Yeah, NOTAMs come into play when the shit hits the fan; radios are dead and you can't reach the tower kinda thing, but in that case just be sure you declared emergency, then they (probably) won't get you much for landing on a closed runway or a taxiway.

It works in both directions. If someone hits a crane in the fog they will present a NOTAM and you cannot argue that you have not been warned.

What if the tower’s radio goes down?

They have backup transmitters and if everything fails (tower evacuation has happened) you talk to the overlying approach / tracon / center frequencies to check. And then after that, land anyway and self announce positions for other traffic.

It's basically the same as landing at an airport that doesn't have a tower. Happens every day at hundreds of airports.

Seems like it's the sort of thing that's borderline useless right up until the moment it's absolutely essential.

Like the smoke alarms in my house, which are actually wired up to mains so don't technically need a battery, but they still have to have a battery in them and will give high pitched chirps ever few minutes when the battery is dead. And since there's so many in the house, and the chirp only comes every few minutes, it's a PITA to find the exact one that needs replacing.

So annoying, and again it's wired to mains so it worked without batteries! Except in a fire the fire might fry the house's magic box of energy delivery leaving only the batteries as the power source and then I really really really want them to work.

I wish those mains-powered ones would just shriek continually if the battery was low AND mains power failed; then you could kill the breaker and find it easily.

I wish they just had a steady LED from the mains power on them when the battery was dead. Seems much easier to me.

Yeah, I bet one somewhere has it, but most just flash-when-chirp which is kinda annoying:

This alarm has a low/missing battery monitor circuit which will cause the alarm to “chirp’” approximately every 30 - 40 seconds for a minimum of seven (7) days when the battery gets low. The chirp will be accompanied by a flash of the Safety Light. Replace both bat- teries when this condition occurs.

Some of the newer ones come with a built-in Lithium battery that supposedly is good for 10 years.

We recently had a house burn in our neighborhood during a power outage that was running on a generator. Generator itself caught fire, so pretty much would cover that exact situation.

Yes, it wasn't like this in my house until some years ago when we had a small expansion constructed and the contractor installed these in the new portions just as a standard thing he does, and said he was retro fitting the rest of the house that way too. ( We know him, friends, so we know he wasn't just trying to get extra $ out of us)

For a few minutes I thought it seemed a needless detail, until I realized that often enough people forget to change their stand-alone unit batteries and maybe deal with the the occasional chirp in an "I'll get to it" mindset until it runs down completely. Having it tied to the mains solves that because it's relentlessly annoying to hear that chirp at random intervals, and the battery is a nice backstop failover from the mains going unexpectedly molten.

I mean, it is code now to have them hardwired up. Actually, I think the requirement might be for interconnectivity rather than mains power. It's probably both in some areas. It's quite possible, depending on the extent of your expansion, that he was required to retrofit all of the smoke detectors in the home.

Interconnectivity is separate from mains (though both may be required) - the modern ones have to be set that if ONE goes off, ALL go off.

For those playing at home, get the ones that are BOTH ionization AND photoelectric: https://www.nfpa.org/Public-Education/Staying-safe/Safety-eq...

I've had way, way, way, too many 'chirp' at 4am situations in my life. The alarms have _literally_ trained me like Pavlov's dogs that it is _always_ a false alarm, to the point where my sleep deprived brain stumbles through the door blindly into the hallway without even checking if there's heat or smoke.

Some of the other solutions here like cry wolf only if battery low _and_ power's out, or a dedicated LED to indicate a problem, or the one I keep thinking of; gently _saying_ there's a problem with a small sound clip of "bat" instead of an angry Chirp that must wake someone up would really help. The alternative that I've heard many others take is to just disable them entirely if you're in a rental. I'll admit that I've been very tempted to do just that for individual alarms which are overly sensitive to bathroom steam.

> gently _saying_ there's a problem with a small sound clip of "bat" instead of an angry Chirp

Those do exist! At least one smoke detector in my house has voice (primarily to differentiate between smoke and CO alarms)

I started replacing the batteries in all of them as soon as the first one starts beeping.

I figure the batteries won't last much longer in the others and I don't need to go hunting.

Your analogy is way off. If smoke alarms worked like NOTAMs, they’d go off every time someone strikes a match, lights the fireplace, turns on the gas cooktop, uses a hair dryer, or plugs in any appliance. Oh yes, when the house is on fire too. All have non-zero risk of inferno.

NOTAMs are full of pages and pages of goop — inconsistent, impossible to accurately filter, hard to read, at times unclear, and mostly noise.

I think you’d not argue that if you flight to the airport without ATC (there are plenty of those), reading NOTAMS about that airfield is kind of essential exercise before flight.

It's all fun and games until you have an F-16 head butting you because you didn't pay attention to NOTAMs and the Vice President happens to be visiting your town.

Interestingly, that seems to be a US-only problem.

The linked booklet specifically refutes that idea, calling out Australia and Greece in particular but implying the problem is much broader.

This is my favorite quote: "In an unintended twist of irony, the agencies seeking to cover their legal ass are party to creating the most criminal of systems – an unending flow of aeronautical sewage rendering the critical few pieces of information unfindable."

"There are three parts to the problem: the system, the format, and the content. The system is actually quite amazing. The AFTN network connects every country in the world, and Notam information, once added, is immediately available to every user. Coupled with the internet, delivery is immediate.

The format is, at best, forgivable. It’s pretty awful. It’s a trip back in time to when Notams were introduced. You might think that was the 1960’s, or the 50’s. In fact, it’s 1924, when 5-bit ITA2 was introduced. The world shifted to ASCII in 1963, bring ing the Upper and Lower case format that every QWERTY key board uses today, but we didn’t follow – nope, we’ll stick with our 1924 format, thank you."

https://web.archive.org/web/20230104130153/https://www.faa.g...

"The FAA is changing the format for Notice to Air Missions (NOTAM) to align with international standards. The transition to the new format will ensure U.S. NOTAMs are compliant with standards set by the International Civil Aviation Organization (ICAO) and promote further global harmonization among neighboring Air Navigation Service Providers (ANSPs)."

ICAO NOTAM format

https://vat-air.dk/files/ICAO%20NOTAM%20format.pdf

Example! (decoder ring not included)

"Q) EGTT/QMRXX/IV/NBO/A/000/999/5129N00028W005"

I want to point out that lower-case is possible with ITA2. Start at 4.6 of ITU-T Recommendation S.2 / 11/1988 in the Blue Book, basically 11111 & 11011 11111 switch case in locking/unlocking mode.

Its even more amazing when you really sit down and think about all of the systems (many highly resistant to change like aircraft navigation systems) that all connect and utilize this service.

What appear like simple changes and proposals in the aerospace realm are extremely hard to execute and many times take decades (multiple) to come into play. Global services/systems are even harder.

OpsGroup, the organisation behind that guide is also worth mentioning. It's a self-organised advocacy organization for flight operations personnel. Apparently they have a longstanding campaign to reform the NOTAM system (as explained in the guide).

They also have a summary of today's incident: https://ops.group/blog/us-flights-delayed-after-notam-system...

This is hilarious. Really appreciate the examples of irrelevant clutter obscuring important information. Any UX designer can learn from this as an example of what not to do.

There's a ground stop for all flights now.

GROUND STOP ALL FLIGHTS / ALL DESTINATIONS EXCLUDES MILITARY AC AND MEDEVAC FLIGHTS DESTINATION AIRPORT; ALL FACILITIES INCLUDED: ALL GROUND STOP PERIOD: UNTIL 1430Z REASON: EQUIPMENT OUTAGE REMARKS: US NOTAMS SYSTEM DOWN

This might be a silly question, but is that a NOTAM saying that NOTAM system is down? How does it get out?

As I understand it there's a hotline and periodic teleconferences and webinars today that for sure everyone at the airlines is participating it.

The website and the system that gets it out to all the planes are related but not the same.

With all the hate in HN regarding SOA, here’s a great example of adding resilience by separating concerns at a systems level.

This is an Air Traffic Control System Command Center (ATCSCC) advisory: https://www.fly.faa.gov/adv/adv_otherdis.jsp?advn=28&adv_dat...

NOTAM is a notice to alert aircraft pilots of potential hazards along a flight route...

Command control can see that NOTAM system is out and publish advisory. (Not via NOTAM system)

Reminds me of that screenshot of a windows popup unable to display an error.

I will never forget trying to set up a printer on a linux desktop 20+ years ago: tried to print a diagnostic/test page. All that printed was a single line of text: "Unable to connect to printer."

Why don't they use lower case letters?

https://fixingnotams.org/wp-content/uploads/2019/11/Field-Gu...

I'm surprised there's not more activity on this here on HN. I guess info is limited, so I found what I could from ATCSCC notices for today [1]. These are all the notices but portions only w/ redundant bits removed. See link in citation if you want full messages. Times here appear to be GMT:

    004 DCC 01/11/23 NOTAM OUTAGE HOTLINE_FYI 01/11/23 00:47
        THE NOTAM OUTAGE HOTLINE IS BEING ACTIVATED TO ADDRESS THE EQUIPMENT
        OUTAGE ISSUES FOR THE UNITED STATES NOTAM SYSTEM.

    013 DCC 01/11/23 NOTAM SYSTEM EQUIPMENT OUTAGE_FYI 01/11/23 04:18
        THE UNITED STATES NOTAM SYSTEM FAILED AT 2028Z.  SINCE THEN NO NEW
        NOTAMS OR AMENDMENTS HAVE BEEN PROCESSED.  TECHNICIANS ARE CURRENTLY
        WORKING TO RESTORE THE SYSTEM AND THERE IS NO ESTIMATE FOR
        RESTORATION OF SERVICE AT THIS TIME.

    014 DCC 01/11/23 NOTAM OUTAGE HOTLINE_FYI 01/11/23 04:19
        HE NOTAM OUTAGE HOTLINE IS BEING ACTIVATED TO ADDRESS THE EQUIPMENT
        OUTAGE ISSUES...

    015 DCC 01/11/23 NOTAM OUTAGE HOTLINE_FYI 01/11/23 07:42
        THE NOTAM OUTAGE HOTLINE IS BEING ACTIVATED TO ADDRESS THE
        EQUIPMENT...

    016 DCC 01/11/23 NOTAM SYSTEM EQUIPMENT OUTAGE_FYI 01/11/23 07:44
        THE UNITED STATES NOTAM SYSTEM FAILED AT 2028Z.  TECHNICIANS ARE
        CURRENTLY WORKING TO RESTORE THE SYSTEM AND THERE IS NO ESTIMATE FOR
        RESTORATION OF SERVICE AT THIS TIME. THERE IS CURRENTLY A HOTLINE IN
        EFFECT WHICH HAS NAIMES/FAA FACILITIES/STAKEHOLDERS IN ATTENDANCE.

    020 DCC 01/11/23 NOTAM OUTAGE HOTLINE_FYI 01/11/23 10:58
        THE NOTAM OUTAGE HOTLINE IS BEING ACTIVATED...

    030 DCC 01/11/23 NOTAM OUTAGE HOTLINE_FYI 01/11/23 12:37
        THE NOTAM OUTAGE HOTLINE HAS BEEN TERMINATED.

In short, the terminology being used so far is "equipment failure".

[1] 1/11/2023 link to the day's messages w/ index numbering for each one per my above quotes" https://www.fly.faa.gov/adv/adv_list.jsp?WhichAdvisories=ATC...

Probably because it’s not about a package manager , a social network, Cloud Service and a chat application.

[heads-up: 'r/He\ is/I\ am' in your personal landing page]

I wanted to keep my above comment to the data only, so I'll post this here: "Equipment Failure" can cover just about anything. I think the "gut" reaction the public might have here when hearing that phrase is that it was physical equipment, but really it could have been software, network linkages, really anything. I hope the real cause here doesn't get buried somewhere in lack of follow up.

Also I really wish I'd been awake when the hotline was active (# was in the full notices linked) so I could call in and see what they were talking about as a silent observer. I can't tell from the context if that was a general status hotline, a failover service to provide the same data NOTAM would have done (though if that were the case then grounding flights wouldn't have been necessary) [1] or discussion of the actual problem and specific efforts to resolve.

[1] Well, only if it was a more benign & limited cause that wouldn't be expected to spill over into other systems. If the nature of the problem included the possibility of a malign actor then other vectors of attacks would not have been (and maybe still aren't, if that was the source) off the table.

> I wanted to keep my above comment to the data only, so I'll post this here: "Equipment Failure" can cover just about anything.

Stuxnet was after all an equipment failure. Sure, with an interesting plot to get there.

(Just commenting on the term "equipment failure", don't know anything about the FAA failure of today.)

FAA and most all global air systems and services tend to use their own nomenclature for communicating to pilots -- I would read "equipment failure" as a term that pilots would use and understand as "a temporary or permanent failure of software or hardware in the system" and not read into it any additional common sense from other areas of expertise.

The limited activity is probably because it's actually not that exciting from a technical perspective.

The answer is almost certainly something like "lack of funding forces FAA to continue using massively outdated systems and this is what happens when someone trips over the power cord of the mainframe in some basement".

It's much more interesting when companies like Google or Facebook go down, because they properly fund their infrastructure and there might be something for the HN crowd to learn.

For anyone looking for previous FAA outages: In 2014, an U-2 plane’s flight plan crashed LAX’s ERAM system because the system ran out of memory.

https://www.oig.dot.gov/sites/default/files/FAA%20Actions%20...

Why do secretive spy planes need to file flight plans?

Because they're not always doing secretive stuff, and even their secretive missions may start and end in airspace they have to share with friendly airliners.

Sometimes it's a deliberate "we see you", too. https://twitter.com/Gerjon_/status/1475444704250630148

That was a training flight, it's in the PDF.

I imagine operational flights might "officially" start and end as routine training flights, at least on paper.

LAX, an air traffic control tower, does not employ ERAM. Los Angeles Center (ZLA), the overlying air route traffic control center, does.

https://twitter.com/AUStinAirport/status/1613134228321087488

> An FAA system outage is causing ground stops at AUS and other airports across the country. Arriving & departing passengers can expect delays this morning & through the day.

If you’re flying today, it’s gonna be a rough one.

edit: BBC now confirming a full ground stop. https://twitter.com/BBCBreaking/status/1613151796955856896

https://www.fly.faa.gov/adv/adv_otherdis.jsp?advn=28&adv_dat...

https://twitter.com/FAANews/status/1613171400801394688

" Normal air traffic operations are resuming gradually across the U.S. following an overnight outage to the Notice to Air Missions system that provides safety info to flight crews. The ground stop has been lifted...We continue to look into the cause of the initial problem"

Very entertaining to watch the local TV news this morning.

The reporter keeps pulling people aside as they arrive at the airport and asking them what they're going to do with all the planes grounded.

None of them know it's happening.

So far, they have all admitted that they either don't look at the news, or don't look at the local news, except for one woman who defiantly said, "It wasn't on Facebook," as if that meant that since the grounded flights weren't in her social media feed that it wasn't real.

REALLY IMPORTANT NOTAMS tend to alert pilots to RESTRICTED AIRSPACE or AREAS OF UNUSUAL AIR OPERATIONS. You can't fly anywhere near the President of the United States, the movements of which can change from day to day.

We get NOTAMS about military aircraft operations; often military airspace which is usually permitted may be shut down for a week or so.

Wildfire tanker operations or other natural emergencies can close an airspace.

It's quite dynamic, all on top of the scheduled flight plans and routes.

They are also for notifying pilots of potential caution areas.

I used to operate a 550+ meter tower and when I had a tower light out, I had to file a NOTAM.

Cool, you filed NOTAMs; I just read them.

If you like lots of daily mail, become responsible for the operation of a Cessna 172 (Skyhawk), the type of aircraft with the most units built of any, ever. Endless variations.

Unending stream of paper NOTAMS via US first class post office... at least, when I had a 172, 20 years ago.

These notices are exactly like a recall notice on an automobile: get someone to validate the steering wheel etc. I would suppose that many HN readers have gone through the car hardware remediation process.

I would suppose that this class of NOTAM, although in my particular case the most common, are not immediately relevant to a total Ground Stop Order for the USA. But -- consider that airline and other commercial operators may have their own internal requirements that could ground a fleet in the absence of an ability to state that they have complied with all relevant FAA NOTAMS. Oh, the liability!!

(Holy cow, just did remedial maths: 550m is a Large Number! Would like to know more details.)

The tower was used by a TV station to broadcast their channels. You can see the curvature of the earth at the top when it's a clear day.

TIL that NOTAM now stands for "Notice to Air Missions."

Back when I was flying it was "Notice to Airmen." Wondered when they would get around to fixing that. Guess they finally did.

Both are still in use. For instance during your pilots tests either is considered an equivalent answer.

that's the test that makes you a Temporary Airman.

They JUST changed it. Made a point of signifying to the masses about it last month: https://generalaviationnews.com/2021/12/07/notams-are-now-no...

What this tells me is a group of government busy-workers are playing around with complex systems that can dramatically impact millions of people.

The NOTAM system is embarrassing. This just tops it off. Actually, our whole ATC system and the way the FAA operates is embarrassing. It all works kind of like waste management in NYC. If they were to add new tech and efficiency, fewer people would have jobs, so we keep doing it the old way to the detriment of everyone else.

It’s easy to say things are about jobs but do we have any evidence of that? There’s a modernization project underway at the FAA so I’d bet that if you looked there’s someone who’s been trying for years to get attention on it before something like this happened.

Shirley you must be joking.

Rumack : I won't deceive you, Mr. Striker. We're running out of time.

Ted Striker : Surely there must be something you can do.

Rumack : I'm doing everything I can... and stop calling me Shirley!

> There’s a problem with the NOTAMs…

>> The NOTAMs? What is it?

> It’s a system for alerting pilots to hazards.

>> I see, thank you

(ETA: haha aw, well I thought it was funny)

What embarrasses you about it, esp compared with more modern systems?

This seems like the kind of critical infrastructure whose uptime requirements can make upgrading challenging once in place.

Others have said it better. If you google 'notams are broken', numerous results will show up explaining various problems with the NOTAM system.

Edit: Here's a good one - https://fixingnotams.org/ending-notam-nonsense-in-the-digita...

A general inefficiency example using two airpots I often have in mind: IAH and JFK. In IAH, traffic is spaced out in cruise well before the arrival begins, where they're burning much less fuel at high altitude. They're then instructed to descend via the full STAR (standard terminal arrival), including the transition to the approach for the runway in use. This looks like a hundreds mile long line of planes all following the same path to the same runway. The only further instruction necessary from ATC is speed adjustment, and allows for an almost continuous descent to the runway provided there's no weather in the way.

At JFK, the spacing occurs much later and, even though the airport has some similar arrivals (though none quite as thorough), they instruct every altitude change and speed adjustment for every aircraft individually, along with vectoring them to the approach for the runway in use. It results in an incredibly busy radio environment and numerous inputs from pilots flying planes capable of simply following a STAR and transition, like in IAH (or ORD, SFO, LAX, etc). There's also much more time spent at low altitude burning more fuel. It reminds me of everything else in NYC, which seems to be inefficient on purpose (NYPD, MTA, DSNY, etc.).

JFK and IAH have a similar number of aircraft movements. Yes, NYC has busier airspace with numerous airports, however, I don't see how this prevents the design and implementation of more modern procedures. Flying around at 4000 feet in an airliner getting numerous vectors just seems ridiculous compared to what I experience elsewhere.

Based on this and the other great PDF someone posted, it sounds like the system itself is fine, but the content is the core problem.

This whole comment seems like a charming case of advanced Dunning-Kruger. You sound very confident in your assessment, but then it seems you are honestly suggesting that NYC TRACON controllers are being inefficient on purpose… for what? Fun times? You said it yourself— it is a busier airspace. The approach and departure spacing of EWR, JFK, LGA and TEB— not to mention the changeable airspace restrictions over parts of NYC itself and managing GA traffic into and out of the Hudson River Corridor— makes a simple single hundred mile long elephant walk to your preferred runway at JFK logistically untenable. Not to mention potential conflicting traffic from smaller airports in Connecticut and Long Island… And just a gentle reminder that as pilot in that airspace you have precisely one small piece of the big picture when it comes to being aware of exactly how complicated that can be for the ATC personal.

So basically it's the same as every other government entity?

I've got a trip to take in a couple weeks. I have a choice: 4 hours of driving followed by two regional flights. Or just forget flying and drive 16 hours point to point. Between today's issues and all the headaches at YVR over the holidays, driving 16 hours seems more comfortable and less stressful than rolling the dice with regional airline travel.

I would rather be stuck driving my car for a day than spend a day sleeping on the floor of an airport lobby.

Depending on the length of the stay and whether you need a car, I'd do the road trip in a heartbeat.

Whenever I opt for 'roadtrip', almost inevitably, about 4-5 hours in, I wish I were flying. When I fly, I don't usually say "oh, I wish I'd driven" (unless there's a delay or problem).

As I've driven more, I need to allot myself more stops. If a trip is over 10 hours, I'll probably split it in 2 days, and stay overnight some place. 20 years ago, I would have 'pressed on', but... it's usually not worth it any more. But some of the 'cost savings' of driving go away when you factor in hotels.

How many people in the car and how many drivers? What do you expect for the weather? Anywhere you want to stop somewhere in the middle?

Probably less effort and time to fly, but a lot of waiting without agency.

99% chance this won't happen to your flight.

It happened last time on a nearly identical itinerary. First flight was late. Missed connection. Landed at second airport too late for the bus (military to base where my car was parked). Had to spend night in hotel waiting for bus the following day.

Time before that, I arrived on time but my bag did not. Had to wait at airport for six hours for bag. Missed bus. Hotel. Bus the following day.

I used to need to be in a city every other week that was only a 1.5 hour flight.

I decided to see if driving might get me there in comparable time if I compared when I closed my front door to when I opened the door of my destination.

I was surprised to find that the time it took as almost the same. That's because flying requires:

* start drive to airport (45 minutes) * park and get on a shuttle (15 minutes) * get to gate (30 minutes) * wait (60 minutes because I'm early by an hour) * fly (1:45) * taxi to gate (5-45 minutes) * walk to light rail stop (10 minutes) * ride light rail too stop (45 minutes) * walk to destination (15 minutes)

Door to door flying: about 6 hours

Driving:

* start driving (6 hours) * fuel stop (15 minutes) * park (15 minutes) * arrive

If I didn't obey speed limits, I could cut an hour or more off that 6 hours.

That said, I still preferred flying. Hard to take a nap or space out while driving for 6 hours.

If I'm understanding the situation correctly: the system stopped processing updates yesterday afternoon and it wasn't a big deal, then they tried restarting it this morning to fix it and it didn't come back up, so now older NOTAMs from before the issue started are not available. Is that right?

Do airlines/planes keep a cached copy of these? How large is the NOTAM database?

Not really. And for most commercial pilots this may come down to whether or not ForeFlight caches theirs or not. I'd actually be really curious to see unofficial commentary from ForeFlight developers about this because it's a pretty unique scenario in the age of electronic flight bags.

The problem isn't keeping a copy, it's sending updates that are very time critical (for example restricted areas around POTUS, any impactful event, etc.)

As of 06:24 US Eastern the appears to no longer be true. There is a NOTAM outage but no full ground stop is mentioned, just a few specific due to weather.

Edit: obviously a developing situation, see responses to this comment for current advisories.

https://www.reuters.com/business/aerospace-defense/us-faa-sa...

Clearly still developing; we'll see.

I don't know how these things work, but the most recent advisory, 027, seems to imply that United Airlines has a full ground stop until 15:00 UTC / 10:00 Eastern.

Edit: Of course as I was typing that, Advisory 028 pops up with a nationwide ground stop until 9:30 Eastern: "GROUND STOP ALL FLIGHTS / ALL DESTINATIONS EXCLUDES MILITARY AC AND MEDEVAC FLIGHTS"

https://www.fly.faa.gov/adv/adv_otherdis.jsp?advn=28&adv_dat...

- "All US domestic flights have been grounded for several hours because of a glitch with the flight control system."

https://www.bbc.co.uk/news/world-us-canada-64236047

Yep, the situation has changed since the post was made. Parent comment was correct at the time, however.

At 5:40am Eastern, the agent at my departure gate told us that there was a ground stop affecting all flights, which was reflected in my original title. This was not reflected in the NOTAM I posted, as was pointed out by several early commenters, and in fact my flight was able to board and depart around 7:00am Eastern before the ground stop took effect.

Why do all aviation IT systems feel like they are so fragile?

It feels like not a single month can pass these days without some "Airline X has major outage and ruins tons of people's days!" story on the news.

What is the tech behind these frail systems?

More modern systems are also down once in a while. With aviation it's just more visible because one system can affect a lot of people.

I'm not sure when ATC systems (which I would count NOTAM to be one of) have last failed, I always had the feeling they're very stable. It's usually airlines that struggle with their applications.

I never know how well it applies but headline outages always make me think of that “The Coming Software Apocalypse” Atlantic article from a few years back[0]

[0] https://www.theatlantic.com/technology/archive/2017/09/savin...

I think it's just a question of media visibility. Aviation outage = queues in airports = easy article to write

All hyper optimized systems become fragile until they collapse.

The NOTAM system is ridiculous, no surprise it collapsed ... you can lose your ticket for flying thru a MOA that went active AFTER you took off.

What could possibly go wrong with a "push" notification system for every tiny little detail of aerospace scheduling? No force is as destructive as the good idea fairy.

For those who also didn't know, MOA: https://tfmlearning.faa.gov/publications/atpubs/AIR/air2501....

I'm not sure your specific example is a particularly good one. I can legally fly through an MOA that's active, without needing ATC or military authorization.

They need Kafka

They use async communication - they need Pulsar!

MOAs aren’t the same as restricted areas. You can fly in them if you want.

The risk of air-to-air collision goes up. Activities are supposed to stop when non-participants unwisely enter. Doing so may be impossible on short notice. https://en.wikipedia.org/wiki/2015_Moncks_Corner_mid-air_col...

That's the curse of niche software. Especially one made to fit, it's easy for penny pincher to decide "it works fine now, it doesn't need ongoing improvements" and you end up with some ancient shit needing java 1.7 (or worse) to run with no plan to upgrade.

Airlines view technology as a cost center.

This is the FAA's system, not the airlines'.

GP asked about both systems. It's worth noting that the FAA's systems are brittle for an entirely different reason than, say, Southwest Airlines'.

The software is tied to a dependency chain of real people (passengers, crew, etc), real equipment, and lots of constraints. So problems have a downstream effect that's more pronounced and visible than many other spaces.

When was the last time NOTAMs went down? It seems remarkably resilient actually.

Wondering if this is just standard aging infrastructure embarrassment or could involve a cyberattack? Cause if I were a state actor with a beef this is exactly the sort of soft-but-expensive injury I'd want to inflict: extremely disruptive, but also hard to assess or explain to the public.

Q: Is there anything like a systems architecture diagram for how NOTAM works? I'm curious about which parts might have failed and what the moving pieces are.

Some here, mixed in with other info:

https://www.faa.gov/air_traffic/technology/swim/users_forum/...

https://www.faa.gov/sites/faa.gov/files/2021-10/NOTAM%20Mod%...

They're resuming operations despite "still searching for a cause." Comforting.

You've never rebooted a computer to get it working again, without immediately knowing why it crashed?

I made that my career

Not since I stopped using Microsoft Windows.

I really like the advisory page -- all info, no sugar.

https://lite.cnn.com/en

It's the best way to get the news. No filler, no adds and it works on a 2g mobile.