HNHacker News
TopNewBestAskShowJobs

kitteh

873 karma · joined July 16, 2019

submissionscomments
kitteh··on Airline pilots landing at LAX report “a guy in jetpack” flying alongside them
They used UHF to talk to ATC.

Related post: https://www.airlinepilotforums.com/archive/index.php/t-56350...

kitteh··on Airline pilots landing at LAX report “a guy in jetpack” flying alongside them
Kennedy Steve is pretty famous (now retired). There's an interview with him on YouTube that's neat.
kitteh··on A Saudi prince's attempt to silence critics on Twitter
The challenge was that depending on the role, the union was involved. When that came into the picture there was a long, dragged out grievance process where someone would have to violate the policy x amount of times in y months before they'd be terminated. It was not unusual from what I've seen for folks to abuse this for years.

To be honest tho - this is the contract that both the company and union agreed to, so bad on the company for being okay and not making this a more serious infraction. I've talked to some of the union stewards about this and they basically said they wanted this data locked down harder. They said it's too easy to access and super temping and wished the company would put more protection around it. Go figure.

kitteh··on A Saudi prince's attempt to silence critics on Twitter
Telcos have this level of monitoring of accounts because employees routinely would abuse this access to find details of exs, family members, friends and celebrities (billing info, call detail records, etc.). The problem was there was no proactive monitoring - it was all reaction based upon complaints that would kick off the investigation. I asked why this wasn't automatic to detect clear abuse and the answer was "do you know how many people we'd have to fire if we went looking for abuse?".
kitteh··on Airline pilots landing at LAX report “a guy in jetpack” flying alongside them
Correct, people keep repeating this story - it's false. I know controllers who have actually controlled the U2 and SR71 and they've explained why it's not true (they don't use the same freq, etc.).

Also Brian Shul (the originator of this anecdote) and his RSO were the only SR71 crew removed from the program. There is a reason for that.

kitteh··on Players are fixing Microsoft Flight Simulator’s monuments with Google Maps
DCS has put some great training tutorials that come with it. That combined with Chuck's super detailed and colorful PDFs guides for each aircraft as amazing.

Agree that multiplayer how-to's is lacking.

kitteh··on Level 3 Global Outage
BGP is a routing protocol that is mostly used for propagating routing/reachability information that also includes additional data that can be used (communities as tags, etc).

A few years ago folks wanted to bake in additional functionality. For example, packet filters (aka ACLs) normally are deployed to router configuration files using each operators own tooling. To deploy this against hundreds or thousands of routers rapidly was a challenge for them (not good at swdev, etc.). So the idea was we already have a protocol that propagates state to every router rapidly in the network, let's find a way to bake ACLs into the BGP updates.

The result wasnt that good for a few reasons: 1) bgp state isn't sticky. If a router goes offline or bgp sessions reset, acls go away. That means if you are using flowspec for a critical need like always on packet filters you've got the wrong tool. 2) the implementation had various bugs. 3) most importantly it gave people a really easy way to hurt themselves globally. There was no phased deployment with pre and post checks. What you deployed led to packet filters being installed across the network in seconds. In most cases (depends on your config) the only way to remove it is remove the specific flowspec route or have bgp reset to it.

I've seen bad flowspec routes core dump the daemon on a router responsible for programming ACLs that led to them being unable to withdraw the programmed entry. I've seen as bugs on tcp/UDP port matches go wrong and eat lot more than intended. I've seen so many flowspec rules installed on a network where it exhausted routers ability to inspect and process packets and you'd see flat lining of packets being dropped.

In my opinion, it's a hack around not having a good ACL deployment tool that has led to many outages in its wake.

Edit: another flowspec gotcha. Some folks like to integrate ddos tooling systems into flowspec. An example of this is if I run a network and some IP address behind me gets lit up, deploy a rule for that specific IP and rate limit traffic to it. Unfortunately, sometimes folks don't put a lot of care into making sure it can't mess with internal IPs that should be off limits. Like route reflectors, router loopback IPs, etc. I've seen situations where some networks have had a bad day due to a ddos or traffic mis classified as ddos by auto installing rules to protect something but actually impair legitimate communications to network infrastructure which then causes the outage.

Also, flowspec doesn't work like regular ACLs where you have input and output on a per interface basis - it applies to all traffic traversing a router, which makes it difficult to say which interfaces should be exempt (think internal vs external).

kitteh··on Level 3 Global Outage
Flowspec strikes again.

Its a super useful tool if you want to blast out an ACL across your network in seconds (using BGP) but it has a number of sharp edges. Several networks, including Cloudflare have learned what it can do. I've seen a few networks basically blackhole traffic or even lock themselves out of routers due to a poorly made Flowspec rules or a bug in the implementation.

kitteh··on Level 3 Global Outage
Not just discuss, but fix too :)
kitteh··on Level 3 Global Outage
It's dated and not particularly useful if you want to learn how things are really done on the internet in a practical sense. So if you read it, be prepared to unlearn a bunch of stuff.
kitteh··on Level 3 Global Outage
Ddos tracking sites are eye candy and garbage. Stop using them.

Outages and nanog lists are your best bet, short of being on the right IRC channels.

kitteh··on Level 3 Global Outage
sneak you should come back to irc :)
kitteh··on Level 3 Global Outage
RPKI is a totally diff problem here, though.

If people refuse to sign ROAs, then they don't get protection. The ARIN TAL thing is real and people have to keep fighting that.

As it is right now you can xfer v4 out of ARIN but not v6. So even if you wanted to you can't.

kitteh··on Level 3 Global Outage
There are plenty of presentations out there. See nanog, ripe.

The books are meh because they're not written by operators. They're more academic and dated.

Plenty of clueful folks on the right IRC channels.

kitteh··on Level 3 Global Outage
Root cause identified. Folks are turning things back on now.
kitteh··on Level 3 Global Outage
Most of level3s settlement free peers aka "tier 1s" have shutdown or depreffed their sessions with them.

Example: https://mobile.twitter.com/TeliaCarrier/status/1300074378378...

kitteh··on Level 3 Global Outage
Massive reconvergence event in their network, causing edge router bgp sessions to bounce (due to cpu). Right now all their big peers are shutting down sessions with them to give level3s network the ability to reconverge. Prefixes announced to 3356 are frozen on their route reflectors and not getting withdrawn.

Edit: if you are a Level3 customer shut your sessions down to them.

kitteh··on Accused spy Alexander Yuk Ching Ma evidently beat the polygraph
Someone I knew involved in defense procurement explained the proliferation of such devices. Basically these small companies who make these things will hound politicians saying they've got a device that can save soldiers/police lives but your bureaucrats who handle supply chain sourcing and testing are slowing us down - please help us accelerate this and we can save lives. They'll usually bring up that this is a small company who is competing with the military industrial complex keeping them out of the game. This usually results in one or multiple politicians pressuring the applicable agency, to the point where they'll threaten to withhold funding to consider a trial or small purchase. Fast forward a year and these devices are deployed and thrown away due to the fact they obviously don't work. It's worse in the law enforcement circles where they actually believe the stuff until a court gets involved and things get thrown out and guidance is given to drop the crap.
kitteh··on Accused spy Alexander Yuk Ching Ma evidently beat the polygraph
US LEOs are big on traffic stops + delay tactics to buy time to have the drug dog arrive and sniff a car. Back when the show "Live PD" was on it was used heavily and officer calling a "hit" on the dog in some areas sometimes seemed like a stretch.

Coworker of mine was a sheriff deputy in Indiana and he confirmed this was a common practice.

kitteh··on What’s Flying Above Us?
206s seems to be the most popular out there. Plenty of room in the back for the surveillance gear and there's a few companies that cater to fitting in all the gear in there (FLIR/TV tracking pod, additional radio gear etc.).

The step up are the B1900s that CBP has which have additional sensor capabilities.

kitteh··on I want to have an AWS region where everything breaks with high frequency
This sounds like a legit complaint. Does anyone call them out on this?
kitteh··on That UPS you bought for your home server may not be as useful as you think
The telco Central Office I worked in had weekly tests where we'd run the generator for an hour. Everyone's desk computer required a UPS as they were on utility. Comically after a few years you'd have to get a new UPS at your desk as theyd die from continual use.
kitteh··on That UPS you bought for your home server may not be as useful as you think
A320 and B737 had special thrust bump modes for short fields like KSNA. Going back many years to the P&W JT8D you could operate in the red provided you did it for only a certain amount of seconds and logged it down.
kitteh··on Photos from Bell Labs Datacenter in 1960s (2019)
Yes.

When I started at the phone company back in the day it was post breakup. The waste that went on when there was no competition was sickening. Many of my co-workers told me about all the abuse and theft that comes with being a monopoly - with the cost being passed down to the customer and the tax payer.

Also the world of the Internet, the openness of IP protocols disrupted them. They finally had to deal with technology that wasn't dictated and controlled by them.

kitteh··on Photos from Bell Labs Datacenter in 1960s (2019)
I liked the Nortel color scheme. If youve spent enough time in COs it just seemed like a nice change.

BTW the scene in Robocop (the first one) where he plugs into a computer is actually a CO with DMS100s.

kitteh··on Photos from Bell Labs Datacenter in 1960s (2019)
I like the FORE ATM switches, the scrolling amber LEDs displaying the hostname I thought was pretty rad.
kitteh··on Adding a fiber link to my home network
The best way I've find out if you're "on net" (have fiber going to your bldg or near it) is to befriend someone in the networking and telecom space who hoard KMZs files that map out what's installed (each telecom develops them and shares them internally and sometimes with customers). I know of folks who treat these like gold and carry them from job to job and have most of the US and other nations covered. These are the folks I call when I need an address lit up to see which options and which carrier is out there vs. contacting one who is going to try to work it thru them. Most metro areas if you're on net you can get dark fiber for a flat monthly fee. Construction and permitting will bring an additional cost and things are delayed quite a bit with covid.

I've known a few folks who've got their own dark (or point to point wireless) to colos in downtown areas where they and their friends will rent half a rack or less to drop in a router and buy cheap transit and become their own little ISP.

kitteh··on Feds move to block California’s net neutrality law
The problem is that the laws in some places will make it hard for you to become the ISP in some of those spots due to the incumbents.

There are folks on the left who aren't on board with net neutrality. I'd say the common ground between left and right is about having viable competition, which we don't have. But we kinda tried that with the 1996 telecom act and that failed.

I'd rather municipalities get in the broadband access game as a utility and be transparent about costs and utilization and provide interconnection to an ISP upstream at standard tariffed rates.

kitteh··on Why is America stuck with bad headlights?
Know one major US city where the state police ran a campaign on the various interstates entering that city (and the surrounding unincorporated roads) where they were writing tickets for lights that violated state law. Went on for a few months and there was backlash (from suburban teens families) to the right folks in the state that got them to stop the campaign.
kitteh··on A Surprise AWS Bill
I do recommend you spend some time looking at the quality Oracle provides. Their regions plug into transit providers (NTT, Level3, etc.) and their peering footprint is damn near non existent (they seem to be in the process of trying to fix it). Only reason I bring this up is try to send traffic to an eye ball network at peak on Oracle vs. Google/Azure/AWS at peak and you can see the difference in terms of packet loss / throughput. This is because those eyeball networks you have to be directly connected to since they run their transit hot at peak.
← PreviousPage 2 of 8Next →