benjojo12

In the era of relatively complicated company ownership structures (especially in a capital heavy business such as Datacamp), the company on the whois does have the same level of meaning as you seem to expect it to.

207 Regent Street, it's a relatively well known virtual office type address, I would be shocked to learn that there were any datacamp employees at that address)

Great catch, there were like 3 mini incidents this week that I was keeping track of, so wires got crossed, will correct the post in a moment

(author of the post)

Yes, this is exactly what I did in the post I linked to: https://blog.benjojo.co.uk/post/bgp-path-attributes-grave-er...

It's a pretty good way to get the bill paid to be honest

I also run a B2B SaaS and I have found that over time the customers who I think are going to be terrible for paying, typically tend to be very good at paying and the customers who I think are going to be very good for paying (in that they have good standing in the community in things like that) tend to be pretty bad at paying on time (and with NET30 and all)

Ultimately I don't really understand why I shouldn't be paid on time (I've got to live too, and I have suppliers likewise) so I think this is pretty fair game, I sold you a product under the promise that you would pay in 30 days, if you're incapable of doing that (ignoring exceptional reasons) I don't necessarily want to go and lend you extra credit just because you're Mozilla

Having a terrible accounts receivable department isn't a exceptional reason.

Yup, unless that component has been disabled (which is quite rare) or the other side is bird, a bgpd that doesn't buffer anything !

Based on some quick sleuthing, I would assume that the 0xff reserved BGP attribute is likely a huawei quirk. Almost all of the 0xff's visible to bgp.tools (hi) follow the same format as the one in the post, and some of those networks with them seem to be running huawei kit.

Route dampening has mostly fallen out of fashion with networks these days.

Most setups were horribly misconfigured and (most) routers are no longer extremely CPU starved as they once were, That doesn't mean that it does not still exist of course, when I did bgp battleships ( https://blog.benjojo.co.uk/post/bgp-battleships ) I found that 3356 (at the time) was doing route dampening, so play had to be paused for a while.

Hetzner sell VMs located in the East Coast US as well: https://www.hetzner.com/cloud/

> i have yet to see any service provider use BGP confederation in production.

There are at least 26 publicly visible routes on IPv6 that have visible confederation in them.

Maybe the best/worst example is 2402:8100::/32 ( https://bgp.tools/prefix/2402:8100::/32 ) :

    # birdc s ro 2402:8100::/32 all
    BIRD 2.15.1 ready.
    Table master6:
    2402:8100::/32       unicast [transit6_orange 13:42:56.368] * (100) [i]
        via xxx on xx
        Type: BGP univ
        BGP.origin: IGP
        BGP.as_path: 5511 3356 55644 {36040 38266 45271 55410}

With 4 confederation ASNs in it!

That electro mechanical system also switched significantly less calls than the digital counterparts!

Most modern day telcos that I have seen still have multiple power/line cards/uplinks in place and designed for redundancy. However the new systems can also just do so much more and are so more flexible that they can be configured out of existence just as easily!

Some of this as well is just poor software, on some of the big carrier grade routers you can configure many things but the combination of things that you can figure may also just cause things to not work correctly, or even worse pull down the entire chassis, I don't have immediate experience on how good the early 2000s software was, but I would take a guess and say that configurability/flexibility has had a serious cost on reliability of the network

(blog post author here)

No in fact, it is so much worse! I do agree we should use IPv6, The ending of the blog post says as such :)

> If we are going to start using address space that might not work for all users, it would be wise to pick the address space that we already have a considerable head start on getting accepted: IPv6.

systemd's resolved commonly uses 127.0.0.53, Given the deployment of systemd I would not really call that a niche cases in terms of volume, but I do agree in general that 127.0.0.0/24 would in theory be enough. However the whole discussion on repurposing 127.0.0.0/8 is mostly academic as it will be basically impossible to use reliably

> That thing retails new for 10k. You got awesome friends if they have such a thing lying around unused and willing to sell it to you at a discount!

Tactical eBay (or whatever is the robust 2nd handmarket in your region) can yield similar discounts on such hardware. Retail price is often list price, and that is often a large mark up regardless. The 32x100G port version of the same switch goes for around £2000 in the UK 2nd hand market

> what's the smallest and reasonably affordable Mellanox model that has a similar featureset in terms of native Linux support?

The SN2010 is likely the smallest, the SN2700 is likely the cheapest

Sure you can install basically any EFI+amd64 distro you want, that also has the correct kernel module.

I don't really know why you would want do that (though, RHEL based would be a reasonable 2nd option)

Performance of the distro isnt really a big deal because the OS has so little to do with the day to day packet shuffling, My switch runs at a load average (crappy metric I know, but to give you a idea) of 0.04.

I apply updates with apt update and apt upgrade, as you would a normal Debian server. Only the kernel is pinned since it is special for the switch, however you can rebuild the kernel deb's when you want, as the driver is in the mainline kernel repo.

It really depends on how much you know what you are doing, If you stick to:

*) IP Routing that would normally "fit" in a vendor switch

*) Bridging

*) VRFs

You will be fine

If you try and do some weird stuff then it's best to check with "ip route" to see if it was actually installed into hardware or not, but I would simply not do anything weird on such hardware

> But that limits what you can do considerably, to the point that an off-the-rack Juniper or CSCO or whatever probably has more features than you can do here without writing your own code to hook into the mellanox sdk?

I mean, I'm not touching any mellanox sdk here, I am using the a very similar stack that someone on a "software router" would use, on a switch that can automatically accelerate it to 800G+ throughputs, while hitting a 60W power target.

You can hit some of those performance/power numbers in vendor hardware like Juniper/Cisco/Arista, however you have to also put up with their software, I (and others in my group of peers) have not had great experiences with vendor software, and in this setup I am able to patch/fix the software on my own terms.

If there is a security vuln in one section, I can fix that, and call it a day, I won't be forced to upgrade parts of the system I do not want to. I cannot do this with Juniper/Cisco/Arista always.

In my eyes, MultiModeFibre is on it's way out.

The high speed (25G+) does not have good solutions for multimode, and the length limits that physics enforces with multimode mean that it's not "no-brainer" applicable for going any more than between the same rack row.

So if you are dealing with datacenter cross connects that can exceed the max distance for MultiMode then you might spend hours debugging broken stuff for no real gain. MultiMode is slightly cheaper, but it's a false economy the moment stuff does not work correctly. I've spoken to people with DC cross connects that go into the 5km+ of cable distance. So it's easier to just stock one kind of optic per speed, and call it a day.

Equinix I think already phased out MMF XCs

> But the most salient point is that, even with "Linux on a router/switch", there's no guarantee that you'll get decent performance

As long as the ASIC is programmed the performance is the same between kernel versions. If the ASIC does not get programmed then it's like the entire device does not work at all (so you likely need to roll back the update you made).

> which usually means that you're squarely in "well, here's this blob that works on certain kernel versions, and good luck with that!" territory.

As mentioned in the post, the switch in question has blobless drivers, unlike the broadcom stuff you likely have experience with based on what you are saying

> Woah these are around $5k.

For what it's worth I did not pay 5k USD for it, that is significantly overpriced for it on the 2nd hand market.

> for business, just pay the bucks to be able to pass the buck - ie get a packaged solution not homebrew

There is a significant cost difference here,

The overall point of the post is that the "packaged solution" is not viable when most switch vendors have what I can describe as "crap" software quality and/or support.

So if you are in the land of simple L2 switching and L3 routing, this switch is amazing because you can escape the crap vendor software.

Quiet in the terms of DC grade switching, I'm not sure you could get away with such a switch in a home environment. The 100G optics need plenty of airflow to keep cool so it's not just a case of swapping the fans for something smaller

> How disruptive are switch config changes? If I edit /etc/network/interfaces to add a new port, vlan, etc

They are seamless as long as your configuration does not do something stupid like tear down interfaces to reconfigure them. The switch takes no noticeable time to program from the Linux space to the ASIC

> - does a `systemctl restart networking` (or whatever equivalent) bounce ports or halt switching for a moment while changes are applied?

"systemctl restart networking" will typically blow away even the well most well configured systems in my experience.

In the post I suggest using ifupdown (the first one), since it's the most "easy" to debug, but I'm sure networkctl works too, with a healthy amount of systemd restraint

Hi, I am the guy.

> what did this guy connect to several 100 GbE

They have 100GBASE-PLR4 optics in them, that allow the 100G ports to be split up into 4x25G ports (or actually in this switch, 2x25G ports due to a hardware limitation with this switch)

> how does the upstream connections he mentions look like, and from which provider

They are just normal 10G-LR Single mode optics, in a data center.

> The device is second hand, so the likely use-case is a Home(?)Lab type of setup.

Nope, this device now runs my business bgp.tools

Amusingly, the project/company that the person you're replying to is using .AI domain (see their HN bio), so clearly their opinions about how ccTLDs shouldn't exist can't be that strong

I am one of the programmers that worked on this port back in 2013, I can assure you that we (the programmers) also wanted to make a scenario where the plant would explode, however given it was designed to promote nuclear technology to school children (generally speaking), the people funding the project were not so keen to have nuclear technology shown off in that way!

How bizarre! Me and a colleague ( who's still a good friend ) wrote this in like, 2012-2013! Well, more accurately we ported this from a older Delphi/Pascal program that was using raw OpenGL calls ( a legitimately very impressive bit of code for what it was written in ) into WebGL ( a relatively new piece of technology at the time )

The primary reason for porting this to a modern platform was ( as far as I understood ) so that it could be run on interactive events for schools and such. It's definitely more optimised for fun than realism. I remember a few good memories of various people who were actually nuclear engineers complaining that realistically the demand would never change that fast on any commercial nuclear power plant!

LCY has a Saturday Noon to Sunday Noon curfew.

It makes the life's of people like me who live near it much better! Even if it means that weekend flights from my nearest airport can be a pain

Yes, Not only is it a visualisation of a SHA256 hash of the public key, but it's hard to turn the visualisation back into a SHA256 hash itself.

There exist other free ACME providers, let's encrypt was just the first and biggest

For what it's worth, I've these endpoints in use for VPCs that still had internet access. Meaning that if you attempted to read the "real" internet address you put your VPC subnet on, they would be unreachable.

It's hard/impossible to figure out if the VPC in question has been setup this way. But I agree that it would be likely that most of these VPC with the endpoints on don't have internet access.

However if we assume (dangerously I suppose) that the VPC subnet distribution is similar to other VPCs without private link, we can imagine how many other VPCs are squatting on space that do have internet access!

(Assuming any of this makes sense)

My personal experience with UCEPROTECT was that they had blacklisted 2 or 3 IPs in my /24 that were not routed to anything, nor had they ever been routed to anything, a fresh new block from RIPE NCC too.

Of course they offered to unblacklist them in exchange for payment, or wait.

Waited 2 weeks and they dropped off. I've yet to hear about anyone using their DNSBL for anything serious in 2020/2021/2022

I only knew about the listings because a monitoring service emailed me about it.