Linux Networking: MAC VLANs and Virtual Ethernets | Hacker News Reader

Linux Networking: MAC VLANs and Virtual Ethernets

pocketnix.org

117 pointssusam18 comments

I think the article could do with a bit more information about _why_ you'd want to use macvlan vs a bridge.

macvlan lets you offload switching to the network--so more of your CPU power can go towards running VMs. Since all traffic, even that between two VMs, goes via the external switch, that external switch can also perform auditing, accounting and policy enforcement. If you're a big enterprise with lots of expensive switches then you probably want to use them for these sorts of things, rather than push the job down to the hypervisors.

In my experience with said expensive hardware, it greatly depends on your use-case / scale.

You have to pay attention to stay within the limits of the hardware the switches (TCAM, etc...). And correct me, if I am wrong, but most NICs are fairly constrained in their hardware support on the numbers of MACs they can filter for. And those two combined, you might end up with a lot of flooding, which needs to be handled by the CPU again.

You also have to rely on the implementation of your vendor. And at least with one major vendor, who shall remain nameless, it is a major effort just to get acknowledged that there is a bug.

The point about number of MAC filters is fairly moot as when you do bridging in hardware you are not going to utilize any kind of hardware MAC filtering capability anyway. Also the issue of flooding tends to be solved by the physical switches in the network.

So, we are in agreement in that point: if you are going that way, you are relying on the physical switches.

I'm not sure about the problem being solved, depending on the scale.

If you are with your use-case (number of rules, macs, five-tuples, ...) outside the capabilities of the hardware, then, in my experience, it breaks down quickly. (I.e. flooding, or becoming unresponsive to configuration commands and needed to be _power cycled_, ...).

And those are switches costing easily five figures, and from two different well known vendors.

If you have practical experience with a setup, which scales more gracefully, please share.

But I'm sceptical that it exists, considering Google, Facebook, AWS all went with custom hardware.

I'm wondering, if at a certain scale of virtual NICs/flows per port, you are not better off with a hybrid approach, where software is part of a decision hierarchy. E.g Openflow.

No practical experience though.

One use-case I’ve seen for MAC VLANs is specialized audio transmission (AES67 maybe?) where data is sent over multicast.

Never really fully understood how it worked, but when developers wanted those applications to be hosted in containers, there was a lot of talk about MAC VLANs :-)

Interesting. Could you or someone else say what benefits does MAC Vlan offer over IP multicasting?

So it's kinda like doing VFIO, but with less things handled by the NIC, but externally from the switch point of view it looks like the same (several MACs going out of a single port), right ?

> macvlan lets you offload switching to the network--so more of your CPU power can go towards running VMs. Since all traffic, even that between two VMs, goes via the external switch, that external switch can also perform auditing, accounting and policy enforcement. If you're a big enterprise with lots of expensive switches then you probably want to use them for these sorts of things, rather than push the job down to the hypervisors.

The current trend is to push that to DPU/IPU-style NICs which have their own cores. Previous generation thing was to reduce CPU load on hypervisor by offloading the packet processing so (smart)NIC.

Unrelated hint to the author of this page:

Please enable Https for your Webserver.

It’s a static, non-interactive web page. Give it a rest.

It is if you are actually talking to the real server. But with plain HTTP it could also be a JS advertising laden hell hole, depending on your ISP and other middle boxes.

It's not static if it's http: any of the network hops can make it dynamic at will.

At which point the end user will see a non-static non-https link that they are free to distrust (and which browsers warn or will soon warn about). That complaint is not specific to this website and can affect any HTTP link any customer ever encounters.

Can someone explain this sentence: "instead of taking a single interface on the OS side of a network interface and mapping it to multiple virtual networks on the Network side of the interface (one to many)." How/why are multiple virtual networks on the network side implemented if not by the OS? Using an external switch?

Exactly, you end up using the external switch to do any of that stuff. There's a sibling comment to yours that talks about why you might want to do that, beyond the local performance concerns.

Title should be changed to article's: Mac to MAC (this is about ethernet MAC addresses, not macOS computers)

In fairness, if you're talking about networking the word "mac" always refers to the level of the iso/OSI stack, not to apple computers.

In fairness, capitalization matters.

Reply on news.ycombinator.com