Add 10 GbE to your system with an M.2 2280 module
cnx-software.com
cnx-software.com
As a counter-point, I'm regularly limited by 1 Gbps PHY limits. During the 2 years of the pandemic I've been working from home with gigabit Internet. Why gigabit? Because that's the "speed limit" of the Ethernet PHY in the fibre broadband box, and also my laptop's ethernet port.
My laptop can write to disk at multiple GB per second, or tens of gigabits.
I transfer large volumes of data to and from a "jump box" in Azure that can easily do 20 Gbps out to the Internet.
I regularly update multiple Docker images from scratch that are gigabytes in size, each. I then upload similarly large Docker images back to Azure.
Even when physically present at work, I'll regularly pull down 50-200 GB to my laptop for some reason or another. One time I replicated an entire System Center Configuration Manager deployment (packages and all!) to Hyper-V on my laptop because the NVMe drive let me develop build task sequences 20x faster than some crappy "enterprise" VM on mechanical drives.
I have colleagues that regularly do big-data or ML queries on their mobile workstations, sometimes reading in a decent chunk of a terabyte from a petabyte sized data set.
All of these are still limited by Ethernet PHY bandwidths.
Note that servers are 100 Gbps as standard now, and 200 Gbps is around the corner.
The new Azure v5 servers all have 100 Gbps NICs, for example.
Do you have home servers you forgot to mention that you’re uploading docker images to, and those could benefit from 10gb Ethernet?
(For me, I use gigabit Ethernet everywhere at home but sometimes need to transfer large disk images from my desktop to my laptop, and using a thunderbolt cable as a network cable helps here, I can get closer to 2 gigabits of transfer speed before disk writes seem to be the bottleneck.)
I work inside a workstation but occasionally on a laptop as well.
Having 10GbE will help me when I'm on my workstation. It's not fatal, definitely, but it adds up to lost productivity.
Additionally yep, I plan to have one NVMe dataset sometime this year.
And even further, I plan to add a few enterprise HDDs in stripe and mirror. If you have 8 (with capacity of 4 for full mirroring) and each is at 260MB/s then et voila, you get 10GbE again.
But the 10Gbit network is there for the flash pool.
Consumer SATA SSDs are plentiful enough that I'm just going all in on flash when I revamp my server and do an 8 drive (+1 hotswap) RAID10 build or whatever. I don't need +60TB or anything and 8-9 drives is about the limit you can find on high end consumer mobos I think...
It is not required for the NAS to be in the same room where you sleep.
With a proper drives (and maybe some APM tinkering) you can make a very quiet one. Also using 2.5" drives is an option too.
Though if you can throw money at the problem than of course you can just buy a bunch of TB SSD drives.
So I read a 10GB chunk of a file that I know is not cached:
$ dd if=/some/large/file of=/dev/null bs=1M count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB, 10 GiB) copied, 11.3378 s, 947 MB/s
1 Gb/s network is probably enough for most people, I agree. But I certainly think there are many use cases for faster networks, especially in digital-heavy households.
> Conclusion:
>The majority of 4K movies (75%) I tested have bitrates over 100 Mbps and many seconds where bitrates spiked over 100 Mbps. Some have 100s of seconds where bitrate spikes over 100 Mbps, and will most certainly cause problems if played with bandwidths less than 100 Mbps on devices that don't buffer well such as the LG TV or Roku TV. To make sure you get the best experience without any buffering or transcoding on such devices, you need to make sure you have a bandwidth that exceeds at least 150 Mbps to play most 4K movies properly. Ideally, it should be higher than 200 Mbps.
"If" is doing some heavy lifting there.
The linked post shows that the average bitrate of every sampled 4k movie was less than 75 Mbps. The author even bolded "on devices that don't buffer well such as the LG TV or Roku TV"
Plus I have a service which downloads stuff for the archive team so that’s always doing some network traffic.
There is also a CI gitlab worker and that is also always doing some build with docker images from scratch.
I just wish more than 1Gbps was something that was offered and I can upgrade but so far I’m limited by my ISP with no way to upgrade. Inside my network I have 10Gbps and I have never hit that limit. It was expensive and I needed it for a now deprecated servicing.
Docker images are layered for a reason. If you’re having to download multi gigabyte docker layers multiple times, something has gone very wrong.
Azure charges (like the other major cloud providers) by egress data. I feel bad for whoever is paying the bill for 20gbps going out.
In general, people in ML shouldn’t be pulling raw data in the hundreds of gigs down to mobile workstations. That’s a security and scaling disaster.
Trying not to be too much of a curmudgeon here, but this is the problem with 1 gbps connections. They are so good they prevent you from doing the right architecture until it’s far too late.
A good remote work setup should leave all of the heavy lifting close to the source and anything that does need to be downloaded should be incremental/shallow copies. You should be able to work from a stable coffee shop WiFi connection.
When I worked at Vercel during the days we supported Docker hosting (back when it was still ZEIT), we saw that even among many tenants on the same machine, the "layers" thing wasn't really that beneficial - we still had to fetch a ton of stuff over and over again.
So this is great in theory, but it doesn't really pan out in practice.
Reasonable, IMHO.
ZEIT had a pretty aggressive cache anyway, for the short time it was implemented (before ZEIT moved away from docker hosting, at least).
I think there are different philosophies, just like 'no comments allowed in code', this is a matter of judgement.
I saw 5 different corporations setup centralised workflow, and usually you end up with a massive cloud bill, incorrectly configured admin rights, under-resourced VDI, and a ticketing system that takes a week to resolve the smallest issue.
If we assume for the moment that data in question is not highly sensitive (for example public satellite imagery), there is nothing wrong with getting every dev a beefy workstation with 50 TB of storage and a 20-core CPU, and it's often even cheaper.
If you are working with ML and have public datasets, just use the original source. Make a public mirror if you want to collaborate. Make it accessible via BitTorrent. You can bet that the amount of savings you will get from AWS will be enough to give each dev a NAS with a few tens of TB, which then they can even then use to host/mirror the public data by themselves.
The analysts were pulling down a ~0.1% sample to develop the ML algorithms locally, and then deploying the final thing in the cloud to run through the whole petabyte.
Local development has its advantages, such as high flexibility and very fast cycle times. It's fantastically expensive to outperform a modern PCI 4.0 NVMe drive in the cloud. Locally it's a $200 part.
Well the images need to be in some order, and if you are fine tuning the very initial parts of the image, this can happen.
But it seems a better transfer algorithm could be used for those individual layers. It rather seems like if they don't match, everything's transferred, while there could be a lot better options as well. If we consider that they are structured like file systems then it opens even more options.
---
So I took a peek on my local Docker installation. It seems the layer files on the drive are gzipped json.
That's right, .json.gz, _that includes the files BASE64-encoded_.
Yes, I believe there is room for improvement, if this is also the format they are transferred in.
[1] head -c 100000 /dev/urandom | base64 | gzip | wc -c
But as commented to me, that was incorrect, the payload in the files I saw was actually binary-encoded metadata. The actual archives are stored only in extracted form in the client and .tar.gz in the registry. Not that putting binary metadata in JSON is really optimal to me either, but maybe there are terrific reasons for doing it. Just check /var/lib/docker/image/overlay2/layerdb/sha256/*/tar-split.json.tar.gz; the fragments seem to be straight from tar, so maybe tar is used to interpret them?
Certainly .tar.gz files are not the most optimal for the purpose of incremental updates either. It's difficult to say how much a more sophisticated transfer protocol/storage format would benefit registry servers, though some clients could benefit a lot from it.
If a space-efficient merkle-tree-hashed archive-format suitable for efficient incremental updates doesn't exist, then someone should make one ;-).
Nevertheles, an archive format other than .tar.gz could be transferred more efficiently given a previous version of the same image. Or, even .tar.gz could be transferred more efficiently if both ends extract the file first, and if there is a need to preseve sha256, then that needs a bit more work.
There are some times when your thinking/workflow is constrained by your environment, sometimes significantly.
In what world?
Network cards: intel, mellanox, solarflare. Intel has a lot of drivers for all OSes, mellanox is most stable electrically (you can provide it with separate power, no need for fancy adapters, just plug any 12v in), solarflare is cheapest on ebay if you buy 2-3-generations old cards. Actually, any 10G network card will do, just check if they have drivers for your OS.
Thunderbolt: there are ready-made eGPU cases, they are costly but easier to handle. If you want to be cheap: wait for sale of any thunderbolt3-to-nvme adapter. PCBs for all of them are made by the same factory (you can find it on alibaba/taobao) they differ only by box and branding. Based on intel JHL6340
edit: I have got this, it was on-sale at the time https://www.aliexpress.com/item/4000200151507.html
edit2: just checked eBay, the cheapest solarflare "S7120" (it is SFN7022F actually) is $15 (!) + free shipping, wow. Though you'll need some sfps, they start from $6 (FTLX8571)
The only use internally would be central storage server. But all the new NVMe SSD's are faster than 10 gigs, so my internal network would be the bottleneck.
The next logical step would be upgrading to 100 gig network, but that is too expensive right now. I should buy more fast NVMe storage for that money.
If I could get 10 gigabit connection to internet, it would probably make more sense, but I don't see it coming that soon. And I'm paying 15€ for 1 gigabit right now, I would not be willing to pay much more for 10 gigabit connection.
On top of that, my Zabbix monitoring data shows that I rarely saturate the 1 gigabit connection to internet. Most of the time I don't need more than 10 megabits.
Some advice: make sure you know the form factor of your NICs. I accidentally bought FlexibleLOM cards. They look suspiciously like PCIe x8, but won't quite fit. FlexibleLOM to PCIe x8 adapters are cheap though.
I've never seen docker images of that size that can't be shrunk with multistage, or docker shrink. I can see how someone could get to that size with just baking in all their models to the image itself but yuck.
They are no-where near that size ...
they only get larger from there. Largest on my machine right now is showing to take 20GB. Granted there are a lot of shared layers between images, so the amount of space on-disk is much less than 20GB. But if you don't have any of the layers already? You're looking at 20GB easy.
I just downloaded: https://hub.docker.com/_/microsoft-dotnet-framework
It was about 2.7 / 2.8 GB in size.
Total size on disk was 8.7GB but we are talking about download here.
I would say you should be moving to .Net 6 it's been several years now that .Net Core has been out so there is no excuse.
None of this is an excuse to get a 10Gbps internet connection to your house. I have a 500Mbps connection and with the corporate firewall my laptop gets 100-200Mbps connection and pulling those containers was like 10 minutes.
Edit: In case anyone thinks I'm lying:
https://devblogs.microsoft.com/dotnet/we-made-windows-server...
I ditched the 10G because the 10G switch was hot and noisy and had a fan, whereas my 1G equipment was silent and low power. This could be overcome by putting the switch in another room, obviously, but I haven’t missed it enough to go through the trouble yet.
> During the 2 years of the pandemic I've been working from home with gigabit Internet.
Doesn’t this make every other point in your post moot? I’m similarly limited by Gigabit internet, so the only time I benefit from 10G is transfers to/from my local NAS. I realized that I almost never lose time sitting and waiting for NAS transfers to finish, so even that was only a rare benefit.
If I was buying new equipment and building out something new, I’d pick 2.5G right now. It’s significantly faster than 1G but doesn’t come with the heat and noise that a lot (though not all) of 10G switches come with.
I’m sure I’ll go back to 10G some day, but without >1G internet and no local transfer use case, I found the extra hassle of the equipment (laptop adapters, expensive/hot/noisy switches) wasn’t really worth it for me yet.
10G was cool but I can’t recall a single instance where I really wish I had it back.
Same. It only really made a small difference to my most transfer-heavy workflows. Even a fast NAS is still slow relative to local storage (it’s not just about sequential transfer speeds).
I’m sure I’d feel differently if I was doing something like video editing where I had to move large files back and forth frequently and wait for them to be ready.
same, I live in a rural village in france where I get 2GB fiber. Yet all my computers aren't able to leverage that.. quite frustrating to see speeds being limited at 110/120 megabytes/s where my drives are easily able to handle gigabytes.
A 4kx4k map layer section, in RGBA is 64MB. Each layer on the map has multiple types of mask applied (brushed alpha, brushed noise-function,...) so there's another 64MB of XXXX data per layer. A map can have (say) 10 layers on it, so we're up to 1GB, and then there's the data for ephemeral layers ("stamped" images at co-ords and scale, alpha,rotation). The stamps themselves are usually stored in an 8kx8k layer so that's another 256MB
When your 'save' action is 1.5GB of data, a 10Gbe network is much preferred over a 1 Gbe network...
All of this is rendered down for production of course, but during development, each layer is kept separate along with its masks.
Wait, why are the masks RGBA instead of grayscale?
1st one is typically a brush-in one - basically just Alpha brushed in from various alpha-brushes over the map
2nd can be a noise function, where the 1st alpha mask is multiplied by the second to get a brushed noise, and the noise might be useful for randomly-shaped patches of different-colored grass in a scene
3rd might be a shadow mask, where any objects "stamped" on top apply a default shadow at a given angle, and that default shadow is written into this channel. Then the map-maker can alter those if (for example) a shadow hits a wall.
Then there's the shader mask - so you can select a given shader for a given area of the map. These'll all be batched up later, but if you want flowing water, brush in where you want the water shader to be applied. Same for smoke, lava, etc. etc.
I have tested Azure and AWS speed out to the internet and they too won't get 20/100 Gbps. That high bandwidth is for connecting instances within the AZ. Once any of the traffic has to go over the internet it's much lower.
> I'll regularly pull down 50-200 GB to my laptop for some reason or another
This is a serious failure in your workflow. You should be looking at reducing that size considerable in terms of a docker image.
Exactly. I understand why some may want or need 10Gbps. But OP comment should not even be top voted.
Yes, virtual desktops suck in several aspects, but we couldn't expect everyone to have gigabit Internet at home.
1GbE networking is one of the slowest interfaces on modern computers.
- The slowest USB3 spec is 5Gbps. USB3 is a mess, but you can easily far exceed 1Gbps with the absolute worst of it.
- The current fastest USB spec is USB4, and is at least 20Gbps, and sometimes 40Gbps
- Thunderbolt/Lightning both push 10's of gigabits depending on spec
- 802.11ac ("Wifi 5"/"Wave 2") is capable of Gigabit speed, and 802.11ax ("Wifi 6") easily exceeds 1Gbps
- Even Cellular connections can exceed 1Gbps these days with 5G, though admittedly realistically you're looking at 100Mbs-1000Mbps
- Individual SAS/SATA rotational drive easily push passed 1Gbps in seq workloads
- Individual SAS/SATA SSD's often push 3Gbps, and SATA itself supports up to 6Gbps, with SAS up to 12Gbps.
- PCIe NVMe in it's slowest spec (Gen3 at 2 lanes) is still 16Gbps, most typically running at up to 32Gbps at Gen3 x4
- PCIe5 consumer devices have launched (Gen5x4 lane NVMe is 128Gbps)
- PCIe6 is around the corner for enterprise
To add insult to injury, in the consumer space they're trying to tout 2.5GbE as some sort of premium new NIC, despite it replacing 10GbE in the premium NIC space from several years ago (eg Asus X99-E-10G-WS). It's maddening. 10Gbase-T, 10Gbe, and Cat6a has been around for well over a decade (to say nothing of SFP+ 10G). Granted they're starting to reposition 2.5GbE as the new 1GbE, which is far more sensible, but still positions your NIC as one of your slowest interfaces
1GbE is the limiting factor to speeds these days, with internal storage/devices and nearly all alternative interfaces vastly outspeeding 1GbE. Even cheap external SATA SSD's and a common usb3 cable can greatly exceed your 1GbE network speed. If you have multiple devices, and want to transfer data between them, a network cable is supposed to be the superior choice. Not Wifi, Not USB, but a network cable over the local network. But it's actually much faster to plug a drive into one device with USB, transfer the data, unplug, plug into the other device, transfer again, than to use a standard 1GbE network interface. Shameful. Just because your WAN interface is likely 1Gbps or less doesn't mean there isn't a use at the prosumer level for >1Gbps network connections for use at the LAN level.
Wifi 6 officially has a real world limit of 700mbps [2] and drops to half that if you're more than a few feet from the router. Passive USB 3 and thunderbolt cables have a max length of 2 meters [3]. The rest of the interfaces you've listed like PCIe all operate on the scale of inches and plug straight into the motherboard.
Plus it really isn't very expensive to upgrade to 10G given the use case is pretty niche for home use. You can get a budget SFP+ Switch and 2 NICs for ~$300.
[1] https://en.wikipedia.org/wiki/10_Gigabit_Ethernet#10GBASE-SR
[2] https://www.theregister.com/2019/12/05/wifi6_700mbps_speeds/
[3] https://www.newnex.com/usb-cable-maximum-length-limits.php
> You can get a budget SFP+ Switch and 2 NICs for ~$300.
Yes, this is part of the problem. In 2016's we started to see mainstream adoption of 10G, which hopefully would lead to the more availabile, more affordable, consumer friendly ease of use twisted pair Cat6a and RJ45 10Gbase-T Switches. Instead, Enterprise had already largely moved on from 10G (largely supporting it as a legacy connector via SFP28 being backwards compatible with SFP+) and Mainstream/Prosumer manufacturers instead decided to regress from the 10Gbase-T deployments to 2.5GbE.
As a result we're not seeing a lot of development in the 10G space, and in many ways a regression towards 2.5GbE. Sure, if you don't mind the noise and know your way around a cli, there are plenty of surplus enterprise switches on the cheap. And I don't mind Optics/DACs. I run 10G at home (cat6a, Intel X550's, Arista 7050T-64, which used to be a few hundred bucks) and 25G/100G in the lab (CX4 LX's/CX6 VPIs connected to an Arista 7260CX-64). But it is unfamiliar to most consumers. Not a huge hurdle, but a strange ask considering 10Gbase-T with familiar twisted pair and rj45 aint new by any means. Mikrotik and UBQT both have some options in the space, but the cost of these newer switches certainly doesn't fall in the $300 budget, unless you only need like 2-4 10G ports on the switch and are fine with SFP+. And yes, >4PHY 10G SFP+ and/or 10Gbase-t switches exist, but you're not getting them and NICs w/ DACs (or especially fiber if you need a longer run akin to a standard Cat5 deployment) inside your $300 budget, and there's very few on the market that'll fit in your budget even if you stretch it to $500.
TL;DR: Yes, 10G is technically available, but it's a strangely esoteric niche that exists after the mainstream decided to regress from 10GbE towards 2.5GbE.
The Intel 10GbE controllers are more expensive, but much better in my experience.
For anyone looking for an alternative, I use an M.2 to PCI-e slot riser and a regular HHHL PCI-e card. Example product: https://www.amazon.com/ADT-Link-Extension-Support-Channel-Mi...
But that's acceptable only because that machine has a workload which can tolerate not having network access for a few minutes per day or so.
↑ this one does support gen3. Anyhow, with 4x@gen2 you’ll be fine at 10G speeds (I checked). Gen3 is strictly needed only if you want 2 (or 4) ports, or 25G...
I guess there really isn't much demand for faster than gigabit speeds even now (outside of servers?)
Also, the main competition is WiFi which is significantly slower than 1GbE in most cases.
Also SSD are not 100 EUR / TB. A 4 TB SSD is like 300 EUR.
If you are talking about only two drives than no. Multiple drives can give you more throughput, but at that point it is RAID-10.
People who notice mine in the corner mostly don't even know what it is.
The simple spinning hard-drive was already faster than gigabit ethernet, and the NAS access speed was 90-95% of ethernet. So I am prerty sure that NAS is the bottleneck here. And this is on a WD-red 5400RPM harddrive, nothing special.
But I guess most people don't transfer files in their local networks anymore and use their network purely for internet access.
I think this is clearly the case. Most new laptops don't even bother with a wired network port. I've got a new "pro" HP laptop the other day, and it only comes with some cheap Wi-Fi card. And it's not an "entry-level" laptop, and it's thick enough for an RJ-45 plug to physically fit.
I also see more and more desktop motherboards come with integrated Wi-Fi. The desktops at Work (HP) also have had integrated Wi-Fi for a while, and it's not something we look for (they all use wired Ethernet).
Some people just don't care.
I'm busy retrofitting Ethernet in my house by pulling cat6 through the walls and pulling out the old cat3 phone cabling. It's much harder work doing two cables (not least because none of the phone cables were in conduit, so it just starts off harder already) for each pair of ports, but it's very much worth the effort.
They didn't care, they just laid some sort of cable because that's all they had to.
Cat6a is probably the sweet spot for home/office/etc structural cabling. It's not much more expensive than Cat6 (or Cat5e in case people are still putting that up), and has more than enough legroom.
If each run is <55m then Cat6 can still do 10Gbps.
there are a variety of ways to screw that up. i've seen a house that has a bunch of cat 5e run, but the installer stapled it down, and most of the cables have a staple through them somewhere along the run, killing a pair or two.
Most people don’t have anywhere in their local network to transfer things to. I still laugh when people see my home server and assume that’s “your work thing”. I do use it for work, but 99.99% of what’s contained in there are family pictures and photos.
What you need for the transition is for premium brands to start pushing 10G ports as a feature, e.g. Apple needs to add it to Macbooks and the Mini and start using it to bludgeon competitors who don't have it. Then once their customers have several 10G devices around, they buy a 10G switch and start demanding that every new device have it. At which point the volume gets high enough for the price to come down.
https://www.apple.com/mac-mini/specs/
> Gigabit Ethernet port (configurable to 10Gb Ethernet)
I really want to start seeing 2.5gb/s becoming standard on Desktop motherboards asap.
You don’t need to go fully 10G, though, I mainly prioritized my workstation for example, and if you don’t want to splash out for switches, you can do a direct connection on 10G to a computer that needs it, and use the 1G links to the rest of the LAN.
1. Consumer broadband speeds rarely exceed 1Gbps
2. 1Gbps Local network transfers are seldom slowed by the network (as large file work typically involves HDD still)
3. Where local network transfers are impeded by the network speed the transferring itself isn't a frequent enough and blocking thing that people feel they need to fix it... they just go make a cup of tea once or twice a day
4. There are a lot of old network devices and cables out there, some built into the fabric of buildings (wall network sockets and cable runs)
5. WiFi is very very convenient, so much so that 250Mbps is good enough for almost anything and most people would rather be wireless at that speed than wired at a much higher speed (gamers and video professionals being an exception)
And ultimately, the cost and effort of investing in it doesn't produce an overwhelming benefit to people.
Even in quite large offices, it's hard to argue that this is worth it when the vast majority of people are just using web applications and very lossy audio and video meetings across the internet.
> (gamers and video professionals being an exception)
I play Stadia, 4K HDR 60FPS, just fine on a gigabit ethernet connection and 150MBit internet connection. Any games not streaming the entire video are fine with just kilobits/s of data, as long as the latency is good.
So the case for home 10GE is even weaker ;)
[1] https://www.machinadynamica.com/machina31.htm
[2] https://www.synergisticresearch.com/cables/srx-cables/srx-sc...
[3] https://www.synergisticresearch.com/cables/srx-cables/srx-ac...
[4] https://en.wikipedia.org/wiki/The_Emperor%27s_New_Clothes
Why not compare the insane prices of thermal paste and compounds per volume/weight with the stuff that is used industrially?
Not always, but often used for results which could be called statistic noise. Very diminishing returns.
All for pushing reviews with many bar graphs over two dozen pages, and so much more page impressions. Like the rest of gamermedia, too.
Woo!
So we have the very few audiophiles which totally ignore the thin copper layer on cheap PCBs, versus the whole market segment of twitchy gamers going for Bling, Bling, Bling without function on their RAM, coolers, vs. the equivalent stuff installed in laptops and servers which lacks that :-)
Which is what rules out wifi :)
10GBASE-T exists, but it turns out pushing 10gbit/s over 100m of twisted pair requires chips that are hot and power hungry. Again, not great for desktops or laptops. And because it’s not used in data centres, there are no economies of scale or trickle down effects.
Gigabit Ethernet and wifi being “good enough” combined with 10gig over twisted pair being expensive and power hungry means that the consumer space has been stuck for a long time.
Maybe this used to be the case long ago, but I don’t think it’s true today. Personally I’m pretty rough with fiber (having slammed cabinet doors on fiber, looped fiber around posts with a low bend radius, left the ends exposed to dust, etc) and had no issues within a data center. Can’t say the same for copper. Even the cheapest 10km optics have more margin when your link is only 50m.
Oh and bend insensitive fiber is dirt cheap and works just fine when it’s tied into knots.
I suppose forwards compatibility is good, but unfortunatel unlike 2.5Gbps, 5Gbps is practically unusable on existing CAT5e cables.
It's easy enough to get G.657.B3/G.657.A3 cables, and you can wrap them repeatedly around a pencil and they are fine.
Also, most consumers would not notice the bend attenuation anymore becuase they aren't trying to get 10km out of them :)
i buy armored multimode patch cables for, i dunno, 10 or 20% more per unit length, and they seem indestructible in my residential short-distance use cases.
> Gigabit Ethernet and wifi being “good enough”
i think this explains it all. when the average user prefers wireless to wired gigabit, we know how much bandwidth they actually need, and it isn't >=gigabit.
i don't know what you're using, but the LC/LC connectors i use seem pretty durable, and the springy bit that wears out can be replaced. the SFP+ modules are all metal; i am unwilling to believe they can wear out.
> and cables left disconnected with out the risk of an accidental laser eye surgery
single mode, which isn't human-visible, is of concern but the multimode (which i mentioned using) isn't any worse than a cheap laser pointer. it isn't strong, and you can see it.
Maybe a laser in your eye is an acceptable risk in your home but I doubt it is an acceptable hazard in a work place.
That is finally improving. The technology improvement meant we get less than 5W per port on 10Gbps. Cost will need to come down though.
I think what's new is the prevalence now of 2.5 and 5GBaseT ethernet chips that are cheap enough companies are starting to build them into any $125+ ATX motherboard. At short lengths even old crappy cat5e has a good chance of working at 2.5 or 5.0 speeds.
100Gb links do take some thought and work to saturate, but that's improving at a good rate lately so I expect it'll become more common rather soon.
The main downside to 25Gb and 100Gb links still seems to be hardware pricing. At these speeds, PCIe network adapters and switches get rather expensive rather quick and will make you really evaluate if your situation really demands those speeds. 10Gb SFP+ and copper network adapters and switches are quite inexpensive now in 2022.
But that's tweaking the setup already, it requires changes, testing and verification, and can cause problems in downstream equipment. And for a lot of applications, a 9K MTU will not be enough to saturate the link because they'll need NUMA awareness, or the NIC queues will need tweaking to avoid imbalances, or the application is not ready to send at that speed...
I'm not saying it can't be done, of course it can. But it isn't "plug a bigger card and it'll go faster".
That doesn't seem right. When I got my first 10G server, it was running dual Xeon E5-2690 (either v1 or v2), and I don't recall needing to tweak much of anything. That was mostly a single large file downloaded over http, so not super hard to tweak anyway, but server chips are a lot better now than sandy/ivy bridge. It could only get 9gbps out with https, but the 2690v4 could do 2x10G with https because aes acceleration.
Well, my point is that most servers don't just download single large files over HTTP. Even if you only look at storage servers, going into multiple files and connections you can easily find issues and have downgraded performance if you don't prepare the system for the workload.
https://www.cablinginstall.com/sponsored/berk-tek/article/16...
... points out that 5GBASE-T needs 200 MHz channel bandwidth which is past what Cat 5e is specified for. So perhaps for runs approaching 100 meters or in noisy environments, Cat 5e won't be reliable for 5GBASE-T after all.
Unfortunately, I'm finding switches with 2.5 to still be overpriced.
I'd rather have a 100Mbps home DIA circuit that was symmetric and seriously reliable, than the typical Comcast circuit.
I would also very much rather have a symmetric connection than what I have now. We are within range of a fiber tap for Comcast's 3Gbps symmetric ftth service but construction costs are too high right now for them to do it. I keep asking every six months regardless. There's no other game in town as far as I can tell.
Comcast Business will run a 100Mbps symmetric fiber for ~$750/mo, maybe that would be a thing to do to get fiber in the door and then switch back to residential after the contract earns out.
Instead we run 10G signaling at half or quarter clock rates and get something that works on the majority of existing wiring.
AFAIK the IEEE was initially resisting supporting this, but enough vendors were just doing it anyways that it was better to standardize.
- An 8x cdrom narrowly beats 10meg ethernet.
- 1x dvdrom narrowly beats 100meg ethernet.
- ATA133 narrowly beats 1gbit ethernet.
that doesn't fit with my recollection of reading/transferring DVDs, or with https://en.wikipedia.org/wiki/Optical_storage_media_writing_...
> - ATA133 narrowly beats 1gbit ethernet.
the electrical interface/protocol, sure. i don't think any ATA133 drive made could actually saturate its interface, or a gigabit link.
The cabling starts to become an issue otherwise. It's also hot and power hungry over copper.
Having gotten into the home fiber side of it for various reasons, it's clear on the fiber side there is plenty of innovation/cost lowering/etc.
20 bucks over MMF, 27 bucks over SMF doesn't seem expensive.
If you want to do it over single fiber rather than duplex, it's 40 bucks: https://www.fs.com/c/bidi-sfp-plus-64
25Gbps is 39 bucks over MMF, 59 bucks over SMF. 40Gbps is also 39 bucks on MMF (more expensive on SMF)
I don't think any of these are very expensive.
The cards are also ~same price between SFP+ and 10GBaseT from places like startech (outside of that, the 10gbaset ones are actually often much more expensive)
Flash drive writing speeds are my biggest bottleneck, wish someone would tackle that at a reasonable cost.
That being said, I generally agree that it has been moving too slow in consumer electronics and 2.5G is a pretty long overdue step-up for a moore-adjacent technology. Another factor is the humble reality that infrastructure (physical cables installed in walls) is the (s)lowest common denominator in this advancement.
Does anyone know why is this useful in casino gaming machines?
In my home server for instance, which is built on a consumer μATX board, the wider PCIe slots are filled with a GPU and HBA, which leaves no way to add 10GigE without spending a lot of money on a new motherboard and CPU - or finding a way to use the M.2 slot.
Isn't it enough given PCIe 4.0?
There's also the fact that people usually use "older" components to build home servers, and if I'm not mistaken, PCIe 4.0 is only supported in fairly recent CPUs, and with not that many lanes; whereas my desktop from circa 2012 comes with something like 40 PCIe 3.0 lanes.
I never really understood why motherboards didn't spend a few extra dollars and make all ports 16x ports (switched), so that you can use full CPU bandwidth to any one device and not have to mess with different types of port for different devices.
Putting the HBA in the second x16 ("electrically x8") slot makes both work at x8 speeds, as those lanes are "shared".
I’m using 10G in my home LAN already for ~5 years. And just a month ago I contemplated about upgrading my notebooks to 10G.
I ordered a cheap thunderbolt→nvme adapter (for ssds) + m2→pci-e adapter on AliExpress. And they all work like a charm! Total cost was about $55(tb3 adapter)+5(m.2)+25(network card)+$8(SFP+) = 93USD. A lot cheaper than other options like QNAP or Sonnet Solo (which are in $200+ range)
Can someone recommend quality USB(-C) ethernet adapters brand? I'm building an embedded system in a professional context and need to connect some of our own custom in-house built embedded devices (usb-c only, no ethernet) to LAN. Right now, when some device goes offline, I don't know wether it's our product or the cheapish usb-ethernet-adapter which is at fault. Would like to have something 100% reliable.
Shall I buy Lenovo or Dell?
Since it's built in-house, why are you relying on retrofitted USB dongles if reliable Ethernet connectivity is important? Unfeasible to make a revision with a port?
Anyway, if I were you I would probably just go ahead and buy one each of the top handful of contenders and try them out myself - they're not expensive and it makes sure that it really works for you, and if not, where the problem lies. If you have the same issue on several adapters with different chipsets, well...
As far as I can tell they seem to be the “go-to” in the USB-Ethernet game and are well supported on Linux and anything else I’ve used them on.
As others have said, there are advantages to pushing people to use normal spec machines to remind them that most of the world don't have 10GbE or even 20Mb broadband but would still like apps to start quickly.
Would love to understand this a bit better.
(edit: corrected the units.)
Power consumption (Full bidirectional traffic, 100m cable):
10G speed: 6.41W
5G speed: 4.83 W
2.5G speed: 3.97W
1G speed: 2.94W
100M speed: 2.21W
The MikroTik adapters are a bit special in that they are more a 2 port transparent bridge where the inside facing portion of the module always runs at 10g and the outside facing portion auto-negotiates. This allows 10G only switch interfaces to support 10/100/1000/2500/5000/10000 clients. In a MikroTik switch it reports back the negotiated speed and the switch can shape traffic to the appropriate bandwidth instead of letting the adapter do it (which I assume is just policed but could be wrong, never tested). I have a 100G switch which is backwards compatible with 40G and allows breakouts so with a QSA can support SFP+ modules... putting this MikroTik module in I can plug a 10 megabit half duplex device into a 100 gigabit port!
The whole article: https://www.servethehome.com/sfp-to-10gbase-t-adapter-module...