I understand the need for independent fiber ISPs. But are gigabit speeds really necessary? For me, a 300 Mbps connection is way more than enough for a four-person family.
I'm certain that one could make a sound argument that 300 Mbps is not necessary for that four-person family, and they could make do with a much slower connection. Back in the day, folks would be asking if it's necessary for your Internet connection to be always on. After all, it's no hassle at all to plug the modem into the house phone line and unplug it when you're done!
For me, switching from a 1400/40mbps cable connection to a symmetric-but-variable 300-1000mbps Ethernet connection meant that I was doing the same sorts of things, but often spending much less time waiting for them to complete. Related to that, it also made "content creation"-esque things [0] much, much easier.
[0] Which I'm declaring is a category that includes uploading and downloading huge files while working from home as a programmer/"DevOps" guy.
It's not a committed rate. Your individual line is a gigabit, but the upstream from your whole block is 10 gigabit so you can't all use it at once. Your guaranteed rate is probably have more like 20-50 Mbps, if that's what's confusing you. But it's extremely rare that everyone tries to use their gigabit all at once.
If it's a Passive Optical Network, you might be sharing a gigabit download with your block - you all share the same fiber - and you get substantially less than a gigabit upload due to the need for timeslotting. Gigabit PON is obsolete though, now you'd get at least 10G PON.
But I'm a realist so I'll take what I can get.
As your remote resources get faster and faster you start using them more like local resources, which can change (often for the better) how you do things.
If all you're doing is watching netflix then 25mbps per user is probably fine.
If you're working or creating or earning, then you want your connection to be as fast as possible, your distant end to be as fast as possible, and the hardware in between to be fast as possible.
The scarcity of upload speed is why I learned how to setup OpenWrt's SQM feature, and upgraded to the highest package my ISPs offers.
I myself have fiber, and speeds are much less than 500 Mbps, but symmetric (with likely maximum two people using video calling at a time), so haven't faced that issue.
> I also occasionally rsync large directories to/from cloud storage and that can also saturate
For convenience, the rclone tool is nice for most cloud storage like google and stuff that make rsync annoying[0]
rsync also offers compression[1], and you might want to balance it depending if you want to be CPU bound or IO bound. You can pick the compression and level, with more options than just the `-z` flag. You can also increase speed by not doing the checksum, or by running without checksum and then running again later with. Or some intervaling like daily backups without and monthly you do checksums.
If you tar your files up first I have a function that is essentially `tar cf - "${@:2}" | xz -9 --threads $NTHREADS --verbose > "${1}"` which uses the maximum `xz` compression level. I like to heavily compress things upstream because it also makes downloads faster and decompression is much easier than compression. I usually prefer being compute bound.
Also, a systemd job is always nice and offers more flexibility than cron. It's what's helped me most with the wack-a-mole game. I like to do on calendar events (e.g. Daily, Weekly) and add a random delay. It's also nice that if the event was missed because the machine was off it'll run the job once the machine is back on (I usually make it wait at least 15 minutes after machine comes online).
Great tips! I'll definitely be using your tar command
Let's take video streaming. I have a pretty compressed version of Arrival that's at 2GB and is a 4k movie ~2hrs long (the original file was ~2x the size). To stream that we need to do 2000Mb / (3600s * 2) = 277.8Mb/s. This also doesn't account for any buffering. This is one of my smaller 4k videos and more typical is going to be 3Gb-5Gb (e.g. Oppenheimer vs Children of Men). Arrival is pretty dark and a slow movie so great for compression.
Now, there's probably some trickery going on that can get better savings and you'll see used with things like degrading the quality. You could probably drop this down to 1.5Gb and have no major visual hits or you can do a variable streaming and drop this even more. On many screens you might not notice a huge difference between 1440 and 4k, and depending on the video, maybe even 1080p and 4k[0].
For comparison, I loaded up a 4k YouTube video (which uses vp9 encoding) and monitored the bandwidth. It is very spiky, but frequently jumped between 150kbps and 200Mbps. You could probably do 2 people on this. I think it'd get bogged down with 4 people. And remember, this is all highly variable. Games, downloads, and many other things can greatly impact all this. It also highly depends on the stability of your network connection. You're paying for *UP TO* 300Mbps, not a fixed rate of 300Mbps. Most people want a bit of headroom.
[0] Any person will 100% be able to differentiate 1080p and 4k when head to head, but in the wild? We're just too used to spotty connections and variable resolutions. It also depends on the screen you're viewing from, most importantly the screen size (e.g. phone).
First, if it was 2GB * 2 for the source of your recompressed copy, that's 4GB * 8 bits per byte = 32 Gigabits (Gb), or 32,000Mb. Two hours in seconds is 60 * 60 * 2 = 7,200 seconds.
32,000 / 7,200 is 4.444Mb/s. Streaming your 2 hour long 4GB movie could be done with ~5Mbit. A 1Gb/s connection could handle streaming ~200 of these movies.
Going back to Blu-rays as a source, an Ultra HD Blu-ray maxes out at 144Mbit but in reality most movies are encoded at a much lower bitrate. Most movies will cap out around 40-50Mbit. You could do 20 of these straight Blu-ray movies on a 1Gb connection.
https://en.wikipedia.org/wiki/Ultra_HD_Blu-ray#Specification...
Your math is WAY off.
2 gigabytes / 2 hours is only about 2.22 megabits/sec.
2000Mb / 7200s = 0.278 Mb/s, or 277.8Kb/s
> It is very spiky, but frequently jumped between 150kbps and 200Mbps. […] I think it’d get bogged down with 4 people
That’s just burst buffering as fast as it can, you didn’t capture the average. It doesn’t suggest it would slow down with 4 people. 4K on YouTube takes 20Mbps, so to parent’s point, you’ll have plenty of bandwidth to spare if 4 people do this at the same time on a 300Mbps line.