My Overkill Home Network
blog.networkprofile.org
blog.networkprofile.org
These days i'm more pragmatic. I have exactly 2 wired machines, and everything else runs Wifi. The network "rack" is simply a firewall and a 16 port POE switch, everything else network related has been retired to the closet.
My NAS is gone. With electricity frequently going above €1/kWh last winter, it seemed like a bad idea to have a NAS sitting there, consuming ~35 kWh/month. Instead i spent about €20/month on cloud storage, and simply upload everything to the cloud (encrypted with Cryptomator), and i have redundant cloud storage (different provider) included in the cost which is where my backup goes.
All that's left at home is a small ARM based machine that handles making daily backups of my cloud data locally, as well as run a small Plex library on a single USB3 external drive.
Besides saving about half my previous electricity cost (as well as all the hardware cost!), i have gained SO MUCH spare time. My firewall is completely closed (save a wireguard VPN), and i no longer have to maintain anything except the normal patch routines.
Personally i have never experienced any of the issues you mention, but my Cryptomator vaults are also mostly below 1TB in size.
My local backup is also done with Arq backup to a local USB drive, though i have a long time test running with Kopia backing up to Minio on a Raspberry Pi.
I moved house recently and the only things I did were running CAT6 from where the internet enters the house, up to a PoE switch in the loft that then has a few Wifi6 APs connected to that.
Everything is wifi (all cameras, smart switches etc, office laptop, a single RPi4 for "projects" and so on) and all seems fine. The only device that is directly connected via ethernet is a headless gaming desktop that lives in the loft that I occasionally use for streaming games to a laptop.
But kudos for this guy if that is what he wants to do. Personally I did not enjoy doing the cable runs (especially for external-grade CAT6).
My "server" is also wired, as is my Hue Bridge, Tado Bridge and other "IoT" devices that doesn't support wifi, but when it comes to user accessible machines there is exactly one, which is my "go to" machine in case the router/switch dies and i can't get on the WiFi.
I also run a HomeBridge to bridge protect to Apple HomeKit, to have a backup of the footage.
Really curious how you set this up, sounds like it could be the perfect solution for me.
That's it, the entire network. Just enough storage to enjoy some media, the QNAP runs HomeAssistant (literally just to bridge some stuff to HomeKit) and Tailscale, and that's all I really need.
Translates to 50W.
Watts is the energy consumed "right now" (which is the product of Volt x Amps), but needs a time dimension to give a measure of historical consumption, which is where the Watt Hour (Wh) comes into play.
While most people are aware that X consumes Y Watts, few consider how much that addes up to over the course of days or weeks.
Few people are aware that a simple ethernet switch will roughly consume 1W per plugged in port, so an 8 port switch might consume 8-16W of power just sitting there, so 4-9 kWh/month.
My old homelab used to consume around 300W when idle, meaning it was consuming 300 * 24 * 30.5 = 219 kWh/month, and with electricity prices reaching €1/kWh, that suddenly becomes A LOT of money to have a Plex server available 24/7.
But that's my price of freedom.
My Synology NAS was failing. Saw that as a great opportunity to set up a proxmox box on a reasonably powerful used HP SFF, run TrueNAS on it. Plenty of time to get proxmox, TrueNAS, Gitea, a docker host, … just right.
It’s sat there with 4/16TB used, no real data transfer apart from TimeMachine from my Mac and my git server, sucking power at European rates.
Wish I hadn’t bothered, and had just gone with another Synology.
I have a similar setup with SFF running backups & ProxMox. However, I have smart plugs for my SFF and have a script to turn it on/off using cron (or on demand) from an always-on Pi. Wake on LAN would have worked too.
I'm considering making my Raspberry Pi the gateway proxy/"Load Balancer" to the SFF; turning it on if needed. It's trivial to proxy HTTP (Pi works as a Caddy-based proxy with long timeouts, powers on SFF in background if off. Things got trickier for proxying NFS & iSCSI)
The box is on another routed subnet - I wonder if I could firewall rule a “start” trigger.
Extra note- I would NEVER recommend a business use their home resources for work unless you know what you're doing (and why you're doing it). Even my homelab has vulnerabilities and I'm a security professional- more moving parts = much larger attack surface. There's a reason why corporations pay big buck$ for managed security services. Most homelabs I've seen are mostly for fun and personal comfort.
"What good is knowledge that is never applied" is what drives me to stuff like this. I can't speak for others but I'm sure I'm not the minority here.
If it makes you feel better, megacorps are getting out of the "self-managed data center" industry and embracing the cloud, to exemplify your very point.
I certainly have hobbies that go above any need or reasonable collection, namely synths and books. Haha.
He eventually got rid of the cameras because, well... They were shitty and only told him the critters were near AFTER he popped 'em. I think he's got a for-purpose system (in his own words "the new cameras didn't fall off a truck") now but it was a fun project!
Some tech people enjoy "homelabbing" and happily throw money at their hobby.
It's the same thing with 3d printers, some like them because they want to install clipper and tune the best/fastest benchy. I do it because they love being able to CAD something and have it in my hands as soon as possible.
That said, nobody goes as far as this guy without really enjoying the tinkering
My big services are a Ceph cluster, a VPN, Borg (backups) and a K8s cluster (using kubespray).
In K8s I have the main stuff: Plex, Gitlab, Gitlab runner (for CI), vaultwarden (passwords), miniflux (RSS, might be moving to freshrss though), rust desk (remote-desktop), home assistant (smarthome).
My next project is to stream metrics into grafana. I have soil moisture sensors in my garden connected to stm32 boards, I just need to setup the receiving side and I can control my drip lines (they're using opensprinkler) with soil moisture information along with weather info.
This is a use case close to my interests. I want a network of environmental things monitoring and doing some detection stuff.
Wifi is rock solid, networking is fast, backups good and outages are almost nonexistent and are always due to me messing with things. I have Netflix, Apple TV+ (is that what it’s called?) and a few others. Particularly with Netflix, the quality is junk so I watch off Plex.
Other benefits are pretty neat too. POE just works. You can power cycle things remotely if you want (have never needed to) and the abundant local storage makes everything easy.
Enterprise stuff that is getting old is also dirt cheap. Converting sections of the network to 10gbe was very inexpensive.
It’s all rather addictive…
But mostly? It felt exactly like exploring computers in my childhood for the first time (I'm 43 now). I was bumping into things I had never bumped into it before, and it was really satisfying to figure all that stuff out.
I'm a game developer, too, and I realized awhile ago a big part of that is I like technical puzzles, and game development is very fertile soil. But there is a whole new world of enterprise networking/storage/virtualization/etc things that you wouldn't normally bump into in the course of software development.
And as a bonus, I have set up a lot of build systems for friends; being infrastructure-savvy is like the digital equivalent of owning a truck and helping people move all the time.
That is very good advice. I know about a datacenter outage/degradation in a very large company, which all of you know.
A couple of network switches got overloaded. The switches connected the SAN to the VMs, so it caused all kinds of weird problems in different applications.
One handy trick: I had at least one drop in each room, often one for each wall of the room. Behind the wallplate was a NEMA box with a conduit running straight down to the crawlspace (6' high in my house so hardly "crawl"). That made it easy to pull cable, not only initially but if I found I needed an extra run, which hardly ever happened. Instead of trying to run through a rat's nest of conduit the conduit was a straight shot and then the cables could easily be managed.
Now, I have three Internet connections bonded and balanced. It is not about the minutes of disconnection but more about the disconnection when I needed it most. I have ample non-internet time and personal/family downtime. However, when the time comes, I’m happy that the Internet is never a bottleneck in my work and play. I have had this setup since around the early days of the Pandemic and our home, per se, “never had an Internet Outage” since.
I'm not into servers, devops, networking or anything of that sorts, but I love tinkering with them. I would love to have an "overkill home network" one day.
Internet is super cheap in India and I can afford all three for a really decent price. https://www.instagram.com/p/CUWeopdPVOp/
Just wondering: 1. How often do you actually need the third? As in, both primary and secondary are down. 2. Is it that important that you always have an internet connection?
First it’s a useful skill set that helps troubleshoot problems in code and apps that run on networks by knowing how such things work.
Second, try to think of it as a private cloud instead of a network. Because it uses a lot of the same types of software cloud provider do. Proxmox is one example that is a self hosted vps provider that is tremendous. So, it’s a private hybrid cloud that can push from your cloud to the other voids (or back). Build using a private hybrid cloud and you can push to,or between manu clouds
Wifi is for convenience, wired is for reliability. When wifi gets jammed, spotty, interfered with which can happen more often than imagined, especially during break ins, wired wins. Transferring files? Wired wins. 4K streaming ther doesn’t cut out when multiple devices are doing it at once? Wired wins.
Self-hosting is much easier than it was 5,10,15 years ago. Tools like proxmox loaded with yunohost running on a 1L usff/mini/tiny pc with 64 gb of ram and mirrored ssds ther maybe mirror to another i de oval box can sip power but power production grade apps for you personally. If you buy big beefy servers it will cost a bit more electricity.
Still, hosting locally can quickly pile up the savings on saas not spent.
Imagine being able to keep up all your test projects you might spin up a paid VPS for and run for way too long. They might not be production grade, but there is something valuable about having them around.
Data backups - the cloud is just someone else’s computer sold as convenient but not secure. Having your own local copy when the internet goes down means not as much of your life or work goes down. Remember it’s not a backup if there is only one copy of it.
Backing up your computers in the cloud are only so helpful when they are down, a lot of time to download them. Local backups win again.
Multiple connections can matter for people who wfh and can’t be down, or have spotty internet that is up.
If you like smart home tech, it’s a ticking clock until the cloud based supports for it that are usually free dissapear leaving perfectly good gear unusable. Instead you can run a local instance of home assistant, etc.
I used to host morethan I wanted ina. Data centre much like this rack. I was hesitant to come back to self hosting or homelabs but j have realized a home server that runs like an appliance (in between that wiring) capturing the sum of all my data and worn as services come and was pretty much unavoidable. Luckily it’s getting easier and easier to do.
Hope that helps.
This is exactly the kind of nuts and bolts guy that is indispensable when keeping a cloud running.
If, say, you use AT&T and you need Internet for your job (WFH?), redundant Internet is crucial. It's not exactly "minutes" they are out. (Their outages states are often "meh, we'll get around to it in a day or so", figuratively)
If you live in a rural area, even more so. You're not high priority for fixes in the first place. (And AIUI, the author lives somewhere in rural Texas)
Do you need to run it as high tech as that? Probably not. My alternate Internet is my cell, tethered. (And if that breaks e.g. during travel, yes, I have a second cell with a separate provider, because heaven forbid the US had consistent cell coverage)
And most of us probably don't give a damn about e.g. 1,000 year photo retention. Or many of the other things he's doing. But it sure is fun if it appeals to your personality, and you can afford to keep it running.
@monstermunch You're living the dream :-)
Cameras and weather station are nice though.
Just built one of my new servers (Cisco UCS 220 M5) with 2x 4TB Enterprise NVMe drives, 4x 2TB enterprise SAS SSD drives and 4x 2TB SSD drives, 256GB of ram and 2x Intel Gold 6230 (20cores). I don't even need this, I just want it.
Just deciding on if to keep with FreeBSD and bHyve which I love, or go with VMware.
If owning an IP block wasn't €LOL, I'd complete my monopoly set.
(Sorry, had to brag somewhere)
I had a single SuperMicro in the basement one summer and it had to go after a month or so. You could hear it everywhere.
I can tell you my 1U R620 servers lasted less than 6 months before I sold them for 2U R720s. I still wouldn't want to sit in a room with the R720s but you couldn't hear them through a door. Sold those when I moved and downsized to a tower server R430, equivalent to a 4U. It's in my office now and no louder than my desktop. I'll get rid of the raid array and downsize to a single hdd at some point to silence the heads clicking though.
Do like the fact that he runs a RIPE probe though.
> Who are you to decide if it is necessary or not? > Why are you justified in negatively stereotyping AmericansThe people who complained about those issues thought they were “on the right side” as well.
But in the end it’s all just busybodies trying to force other to live their life the way they think it should be lived.
Do you disagree that humans need to minimise waste in order to fight climate change?
Allow me to respond with a slight tangent:
Suppose you care about reducing plastic in the ocean. So you try to reduce your usage of plastic straws. You spend a whole lot of time convincing everyone you know to buy bamboo straws or such.
Plastic straws make up less than 0.1% of the plastic in the ocean. Meanwhile, fishing nets make up ~40%. Plastic straws are a hard problem to solve, and solving them accomplishes less than 1/400th of the fishing net problem.
And yet, we hear a lot about plastic straws but nothing about fishing nets. Why?
Because it's the more consumer-facing solution. It's vital that people focus on the solution that solves the problem, rather than wthe one that is most easily pointed to.
Should we do both? In theory, sure. But in practice people have finite time, and you have 30 seconds max before they start tuning you out, so you're better off giving your elevator pitch on fishing nets.
When is it net-negative compared to pulling energy from the grid?
Why ask something akin to "why does anyone ever even try to audit stuff, tho?" What's with the stark epistemological nihilism?
Why do you even respond if you won't actually address any of the questions? How is that not exactly the bad faith argument style you imply I make?
I have no doubt we could examine OP’s life and find plenty of “wasteful” use of energy but no doubt OP would say “that’s different”. It’s always “the other guy” in these cases.
Add on top it’s a lazy comment that adds nothing to the conversation.
I've seen stuff that or more vile etc. of course, but as far as sanctimoniousness goes, this is the worst I've seen in my several decades on the internet.
Even many northern US states (around the same Latitude as Madrid, north), on average, are 30-35+ every day during the summer and 0C to -10C in the winter. Many Canadians have air con.
So yes, heating and air con is a requirement in many areas. I live in a northern state, and we regularly have full weeks that are 38-40C.
A part time job of mine is helping a buddy's solar company and right now I'm working on a project, for an older couple who are building a 4200sqft house with 4 mini-splits, 2 electric boilers that switch to gas when it's cold, 3 refrigerators, 2 deep freezes and a potential future electric vehicle. There's a pool too but that changes the dynamics of the system greatly but I don't design and quote that unless they really want to spend $$$.
I'm glad they're deciding to go solar but I see this so often it hurts. An older couple without kids in a giant house in the desert that requires immense heating and cooling wants some solar to help offset their gross over consumption. Rather than scale down to a, still large, 2000sqft high efficiency house they go with cheap construction so they have more than enough space for the one time a year all the family shows up. Oh yah then they fill it with crap that's rarely if ever used.
There's no judgement thrown to the customer as this is the reality of the world we live in, but I see this as seriously problematic. Especially going forward as these older individuals die. Can a family of 4 or 6 afford that place? Do they even want a place so large which requires such an inordinate amount of maintenance? Three 1400sqft high efficiency homes could be built on approximately that same space bringing down the price of the home, reducing maintenance and providing more than one couple space to live.
Sorry, that was more of a rant than anything. Europe is not the promised land but the US really has some screwy priorities when it comes to housing.
Lol, you write that as if the Europeans were consuming half as much because they are morally enlightened.
Europeans use 6Mwh/person because thats as much as their emasculated societies can steal from Africa. May I remind you they didn't leave Africa very willingly in the 50s-70s. But they kept on stealing anyway!
Thats right. Steal. Libya, Algeria, Niger. The French convertible Franc. The French senate bemoaning just this week that Macron "lost" Africa. Lost Africa? Africa is not France's to lose!
Unlike, say, Japan or Korea that have more than just a token few LNG terminals and pay for their gas with monitors and great cars.
I'm tempted by the new Alder Lake N100 mini PCs that are available on Chinese websites. But I need to wait until the laptop dies before I can justify it. It's not just about power consumption but the (hidden) cost of manufacturing. Make use of the stuff you already have before buying new stuff.
At least he had a big solar array...
You need low power server? Go for Atom like CPU with good case. My home server running D2550 and 2x 1TB HDD is less than 40W. PSU is cold.
You do have a point in that certain devices can suffer from overcharging and bulging batteries. This issue is largely addressed in newer devices. I can see that the BMS in my laptop server is holding at 90% charge. Definitely worth checking if you plan to use an old phone/laptop/tablet as a 24/7 device.
> Make use of the stuff you already have before buying new stuff.
this is the way!
if you need a heavy duty machine, use it, but most of the time you don't, that's the point.
(for comparison; skylake laptop uses about 20W under max load, my 12-gen desktop uses 40W while idle)
Don't bother appeasing climate hystericals.
And my opinion is that the setup is indeed overkill. The author(s) literally admit so themselves.
The point that this is overkill and power hungry is completely irrelevant. OP's comment is coming from their high horse.
I get it, it looks cool and damn fun. I dream of this kinds of setup. but brushing off the environmental impact is also wrong.
Now admittedly, he is but a drop in vast ocean when it come to pollution and waste, and there is definitely far worse offender. But keeping our environmental impact in mind should be a necessity in this era.
Individual accountability is important, but self-flagellation or (vicarious) shame because forces bigger than us are causing problems is a waste of energy (ha!).
It's similar to when there's hypocrisy charges levied against "champagne socialists." Nice virtue signalling for worker's rights for someone with a 6 figure income... meanwhile that 6 figure salary accounts for 0.01% the net worth of a billionaire.
Go for the high stakes first, then we can worry about homelabs pulling more power than they should.
Holier-than-thou much?
city dwellers not so much
i'd like that number put into perspective, but it does not feel like on the low end.
> Yet the most popular vehicle for the past few years has been the ford F150. It's not being bought by rural or working class people, it's a luxury status symbol now.
thanks for the insight! but i struggle with the combination of "most popular" and "luxury status symbol"
Sometimes people who live in urban areas like to pretend that no one lives outside of cities except cows and guns. 20% is still quite a few people at about 60 million. Not a exactly a rounding error.
My childhood in NZ was like, medium sized perfectly reasonable Japanese cars on the road, some small-ish (by American standards) utes where people needed to haul a month's worth of groceries back to the farm. Now everyone & their Mum is driving an SUV in the UK (where I'm living now) and this is made even more obscene by the narrow, historic roads here.
And you look at the inside of these plush, rounded range rovers designed not for crawling up a mountain but the urban commute and if you're lucky enough to spot one not being driven by a single occupant (also common), expecting ridiculous cargo and passenger space and...no, they've just somehow managed to fill the space with more nothing. 2-2.5 tonnes of nothing. A >3L engine to move your nothing around.
Even American friends of mine here agree & laugh at "oh v6/v8 is common and they're 3-7L and yet aren't really that efficient at making power".
It really IS unnecessary, as you can tell by the author of the article referring to their home network as "overkill".
5-10 years ago I would think this is perfectly fine. I believe I was not alone in this, but maybe I was. The energy would have cost pennies too and why whine about it?
Now I think it’s exorbitant as well, for me. The F150 etc, it’s telling a different story now.
I do think it’s kind of below the belt to snipe any single culture for this. It’s too easy to target Americans. I don’t know about this guy’s life. Maybe he has a ranch with 20 square miles of renewables and I don’t care actually. It’s just personal reflection.
When I moved into my own home (8 years ago) I brought my 'homeserver' with me. Which was just a simple i5-2600 build with some shucked drives in it. I never thought about electricity prices when I lived with my parents. But that changed rather fast. With the server gobbling up a constant 90W, I quickly realized that, even back than, it would cost me 15 euro's a month on electricity alone.
I than proceeded to put a Pi next to it, that would listen to incoming Plex requests and would start up and shut down the server with WoL. That only reduced costs by about a third. The next couple of years I would move on to a NUC with a NAS that would only consume about 29W/h on average. Which was decent, but not great considering the poor performance of both machines (J4105 and J1800).
Last month I have gone back to the DIY route. Now with a i5-13500. I'm still in the process of optimizing it for power efficiency. Although much more stressful than the prebuilts, I love the hunt for the last watt.
Anyway, what I wanted to say is that I notice that family and friends don't really care about saving power in general. They mostly just pay for it and there's that. While my house runs 100% on electricity and I'm really proud if I can get 9kWh/day on average. Even when I see that (for example) the 8-bit guy uses 100kWh/day on average [1]. Which seems out of this world for me.
I'm environmentally conscious (0 cars, 0 pets) but I haven't spent any time measuring and optimising electricity usage.
I've had a look at my own electricity usage every once in a while, now averaging 48 kWh/month (between 38 and 68) for a larger than average apartment for two people.
How hard is this to configure? I have a server at home I use to run a database and computational heavy code, however I am the only user so realistically it is only in use 8 hours a day and some weekends etc. However in the fear of forgetting to turn it on before I go to work (or if I suddenly find time to work while away) I find that I default to leave it on. Being able to control it would be fantastic.
You can make it simpler to use by making an alias in your shell, or a button on your phone (with one of the countless "ssh button" apps). Or even make a web page for it (some php or python that just calls the WoL function).
OP describes a more transparent (and complex) setup where the Pi presumably acts as a reverse proxy. I'd be curious to know the exact setup too, one of the simplest ways would be to use wake on unicast: https://news.ycombinator.com/item?id=35627107
Other ways include wrapping some scripts around socat, writing your own proxy, systemd socket activation, etc.
A reverse proxy would of probably worked too, but I didn't want to be limited by the 100Mbps network interface of my Pi 3B.
1. https://gist.github.com/alex3305/8cc73ddd2c8ca6328f20235480a...
Not at all. Just ensure that you have WoL enabled on the host machine and than proceed to send a magic packet. You could even do this with Home Assistant [1] if you are into that. I did this with a script that used tcpdump to monitor for incoming traffic [2] for Plex with an additional (dummy) Plex server on the Pi. I also remember faintly that I had to add 1 library and 1 video file to make this work though.
Powering down - or sleep - is a bit harder. I built a 'Sleep on LAN' app [3] for myself years ago that could power down (or sleep) a system on demand using a REST API. I used this and Tautulli [3] with Home Assistant that would check if there were any active streams and if there wasn't any activity for a specified amount of time I would send a SoL request to my service.
As you can see it isn't super hard or complicated, but a bit cumbersome to find all the moving bits and make it work. But when it does, it's IMHO fantastic.
1. https://www.home-assistant.io/integrations/wake_on_lan/
2. https://gist.github.com/alex3305/8cc73ddd2c8ca6328f20235480a...
Second, this person seems to have a very large house anyway. My house manages with a modem, a wifi router and two IP cameras, but that can just scale up linearly.
When I grow up I want a switch rack like this in a basement or closet, wired network everywhere and PoE security cameras. Bigger house also means I can get an array of solar panels and become net positive.
Most AWS (and generally cloud) instances are already virtualized from bigger machines. Power, heating, and cooling in these large racks is running much more efficiently than running these things in your home on dedicated compute that is only being used for your purposes. Even more efficient is using something like serverless compute or something like Cloudflare workers. While cloud compute probably is not running at low idle power, it's probably maximizing it's power utilization as much as possible.
> Bigger house also means I can get an array of solar panels and become net positive.
There's a lot of hidden costs associated with becoming "net positive", everything from adding more peak load during inclement events to issues associated with grid fragility.
Look I don't think it's good form to roast OP on what's really standard behavior on the internet: showing off your cool digs. But I also think we should be intentional about what it is: showing off your cool digs.
The rest of the services that OP runs I run on a combination of a 12 year old SBC and a Raspberry Pi. Modern computers (read < 15 years old) when used as server appliances are very capable and can packet switch very quickly, even while running a kernel and a usermode. Doubt me? Run your own iperf tests and check where things are bottlenecking.
My IP cameras are all Raspberry Pis that use the camera module and stream using RTSP back to the SBC. Instead of managing VLANs (I do offer a guest VLAN but my partner and friends found it a pain so it doesn't get much use shrug), I run Zerotier through the house. My IP cameras sit on a single Zerotier network. All other services I run internally use Zerotier and if I do accept external traffic, I expose a port on a VPS (like OP) and have some nftables rules which NAT the traffic through Zerotier. I've thought about getting a failover ISP (our area has 2), but our primary ISP offers us fantastic peering (I've exhaustively tested the bandwidth and jitter to the VPS I proxy traffic through) and good enough uptime for me to not bother.
Like a lot of other hobbies, people can get very consumeristic. This is what a consumeristic version of homelabbing looks like. This shouldn't stop you or anyone else from building a humble homelab that can do almost all of what OPs home network does. The key is to test where your network has bottlenecks. I've iperf tested my network exhaustively and frequently look at kernel counters and other things to make sure that my network is performing up to spec, but that's a lot less fun than buying lots of cool networking gear. Much like you don't need an F-150 to get around day-to-day, maybe even just a bike :)
This is also my case. For Internet access, I use a router which has 4 2.5 GbE ports for the internal LAN, which was made several years ago with an Intel NUC and 4 USB Ethernet dongles, but which could be done today with a small and cheap fanless router using an "Alder Lake N" Intel N100 or N200 CPU. There are also such small routers with more 2.5 GbE ports for the internal LAN, e.g. 6 or 8. That router implements a lot of network services, including DNS servers for my domain and my own e-mail server.
I also have 5 servers that are interconnected by a 10 GbE subnetwork, but that is done without external switches. Each server has a dual-port 10 Gb/s NIC and they are interconnected directly, in a ring. The servers are normally powered off. Whenever needed, I power on the first of them through Wake-on-Ethernet. Through it, I can power on, also using Wake-on-Ethernet, as many of the others as needed.
When a part of the servers are powered off, the topology of the subnetwork becomes linear and the throughput becomes limited to 10 Gb/s, but with less active servers the traffic is also less.
Greater performance can be obtained by using two Ethernet cards with dual 10 Gb/s ports in each computer. With 4 ports per computer total interconnection can be achieved up to 5 nodes and above that each node can be connected to two intersecting rings, which is a topology very frequently used in HPC.
An alternative is to use a single card with dual 25 Gb/s ports, per computer. Such cards are not much more expensive than the 10 Gb/s NICs, but the 25 Gb/s switches are very expensive, so a ring topology without switches is even more attractive.
Currently having a house remodeled, which I'm putting as much solar (and battery) on as possible, and the gas is being removed entirely. Going to try not to use more than the system can generate, though I am not sure I'll be successful.
We really need to stop demonising the use of energy. Energy is amazing. It is the reason we have seen to much prosperity in so little time. It's the backbone of our economies. Of medicine and the internet and EVs and well insulated homes. We should be doing everything we can to make energy cheaper, enabling more people to use more energy. Few things have been as damaging for climate activism than the war on energy. It's unnecessary and misses the point of the entire movement: saving the environment. Focus on cleaner sources of energy rather than demonising people using energy.
Like me.
Source: https://www.nei.org/resources/statistics/state-electricity-g...
Some people fly to Africa and shoot elephants for fun - as far as hobbies go, this is fairly benign. If you care about the environment, I mean actually care, then you should recognize that policing how many computers someone have in their closet is counterproductive. I don't buy the "every little bit helps" argument because there is always an opportunity cost. Attention spans are limited and focusing on this non-issue does nothing but draw attention away from things that actually matter like meat consumption and fossil fuels.
You can't tell from a blog post if what he's doing is good or bad for the environment. And further more, its unfair to tell people that they shouldn't improve their niche skills when those same skills are what they use for employment.
I don't even know why I'm replying to you, but here we go
So I should ditch all this and rely on cloud services I guess. Cloud services are hosted in magic land where they don't use any power, right? If you use any cloud services, you are just as bad as me.
I have 17kw of solar on the roof. As we type this, I'm exporting 10kw of power back to the grid. I generate enough solar to completely cover all of my electricity usage, and during the night when I'm pulling from the grid, my power plan is sourced 100% from wind or solar.
All of this is my hobby, but also helps me learn skills for my job. That job is 100% work from home, which means I don't drive pretty much at all. I'm sorry my F150 offends you, which has less than 10,000 miles on the clock despite being 5 years old. That same F150 also gets better miles per gallon than a lot of regular cars on the road. My wifes Ford Escape (A compact "SUV" if you can call it that" gets worse milage than my truck. But I'm sure if you saw a Ford Escape, you'd be fine with it right?
Where in the post did I say how much power all this uses? You said "power... so much power" yet you don't even know how much power it uses...
I won't even go into detail how I've removed over 1000sqft of concrete on my property, how I've planted an 800sqft native wildflower meadow, have a bat house, use rainwater for plants, etc etc. But no I'm just such a terrible American. Grow up.
Its not quite as green as that right now, since we have a pretty bad drought here in Houston, but it will come back.
I had to make the small fence as it just kept trying to spread out. The fence is made from scrap 2x4's ripped down the middle to make 2x2's, and the rails are old scrap cedar pickets pressure washed and ripped down
This is the seed mix: https://www.dkseeds.com/shop/flkbee-website-king-s-feed-the-...
I'll have to take some pictures of the bat house, maybe I should just make a post on all this, maybe that will please them
My homelab is a mess of cables, and half-baked deployments running on old Supermicro and custom servers, that is way too noisy, and probably pulls a lot of power. I wouldn't know, as I've yet to track power usage... (One of those projects I never get around to ';-D)
Everything's stacked on a Lack rack, even though I have a 24U rack in a closet doing nothing. This is partly because of space and cabling constraints, but _one day_ it will all be neat and tidy, clean and efficient. One of these days...
I dunno man, sounds identical to mine...
1. 24p Patch Panel (Odd ports of switch) 2. 48p Switch 3. 24p Patch Panel (Even ports of switch) 4. 1u Cable Mgmt
Then you could have bought 4-6" Cat 5e/6a pre-made patch cables to connect in the majority of your patch panel ports. The only cables that would traverse through the cable mgmt panel would be longer cables to the servers in your rack.
I have a similar Cisco switch as your 24 port. I am not fan of how they arranged all the ports at the right side of the enclosure. Why not make the 24p one row along the top (or bottom), like the 48p with the 2nd row removed? It would be a lot easier for cable mgmt.
If I could re-rack it all, I might make some changes. But that means turning everything off, and I'm not sure when, if ever, I want to do that
Decent robustness check
When I put together my own home setup (see https://www.reddit.com/r/HomeDataCenter/comments/ktz6yo/my_s... )
I had the patch panels above a collection of switches in both the MDF and IDF closet. About a year in I rearranged it to the more common interleaved switches and panels, mostly just to make it look a bit cleaner.
I moved from 10g to 40g for switch interconnects in a few places, as well as used the fiber I installed to do 10g to most of the desktops, etc. Fun stuff for sure!
This post is overkill but what mostly works for home networks:
SQM - like cake sqm in openwrt - will work wonders if don't have a great ISP. I have seen as much as 5% packet loss on one ISP who said thats acceptable for a home network. Just reduce the load until a point where packet loss is really low. Then use a DNS resolver. DNS resolver will be much more reliable like unbound for your local network as packet loss has cause really problems with DNS for me. Cheap home network equipment dns is not reliable. You could really just buy a cheap Pi and run a local resolver. It would work a lot better than your home router. You may also want to consider replacing the ONU. This is rather easier than you think. The ISP ONU are really bad. A cheap NOKIA SFP GPON is like 20$ on eBay and if you can figure out the PLOAM password and Serial and Mac and VPN ID of your ISP, you will be able to simply swap it for a more stable link.
All in all I would suggest for a simple home setup: get a cheap SFP GPON and replace your ONU. Get a cheap router (either router on stick with Rpi4 for <500mbit or x86 box with SFP for hitting 1gbps with the SFP GPON) and run your own unbound resolver. Get some APs and either wire them or get a cheap mesh router 3 pack on amazon (you can get a wifi 6 3 pack for under 150$ these days on sale) and run it in AP mode.
GPON networks are often vendor locked. This means you cannot just buy any random SFP ONU and expect it work.
Even if you do get it to work, no ISP will offer you any kind of support if you do this. Some might even take to it poorly, if they find out.
Generally, the best option is to ask your ISP if they offer other ONU models and choose from those.
ISP ONUs are locked for easier management for them. They will give you 1/10th of the speed promised if it helps them deal with support calls remotely rather than visiting your place to fix it. Support calls where people complain for things like 'I forgot my wifi password' cost ISPs a lot of money. So they can basically just login to your ONU at any time remotely and change settings for you like your wifi password. They do remote firmware updates and what not remotely. This is the biggest reason why ISPs love such modem router combos. Support can be guaranteed with a phone call to fix your wifi for the average Joe.
You will not get ISP support if you use your own ONU but if you are using your own ONU then you are already at that point where you know what you are doing. As far as signal issues are concerned, like I said ITU specs mean they can see the signal strength remotely. Everything else they don't have access to but you don't need them to have that access. You can always swap out the ISP provided box to troubleshoot.
If you actually swap out the ONU to a better one, chances are you'll never need to call your ISP unless there is a fiber cut or some serious signal loss somewhere.
"it's always dns" should ring a bell :)
not really the fact that some domains i don't care much about are blocked, but this is a dangerous precedent in deviation from facts.
A local network resolver instead of forwarder on a client seems to work a lot better in my experience especially when I use root DNS on unbound. Yes a root DNS is a lot slower than something like 8.8.8.8 but you could use an upstream dns with unbound if you really want something faster. Either way, a local DNS will be lot less problematic compared to if you devices has to handle all dns over wifi including the redirects (which happen a lot).
Almost every time I see a network problem, it's always a DNS problem. In a typical home network setup these days:
client DNS cache -> upstream DNS -> root DNS. client DNS cache -> local DNS - > upstream DNS -> root DNS.
My setup:
client DNS cache -> local DNS -> root DNS.
The first one works better if the network is wired. Second and third one works better on wireless.
One alternative that works 'okay' is if the client directly uses DoT or DoH for DNS. TCP works little more reliably than UDP (wink) but neither Google nor Cloudflare offer signed profiles for iOS and macOS (third party is there). Stubby works good on linux and android and windows have a built in solution.
It came up the power connection and the copper phone connection. The modem/router/switch fried, then it spread out though the twisted pair Ethernet and fried every interface connected to the network.
These days I use WiFi unless I really need low latency / high reliability in a particular location. Optical would be nice, but WiFi is the poor man's lightning safe network.
And then standard power isolation and surge protection will handle all the power connections.
I've yet to loose equipment to lightning from the power cables but I literally can't count the amount of equipment that has been killed by lightning striking a copper data line and then surging through my network.
What’s the best way to implement power isolation and surge protection?
Something as simple and cheap as this would do:
https://www.tp-link.com/us/business-networking/accessory/mc2...
It was a lot easier to run a pair of fiber-optic cables there than it would've been to do the same with cat6. Also I got to play around with fiber for the first time.
Never thought of isolating my modem from the rest of my gear though, I need to look into that.
And then you can put surge equipment on any other incoming wires.
With only one connection (power or phone) the whole device tends swing at the same potential. Damage can occur if it arcs to ground, but that's less likely as in a device the isolation to ground is typically higher than between the power and data parts of a circuit. (Extra bad news if the arc to ground happens via a person.)
Incidentally, if a lightning strike is nearby a typical consumer surge protector will improve the odds but probably won't prevent damage.
When it came back, that modem still worked, but it was stuff "off the hook" forever. I had to unplug/replug it every time I wanted to dial out, and the phone line was permanently busy.
A few months later we got struck again. The router survived but the building caught fire.
Now that I'm older I just accept the risk that "sometimes nature hates you". Instead of jumping through hoops to protect my gear from nature I just set aside money to replace anything I can't afford to lose.
Life's too short to play tug of war with the planet and I'd rather set aside $50/month to pay for replacements when I need them.
I wonder if this person works in IT? I’m a dev lead, I do silly amounts of hours and when I actually have a decent amount of free free time up my sleeve, the last thing I want to do is tinker with my homelab.
Overall power usage throughout the year for me is pretty high, on the order of 8MWh/month, and I get about 25 MWhs of total solar production. My servers and network gear take about 4kw sustained, which is 35MHws, so my solar covers about 70% of my computation and network power usage. I live the NW, so I produce most of that power in the central 5 months of the year.
If you don't mind me asking, does your energy bill take a huge blow because of this? I had a modest homelab set up and had to start shutting things off due to how much it costs to keep it running
I apologize if I missed this info in the blog!
I have a rack setup and most of the time it consumes around 200 watts during the daytime and 100 watts at night, but can spike upto 600 watts if I put a heavy CPU+GPU load on it.
I also put my desktop into suspend at night, something which I think a lot more people with desktops could do. Don't run 24/7 services (e.g. Home Assistant) on your massive desktop with an i7/i9 and a GPU. Run that stuff on a NUC or Pi4 or anything that has low power consumption. Then turn your desktop on only when you're actually using it.
I'm curious as to what others are doing to save costs if anything. I love the hobby but we're in a recession, lol!
In an era of abundance we seem to be trying to use more power to support what we already do, rather than rethinking what we do to use less power. It's a shame!
But at last, I'm a renter and my property management isn't going to allow that.
We're literally wasting clean energy because of a bunch of rich turds hogging real estate with shitty rules about what you can't do. When my property changed management the new management even banned EV charging on the property and locked up all the outlets in the parking lot. These kind of management are not welcome in Silicon Valley but somehow they are here.
I'd almost be willing to bet money that your landlord would charge a "renewable energy" fee even if they're just dumping power back to the grid for $$$ even if they had the panels
My personal approach is separate archived storage from working storage and keep archive storage offline until I need it. Keeping hard drives spinning is costly.
That’s what the generator is for.
In all seriousness, thanks for sharing, this is really incredible. I see a few similarities (fellow Harbor Freight shopper, ADS-B receiver)... but I took the mostly lazy way out and just use the TP Link Omada router, controller and access points. Works great for well over 50 wireless clients that we had at a recent BBQ. I particularly love your note about encrypted LoRa networks at the bottom there, I'll be interested in a follow-up on that topic. Thanks again!
I've been very interested in the new TP-Link stuff, they have really come a long way. I don't know if I've really ever heard of much wrong with the TP-Link stuff
but is astonishing how expensive is the connectivity in other countries.
>>> Verizon Gateway sitting on top of the rack. I get pretty good signal here and get the rated speed, of around 300Mb/s down and 20Mb/s up. This costs $50/mo.
There are offers [0] in the EU of 10Gbps / 25 Euros/mo.
I'm confused, what's faster than what?
Curious to know how hitting HN impacted your setup (if at all)!
Total bandwidth over the past 5 hours is 253.37 GB. But total cached and served by CloudFlare is 237.82 GB. Uncached is just 15.54 GB
Here is CPU load on the VM: https://files.networkprofile.org/apps/files_sharing/publicpr...
Networking on the VM: https://files.networkprofile.org/apps/files_sharing/publicpr...
I get blisters on blisters on my finger tips when making lots of cables.
Gave up on that years ago
Today, Ubiquity HW. Enterprise 48 PoE, 1/2.5/10G core, small PoE powered 5 port switch behind TV, etc. for fan out where needed (I did rewire house with Cat6A). 3 of the latest APs that support 6E. NVR for the cameras. TrueNAS MiniXL+ for NAS that I can run K8s on if needed (cheaper to pay IXsysten for the setup then build myself, including time). AppleTV for media. Leviton Wifi switches for remote control power/lights. 5 gig symmetric fiber internet with a Comcast backup that auto fails over (UDMP-SE for firewall). It all just works and I spend zero time on it.
About the only tech head thing I still have is a stand alone 4 generation old CPU (i7) running ... gentoo targeted at that CPU for speed, as a Minecraft server. And honestly that is going to be replaced with a 2 year old SFF box soon with a less complex version of Linux :)
But the main reason they’re gone now is that people put the laundry on the same floor as the bedrooms in modern homes.
Non rated cable more easily catches fire and makes a lot of smoke. Plenum rated cable is about twice the price, but ethernet is pretty cheap for a single home.
ESXI 8.0.1 worked out of the box with the 7090 I was given.
I'm migrating it from 6.7u3, which is running in 14 year old hardware (injecting the drivers was easy btw - finding the drivers was difficult).
ESXi is not managed with vCenter, it's a standalone host. I'm hosting a handful of VM's that do everything I need.
Critical items are never sorted solely on the server; they're stored on three remote sources, with integrity checks , and mock restorations.
Fortigate 140d PoE, which handles a phone system and all of the PoE I need.
Backups are simple. No RAID, again, critical stuff is not solely on the server.
I use enterprise disks, and SMART - I'll get a notification with ntfy if something gets wonky.
Server is consuming 60-160w. 16gb of RAM, non ECC, 5th gen i5, 12TB of storage.
Homebridge, home assistant, scrypted, Asterisk, Samba, Timemachine backups (all containers configured with compose, checked into Git) windows 10 and 7 VM's
All of the hardware is second hand, with exception to the disks. Ambient in the closet is 29, no dedicated cooling.
UPS with cyberpower.
Glad to see you're using solar, and that your hardware too is used. When I worked in the industry they'd toss stuff I could never afford...
FWIW: Proxmox seemed like something new to learn entirely, ESXi seemed similar for me; I don't mind the RAID stance, or the licensing...
1g lounge: must pad message with 1gig of zeros
10g lounge: must pad with 10 gigs of zeros
100g lounge: must pad with 100gigs of zeros
1t lounge: must pad with 1 terabyte of zeros
100t lounge: intelligence community meet-and-greet
Edit: relevance :)
Competition here has gotten pretty good with lots of virtual carrier being allowed by law to use telefonicas ftth which has driven prices down.
I’m the opposite to you where I’m always surprised when I hear people in fairly big towns/cities in US/UK etc are still on less then 100mbit symmetric!
A kilowatt?
otherwise, i can imagine it'll still be cheaper to not run things than to run things. but it's a hobby, people spend energy and resources on more frivolous things, and those frivolous things make us human.
That Generac generator though: it powers everything but only in case there's a power outage right?
P.S: there's a slight typo in TFA: it's Mellanox, not Mollonox.
Thanks, I'll correct that!
My only complaint is that there’s no public API and thus no official Terraform providers.
The UDM in particular is a masterclass in how to upset all your customers. (coming from a previous all-in UBNT customer that had a first gen UDM Pro).
Ebay Ruckus + OPNsense and my network has never been more stable and performant.
I don't need this setup at all, like I don't need cat6 ports in every room, there's definitely an element of it being a hobby. Maybe I just like flashing LEDs.
Sonos gear still using STP was an unwanted pain, all part of the fun I guess.
I also read some bad stories of the Dream Machines unable to restart after a power loss.
And thanks for sharing your knowledge. Infrastructure has been an off and on hobby, but it’s hard to know how to take it to the next level. Or absurd levels (in the best possible way), in your case.
My setup currently consists of a single server with an i3-12100F, which does all of nas + hosting (9TB array for windows image backup of all local machines over smb + Home Assistant + AgentDVR) and networking (dual ISP in, single gigabit out, all handled by systemd-networkd and ip + firewall), and several 1G-CAT6 runs connected to an unmanaged switch. Wireless as several random brand APs scattered around the house, all running on low power mode and being pretty much room-confined, 5GHz everywhere, had mesh before and my experience is it just sucks!
Your post made me interested in those RIPE Atlas boxes.. I might see if they are willing to send one down here for me.. with two different ISPs on different upstreams I might have some interesting data.. and maybe use that raspberry-pi I have for a stratum gps server like you did as well... and maybe a weather station too.. down the rabbit hole I go again!
Currently I have four small racks:
Rack #1 & Rack #2 dual side-by-side (custom built)
Rack #1
NAS x1 4U
NAS x2 4U
Patch panel 1U
Rack #2
PDU 1U
UPS 4U
UDM Pro 2U
24 Port switch 1U
Rackmount cable modem 1U
Rack #3 (it's an arcade cabinet)
Workstation 4U
Rackmount Mac Minis (Intel & ARM) 2U
UPS 4U
Rack #4 (custom built)
Supermicro 4U server (330TB) that doubles as the VR computer
Supermicro 4U storage box (700TB)
PDU 1U
UPS 2U
I appreciate your hobby of a well installed home network. Add the diagrams and you will increase your readers' comprehension several fold.
My only recommendation would be switching your virtualization over to Proxmox (LXC / KVM) and setting up an HA cluster with Ceph and MLAG. It's relatively easy and free and will give you a lot more features than plain ESXi and even free vSphere/vCenter.
Yeah, the price on this genset I think has gone up around $4000 since I bought it, not including the install
I've been meaning to try Proxmox, but my day job heavily relies on ESXi, so its nice having something to mess with at home. I am also running vSphere with an Enterprise licence, so I get all the fancy stuff
They can legally do that as we don't have Net Neutrality anymore.
I'm sorry for not contributing with hard evidence but I've seen this behavior (though I never heard about it from anyone else before now)
These days I rarely need large bandwidth to any single host, but have a lot of bandwidth use between many hosts including streaming applications. Bufferbloat is far more apt to be problematic under this workload than ISP priority, unless there is a bandwidth shortage on the local links.
Cloudflare's - https://speed.cloudflare.com
Cloudflare's speedtest often performs worse for me.
iperf3 - https://iperf.fr Is the most accurate CLI speed test I've found (depending on which server you choose).
The best speed test is to download some large linux ISOs with 100+ peers, so you max out your connection to tons of networks.
Downloading packages with $PACKAGE_MANAGER, downloading files from $CLOUD_STORAGE, downloading videos from youtube, downloading linux ISOs, receiving files through $INSTANT_MESSENGER, ... they all are very close to the speed I get with fast.com.
I like this one because your speed really depends on what you're downloading and this one tests some of those scenarios.
[0] https://www.ebay.com/itm/166190039675?epid=17034031881&hash=...
Supermicro did have RAM issues in the early teens.
This sort of research is the non-dollar price of finding deals on server-class hardware.
If you want to get something pre-validated, and usually with a warranty, it can be worth it to look for resellers for the major OEMs that also offer refurbished hardware. It's more expensive than the typical options available on eBay, but still much cheaper than new hardware.
The killer is the power consumption. It's better now compared to old enterprise gear of 20 years ago, but you can sometimes still pay for a brand new low-power system just in power savings alone.
Of course, if you have solar onsite that doesn't matter.
Especially putting one of these not in a rackmount chassis, you can opt for quiet cooling and fans that do not draw 10s of watts each. The idle loads can get fairly low, depending on your additional peripherals.
For very low power, I actually prefer small form factor machines, either used 1L SFF machines from the big OEMs (see STH's Project TinyMiniMicro[0]) or one of the many AliExpress mini PCs[1]. These sorts of machines tend to use low-power CPUs and a lot of the AliExpress ones are laptop parts.
As with all things, there are always tradeoffs. If you enjoy researching and bargain hunting for hardware, then this is a viable path. If you just want a machine that works, then this is onerous work.
[0] https://www.servethehome.com/introducing-project-tinyminimic...
[1] https://www.aliexpress.com/w/wholesale-mini-pc.html?catId=0&...
https://www.klennet.com/notes/2022-11-25-zfs-and-ecc-rant.as...
IF you can get ECC cheap, get it, otherwise, meh.
The Latte Panda Sigma looks pretty amazing too if in-band ECC is sufficient.
You might however want to read about the grocer's apostrophe: https://www.grammar-monster.com/lessons/apostrophe_error_wit...
I wouldn’t bring it up but… glass houses.
I am sure there are PCIe solutions that could easily go into a more standard form-factor machine, but perhaps these are more expensive or less readily available; this is just conjecture. The ESXi machines have minimal PCIe connectivity. Perhaps such a card could be put into one of the Supermicro chassis.
Edit: USB to TTL costs like $3.5, it's like a standard RS232 serial port but much simpler and faster.
Your accuracy will be less otherwise.
I also like having them bare metal
[1] https://eu.store.ui.com/eu/en/collections/unifi-switching-ut...
But I really appreciate your suggestion, perhaps I am going stick to it as I've seen no alternative so far and my previous experience with Ubiquiti makes me confident it will do the actual job great.
You really helped me a lot! I wish you could feel how my mood has improved. Now I feel empowered to implement the set-up I wanted. This is a minor home improvement task but it sitted in my to-do list (after numerous attempts) for over a year.
Is there a place nearby for cabinet, table, or shelves? Those are good for hiding network gear.
The catch? _No IPv6 support whatsoever_
I wasn't even aware you could use HE's services without being a customer!
It's been around for a long time! If you do the IPv6 training certification, they do eventually send the t-shirt :P
EDIT: They also provide free DNS for up to 50 domains at dns.he.net which is also worth looking at if you don't want to run your own for every little thing.
I had to run a DNS proxy to filter out AAAA records for any Netflix domains.
For sites that use HE.net and have more than one IPv6 subnet, I use the initial /64 for the DMZ, so that didn't matter on most of them. Small sites had just been using the /64 though, and I had to renumber those. Total pain.
The norm seems to be for hosting services to contract with both HE and Cogent if they decide to go with one of them. This is why for typical client browsing you wouldn't notice--the other side has ensured you can reach them. HE seems to be the cheaper option, so people will route most of their traffic through HE, and then separately contract with Cogent for the smaller amount of traffic originating from/to Cogent networks. Alternatively, you contract with another network provider who handles that idiocy for you.
Honestly I just use ipv4 and turn off ipv6 everywhere. Then I have just one configuration and one set of firewall rules (in and out).
In some magic happy land where you can use your IPv6 prefix over both WANs, then yeah, things can work, but that doesn't work for residences, generally.
To be completely honest, I find ipv6 wildly alloying, but I guess that means I need to learn more!
> Raspberry Pi 3B+ NTP server with GPS/PPS time source
Just one though? I have a triple LeoNTP device. Terminated my own cables to outdoor GPS antennae. Been working on designing my own but it's slow going.
I also have UPS power routed elsewhere in the house. But just to 1 room, where I had a powered ISDN phone. I couldn't find an ISDN phone I liked that took power from the line, if that's even a thing. I don't remember as that's ancient history now and I don't use the remote 240v power at all, just PoE everywhere.
The one thing missing is EPO (Emergency Power Off). I built my data closet into the garage, stealing part of a garage bay. It is the size of a smaller sauna room. I have an EPO on the outside, wired to the UPS.
I added card access to the data closet last year, because I also added it to the outside entries of the house, and I had one controller port left over.
I don't have nearly as much switching, and no fiber. But many servers. I would buy an 0xide rack if they weren't $500k.
Can someone with more knowledge comment on whether that particular design is more common in Houston or was it really just a lucky guess? I kinda feel like that carport-style design is very Houston, haven't seen it as much in other cities.
But I suspect it was popular in the south as a whole for a short time, that time being when major sections of Houston were built.
No fancy CPU just a dual core MIPS based MediaTek 7621AT - the key difference is the software, stable and well designed and not some hacked together version of OpenWRT and some old version of Linux from some random OEM that never get's updated after release.
There's cheap and reliable ZigBee/Z-Wave/433MHz USB dongled antennas you can replace the Hubitat with.
My main feedback is give those fibers a bit more breathing room. The bend radius is a little tight.
JK, wife would kill me
Brightspeed did basically the same thing a few months ago, but they kinda suck.
But I also now have a bunch of electrician tools for running wires, some private cameras that aren’t google or Amazon spying on everyone that walks by, and a network that isn’t bogged down by a couple people watching Netflix.
And I have plans for MORE.
I’m not in any position to state anything about the competition.
I am glad to have went the PoE route, even if running those wires sucked ass.
I only did UI cameras cause everything else I have is ui. There wasn’t much more thought than that.
You really gotta do some due diligence to make sure you're not buying lemon parts but it's very possible to get a beefy homelab at fair prices!
My 10G switch with PoE and 4 SFP+ ports cost only $100, for example, while many new products with similar specs cost $600+.
Best $60 purchase I ever made!
But, its all used and all over a number of years, so not as much as you'd think. If you went out and tried to get it all at once, it would be crazy. Most things I spent a good while waiting for the right deal to come up
Like the UPS alone, I saved probably close to $3000 off MSRP
I’m in a very similar situation when it comes connections, FW and needs. I’m about to get a backup line for internet and it’s between Verizon (just like you) or Spectrum (same price, 500mbit down). What was your reason for VZW? Was ATT your only wired option?
Also, the less money I give to Comcast the better.
Plus, I always think fiber is cool
Within my rack I use DACs to connect all the gear, which works well since it allows maximum performance over short runs with minimal effort.
Nice setup though, my comment wasn’t meant to criticize, I think having a proper network at home is a big boost to quality of life that most folks don’t realize because they’ve never experienced it.
But, It would have been better if I fed a small sub-panel from the UPS, and then went out from there. Shoulda-coulda-woulda!
I have way too many power cords under the desk but use fused plugs and boards. Want a small cabinet just to keep cables and devices away from small curious people.
My favorite is ZPE Systems HSR, comes with 2 LTE modems that supports GPS and I can run some containers on it too.
I get using one for NTP.
I guess a second for redundancy even though it is picking up the same satellites as the first.
And the rest?
Garage rack has a NAS that mirrors storage of the closet rack.
Garage rack has one Raspberry Pi GPS for time services.
Closet rack has two (different) Raspberry Pi GPS for time services.
Looking at it that way, if you accept the 2x Internet connections each being independant and acting as cross backups in case of ??? then it's just the second GPS on the closet rack that's "extra".
That can likely be written up as a home experiment in messing about with a different Pi setup.
It's overkill .. but often these home setups are there to keep current on new technology and different ways of doing the same thing .. in that light it's not uncommon to have multiple ways of achieving the same end just to gain knowledge on the pros | cons of different approaches.
Got anything more I can read? Dunno if I'm just blind but I couldn't find anything on this. I'd like to learn more!
We had no problem routing gigabit on an Intel Atom D525 years and years ago at a previous job. Consumer Mini-ITX board, onboard gigabit NIC, and an Intel gigabit NIC in the one expansion slot. It did require minor tuning, but nothing that couldn't be done thru the web UI.
Everything I have to manage runs plain OpenBSD managed with Ansible now, so I don't know what the current state of pfSense/OPNsense throughput is.
edit:
RUCKUS R760 - $1800 - Wi-Fi 6E 4x4:4
RUCKUS R560 - $950 - Wi-Fi 6E 2x2:2
RUCKUS R750 - $820 - Wi-Fi 6 4x4:4
RUCKUS R650 - $520 - Wi-Fi 6 4x4:4
Even a few year old AP will work great. My R510 is pretty old, but works flawlessly
Granted, that means you NEED an Internet connection... But it's as close to "walking data center" as anyone can get.