Making a Linux home server sleep on idle and wake on demand (2023) | Hacker News Reader

Making a Linux home server sleep on idle and wake on demand (2023)

dgross.ca

297 pointsAgaoAnar108 comments

Ha ha, I tried something similar when I had to go to Thailand for my wife's treatment. And failed spectacularly. Fortunately, my laptop had all the files. I didn't have Tailscale at that time.

My Desktop, WoL on, tested to be working Android Phone, always on

All have SyncThing installed. Mobile had Tasker installed.

So, the idea was to have Tasker monitor a folder inside SyncThing, When I need my computer, I put a file inside that folder, when Tasker finds that folder, it sends WoL to my desktop, and deletes the file, Computer wakes up. When I see the file deleted I know, the beast is now awakens ....

When I actually did try that from Thailand, the file did not got deleted, nor the beast woke up.

What happened? Turns out, my mobile restarts automatically after some time of inactivity. Which, locks Tasker out, and the whole process fails.

So, operation wake the beast was busted.

Since you're already using Tasker, you could use it to launch Tailscale on the phone at boot - I've got mine set to do this so my phone reconnects to tailscale after a reboot.

Now I have another mobile with more control, so restart is not a problem anymore, and as I can SSH in to it using Termux. This basically handles all of my needs.

Now I just need to get enough momentum to set up the process of waking up my desktop as needed and set up radical caldav server in the mobile.

Why don't you send WOL directly over the internet? (You'll need to set up a permanent ARP record on the router.)

Or login into the router and send WoL from there?

Or have Raspberry always on with ssh?

Or set a power-on timer in BIOS/UEFI once a day and a shutdown/sleep in cron? (This is also a good failsafe if WoL doesn't work.)

> Why don't you send WOL directly over the internet? (You'll need to set up a permanent ARP record on the router.)

Hm, so never tried something like that before actually, I think for OpenWrt I need to install arp packages. I do not have public IP any more though, was getting hammered by bot network and found Tailscale to do what I need.

> Or login into the router and send WoL from there?

Restricted outside access to the router management panel, only port forwarded to the desktop.

> Or have Raspberry always on with ssh? I do not have one, which is why used a mobile, thought to be next best thing. As, power outages happen regularly, I need something that would be able to keep on running.

> Or set a power-on timer in BIOS/UEFI once a day and a shutdown/sleep in cron? (This is also a good failsafe if WoL doesn't work.)

That's easily workable for my setup, actually!

Thank you for giving me the ideas!

>> Why don't you send WOL directly over the internet? (You'll need to set up a permanent ARP record on the router.)

> Hm, so never tried something like that before actually, I think for OpenWrt I need to install arp packages.

No, you don't. Put this in /etc/firewall.user or /etc/rc.local (you can do it from UI) and forward a UDP port to 192.168.0.254 or whaterver subnet you're using:

  ip neigh add 192.168.0.254 lladdr ff:ff:ff:ff:ff:ff nud permanent dev br-lan extern_learn

The incoming packet will be broadcasted (ff:ff...) on the br-lan interface. 192.168.0.254 doesn't need to exist - the sleeping computer doesn't care.

Thank you for the explanation!

The version in the OpenWrt didn't support the full command. Had to update to `ip-full`.

After the command was successful, I can see the entry in the `ip neigh` table.

Now digging around

PS:

Correction! /etc/rc.local isn't OK. The ARP record needs to be reloaded if br-lan interface is restarted, such as when changing other network settings in WebUI. So only /etc/firewall.user will do.

I don't think BIOS/UEFI can do power-on timer unless it's server-class hardware? Have not looked into PC BIOS/UEFI for a while.

For failsafe, KVM might be the ultimate solution in case boot got stuck.

I don't know about UEFI - I don't have any UEFI system - but my Gigabyte 880GA motherboard can certainly do RTC power-on from BIOS settings and it's far from being a server motherboard.

You worked with what you had and understood, it was beautiful.

I for one just leave the beast on, I understand that spin-ups and downs are the main thing that wreck harddrives (and my beast has a lot.)

Thank you!

For me, power outages were/are a big issue, lost one of the HDD recently then, did not want to risk the new one.

20 years ago, I used to have a Linux server running Slackware at home, that would wake up the two PCs we had at home to back up their data if they were turned off. If they were already turned on, they would send a WOL packet to the Linux server to turn it on in case the that server was off, and then start the backup routine. And the last one would tell the Linux server to turn-off itself. It used to work really great, good times.

Some motherboards disable power to their ethernet port upon sleep and so WoL will not work.

This is particularly common if the NIC is a power hungry 10GbE port.

However, in the particular case I found, the motherboard also disables oower to any usb GigE adapter attached.

The solution I found was to attach a USB hub with (empty) SD slots and integrated GigE port. As SD cards require power to remain mounted, the motherboard did not shutdown power to this adapter and WoL worked.

This reminds me of the empty box in a corner of the workshop with the written note "Don't remove!".

There's a "EuP 2013" setting in the BIOS/UEFI. Disable that.

If you use Windows, check in device manager for a setting to allow the device to wake up the computer. (I can't remember the exact name.)

Well done. But note: you can grab an ATX control board, or configure the RPi as a USB Gadget to wake the machine via power button or keyboard.

I did this recently as I was struggling to get WoL to work with my consumer PC. It seems like this ultra low-level stuff is a total crapshoot so if you can dodge it by just wiring up the power button, that's a good option.

In in the end I just went the whole hog and set up a PiKVM, so now if I mess up the machine's networking (or even completely break the OS) I can still recover it remotely even though it doesn't have a proper BMC or anything like that.

In general this approach seems ugly in principle but I really like it in practice. It lets you retrofit solid remote capabilities onto consumer hardware. That way you have such a broader market to buy from.

I'm absolutely excited for the nanoKVM-PCIe. They were out of stock last I looked, but they've released the firmware source as promised.

I hadn't seen that before, seems neat as the physical inconvenience factor of the KVM approach is definitely real.

It's pretty funny that you have both the input and output physically installed in and powered by the chassis, but then you run cables to connect them!

It would seem reasonable to at least have the option to have it directly enumerate as a USB hub/display device on the PCIe bus it gets power from! But maybe that would add a lot of bulk/cost?

NanoKVM looks really neat, thank you!

I take it you're in the US? Because it seems I (in Europe) could order it from Aliexpress, see the link on https://github.com/sipeed/NanoKVM.

EDIT: It seems the NanoKVM is struggling with one of the most fundamental KVM tasks, though: https://github.com/sipeed/NanoKVM/issues/594

I really liked the PCIe form factor, but it looks like I'll have to go for https://tinypilotkvm.com/ instead.

This seems like a smart option, that would also allow power cycling the machine remotely I’m assuming, in case it goes totally out to lunch.

Note: If you're going to use an SBC _only_ for wake up signals, you might want to look into alternatives for the RPi such as the Radxa RockPi S [1]. My home server, for example, runs continuously at 7W, which beats many RPi models. Of course, a Pi to wake things doesn't need that much power and could be an older model, but even then, you'd still be burning "empty Watts".

Of course, the RockPi doesn't give you any KVM like functionality, though.

[1]: https://wiki.radxa.com/RockpiS

Apparently RPi Zero idles at 0.4 W: https://www.jeffgeerling.com/blogs/jeff-geerling/raspberry-p...

Yea, technically an esp32 board with ethernet would work for this job. They're typically limited to 10mbit, but if you have 10mbit/s of broadcast traffic you can afford something better.

Not as sophisticated as your solution, but works with all hardware for the "backup use case": use a mechanical timer set to 10 minutes before backup starts (and power off 30 minutes after backup normally ends):

https://www.amazon.de/Mechanical-Analogue-Switching-Christma...

It's a more rigid solution that doesn't let you ssh in (unless during backup time ;-), but it saves electricity and it is implemented in 10 minutes (5 for an Amazon order and 5 to plug in and set the timer to your backup hours). It's also a more robust solution - little can go wrong (the only thing is you need to balance backup time against electricity/time savings as backup size grows).

Or just suspend to RAM on backup finish/time and use wake on RTC alarm (and set to turn on after power return on power failure if needed), no need for extra HW.

That's what I was thinking too. My home server consumes like 15 W and is silent. If you get a rack mounted server made for data centers and stick it in a closet so you can't hear it then yes, i guess this approach makes sense.

I use a Mac Mini. When nothing it up, it draws like 7W!

How do you accurately measure how much current a PC is drawing at any given time? Do you have some kind of measurement device inline with the power cord?

You can use "smart" plug such as the Sonoff S31 (flash ESPHome to make it cloud-free).

And for one-off testing there's the standard P3 Kill-a-watt.

I have it plugged into a IKEA smart plug.

https://en.wikipedia.org/wiki/Kill_A_Watt

Works good IME, but you have to plug it into the wall first. Sometimes if you plug the appliance in, and then plug the KAW in, it will miscalibrate

Yes, I have one of those for my homelab. I draw about 200W on average, but it can spike to about 1000W.

https://www.brennenstuhl.co.uk/en-GB/products/travel-adapter...

In the UK, every 10W of 24/7 load is ~£25/year (33 USD).

It’s very easy for even the small things to add up.

In contrast to many common consumer subscriptions which start at double that it seems like decent value.

(2023), discussed at the time: https://news.ycombinator.com/item?id=35627107

Thanks! Macroexpanded:

Making a Linux home server sleep on idle and wake on demand – the simple way - https://news.ycombinator.com/item?id=35627107 - April 2023 (237 comments)

Thank you for the post, very informative. I do this half manually. I have a cgi script on the always on very power efficient sbc server that wakes up the bigger server if someone needs it. The big server powers itself down when no backup is running at a time everybody using the server usually sleeps. I thought about improving this and measure server usage and solar power generation to decide the shutdown, maybe with an additional warning email. for example: "The server shuts down in 5 minutes due to no demand and no solar power, if you want to prevent this click this link: http://server.lan/cgi-bin/keepalive "

Have you measured the power consumption of your "high power consumption" server?

Seriously. You can get running states around 1W these days.

What to do about this? https://forums.developer.nvidia.com/t/phantom-rtx-5090-high-...

I suggest not putting a ridiculous GPU in your backup server.

Not everyone's "home server" is a backup server - some are for AI experiments.

That one should be Wake-On-LAN

I wouldn't want to fiddle with WoL just to get my self-hosted LLM-assisted code-completion working when I pickup my old and underpowered laptop on the couch for some quick hacking. Wake-on-any-network-activity would be perfect, and has superior UX.

You can use nvidia-smi to set a target maximum power draw and performance mode to bring idle power levels down. Also make sure your computer is using the server/headless mode driver to keep idle power consumption down.

If one can afford a GPU with a MSRP of $1999 and was scalped for $2999 during the initial craze, you are probably not struggling to pay your electric bill.

Running states around 1W? How?

Only put the RAM you need in the box, use peripherals with working ASPM, attach them to the northbridge PCI ports instead of the CPU's root ports, use wireless instead of wired networking, and don't attach a display.

I do basically all of that, CPU idles around 5% but still consumes over 100W. (Minus the Wireless part)

(5800X + 64GB)

I can enable Eco Mode in the BIOS, which will bring down the CPU to about 65W max although its still at about 100W total system.

Whatever software you are using is totally, utterly broken. Not sure what else I can tell you. Even a completely decked out Ryzen AI Max Pro 395 idles at 5W in Windows S0 (see: https://h20195.www2.hp.com/v2/getpdf.aspx/c09133726.pdf)

Chiplet based Ryzen CPUs inherently have higher idle power draw. Monolithic chips like 5600G have lower idle power draw. The motherboard, power supply, and internal peripherals all need to be carefully selected to get a really low idle figure.

1W is a unicorn as just plugging in a power supply with no PC parts hooked up will register 1W. <10W is more realistic. Select a PC that can run off a laptop charger. Check manufacturer spec sheets for the idle power consumption. Don't install any PCIe cards or hard drives. Use powertop --autotune.

> Select a PC that can run off a laptop charger.

(looking at my 330W laptop charger)

I’m not sure that is the constraint you think it is.

To be more specific a 65W or 90W laptop power supply.

Are you talking about processor's C-states? My old 6th gen i3 stays most of the time idling around C8, averaging 5w, really impressive, I suppose newer gens will be even more efficient

C8 is a good state if you can get it. Intel is really good at this. They don't even bother energizing the L3 caches immediately when exiting deep package C-states. But there are lots of conditions that will inhibit C8, notably an Ethernet link on a NIC capable of PTP. This is why wireless is better.

Although Intel processors are efficient, modern AMD processors have much higher idle power usage, due to their chiplet design. They typically use at least 20W more power.

My home server setup when running at full tilt consumes approx 160 watts iirc.

I'd love it if I could turn off my systems for when I really need them.

Do you usually run it full tilt? What's it typically using on average? I've forgotten how much mine averages, I only remember being surprised how little it sips because it's mostly idle.

It's surprising because reddit (and HN) would make you think you're throwing away tons of money unless you go with some tiny ARM board and that's not true.

Mine idles at around 130W from the walls. I think it's mostly the hard drives, maybe the SAS controller. I've migrated a few services to a miniPC and started turning it off too.

The main issue for me is the heat. I've got it next to me and 130W of heat adds up in the summer.

I considered doing something like that but eventually I went for the simpler solution of plugging my little servers into smart plugs. I shut them down, then power off the plug over wifi. I start them by powering on the plugs. The plugs draw very little power. The servers are ARM SoC and draw 1 to 4 W. One of them has an HDD that draws about 10 W but I can unmount and power off it when I don't need that disk and I still need the server on (it's also got a SSD.)

Sounds like a neat setup - can you self host the smart plug control (Home Assistant?), or do they connect to third party servers?

It's a Shelly S. Among the many ways to control it I'm using its local web server to turn on and off the power from my phone, and a curl to do it from my scripts. I'm also polling the url for reading the power level and know when a shutdown is complete, before turning off the power. I think it can be interfaced by home assistant and similar platforms but I never tinkered with them. Self made small scripts are more than enough for my use cases.

This reminds me of this old plugin for Windows Home Server: https://www.green-it-software.com/products/lights-out-for-wh...

You could set a calendar schedule for waking up itself and backing up the clients, and at night the server would go into standby only if no clients were running anymore since X minutes.

My server stays powered off but will boot on receiving wol packet. It’s not a sophisticated set up but works for me. I have an always on Rpi running tailscale. I can ssh to it and send a wol from there. Once my server boots I can ssh to it and run the script to mount its data disks - they are encrypted so I need to type a passphrase. Quite a bit of work but I don’t have to do it often.

I would be tempted to try using the Pi as a router & firewall with the server on another subnet, having it wake the server using traditional WOL as needed. That feels simpler to me and more controlled. But my overall feeling is that not much power is saved here overall compared to a well set up server. Good project though quite educational.

Use `powerprofilesctl set power-saver` to set to a low use profile

https://blog.ctms.me/posts/2024-08-29-running-this-blog-on-a...

Does a Pixel 5 have wake on Lan? Do any cellphones?

I recall this being quite simple last time I tried, just enable WoL in bios and run etherwake from my router (or from some other machine on the network).

But this is about waking up on non-magic requests, just any request?

Most of the time you don't need data center sized home servers even if you do test data center sized software solutions on them.

So instead of the Pi solution, how about using a server that idles at 10-20 W?

This. My Lenovo ThinkStation with a Core Ultra 7 265K idles out at ~10W headless (measured with a smart plug), while it can be very fast when needed.

I don't have a homebrew server for all the coolest open-source apps or a device to wake it up but it would be awesome if someone had a guide for dummies, including what to buy while keeping things relatively future-proof.

While not Linux I have my Windows 11 rackmount gaming server sleep after 30 minutes but wake every morning at 8am for backups using WakeupOnStandBy which works great. I tried using built-in Windows task scheduler but it never worked correctly.

for ideal I need port knocking for wake up

Nothing to stop you setting that up on the secondary device to trigger the listen/wake scripts, but if someone malicious is on your local network and has permission to trigger WOL, chances are you have bigger issues.

If you just want it to sleep during the night you could do something like this:

    sudo rtcwake -m mem --date=+15hours

Well done. I really enjoy blog posts that dive into topics that probably cross the minds of most home lab-ers at one point or another.

I built similar functionality into a DNS server so the device can be woken from beyond the local network: https://github.com/RyanGibb/eon/tree/main/bin/hibernia

There's a bit of a writeup here: https://www.sicsa.ac.uk/wp-content/uploads/2024/11/LOCO2024_...

Is a Pi even necessary? Wouldn't a microcontroller be enough?

any half decent router will allow you to create static arp entries for another host

no need for a standin

Yep, here is the page for it on FreshTomato (the firmware the author switched to): https://wiki.freshtomato.org/doku.php/basic-static#static_ar...

The question now is why didn't Wake on PHY work as intended?

This sounds very similar to Apple’s DNS-SD Sleep Proxy Services.

Indeed. Frankly, that would be a nice standardized solution to your machine register itself on a Bonjour Sleep Proxy and can be cheaper than Pi with a used AppleTV off of eBay.

Seems to me that if you want to waste time and money engineering your setup more net efficient, just buy a few solar panels and LiFePo4 batteries to buffer. You can run other stuff off of it, too.

I always choose “make more money” over “pinch pennies”.

If that were the case you wouldn’t waste valuable time posting snide responses to hn posts.

Start a blog about making more money instead of exploring technology and see if anyone gives a shit

All this complexity to save a few bucks per year on your electricity bill? This is ridiculous, the Pi costs far more than what you can be expected to save.

Are you a programmer by chance? Some are, and they enjoy it!

I know a lot of sysadmins / platform engineer types. A lot of them really enjoy the mix of hardware and software integration challenges.

I don't know any sysadmins who would do this. Task scheduler for Windows or rtcwake for Linux. We try to reduce complexity with existing battle tested tools, not create whatever this is. This is definitely not the easy way.

Ah, you're not a tinkerer!

I definitely am. But this is a very solved problem. It just adds brittleness to the system. Maintenance for this is going to suck. It's going to break in less than a year because you forgot to set a static IP address, or a Python library changes, or the SD card in the PI gets corrupted, or the jack on it fries, or its cheap PSU died and fries the board, etc. Then you're going to have to try to remember how the damn thing works, then figure out what other tiny change somewhere broke something. You should never add more failure points to infra, ever.

I guess everyone has to go through it.

> I guess everyone has to go through it

Bingo, but some people never stop tinkering - their joy is in avoiding the well trod path.

I do agree that it would be nice if no external device would be needed. It does make me wonder if the Pi is truly needed. I'll be looking into this myself as well, I also have power hungry servers that are mostly idling and would benefit from WoL. In some EU countries, energy prices are truly a scam.

> Then you're going to have to try to remember how the damn thing works

Fortunately, the author wrote a derailed essay explaining all of this…

> You should never add more failure points to infra, ever.

Every time you add a new system or a new feature you necessarily “add more failure points”, there's no way around that.

One should avoid introducing more failure points than needed for the functionality you want, that's it.

You say it's a “solved problem”, but you only give solutions to a different problem (starting the server at a scheduled time, when the author wants to start the server on demand).

In many European countries electricity is quite expensive. In the U.K. for example, running 20 watts nonstop for a year will cost you around $65 on a typical tariff. If you have more than one home server the savings can quickly add up.

Also 20W is fairly low for idle draw right? Like I think you can get proper machines down that low if you know what to look for but most stuff bigger than a mini PC is gonna be drawing 40+? I might be slightly miscalibrated though.

Anyway it's not about the money for me it's any the aesthetics. Burning power for nothing is yucky.

Edit: just been Googling around. OP is running one of these HP mini PCs. They are pretty efficient! Some go well below 10W. So yeah I would say for the specific use case it's unlikely to matter very much. But still it's a useful thing to be able to do in general.

My home server is a pig and draws about 100 watts idle.

You should try running powertop on it. It will scrape sysfs and look for things that seem misconfigured, and suggest changes to fix them. On one of my machines it enabled some peripheral power saving mode that made a pretty dramatic saving!

(I also heard that it sometimes suggests power saving modes that are usually switched off for a good reason, like apparently you really don't want some USB controllers going into certain sleep modes as they take seconds to come back).

It's a server motherboard, not a consumer motherboard, so it doesn't surprise me that power management is not great. AMD Rome is also not exactly known for its low idle power.

Looking at powertop, the various qemu virtual machines have high events/sec compared to host processes. There are also a couple of timers - tick_nohz_handler and hrtimer_wakeup - which seem to generate hundreds of events per second.

I think it turned out a lot more complicated than the author expected, but that their solution they kindly wrote up will be pragmatic for someone.

(For example, imagine a big home GPU server that is needed only intermittently, and you want it to spin up automatically on network traffic from family's various devices that you can't modify.)

Of course, if you have simpler needs, and you're willing to send a WOL magic packet from the using devices, you can do in a few lines of shell script. It's a 1-line ssh-to-something-that-can-etherwake-on-that-vlan script, then wait in a loop for the service you need to appear, then 1-line ssh-to-server-to-shutdown when you're done.

The old saying: Linux is only free if your time is worth nothing.

> a few bucks per year

Crunched the numbers for mine - about 150 bucks a year in potential savings.

Not sure where you are but energy bills are sky rocketing in the US.

The complexity in TFA is due only to author's desire to not use magic packets for waking the server, making thus the state of the server transparent for users.

If you are willing to send magic packets to wake up the server, before using it, you can save money from the electricity bill with negligible complexity.

Reply on news.ycombinator.com