Does anyone have any useful rules-of-thumb or heuristics for balancing this trade off of upfront cost v.s. power cost? e.g. how much does an N100 cost to run for a year v.s. say a i5-2400s (the CPU for the first row on the linked site)?
In the US electric power might be cheaper. And if it's running only part of the time, you should adjust the calculation.
My desktop/server runs 24/7, so I prefer having a CPU with 65W TDP over one that is 125W TDP. That might run up to 120 Euro per year difference for me (if it would be running at 100% CPU).
Edit: I should note that there's no fan drawing power because I put it in an Akasa passively cooled case.
I ended up going with a HP EliteDesk 800 G5 Mini I5-9500T (35W) off of Ebay for $100 and it does the stuff I need it to do just fine. According to my current monthly power usage graph, it's averaged 7W which accounts for $0.61 of this month's power bill.
Just from tweaking my laptop, I’ve noticed that when it is really idle (or I’ve intentionally put it in a low frequency mode), the big power drains are the wireless interfaces (don’t forget bluetooth) and the screen (OLED helps as long as the screen is mostly black). Gotta tweak the whole thing.
When I did this I was surprised by how much - or how little - it cost to run various devices. It's quite addictive.
It's not always accurate because a lower-power machine doing the same task will often need to work at its full power more often, so the savings may be less. For example, a Raspberry Pi 5 may often be more power effecient than a Pi 4, despite drawing more power at full capacity on paper, because it spends less time at full capacity than the Pi 4 does.
On the other hand, when I upgraded my work PC I found it used less power but I also had to run my office heater more often in winter, as the new PC wasn't as efficient at heating the space.
So, it goes quite quickly. Savings of 20 watt save you $ 52 a year.