Hard disk LEDs and noisy machines
blogsystem5.substack.com
blogsystem5.substack.com
These days, any number of people feel entitled to use our computers whenever they like for whatever they like. The company who makes your mouse driver might want to phone home, then silently download and install a bunch of software you never asked for. The maker of your OS sends themselves a steady stream of data about what you've been doing on your computer: which files you've had open recently, what software you have installed, etc. With countless people using our devices for their own benefit on their own schedules it's nowhere near as useful to know when your computer is doing "something" because it's always doing something.
Personally, I've always preferred the soft whir of fans and the clicks of old hard drives. For a time they were a wonderful source of white noise for sleeping and I would sometimes find myself waking up if a computer I'd tasked with something overnight suddenly fell silent because it encountered a problem. I'd get up, fix the issue, and go back to sleep comforted by the sounds of my computer working away for me once more. Today our machines can be always silent while never being still, and they can spend at least as much time working for others as they do for us. Indications of activity can still be useful to me at times, but they're never comforting the way they used to be.
But as life moves on, and you get older, you start to appreciate the times as a child when your grandparents would reminisce the same things, and you would think old folks crazy as your baseline of life and technology is still being set at a fast pace, that eventually slows down. Until, you're telling tales of whirring media and 5MB hard drives the size of a washing machine and costing more than a house at the time. Which is just crazy talk to kids today.
That's nostalgia, which is always a good positive comfort go to place as you get older. I'm equally somewhat reminded of the Star Trek TNG Episode - Relics when Scotty is in the holodeck simulating his Enterprise and tells Picard how he knew the Enterprise, his Enterprise and how fast it was going and the health of the engines by the feel and vibrations in the deck plate and none of that in the future he is on Picard's Enterprise. Great episode.
I am not worried about my Fedora/Debian/Arch/whatever systems collecting personal data.
I am not worried about GrapheneOS scooping up the location information from my phone.
Home Assistant developers don't want to know when my lights and oven are turned on.
You should check the source code for every project you're using as much as possible to really ensure that there isn't anything obviously nefarious like telemetry included, because in some cases it's still there.
See Sunxi's Linux SBC backdoors on git for one of many examples of undocumented insecure or privacy-breaking code. In other cases it may be more documented, such as Golangs baked-in telemetry.
There should be better ways to check these problems. The best I have found so far is Crev https://github.com/crev-dev/crev/. It's most used implementation is Cargo-crev https://github.com/crev-dev/cargo-crev, but hopefully it will become more required to use these types of tools. Certainty and metrics about how many eyes have been on a particular script, and what expertise they have would be a huge win for software.
Something that really taught me to look for things like the "HDD Light" or Fan speed was starting my career in embedded systems. 16 bit MCUs really let you know if you are trying to much on them. They also let you know if you toggle the wrong pin by going up in flames.
The disconnect between your fingers and what actually runs the code is becoming greater and greater for newer developers. It will be interesting to see how computing power keeps up with bad code (it had been doing a good job so far).
Most apparent are the web programmers, most of them assume everyone has 10gbit fiber connections with 16 core CPUs and 128GB of RAM to feed to Chrome. And then they wonder why their shit runs like shit in the real world.
Not all developers. In my entire career, I've never had a development system that was better than the average consumer machine. And, honestly, I wouldn't want one that was, for the exact reasons you state.
Sure, you do occasionally need to do a full recompile, and if the project is very large then this can take a nontrivial amount of time, but that doesn't really happen often enough for me to be bothered about.
If the project is of really significant size, or if the compiler/language/code structure is such that incremental compilation can't give you significant gains (or isn't even possible), you may need more grunt of course.
Facebook marketplace. I don't know how you can make a grid of images max out a 5800x, but they've managed it. A markedly inferior product to every one of the classified ad competitors they've squashed out of existence.
https://en.wikipedia.org/wiki/Jamie_Zawinski#Zawinski's_Law
https://en.wikipedia.org/wiki/Jevons_paradox
https://en.wikipedia.org/wiki/Parkinson%27s_law
https://en.wikipedia.org/wiki/Induced_demand
I wonder if this phenomenon can be generalized under one law, because clearly it pops up anywhere humans are concerned.
Its funny how we all differ in respect to what kind of feedback we find acceptable from our computers. I turn off all audio notifications. I MUST have a hard disk light. I need graphs running at all times. I like to hear fans spinning up when the CPU/GPU is hot. And I never want to have my computer talk to me. ethernet jacks MUST have link and activity lights.
This is the way.
My keyboard came with RGB. I don’t care for it but I let it light up if any kind of lock is engaged and it glows a low red colour beneath the keys that I click.
That’s the only step I go beyond you. And the only reason I did it was because no lights at all for the keyboard meant cycling through every possible setting.
My computer is a machine. Its beauty is in its functionality. Being able to look at it and know what it’s doing is important.
Once every 2 years I gotta do data entry with something that can’t be OCR’d for whatever reason and I need those keys.
I regularly use the numpad, often in the context of a throwaway spreadsheet where I'm doing some napkin-math. (Or sometimes a tiny Python script, depending on what needs modeling.) Not true "data entry' in the "finish these X pages" sense, I guess.
And for istat menus specifically: it's super stable and they're extremely fast to support new OSes, unlike many fancy menubar apps. Easily worth the money for me.
Of course, a performance monitor which updates less often can avoid that issue (for instance, atop by default updates every ten seconds), but most people who run these graphical performance monitors want second-by-second updates.
Note that iStat Menus have its own set of daemons it uses for keeping tabs on system usage. It doesn’t just hook to the kernel via interfaces and display stats.
Back in the days I didn't enjoy the LED or the sound of disks. They disrupted my workflow and not always actionable. I mean, I don't really need to know that the machine is working hard on reading the optical discs I just inserted. I will take action if the reading slows down my foreground program.
It's no surprise this has resulted in removal of things like status/activity lights. The same thing happened to automobiles when they went mainstream. Now you have to get an enthusiast/sports car just to get oil pressure and voltage gauges, and even then they're likely to be nerfed to non-linearity so they're always in the same place until so far outside of normal that it's too late to prevent any negative consequences.
I'm looking at you, rental Turbo Passat coolant gauge that stayed perfectly still until overheating in Death Valley accompanied by an instantaneous jump from Normal to pinned in the red on the temperature gauge I was attentively monitoring. smh.
Another example is the original MX-5's oil pressure gauge that was so active you could use it as a proxy for engine RPM, which is normal. The next generation "improved" this with one that stayed stationary in the upper range once the engine had any pressure at all, compressing all "normal" pressures to that needle position, effectively turning it into an oil pressure switch+light in analog needle form.
I'm still waiting for a PC/Laptop manufacturer targeting specifically this market. Clicky keyboards, hardware connectivity switches, status/activity lights galore, i.e. a rebirth of the OG classic ThinkPads w/modern SoCs/displays/batteries. They'll be expensive, but so are Porsches, I can't be alone in wanting this...
One of the special infected was the Tank, which is quite strong and can throw chunks of pavement. I always knew when he was coming because my PC was kind of weak and the fans would go nuts about 30 seconds before he made an entrance. The other players were surprised that I had a sixth sense when it came to knowing when it was happening.
If someone was clever about it, you could probably even feed data back to the external display via SSH from remote boxes.
It's underneath other windows, but usually visible off on the side.
This might be a dark hole for me, I'll end up with an office dressed up like an Soviet reactor control room.
One thing that was cool though, was way back in the C64 days. If the SID chip's volume wasn't set to zero, then background sounds would come out of the speaker that were directly related to what the computer was doing. It was neat because it kind of sounded like an ethereal pipe organ. But you could "hear" basic things like complex computations vs. a short loop. I think disk drive data transfers were recognizable, but it's been a while.
And, of course, the spreadsheet I used most often at that job.
It was enough to turn off the monitor and kick back during a long modem download, and the repetitive ticking of what I presume was the UART ISR, could've been the PATA chain, would tell me it was still in progress. When the ticking stopped, the transfer was done and I'd turn the monitor back on and get back to work.
When I first started using Linux (kernel 1.2.8, I believe?) it took 8 hours to compile a kernel on my 486dx/33 with 4MB of RAM.
The endless hours of hard disk thrashing only told me that my computer was working very hard to compile lots of lines of code.
It was only years (?) later when I upgraded to 16GB that I realized this should only have been taking minutes.
Tangentially relevant but since you mention Aussie50... brings back the memories. At the end of primary school, I used to watch Aussie50 on YouTube a lot, and discovered a number of excellent engineering channels they were associated with. He was very good at explaining how various machinery worked, and YouTube videos like these played quite a role in getting me interested in tech.
Sadly there won't be any new videos anymore, but hopefully the channel stays up/someone has a backup of it.
And don't worry, I've got Aussie50's entire channel backed up, myself. I'll post it if it ever goes down.
Since the first time I got a second HDD, I was always disappointed that there wasn’t a dedicated LED per HDD. NAS cases actually have that.
As for servers, embedded systems or any other situation where a traditional monitor can't be used, cheap external LCD modules showing system stsatus can be interfaced via serial, USB or parallel ports through lcdproc.
Going back a bit further, I fondly remember my dad yelling "did you PARK the hard disk?" whenever he heard me power down the old TRS80 with "state of the art" 5 meg HD physically bigger than my first PC (which had a 40meg HD - dad was jealous). If you didn't manually issue a park command, the read/write head could flop around and cause damage if you bumped or moved the HD.
I’ve had to calculate maximum possible data transfer rates for RAM to convince devs that their query was the issue.
“This has an index, it should be fast!”
“OK, but you’re hitting 800 GB of pages…”
And if the fans on your MacBook Pro do happen to spin up when you hit ‘run’ you know you’ve made something at least O(n^3)
I've repeatedly removed and replaced the batteries on my thermostat's remote temperature sensor, because there's no visual feedback that its otherwise working, and my house was cold.
The absence of "working" lights when I know I'm doing a processor-intensive operation on a computer is similarly unsettling. Its not directly actionable feedback, certainly, but it can be a proxy for a more directly actionable, but more onerous to implement or use, feedback mechanism.
I have a 2u server sitting in a closet running a lot of things, it sits over a pool of spinning rust, one day I happened to be in the closet and hear the disks getting hammered, no alerts about any network hijinks, no login alerts or high volume traffic coming or going so I log in and it turned out to be a friend I had given a shell to because he wanted to learn linux and he was messing around with dd... His account now has an alert on login so I don't have a stroke.
Not exactly fun but did bring back memories.
My first PC - much like the one pictured in the story, had a fan that was spinning 100% of the time, because most PCs then were too primitive to have a temperature sensor that controlled fan speed. And you couldn't hear the hard drive because the fan was so noisy. So much for these performance indicators!
The only thing audible above the loud PC was when the hard drive died, and spent all its time making nasty mechanical clicking sounds. But since the computer had locked up by that point, it wasn't much of a help!
You just described my reasonably current water-cooled PC (except the disks - it only uses SSDs). I have a direct die water-cooled ryzen 7950x and 2x 3090 gpus. Normally my PC is completely silent. The massive radiator surface is fine for passive cooling until about 300W. I can tell the cpu is doing more than usual, because I can hear the water pump (normally I can't hear it - yes I need to add rubber feet, then it'll be silent even when ramping up) in addition to all the hardware monitoring I do.
Also leds are extremely important for troubleshooting. Anyone who built an am5 PC knows the pain of "memory training". Having leds on the MB to tell you what is going on is very useful.
As for the typical "developer laptop" in my current place of work (and 3 previous ones - all fortune 300 companies) developers use either Windows or Linux laptops. Only managers use macs. How is this relevant? Well, because all these laptops have fans and you can tell when they're busy.
It's a windows only application though.
Not an edit:
Found it!
Despite this, and in case I can't hear it for some reason, I have load and temperature gauges on my monitors at all times.
But hard disk noise, I completely forgot that was a thing until fairly recently something caused me to remember it. I don't miss the noise, but it was definitely a good canary or just a sign something was going on. I wonder what gauge I can add to my Linux system as a replacement for disk noise?
Not good for the project schedule, but great for better understanding the system you are working on.
The embarrassment of riches of gigahertz quad core cpus available for 20 bucks or less... So much excess labor that can sneak by.
With some approximation of AI, those scifi stories of the distributed rogue AI living on ample excess cycles gets slightly more plausible every day
LiteStep folks liked it. Personally I never bothered with it, because, well I could hear it, lol, but I do miss LiteStep dearly.
The author of the article has iStart on the menu bar, so the performance indicators are visible even when application windows are covering the desktop. The GNOME extension system-monitor-next[1], as mentioned in the article, is also similar.
Does anyone have recommendations for something similar on Windows?
[1] https://extensions.gnome.org/extension/3010/system-monitor-n...
(also I think it has a topmost option for some skin, so not only desktop)
I can't vouch for any of them because I don't use them, but just a quick search for "cpu utilization hdd utilization in the taskbar" shows a lot of options, eg https://entropy6.com/xmeters/
Or even running the default Task Manager so it would be only in the notification area and not in the taskbar: https://superuser.com/questions/666597/always-show-windows-c...
Task Manager is very limited, unfortunately. You only get CPU usage in the notification area.
If people don't use them, if they don't care about the performance, it is on them, not in the lack of noisy spinning disks and machines behaving as if they were hair driers.
Is it? Do I belong to the minority of developers that write code on their local machine?
Performance requirements should be covered by tests. Relying on the dev to notice is a form of manual testing. Nice if regressions are discovered that way, but don't rely on it.
On the flip side, I once had a mainboard where psu noise leaked onto the audio path. The little chirps were only audible when it was quiet and gave me nonintrusive feedback on the activity level of the hardware. I still miss that board.
I remember that computer wasn't like this from the get go but started exhibiting this, after an year with it, I used it like this for at least three more years and then got a new one and the old one was converted to a media center.
For years, noisy hard drives with blinking lights served to highlight the difference between thrashy Microsoft operating systems and smoothly running, performant alternatives.
It doubles as a way to quickly eject and access mounted volumes as well.
CPU usage being high doesn't really tell you anything. You may suspect that there's an issue if you are familiar with the system, and you have only one system to worry about. It's definitely not OK if you have a 'modern' architecture and a fleet of machines running a distributed system. There's a whole lot of noise and you can't tell if traffic has increased, if someone pushed inefficient code, if there's a cooling problem and CPU is throttled, if a "batch" process started, or a million of reasons.
And even if it is high, and it is an anomaly(anomaly detection on those metrics can be useful at times), is it causing any issues for your customers? CPU usage can't tell you that.
Every piece of software has some work to do. What is that work? Figure that out and monitor that. Golden signals on the relevant metrics and you'll be in a far better position. Did the error rate increase? Are we getting higher response times? In that case it doesn't matter what the CPU metric says – it could even be lower than usual if there's a bottleneck somewhere else – it is a problem that needs to be addressed. You can then use the other metrics to confirm, or try to isolate the problem. But they should be supporting information, not your main diagnostics tool.
> In a project I worked on, our development builds started writing about 80 MB of log messages per second to disk.
Well, there you go. If you just look at disk I/O you'll see that it is high. But if you track metrics from your logging system, you'll be able to immediately see _why_
Datadog (no affiliation here) has a good blog that set me on the right track: https://www.datadoghq.com/blog/monitoring-101-collecting-dat...
By all means monitor those metrics on your work laptop - very few people are going to bother running prometheus in their dev machine and setting up alerts for third party apps. But for the apps you are responsible for? Figure out, from day zero, what metrics you should collect for your app and monitor those instead.