Computers should expose their internal workings as a 6th sense

The funniest demonstration that I watched was at the computer museum at the University of Stuttgart (it's just a single room, but it contains a lot of history!). The guide took an old, butchered radio that was reduced to a coil attached to a speaker and put on top of the front panel of a PDP-8. Then he started a Fortran compiler, which would take several seconds to complete. During that time, the radio made kind if hideous digital beeping noises from the CPU's EMV radiation that got picked up by the coil inside. You could easily learn to distinguish different compiler phases and tell whether the program made progress. The guide explained that this was a common way for operators back in the day to keep track of the jobs they were running while taking care of other tasks: were they still running? Did they get stuck? Did the job complete and is it time for the next one? Some inventive guys figured out that when you wrote certain instruction sequences, the EMV noise would become tonal and the pitch could even be tuned to some extent. That got them to write programs which would compute nonsense, but when you picked up the EMV emissions, you would hear music! The museum guide ran a few of these programs to our great amusement :).

I've yet to see this mentioned - or demonstrated - anywhere else.

In the late 90's, the ops mgr at our studio hooked a network cable up to a little motor and attached a 5-6 foot long string to the motor shaft. He hung the motor-string thing in the corner of his office with the string dangling.

The motor made just a little bit of noise and the string would wiggle around indicating network activity. Soon he was able to know what was typical string movement and what was atypical frenetic motion that indicated a need to investigate.

He called it an "ambient interface" and said he had read about it somewhere.

In the mid 90s, I worked with magneto-optical disk systems. The noises they made helped me (and others) diagnose their problems.

This type of "sixth sense" is also not limited to computers. When I worked in the aerospace industry, I heard a story about McDonnell Douglas replacing the F-15 cockpit fairing with a sleeker, fewer-piece version that reduced drag. Pilots found that without the noise from airflow over the metal joints, they didn't have as good a feel for speed and maneuvers.

One fun example is the "malloc Geiger counter", which clicks whenever memory is allocated: https://www.youtube.com/watch?v=7vn6aGgLKfQ

(Previous HN discussion: https://news.ycombinator.com/item?id=24303832)

I miss being able to know if my program is frozen because of CPU (fan noises), disk access (HDD noises), or network (silence).

I've thought of "reimplementing" that for troubleshooting my own projects. Imagine `clang --sounds`, so that it makes a sounds when

- Accessing disk.

- Sending something on the network.

- Allocating large buffers.

- Waiting for locks.

Or, alternatively, there's a constant background sound that is modulated on every function call depending on

- Call stack depth.

- If it's my code, library, or syscall.

- How long the last call took.

- Or just a unique modulation for each major function.

I think that's a nice, easy, and useful step before the more advanced applications mentioned in TFA.

This alludes to a concept I've been wrestling with: "hard" versus "soft" understanding.

A professional in _any_ industry will pick up a notable difference between extremely similar states. This article is expressing the fact that older computers were easier to read.

I'm convinced it's a product of sophistication. Distributed systems used to be an enterprise thing, but now everything is technically a distributed system. Memory registers are quantum scales more than they used to be. Drives now have no moving parts.

There are still ways to get an intuitive understanding, but they're...different, and certainly not audible. I've noticed that I can feel out I/O speeds when I'm power-using. I'm fairly convinced that many knowledge workers prefer a specific OS because this intuition pulling up false-positives in a new environment.

This is an important but seemingly understudied subsection of my field (human-robot interaction). From some well-cited work:

"Our goal is to enable robots to express their incapability, and to do so in a way that communicates both what they are trying to accomplish and why they are unable to accomplish it... Our user study supports that our approach automatically generates motions expressing incapability that communicate both what and why to end-users, and improve their overall perception of the robot and willingness to collaborate with it in the future."

I'm not as plugged into human-computer interaction work, but as a user, it seems like this is sorely missing and getting worse. I wish I could get a happy medium somewhere between a full stack trace and silent failure, e.g. when my iCloud documents won't sync.

[1] https://dl.acm.org/doi/abs/10.1145/3171221.3171276

I’m reminded of a programmer who would add click on the PC speaker inside function calls so he could listen to the timing of them being called.

As computing environments mature, a lot of the instrumentation that used to be presented to users has been going away. I do think that's too bad. One of the first things I install on any new machine is a visible network/CPU/memory monitor.

Humans constantly monitor their environment, and tend to only become conscious of something when it is behaving differently than it had been. Think of long drives - unexpected motions, odd noises, etc. trigger conscious attention.

Computers, especially of the 'cloud' variety, lack the incidental physical environmental interactions that give us those, so intentionally building them in is required. (And because they're intentional and artificial, they're at risk of manipulation, something else to worry about.)

A previous PC I built had faulty grounding with the headphone jack and would constantly play static mixed in with whatever intentional analog signal was being played. When I played single player games certain patterns signaled a big boss fight was about to happen.

Computers should be noisier is an interesting take.

I can't tell you how much of a relief it was to have finally have a computer with solid state everything, an insulated case, 120mm fans. Turned it on, silence. It's good for recording sound, it's good for anyone else who happens to be in the room.

Why sound? why not just a.. oh idk. disk access light? Maybe add other little leds for other stuff?

One of my ideas is for application windows to have a button to flip to the internals of the application - see threads, connections, progress bars, concurrent loops, memory allocations, even stacks. This is what I call an encyclopedic desktop.

In sixth-sense terms, I once heard of experiments with a belt which would vibrate to indicate magnetic north. Apparently people really missed it when they had to give it up.

There's nullsoft's beep: http://www.1014.org/code/nullsoft/nbeep/

Feels like the culture series "aura" for droids could fit here. Subtle colours indicating mood in the books by Iain M Banks, but could indicate system health.

I never tried it, but ~20 years ago there was a project called Peep for auditory monitoring of networks.

From the Usenix (https://www.usenix.org/legacy/publications/library/proceedin...) abstract:

> We created a network monitoring system, Peep, that replaces visual monitoring with a sonic `ecology' of natural sounds, where each kind of sound represents a specific kind of network event. This system combines network state information from multiple data sources, by mixing audio signals into a single audio stream in real time. Using Peep, one can easily detect common network problems such as high load, excessive traffic, and email spam, by comparing sounds being played with those of a normally functioning network.

The SourceForge page is still up: https://sourceforge.net/projects/peep/

I do really like the concept of this and keep a full bar of monitors going in my top panel on desktop. I've also done things like tailing the ssh apache logs to watch real time traffic. I've also setup a streaming audio server that played sounds on various events like someone visiting a website or auctions, triggering the stairway camera, making a sale and tracking updates.

I also thought about, but never implemented some sort of lighted display, possibly a dollar sign that slowly turned from red to green as an indicator of the days profit from sales on my e-commerce channels.

I'm just missing the small led that used to indicate disk activity until someone decided laptops looked sleeker and less cluttered without it.

This makes a lot of sense to me.

MS Windows could really use a "WHAT THE HELL ARE YOU THINKING ABOUT?" feature. From the first version of Windows with networking, a Windows install had a lifespan of a few years, after which simple things like clicking on the Start menu would take several seconds to respond. You expect this when connecting to an external disk share, but it was woven into the OS so that it happened at weird times.

I didn't know enough about Windows internals at the time to figure out why this was happening. After a few days/weeks/months of irritation, I usually ended up doing a fresh install and re-installing/configuring all of my apps.

Just in the last month or so, my Windows 10 development system will sometimes take several minutes to pop up a File Explorer window. Default File Open/Save dialogs are affected as well. I'm not using any shared drives and I disabled the stupid One Drive thing. At this point, a reboot resolves it, but I sense another reinstall in my future...

In my old office I used to be able to tell if any of the machines in the lab were having issues just by walking in; the machines were old enough that if there was any issue (too much load, hard drive failing,etc) the difference in background noise was noticeable. My new office is unfortunately too loud by default and everything is too new for me to do it anymore.

Similar issue with the newer laptops that include power buttons as part of the keyboard. The power button no longer has a noticeable change when pressed, and if the laptop is fanless or just very quiet, it can be impossible to tell if the damn thing is turned on or not.

Sometimes I wonder if the manufacturers are doing this on purpose, to make the customer feel more distant from their devices and willing to part with them easier.

Maybe a smartwatch with more detailed tactile feedback could be a conduit for this communication. There are some truly tiny 5v mechanical relays that could be turned into little fingers for touching/vibrating specific spots. Bluetooth would be convenient too.

Dell got you covered: get a pleseant humming noise when your GPU is busy: https://m.youtube.com/watch?v=ATxR9FyBrVw

Modern computers and peripherals tend to come with a lot of RGB lighting that's mostly there to boost performance, but could ideally be used to expose this kind of background information.

I remember having this thought while watching TNG - if the computer was having trouble processing a "recursive algorithm" you'd know about it from a series of hesitant chirrups.

I've thought about making an app that lets you know what's happening in the background with a black bar that has jets of colour move across it when something happens, with different speed, colour, and shapes based on various attributes, like which app the activity came from, and what type of activity it was.

I like this idea at first pass! I think augmenting reality with a tap into the digital could be a nice new dimension to how we look at the world.

I do however think that giving tech companies + ad companies the ability to tap into this is risky. As a result, rather than have a call-to-arms to invest more in such tech, maybe it's better to just start the conversation on how we can interact with the digital world safely.

...Or maybe which digital worlds are safe to connect to? :thinking_face:

As field support engineer for a large company I can relate to what the author is exposing... I work with LED screen installations and most of the times I can detect a bad installation issue way ahead of any failure because this screens have some High Voltage capacitors that may produce sound if there is something wrong with the power quality, once I detected harmonics way ahead of the initial test, because of that we saved like 2 o 3 days of installation

Linux has a very flexible LED framework, and there's a variety of different "triggers" one can use to drive the LEDs. CPU usage, disk usage, network device usage, backlight level, usb port usage, audio are all different triggers one can control an LED via, that are baked into the kernel[1]. There's userland devices one can use to do whatever else one wants.

This isn't as aesthetic as say, the LiveWire[2] mention in the comments. But it's readily available on almost all systems, and is a very flexible ambient indicator.

There's a lot of really really fun good stuff in the comments here. Ambient is good, but to me, I want computing that exposes the causal relationships of what is happening as it's processing, as it's running. "This button was clicked so I'm trying to change the screen brightness now." All of the entities of computing, the data, these user events, should be reified, should be made into a logged sequence of what is happening. From that basis, we can all be free to explore computing, and to- EventSourcing style- extend the graph of computing as we might see fit.

[1] https://github.com/torvalds/linux/tree/master/drivers/leds/t...

[2] https://news.ycombinator.com/item?id=28348148

I second, we have improve in many aspects of the UI, others like visibility, no so much. Take a look a my experiments exploring ways to sense the virtual environment

https://mymakerspace.substack.com/p/another-look-at-infrastr...

My next motherboard will have 8 rgb LEDs on the bottom (vs 6 on old gen). I'd like if these could be colored per CPU core load. Say a dim blue for idle ramping up to bright red for full load.

Which reminds me I need to update the case for 8 lights now that the 5700G is available and worth doing an upgrade to the Mellori_itx.

I recall a classic Mac OS hard drive indicator icon to the left of the Apple menu that would roughly correspond to the crunching of the physical disk.

I had no idea what this article was going to be about, but determining behavior based on physical characteristics of the hardware is something I miss with modern, quiet machines.

i think if we combine modern monitoring and metrics systems for cloud computing with sound synthesis we could get valuable intimate insights that offer something dashboards or anomaly detection and alerting systems can not reach. if we listen to our companies systems noises, sounds and clicks thorughout the days and weeks we would naturally learn to read them and get that connection we had to our 90s workstations. the problem is its easy to make something like a novelty show room in eg. a facebook foyer, but to get the perfect balance of not being annoying or drawing too much attention from work and still representing enough real world data requires the best sound designers we have. ideally the result would be something that the kind of devs that like to listen to white noise and rain or ocean recordings at work would enjoy.

As much as I enjoy quiet computing (finally!), I have to admit that all these noises (hard disk access, fans spinning up, etc.) provided important feedback. I could think of a thin, rather dimm light bar with various sections for indicating processor load, bus and IO and network activity, etc.

Back in the days of 8 bit microcomputers running in the 1-2 MHz range, this was possible through EM interference that you could hear by putting a radio nearby.

I could tell whether thee machine was crashed, and a lot about what part of a program was running by the texture and tone of the noise.

In the past, the clicking of hard disk drives has told me a lot of things: An unexpected I/O intensive application running, lots of random seeks, or even failures like bad sectors. Now with SSDs, that aural indication is gone and the only noticeable effect is system lag.

When I'm compiling a big project, I'll often go read some emails or whatever and listen for the fans on my PC to quiet down again to know when it's done. It's actually become a pretty important part of my workflow lol.

I'm not sure this would really work. I mean, remember the modem sounds from the 80s? Could you really tell what was going on based on the sound alone other than the difference between connecting/transmitting/disconnecting?

yes 1000%. I remember my first impression of automatic updates was to feel profoundly out of control; software could now change itself without anyone even knowing. The transition to broadband was cool as I could pirate enough music to fill up my hard drive, but also profoundly unsafe: other things could come down that pipe.

Making the invisible processes of change and exfiltration legible to humans, even only as background noise, would humanize these awful OSes in a big way.

See also: https://justine.lol/blinkenlights/

haha, back in the day my dad used a transistor radio to understand what his mid 70s microcomputer was doing

There is still the occasional time delay - when you say something isn't right.

Well there is always dtrace, strace and ptrace.

This is how you know you need a vacation from computers - when you write an article literally fetishizing sounds they once made. The whole piece is symptomatic of a fried brain. To the author: go outside for a long walk and shut your devices off.

I know this feeling. I remember playing Minecraft in a Windows 10 virtual machine, and the fan was roaring. Windows 10 is bad enough, but throw in a resource hogging game & a VM and you're asking for trouble. The game was so laggy as to be unplayable.

No!

Computers should have normal auditable log files, board schemes and spare parts. If it works right, it should be invisible. If it stops working, I call plumber who will fix it very cheaply.

This "computers are magic" is just BS. I refuse to "interact" with my thermostat. Soon this 6th sense will feed ads into my subconsciousness.