Building a Signal Analyzer with Modern Web Tech | Hacker News Reader

Building a Signal Analyzer with Modern Web Tech

cprimozic.net

138 pointsAmeo30 comments

The overall article is extremely clearly written and well-diagrammed, as you'd expect from someone who writes software for test/metrology equipment. It does look pretty different from your usual web-dev retrospective, and shares more in common with "$BROWSER Internals" style articles.

The thesis:

> I recently spent some time building a browser-based signal analyzer (spectrogram + oscilloscope) as part of one of my projects. I ended up using some very modern browser APIs and technologies that I'd not worked with before, and I discovered a lot of really interesting patterns and techniques that I'd never seen before in a web app.

> Building this application has made it clear to me that the modern web is extremely well-suited to building complex multi-threaded and graphics-intensive applications.

Demo doesn't work on FF in Windows 10, not clear that it's meant to run anywhere other than on Chrome or Safari.

So I'm not really sure the author's thesis has been validated, as interesting and useful as the concept appears to be.

Firefox is lagging behind on a lot of the more esoteric web APIs. I think it's a choice by the Firefox team, and a valid one given the effort required, Mozilla's relatively limited resources, and how few devs actually want or need these APIs. Firefox is more like a "website viewer" than a "web app host" like Chrome and Safari. I think that's what a lot of Firefox users want, and why they choose that browser. It might even be useful to differentiate between them instead of calling everything a 'web browser' in the future.

To that end I don't think it's a fair criticism of web app developers or Firefox that apps like this don't work in your browser.

The specs that firefox chooses not to implement are stuff like web midi for security concerns. Firefox does not try to market itself as a "website viewer" that does not support "web apps"

You must have missed the announcement. Firefox now supports web midi, their security concerns where BS all along anyway.

I can't speak on the specific details of web midi, but after the spate of poorly designed browser features with intrinsic security vulnerabilities (eg canvas fingerprinting), it's reasonable to be wary of things coming down the pipeline.

That always makes good sense. I think Chrome got this one right though, have a reasonable protection against abuse and trust that the end user is capable of weighing the risks.

Really, there isn't an issue around fingerprinting? https://twitter.com/denschub/status/1582730985778556931

The issue around fingerprinting is secondary because it isn't a security issue but a privacy issue, and this is a problem anyway for which there isn't a really good technical solution.

The security issues that were mentioned (someone bricking synths on purpose via SysEx messages) has afaik never actually happened.

This works fine for me (Firefox, Ubuntu, tracking protection set to max, uBlock origin on, privacy badger on, resistFingerprinting on). You need to click the play button to make it go, of course, but then it just works.

Maybe there's a setting you've disabled previously and have since forgotten about?

Hmm it should be working in Firefox - as long as you're using the latest version instead of ESR it should be good; works for me on Firefox 113.0 on Linux.

Some browsers have anti-autoplay-ad protections built in that prevent audio from playing until you interact with the page. You might have to click the page or press a key on the keyboard to get it started.

Works great here (Firefox 113.0.1 on Windows)

"$BROWSER Internals style articles" sounds like something I would crave for. Do you have any examples with actual "$BROWSER Internals"?

Show HN: I've built a spectrogram analyzer web app

https://news.ycombinator.com/item?id=35781959 (19 days ago)

https://webfft.net/dft/

FYI, it supports decoding ultrasound wav files. I use a Dodotronic UM250K mic to capture ultrasound up to 125 kHz. LCD monitor and other electronics emits some interesting sounds. For some reason, getUserMedia doesn't allow high sample rates, but ffmpeg works:

ffmpeg -f alsa -channels 1 -sample_rate 250000 -i hw:CARD=r4,DEV=0 -t 5 mic.wav

That Webfft is fun. Try recording yourself making a few long vowel sounds A I U E O AE UH , and take a look at how the harmonics/formants/ the pattern of peeks at different frequencies is distributed between sounds.

Being able to run an FFT on audio signal in real time in a background thread seems really cool. Not sure how feasible it is but could be the basis for speech recognition engine running in browser (I'm not expert but I know Speech detection machine learning models are trained on range of FFT derived signals) - if you can pipe the FFT output to ML endpoint you could potentially have speech to text generation running in the browser - or something like automatic closed caption generation from an incoming audio signal.

Definitely possible to run high-quality speech-to-text in realtime in the browser: https://whisper.ggerganov.com/

Made by the same guy who created the popular llama.cpp LLM library. The model uses log-mel spectrograms as input.

Using modern algorithms, FFT is actually really fast to compute. Definitely dwarfed by the evaluation of the model itself, even when using many threads and Wasm SIMD.

Isn't that already offered out of the box via WebAudio's `AnalyserNode` ? https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNod...

Yep looks like it - wasn't expecting browsers to have it baked in. I'm not all that familiar with what is available in the world of audio.

I've done some rudimentary signal analysis on other types of sensors - accelerometer (for detecting vibrations of specific frequency) and millivolt probes on a hardware instrument - I've used stuff like Matlab or Python to do the signal analysis.

I think the biggest problem with SharedArrayBuffer is that you need to set the Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy headers for it to work at all (due to Spectre), which means you need a host or a web server where you can configure that.

Very cool project and great write-up, nice work OP.

This doesn't seem like something you should be able to do with a web browser.

Very dope. Glad I saw this. Great work.

People were running full 3D engines in the browser 10 years ago (the original asm.js).

3D in browsers was even in the mid 90s possible. But the hardware was not ready, so VRML never really became a thing.

https://www.howtogeek.com/778554/remembering-vrml-the-metave...

You really need a realtime-OS for this if you want to prevent buffer underruns from happening. Otherwise, it is certainly not audiophile-safe.

I mean… Pro Tools, Reaper, Ableton, Logic, Cubase… these all run just fine without an RTOS. But you did say “audiophile” specifically and not “audio professional” so you might be right, but it likely doesn’t matter in practice.

I guess I'll stop dealing with exceptions in my code because they don't happen in practice anyway.

Does your audio input device run an RTOS? Probably. Audio output device? Also likely. Does the intermediate processing need to be an RTOS? Or does it just need to be able to operate fast enough to consume the input buffers and keep the output buffers from starving?

> Building a Signal Analyzer with Modern Web Tech

No info about HW ? Does he uses the soundcard ? Then it is nothing new.

Reply on news.ycombinator.com