Hacker News Reader

Hi, I work at Mozilla doing audio programming for Firefox, I see in your about page that you have issues with the Web Audio API, ping me (same nick a bit everywhere on the web, and also @mozilla.com) and we'll answer any question or fix any issue you have. Lots of things have happened in the audio space on the web since 2018!

In particular, low latency (like native roundtrip latencies, so <10ms easy, but depending on the OS) no-jitter/real-real-time audio programming is now something that developers can do. Lock-free/wait-free programming and SIMD are coming in the next weeks/months.

Very cool project in any case, I'll use it when I need to quickly do very high zoom on wave forms to debug things for Firefox.

When can have we web midi (and no not sysex ;)

Sooooo?

Seconded, web midi is a big showstopper for Mozilla and should have more priority than the various bells and whistles that Mozilla does seem to prioritize but that ultimately are not part of a browser.

Not being able to use Midi from FireFox means that for a whole raft of possible applications Chrome is the only option, which is a real pity.

Please, please, pretty please, give web midi a higher priority.

Curious, what applications need web midi?

Just one example:

https://sightreading.training/

wow that's a pretty cool app

Most music-related applications wanting to offer generic hardware support and hackability. Midi is an amazing standard. It allows you to hook up a synthesizer from the early 80s to your 2020 laptop (via a midi-capable audio interface) and control almost anything in your modern DAW (Digital Audio Workstation). And if your Web browser has support for it, control a software synth or any application as well.

In addition to just musical instruments, a huge number of different hardware still use midi. I'm personally interested in DJ-world where things like DJ- and light-controllers use midi and therefore are very easily remappable and very hackable (if the sw is built correctly).

I've wanted to build some DJ-related sw myself that would work via browser but the fact that currently only Chrome supports the standard has so far kept me off it. I'd very much like to see Firefox supporting Web midi.

Right, I understand what MIDI is for, but usually music software has these real-time requirements on performance and the browser runs javascript, so... What are the applications for web midi? People aren't mixing their music on the browser nowadays, right?

You could have a whole DAW in the browser. There are already some web based synths

I haven't seen significant problems from javascript being too slow on things that use web MIDI. Here is a web based "piano karaoke" app that runs quite well in browsers that support MIDI (and since it syncs with YouTube videos, it really kind of has to run in a browser vs native)

https://www.youtube.com/watch?v=NV6rdmdZnkA

Get in touch if you want to beta test it and let me know if you think it is fast enough on your set up, etc. Hoping to put it on "Show HN" soon. rjbrown at gmail More about it at https://pianop.ly/portfolio/

Just to throw out an example from a friend's company: online piano lessons. MIDI input in the browser makes it possible for them to tell if you played the correct notes on your keyboard.

If it was WebMIDI that actually transmitted -MIDI streams- (instead of just shoving them, with limited options, at some flaccid GM soundfile) that'd be really cool. Then you could choose your own MIDI playback engine (application, external hardware, ... ).

In an environment that interprets -all- the codes, MIDI's almost unlimited. Hollywood's used it for a long time. It's very undemanding so people could -certainly- mix their music on the (right) browser. And, if people could add javascript routines to it? Ay-yay.

Mine, for one.

https://www.youtube.com/watch?v=NV6rdmdZnkA

(piano karaoke that overlays and syncs with youtube videos)

The use of the phrase "bells and whistles" seems pretty ironic here.

When you think of the web as a platform to be used for audio applications, it seems like a reasonable feature

The thing is that "the web" doesn't make sense as a platform for audio applications when to build upon it you need to throw away realtime safe subroutines.

Realtime audio rendering is a soft realtime problem. If the web audio APIs don't have methods for guaranteeing deadlines for rendering audio, it's only possible to build toy audio applications.

I'm aware of things like bandlab and such. They're still toys and too limited for serious work, where the money is.

you can actually use it today - there is a plugin for it which adds Web MIDI API to Firefox - https://jazz-soft.net/download/web-midi/

I have tried some Web MIDI demos (and online CSound IDE) this way.

Will we get jack support back on Linux one day ? :/

The mess with audio servers is one of the main reasons I could never stand Linux as my primary laptop OS. Jack, ALSA, OSS, Pulse, etc. Far too often the answer to my problem was that it didn't work on that specific audio server, or that I needed to compile my own server/kernel/app from source.

Easily, reliably, and quickly playing audio is table stakes for Windows and Mac OS. They've been doing it for years. But instead of improving one or two audio APIs, on Linux it is time for a new audio server that will, but doesn't, solve all of the problems of every predecessor.

> Easily, reliably, and quickly playing audio is table stakes for Windows and Mac OS. They've been doing it for years. But instead of improving one or two audio APIs, on Linux it is time for a new audio server that will, but doesn't, solve all of the problems of every predecessor.

I mean, I have currently 5 soundcards on my desktop computer, 1 pro over PCIe, 2 pro over USB, 1 standard HD Audio and 1 output in my screen's HDMI, and I can assure you that even on windows it's definitely not a smooth ride between WASAPI, ASIO, MME, WDMKS... things crash or get stuck routinely, and I sometimes get random loud buzzes or no sound until I reboot. With my band we had tons of issues with a M-Audio card on windows fixating itself on the wrong sampling rate as soon as a desktop music playing was launched (this includes web browsers), entirely preventing playback on Ableton Live for instance.

It's barely better on macOS, e.g. look at that shit: https://github.com/OSSIA/score/issues/778. macOS also sacrifices some low-latency when comparing the same hardware on it versus Linux (with raw ALSA or JACK) and Windows with ASIO.

What exists as far as open source digital audio interfaces? Meaning only digital i/o — no analog.

It seems like a relatively contained problem to take multiple streams of PCM audio into a computer, locked to word clock. Then the hard, fiddly aspects of analog design can be left to high-quality outboard converters which speak MADI or whatever.

I say "contained" because I don't want to minimize the difficulty of getting such a design right, but it seems like you only have to get it right once.

Is USB not sufficient?

What I'm looking for is a way to hook up any converter which outputs via a standard digital audio protocol (SPIDF, AES/EBU, ADAT lightpipe, etc.).

Then we don't need a USB driver for each converter! We only need one USB driver, for the digital i/o interface. And we can polish the driver for that one interface until it's actually reliable, instead of relying on the sketchy one-off driver for this year's soon-to-be-obsolete USB audio interface.

ETA: Many such converters (most of them high-end) listed here: https://www.sweetwater.com/c796--AD_DA_Converters

> SPIDF, AES/EBU, ADAT lightpipe, etc.

If I'm not mistaken the underlying transport protocol is the same for all of those - you can get a coaxial to XLR to get S/PDIF into an AES port or a coaxial to TOSLink converter to get S/PDIF data into an ADAT port.

You can look into the Madiface XT maybe ? https://www.rme-audio.de/hdspe-madi-fx.html or the other RME products which have enough digital I/O to cover a lot of needs

> If I'm not mistaken the underlying transport protocol

Some quick usage of Wikipedia suggest that you are, sadly, mistaken. ADAT lightpipe seems to use a completely different protocol (more capable?), and while S/PDIF and AES/EBU use quite similar protocols, they differ in impedance and max and min voltages, so I'm guessing that directly electrically connecting the two controllers is a bad idea for both the controller and the data.

It doesn't matter for the purposes I'm investigating. So long as we can get the bits out of the wire and into open source software, we can make sense of them.

(Licensing could be a concern though if ADAT lightpipe is proprietary — I'm not clear on that. But then we could just use other open protocols instead.)

In terms of i/o capability, the specs of the Madiface XT would work well for the purposes I'm proposing.

But the point is that I want open source drivers, and open source hardware! I don't want to depend on the health and the priorities of a commercial entity. I want to be able to inspect the driver software and contribute towards perfecting it!

Compared to, say, GPUs, the needs of multichannel digital audio i/o are modest and not changing very much over time.

unrelated, but what do you use 5 sound cards for?

I only have a "main" soundcard (the PCIe one) - HD audio and HDMI thing are unused 99% of the time, but if you have a computer more recent than 2007 you have them. The two USB soundcards are temporary and here currently because I need to test stuff for the sequencer I'm developing (https://ossia.io)

I think the guy with 5 sound cards might be the exception that proves the rule - the conversation was about playing back audio from the browser not which OS starts to fall apart when you need a DAW for your band.

> the conversation was about playing back audio from the browser

well, no, the web audio API & stuff is explicitly being sold as something that will be equal in capability to other solutions, so definitely things used for pro audio.

Even quite modest normal setups will have several input or output devices. Just on my laptop I have hdmi audio, built in speaker and mic, USB headset, bluetooth headset and remote audio from an RDP session.

My Chromium plays audio to my external and internal cards fine without Pulseaudio, I have just ALSA and don't know who still uses OSS (no judgement, I just use the default). I think your comment can only have an adverse effect on the conversation if you don't give specific examples.

(Nitpick: Linux has nothing to do with Pulseaudio, it's just the kernel.)

I've had many issues with OS X audio on Catalina - all of my USB audio devices skip and pop on it. It's also a pain to configure, with options bizarrely hidden in the MIDI app.

Windows probably has the best audio stack of them all (battle tested, easy to use UX). In comparison, JACK/pulse are pretty good as well and people use it professionally. Like most things on Linux, the initial UX is a bit of a pain, but once you get past it you get amazing customization, even more so than Windows.

These are the only abstractions I'm aware of on Linux systems - and they are simply built on top of ALSA which is the kernel's core sound API. Almost all desktop environments simply install PulseAudio, and you never have to worry about sound.

How old is your Mac? That issue sounds very strange to me.

CoreAudio is far and away the most impressive and powerful audio API across the board. It's ridiculous how much you can do with it and the abstractions it provides.

It's also been better, faster, and supported lower latency than Windows up until WASAPI came around - you needed to use ASIO on Windows for years. Even today you can't programmatically change the sample rate of a device on Windows, which means that engines like RtAudio have to do resampling under the hood to support basic functionality that's been in CoreAudio.

The only problem with it is that the documentation has evaporated. Which admittedly is a big one.

> How old is your Mac? That issue sounds very strange to me.

New MBP 16". Apparently it's a problem with the T2 security chip and affects almost all external USB 2.0/3.0 audio devices. Just do a google search for audio drop outs and T2. All digital audio goes through the T2 chip which seems to add latency and these drop outs. I think the only solution at the moment is to use a USB-C native device (but I haven't tested this). If you already have substantial hardware like I do, Catalina is basically unusable (and no fix in sight from Apple...).

fwiw I don't think any audio manufacturer (hardware or software) has advised it "ok" to use Catalina, and OS upgrades have been a problem for as long as I've used professional audio tools.

I understand with brand new machines it's a problem, but buying less-than battle tested hardware/software has always been problematic. If you already have substantial hardware, you probably shouldn't be buying a brand new machine and expecting it to just work after a major OS upgrade.

I shouldn't expect a widely used standard (USB Audio) to work after upgrading my hardware? All the Windows and Linux machines I've had have no problem with USB audio and I don't expect they ever will.

I mean I can't even get (all) my USB audio gear to work properly on Linux, at all. For example system settings on Ubuntu Studio misinterpret anything with more than 2 channels as a surround device. But that's OK because the hardware manufacturer has advised they don't support Linux for their proprietary device management software because "widely used standard" like class compliant USB audio is not sufficient for their use case.

And I've been burned enough times by OS upgrades that I know to be cautious. I'm not saying the situation is good, just that it's a shitshow everywhere. I got around a dozen emails from my device and software vendors telling me not to use Catalina, so I'm not yearning to go out and buy a new Mac to run my software on and connect my hardware to it. I'll shit on Macs for that, but under the hood, CoreAudio is much more impressive than anything on Windows or Linux.

After a bit of struggle I found that Pulseaudio over Jack adresses my needs. Music production software connects directly to Jack and all other software (browser, video player, etc.) uses Pulseaudio. Also the Cadence GUI really helps to set this up correctly.

It's already enabled and lots of people use it, although it's not compiled in on the mozilla-provided builds (but downstream builders enable it).

What really? How do I get it?

Are you open to a couple of feature suggestions from someone who uses Audacity for scientific applications?

1. An port of Audacity's "Noise Reduction" filter. I have done this once [0] (with some difficulty and no doubt with errors) and I am so tempted to just translate it to JavaScript and put up a PR, but I'm slammed right now.

2. It would be very useful to be able to view the spectrograph as an alternative to the waveform, rather than having it live in a separate "spectrum analyzer" pane. Especially if zooming along the frequency axis were implemented, this would make it much more useful for, well, spectrum analysis.

Thank you for making this. It's impressive in its own right but doubly so as a web application.

[0] https://github.com/robin-labs/robin/blob/master/noisereduce/...

Those are both great ideas, especially the noise reduction function might be something that makes this tool quite useful for quick recordings! I actually tried implementing noise reduction algorithms in the past, but sadly could not get it to work properly. Maybe time to revisit! (ditto about the spectograms too)

Ooh, I actually was looking for a js noise reduction library recently. I might just translate it and put it up myself. :)

The next plans are - redo drawing library to further improve performance!

I'm amazed that you still have ideas regarding improvements here, because at least the selection performance is buttery-smooth even on a phone.

Great job, very useful product.

Soundforge on a 486 100Mhz was already pretty fast at editing a single track of audio. It's amazing how much heavy frameworks have warped our perception of normal interactivity.

I had a 486 and did some rudimentary audio editing back then.

I remember the performance to be decent, but not 60fps-decent.

But aside from that phones usually have relatively infrequent touch updates, which usually results in a single repaint on every such event, so 10FPS or so.

And took more time and expertise to develop and only worked on one computing platform. The software industry has been a cycle of trading hardware performance improvements for developer productivity since the beginning.

Very cool.

Any particular reason you're not using standard keys for cut/copy/paste/select all etc...?

I did not want to bind control/command key as it is used by the browser itself (eg control + t opens a new tab). So instead I used the shift key, (so shift + A = select all, shift + C = copy, shift + V = paste, etc).

interesting choice to be different than all other webpages and user experience for no good reason.

It is a non-optimal choice, but I believe there is a very strong reason. Not competing with the browser's native functionality.

Some of the most horrible web experiences, occur because websites try to take control of native functions. For example "smooth" scrolling, or highjacking the back button, or trying to abuse the clipboard! Confusing the user, breaking navigation, making the page less accessible and in some cases breaking after X months when browser or OS behavior changes.

What I have now is unfortunately far from "good" and there is improvement and experimentation waiting to happen, but the straight forward approach is not much better either...

What would you be competing with? Cmd/Ctrl-C/X/V for cut copy and paste are the norm. By allowing your app to use those keys you aren't overriding the browser, you're making your app conform to the standards. Even native apps do this. The OS may provide cut/copy/paste in an OS level text widget but it doesn't provide it in Word, Excel, Photoshop, or Audacity. That's up to the app. In the same way doing the same thing is up to the web app for anything not plain text.

Most web major apps do this. Office365, Google Docs/Sheets/Slides. Even Facebook/Slack/Gmail since they re-interpret the text (converting :-) into emoji and or replace with images.

Fair enough :) for cut/paste/copy it might definitely be a good option. Though what about control + S, should that prompt the export audio, or let the browser download the page?

Perhaps I should add support for the control key, but let other combinations were conflicts might arise. Getting good UX is a such fascinating topic.

Please, specify license of your code, e.g. GPL3.

Multi-track! That would be awesome! Take you time tho, keep the pleasure of working on it and don't say yes to all feature requests.

Also, looking at the page load and see less than 80Kb transferred: this is absolutely beautiful! Amazing work!

This is one of the coolest web apps I've ever seen. How it handles multiple windows, docking, local drafts. Wow, well done!

Really hoping you add in spectrum editing, too, because waveforms are fine, but the power of RX and tools like it is that you can view the FFT spectrum and cull parts that are entirely useless (cutting out hums, >12k painful harmonics, etc =)

What's RX?

Reactive programming, possibly in js

Not in this context.

"RX 7, The industry standard for audio repair": https://www.izotope.com/en/products/rx.html

No, not even remotely. Izotope RX: https://www.izotope.com/en/products/rx.html

A tool used by anyone and everyone in the audio space: https://www.izotope.com/en/products/rx.html - it's most important feature is that while you can work on the waveform, that's pretty much the least useful way to work with audio, and so it also lets you work directly on the spectrogram of an audio source instead, which makes editing the actual sounds you hear much, much easier.

This is great, thanks. Is there a way to access the menu via a key shortcut?

Right now no, but it would be very useful so i 'd like to add this, since you can use the keyboard to control the menu... (except well... opening it!). Any preferred key you 'd like this assigned to?

Great! No strong thoughts on a specific key. Maybe Shift-F, or Shift-F10? Or Esc-F, since another comment suggested Esc as a general prefix key?

You should make your modal/dialogs close when the escape key is pressed.

politely asking for more formats too, it won't load my flacs

presumably it loads whatever the browser can natively load. I don't think flac is one of those?

Chrome can load FLAC.

Fair point, looks like FF also supports it.

Well done, and thanks for making it available so easily.

Slick UI!

Congrats!

No keyboard cut, copy and paste?

The keyboard shortcuts are signalled in the menu; they are Shift-C and Shift-V