Triforce – a beamformer for Apple Silicon laptops
github.com
github.com
The use case was for lectures, you could tell the laptop to just record from behind it, pointing the beam in the direction of the professor.
Amazing idea and something I haven't seen since.
Another great idea.
Oh. They still make similar stuff: https://electronics.sony.com/imaging/imaging-accessories/all...
https://devstreaming-cdn.apple.com/videos/wwdc/2019/249a0jw9...
https://www.sennheiser.com/en-us/catalog/products/meeting-an...
The mic array for the room figures out who's talking and isolates the audio from them.
(Videoconferencing in large rooms has long picked the loudest microphone to use at any time, to avoid mixing in noise from other mics, but then the beamforming makes it that much better.)
Use a microphone array and LIDAR for ground truth, and train a diffusion model to "imagine" what the world looks like conditioned on some signal transformations of the microphone data only.
Could be used by autonomous vehicles to "see" pedestrians through bushes, early detect oncoming emergency vehicles, hear bicyclists before they are visible, and lots of other good things.
I like it. I think you'd need to be in known motion around the area to build up a picture -- I don't think it would work with a microphone just sitting in place.
If you shut your eyes off and you hear footsteps to your right you have a good idea of exactly what you're hearing -- you can probably infer if it's a child or adult, you can possibly infer if they are masculine or feminine shoes, you can even infer formal vs informal attire based on the sound of the shoes (sneakers, sandals, and dress shoes all sound different), and you can locate their angle and distance pretty well. And that's with just your organic 2-microphone array.
I imagine multiple microphones and phase info could do a lot better in accurately placing the objects they hear.
It doesn't need to build an accurate picture of everything, it just needs to be good at imagining the stuff that actually matters, e.g. pedestrians, emergency vehicles. Where the model decides to place a few rustling leaves or what color it imagines a person to be wearing is less relevant than the fact that it decided there is likely a person in some general direction even if they are not visible.
I just think diffusion models are relatively good at coming up with something explainable and plausible for a given condition, when trained on some distribution of data.
Like "oh I hear this, that, and that -- what reality could explain those observations from the distribution of realities that I have seen?"
Conceptually, it's quite simple, you need to derive a gradient of the output error with respect to the sought information. And then use that to minimize the error (= "loss function" or "objective" depending on field terminology), like you do in neural networks.
In many cases, the solution is not unique, unfortunately. The choice of emitters and receivers locations is crucial in the case you're interested in.
There's a lot of literature on this topic already, try "acoustic inverse problem" on google scholar.
A few takeaways:
-The sampling rate is slightly off between devices - approximately ±1 sample per second - not a lot, but you need to take that into account.
-Spectral characteristics in consumer microphones are all over the place - two phones of the same model, right out of the box, will have not only measurable, but also audible differences.
-Sound bounces off of everything, particularly concrete walls.
-A car is the closest thing to an anechoic chamber you can readily access.
-The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely.
I recall a youtuber solving the anechoic chamber problem by finding a big empty field - nothing to reflect off of except the ground - and maybe putting some foam below the experiment.
It doesn't kill environmental noise, of course, but it apparently did a very good job of killing reflections from his own instruments.
I get the gaussian link. But, can you explain your point with more detail?
My plan was to detect the frequency of the speaker by counting (with weights) which distance between peaks is the most common. I wanted to avoid calculating the power cepstrum as I felt that I was running out of computing power on the devices already[0] - a mistaken belief in the long run, but I was too proud of my little algorithm and how stable it was to let go.
[0] Sending raw sample data to a more powerful machine was out of the question as I wanted to remain within the bandwidth offered by Bluetooth at the time due to power consumption considerations.
Sound deadening generally requires mass to work for lower frequencies and the seats absorbed them all nicely. I got some reflections from - I assume - the windows, but they were manageable in comparison. Didn't even produce much of a standing wave when I tried.
Or, to quote the progress report (https://asahilinux.org/2025/03/progress-report-6-14/#is-this...): "This is Apple though. Nothing is ever simple."
If you've got headphones with a wraparound microphone on its own arm then it could be better, but everyday headphones are limited by the position of the microphone
Apple puts a ton of R&D into making things work well. As another example: Macbooks have been, for 15+ years now, the only laptops that I can trust to actually sleep and conserve battery when I close the lid and slip into a backpack for a few-hr flight. Windows and Linux on laptops seem to have about a 70% chance of either not sleeping, not waking up right (esp with hybrid graphics), or trying to do forced Windows updates and killing the battery, then waking back up to 20+ minutes of waiting for updates to resume / finish with no meaningful progress indicator or way to cancel / delay.
Not everything they do is perfect, and I'm not some huge Apple fanboy, but they do offer a significantly better experience IMO and feel "worth" the premium. It's not as if modern gaming laptops are any cheaper than MBPs, but they certainly feel much jankier, with software and UX to match. As an example, the IEC plug on the power supply of my Asus Zephyrus Duo wiggles enough that it disconnects even with different IEC cables. I've had to wrap some electrical tape around the plug body to get it to be less flaky. Asus Armoury Crate is a terrible buggy and bloated piece of software that runs about a dozen background processes to deliver a "gamer" UI to...control fans, RGB lights, and usually fail to provide updates. They also have utilities like https://www.asus.com/us/content/screenxpert3/ and "ROG ScreenPad Optimizer" that are largely buggy garbage, but sometimes required to get their proprietary hardware to work properly.
Does Apple gouge users for extra RAM and SSD space? Absolutely, but you're paying for the R&D as much as the actual hardware. I wish they'd just price that into the base models and make upgrades cheaper, but their pricing strategy seems to be lowering the base entry point to something more appealing with "it barely works" levels of spec, while making increasingly ridiculous margins on higher specs -- an additional $4,600 to go from 1TB -> 16TB on the Mac Studio is pretty bold considering consumer QTY=1 pricing on a fast M.2 SSD is around $600 for 8TB, and I'm sure their BOM costs are around the same for 16TB worth of silicon in huge quantities.
As a photographer, this is a bit maddening.
In my experience, the fastest option for this is NFS without encryption, which is only really viable on a local network as it's hecking insecure (sure, wrap it in Wireguard, but now you're slowing it down again) and over Wifi at least, it's definitely slower than using an NVMe drive plugged into the Macbook, at least for 40 MP files coming out of my Fuji.
The external NVMe drive w/ Thunderbolt works... OK. But it's annoying (both physically and in terms of sleep/wake causing dismount warnings, etc.)
I just want a heatset aux port and I'm GTG. I want my money put into the GPU/CPU/Display/Keyboard.
Now my macbook pro for work? Yeah; high expectations there for AV quality in terms of joining meetings etc.
+1 on this one... I can close my lid (from on) and set my M1 air aside for a few weeks and still have plenty of battery left. I don't use it much when not traveling, it's mostly my desktop, work laptop or phone.
Also +1 on the hardware feel... it's got an above average stiffness, keyboard feel (for what little that's worth) and the best touchpad experience hands down. The screen is also on the higher end (I've seen slightly better in some really expensive laptops). All around, it's a pretty great value on the mid-high range. What I don't like is the aging UI/UX, the variance from other platforms (I use Linux and Windows pretty regularly) and some things that I just find harder on the platform in general.
I don't think I'd every buy a maxed out Apple product all the same, I don't use an iPhone or anything else but my laptop. That sometimes makes the ecosystem integrations slightly annoying. That said, my current laptop is still running well, and my prior laptop from over a decade ago is still running fine for my Daughter's needs... though she may get my m1 if/when I move to a Framework 13 (strix halo).
Even the cheapest of Chromebooks sleep and resume reliably. I suspect the reason is not purely R&D, but limiting the number of supported devices/chipsets and testing the supported configuration thoroughly. Chromebook OEMs can only manufacturer specific hardware combinations blessed by Google, and in exchange Google updates the drivers during the support period.
They don't actually sleep. Apple remarketed the concept of never sleeping as "Power Nap".
You can choose to have it actively updating the system or not, but it never actually sleep, just go into a ridiculously low power mode. You'll get the same on Surface Pro laptops or Chromeboks for instance.
Actual sleep only happens when the battery is about to die.
https://support.apple.com/guide/mac-help/turn-power-nap-on-o...
Power Nap is just fancy name for scheduled wakeups; it was supposed to be more but my understanding is that this never really materialized.
I might be confused on the name they chose to market never sleeping (never do full suspend to RAM with CPU shutdown except on special circumstances), as it was announced with Power Nap as the front facing feature.
From MS's doc:
> This option was designed for laptops and might not be available for all PCs. (For example, PCs with InstantGo don't have the hibernate option.)
https://support.microsoft.com/en-us/windows/shut-down-sleep-...
In the interim, they raised the price and added a ton of bloat so I don't use it anymore. (The bloat killed it, not the price. And the popup that's like "you're so stupid that you can't even figure out how to enable Krisp Speaker, you idiot". I'm well aware of how to enable it, but I have chosen not to, as I do not want to heavily process the audio that I'm listening to. Only emitting. "Don't ask again" would have probably made them an extra $110 at least.)
Don't know that I've had anything as loud as a fire truck, but more than a few times I've had a 75lb dog a few feet away from me barking like mad, whining at me, playing by throwing a cow femur up in the air and letting it crash down on the vinyl floor, etc and apologized about the noise only to have people look at me funny and tell me they didn't hear anything but that explains why I seemed like I was having trouble speaking.
I think the only time I had anyone say anything about anything was when I accidentally had an air conditioner blowing directly on my microphone. They couldn't hear it, but my voice was coming through a little less crisp than usual as the noise cancellation was trying to remove the constant, high volume white noise.
Don't know what OS you're on, but on Linux I can definitely recommend Easy Effects (https://flathub.org/apps/com.github.wwmm.easyeffects). Been using RNNoise + Speex along with some other filtering for quite a while now to great effect.
One thing I found worked _really_ well if you're already using an external microphone of some sort--using the webcam microphone as part of a noise gate. On top of the filtering and existing gating, my audio only opens if my webcam _also_ picks up sound of sufficient volume. Lets me keep the microphone in front of my face fairly sensitive while still entirely eliminating echo and most off-axis sounds.
Doing some things like disabling an input/output device, or an internal keyboard, or a webcam. Almost impossible. Even if there are some ways, they change so often. Let's say you have two cameras and an application that always picks the internal one. I couldn't find a way to disable the internal camera so that this app would pick the only available one.
mic = hs.audiodevice.findInputByName("MacBook Pro Microphone")
function handle_deselected(_, type)
if (type == "gone") then
if not mic:inUse() then
mic:setDefaultInputDevice()
end
end
end
mic:watcherCallback(handle_deselected)
mic:watcherStart()LE Audio is great though - and is already, as "the full stack" has had support for quite a while... Assuming you don't happen to get your equipment from a certain fruit supplier that is notoriously slow at implementing open standards, almost as if they want to not give you a choice outside buying their own proprietary solutions...
It reminds me of the telephone network, where even though the whole thing is just another packet-switched network these days, the abstraction exposed to the handset is an analogue baseband audio signal.
---
Why can't we get another type of "Bluetooth audio", that works like VoIP does between handsets and their PBXes — where the two devices will:
1. do a little handshake to negotiate a set of hardware-accelerated audio codecs the devices (not the Bluetooth transceivers!) both support, in descending order of quality, constrained by link throughput + noise; and then
2. open a (lossy, in-order) realtime "dumb pipe" data carrier channel, into which both sides shove frames pre-encoded by their separate audio codec chip?
Is this just AVDTP? No — AVDTP does do a capabilities negotiation, sure, but it's a capabilities negotiation about the audio codecs the Bluetooth transceiver chip itself has been extended with support for — support where, as above, userland and even the OS kernel both just see a dumb PCM-sample pipe.
What I'm talking about here is taking audio-codec handling out of the Bluetooth transceiver's hands — instead just telling the transceiver "we're doing lossy realtime data signalling now" and then spraying whatever packets you the device want to spray, encoded through whatever audio-codec DSP you want to use. No need to run through a Bluetooth SIG standardization process for each new codec.
(Heck, presuming a PC/smartphone on the send side, and a sufficiently-powerful smart speaker/TV/sound bar on the receive side, both sides could actually support new codecs the moment they're released, via software updates, with no hardware-acceleration required, doing the codec part entirely on CPU.)
---
Or, if we're talking pie-in-the-sky ideas, how about a completely different type of "Bluetooth audio", not for bidirectional audio streaming at all? One that works less like VoIP, and more like streaming VOD video (e.g. YouTube) does?
Imagine a protocol where the audio source says "hey, I have this 40MB audio file, it's natively in container format X and encoding Y, can you buffer and decode that yourself?" — and then, if the receiver says "yeah, sure", the source just blasts that audio file out over a reliable stream data carrier channel; the receiver buffers it; and then the receiver does an internal streaming decode from its own local buffer from that point forward — with no audio channel open, only a control channel.
Given the "race to sleep" argument, I presume that for the average use-case of "headphones streaming pre-buffered M4As from your phone", this third approach would actually be a lot less battery-draining than pinging the receiver with new frames of audio every few-hundred milliseconds. You'd get a few seconds of intensive streaming, but then the transcievers on both ends could both just go to sleep until the next song is about to play.
Of course, back when the Bluetooth Audio spec was written, something the size of AirPods couldn't have had room to support a 40MB DRAM buffer + external hardware parse-and-decode of M4A/ALAC/etc. But they certainly could today!
So we define a custom BLE service and blast audio file through it
I think this isn't a problem if you're using Apple headphones with Apple devices, but anything else falls back to crappy BT quality, usually with some kind of terrible ANC to boot.
FOr me, crappy audio setups and apps trying to do too much audio processing are the primary reason of "Zoom fatigue". I've done a lot of calls over apps that transmit raw, high-quality audio with no processing whatsoever, and the experience is just so much better.
On an unrelated note, I tried doing calls with a stereo mic setup but participants were actually uncomfortable with the ASMR-like effect of the audio.
The MBP mic is generally preferable to most headset boom mics in my experience with good noise reduction. You also get the benefit of not picking up extraneous mouth noises (gum chewing, coffee slurping, whatever)
I feel like 99% of people I conference with use regular headphones + MBP mic
Main problem with that setup is not being able to hear your own voice in the headphones (feedback, or whatever that's called) which can be annoying sometimes if using NC headphones
Monitoring.
There are umpteenth ways to do that, and I find headsets themselves do it the most poorly of all (if they have the feature at all).
> The MBP mic is generally preferable to most headset boom mics
Another benefit is not paying the '90s GSM handsfree BT profile codec pain (at the cost of A2DP having slightly higher latency)
It's called sidetone. Headsets do it so your ears don't feel clogged and to avoid subconscious yelling.
Some headsets let you adjust it either through a regular Sidetone volume control or some dedicated app. Soundcards also often have this feature in the form of a Mic output volume control, done in hardware to reduce latency.
A significant difference in headset quality is in sidetone latency. The heavier the DSP processing required to get a reasonable mic output, the harder it is to hit latency targets. Headset SoCs have dedicated hardware for this - a user-space solution like Apple pulls on their laptops would not be able to meet a usable latency target.
> Another benefit is not paying the '90s GSM handsfree BT profile codec pain
LE Audio includes the LC3 codec, solving this once and for all.
In the meantime while this rolls out, various alternate codecs exist that are fairly widely supported. This is especially true when using fancier headsets with a dedicated bluetooth dongle as they have more flexibility when it comes to codecs and compatibility.
AirPods of various versions is common, as is many other buds. Enterprise headsets like those from EPOS (the old Sennheiser Communication) and Jabra (with or without boom) and speakerphones are common in corporate settings, casual headsets (e.g., Sony, Bose) and wired gaming headsets are common at home.
The point is, everything they make is vertically integrated. They want to deliver a feature (like Airdrop or Continuity), they will cut across the stack to get it done. If you go the DIY route (which is what Asahi is effectively all about), you get to also DIY the missing software pieces.
The upside is that the entire ecosystem gets to benefit from that work (see e.g. the new DSP in PipeWire). PC hardware is generally crap, and so is Apple's if you omit these extra bits. But "the whole package" sets the bar quite a bit higher. I want to see the FOSS ecosystem meet that bar.
I already use the Bankstown bass harmonics synthesis plugin developed for Asahi and a convolution EQ on a cheap HP laptop, with startlingly impressive results, using the Pipewire plugin chain autoload feature also developed for Asahi.
I suspect there are quite a few use cases for this beamformer outside of the Asahi ecosystem as well.
I've got a blog post and associated podcast on Rust SIMD in the pipeline, we'll touch on this.
Does it mean M2/M3 don't have similar array of microphones or rather not tested?
I'm even curious if this is only supported on Linux or MacOS as well - not sure if apple provides dedicated microphone stream for each mic?
MacOS has its own software deep within the system for handling this; it's only exposed as a normal microphone to application software.
> Unfortunately, PDM mics are very omnidirectional and very sensitive. We cannot get by without some kind of beamforming.
https://asahilinux.org/2025/03/progress-report-6-14/
Also, it turned out that some previous work done for the speaker output was reused here for mic input.
> Thanks to the groundwork laid in PipeWire and WirePlumber for speaker support, wiring up a DSP chain including Triforce for the microphones was really simple. We just had to update the config files, and let WirePlumber figure out the rest!
Much like with the speakers, Apple are trying way too hard to be fancy here
What's overly complicated there? The hardware? The software?
As a MBP user and hobbyist audio guy I've been really impressed with the implementation of those speakers, particularly on the larger MBP models.
But I'm just a hobbyist and don't have any knowledge of them other than the driver arrangement (tweeter + dual opposed woofers). It certainly seems like they're pulling the same tricks used by "good" bluetooth speaker designers in order to wring acceptable perf and bass extension from teeny tiny speakers (adaptive EQ etc)
Dumb, noisy, cramped, unbalanced audio just doesn't cut it anymore.
This is about laptop speakers that just pass audio directly through, vs. laptop speakers that process the audio including spatially. Yes, it sounds dramatically better. And it's not just about "fake 5.1" but even just mono or stereo.
External speakers are a totally different conversation.
Similarly they can't be used very effectively without special, complex software that involves physical simulation of the speaker hardware. Doing things this way allows them to reach an amazing level of compactness + volume, but at the cost of complexity
If Apple intended to support platform openness, they'd likely have made such software available to hackers. But they never enthusiastically encouraged that, so people like the Asahi team are left to reverse-engineer and reinvent everything they need that lives in software
Probably the best overview to find out more is here: https://github.com/AsahiLinux/asahi-audio
In some ways speaker design is all about trying to cheat "Hoffman's Iron Law": bass, efficiency, and compact size.... you can only have 2 of the 3.
Part of it (as we know thanks to Asahi's work) is that you are varying a lot of speaker parameters dynamically. For example at low volumes you can dump extra energy into the bass frequencies. But at increasingly higher volumes you need to limit that. To get really dynamic you need to know not just the user's volume setting but like, real time spectral analysis of the actual program material
A speaker designer from 50 years ago would not be impressed with a modern pair of $500 or $1000 bookshelf speakers, those have barely changed. But they would be absolutely astonished at how some semblance of performance has been extracted from teeny tiny speakers on a Mac laptop or a high quality bluetooth speaker
It is just a reference that Apple Laptop speakers have been waaay above anything the competition uses - and this is true since multiple generations. Had a MBP from 2014 and multiple friends were astonished about the sound when we watched a movie on the go. Same with the M4 MBP - sounds quality from the speaker is at a level that you probably don't actually need.
Language is fascinating - I can convince myself with enough effort that the latter is just as valid as the former, given the literal meaning of the words, but my linguistic intuition is screaming at me that it’s wrong. How does someone ever learn that? How would a textbook ever explain it?
More like the opposite. The MacBook speakers are absolutely rubbish, just like all laptop speakers (there's only so much you can do when constrained to a laptop body). The reason why MacBooks sound good is entirely god-tier signal processing which manages to extract extraordinary performance out of some decidedly very ordinary speakers.
https://github.com/AsahiLinux/asahi-audio#why-this-is-necess...
If they are all rubish together, well, they are laptop speakers - and as such you have to treat them. Still there is nothing preventing some set of laptop speakers being objectively better than others.
They're extremely omnidirectional and very sensitive. With a single mic with no beamforming you get basically all of the sounds from every part of the room, including and especially horribly loud sounds from (eg.) the keyboard and mouse.
Apple selected their microphones based on the constraints their system had (beam formed array) rather than the "usual" laptop microphone which is physically not very sensitive and highly directional towards the front of the laptop, and in turn, those microphones are not particularly useful without beam forming.
Other laptops with beamformed arrays simply don't expose the raw mics to userland, by doing the beamforming in firmware, but this of course comes with its own set of issues.
Not always true, back in the Windows XP days (!!!) some laptops would expose the array to software and let the user configure where the mics record from.
It is unfortunate that user control has been polished out of modern systems in exchange for "it just kind of works".
Not certain if OP is saying they are currently an undergrad, but impressive if so
I'd search to see, but reading patents is an info-hazard which increases your chance of infringing, so I've quit reading them entirely.
But all joking aside, there is a tremendous amount of literature on the mathematics of beamforming. I'd be surprised if any of it is patented in a way that isn't circumventable.
I do think most such devices will present themselves as less capable than they actually are (I.E. just a stereo input) for maximum OS compatibility, but the technique isn't Apple exclusive as far as I know.
Are you sure? I’ve never heard a laptop microphone better than the MacBook. Maybe they do beamform and there’s other issues, but
is there a reason apple hasn't exposed a higher level api for this given the hardware (mic array) looks like it's already sufficient in macs?
Then in software they use digital signal processing. For speakers they modify what gets sent to the hardware so that the actual output then does match the frequency response, and for the microphones they do this work to extract the desired signal.
If Linux addressed the speakers as is, you would get unpleasant sound, and if it read the microphones as is, it would get a lot of noise. That is why Asahi had to add digital signal processing to the audio input and output, to get the "correct" audio.
It does mean the processing is specific to the analogue audio hardware in each of the different Mac models.
The processing could be done in additional hardware, but why bother when you have a very good CPU that can do the work.
As I understand, this is not a magic pill: it probably won't help to pull out frequencies which are suppressed by 30-40 dB and I assume that if the frequency response graph is too wavy (lot of narrow peaks and dips), it won't help either.
Also, you need to have calibration files to use this method, right?
Yes, I would be very surprised if they weren't using specific calibrations for each model. That's pretty basic.
eg
A ------ MIC1 --- B --- MIC2 ------ C
any sound coming from A, will be picked up by MIC1 well before MIC2, same for sounds coming from C. If you delete that audio from the income waveform, you have beam forming. And thus much better audio noise filtering.
And as it says in the link, Apple decided to implement this is software, not hardware, so you'd need to reimplement it if you're not using macos.
Good hardware definitely beats software trying to make something out of nothing: can't make directionality out of 1 mic, so Google Meet couldn't filter out background noise in that situation. Though it didn't help that these USB-C DACs seem to all be terrible (I tried several with the best findable reviews) compared to any old headphone jack where the device's internal DAC just worked
You’d think a phone could do cancellation between the bud microphone & it’s own microphones; maybe once the audio data from the bud has been pushed through bluetooth audio compression there’s no longer enough information to do that effectively?
In comparison, if you're on Meet with a crappy mic, it will still remove background noise, but your voice will still sound crappy. I.e. like a crappy mic in a quiet room.
Which is, as I'm sure you agree, is unfortunate and at least deserving of some (minor) reprisal.
Might be fancy, but it does make for surpisingly good audio from a laptops.
A lot of similar stuff is done in firmware on x86 laptops, to the point that both AMD and Intel now share considerable portion of the stack, with both using Xtensa cores for DSP, with Sound Open Firmware as SDK. When I use built-in microphone array on my laptop, it's parsed through the DSPs "transparently" to end user.
But technically you can load your own firmware there.
Apple does have some excellent audio engineers for the speakers, although these days the difference isn't as stark as it was five or ten years ago.
Of course you need to get a good Windows laptop to get any such quality, and many people and companies seem to only bother spending money on premium laptops if they're made by Apple.
Compared to the two new-ish AMD laptops? For the rare use case that warrants using built in speakers and mic, I see no real difference. Maybe latest macs are better, but... Usually the only use of built in speakers and mic are as last chance backup, or watching movies in bad conditions. Otherwise it's always a proper headset or standalone speakers
It’s night and day.
I haven't bought for built-in sound quality in the past (or ever, it's backup's backup after all), but I do remember lots of laptops offered with Harman-Kardon sound system, including hardware implementation of the dynamic compensation system in M-series Macbooks. Except also usually with way beefier speakers. Microphone arrays came in later arguably, but that is more correlated with availability of audio coprocessors - T470 had not great twin mic (mac was better there), but new ones easily handle it beyond my needs.
Way above "last choice backup solution" that "built in speakers and mic" are used for by me.
I've been trying to record audio in my noisy server room but only deepfilternet is able to deal with the fan noise in the background.
Instead, sound engineers spend weeks cleaning up spoken dialog by hand in spectrogram editors. It's honestly astounding the magic they can do, but it's also labor-intensive and therefore expensive. They're literally individually EQ-ing every vowel, splicing consonants from one word to another... it's wild.
There are many advantages to vertical integration as regards end-user-experience.
of course Apple has this implemented.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
That’s a rather unfair characterization. I’ve found the array to work very well in practice. It’s hardly trying to hard.
/_\Triforcin'
From Gemini:
```
Imagine you're trying to record someone talking in a noisy room using your MacBook's built-in microphones. This software acts like a super-smart filter:
* It knows where the microphones are: Apple laptops have multiple tiny microphones.
* It listens to all of them at once: It takes the input from all the microphones.
* It figures out where the person talking is: It analyzes the sound to find the direction of the voice.
* It focuses on that voice: It boosts the sound coming from that direction.
* It quiets down the other noises: It reduces the sound from other directions, like background chatter.
So, instead of getting a muddy recording with lots of noise, you get a clearer recording of the person you want to hear. Basically, it makes your MacBook's microphones sound much better in noisy environments. And it's designed to work within audio programs that use a specific plugin format called LV2.
```