MediaSquirrel

Summary: https://gist.is/arxiv.org/en/YVxP3o7y3Dq1

Ha, I built a version of the same thing as a Cursor plugin. Check it out:

https://open-vsx.org/extension/Transcendence/gist-discover https://gist.is/discover

We literally posted our Show HNs within minutes of each other: https://news.ycombinator.com/item?id=48768342

Great minds think alike.

Dr. Alex Mathiasen, PhD wants to build a pause button for human biology – so that you can live forever. All he needs is enough GPUs to simulate quantum physics.

Do share.

yeah, it came out after I stared on my project last year. Only issue is that you can't fine-tune it on Apple Silicon.

depends on the model!

If you run a smaller whisper-distil variant AND you optimize the decoder to run on Apple Neural Engine, you can get latency down to ~300ms without any backend infra.

The issue is that the smaller models tend to suck, which is why the fine-tuning is valuable.

My hypothesis is that you can distill a giant model like Gemini into a tiny distilled whisper model.

but it depends on the machina you are running, which is why local AI is a PITA.

Look inside here: https://github.com/mattmireles/gemma-tuner-multimodal/tree/m...

Here’s the trick: use Gemini Pro deep research to create “Advanced Hacker’s Field Guide for X” where X is the problem that you are trying to solve. Ask for all the known issues, common bugs, unintuitive patterns, etc. Get very detailed if you want.

Then feed that to Claude / Codex / Cursor. Basically, create a cheat sheet for your AI agents.

This will unlock a whole new level of capability.

I’m @mattmireles on Twitter — feel free to DM me.

More data -> better, faster on-device models

The actual plan was to distill Gemini 2.5 Pro into the best on-device voice dictation model.

Pretty sure it would have worked. Alas.

re: Whisper v3 -- how is this possible? Whisper has a 30s context window. You have to chunk it.

Great minds think alike!

Also, I had a huge head start, as I spent a month or two working on this in September 2025, shelved it and dusted it back off this weekend.

Haven’t tried yet. That’s on the do list. But good suggestion.

you are welcome! It was a fun side quest

Memory usage increases quadratically with sequence length. Therefore, using shorter sequences during fine-tuning can prevent memory explosions. On my 64GB RAM machine, I'm limited to input sequences of about 2,000 tokens, considering my average output for the fine-tuning task is around 1,000 tokens (~3k tokens total).

Here's the gist of the paper for anyone interested: https://gist.is/science.org/en/VdSDF9qjxbH8

Nukes gave us peace and freedom.

We've had no WW3 (so far) and no one here needs to worry about being drafted into a war. Gatling might have thought his gun would reduce the number of war fatalities, but but Oppenheimer thought he would end the world. Both were wrong.

Alternative take: Inventors are bad at predicting the downstream societal effects of their inventions.

In my conversations with people at Apple, my understanding is that they do not use CoreML. Instead, they have access to lower level libraries that allow more direct programmatic control of the hardware.

CoreML is the crappy middleware they made for 3rd party devs, but never got much love and never took off.

Re: ANE — as stated, ANE is crazy fast when it works. Yes, it’s also more power efficient, but the reason I think it’s actually worth building on is being able to make consumer products where the entire experience depends on speed.

I think you can agree that 5 milliseconds to generate a 512 px resolution image is absolutely insane speed.

Yeah, this is why I was begging investors for a high-level intro to someone at Apple, because I was very afraid that I’d run into some random bureaucrat that would have a similar response.

It sounds like your experience differs from mine. I oversaw teams trying to use CoreML in the 2020 - 2024 era who found it very buggy, as per the screenshots I provided.

More recently, I personally tried to convert Kokoro TTS to run on ANE. After performing surgery on the model to run on ANE using CoreML, I ended up with a recurring Xcode crash and reported the bug to Apple (as reported in the post and copied in part below).

What actually worked for me was using MLX-audio, which has been great as there is a whole enthusiastic developer community around the project, in a way that I haven't seen with CoreML. It also seems to be improving rapidly.

In contrast, I have talked to exactly 1 developer who have ever used CoreML since ChatGPT launched, and all that person did was complain about the experience and explain how it inspired them to abandon on-device AI for the cloud.

___ Crash report:

A Core ML model exported as an `mlprogram` with an LSTM layer consistently causes a hard crash (`EXC_BAD_ACCESS` code=2) inside the BNNS framework when `MLModel.prediction()` is called. The crash occurs on M2 Ultra hardware and appears to be a bug in the underlying BNNS kernel for the LSTM or a related operation, as all input tensors have been validated and match the model's expected shape contract. The crash happens regardless of whether the compute unit is set to CPU-only, GPU, or Neural Engine.

*Steps to Reproduce:* 1. Download the attached Core ML models (`kokoro_duration.mlpackage` and `kokoro_synthesizer_3s.mlpackage`) 2. Create a new macOS App project in Xcode. Add the two `.mlpackage` files to the project's "Copy Bundle Resources" build phase. 3. Replace the contents of `ContentView.swift` with the code from `repro.swift`. 4. Build and run the app on an Apple Silicon Mac (tested on M2 Ultra, macOS 15.6.1). 5. Click the "Run Prediction" button in the app.

*Expected Results:* The `MLModel.prediction()` call should complete successfully, returning an `MLFeatureProvider` containing the output waveform. No crash should occur.

*Actual Results:* The application crashes immediately upon calling `model.prediction(from: inputs, options: options)`. The crash is an `EXC_BAD_ACCESS` (code=2) that occurs deep within the Core ML and BNNS frameworks. The backtrace consistently points to `libBNNS.dylib`, indicating a failure in a low-level BNNS kernel during model execution. The crash log is below.

Your bar is higher than mine.

Unlike Apple's formal "developer evangelist" and several others I contacted, the guy actually took the time to talk to us, and I was/am grateful for that. He's a cog in a very large corporate machine. Apple is Apple. He's not the CEO. He was doing his job and did me a favor. I am grateful to him.

I mean, to be fair, these events happened several years ago. My memory is as faulty as any other human being's, but as far as I remember, this is exactly what happened. These were very memorable events that I remember distinctly. It's possible my memory is distorted, but this is literally what I believe happened to the best of my ability.

Obviously, there was more to the conversation that what I wrote, but these are the actual words that I remember being said.

For more context, the PM at Apple in question was a former colleague of my then girlfriend. I reached out to her to have a friendly catch up. It wasn't positioned as meeting officially with Apple. I was literally just going through LinkedIn trying to figure out who I knew at Apple. So I hit her up on LinkedIn and asked to catch up, then told her about the situation. And this is how she responded. Worth noting: English is not her native language.

Yes, MLX is for research, but MLX-Swift is for production and it works quite well for supported models! Unlike CoreML, the developer community is vibrant and growing.

https://github.com/ml-explore/mlx-swift

Maybe I am working on a different set of problems than you are. But why would you use CoreML if not to access ANE? There are so many other, better newer options like llama.cpp, MLX-Swift, etc.

What are you seeing here that I am missing? What kind of models do you work with?

> Why would someone say "You're a mosquito. Apple will just stomp on you and you will not exist.", it makes zero sense to me given the context laid out here.

I'm telling you what I was told. It's a true story. I was there. It happened to me.

Why would I make up a detail like that?

OP here: I can see why you'd think "this can't possibly be real," but I assure you that the story is real, not exaggerated. I was there.

I don't get what's so weird about MLX. Apple's focus is obviously on MLX / Metal going forward.

The only reason to use CoreML these days is to tap into the Neural Engine. When building for CoreML, if one layer of your model isn't compatible with the Neural Engine, it all falls back to the CPU. Ergot, CoreML is the only way to access the ANE, but it's a buggy all-or-nothing gambit.

Have you ever actually shipped a CoreML model or tried to use the ANE?

Our app wasn't running on CPU or GPU –– the actual software we built was running entirely on Apple Neural Engine and it was crazy fast because we designed the architecture explicitly to run that specific chip.

We were just calling the iPhone's built-in face tracking system via the Vision Framework to animate the avatars. That's the thing that was running on GPU.

I can see why you'd be skeptical. It was pretty insane what we did.

I built a tool to ask Google, Perplexity, and ChatGPT the same question — and now I don’t trust any of them.

Did you read it? And why do you assume that it can't be interesting?

Honestly, as a writer, I sometimes spend a bunch of time writing an essay that's the result of a conversation, and I realize –– hours later –– that the conversation actually does a better job of exposing the idea and explaining it than the essay I worked so hard to produce.

I agree that Claude convos can be slop, but don't bash it before you read it!

LOL

Why?

In 2021, I built the first real-time video generation AI app for Apple Silicon. It was revolutionary - until it started melting iPhones. When my demo literally burned Sam Altman's hand, I discovered that Apple's on-device AI hardware was not designed for modern AI.

Look inside any M-series Mac today. Nearly half the silicon - the Neural Engine cores - sits idle, designed for the pre-transformer era of AI. Meanwhile, my ninety-five-year-old father struggles with a seventy-five-inch screen he can barely see, finding brief moments of magic only when speaking to Siri.

The problem isn't just technical - it's ideological. Apple's privacy stance prevents them from gathering the real-world data needed for truly adaptive interfaces. But there's a solution: AppleCare Platinum, a premium human support service that could evolve into a radical new kind of UI - by serving as an Enders Game for AI computer use training data.