znagengast

Takes me back immediately, really cool!

Maintainer of WhisperKit here, confirming we do exactly that for longform. We search for the longest "low energy" silence in the second half of the audio window and set the chunking point to the middle of that silence. It uses a version of the webrtc vad algorithm, and significantly speeds up longform because we can run a large amount of concurrent inference requests through CoreML's async prediction api. Whisper is also pretty smart with silent portions since the encoder will tell it if there are any words at all in the chunk, and simply stop predicting tokens after the prefill step - although you could save the ~100ms encoder run entirely with a good vad model, which our recently opensourced pyannote CoreML pipeline can do.

Turns out third party actually gets priority for ANE

This did not age well https://twitter.com/Shopify/status/1724079230663979066

I have to play devils advocate here because for every one of these cases, there is probably a dozen of similar stories where the ambitious new guy actually did nuke the system with a risky friday release and then logged off for the weekend xD

The thumbs up icon killed me

It's also largely dependent on the embeddings model as others have mentioned. Even if wikipedia doesn't have any words specifically referring to that monkey as "weird", the model itself would know to correlate this monkey's embeddings with the "weird" concept. The main issue with this particular implementation is the model used (all-minilm-l6-v2) which is designed for speed and efficiency over accuracy.

Found the cake https://noclip.website/#Portal/escape_02;ShareData=AZZCH9W7f...[

Isn't it crazy that the entirety of human knowledge can be condensed down to fit on an SD card.

The page has changed over the years, there used to be a much more apologetic version "Reddit broke (sorry)" https://github.com/reddit-archive/reddit/commit/4778b17e939e...

So they actually made the intentional decision to change it back to the much more accusatory "you broke it", I'm not sure exactly when that occurred but I suspect it was roughly the time spez took over.

Compared to how much progress the iPhone made from initial launch to now, the potential for this product line is very exciting.

What if you split up the training down to the literal vector math, and treated every macbook like a thread in a gpu, with just one big computer acting as the orchestrator?

Whoa didn't know about this, cool

How feasible would it be out crowdsource the training? I.e. thousands of individual macbooks training a small part of the model and contributing to the collective goal

Hey HN! I recently faced a challenge of generating vector embeddings within my Swift app, so I built this library as a "good enough" solution for on-device similarity search without relying on an API. I've got some exciting plans for future development, but I'd love to hear your thoughts and any interesting use cases you can think of!

How are you guys thinking about the embedding generation side of things? It seems like that part has a generally hefty compute cost before it even gets into the index - I just open sourced a swift package to try to make that part as easy as possible, the example project exports directly to pinecone. https://github.com/ZachNagengast/similarity-search-kit

Brings to mind a quote from The Hitchhiker's Guide to the Galaxy:

> "Only once you do know what the ultimate question actually is, you'll know what the answer means."

This would be amazing! At lot of ML use cases are largely unfeasible in lambda on python without serious pruning. Latest version of tensorflow is 150mb uncompressed, add numpy pandas etc to that and it adds up fast. I think 1 GB uncompressed would be pretty reasonable in the current state of ML tools, personally.