Tsarp

There are direct prompt tests and then there are tests with tooling.

If for example you use controlnets you can pretty much get very close to a style composition that you need with an open model like Flux that will be far better. Flux has a few successors coming up now

For folks looking to running speech to text locally(in the context of dictation) and dont want to do a subscription there are a few apps that run whisper large turbo locally (optimized to run really fast on apple silicon).

1. https://goodsnooze.gumroad.com/l/macwhisper (dictation + transcription)

2. https://carelesswhisper.app (does dictation only, and does it really well; cheapest)

3. https://superwhisper.com (both local and hosted models + lots of bells and whistles, but much higher pricing)

Curious if the the paid tier helps support development of the project

1. Brain dump

- Voice based note taking tool that does transcription locally

- Markdown files to any folder(s) you setup.

- Help with ideation, rumination, todolists

- Alternative to when writing is too high friction

- https://voicebraindump.com

2. VoiceType

- wisprflow, superwhisper alternative

- runs locally so no subscription

- dictation tool, best for when using cursor, windsurf, chatgpt or talking to an LLM

- https://carelesswhisper.app

What tps are you hitting? And did you have to change KV size?

Atleast on macos cant the sandbox-exec be used similar to what codex is doing?

I tried this path and found that MacOS has horrible support on firecracker and similar.

For a lot of basic applications that do not need a lot of the extended browser APIs, Tauri is an amazing option. Consistency isn't really required in a lot of use cases. You just want things to work, you have a small team and Tauri works perfectly.

Some Linux distributions still dont work smoothly. If you could optionally choose to bundle verso just there I think its a great cross platform option.

Just came here to say, the demo video was awesome!

Refreshing to neither see a loom recording or a high budget video set in a Japandi architecture style office designed to go viral.

Great coverage. I was first hesitant to watch the 1hr45min long video. But well worth it!

Wondering how useful this would be for the agentic workflows that need browsing. The open deep research tread from yesterday mentioned using a pure text based browser sort of thing to quickly get info.

Side project not a business. I have a tiny dictation app for Mac on the app store.

One time payment since it runs whisper locally. Autoupdates through the app store, and I have a lot of folks emailing me positive, negative and improvement feedback.

It is a lot of randomness. Some weeks are low and when it got a small mention in a popular article I saw a sudden inflow of traffic, downloads and purchases.

So far Ive been ok paying the apple tax. Its a little hard going through the hoops to get it through the app store( I kinda understand why they do a lot of it ) but it provides a lot of free discovery and I spend 0 time on payments, refunds, disputes, handling a CDN to distribute binaries etc. Negative reviews without basis are the only thing that bother me, for some reason I seem to take it personally.

AI engineering is more applied than research and in that regard this list is great.

swyx and teams podcast, newsletter and discord has been the highest signal to noise ratio for keeping up and learning.

Thank you. Hope you try it and find it useful!

A lot of folks use it for this. But its not very straightforward like the tools that join calls and listen in.

It only transcribes what it can listen to. So if you are on a zoom call on your computer it can listen to both the audio from the PC and you speaking and transcribe.

Ive been looking into tapping in directly to both the audio out and mic in on apple devices. Im making progress but its not stable yet. Hoping to get it done during the holidays.

BrainDump: Apple Notes + VoiceMemos + ChatGpt in 1.

What is it? - Apple Eco-system based voice note taking/journal. - Tried various note taking apps. Nothing stuck like Apple Notes. - Writing by hand was the best way to get clarity but seemed high friction. Talking about ideas with friends/colleagues was the second best to thought clarity. - Basically combined it. Voice -> Transcribe (whisper accuracy locally) -> Rewrite with COT LLMs. - Helped me maintain journal now ( life events, ideas, anxiety days etc ). Added some prompts to help me. ( researched what experts in the field recommend; as an example - How do I start? And what do i say if i want to gratitude journal ).

Since launch, more than 50% end up using it for meeting notes.

https://voicebraindump.com/

VoiceType: Mac Dictation tool. Running locally with WhisperLarge Turbo. So its fast accurate and can get a lot of Rust Library names correctly when I use with Cursor.

Why I Built It?

I ended up getting RSI over the last year and half. Despite lifting 3x a week and physio-therapy, circumstances had me working >12-15hours 6-7days a week for a few months straight. I’d read about folks using diction but never worked well for me or was pricey software.

At the same time I started using ChatGPT, Claude, Cursor etc quite a bit. When the Whisper Large turbo model was released by OpenAI, I tried transcribing some technical terms and it got transcribed quite well. ( still makes errors, but its within tolerance of what ChatGPT et.all can understand). I mostly talk type to my Mac now.

https://carelesswhisper.app/

Ive worked on some enterprise NER systems (specifically privacy/redaction), and in almost all cases the cost of missing out masking was significantly higher than latency (ofc in an ideal world youd have both).

And in all the research we did, the best solutions ended up passing through a workflow of 1.NN based NER, 2.Regex and 3.Dictionary look ups to really clean information. Using a single method worked well in customer demos but always ended up in what we thought were edge cases in prod.

That being said, latency stuff makes sense. This might work great in conversational use cases. Picking out intent and responding. Every millisecond helps in making things sound natural.

Wouldnt it be better to run normal Whisper and NER on top of the transcription before streaming a response or writing anything to disk?

What advantage does this offer?

After a lot of experimentation, the only thing that worked in a chat style application is to pass maybe the last 4-5 messages (ideally the entire conversation history) and ask an LLM to summarize the question in the context of the conversation.

Without that it often failed when users asked something like ("Can you expand point 2? , Give a detailed example of the above").

Current implementation(I have 3 indexes) is to provide Query + Past messages and ask an LLM to break it down into Overall ask: BM25 optimized question: Keywords: Semantic optimized question:

Perform RAG + Rerank and pass the top N passages after this along with the Overall ask in the second LLM call.

quite impressive! This is really more like what I was hoping Elicit would be.

Are you breakdown the question into subtopics, doing a broad search and then doing some sort of dim reduction -> topical clustering to get it in the format?

How does it compare with something like Bloop, which also uses a combination of a syntax tree, Embeddings, FTS and LLMs?