HNHacker News
TopNewBestAskShowJobs

pancomplex

630 karma · joined October 14, 2015

CTO @ Frigade (YC W23)
submissionscomments
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Sorry about these issues - appreciate all the details. Are you on an Intel Mac by any chance?
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Apple's built in dictation still executes parts of it online and even has a disclosure that it "sends information like your voice input, contacts, and location to Apple."

This project uses the on-device SpeechAnalyzer API directly and makes zero network calls, and since it's open source you can verify that nothing leaves your Mac.

pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Thank you for this feedback! Heard from other folks the menu bar is redundant too - will kill it in the next update and fix the F13 issue.

And interesting re: history - it does have the option to clear, but you’d want that as a setting?

Trying to keep the amount of settings to a minimum - personally finding it overwhelming when apps have too many bells and whistles lol

pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Totally! I honestly think so many frontier AI models will be running locally in the future. This is just the start.
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
It's a great name. Glad someone else thought of something similar!
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
I think it once did! Now it doesn't - ways to easily verify this:

1) install the dmg in the GH repo

2) turn off wifi

3) use it offline

pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
That would be a super trivial fork actually! You basically just need this file https://github.com/FrigadeHQ/yap/blob/main/Sources/Services/... and then wrap a cli around it :)
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
I don't think you're missing something - Apple's built in dictation is pretty good. The biggest issue is that it still executes parts of it online via Apple's servers and even has a disclosure that it "sends information like your voice input, contacts, and location to Apple."

This project uses the on-device SpeechAnalyzer API directly and makes zero network calls, and since it's open source you can verify that nothing leaves your Mac. It also keeps a local searchable history. Finally, I also added press-to-stop with a preview so you can cancel a bad take before it inserts, instead of typing live (very useful in terminal).

What type of package were you hoping it was?

pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Thanks for linking! I've used both Apple's and Parakeet locally for a while and haven't been able to spot the difference in quality and speed is better on Apple's model (+ doesn't take several gigs of memory). Have you noticed any quality differences anecdotally?
pancomplex··on Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
My bad - this is now fixed. Thank you!
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Luckily no. The calls are executed client-side just like they would if you were using the UI.
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
We only do this with consent from the application owner - the use case is typically companies who don’t have an AI agent yet who want fast path to get one.

This is the process we usually do: https://frigade.com/how-it-works

pancomplex··on Show HN: Reverse-engineering web apps into agent tools
This works for non-authenticated sites but wouldn't work for anything behind a login wall that requires a JWT or similar - right?
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Yup just another user account. It can work without this as well (we support ingesting other resources such as help center articles, pdfs, etc.) but at that point it's no different from any other (dumb) AI chatbot out there that just spits out a bunch of itemized bullets.
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Even with an API spec it wouldn't "just work". You'd still need to handle authentication and have a place to manage which APIs you want the in-app agent to have access to / when to call a given tool.

And believe it or not, even big companies with big eng teams don't have API specs available for their applications ¯\_(ツ)_/¯

pancomplex··on Show HN: Reverse-engineering web apps into agent tools
We can either replace an intercom widget entirely or do a hand off with context to a live intercom chat agent.
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Here's a more thorough explanation of how it works under the hood: https://frigade.com/features/skills
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Agreed - and especially when the API changes..
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
We use a browser-based agent to learn all the APIs and turn them into skills. Most users will run this in a staging/test account to create all the recipes/blueprints. Our agent is also instructed to not take any destructive actions - but of course LLMs make mistakes (hence the test account :) ).

You can see more about how it works in detail here: https://frigade.com/how-it-works

pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Works with any frontier LLM model
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
No definitely not!

Our customers run the training on their own applications using a demo or staging environment. Then they install the in-app agent you see in the demos and turn on the tool calls they like. All the API calls are executed client side and never touches any of our servers.

pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Not at this point - but we’re considering it! What would you use it for?
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
Thank you! Along those lines, we did Airbnb too: https://demo.frigade.com/
pancomplex··on Show HN: Reverse-engineering web apps into agent tools
thank you!
pancomplex··on Why most product tours get skipped
100% - that's why it's so confusing why PMs/PMMs think they need to keep adding these to their products.
pancomplex··on Why most product tours get skipped
tbh adblockers should just filter these out. I guess the reason they don't is it's "technically" the product ¯\_(ツ)_/¯
pancomplex··on Why most product tours get skipped
couldn't agree more - they always pop up at the right time. I don't know why every PM thinks they can save retention by spamming users :(
pancomplex··on Show HN: Mastra 1.0, open-source JavaScript agent framework from the Gatsby devs
seems non of these are typescript-based? Strand appears to have a typescript sdk available but not natively ts.
pancomplex··on Show HN: Mastra 1.0, open-source JavaScript agent framework from the Gatsby devs
We use typescript for all our entire stack and it's super dope to see a production-grade framework (with no vendor lock in) launch!
pancomplex··on Show HN: An AI agent that learns your product and guides your users
Yes
← PreviousPage 2 of 4Next →