Rewind: The Search Engine for Your Life
rewind.ai
rewind.ai
Overall it has helped me in a couple situations. I will say when I was onboarded I don't believe audio was mentioned and a quick search for things I know I said in Zoom recently comes up empty--perhaps that functionality is new?
For now, I've found that I haven't yet hit the equilibrium point where I've forgotten something badly enough to replace my instinct to use Spotlight, browse Finder, or even search Google Drive (shudders). The Fitness app on iOS needs 180 days of data before it shows you trends; I feel like Rewind is similar in that I need a critical mass of data and time before I truly don't know where to look and go to Rewind first. Audio support would greatly reduce that window, however, so maybe I am missing out.
Overall I definitely recommend trying it out, however!
Its great that you have leveraged the power of Apple Silicon to analzye and index the recordings as well as keeping the size of the recordings very very small. Do you see this becoming available for Intel Chipsets a.k.a Windows and Linux anytime soon?
From the docs:
> Video data is never sent off your Mac. The only data sent to a cloud service is the audio. We send it to a cloud transcription service in order to generate a transcript of what was said. This transcript is created so you can read it and search for specific words that were said. [1]
Yeah, we only "store locally", but hey, that doesn't stop us from sending it to a third party service for transcription. I was already uncomfortable with the idea of being voice recorded. Reading between the lines of how it works doesn't inspire a ton of confidence.
[1]: https://help.rewind.ai/en/articles/6526620-recording-meeting...
The only time this cloud-based transcription is used is during a Zoom meeting, which is already in the cloud.
When you join a new Zoom Meeting we prompt you to ask if you want to record & transcribe the audio. You can choose to do this selectively for each meeting, or not record & transcribe any of them if you aren't comfortable with cloud-based transcription.
That said, we do plan to move away from cloud-based transcription, likely with Whisper from OpenAI.
In the modern world even the most genuine and honest people can't be trusted on black boxes so such kind of memex while wonderful and potentially useful have a so big safety trade off that's a no-go for me... Personally I do not even live my smartphone near me at home since I consider it a macro-spy tool operated by someone else. I can't do much for in-car stuff since I need cars and can't live on historic one for mere comfort. But more than that is beyond personal Overton window of acceptability...
Anyway, that's a very nice implementation of a very nice idea.
Edit : do you hire EU citizens ? :)
Edit edit : most of the world doesn't use macOS : what's your long-term vision for non Apple silicon users ? On smartphones ? Any plan on a hardware product that literally records everything you say ?
The UI also looks very good for a first version. Excited to give this a try!
[1] https://news.ycombinator.com/item?id=27965979
Also, I called it out that time, I knew this was going to be huge: https://news.ycombinator.com/item?id=27965979#27970968
I've been recording my desktop on Mac, Windows and Linux, recording audio in my office, including transcription of the audio, and multiple camera capture for over a decade since early 2000's, and using a SenseCam-like device, on-and-off, for almost 20 years. My current setup of an NVidia Jetson does an admirable job of determining if one or more people are in the room, the activity they are engaged in, multiple camera angles, transcribing what is being said, music identification, desktop image capture and OCR of the images is very much kept locally and I can be assured nothing is leaking out without my explicit knowledge. I have a custom device on my desk that I built that can "pause the recording" or "forget the last 30 seconds" or "forget the last five minutes" for those events that probably shouldn't be captured, e.g. I'm picking my nose and carefully inspecting whatever was retrieved.
This is more akin to Raindrop or a bookmarking addon with notes and folder structure built int
Signed up for the beta. Really curious about it.
I could be wrong but I'm skeptical about this claim because video compression can't be done so simply just by creating a new algorithm and writing it in software. Especially the newer codecs such as h265, h266, and AV1 must be in hardware or else it would be too computational heavy and your device will just choke up trying to spend so much cpu time compressing when you're trying to run other processes.
Also if it was their own video compression, they would have to write their own decoder for their own player for every device. That is way too much for a start up to take on. It just smells like BS to me.
This is the key information, as they support M1/M2 which have en/decoders in silicon, they definitely use some off the shelf codec. So they don't use any new breakthrough tech, but marketing existing tech in a shady way.
Thank you for your insight!
Not only that, but we have the following features focused on privacy:
1. Full control: users can pause and delete recordings at any time
2. Exclude apps: users can exclude specific apps they don’t want recorded (e.g. Signal or 1Password)
3. Incognito: by default, we don't record Chrome Incognito or Safari private browsing windows
We have a long roadmap of things we have planned to further our commitment to privacy. Anything in particular you'd love to see us implement that might make you feel more comfortable?
Recording meetings for later searching still feels weird in a similar sense. Even if I trust the way my data is handled when I use it, it's a violation of privacy norms in a way that feel "creepy" to be making a persistent record of things other people said to me in what is traditionally an ephemeral context.
(That said, this sounds like something I've occasionally wanted for a long time.)
I'd like two AI toys on top of it - embedding similarity search and question answering. The AI part can be executed easily with HuggingFace. There is a great QA model (FLAN) trained with multiple tasks, it can sort of do what GPT-3 does but locally, the private & cheap option.
Actors can easily access data on a local machine, with the necessary tools and desire. Still seems like something that could be accessed and abused.
I am eternally grateful to Hacker News for helping me launch my first startup, Optimizely (YC W10), so I’m back here to launch my second one: Rewind.
Rewind is the search engine for your life. Check it out: https://www.rewind.ai
It’s a macOS app that enables you to find anything you’ve seen, said, or heard.
We’re also announcing that we’ve raised $10m at a $75m valuation led by Andreessen Horowitz.
In case you are curious, the inspiration for Rewind came from an unlikely place:
Our founding story
I started to go deaf in my 20s. When I turned 30, a hearing aid changed my life. To lose a sense and gain it back again felt like gaining a superpower. Ever since that moment, I’ve been on a hunt for ways technology can augment human capabilities and give us superpowers.
That hunt ultimately led me to memory. Studies show that 90% of memories are forgotten after a week. Just like our hearing, our memory gets worse as we get older. But does it have to? If we have hearing aids for hearing and glasses for vision, what’s the equivalent for memory?
What if we could use technology to augment our memory the same way a hearing aid can augment our hearing? This question is why we founded Rewind. Our vision is to give humans perfect memory.
Introducing Rewind
Here’s how it works:
1. Local and Private By Design: We record anything you’ve seen, said, or heard and make it searchable. For your privacy, we store all of the recordings locally on your Mac. Only you have access to them. Learn more: <https://help.rewind.ai/en/collections/3616305-privacy>
2. Mind-Boggling Compression: Storing all the recordings locally means compression is very important. We compress raw recording data up to 3,750x times without major loss of quality. For example, 10.5GB of raw recording data becomes 2.8MB. That means that even with the smallest hard drive you can buy from Apple today, you can store years of recordings.
3. No Cloud Integration or IT Required: In order to enable you to search for anything you’ve seen, we use native macOS APIs and Optical Character Recognition (OCR) to recognize & index all the words that appear on your screen. This means you don’t need to integrate with cloud products like Gmail, Dropbox, or Slack. Rewind starts capturing these apps right away, with no IT required. Also, OCR enables you to copy & paste anything from your past.
4. Meetings: In order to enable you to search for anything you’ve said or heard, we use state-of-the-art Automated Speech Recognition (ASR). This is particularly useful for meetings, where the content of discussions, debates, and decisions are often lost forever as soon as a meeting is over. With Rewind, you never have to worry about losing this content again. Rewind can capture the content of meetings, including everything that is said and everything that is shared visually. All of the content is searchable. You can go back to the exact moment in a meeting you are looking for by simply searching for a word that was said or a word that appeared on your screen. Learn more: <https://help.rewind.ai/en/articles/6526620-recording-meeting>
5. Unleashes Apple Silicon: The key enabling technology that makes Rewind possible is Apple Silicon. We utilize virtually every part of the System on a Chip (SoC) so that running Rewind doesn’t tax system resources (like CPU and memory) while it is recording. It feels virtually imperceptible.
Rewind is just the first step toward our vision of giving humans perfect memory.
Please share any and all feedback you might have! We’re all ears
Until an unauthorized person gains access to your computer.
I'm assuming there is some sort of encryption, but if the software can transparently decrypt the data to access it, then what is to prevent someone stealing the key from the software?
How is this affected by wiretap and privacy laws, where you would need the consent of other people in the room?
At home, I certainly don't want something recording every sound.
Even at work, I don't want everything recorded. Sometimes people need to communicate freely and discretely, e.g. your manager says to you like "Don't prioritize the CEOs request. It's a bad idea and won't go anywhere. If he asks you, we're working on it. I'll deal with the fallout later."
You are right that if someone compromises your physical computer and knows your local Mac user account password then encryption won't help.
> How is this affected by wiretap and privacy laws, where you would need the consent of other people in the room?
While the laws differ from state to state, we believe privacy is so important that we recommend users of our product hold themselves accountable to a much higher standard than the bare legal minimum. We believe all users of our product should proactively seek consent from everyone they record even if they are not legally obligated to do so. Not only is this is the safest approach legally, but it is also just the right thing to do.
> At home, I certainly don't want something recording every sound.
We don't record every sound. We prompt you to record audio each time a new Zoom meeting starts. You can choose to record or not to record. You can also pause at any time and delete anything retroactively.
If you could bump me on the invite list (mike@luabase.com), would truly appreciate it. I'm a good beta user, promise.
Yes! Here's the tweet you are referring to: https://twitter.com/patrickc/status/953011978217205760
And his more recent reaction to our product: https://twitter.com/patrickc/status/1587450179069743104
> Amazing work. There's a mic drop moment in the demo when you're able to click on the link shared in the zoom, totally sold me.
Thank you! Flattered. :)
> If you could bump me on the invite list
Absolutely! I'll move you up the wait list.
How should I ask for consent?
Here is how we recommend you ask for consent:
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I use Rewind to record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
In addition to saying this verbally, we recommend including this message in your calendar invites so people have a chance to think about it ahead of time. Feel free to link to this help center article as well.
What are the laws around consent?
The laws pertaining to recording other people depend on where the people being recorded live.
While the laws differ from state to state, we believe privacy is so important that we recommend users of our product hold themselves accountable to a much higher standard than the bare legal minimum. We believe all users of our product should proactively seek consent from everyone they record even if they are not legally obligated to do so. Not only is this is the safest approach legally, but it is also just the right thing to do.
You aren't asking people to be conscious of privacy, you are asking people to be a free marketing platform.
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I use Rewind to record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
to
"This meeting is important to me so I want to make sure I don't forget what we discussed. Is it ok with you if I record and transcribe the meeting? For our privacy, all data is stored securely on my Mac and not in the cloud."
by all means go ahead.
1. You claim that the user data is stored locally, that no data is sent to the cloud but audio is sent to a cloud transcription service. How do you reconcile these things?
2. What languages does the app support?
Thank you.
For 5, start with "It feels virtually imperceptible." rather than end with that, and don't define/use SoC. Maybe something like "Unleashes Apple Silicon: Thanks to the advancements with Apple Silicon, the overhead of Rewind is virtually imperceptible by utilizing every part of the chip including the Media Engine."
Really intrigued by the product and hope I can try it soon!
What companies do you think would be interested in purchasing Rewind and why?
Why focus on Macs when the majority of the world uses Windows?
Compare a base-en whisper transcription to a human transcription. This is the A Powerful Theory of Why the Far Right Is Thriving Across the Globe latest episode of the Ezra Klein Show, transcribed just now:
1. [whisper] https://modal-labs-whisper-pod-transcriber-fastapi-app.modal...
2. [human] https://www.nytimes.com/2022/11/01/opinion/ezra-klein-podcas...
(You can use the same app to transcribe other episodes on-demand and see the base-en model's quality)
> ..We compress raw recording data up to 3,750x times..
Richard Hendricks should've run "Pied Piper" with this use case :-)
In any case, I may be wrong, but I don't think anyone's gonna install that on their computer.
how does data transfer look like, when changing your mac?
I've started keeping a mood journal after a mood swing caused a very negative incident last year. It's been absolutely indispensable.
I could have used "less tech" and kept it in notebooks, but (a) handwriting is time-consuming, (b) handwriting a lot is fatiguing, and (c) I could lose the notebooks (like I lost the diary I kept when I was younger or the notes I exchanged with my first girlfriend that went into detail about what I did that day, both of which I wish I had back).
Using an app to keep this journal has made it really easy to correlate my mood to external events. Since my mood is strongly correlated to the amount of sleep I get, I can't wait to stitch my sleep and calorie consumption data from Health with my mood journal entries.
Ironically, the journal has helped me remember things I would have long forgotten!
I'm using Daylio for my mood journal. It's private. I use it every day, sometimes multiple times per day. (I _always_ use it when a significant emotional event happens.)
I love it because I can explicitly pick my mood based on five options (rad, good, meh, bad, awful), can create and pick unlimited additional identifiers to associate with each journal entry, and can see reports showing me not only my average mood for the month/year but also which identifiers corresponded to what mood.
Example. I slept well today, felt really good, wrote code, spent time with my wife and worked out. So in my entry today I picked the "Good" mood (internally a 4/5), then picked the "good sleep", "home", "sunny", "time together", and "wrote code" identifiers. I also have a custom identifier called "do i need alone time" wherein I can pick how badly I need it on a scale of 0-2. I didn't feel like I needed time alone, so I picked "0". Then I wrote my entry.
At the end of the month, Daylio will send a push asking me if I want to see how I did this month. In that report, I'll be able to see a line graph of my mood along with the number of times I selected identifiers like "good sleep" or "time together", in descending order.
It would be REALLY powerful to connect these entries to Apple Health, as I track my sleep and what I eat. This would answer questions like "how does my mood change when I cut carbs to 150g/day max and 1800 cals/day max? how long does it take for changes to set in?" or "how does my mood affect how much I can push it in the gym, or vice versa?"
For my work journal, I track that in Git and have Alfred shortcuts to make that easy.
I also track weekly goals at https://retrospectives.carlosnunez.me.
Aye captain - tis what I'm working on. We should zoom sometime.
[0] https://www.apple.com/newsroom/2020/11/apple-unleashes-m1/
Still a damn noble aim, good luck!!
Looks like a cool demo with neat tech though. Maybe someone will reimplement as OSS?
The problem with relying too much on things like photos to recall your memories of previous events is that typically the photos overwrite the actual memories, i.e. you lose your built-in record of sensory impressions of events.
Used sparingly, it's probably okay, but working on improving your built-in memory is also a good idea (i.e. the whole memory palace approach).
This kind of thing could be a great aid for early-stage Alzheimer's patients, however.
As far as being used for mass surveillance and social control by authoritarian states bent on managing their captive populations in some dystopian social-credit-scored plantation-labor system, well, I imagine the NSA's DARPA-sourced 'Total Information Awareness" programs are well ahead of anything seen in public by now.
All OCR is done locally on your Mac.