I record myself on audio 24x7 and use an AI to process the information | Hacker News Reader

I record myself on audio 24x7 and use an AI to process the information

roberdam.com

788 pointsroberdam400 comments

This is known as life logging with adjacency to sousveillance and it’s a fascinating topic.

https://en.m.wikipedia.org/wiki/Lifelog https://en.wikipedia.org/wiki/Sousveillance

We in general don’t want to be watched by others, but a managed record of our own activities can be extremely valuable, and even more so if you find yourself wrongly accused. Further it can be used to shine a light on corrupt officials, one example of this is the nycplacards exposes on twitter.

The trouble with any such footage is that it can be used against you ("as the the defendant's own records show, they were present in the murder area") but they generally won't extricate you when produced by you, since you clearly have a motivation to use it selectively. So you showing a picture of yourself reading a book during what you claim is the murder night is not an alibi, because it could have been produced at any other time, and you will have a massive uphill battle in the court to authenticate that image, and risk even sink you further if you fail ("the defendant even prepared an alibi").

The only way I would accept a commercial product performing this always-on archiving is if:

1. It's encrypted by default with a strong key that can't be subpoenaed or circumvented.

2. The encoder generates its own random key upon installation which I don't know (recoding effectively random, undecodable data), and I then have to manually change the key if I expect to ever read the recording.

Number 1 allows me to review and footage and only release it if it's in my interest, and number 2 affords me plausible deniability, if I don't release the key I can claim I did not know you need to set it manually.

Sure, as long as you are the only nerd doing this, you don't need this complex setup, and you will probably get to use it the unencrypted footage only in your favor. But when it becomes widely accepted as a social norm (say, everyone wearing Google glasses), you can expect law enforcement will become aware of it as a cheap source of self-incriminatory evidence.

I‘d be interested in knowing of any cases where someone who recorded their own activities used it as an alibi. Right now it's all theoretical. Dash cams are really strong evidence in traffic court, but this isn’t criminal so it has a lower bar. From the other end, body cam footage is powerful when worn by police, and cell phone evidence by bystanders are also strong evidence.

I was falsely accused of some serious crimes and abuse.

It turns out, having security camera footage of the entire period of time showing that not only did you do nothing of the sort, but fundamentally could not have done anything of the sort, and was actually just playing with your kids the entire time, and the accuser had to have known so, changes the dynamic of such a case quite quickly.

Unfortunately, no one seems to take Perjury seriously in California, at least when the accuser is a woman. Despite it being a felony with penalties of 2,3, or 4 years in prison, and airtight evidence of multiple repeated occurrences.

Time is really critical. Alot of police investigative work is about stitching camera footage together in a timeline.

I served on a jury where a case was built around camera evidence immediately before and after an event. Of about a dozen relevant data sources, only one had verifiable, correct time. The defense was able to impeach that evidence, and the whole case collapsed. A dude got away with manslaughter.

The body cam footage is a good example, it's deeply hated by the police and a frequent source of incriminatory evidence against the wearer.

Since "you got nothing to hide", as the old saying goes, why not bodycam yourself and offer the authorities a great source of evidence they can use against yourself?

Why not just have the device occasionally send sha256 sums of chunks to a third party service, like Twitter tweets, where it's clear you can't forge the date of the message? If you need to produce some chunks, the matching hashes provide an independent time stamp showing you didn't just produce the content at any time. This sort of trick is already commonly done to demonstrate prior knowledge of something at a later point in time without having to reveal it just yet (if ever).

Great idea, but are courts tech savvy enough to accept this already?

Even with a tech expert to explain it, I worry the opposition would just get their own expert and make a whole mess of it, confusing both the judge and jury enough to cast doubt.

Perhaps I have a very wrong view on how both such evidence is presented and accepted though.

I had an awful breakup with a woman with tortuous tendencies and a false sense of how to abuse personal injury law and she keeps google home devices and car devices recording practically every moment of her life. as well as apple watch broadcasting her location at all times.

I observed she would always accuse me of things i never did in front of these cameras to get me to give false confessions. thankfully i am as blissfully honest to all people i meet, sometimes to my own detriment, so i never admit to fabricated stories.

given that context, can you point me to judicial precedents where plaintiffs had their self provided footage weakend due to the idea of fabricating false narratives with devices, selectively natrowing contexts etc.

> Further it can be used to shine a light on corrupt officials

Little Brother surveillance. It would be nice not to be surveilled at all, but since that's not an option the answer to "quis custodiet ipsos custodes?" is us.

Little Brother is one of Cory Doctorow's books: https://craphound.com/littlebrother/about/

I read it and it was as fun as it was chilling.

In times past, it was obvious that it wasn't an option to avoid pervasive violence (by orders of magnitude compared to today).

It was, though.

Also it was obviois that ut wasnt an option to avoid pervasive slavery.

Or a class system where lords and kings have more right than you do, although we are kinda bringing that back.

Who watches the watchmen? The watched.

How many Watchmans would a watchman watch if watchmen could watch Watchmans?

a managed record of our own activities can be extremely valuable

I've thought of this as a hardware product: A device that records your own voice and non vocal sounds, but which does not record the words of others. (That, plus maybe location and a video stream, provided one is in a location without "a reasonable expectation of privacy.")

Perhaps it doesn't even have to be hardware at this point! Maybe this could be installed as an app on an older smartphone?

a throat mic would do the trick

Except, if it still faintly picks up the speech of others, then that's a violation of the law, and it opens one up for lawsuits.

Since everyone is interested in the hardware:

https://www.aliexpress.us/item/3256803349510543.html

https://www.aliexpress.us/item/3256803085687061.html

the particular choice was for the battery and the other for the size, both are generic and come with the same software and bios, several vendors, if I could buy something better I would look for one that can have a lavalier microphone

I wanted to do this exact project - record audio all day and then have AI process it - to identify behavior outburst of my autistic toddler.

It's critical information for early diagnosis and treatment, but it's really hard to capture the data while also dealing with the actual situation. Being able to send the sounds he makes to his therapist could also be usefull when then are trying to get him to mimic sounds and talk.

With that said, is the audio AI open sourced? The part that analyzes the audio stream?

Thanks for the links to the hardware, also a really important part!

I would guess that they're using OpenAI's Whisper, which is open source: https://github.com/openai/whisper

It does speech-to-text, then you can use the full force of all the text analysis tools that are out there.

I've thought about this a lot.

My 8 y.o is Autistic and when he was little, I was struggling to catch evidence to provide to Speech and Language Therapy. I wanted a way to always record and have an easy way to pull out the key points.

Now I would love to correlate background noise (level and context) with meltdowns. We know babies crying set him off, as that's obvious, but would love to analyse further to spot other trends.

this one will suit you well, even has a magnetic back so you can attach it to something, https://www.aliexpress.us/item/1005003535825295.html

that's a fantastic use case!, the easiest way (and the one i'm currently using) is by upload the audio manually on :

https://replicate.com/openai/whisper

Thank you for the links and for the article. How long can record the smaller one? Actually if it can record for a day, it'd be enough for me.

I used to record all phone calls, until EU made Xiaomi remove the feature. It was very useful because I always could take notes later if they sent me a number, contact name or appointment hour.

At 128kbps the MP3 takes about 56mb per hour, I got the 16gb, so you have a lot of time, the battery of the smaller one I read is 800 mAh , according to the docs should last around 2hrs, but I try to recharge it as soon as I can

Thank you, hmmm... you wrote 2hrs, I guess it's a typo. In the page it's 20 and more than enough for my use case. And even the 4GB is overkill if you make a daily dump.

The Ali Express link says "Continuous recording:20hours". But since they offer sizes from 4GB to 32GB it's unclear which storage size that's for. That 20 hours could also be how long the battery will last while recording. But 20 hours is still enough to last the day and then some.

It's probably the battery life. 128 kb/s AAC is effectively transparent even for music, and only translates to 1.1GB. Even if it's uncompressed (1 channel 16 bit 44100hz PCM), 20 hours only translates to 6.35 GB.

Super cool project! How do you carry the microphones on your person? The big one looks like it wouldn’t clip to a shirt very easily. Does it pick up your voice from your pocket?

thanks!, I try it on my shirt pocket but now I have it hanging from my neck with a badge rope as close to the mouth as I can

Yeah but I want the software! Will you open source it? I'd contribute!

I'm doing it simple for now, transcribe it by uploading the files to colab or replicate.com, then using regex to extract the commands, the panel is in rails but nothing fancy so far.

As I clarify in the article: This is a “proof of concept” and not yet ready for production, everything described here works but probably “glued with tape”, several of the processes are probably not automated or polished.

Not as hardcore as OP, but after Whisper came out, I quickly built an app that allows me to record from lock screen: https://whispermemos.com/

This app apparently sends data to their servers. If you don't want to share this information, you can use an app like Lockflow (https://apps.apple.com/us/app/lockflow-lock-screen-shortcuts...) to put an Apple Shortcut on your home screen.

That Apple Shortcut could be the Dictate Text action hooked to create/append to an Apple Note (thereby not leaving your device) or fire off an email or send a message via your favorite bot service (Discord/Telegram/Slack/etc).

Bonus: That Shortcut will also work on your Mac.

There's also the minimal friction app Just Press Record (https://apps.apple.com/us/app/just-press-record/id1033342465), which will transcribe and has a decent Shortcut library.

Yeah I tried JPR before but missed the workflow of sending it to my email. (Maybe they have it and I didn’t notice)

Also Whisper is better for my Slovakian accent.

That looks like it's iOS only; I've been using a similar app on Android, Voiceliner, but it doesn't yet also record from the lock screen. That would definitely make it more useful!

This is a cool project. One of my pet ideas that I haven't done is to build a home assistant where all data is stored and processed by a home "server". The biggest benefit I see is that it could truly be omnipresent. There in the background, answering questions, jumping into your conversations without prompt. And it's much less creepy if all that data isn't going to someone else's computer.

Also piping in and processing the data from my mobile would be cool, but I wouldn't want to invade other people's privacy if I'm in public.

SAID this before physical VPN and open source cloud is exactly what Im trying to make a reality VM to the TVs not to mention we all need that Hillary Clinton privilege AS WELL as when you pull in at home you car uploads. updates, charges (manuel plug for now) and tie that in to the obd reader software that alone could invaluable also having two devices is the best solution one staying the car gives me a internet connection to my car for remote access does my maps or music the other does my Hud

It does not sound like a realistic capacity plan. The reason this works in the cloud is the inference can be run in parallel on a huge amount of hardware for a short time. To run those kind of models on your rinkydink computer would take forever.

An Nvidia 3090 GPU can run open ai's whisper at 17x realtime[0]. they're not exactly cheap (~$500?), but they're cheap enough that running the transcription end at home is quite feasible. And, it includes translation, so you don't have to do it in English.

Searching all of a downloaded copy of Wikipedia wouldn't be that computationally expensive either if the assistant has hot words it picks up to look up.

[0] https://news.ycombinator.com/item?id=32928207

It's also possible to use local AI processing chips like Coral or Gyrfalcon for this.

Could just load up a pcie card full of them if necessary. A local home AI would be such a boon to people, not just the average person but the elderly as well, combined with a refined GPT etc it could conversationally respond to requests rather than most assistants' current request->response "I am a robot" scheme.

>Your son called when you were asleep to ask if you wanted to get coffee today, shall I call him back for you or put you through to him? >X, you've fallen! Please let me know you're okay or I will call emergency services for you

It's sad that we have the technology to do this already but haven't.

Respectfully, I don't think that's true. "The Cloud" is just computers in a warehouse somewhere

$5/month's worth of "cloud" is going to work out to be less actual raw CPU resources than a low end raspberry pi running full time in-house

I don't actually think they're true.

One second of Google cloud TPU has roughly the same number of floating point operations then 4 hours of raspberry pi 4B time.

So 3 minutes of cloud TPU time already covers your whole month of raspberry pi usage. Pretty sure it costs them less than 5$ as well, since they have the hardware anyways.

"The cloud" is also massively parallel software. If I run a Google search, many thousands of CPUs will be brought to bear on my query, and a gazillion DIMMs, and all the throughput of a hell of a lot of SSDs, and so on. If you just happened to have a copy of the web, and an index of it, on "a computer" no matter how big, it would be impossible to get prompt answers.

If Google (or whomever) needs to run voice models, they take your query and all the other queries that arrive in the same millisecond, smoosh them all together and shove the batch into a TPU and run it. You don't have any TPUs and you also don't have any traffic you can use to amortize the cost of your infrequent queries.

The idea that you could run these kinds of ML inference tasks is economically fanciful. You would need a huge investment in hardware and the opex would be ridiculous.

I don’t think a $5 instance is enough for ML/AI workloads. You need something with a GPU.

this is also one of my pet ideas, but I keep procrastinating. Have your idea transformed into any kind of repos that we can contribute to?

> One of my pet ideas that *I haven't done*

I suspect OP was clear enough.

But there exists https://mycroft.ai/

https://github.com/MycroftAI

MORE INFO ON THE DEVICES:

https://www.aliexpress.us/item/3256803349510543.html

https://www.aliexpress.us/item/3256803085687061.html

both recorders are using the same generic bios, you have a .txt file called FACTORY.TXT, by changing the values of the file you configure the device, this is the content of the file.

---------------

TYP:1 (0:WAV 1:MP3)

VOR:0 (0:voice-activated off 1-7:voice-activated sensitivity,higher means record less)

BIT RATE:2 (0:32Kbit 1:64Kbit 2:128Kbit 3:192Kbit 4:Translate ON 5:512Kbit 6:768Kbit 7:1024Kbit 8:1536Kbit 9:3072Kbit)

GAIN:5 (0-7 record sensitivity 8 grades)

SECTION:(30) (1-999 record time exceed this,file will auto save,uint minutes)

DATE:2022-10-15 (year-month-day)

TIME:08:36:24 (hour:minute:second)

TIMER:1 (timer record 1:on 0:off)

START:08:39:32 (timer record start time)

TIMELONG:(120) (1-720,timer record length,uint is minute)

CYCLE:(030) (1-999,how many dyas,0:everyday)

--------------------------

I got the 32gb version of the bigger one and the 16gb version of the smaller one.

I configure the device to save a file each 30m, each 30m mp3 file takes 28.125kb, so around 56mb per hour at 128kbps

Thanks for the post! I find that other voice assistants (eg. Siri) are not particularly able to detect the activation command when there's any background sound (like music with lyrics). How does your system perform against this?

I understand that you're doing batch processing asynchronously and so any immediate task isn't affected; but it's arguably even more of a problem where you record a task, put it out of your mind, but then the AI fails to detect the command because it got confused by the background?

[EDIT] I see you've sort of responded to this already at this comment: https://news.ycombinator.com/item?id=33612155

How do you get the files off the device? Do you have to manually take out the SD card, put it into your computer, and copy the files over, every single day? I'd never be able to keep up a habit like that consistently, so I'm wondering if you found a more convenient way to transfer the data.

both recorders work as USB drives, once a day upload all your files, but is just drag & drop

Interesting work, glad to see I am not the only crazy one left in the life logging scene after all these years. Have been lifelogging since 2004-ish, and built a few custom bits of software and hardware to support it. I don't record 24x7 anymore, but I used to. Now my recordings are limited mostly to my office environment, and when I am out and about using a Sensecam-like device with custom firmware. When in my office I capture video, audio and depth data from multiple view points, along with images of the desktop of whatever computer I am on, and process most of it on a Jetson.

How's the audio quality on those devices you link to in other comments? I find I pick up a lot of ambient noise when outside of the office, and always struggled to come up with a viable algorithm and model to differentiate "background chatter" from the main conversation, and it is a problem I've never really managed to solve so I am interested in your experiences on the subject.

> Have been lifelogging since 2004-ish

Hopefully new advances in AI will let you try new things with your old recordings

> How's the audio quality on those devices you link to in other comments?

Decent, quality is directly proportional to the distance between the microphone and the mouth, but can't expect too much from 30$ devices.

>and always struggled to come up with a viable algorithm and model to differentiate "background chatter" from the main conversation

Yes, that's a big problem to solve, you can try Pyannote's Diarization https://lablab.ai/t/whisper-transcription-and-speaker-identi...

that will be a next step for the experience

Do you mind sharing your experience, why you started, what you want to get out of this etc? I'm interested to read your experience.

Have you seen the show "My Strange Addiction"?

This one https://www.wikiwand.com/en/My_Strange_Addiction?

No i haven't.

Not interested.

This is really interesting and many of the comments here go into the utility of this, however, verify that you aren't recording somebody else without their consent, in many places it is illegal to record to conversations without the other party's consent.

I only ran across this problem years ago when, due to a serious potential workplace issue I suggested somebody basically "wear a wire" and record their workday to catch some HR problems. We found out that the state this was occurring in had a two-party consent law and violating it was not a great idea.

This "be careful of recording" line is harmful, because folks tend to read it and assume that it is universal - there are places where it is unilaterally allowed if you have good reason to suspect not having things recorded will result in your rights being abused. Then there are places that just don't care if you do.

I don't know of any juridiction where it is illegal to record someone else.

What is usually illegal is broadcasting or making that record available to someone else.

There are many examples, for example in the state of Pennsylvania, all parties must consent. While it is a felony to record and then playback without consent (you are correct in this case), it is a punishable misdemeanor to violate privacy (broadcasting is not required) with a fine of up to $5000 in the first violation and 2 years of prison and/or up to a $5000 fine after the first violation.

Does this mean it's illegal to have an Alexa in your cubicle?

No because the terms of services and privacy policy says that you accept being recorded if you use alexa.

I think it does, but I believe that we are still waiting for someone to test that theory in court.

Expanding on the structure the OP created, this is how I see us getting to human level AI:

1. Record video sound etc... (trajectories) egocentrically

2. Analyze the data and assign reward labels (more/good, less/bad) to state and transitions actions

3. Use the reward feedback and trajectories to build the policy for some set of actions in certain environments

This is why I'm bullish on anything sousveillance - so AR cameras on your head, always on mics etc...

The challenge is doing this democratically, without it being intermediated by a giant for-profit mega corp that doesn't care about you and wants to mess with your head

Honest question,how does this make the lives of humans better?

Well, for example. Lets say that I have a goal BMI I want to maintain

If I reach for the Oreos, I can choose to have a flag set with a heuritic I created myself that will tell me:

"Having 5 oreos means you need to reduce other calorie intake by n calories to maintain your BMI"

That data can also be aggregated to give me my macro/micros for everything I've eaten etc... without me having to log it like I do now

Think about it as the ultimate personal assistant and all you need to do to instrument it is attaching a camera and mic to your face. You can decide what your goals are, and this kind of instrumentation will capture the data that you need without you having to actually annotate everything.

Your personal life API

> all you need to do to instrument it is attaching a camera and mic to your face

It's funny that this is a reasonable thing to say.

I've been thinking about doing this for a while now, cameras all over the house hooked up to ML algorithms that help you audit and tweak your behavior towards some specified goal.

When I used to play video games ultra competitively, I would analyze recordings of my gameplay to try and get better, and it worked wonders.

I think the oreo thing is probably doable now with things like habitaware. It’s most likely a very easily distinguishable motor pattern. Not sure they could be programmed to give you an oreo-specific-reminder, but that’s a design gap more than it is a technological one.

I'd just get really annoyed at that AI

...being annoyed increases stress, which increases appetite

Sounds like a nightmare!

Another challenge is industry-wide bad security fundamentals.

Godspeed to work/people like agoric.com and seL4.

All of the rooms/corridors in my house except my bathrooms are covered by cameras. My initial motivation for installing them was to keep an eye on what my pets were doing when I'm not around, but I find in recent years that if I misplace something, I end up tracing back my history on the cameras and finding where I left it.

It seems obvious that at some point, AI will be able to do that for me and I'll just be able to say "Alexa, where did I leave my glasses?", "Hey Google, where did I put my box of spare fuses?".

I would 100% prefer to lose my keys rather than letting Amazon or Google in.

FYI: I have zero Alexa/Siri enabled device, zero automated home device, a degoogled phone, etc etc. So we might have different perspectives on the matter.

Each to their own. Personally the value these cloud/AI assistants give me is worth the loss of privacy. There's nothing I do that I think anyone would be especially interested in spying on, other than to try and sell me things.

Note that I don't think anyone should be forced into this sort of surveilance. It should always be a choice. I also support the open source projects to bring it back to individual control - it's just too much hassle for me, personally.

There is no reason this technology needs to rely on consumers sacrificing privacy. The big players are trying to create that perception in the public so consumers will willingly sacrifice their privacy regardless.

The tech is there so someone could make a box with no external data transferred that could store and analyze video data. I would be a customer for sure for something that had this capability without the privacy concerns.

Google and Amazon say they want this data for quality control, but I suspect each of them have plans (if not active projects) for converting video inside people's homes into actionable marketing data.

> the value these cloud/AI assistants give me is worth the loss of privacy

they've got you right where they want you.

Just because the data isn't interesting to anyone right now doesn't mean that a future oppressive government won't use it against you

> There's nothing I do that I think anyone would be especially interested in spying on, other than to try and sell me things.

Do Uyghurs have something to hide and are worth spying on? How many times are we going to hear this argument? It comes only from a position of privilege. You're only uninteresting to be spied on as long as it's allowed by the security apparatus you depend upon. There's a reason we have sayings like "power corrupts"; dismissing the potential for abuse of a cloud-based unencrypted surveillance system is narrow-mindedness at best and subversion at worst.

Note: the above hardly represents me politically, it is just a counterargument against the perennially repeated "I have nothing to hide."

i mean you could always run a home server for the automated home things. heating/ac and lights are nice things to automate

Because spending tens of thousands of dollars in home infrastructure to avoid fiddling with the thermostat four times a year definitely makes sense.

controlled heating for my flat with open source and Zigbee compatible devices would cost me ~1k. I did not calculate the ROI but break even looks like it’d take many years.

Curious: are you concerned about data leaks or you don't trust the employees to not access your user data? Or something else?

Not who you asked but: I’m afraid of the data being stored and available to anybody. As long as it’s out there, the government can compel others to give it to them; and companies can get acquired and structures and laws can change in such a way that the data gets in others hands perfectly legally.

Thus, I should only be okay with it if I’m okay with the “nothing to hide” argument, which I’m not.

Yes, yes, and I don't like anything home automation to be dependant on anything cloud. Enhancing function is fine but house that stops working right the moment internet link is down is a dystopia.

Both and more. Having my data sold to 3rd parties is an obvious first. And if you think the terms of service are enough to cover you, see how fast they can change in everyday life and please reconsider. Plus, data can be sold pseudo-anonymously and build up a profile against which your identity is compared and metered, as in, for example, health insurance risks or crime potential.

Additionally, we, the consumers, have lost the right to own things. Or at least, if we do own things, it comes with all sorts of strings attached in the form of "features" or "connectivity". Which is just marketing lingo to say that you're feeding the cash cow.

Nah, is it much more efficient to distract you into loosing your box of fuses and manipulate you into buying new ones.

Alexa AI doesn't work for you, it's a hired gun in your house.

Just like 2001: A Space Odyssey - but instead of Hal trying to kill me, it just tries to get me to buy things I don't need.

"I'm afraid I can't do that, Dave, not until you watch this advert"

Wish they’d at least release a Christopher Walken package.

“I can…NOT find your ANswer.”

I agree that it is an obvious extension for AI to use this data at scale to help users. It also is obviously a huge temptation to abuse it for other purposes.

Wasting a few minutes in the morning to find my glasses is a small price to pay to not be watched and analyzed all the time. Let's not build our own panopticons.

When you do, can you invite me over to show me how it works? Then I'll test it for "Alexa, in which mattress does cameronh90 keep their savings?".

"Voice not recognized. Releasing the robotic attack dogs"

Like pretty much everyone in my country, I already entrust a bunch of private corporations to safeguard my wealth. Worse, there's nothing to stop my bank from suddenly deciding tomorrow that I don't have any money, and I don't really have any paper records to prove otherwise...

I figure any AI advanced enough to monitor everything I'm doing and where all my stuff is, is probably smart enough to know if it's me asking.

Just having a front door cam paid off immensely this year when I was able to prove that I had left the house with an item (that I later misplaced, and was able to recover with that knowledge.)

Excellent idea. You can later search through your logs in the future for reference. As it's all in text.

Prior solutions posted on the net, had this take photo / record audio 24/7 features, but then those were stuck there. What next? What would anyone do with these data?

But this Hi Jarvis styled recording of text on the go is a very useful feature.

Another step ahead.

I've wanted to do the same thing with my online activity as well. Chat logs especially. They tend to go into a void and finding an older log is weirdly difficult. I've wanted to log everything and then be able to apply better search algos (semantic search perhaps) to try and make my chat logs useful.

Cellphones are placed amazingly well to provide this sort of search. Seeing the post about BeOS and its amazing metadata-driven BFS filesystem yesterday really makes you think what might have been had iOS and Android been more ambitious about filesystems instead of just re-applying the same old conventions from our desktop computers.

You should be able to just text search every phone call you have made on iOS/Android, today, similar to the automated voicemail transcription features already present etc etc.

I think the "total recall" search can be a killer feature

I remember an Asimov short story in which scientists developed a machine that could see backward in time.

If I recall correctly, the upshot was the government became terrified because any machine that can see 1000 years into the past can also see 1000 milliseconds into the past and therefore functionally be used to spy on anyone in real time.

There was an article some years ago (2 or 3?), that described a drone (or drones?) that flew 24/7 over Mexico city taking high resolution video of the entire city at all times.

Whenever there was a crime, the police could zoom into that location at the time of the crime and then run backwards to see where the vehicles came from. They then knocked on that door.

I'm disappointed that I can't seem to find it using Google anymore, maybe it was from a movie or TV show?! That would be weird though, because it seems technically quite reasonable to achieve and hard to believe governments wouldn't jump on it.

The term for this is 'WAMI' - Wide Area Motion Imagery[1]. Here's a Bloomberg article about an instance of it in Baltimore[2] (although this wasn't where I learned about it first, like you I can't find my original source either)

1: https://en.wikipedia.org/wiki/Wide-area_motion_imagery

2: https://www.bloomberg.com/features/2016-baltimore-secret-sur...

I remember the same article, although I can't find it now.

These two seem to reference the same demo, although neither are the article I remember: https://www.bloomberg.com/news/articles/2016-08-23/watch-thi...

There's this reference to it: https://www.pressreader.com/usa/the-washington-post/20140206...

https://www.bloomberg.com/features/2016-baltimore-secret-sur... has a lot more details

I've read about that happening in Cleveland, using tech developed to find insurgents leaving IEDs in Afghanistan. Yeah, citation needed...

https://www.pss-1.com/media-contacts

Not sure about the Mexico City drone, but a similar thing was developed by the US military: https://en.wikipedia.org/wiki/Gorgon_Stare

I know some folks who deployed during OEF/OIF and used these types of systems. Many a night raids were conducted simply by watching where attackers originated from.

Different author, but sounds somewhat similar to 'The Light of Other Days' by Arthur C Clarke and Stephen Baxter.

Although iirc correctly it starts with being able to see other locations in space but at the same time, and the historical viewing is a second development.

Fantastic book, even if it's not the same one you were thinking of.

Pretty sure both those authors wrote similar concepts, with the same creepy conclusions of taking the technology to a limit.

It came up in an acoustics class once. I said that sound never really dies. It just bounces around until it becomes thermal energy, thus warming the room a little as a prelude to joking about professors talking hot air.

A student asked whether, one could recover sound from reverberations that had fallen below RT60? Could you listen back in time to conversations that had happened hours ago?

Obviously entropy can't be put back in the box with the technology we have now, but it makes you wonder.

Two things have since made me revise the question. One is recovery of sound from video images. The other was an archaeological recovery of sounds from a ceramic vase spun on potters wheel many centuries ago. Sorry but the references for both escape me atm.

Clark also used it as a throwaway line in Childhood's End. IIRC, humans were given a device that would allow them to see the past--most religions didn't survive seeing the true origins of their faith.

It was The Dead Past [1]

The idea of it was that it was known that the technology existed, but the government went to great lengths to imply that it could only see into the far distant past. The reality was it could only see 20 years back or so, and the government was covering it up because of the 1000 milllisecond issue.

[1] https://en.wikipedia.org/wiki/The_Dead_Past

Yes, that was it! Nice find :)

I wonder if this was an inspiration to the "Devs" miniseries. Won't say more about it for fear of ruining it. Amazing show.

Sounds very similar to the guy talked about in Albert-László Barabási's book (either Bursts, or Linked ... don't recall which atm) - he was photoing/videoing his whole life, but never of himself - ie, the camera was always facing outward (like a policeman's bodycam)

The entire topic and many posts in this comment page also sound like things straight out of The Circle and The Every by Dave Eggers.

I had forgotten about The Circle :)

I did an experiment where I lived for awhile with a sony recorder/mic on me 24/7. It was nice to be able to refer back to conversations and events when I wanted them. Biggest issue was sorting through the data-- timestamps and recorder bookmarks were OK but I really needed full text search on the audio. It would have been great to tag via `Robert, mark timestamp, end Robert`. AI seems to be required, especially when dealing with wind noise and other issues (like the mic twisting around and all of a sudden one channel is my heartbeat.)

The sony voice recorder out there easily last 24 hrs on 1 AAA battery.. dumping to mp3 on a large sd card.

I did a similar experiment in about 2005 using a small iRiver iFP [1] and reached the same conclusion.

It needed a physical "Something interesting just happened" button that could be annotated later. At the time, creating custom hardware as well as the entire software/service stack was more than I was willing to bite off.

The iFP is tiny, roughly a 4" long by 1.5-2" cylinder. It easily covered a full day, the silence detection worked great, and quality was fine when used in a pocket or on a belt. Basically, the stuff that I expected to be difficult was already solved.

[1]: https://en.wikipedia.org/wiki/IRiver_iFP_series, https://www.cnet.com/reviews/iriver-ifp-790-digital-player-r...

Excellent. Just terrific.

My future perfect system also logs my location and what I'm doing. And probably health metrics too, like heart and breathing rate.

Instead of initiating my exercises, I just want to say "Robert, start jog". The "modal" nature of my Apple Watch's Activities really frustrates me.

I don't want to take notes while I'm listening to a podcast. I'm generally doing something else at the time. I just want to say "Robert, bookmark". And magically a link will be made to whatever I'm listening to at the time. (Audio book, radio, stream, podcast, whatever.)

Ditto identifying songs (Shazam!).

I don't want to fart around with exchanging contact information. My hands are usually full or whatever. Just say "Robert, contact info" and then repeat out loud whatever I hear.

I also want to rewind after the fact. When trying to recall a tidbit, I'll remember the song, where I was (eg while walking the dog), who I was with, what I was eating. So if I want to remember which podcast I was listening to while at the park, I'd just start with my location log and jump over to my podcast listening log.

What could be more simple?

FWIW, I'm still waiting for my "bicycle for the mind".

PS- I've tried, half-heartedly, to use the voice recorder app, and notes with voice transcription. But then it quickly becomes a treasure hunt. And my attempts to do this stuff with Siri just leaves me more frustrated.

Thanks for listening.

Great project. Please keep us posted on updates.

thanks!, you should try to transcribe your recordings now for free with whisper and see what you can make of them: https://replicate.com/openai/whisper

I've been experimenting with this recently as well, but with an app on my apple watch. Looking for a method/model to split different speakers into different tracks to only look at audio from myself and certain people.

Someone is experimenting with diarization (speaker identification) + Whisper here https://github.com/openai/whisper/discussions/264

If you know how many speakers there are, https://twitter.com/dwarkesh_sp has it working here:

https://colab.research.google.com/drive/1V-Bt5Hm2kjaDb4P1RyM...

Ahh I’m working on exact same project. I applied to YC with the idea and was told that “nobody wants this” during the interview.

There’s a ton of problems in the space around privacy and UX. But I’m incredibly excited about projects in this space because in modern society we’re basically surrounded by a million unhealthy things designed to tempt us. Logging forces you to “stay honest”. I’ve been shocked already by how many unhealthy habits I underestimated and how many healthy habits I overestimated.

My #1 priority is just to improve my own physical and mental health. Whether there’s a market for this stuff, who knows.

Good luck!

My original inspiration is to better understand how I talk to others and study my own behavior

A noble goal. One of my bad habits I've been tracking and trying to reduce is rude behavior to people, online or in-person.

Check out this model, I've had limited success with it. Best I've done so far is to just add the labels it gives to the overlapping segments whisper spits out, which means some sentences have multiple speakers, but that's mostly the case because of cross-talk. I'd say it gets it right ~80% of the time with the 5 speakers I've done it on across ~16 hours of audio.

https://huggingface.co/pyannote/speaker-diarization

I will!

Speaker identification is the next step, you might want to read about Pyannote's Diarization:

https://lablab.ai/t/whisper-transcription-and-speaker-identi...

we're experimenting building out a version of this too, but on desktop with www.usebacktrack.com - should have splitting speakers/inputs early next year and seeing what that's like

what app are you using on the apple watch?

Here's a 24/7 background audio recorder app I made for Android. The impact on battery and storage is surprisingly reasonable.

https://github.com/miguelrochefort/eardrum

I like this. It vibes with a language learning app concept idea I recently shared out loud.

https://twitter.com/kuizinas/status/1591867392220594183

I've been doing this with Anki.

When I have a conversation with someone in a language that I'm learning (was Russian and Greek, now Arabic) I record the conversation. I then get both native-speaker audio to add to Anki for the things they said, plus I get a list of words that I either needed to use or that the other person used, to add to Anki.

A secondary benefit is that this system encourages me to go out and seek interactions with people, a clear benefit for a natural introvert.

Well done!

Got a similar PoC that uses Tasker to record sound on my phone, Whisper to convert it to text, and neatly organizes everything into Obsidian.md. The continuous recording kills the battery life on my phone so it's only usable if you don't mind going around with a powerbank. Would be great if a manufacturer would put in a separate low-energy chip with a good ADC.

P.S. "Active functions" with custom home automation is easy as pie with joaoapps's suite. I use BusyBox to SSH into a Pi with a Tellstick Duo. And some RFID tags for the system to know where I am (e.g. bedtime routine gets triggered when I place my phone on the bedside table). But yeah...traffic goes thru Google.

you should write about it!

How would this work with other voices, like a coffee shop, would it hear those simultaneously, and interupt a command?

Also, how do you handle using OpenAi whisper, seems like they do 30 second intervals - would that be an issue if your command is cut off mid word?

For now I try to give the commands when there is not much noise, but you can lower the gain of the microphone so that it only record my voice.

The 30 second limit is not a Whisper model limit, but a limit some of the free online "try whisper" put.

I think he means that even whisper segments the audio into 30 second bits and does transcribing on them and then stiches everything together.

The future will definitely have devices which record visually/verbally all your life. VR headsets are already able to record all your facial expressions. A google glasses like gear which records all your life is pretty much possible in the near future. The future influencers won't have to carry a phone/camera to create vlogs, they would just see wherever they want and the glasses will record not only the thing they are seeing but also their expressions. Privacy will probably not be such a big thing as now given most people with each generation are increasingly becoming more and more comfortable sharing their whole lives online.

Ted Chiang explores this idea in a short story called "The Truth of Fact, The Truth of Feeling" (https://devonzuegel.com/post/the-truth-of-fact-the-truth-of-...), which takes place in a world where commercial, individual, always-on recording exists. Ted Chiang also wrote the short story that the movie Arrival was based on.

Halperin's debut novel _The Truth Machine_ also has people living "documented lives", with always-on video and audio recording, as a primary plot element.

> Privacy will probably not be such a big thing as now given most people with each generation are increasingly becoming more and more comfortable sharing their whole lives online

That's not quite right. People are just unaware of the power of what they share and usually react quite negatively if their data is used against them and start shielding themselves from surveillance of the sort that might affect them.

So predicting "people won't care about privacy because they share their whole lives online more and more" is a bit disingenuous.

It's what the tech companies like Facebook want but it's far from the truth.

Reminds me of the Black Mirror episode “The Entire History of You.” Could be pretty scary if misused.

I've thought for 20 years that the Life Recorder is inevitable. I figured it would be like journaling constantly, getting insight and guidance to improve.

Now I think it will result in unbearable self-consciousness. You will yearn to be offline, quiet, to just forget, maybe to enjoy the moment without it going in your permanent history.

Arguments in relationships are messed up when you can rewind and debate what he said she said. The actual words are often not important, it's the emotions. The permanent record makes it harder to forgive and move on. It's like being in court, everything transcribed.

> given most people with each generation are increasingly becoming more and more comfortable sharing their whole lives online.

There is an entire generation who learned not to post, many who are very anti social media, many who stay anonymous. Chat is much bigger than public social these days.

There are also scenes that avoid digital. They make cassette music and black and white photocopy artwork.

Arguments in relationships was one of the first “off-label” circumstances that came to mind when I saw this post.

Thinking about it a bit past the initial reaction though, I think it’d actually be a massive boon for relationships. Sometimes we get so hung-up on the wording, that may or not may not have been clearly expressive, but due to some slight subtlety, changed the course of the interaction from a potentially positive one to a toxically negative one. I think if you can just subvert the “you did say that! You don’t remember??/No I didn’t/Yes you did” you can focus on the actual content of the conflict resolution.

I’d personally want to know if I said X-potentially-hurtful-thing. Or was the other person just hearing it? What’s workable, what’s not workable. Post-mortems for arguments (assuming the relationship is a viable one, ie genuinely collaborative & not contingent on point-keeping) would be a lot easier & constructive conversations can be realized much more quickly.

I'm not sure people are increasingly willing to share their whole actual lives online, they are mostly presenting a curated and tailored image of themselves to project the self that they want the world to see. 24/7 straight-to-internet type recording doesn't serve that goal very well. I'm not sure that something like this would be popular among streamers, other than maybe a small niche for who that is their whole thing.

I predict future generations (maybe even those being born right now) to start moving away from recording and uploading everything as it's the uncool thing their parents are doing.

Current gen Tiktokers ran away from facebook and instagram, not from oversharing on social media. I think teens will always be very active on some kind of social media, but yes one can certainly hope that all of it is seen in some negative light by the coming generations.

After reading The Circle, I'll definitely pass on sharing my whole live online.

Awesome idea. However people would find it weird that I talk to myself all day long.

Yeah, I can see how this would be easier if you work from home, but you could explain that you are running a long lived experiment on yourself. Then again, that doesn't exactly scream "I am fully sane" :)

Well, genius borders madness, and it's been said that talking to oneself is a sign of intelligence, so you may be right.

A case for neuralink.

This is awesome! I've been recording myself (video/audio) for the last couple years on and off (thousands of hours) and have no efficient way of processing the info. Was not aware of Whisper and what he's done is exactly what I'm looking to pull off.

The GPT-3 idea is scary and most certainly the future. I can't stand the world of never ending 'Moviefone' menus and chatbots, but when it's me that gets to be the machine response the future doesn't seem so annoying. Would be nice to have my own GPT-3 model that I can use to "get to a real person" when calling places.

The passive information would be useful if that would work with your inner voice.

I read a research paper quite some time ago that most inner monologue is articulated at least partially by the mouth, throat, & tongue, and can be detected. Might be a better approach than having a chip in my brain.

If I find the article I will update.

Anecdotally, my inner monologue certainly is physically articulated. It would be really interesting to build tech that can understand it.

what's fascinating is apparently there's a connection with schizophrenia, where the "voices" are actually the person saying things to themselves at a level too low for other people to hear

also suggests a throat mic would do wonders

Love this idea. Most people subvocalize when they read, and my guess is the same for when they write. As for subvocalizing when we think, I have no idea, but I suppose if one were engaged in “talking to oneself” type thinking, that it could be possible that many subvocalize their internal monologues.

Is there some kind of sensor that one could wear that would be unobtrusive enough to measure subvocalization? If so, building a training dataset for an ML model would be as simple as having many people read and write a significant amount of text while using the sensor.

Even better if that text corpus overlaps with text that has been used to train text-to-speech models like Whisper, as you might get away with knowledge transfer with such a model.

It’s definitely worth looking into!

With feedback, it is likely possible for you to train yourself to subvocalize your thoughts as an act of recording. I thought this sentence before writing it, but I subvvocalized while typing.

That would be creepy and next level.

Isn't that what Musk's Neurolink is trying to achieve?

That technology doesn't exist yet.

I have recording turned on on my phone. Usually it records 6 hours at a time, so it is annoying that I have to manually restart recording. Another annoying thing is that it will pause the recording when I pick up a call.

Why Google decided to block call recording is beyond me. In the past when I was able to record calls it saved me a lot of trouble - for instance when insurance company lied to me over the phone about the product I could confront them about it and get my money back. I wish I could be able to record calls with my relatives as well. Call recording is legal in my country.

So the author is recording all their interactions with everyone they meet and then processing and analysing those intetactions. How is this not a massive invasion of privacy?

I see he is concerned about his own, but he doesn't seem to be at all concerned with anyone elses. Personally if i discovered that someone I interacted with was doing this I would insist that they deleted any data concerning myself they had gathered and pursue whatever means were available to me if they refused.

I was pondering this as I read, as well. I would be a bit disconcerted to find that a coworker was recording EVERYTHING they say all day and that meant I was on those recordings throughout the week. The lack of any acknowledgement of that piece in this writeup is a little confusing... I don't question their intent but I do question their lack of awareness of others (based on the writing).

I am using it only with friends and family around, and tell them beforehand about the experiment

Very interesting idea.

Would you be willing to share more info on the tech used in the process?

>I bought a couple of Chinese microphones

Which exact microphones? How long does their battery last?

As well as other parts of the process.

https://www.aliexpress.us/item/3256803349510543.html

https://www.aliexpress.us/item/3256803085687061.html

Not the OP but I've been tinkering with the same concept (24/7 processing).

'm using vosk browser: https://github.com/ccoreilly/vosk-browser

To do speech to text locally and it works very well for English.

Browser models are too small, unlikely they recognize accurately. They are more for simple predefined phrase.

You can probably try vosk-api on the desktop-grade machine. You need big models from https://alphacephei.com/vosk/models, they require like 8Gb to run but they are much more accurate.

What a bonanza for opposing counsel of any kind.

(which is a bummer, as there are lots of interesting uses for digitizing our lives if the data could be guaranteed to remain private)

An off the shelf solution for recording your whole life:

I have a Sony recorder, ICD-UX570, and it has a setting where it turns on or off based on sound, and also adjusts the gain to best record. It takes a micro-SD card and has pretty solid battery.

I think you could put it in a breast pocket and run it for several days on a single charge. Because it would just record when you are talking or making noises you could likely run it for a year on a big SD card recording in mp3.

Change to a wifi SD card and suck the files off and process them and you might have something kind of cool.

Would the sound of jostling in your pocket/bag not set off the recording?

I'm not sure about jostling, but lots of noises would set it off. It's erring on the side of recording when there is sound, and it can't tell speak from not speach, though you can set a sensitivity level.

A related question, is there a ready solution to do constant recording using some Linux box (e.g. Raspberry PI)?

AFAIR I've seen someone recommended such software on HN but I can't find it right now, it was something for recording radio stations or similar.

I would like to get some kind of sound monitoring of my house when I'm away or sleeping and besides using arecord I couldn't find anything useful.

Even better is something like the Guardian Project's Haven(https://github.com/guardianproject/haven) which IIRC Snowden contributed to.

It's incredibly cost-effective to just buy an old Android phone (which comes integrated with multiple microphones, with good signal processing and noise cancellation), instead of building it with components.

Haven is specifically designed for intrusion detection, and for preventing people from tampering with your laptop for instance by detecting activity on the Android phone's sensors.

Interesting concept, but the issue is that one would need to constantly charge the phone and batteries have a tendency to go bad after some time with such usage.

An old laptop will work better.

Raspberry pis don't have Audio in. You need USB microphone and drivers, which are hit or miss.

I was playing with this idea of recording everything using a 1st gen AIY voice kit. The driver is a linux kernel builtin module. The recording quality is good if I'm in the same room. But I didn't find the recording that useful, so I stopped doing that.

Well yeah, but I wonder what software to use for constant recording that could be worked on outside of the recording process.

As you'd be recording all of your conversations, this is illegal under some legislations, unless all your conversation partners agree with being recorded/their convos being stored.

In some countries it's enough for one party to consent, which can be you, so it's legal.

In the US it even varies by state, which has gotten people in trouble with recorded phone calls.

I am using it with friends and family, and tell them beforehand about the experiment

What about going out to a restaurant, a party, a date, or even somebody you bump into on the street? You're recording a lot of people, which to me is a huge invasion of privacy. What do you do if someone is uncomfortable with being recorded?

Is it illegal to record it, or is it simply inadmissible in court?

Interesting article. Thanks for posting this. I think when wearing something like google glass and recording everything the potential is even bigger. The AI can extract so much more context. Analyse faces, gestures, locations and more. Dystopian and yet so interesting.

This sounds horrible. Nobody should freely record and analyze others without consent. This is not only dystopian but also very rude and possibly illegal.

As far as recording oneself to capture thoughts, processes, this is a fine idea that I'd like to give it a try.

Are you planning to combine it with other info? Your smartwatch already knows how long you've slept, getting that info directly to your database seems more efficient and less error prone. The same goes for the amount of money you've spend, if your bank allows you to export that info it'll save you a step. Your bank doesn't know what you've bought, only the total cost, time and shop, but if you scan and upload your receipts and use ocr you'll have a detailed record of that too.

And you could also keep track of your location, so you know where you had a conversation or at what gas station you spend 250,000

Diarization might be my next step,(recognizing the speaker on the recording).

Combining the information from multiple sources as you say will get you a complete view (location history and time of the recording will let you know if you where speaking with a college or your spouse for example)

If you are going through that much trouble you might as well get a WiFi scale, wear a tracker that has an API, etc. I’ve definitely thought about taking speech-to-text notes at work, nice to see somebody did it.

The problem is all those gadgets have their own APIs and quirks. For example, my WiFi scale often decides to silently not sync to my phone. Or decides to update my partner's weight instead of mine. OP is building on arguably the most natural API - speech, which is what all those smart assistants have been promising us for a decade. I think there is a lot of convenience as well as unexplored ground to be found in such a system.

Well yes, but this is a much cheaper option. Instead of having many smart gadgets you only need one. The mobile phone or some other microphone but that is the most obvious option.

Instead of paying hundreds of dollars for all these gadgets that have to be charged and kept safe you can buy cheap variants and still have basically the same benefits.

But your time is exchangable for money. (Something almost never reflected in hobby behavior: that's why the crafts store has 130 kinds of acrylic yarn and 1 kind of wool yarn.)

A $150 scale is expensive but buys a very small amount of software development.

Interesting to see comments suggesting use of constant recording as a defense against invasion of privacy via constant surveillance.

Interesting that the defense against the harms of technology is technology itself. When the humankind unlocks a new powerful technology, and when it is possible for criminals to use the technology to harm us, our best course of action may be, not to look away from it out of fear, but to spend more time understanding it and its implications, and to get there faster than adversaries do.

Handling the privacy of other people might be oddly easy. If you can detect the voice accurately enough the AI might be able to _drop_ the other participants.

Dropping after the fact still means it was recorded, this violating two party notice statutes.

Does that mean that every phone recording audio to detect its assistant trigger phrase villages two party notice statutes?

What if it was never written to disk? The identification and trimming happening before any audio is actually saved.

Is picking someone else up on microphone at all a violation of two party consent? For example walking past someone in public with a loudspeaker call active.

This reminds me about Black Mirror's White Christmas episode where they create a digital clone inside a white "cookie" and then they use it for receiving tasks such as making toasts. This project is very similar actually. I found very interesting the part where you can track your food and automatically calculate the calories every day without writing anything anywhere.

I'm curious if there was other work you were inspired by. I have also been a bit interested in using this style of "personal database/logger/journaling";

task-agnostic input -> processing -> visualization/recall

My assumption is you are just storing post-processed conclusions in a local db on your computer + raw audio for possible future re-processing, and not currently storing other media input (ala food pics)?

I'm trying to catch up with all the opensource AI stuff out there and explore the posibilities, on the spanish part of the website is a test with stable diffusion and twitter, and now I'm trying to finetune Donut document transformer.

I'd be so self-conscious with my speech being recorded roughly 24/7 by myself. I'd probably get used to it but it'd take some time.

People who take notes in life (the org mode people): oh cool. Everyone else: why would I want to know what I ate, weighed, or thought last week?

Great idea. However always recording is a disadvantage for me.

I thought about a device that could look a bit like Star Trek badge. It should react to pushing it slightly and it would have a microphone. It would connect to a phone with Bluetooth.

Main use for me would be push-to-talk as I use Zello with my wife quite a lot. But all those reliable assistant/voice-notes uses would be also sweet.

The advent of Whisper gave me a similar idea, except instead of uploading the recording once a day, it worked by calling my computer from my phone and recording and processing in batches. Realized pretty quickly that most of my day is silent though, and would rather be able to trigger it on demand, which I haven't gotten around to.

Pretty sure Bill Gates wrote about this idea in his book. It SEEMED like the future, but software/hardware innovation goes where the money is, and no one is interested in recording their own lives. Maybe once the AI to make use of it gets better, it'll find product-market fit.

In some jurisdictions it’s illegal to record a conversation without all-party consent. Example:

https://www.rcfp.org/reporters-recording-guide/massachusetts...

> The law only applies to secret recordings, however, so affirmative consent is not necessary when all parties are aware of the recording.

Hm. Will a T-shirt with “my phone is recording everything” be enough?

Interesting- secret recording is punished more severely than use of such recording. Logic?

Also, the difference between image and sound recording. Secret image recording in public space is basically ok, while sound recording is not. That probably is caused by a wide availability of photo cameras, with known “fair uses”.

These states are: California, Delaware, Florida, Illinois, Maryland, Massachusetts, Montana, Nevada, New Hampshire, Pennsylvania, and Washington.

Edit: I should add that in some of those states, it would still be permitted to record others in public without consent, where there's no reasonable expectation of privacy (e.g. a coffee shop or gas station).

Obviously the solution is for the government to record audio in all places at once, and then those with $$$$ can just pay for the audio feed. Win/win.

Pretty well known in the states, luckily he seems to be talking to himself :)

I think the passive part of this could be really interesting - starting with a simple "tag cloud" of keywords by frequency linking to audio snippets that mention them, it'd provide a way to index conversations during the day for future reference (or processing).

Further to indexing conversations, it would be interesting/helpful to be able to pull out:

  * conversation length
  * participants
  * location

So you could search for, say, that long conversation I had in the park with Bob.

I'm not sure how easy it is to identify/track different participants in a conversation.

Edit:formatting

indeed, that I guess will be the best part of the expermient, but the longer one.

Anyone familiar with rewind.ai which seems to be building a product on similar lines?

Looks like a slick wrapper around Apple's ImageAnalyzer and ScreenCaptureKit

Anybody made some progress with using google assistant with arbitrary commands? I know there are a few integrators online that could, in theory, get commands and send them to a spreadsheet, but I couldn't get them to work.

I liked this article and find it intriguing. That said, I would set the original sound data to expire relatively quickly, perhaps erasing everything week or so. I like letting the past be the past.

Would you mind linking/listing what microphones you're using?

I bought two, both from Aliexpress, no brand both, the one on the picture has a 5000 mAh battery, bulky but last a lot, and the other one is tiny but with short battery life, a lot of sellers on Ali, I pay around 30$ for each, both have the same software and bios, only difference is the size and battery.

for a while I had my laptop set up to take a photograph and and screenshot every ten minutes. The information was completely useless, but I got some great candid photos of myself

Are there any good (discrete) wireless throat mic patches out there that are sensitive enough to pick up subvocalization / whispering?

I was curious about this as well. It seems there are bone conducting microphones, but the ones I've found so far go in your ear (so it's visible). I'm going to just hide a mic in my beard.

Are you planning to make it an open source project?

for now it's "glued with tape", but I'm going to try to make it presentable to post something on Github

Please do, this is a very cool project and I would like to give it a try.

I'm looking forward to it!

I do that as well. I had a few arguments with police patrol over driving tickets. Once dog attack and very aggressive dog owner.

I thought this would be recording everything to generate an AI model that could sit in zoom meetings for you.

Did I miss something or is a description and/or link to the software used not in the article?

You can use you smartphone for the recording and transcribe it here: https://replicate.com/openai/whisper

regex to extract commands from the transcripted text

You can download whisper at https://github.com/openai/whisper

That calorie tracking will be WAY off and useless if not dangerous. Nothing beats a kitchen scale.

Very cool idea, the only question I have is how fast does this not drain the battery of the mics?

the device from the picture has a 5000 mAh battery (around 10hrs), and and bought a smaller one (1-2 hrs)

I must be blind, where do you list the microphone(s) you are using?

This is the sort of thing Nietzsche alludes to as his "last man".

What are the limitations of numbers as descriptors of Being?

Doesn't this break wiretapping laws (depending on the user's geographic location) and possibly GDPR/NDAs(if left on while at work)?

Ethically most people probably wouldn't be happy to find out that you recorded a conversation with them.

I am using it with friends and family, and tell them beforehand about the experiment

Forgetting stuff is a feature, not a bug.

Depends on the human haha for me it is for sure a bug because it is way to prevalent...

Why is this being upvoted ? There is no code

Did you think there was a requirement to have code?

So where was the remote control ?

yeah but there's 3 toasts

Why are you so self-obsessed?

Why are you so mean-spirited?

Too bad Abbot and Costello aren't around to attend a standup with a bunch of people using your app.

  - Robert Robert what did you do yesterday End Robert
  - Robert I met with Robert in accounting to finish Jira 12392 story on the Robertson patches End Robert
  - Robert Then I took a break to have coffee and watch a Julia Roberts short with Robert in System Admin End Robert
  - Robert Robert what are you going to do today End Robert
  - Robert It depends if Robert Roberts has internet access End Robert
  - Robert If he doesn't, that'll be the end of Robert Roberts End Robert
  - Robert No impediments for either Robert in the Robert epic End Robert
  - Robert Robert's on Help Desk, Jean Robert's in Code Review, and Sam Robertson's running the Roberton's retrospective End Robert
  - Robert But then who is Robert Robertson? End Robert
  - Robert Oh Robert Robertson's our scrum master! End Robert

I'm a bit concerned about the calorie level I see here, 832/day. That is about 1/3 of the NHS recommendation [1] for males.

1. https://www.nhs.uk/common-health-questions/food-and-diet/wha...

The lamb sandwich, orange juice and almonds alone could approach 800 calories, even excluding the other three meals. People are quite bad at estimating calories though, it takes practice.

It's typical in calorie tracking that people start the day strong, but then forget or lose will to track later in the day. Ideally days without a full record should be excluded; I'm assuming that's just not happening here.

I agree, but looking at the actual food eaten, it doesn't seem to line up.

Those recommendations are for people that are on their feet all day, not for office workers and home dwellers that never go to the gym.

I think your last sentence summarizes the sentiment really well: “The difference between utopia or dystopia is who has access to that information”

> My biggest problem with “OK Google” is that I don’t know by heart what it can do interactively

Maybe it’s just me but this feels unaddressed and that seems ridiculous.

Why is it so hard for me to find a single, precise location on my phone with an enumerated list of every command Siri or Google can work with?

> Why is it so hard for me to find a single, precise location on my phone with an enumerated list of every command Siri or Google can work with?

The likely answer here is that the engineers who work on such products would scoff at the idea that their work amounts to a simple list of commands. In their minds, they’re working on a natural language virtual assistant, whose understanding of user input is “intelligent”, and it should know what you want regardless of how you phrase it. Want to do something? Just ask! Treat it as if it's a person! The possibilities are endless!

Never mind that its actual functionality (y’know… the things it can do when it understands you) is embarrassingly finite and boils down to a “list of commands” anyway.

It also annoys me that there's no (obvious) meta-interactions with most smart assistants to explore what's possible. I can't easily ask "can you do X?" or "what can you do with Y?"

There isn't even an enumerated list of all the features of the Google search engine (i.e. quotes for full expression, minus to rule out words etc.) And this might be the most popular web service in the world!