Hacker News Reader

Comment by llm_trw | Hacker News Reader

> - the push back against remote work will be real and large. Not sure what side I am on

I'm at a small company where we've started recording, transcribing and taking minutes of all meetings with AI. Then that gets fed into a RAG system and we can see what's been said when.

If you're doing it in person than all of that information will just float out of everyones heads by the end of the week.

There's a push _for_ remote work because it's actually convenient to find out what you need to do and when it needs to get finished by.

The next thing we're doing is buying everyone an intous screens so we can share whiteboards which is the last thing that in person meetings were good for.

View on HN

This sounds like a recipe for getting people to only speak after careful curation.

Which is a good thing as there is no such thing to unsay.

Its a great way to stiffle innovation, though... Lots of ideas start out as dumb, and then get refined... We have also been using AI, meeting recordings, and summarization to some extent, but I dont see it being used "for all meetings", it will just be very counter productive IMO.

By that logic, company cultures that don't punish dumb ideas will remain innovative though.

> recording, transcribing and taking minutes of all meetings with AI. Then that gets fed into a RAG system and we can see what's been said when.

Could you please share details on tooling for this?

Not much tooling I do it all on Linux.

I use audacity to capture the audio stream live, whisper.cpp with person segmentation to extract the text with notations on who was talking and a fine tuned llama model which is trained on data specific to the companies market to extract the minutes. It's a couple of passes for the model to write the prompts to extract the data into minutes.

The RAG system is just untrained sbert weights of every sentence in the recording and minutes shoved into the vector postgres. Queries are @mentions in the company chat.

Basically a bunch of python scripts held together by duct tape and chewing gum.

When querying, can the system answer with link to the minutes and the recording ? Lack of source citation for facts is a problem with what I've seen of LLMs.

Yes, part of the metadata is the date, person speaking and line number where the hit happens for each sentence if it's a hit in the transcript. If it's a hit of the minutes it's just a link to the date on which the minutes happened.

That's got nothing to do with the LLM used to do the summary but part of the RAG system.

Sounds like a fun project!

> whisper.cpp with person segmentation

Super curious about this bit. Did you use the --diarize or the --tinydiarize flag? How accurate/reliable was the segmentation? Sounds like it was good enough to use at the very least. I had looked into this a few months ago but thought it wasn't good enough yet.

Me too! In particular, assuming whisper segmentation works, how did you recognize and label the individual speakers?

Very cool. I tried to do this a couple months ago, but really had trouble recording the audio from the call. Are you recording the audio from the call on an external device ie. your mic?

To be clear, my issue was I could only capture the streaming audio of my mic or the people on the call. Never both at the same time.

I have no idea.

I just have a dummy user that joins the meetings so people known when they are being recorded. I imagine I accidentally dodged a bullet there since that user isn't supposed to have audio in and the audio out is directed to audacity and muted otherwise.

MS teams has this, don't bother cobbling together and maintaining your own solution.

>Don't bother doing data science in python, Microsoft has excel.

And excel has python!

Jokes aside, excel is the most powerful business software ever. Everyone use it.

Don't bother using it either, have you seen one of those transcripts?

Maybe it works better with an American accent.

I’m pretty sure you could record and transcribe in person meetings just as easily. I’d bet there are several services doing that now but it would also be an incredibly simple project to build in house. The only problem is identifying who is speaking which could be a deal breaker for some uses.

>I’m pretty sure you could record and transcribe in person meetings just as easily.

The issue is that in person you constantly have informal "meetings" where information is shared and impossible to record, unless you force everyone to walk around with sound recorders on. Which come to think of it is something that might well be done in some companies I've worked for before.

>The only problem is identifying who is speaking which could be a deal breaker for some uses.

Trivial, it's a problem that's been solved for a while and I managed to get it working in a couple of days of messing about.

>The issue is that in person you constantly have informal "meetings" where information is shared and impossible to record

I'm positive that people working remotely can just as easily have informal meetings were they won't be recorded. In fact, I'm absolutely sure that a good segment of the workforce will make sure they have meetings in that informal sense, specifically so the recordings aren't happening.

> The next thing we're doing is buying everyone an intous screens so we can share whiteboards which is the last thing that in person meetings were good for.

HHHa, tell me you're middle management without telling me you're middle management.

Reply on news.ycombinator.com