oceansweep

You could try my app https://github.com/rmusser01/tldw

Supports arbitrary length videos and also lets you choose what LLM API to use.

Not the person you asked, but it's dependent on what you're trying to chunk. I've written a standalone chunking library for an app I'm building: https://github.com/rmusser01/tldw/blob/main/App_Function_Lib...

It's setup so that you can perform whatever type of chunking you might prefer.

I've assumed that Google's approach for NotebookLM is similar to this, given their release of https://huggingface.co/google/gemma-7b-aps-it :

Gemma-APS is a generative model and a research tool for abstractive proposition segmentation (APS for short), a.k.a. claim extraction. Given a text passage, the model segments the content into the individual facts, statements, and ideas expressed in the text, and restates them in full sentences with small changes to the original text.

Anthropic:

When Citations is enabled, the API processes user-provided source documents (PDF documents and plain text files) by chunking them into sentences. These chunked sentences, along with user-provided context, are then passed to the model with the user's query.

Claude analyzes the query and generates a response that includes precise citations based on the provided chunks and context for any claims derived from the source material. Cited text will reference source documents to minimize hallucinations.

Ahh, thank you. For what it’s worth, I agree.

You’re making assumptions about my original question. I wished to know their opinion on what made NotebookLM ‘look like an Intern’s project. If what they shared was something I agreed with and relevant than sure I would apply it to my project but I already have plans for improving my project to my own standards.

Thanks for sharing your perspective though, I will keep it in mind. I disagree regarding podcasts being a ‘big thing’ past the initial honeymoon phase. I do think that custom generated audio is also a big thing and the podcasting is the first exposure a lot of people have had with that level of quality, and since it’s free and everyone already has a Google account, it makes it much easier for it to viral.

I don't think you understand the context, the person I was replying to was making that comment about NotebookLM. I'm fully well aware of how my UI looks, the whole reason I'm using Gradio for right now is that it is a single person project that isn't a product for sale. Not quite an intern, but same amount of funding. The current UI is a placeholder, because the idea is to migrate to an API first design so users can have whatever kind of UI they'd like.

SoundStorm/Podcast creation is one of the big draws, but I would question as to whether its one of its most-used features, considering hallucinations and shallowness.

As someone who's built something like it in their free time as a hobby project ( https://github.com/rmusser01/tldw), could I ask what would make it a professional product vs something an intern came up with? Looking for insights I could possibly apply/learn from to implement in my own project.

One of my goals with my project I ended up taking on was to match/exceed NotebookLMs feature set, to ensure that an open source version would be available to people for free, with ownership of their data.

Hey, I’m working on exactly that: https://github.com/rmusser01/tldw

It’s a work in progress, and I literally just added the first foundation for note taking next to the RAG chat last night(not working yet) but it’s getting there. The plan is to build the functionality out and then migrate the codebase to an API backend so that people can make their own front-ends for their own needs. Also, it doesn’t have podcast creation or TTS, yet. They are planned, but it can transcribe and ingest a podcast for RAG chat currently.

There's multiple. Here is one I maintain, though am very behind on it.

https://github.com/rmusser01/Infosec_Reference