Hacker News Reader

Comment by jcq3 | Hacker News Reader

Hello Alex, I've built a similar app for my own usage, I don't plan to ship it online. How do you handle token limitation for summarizing lengthy videos?

View on HN

I do many requests. Then I summarize the summaries. This approach is well described in this OpenAI article: https://openai.com/blog/summarizing-books/

Useful link thanks. This approach is not cost effective though, I guess you're working on solution to optimize your cost. Fine tuning? Embedding?

Honestly, I haven't worked on cost reduction yet. I have a backlog, but I haven't done anything from there.

One idea is to use other models to shorten text by throwing out meaningless words. I estimate this will reduce the length of the text (and thus the GPT cost) by 30%.

> One idea is to use other models to shorten text by throwing out meaningless words.

Does this mean GPT does not need coherent sentences to understand?

My feeling is that she understands much better than people do. If you understand a sentence from which some of the conjunction words have been taken out, she will understand 100%. And I think this will all be smoothed out on a transcript of 150,000 characters (that's the average size of a podcast).

Hey jcq3

Anything in particular thing stopping you from shipping it online? Or just that it's not necessary for your use?

(Context: I've been working on `Heroku for LLM apps` and trying to understand where the value/frictions are)

Reply on news.ycombinator.com