Hacker News Reader

You can exploit this and build your own stream of streams for interesting use cases: https://chrlschn.dev/blog/2024/05/need-for-speed-llms-beyond... (screen in action: https://chrlschn.dev/img/need-for-speed/generation-example.g...)

Most interesting is combining this with web components and having GPT directly output streams with small web components.

View on HN

That gif is really cool! I built a Python package magentic [0] which similarly parses the LLM streamed output and allows it to be used before it is finished being generated. There are plenty of use cases / prompts that can be refactored into a "generate list, then generate for each item" pattern to take advantage of this speedup from concurrent generation.

[0] https://magentic.dev/streaming/#object-streaming

Reply on news.ycombinator.com