Hacker News Reader

Not open-source, but https://airtrain.ai lets you do this. Disclaimer: I’m an engineer there.

Edit: aiming to have Claude 3 support by tomorrow.

https://github.com/nat/openplayground but seems it has not be updated from 6 months.

If you're willing to use the CLI, Simon Willison's llm library[0] should do the trick.

I already have a cli client, but how to talk to multiple different LLMs at the same time? I guess I can script something with tmux.

Yes, I had in mind that you’d need a simple script for this

Choose Arena (side-by-side), it has Claude 3 Opus, Sonnet and GPT-4

Poe. Poe has Claude 3 right now as well as Gemini pro (not ultra), gpt 4 and 3.5, mistral large, llama, others