ollama is dead easy to use. It's a single binary that downloads models on demand, like docker downloads images.

  pacman -S ollama
  ollama serve
  ollama run llama2:13b 'insert prompt'

https://ollama.ai/

Here is an open source project that supports voice as well: https://github.com/cogentapps/chat-with-gpt I think it is meant to be used with ElevenLabs and OpenAI API, but might be easy to configure for use with local Whisper.cpp + llama?

It's not open source but it is still (for now) free: lmstudio.ai. Chat histories, a good config UI, easy prompt management, model management, model discovery, easy set up, cross platform, able to serve an API for connectivity to other tools.

They're hiring and have no currently disclosed monetization strategy so I expect a rugpull soon where some now-free feature gets paywalled or purposefully crippled, but it's not like porcelain apps for free LLMs that rely entirely on llama.cpp to function can do vendor lock-in. I'd second ollama if OSS is a higher priority than features though.

This is the easiest to setup: https://faraday.dev/

I think Wizard is the “meta” for technical questions now.

Depends on what you mean by "best"? Absolute bleeding edge fastest possible inference? ExLlama or ExLlamaV2 on a 4090.