Ask HN: Which LLM to consult about LLM's? | Hacker News Reader

Ask HN: Which LLM to consult about LLM's?

6 points65105 comments

There are so many AI projects with much overlap, different price tags and new ones released every day. I imagine choices will get more complicated over time(?) Is there a chat bot with up to date information some place? It seems like something that should exist.

View on HN

The data isn't there. Most would be out of date, and there's no incentive to train a LLM on this. OpenAI doesn't even do a lot of things well; their RAG, STT, TTS, voice, embeddings is below average, but it's convenient for people who don't want to look elsewhere.

The best would be looking to AI influencers. The good ones will be familiar with the common drawbacks and look to ways to evaluate the new models. Some do a weekly summary and such.

I think the trick is not so much which model you use, but rather how you ask the questions. Make it very clear you want sources front and center, and use the LLM to try to cut through all the SEO and find the information you are looking for. Don't rely so much on what the model tells you, but rather where it leads you.

I just plugged this prompt into Nova:

I want you to generate a document detailing the current landscape of AI model and agent providers. You should include external citations, a very brief synopsis of each tool, the pricing associated with the tool, and a link to check it out. Limit this to the top 20 tools in the AI space.

It generated a doc that I could download and build off of. Try something like that?

Any of the leading LLMs could answer this, they just need the right context. So paste in the model cards for the various models you care about, or ask a model with search or "deep research" capability.

That being said, there probably isn't too much alpha in chasing down model tradeoffs outside the top few players (OpenAI, Anthropic, Gemini).

try Perplexity. ai, it’s like a research assistant with live web access. You can also use ChatGPT with browsing (Pro version) to compare models, pricing, and capabilities. Also, you can track updates across models using LLM Leaderboard (Hugging Face) and thereisnobenchmark.com. Both are great tools.

Reply on news.ycombinator.com