Hacker News Reader

Full thread

jschveibinz

Some attempts to benchmark and compare LLM's:

https://artificialanalysis.ai/

https://benchlm.ai/

https://tokenscost.com

https://www.vellum.ai/llm-leaderboard

Are there any projects out there for doing something similar?

View on HN

ignoring the results on the website, all of these except for artificial analysis are very low quality. On vellum, it's hard to even understand which model does the bar refer to since the alignment is off and different for each item.

Even if they add value, it's very hard to take them seriously given the quality of the website

Reply on news.ycombinator.com