ChipBenchmark: Open-Source Benchmarking for LLM Performance Across Hardware

We just launched Chip Benchmark, an open-source tool for hardware-centric benchmarking of open-weight LLMs across accelerators like NVIDIA A100/H100/L40S and AMD MI300X. It measures throughput, latency, and time-to-first-token with transparent scripts and an interactive web dashboard—making apples-to-apples comparisons easier.

We're actively welcoming contributions, new hardware support, and benchmark requests.

Repo here: https://github.com/Herdora/chip-benchmark Dashboard: https://herdora.com/benchmark

Feedback and contributions welcome! We made it super easy to add other architectures by including the script we used for benchmarking.