This particular model says it needs 640GB of memory just for inference. Assuming Huggingface also has other large models loaded, and wants to also make them available to a non-trivial number of concurrent users -- I wonder how many GPUs they have just to power this test-drive feature.
So you're paying ~10x the power costs to get worse, unverified, illogical answers faster when using LLMs vs humans. Which then have to be checked and revised by humans anyway.
I know this isn't the spirit you meant it in, but I'm also impressed with humanity that we've managed to develop something as capable as it is (admittedly significantly less reliable and capable than a person) at only an order of magnitude difference in power consumption.
I mean I don't usually plug myself into an electrical outlet, isn't food much more expensive for the same amount of energy?
So the two H100, at 1KW, cost 0.2276×24 = €5.5 ($6) per day, which is nearly my groceries average.
(My meals are powering all of my body though, which is five times the consumption that my brain requires, so all in all, it seems a bit more power-efficient than the GPU still.)
I believe it's many times less for the brain. There's no way it dissipates anything close to 100W without cooking itself.
Regarding quality of the output, it obviously depends on the task, but we are benchmarking the models on real world metrics and they are beating most humans already.