While I understand what you mean, there are already keys which are "infinitely valuable": root CA keys. If you have those, you can essentially impersonate anyone. Therefore, we must have ways to protect these sort of keys.
That being said, a root CA key is probably locked up on an air gapped machine vs some machine which actively using the key to scan through traffic.
Mainly pointing out here that the idea of value vs cost has to have some point at which we protect it regardless of the value because thats how the hierarchy of trust works in our CA system.