Now, "running" may also mean different things. For example, you may want to do things s.a. performance diagnostics (in order to understand if your code uses resources efficiently), and then you'd need stuff like NVML, DCGM and co. Consumer-grade hardware might not be supported, or might not in principle support diagnostics collection / instrumentation. Or, if you have multiple GPU-dependent workloads that cannot saturate your resources -- you might think of MIG as being a way to address that... and, again, consumer-grade GPUs won't help you here...
I'm not saying you shouldn't try self-hosting. I'm actually all for it. But, you also need to be mindful of the pros and cons. NVidia must have some reason to pitch the datacenter family of GPUs to, well, datacenters. They aren't just blowing up prices.
Also, to give some sense of comparison:
Titan X ~= 3.5K CUDA cores.
V100 ~= 5K CUDA cores.
A100 ~= 7K CUDA cores.
H100 ~= 18.5K CUDA cores.
It probably doesn't translate directly into H100 being six times as fast as Titan X, but, I hear that these GPU workloads might be lengthy...