Ask HN: Selecting Hardware for AI Inference?

Hey,

I'm looking into building a AI/ML rig. I want to do some inference (and perhaps training), and considering factors like power consumption costs, efficiency, accuracy, versatility, and general cost (of the hardware). What should you prioritize when selecting hardware for inference for these kinds of things?