Memristor and analogue processors to accelerate ML are doing this. Neural networks are ok with operations on noisy, analogue states. Analogue operations such as op-amp multiplication are more power hungry than individual digital gates, but some are less power hungry than digital multipliers which use thousands of gates.

For digital, noise-free logic, it's generally more power efficient and physically simpler to have logic gates operate on two cleanly separated states with more gates, than to have bulkier, more complicated gates that do the same thing with combined states. A transistor which is fully on or fully off in a logic circuit uses low power either way (like a wire or a gap in the circuit), but the in-between state uses more power (like a resistor, it produces heat). This is one reason why power consumption goes up with the amount of logic state switching (the in-between resistor-like state occurs briefly during each state change), and also why many-level stable logic states aren't so efficient, except with complicated gates that use many transistors to implement many thresholds.

For non-volatile storage, that doesn't apply. Information density is more important, individual memory cells do not change state often so switching power is less of a thing per memory cell, and the logic sits at the edge of the memory array, shared among many cells. The edge circuitry can afford to be more complicated, to optimise the bulk of the memory array.

In magnetic storage and communication, the signal processing to encode many states in a small signal takes considerable power, but the trade off is worth it.