What do you mean by non-deterministic hardware? cuBLAS on a laptop GPU was deterministic when I tried it last iirc
DRAM needs to be refreshed every X cycles.
This means you don't know the time it takes to read from memory. You could be reading at a refresh cycle. This circuitry also adds latency.