Speculative decoding was on in all of my tests, the gain came from moving the round's inputs off host memory.