The AVX ALUs also go into power saving when not used and take a couple of cycles to switch back on, delaying the first AVX instruction. There is afaik even a paper on a side channel attack that uses this.