The short answer is:

Yes, you must disable CPU frequency scaling in your BIOS if you're doing this kind of work (i.e. building cryptographic primitives that don't leak information via timing).

See https://timing.attacks.cr.yp.to/overclocking.html