That is one theory I had but i'm not sure how to determine if CPU is throttling (on Ubuntu Linux.)
cat /sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_cur_freq
where instead of cpu0 you can write any core number, and it will give you the current frequency of that core in KHz.