Hacker News Reader

Comment by geokon | Hacker News Reader

This is a bit tangential but maybe someone could give me a some advice.

When running with many cores is there some way to get the Linux kernel to run them at a fixed frequency?

I've been doing multicore work lately on AWS - which works pretty well but you don't get access to event counters so sometimes I'm having trouble zeroing in on what the performance bottleneck is. At higher core counts I get weird results and I can never tell what's really going on. Running locally I have concerns about random benchmark noise like thermal throttling, "turbo" and other OS/hardware surprises (I know on modern chips single thread stuff can run really different from when you load all the cores). I've been thinking of getting something a bit dumber like an old 16 core Xeon (I'm on a bit of a budget) and clocking it down - or maybe there is some better solution?

View on HN

As was mentioned cpupower frequency-set will sort you out, though you have to do it for every single core/thread combo.

The other option if it's an occasional thing is you can change it in your BIOS and lock them (I believe) to a specific speed.

Since you're not doing it for higher performance just consistent performance you can lock it to your base clock frequency and it should be stable.

if you omit a specific cpu core, all of them are affected by ‘cpupower...’

Really? Maybe I'm thinking of a different tool then, because that didn't work for me (on an AMD CPU)

oh, that's surprising, here is what i did:

  seldon ~ # cpupower frequency-set -f 1.86GHz                                                   
  Setting cpu: 0
  Setting cpu: 1
  Setting cpu: 2
  Setting cpu: 3
  Setting cpu: 4
  Setting cpu: 5
  Setting cpu: 6
  Setting cpu: 7
  seldon ~ # cpupower --cpu all frequency-info | grep -E '^analyzing CPU.*|current CPU frequency'
  analyzing CPU 0:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 1:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 2:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 3:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 4:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 5:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 6:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)
  analyzing CPU 7:
    current CPU frequency: 1.86 GHz (asserted by call to hardware)

this an oldish Intel(R) Core(TM) i7 CPU. just tried it on an odroid, and it seems to do the right thing there as well.

I don't have a answer to your frequency problem but I ran into similar issues with memory usage when trying to process LIDAR data in pandas and keras. I ended up buying an old HP quad xeon 4U with 320gb of DDR3 for $330 shipped. I used https://labgopher.com/ to find the server deals.

Thanks for sharing labgopher... would have made my own search much easier last year.

oh wow. thanks for the info. I saw some decent deals on newegg, but not that low

it's a big chunk of metal, but it would make life easier for some stuff

> When running with many cores is there some way to get the Linux kernel to run them at a fixed frequency?

just a wild thought: have you looked at 'cpupower frequency-set' ? it _might_ help.

You could disable Turbo and EIST in the BIOS - most will have a setting for these, and that will result in the cores running at nominal freq.

I don't know about a nice high-level linux interface, there may well be one, but if you need to access these settings from a running machine there are MSRs you can poke.

Reply on news.ycombinator.com