What should the CPU usage be of a fully-loaded CPU that has been throttled?
devblogs.microsoft.com
devblogs.microsoft.com
This will take into account machine to machine variance, and even environmental factors effecting maximum speed.
Anyway, current processors run a separate clock per core, and maximum clocks are only available when a small number of cores are active; if all cores are busy, that should really be 100%, even if each core is only doing 80% of max for a single core.
Mostly, I want to see % of time cpu is busy, and separately, stats on how throttled the cpu is, because it's hard to combine both into a coherent number. Maybe also some idea of how much of the core is being exercised, if it can be easily measured... I'd love to know when a program is keeping the cpu busy, but not making good use of it.
Both are extremely complex with a lot of nuance after that but the "percentage time spent not idle due to program" type this article refers to tends to be simpler than trying to figure out cache/mode switches/instruction level parallelism/instructions per clock per type/and so on on top of everything you need to figure out in the first case anyways.
That should never happen and it's ridiculous that we just let manufacturers get way with it.
Prime95 is basically a synthetic workload so it makes little sense to optimize for it.
At 100 the processor is at high risk of damage, which is why there's a built-in throttling mechanism.
Just imo, if it was safe it would be running at full speed (or at least max base clock) at that temperature.
As someone else said, they're made for burst operation these days, but again, that does not excuse manufacturers using subpar cooling.
I can see the majority is fine with it, but I'm not. A 15-25% failure rate in 2 years would make any other product a rotten lemon. But somehow it's acceptable for computers. Probably because people replace them every 2 years regardless, which is another insanity on its own.
Temperature junction throttling is a last resort. No laptop should rely on it in normal operation. Of course, both HP/Dell/Lenovo/etc and Intel benefit from increased sales so they don't care.
CPU Usage measured in percentage doesn't make sense on any CPU with modern performance features like speculative execution and branch prediction.
> The etiology of military power is from War Emergency Power (WEP) which in the WWII era was a higher than normal rating power (i.e. >100% rated power) setting on an aircraft engine. Such power settings were approved for short durations (typically 5 minutes or less) such as takeoff and battle maneuvers.
> The term was quickly shortened to military power.
The internet and GPS, the core technologies of our time, are both direct descendants of the US military.
You may not like it, but the entire IT and tech field are built on US military tech
In ancient times both mercantile and military incentives propelled the design of ships. Is fair to say the whole ship building field built on military tech?
Even if it were true that in the distant past a thing had a military application/funding, there's no reason to use their terms for new things which aren't military in nature. No commercial liner calls it's full speed "military power". It's just cosplay at this point.
There's plenty of room to debate how much of our technology came from the military, but I don't think naming is a big deal.
Take the word "screen" for example. The original meaning was a partition to protect from heat. Many of these were fabric, and led to "magic lantern" shows done with shadows. The word was repurposed again for the projection era. And yet again for tube TV's and beyond.
If you told somebody from 100 years ago to look at the screen they would have no idea what you're talking about because the original meaning is lost.
That's why I don't think it matters whether we use military terms for computer stuff. It doesn't have its original meaning anymore.
Good, we agree, no more tacti-cool cosplaying in IT.
https://www.freedesktop.org/software/systemd/man/systemd.res...
> The percentage specifies how much CPU time the unit shall get at maximum, relative to the total CPU time available on one CPU. Use values > 100% for allotting CPU time on more than one CPU.
That’s hard to do without hardware support, but I wonder how hard it would be to get a decent metric with currently available performance counters.
However, when the CPU's speed has been reduced because it's too hot or the system is otherwise unable to allow the processor to sustain its full clock speed, you absolutely should see the Task Manager reporting 100% utilization. This scenario is what more often comes to mind when the term "throttled" is used.
If a hardware platform's power management capabilities make it impractical for the operating system to satisfy both of the above goals, then it should favor the latter goal, and err on the side of not lying to the user when your system is truly operating at its limits.
Maybe just measure overall system load as current CPU temperature as a percentage of maximum?
I've never used a mobile device that seemed to have such considerations. They all seem perfectly happy to make their surface scalding hot. Do manufacturers actually care about this?
I was trying to optimize some FEM code, toying with (hardcoded) solver parameters. On one console I had it spitting out the wall clock durations of time steps as the simulation was running, while on the other I was preparing the next run. I start compiling another version, and inexplicably the simulation in the other console gets faster. Like, 10%-20% less time taken per time step. "That must have been coincidence. There's no way the simulation got faster by compiling something in parallel." But curiosity got the better of me and I still investigated.
Watching the CPU speed with CPU-Z, it turned out that the simulation was indeed getting down-clocked, and that compiling something in parallel made the CPU run faster, speeding up the simulation too. WTF? And indeed, I could make the entire simulation run significantly faster by calling
std::thread([](){ while (true); });
Why? Well, the simulation happens to be extremely memory-bound (sparse mat-vec multiplication in inner loop). So the CPU is mostly waiting around for data to arrive. Apparently the CPU downclocks as a result. That would be fine, if not for the fact that the uncore/memory subsystem clock speed is directly tied to the current CPU speed. That's right: The program was memory-bound, hence the CPU clocked down, hence the uncore clocked down, hence memory accesses became slower.
Knowing that feedback loop, it makes perfect sense that keeping the CPU busy with a spinning thread improves performance. But it's still one big wtf.
This problem eventually went away as we parallelized more and more of the simulation, giving the CPU less reason to clock down. But for related reasons, the simulation still runs faster if you prevent hyperthreading (either by disabling it in BIOS or having num threads = num hardware cores). More threads don't improve memory bandwidth and the hyperthread pairs just step on each others toes.
What exact mechanism controls this is not clear to me (and I'm actually not sure if it's clear to anyone outside of Intel - the one paper [0] I found at the time was based on reverse engineering experiments). Nevertheless, CPU clock speed definitely affects Northbridge speed, as proven by the latter increasing from spinning a thread that never touches memory.
[0]: https://tu-dresden.de/zih/forschung/ressourcen/dateien/proje...
See section V.A:
> The results [...] indicate that uncore frequencies – in addition to EPB and stall cycles – depend on the core frequency of the fastest active core on the system.
(That conclusion is fully in line with my own observations.)
Also see the corresponding patent linked in the paper: https://patents.google.com/patent/WO2013137862A1/en
Linux has done some work in this area with frequency-invariant utilization tracking: https://lwn.net/Articles/816388/
Agree, you really want 2 indicators, vs 1 confusing hybrid.
For Windows, if anything I'd love to see an additional metric, where the app resource usage is presented in % of available cores.
Eg if I have a single thread app (i.e. a game) on let's say a 10 core CPU with SMP disabled, it might report 10% usage. But looking at the core graphs we see one at 100% with the other 9 idling. So the app is really maxed out at its architectural limit - ie 100%. It would be nice to have an additional column representing this - and maybe an idea of the foreground app's available % usage in other reporting (am I cpu limited, or GPU? people have bought Intel over AMD CPUs for this metric even though multi thread workload capacity has been far greater in AMDs).
Taking a step back again, especially for Ryzen CPUs unless manually locked you'll have different per core frequencies - so the article's suggestion becomes an almost impossible mission: A Mandelbrot processor can be split across all threads rather equally for an unthrottled 10 core run say at 4ghz, but a core limited config may be able to run it at 4.5ghz prior to thermal throttling - what is the real max to report in a 10 core run following that then? And on a very cold day, with fans at a higher RPM, the core limited run might hit a higher 4.8ghz - now what? You just can't; reporting on current CPU capability is a separate concern altogether.
* 100% per core's for the factory target speed (what a CPU is sold to run at non-turbo/boost/whatever)
* turbo / boost displays figures above 100% (per core)
* special fake accounting buckets for not just cpu_idle, but also cpu_powersave, cpu_thermal, etc. Any status caused by yielding or reducing the runable time of that core.
* An additional (does this exist already?) counter of executed instructions, in addition to the run time for each task / thread. On multicore systems this should be recorded for each core the thread can run on. Large NUMA systems might disable, or node restrict, this for obvious reasons.
This would give a clear picture of both the utilization of hardware as a whole, as well in the program context.
Early throttles were 50% duty cycles, then L1 bubble injections, then V/F frequency scaling. The author only addresses the early mechanisms, but it gets even more complex with the PCUs in the later Xeons.
It is not an easy question to answer, but I think the question can be modified. A single number doesn't solve the problem, you need to know utilization in the context of throttling (and magnitude). Then decide what you are trying to solve: scheduling or app-level throttling? Personally, the Hz denominator should change if it used in utilization , since that covers the majority of cases. Any other case should read both metrics.
EDIT: Removed generalization and worded as anecdote.
IMHO the reason actually does matter.
Utilization should be relative to the maximum frequency the CPU could be running the same instructions at. If the CPU is throttling to save power, then it could be running the same instructions at a higher frequency, so utilization should be relative to the higher one. If it's throttling to lower the temperature, then it can't be running the same instructions at a higher frequency, so it's already maxed out at 100%.
This gives the ability to go over 100% before thermals equalize, since max power would follow a Newton cooling curve.
How do you define "maximum frequency"? For desktops it's relatively easy to define since they usually have good cooling and power delivery that they can always hit the top frequency, but for laptops this often doesn't apply. It might be only able to sustain top frequency for less than a minute before thermal throttling kicks in.
But okay, so there's some discrepancy, whether 15% or 30%. Okay, so I might underestimate it, and... then what? What would actually go wrong?
I think what you're not realizing is that underestimating capacity is a very different situation from overestimating it (especially when we're talking about underestimating like 0.4 GHz vs. 4 GHz, as opposed to overestimating 2-3 GHz as 4 GHz). When I'm looking at CPU usages it's almost always to figure out who's overutilizing the CPU, not underutilizing it. If I underestimate utilization, then at worst, what happens is I launch a program that needs all cores for max throughput (say a video encoder?), and then get disappointed at its throughput being too small. This is so infrequent (if it happens at all for the average user) that the additional mental effort required to factor in the throttling is quite negligible, and the negative consequences are quite mild. Compare that to the case where I overestimate utilization: suddenly I semi-panic and try to kill the program using the most CPU, and boom, suddenly I'm at risk of losing a bunch of data. The difference is quite asymmetric.
And I'm just talking about what 99% of ordinary users expect most of the time. Nothing prevents you from measuring whatever metrics you want either. Obviously "I think this is what most people expect utilization to mean" obviously does not imply "I think you should be banned from anything else you might find useful"...
Every one of those numbers is different, and frequently the only way to determine if a frequency is possible to use for stable operations is to try it and see.
You cannot deterministically know what max frequency the CPU could be running at in some future demand/power/temp state.
Of course, this opens up other issues with how to aggregate multiple cores, what the benchmark for 'max' should be, etc.
Perhaps the more fundamental answer is that there's no single metric that can sum up the situation for all use cases, in which case displaying '100%' would be more useful for a typical consumer while exposing multiple detailed metrics would be more useful for system admins and power users.
So seeing 200 load doesn't indicate to me that it's throttled, but that it's using the equivalent of two full cores. Or did I misunderstand?
It's more of a "pressure" / "need" measurement. And yeah, applying that to per-process CPU measurement would be interesting.
Trying to express "How much of the hypothetically available computational resources of the CPU did this application consume?" in a single number would seem like a futile exercise at best. VTune used to have something like this which IIRC was based on using all cores in parallel sections and IPC or something like that. It wasn't very meaningful, and is impacted by all sorts of factors.
If you turn off hyperthreading and keep the same exact workload, then instead of "the graph says 50% but it's really more like 80-90%", you'll have "the graph says 100% and it's correct". The numbers now accurately represent your lower capacity.
Do you mean SMT in general? I don't think hyperthreading specifically has ever done that, but if I'm wrong I'd love to know more. (And AMD's version falls under "newer desktop-class processors")
It also describes several instances where Intel's desktop cores used to devote specific resources to each thread on alternating clock cycles, but newer cores have progressively removed those limitations. However, these probably don't quite fit my original assertion because if the OS has literally HALTed one of the virtual CPUs, these alternating clock cycle limitations may have been temporarily removed.
For me, I keep SMT turned off. The latency is more important than the throughput for my workstation, but if you do full builds of C++ regularly, you might want it on. Use the 10% time you're getting back to switch to a build system that can cache things, though.
That's not really true, you can mix and match memory, and you might not get ideal bandwidth, but for a lot of uses it's just fine.
Perhaps that's something Microsoft can borrow and improve upon.
Although then you have users trying to figure out how to kill kernel_task, which isn't great either...
I feel that should be made clear on the various CPU usage utilities.
If the system doesn't do that already, either nobody considered that users would try that particular nonsensical thing, or nobody cared enough to let users find out immediately instead of wasting their time thinking the computer was at fault.
So anyway, it is complicated.
“I’m using 100% of my cpu but I’m only at 1.2GHz, something is wrong”
The right reporting is to report both the numerator and the denominator. I'm using 800 MIPS out of 1200 MIPS available.
One representation is a line graph with two lines -- one is showing the maximum MIPS (right now), and one is showing the actively-used MIPS.
Another is a pie chart, where the size is the chart changes based on current capacity, and it is filled to how much of that capacity you're using.
And so on.
You have two dimensions. You don't want to squash that into a 1D representation.
There is no constraint that one has to use only one number. Perhaps, in 2021, CPUs are complex enough to deserve more than one number to give a useful representation of utilization, as it works with all sorts of factories/plants. Also, who would love to see in the metrics reported CPU usage per core?
Then, implement advanced mode that contains a deeply similar but mildly extended UI. Imagine Office with 1 ribbon tab, cleaned up, vs 3, filled to the brim with choice, tools. In principe, thinking of keeping the diff small but noticeable only slightly, like that.
Which could be useful.
Now throw HT in the mix and loose your mind.
I'm not sure there is a really better solution. Just document the one you choose, please!
Throttling is essentially determined by three factors: Energy usage, thermal saturation, and cooling. A bath tub analogy comes to mind: Energy usage represented by flow from a faucet, the heat sink is the water level in the tub, and the drain represents the cooling rate. Only energy usage could be plotted instantaneously while the others may have to be modeled and could change based on environmental factors.
The downside is of course, that users see this process name chewing up 500%+ of CPU, google "kernel_task cpu", and find the commands to disable the thermal throttling to "fix" the issue!
https://eclecticlight.co/2019/02/25/playing-with-fire-dealin...
Clearly that's a different kind of situation; how do you distinguish the two?
- measurement at thread level (top and the like) should report instructions retired per unit of time, not %CPU. We want to know how much work is being done. The explanation on why so much/so little requires more information (scheduler? cache misses? cpu throttling?)
- measurement at the CPU level (mpstat, etc) should report both %CPU (as "time active" vs "wall clock time") and active clock cycles per unit of time (dividing by clock stretching factor if used, and perhaps scaled to absolute max frequency and/or %CPU if one wants a percentage).
%CPU tells us whether we are making full use of the available time, and perhaps suggests to schedule threads differently if appropriate
Clock cycles tell us how much one CPU is affected by throttling (manual, automatic, due to C-state exits, etc), and this is an orthogonal indication on whether there is something at the system level that is making the CPU underutilized
if they would be available wouldnt the CPU scaling kickin and ramp up the frequency as required? i think in this case it should really show up as 50%...
otherwise its somewhat similar to battery charge status... apparently nobody wants to know what percentage of the initial full capacity is left but just how much is left from what is potentially available... so in a low power state were throttling is done permanently at some frequency i would just like to know how much capacity is actually left...
The problem is what is 100% when CPUs have gotten so complex around the frequency. What's the absolute reference, the base frequency, the single core turbo, the multi core turbo, overclock, AVX?
For batteries you usually get 2 estimates, a capacity percentage of the total practically possible, and one for how much usage you can get out of that based on recent or estimated usage.
https://twitter.com/brendangregg/status/1411654427304333313
Personally, what I'd want to see is the proportion of max achievable IPC, and if that means getting used to numbers well below 100% (even when I'm doing everything right) then so be it. I can adjust my expectations and targets.
That's what we do for other things that have no fixed upper bound, e.g. disk/network i/o. We even do this for memory now given dynamic virtual memory, etc.
AFAIK that's not how it works in modern CPUs. Dynamic frequency scaling is a black box implemented in hardware. Software, and even OS kernels, have very little input over that. They can't subscribe for status updates. Even just getting the current frequency seems impossible, only indirectly by comparing RDTSC output (absolute time unscaled) with performance counters.
How about another measure, % of CPU TDP? Or some form of TDP measuring. If it is 100% CPU TDP, I know it is pushing as hard as it can.
( But then when your cooling aged you will be running at 100% TDP but at lower clock speed without realising it. )
Thinking about it this simple subject is really complicated.
And also, why are systems shipped with inadequate thermal control?
Name is probably wrong but it definitely exists.
> While I sympathize with this point of view, I feel that reporting the CPU usage at 50% is a more accurate representation of the situation.
Unbelievable how dense people play sometimes. It was clear what I meant when I said word, and in fact, I did refer to a single word if you want to play that game: Mandelbrot.
"Maldebrot set" or whatever it said before wouldn't even give you the correct search results. It serves a purpose to correct this, whereas this is just grammatical bickering for no apparent reason other than "he started it". I pointed out an important mistake, you're just trying to find substance in an argument that holds as much water as a sieve.