2, 3, 4 function calls in a row will benefit from that caching, but also help train the speculative execution engine. At some point that speculation may radically alter performance. Later, with a sufficient number of runs, the timing will also suffer from preemption, context-switches, or other interruptions that can invalidate both the cache and the speculative execution engine.
The idea that there is a fixed performance time for a function is naive on modern hardware.