First of all: that technique is not very accurate, calling sleep() is not guaranteed to return in a specific amount of time.

But more importantly, that only covers the basic timing attack side channel and not even that very well. Cache and branch predictor attacks are still possible (and these are a bigger threat when you're using a virtual machine that may be shared with your adversary). Not to mention all the side channels (power consumption, etc) that are visible if your hardware is compromised.

And if you do end up yielding a thread, the context switch overhead will be measurable. It can take hundreds of microseconds to fully recover from a context switch (TLBs and cache misses, etc), which will certainly affect the timing. It would almost certainly be possible to detect if the encryption method yielded the thread or not based on timing alone (e.g. encrypting next block would consume more time after a thread yield).

There are no easy ways out, crypto code should be free of side channels: constant time, no data dependent memory accesses (cache attack), no data dependent branches (branch predictor attack), etc. This requires the crypto implementor not only to be proficient in the mathematics of cryptography but also details of the computer architecture they're running on.

Making the overall computation take a fixed amount of time does not plug indirect leaks. In particular, if the computation makes memory accesses at addresses that depend on secret values, then this will populate the memory caches for these addresses, and evict from the caches whatever data was using the same slots. The attacker may work out, after the computation, which previous data elements were evicted from cache, thereby leaking secret information. Crucially, the cryptographic computation itself took a fixed amount of time, but leakage occurs nonetheless.

This kind of leak still counts as a "timing attack" because the test for cache eviction is based on the time it takes to next access the relevant element (and this occurs _after_ the cryptographic computation, from other code). Notably, this can still be done remotely, possibly from a large distance, thanks to the efficiency of modern networking (this is what makes timing attacks special among side-channel leaks: power analysis, electro-magnetic emissions, sound... require the attacker to be in the physical vicinity of the target system, while timing attacks may be performed from hundreds of miles away). Demonstrations have been made with network access (over ethernet or optic fibre), and, in another kind of setup, when the attacker can run his own code on the same hardware, even with logical isolation (another process, or even in another VM that happens to run on some other cores of the same machine).

Thus, "true" constant-time code will make no memory access at an address that depends on secret data (but the data contents, of course, may be secret). Similarly, it won't make conditional jumps that depend on secret data (because of the cache accesses for loading the code, and also because of the jump prediction cache, both having been successfully exploited in lab demonstrations).

I would imagine that you can detect that in various ways - how fast the server is able to serve concurrent requests, or if you have more local access how much power is being drawn, and so on.

I think the goal is not necessarily constant time, but more like do the same things independently from the bits of the secrets (symmetric keys, private keys, etc.). As a result, you get constant time.

But you get much more, because if one analyzes physical conditions (cpu/memory usage, temperature, time itself), he can't gather information about the secrets.

That's going to vary based on processor, memory, load, etc.