I don't know enough about cloudflare workers, but function cold start latencies are not fully in control of the platform. You can hide the latency of scheduling and loading, but the function might load a ton of dependencies; and it might have its own custom init code that you generally have little control over. So you can hide an SSL handshake's worth of latency, and that's a nice little win, but whether that makes the user-visible cold start latency zero depends on what it was in the first place.
But I'm guessing cloudflare workers are used for relatively simple stuff, so they don't often have dependencies or complex init code.
(And sorry, I don't mean to sound too negative, this stuff is pretty cool!)