Calorie counting is impossible to do right and meaningful. All that experience you mention is simply a chance and has a tone of confouding variables.
Your internals are not fixed like a machine, parts of the system can be shut down (such as reproductive system) and parts of it can demand more energy at specific times (like immune system) and parts of it can appear (like cancer). There are also other factors like hyperinsulinemia which is more like logistics problem (some parts of body get bunch of energy while others are starving). With this kind of dynamic, relying on calorie counting is simply usless as all people are on various states on this system.
Sure, when you bring calories down to 0 you will certainly di of wasting but that has 0 practical value.