Knowing how long things took vs estimates is useful for determining the accuracy future estimates.
Having an absolute sense of numbers for types of projects is useful for estimating future projects of similar scope; where by absolute I mean, not, "I think it took two weeks last time we did this".
Of course, it's a good idea to investigate why expected timescales differed from expectations - but IMHO detailed time recording isn't the way to do that.
Of course, YMMV.
Also, if you're measuring and refining your estimates, they stop being 'estimates' and become something far less useful. If you have problems with estimates being inaccurate, try not estimating.
There are far more interesting metrics to measure rather than flogging a dead horse.
Unless your billing on an hourly basis, chances are that counting hours is too much micromanagement.