I think this is essentially what FogBugz does, so if it works for you, I think there's a good argument for using it.
More info http://www.fogcreek.com/fogbugz/evidence-based-scheduling/ and http://fogbugz.stackexchange.com/questions/4396/how-does-evi...