Even then, you run into issues with gravitational time dilation (though at scales much smaller than whole seconds). For instance, the TAI timescale was quietly changed at one point to follow a new reference altitude, causing the 'calendar second' to run slightly differently with respect to any stationary observer. The only way to really know your age in your own proper time is to have an atomic clock following you everywhere since birth.