The idea was to not use direct measures of time to estimate because someone above you will start assuming those estimates are promises and then life gets unpleasant.
Story points, much as I personally dislike them, were invented for developer defense.