You talk about time to other cities because it's a constant.

We talk about miles because otherwise, we'd have to say,"I live seven minutes from downtown SF between 10pm and 7am, an hour and a half between 7am and 10am, twenty minutes between 10am and noon, an hour between noon and 2pm, forty minutes between 2pm and 4pm, an hour between 4pm and 7pm, and half an hour between 7pm and 10pm."