You can apply calculus to estimate the stepsize you need for the current iteration. If you do this in a principled way then my results suggests that for linear regression you may only need one gradient calculation.
Yes, it's called Linear Least Squares. Gauss discovered it in 1795.