I agree with that. You can hand limits as a tool to someone already comfortable with the concepts and present them as a way to quantify their intuitive understanding.
Hence, the gap between "5 year old calculus" and "high school calculus" is smaller than the gap between "high school calculus" and "actual calculus"