The basic concepts of calculus (area under a curve, areas inside shapes in 1, 2, 3 dimensions, relationships between forces, etc) don't need anything more than some basic physics experiments and the ability to perceive the world around you.

Calculus can be explained in terms of algebra, sure, but it can also be explained with visualizations and with experiments (and/or you can learn the algebra at the same time).