https://en.wikipedia.org/wiki/Euclid%27s_Elements
They aren't really books of equations, just lots and lots of verbose axioms and theorems and conclusions about properties of points/lines/triangles/circles/etc.
A square (the squaring operation that is, not the shape) in this context still means x * x, but it would be described (and reasoned about) in terms of being the surface area of a shape or the length of a line, and logically used in comparison to the surface areas of other shapes or the lengths of other lines. A squared square would still mean x^4, but it would be presented in terms of some quantity (x^2) which was shown to be the square of some other quantity (x), which also happens to be the square root of yet another quantity (x^4) according to the logic of whatever property the proof was trying to establish.
An example of a geometric property of the squaring operation could be a simple statement along the lines of, if the side of one (geometric) square is shorter than the side of another square, then the area of the first square is guaranteed to be less than the area of the second square. Which all sounds painfully trivial, to the point of not even being worth talking about, but in the context of Euclid it would have been a meaningful relationship that could be exploited as part of some much more elaborate proof.
At no point would any of this be expressed in terms of modern mathematical notation, it would all be semiformal verbal descriptions of the relationships between 2D geometric primitives. In the world of philosophizing about 2D geometry, squares and square roots have a conceptual primacy that other exponents do not have (likewise for cubes and cube roots in 3D geometry).
What little I've read of this stuff comes across like the machine language of mathematical deduction, where everything is built up from a very long series of very simple statements that (given enough statements) can eventually produce very sophisticated results. The way math is taught now (at least to non-math majors) comes across more like a very high-level language where we're simply taught rules like x^y * x^z = x^(y + z), and we aren't required to understand it in terms of low-level geometric proofs.
Another way to interpret it could be, "okay, take this cube with a volume, take that volume and pretend it's actually the width of another special mega-cube and never mix mega-cubes and regular cubes".
A square of a square is a square made up of squares :)
(like a Sudoku puzzle)