Well, that depends whether the universe is infinite. In any case, everything was closer together and hotter.

I think "the big bang happened everywhere" is more correct than your take. The big bang dlrefers to the early period of a spacetime with an initial moment. At that initial moment the distance between all points is zero. But there still are a full 3d set of locations. Starting at this initial moment, distances between points grow quickly. Thats the bang.

It is true that in GR you can't speak of "before" the big bang, but the big bang itself is a feature within time. It happens at the first moments of time and everywhere in space. And if you replace the initial singularity with a dense quantum foam and thus are able to extend time into the past in some quantum sense, the big bang doesn't go away.