There's two things that can have symmetry here: The laws of physics themselves, and the system under investigation. The symmetries of the laws of physics don't get broken, but those of the system do. Compare a crystal to a gas: In a gas, the atoms are all bouncing around pretty randomly, so at any given point in space, there's roughly the same chance of finding an atom. Shift the gas to the left by distance x, and the local probability distribution of atom positions looks pretty much the same. In a crystal on the other hand, the atoms are still moving around and vibrating (so there's still some uncertainty in the positions of the atoms), but they tend to stay pretty close to their proper position in the crystal lattice. So the atoms are more likely to be in positions that line up with the rest of the crystal lattice than anywhere else. This breaks the symmetry. Shift by distance x and the peaks of that probability distribution no longer line up. The exception to this is if x is a multiple of the spacing of atoms in the crystal. Then you're shifting the peaks by exactly the right amount that they line up again when you're done. So a crystal doesn't completely break the symmetry of space, but it reduces it from a continuous symmetry (you can translate by any amount in any direction) to a much weaker discrete symmetry (only certain translations of space will preserve the symmetry).

A time crystal is similar to an ordinary crystal except that instead of reducing symmetry of translations in space from a continuous symmetry to a discrete symmetry, it reduces symmetry of translations in time from a continuous symmetry to a discrete symmetry.

EDIT: It's a little ironic that if you ask most people, they would say that a crystal is more symmetric that a gas, since a gas will look completely random and asymmetric if you take a snapshot of the positions of all the atoms at a single time. But since physicists care about the probability distribution of atom positions, they say that the gas is more symmetric than the crystal.