As I understand it (which, like most of the population of the planet, is not very much), solid magnets are also just a lot of smaller magnets floating in a macrostructure.
Permanent magnets are locked in place. If they're floating freely, the "tiny magnets" just snap together and cancel each other out. That's why heating (increasing particle mobility) a permanent magnet makes it lose its magnetization.
Not really floating, since that implies a certain freedom of movement that atoms in a solid don't have. But otherwise you are correct, since each electron is a tiny magnet (protons and neutrons are also magnets but their magnetic field strength is much smaller so they can safely be ignored). A macroscopic magnet is just a material where the electron magnetic fields add together to an macroscopically appreciable value, rather than cancelling each other out, i.e. the direction of the electron magnetic fields are largely aligned.