It's kind of nice. So everything in nix revolves around /nix/store
This is where each package is stored in a form <hash>-<package name and version>
The second part is only useful for humans and is not really necessary. The hash is computed from package source, compilation flags, dependencies, architecture etc. (there's also experimental mechanism where you can use hash that is computed from the contents of the package, which solves some important problems).
The reason why hash is used, is so you can be able to have not only multiple version of the same package, but you can also have even the same version that maybe even depend on a different package.
For example in a traditional system you could have situation that you need to run two apps, but each of them perhaps depend on a different version of openssl. That situation is generally not possible with some kind of tricks, you either have to recompile one application to use the same library or you will have do some kind of tricks, like compiling statically, or placing openssl in some other location. In Nix you can have two openssl side by side and each package is referencing the one it was compiled with.
The store also is designated that it supposed to only adding packages or removing, you no renames and definitively no changes of the files there. If you do that (well it's linux, you can also change permissions and do that if you really want to, but you're asking for problems)
Now this makes caching easy. Because when you need an application you not only know its name, you know down even to compilation of the flags what it needs to be. So you can simply compute the hash of it. So when it is needed Nix will first check /nix/store. If the package is there, success, you use it, if package is missing, nix can contact configured cache (which can be even an S3 bucket) and check for the package. If package is there it'll download and extract it in your store. If object is missing in remote cache, then nix will begin downloading source code and compiling, because that's how every package is defined (how to build it).
Now that explained the basics, when you invoke nix-shell to enter. The Nix will perform these steps, it'll check if you have the package in your store, if it is missing it'll download from cache, if it is missing in the cache it will begin building, if another dependency is needed it will do that step to it recursively.
Ultimately when the package is present nix-shell will enter a shell, with modified PATH that includes the package(s) that you referenced. You exit the shell you land back in old shell with old PATH. If you use nix for development you can also combine it with a tool like direnv, which can make it so when you cd into a project directory it will ensure that your dev tools are automatically available. And they will disappear when you exit it.
If you were paying attention you might realize that this approach will generally make /nix/store just continue to grow, the new packages are being added all the time. This is especially when you use it for development and you're changing your application, you will get tons of versions of the single app. That indeed is happening and is kind of a drawback, but there's a garbage collection method which you can invoke manually, or you can also schedule it to run periodically. There is also an option (I think more useful for a build machine) where you can specify how much disk space you want to use it, and it will just clean old stuff as you're hitting the disk usage limit.
Anyway, changing PATH is what nix-shell does (originally author said it was meant for debugging (you could enter shell that was used for building the application so you could troubleshoot why build is failing), but it turned out a great feature of its own). NixOS on the other hand works in a way that you have central configuration system where you have a configuration.nix file with all system configuration. When you invoke nixos-rebuild command (typically called with switch parameter to immediately switch to the new config) the NixOS actually is rebuilding the entire system from scratch, the only reason it does not take so long is because all needed files are mostly already in /nix/store and all NixOS needs to do is to recreate symlinks in the root to point to proper packages. This is great, because:
1. you get system exactly as described, unlike SCM like saltstack, ansiblem chef, puppet the nix configuration is truly declarative, and not just imitation of it. For example if you define a package in the configuration file and configure system, then remove that package, after rebuild that package won't be referenced anymore and available (it still will be in the store, but it would be just taking space that you can reclaim with GC)
2. because symlinks are used, and packages aren't replaced in-place. You can actually make a rollback. Let say you have your system running and you want to try Wayland. You make changes to the configuration to install it, maybe the process failed for some reason or you realized you want restore things back to where they were. You can just revert to a previous version. This also will be a very quick process since you likely still have your old packages installed and nothing new needs to be downloaded.
It's really a new very different way of thinking about packages and about OS, and this approach solves so many problems with the status quo we have right now. Also, removes need for the SCM tools as mentioned above.