To sum it up succinctly, I would say that Nix helps you manage $PATH while NixOS helps you manage /etc.
That being said if you want to learn Nix for its own sake or because it is awesome, which it is, then have fun!
Perhaps, if you decide to take a look at Ansible, it might help with getting started https://hth.is/2023/01/02/android-ansible/
https://search.nixos.org/packages?channel=23.05&show=awscli2 is 2.11.27 (even on the "unstable" channel), versus https://formulae.brew.sh/formula/awscli#default that is 2.12.1, which correctly is the most current (https://github.com/aws/aws-cli/tags)
- Nixpkgs (3 days ago): https://nixpk.gs/pr-tracker.html?pr=238031
- Homebrew (3 days ago): https://github.com/Homebrew/homebrew-core/pull/133968
Your link refers to the 23.05 stable release channel, so it'd likely stay at 2.11. Package updates don't get backported to stable channels except for fixes. Additionally, the package search page probably isn't updated in real time so the version might be slightly out of sync.
Nixpkgs is generally quick to update packages because Nix encourages automation.