Automatic toolchain switching does not trigger compilation of later Go toolchains; it only attempts fetching a blob (if available) and using that to perform builds[1]. If a system supports Go 1.21 but not 1.23.1 (e.g. Windows 7, mentioned in a sibling comment), then this project will fail to build on said system. Likewise, if a user on Go 1.21 has disabled automatic toolchain switching, the Go CLI refuses to build the project at all.

Overall, I would say the minimum required Go version is indeed the Go version declared in the go.mod, since that is the declared minimum required toolchain.

[1] https://go.dev/doc/toolchain#select