Correct. In every way.
Deming process people call this "one way flow" of information.
Paraphrasing, mostly to continue my rant:
Versions are just labels (tags, aliases) you share with clients, users. Semver is no worse than any other scheme. And fretting about semver gives scrum masters something to do.
Build numbers are used for internal processes. Like the "found", "fixed", "closed" fields that QA tools should have (but rarely do).
As you said, version is annointed AFTER an artifact is accepted, moments before release. Like an imprint. (For all workflows prior to "Test Into Prod", so ymmv.)
Then everyone can also precognitively divine a future version, so stuff like press releases can be front loaded. Genius!
Few things drive me more nucking futts than teams which predetermine version before attempting a build. Oops, build failed. Manually increment semver. Try again. Rinse, lather, repeat. Then ferret all wiki references to semver and update those JIRAs.
Trying to explain this to noobs is harder than advocating using knives for cutting to someone who's only ever seen spoons. People get it when they see it, but can't imagine it, so the push back is exhausting.