Note the example above:
(select a from nums) order by b;
a
---
3
0
1
2
(4 rows)
If things worked as you described, that would also fail.So Postgres is, in some cases, cheating with the "new set" thing. The weirdness is that it does it inconsistently.