I wouldn't call it consensus, just a persistent claim by a large fraction of developers. They finally got to prove their claims with x32 ABI, which was ILP32 in amd64 mode, including the additional registers. Few if any people used it, nor did it show better performance in practice.
Regarding 64-bit support: by the time amd64 chips shipped people had been writing software for the 64-bit Alpha for 10 years, and 7 years for sparc64. IME most open source software was already 64-bit clean and worked well out-of-the-box. Back then the open source community, and especially GNU projects, heavily emphasized platform and hardware portability.
Perhaps the situation on Windows was different. It also didn't help that Windows kept long 32 bits, which had the effect of breaking code that cast between pointers and long (intptr_t didn't come until C99). The 64-bit ABI for all Unix platforms (AFAIU) carried forward the relationship between long and pointer. I don't think I ever recall seeing Unix or open source code casting a pointer type to int, only long; it was Windows software that presumed pointer->int conversions worked.
We use it for our specialized analysis framework. The real world performance gain is just shy of 30% which is impressive since all it took was a bunch of compiler flags.
Personally I think x32 is a vastly underused ABI. Every application that is unlikely to use more than 2GB should use it. It is literally free performance.
It is entirely possible to make a modern system that use far less than they do today. If you're gaming it's a different beast, but for normal OS and programs it's certainly bloat.
If I've learned anything in my computer career, it is that this is an evergreen comment. I remember reading Q-Link posts from Commodore 64 users with this complaint when the wasteful C128 came out.