> so many things are built with QWERTY in mind

I found this to be the biggest issue with using Colemak or any other alternative layout. I use my own computers exclusively so I had complete control over using my layout, but it just got so tedious having to rebind everything all the time and I would often run into games and programs where you couldn't rebind keys at all. Trying to rebind keys to work in a sane way in Emacs was an uphill battle that never ended.