I've considered doing that, was it hard to set up?
Here's one of the latest attempts I've done, although I'll note I still haven't managed to get an actual working Apple Silicon build. It keeps building for x64. The Windows builds were smooth sailing since I didn't need to worry about cross compilation yet, though I'm sure it'll be interesting when I eventually try this for AArch64 Windows.
https://github.com/puyonexus/qt-sdk-builder
(I don't do this for Linux since it's easy to get a Qt SDK on Linux.)