I did GCC too, https://github.com/puppeh/vc4-toolchain.
The assembler/linker I'm using is not ideal, I want to get MC code emission working eventually. I saw that you mentioned limitations on ld/st, why not use lea for data?
For example:
BB1_12: # %sdram_clkman_update_end.exit2
mov r0, 2114982312 # long
ld r2, (r0)
lea r0, .str8(pc) # PCrel load
lea r1, __FUNCTION__.sdram_init_late(pc) # PCrel load
bl xprintf