We have considered it, however, because there's still a need for a firmware, the first stage bootloader and the firmware itself are using a common driver framework, it just makes things easier. Besides, the firmware will later run an RTOS (for example LittleKernel) and Uboot doesn't provide RTOS-like services.
Would be nice to chainload U-Boot from your firmware/loader though, because FreeBSD boots via U-Boot.