Note that currently with ULA if you have dual-stack IPv4 will be given priority over ULA. There is a late-stage—Submitted to IESG for Publication—draft that will change this:
* https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc672...
* https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc672...
That draft doc seems to fix multiple problems at once.