The federal government dominates the mortgage market -- through direct insurance, guarantees, and purchases. It's tough to compete with them. The only comparable market I can think of is student loans where some companies have found a way to pick off the most credit worthy borrowers (e.g. SoFi). But in student loans the government does no underwriting and offers a uniform rate, some of which are quite high in the current interest rate environment. Whereas in mortgages the government underwrites and offers very low rates.

How exactly anyone expects to make money by lending out money for thirty years at 75 basis points above the risk free rate, with a zero premium call option, levered 4:1 or greater, and with low recovery percentages if the security needs to be seized is beyond me. That's even before getting into the high overhead to deal with servicing and regulatory compliance.