Credit unions, or small banks are likely to be helpful in some situations. When building our house 8 years ago we had a lot of trouble getting the construction loan mostly because of 1-2 bad comps in our area. One of the big banks turned us down with no recourse with an assessment that included a picture of the wrong lot (a farm field across the street). Another said no one should build a house under 3k square feet, so no to our plan. Our little local bank was able to actually take a look and approve us.