My father's saying was "I can work 24 hours a day whether I want to or not". He also used to say "If you go to the bank for a loan and you say you are 'self-employed' they hear 'un-employed'"
So yeah, it's a lot harder to deal with banks if you're self-employed, especially as loans seem harder get these days.