While a regular bank won't give you a loan to invest in the stock market, stock brokers will: It's called margin trading, and it's exactly borrowing using the stock you have as collateral, just like you put the house as collateral in a mortgage. Since stocks are far more volatile than real state, you are allowing your broker to sell your assets if your account is about to go underwater, in what it's known as a margin call.
You can have a 5:1 margin account, like the one you describe, as an individual investor: If you have 100K to put in a brokerage account with your own money, you definitely have enough money for a brokerage to let you do it. It's just that it requires attention, because with a 5:1 ratio, a 5% loss just became a 25% loss. Ignore the market in the wrong week(say, in early October 2008), and all your margins get called, and you are in deep trouble.
Still, if you have 5 $100K rental properties, and you aren't bleeding a lot of money for a company to take care of it all for you, chances are you are still paying quite a bit of attention to your apartments anyway.
As far as whether mortgages or rentals are more expensive, it completely depends on the local market and local laws: In a pro-renter environment, rental is more expensive, because nobody wants to rent apartments out: People would rather let apartments sit empty. In a pro owner environment, at a place losing population, renting might be cheaper.