The problem is that people interested in manipulating the election could come to your place and offer you money in exchange for your vote.

That's a damn good reason for using polling booths in public places.