It's not required, always, to pay tithing to attend the standard sunday services in the LDS faith. But if you wanted to get help from your bishop, who has the ability to grant about $5000 USD to anyone of his choosing (I think per month), he will most certainly note if you are contributing to the church if you ask for help paying utility or medical bills, which is not uncommon. Mormons that wish to be married or sealed in the Temple will also be scrutinized of their contributions.
Like you noted, when relating it to Government tax, if you decide to never pay taxes again, you wouldn't get thrown in jail or forced to leave, but you wouldn't get to enjoy the simple things in life, like roads, police, ambulances or the fire department.
You could probably live a life like Henry David Thoreau, but it probably wouldn't be very comfortable.