The tricky part is that you're basically playing a lottery. If they audit you (and there's a very strong chance that in your lifetime it won't happen), you're likely to hit a bunch of fines. And if you know someone in the IRS they'll just tell you what you owe. And if they don't, basically nobody gives a shit.
That's what it is. Is the risk of a failed audit that much of an issue? Or not. And if you don't have the money / assets to pay the fines, they'll likely just give you a loan or maybe a reduced amount. The IRS rather get some money than no money.
Its... weird.