Fair use is a defense. You, at best, get sued, have to pay a ton of money in attorneys fees, on the chance that you "bet" it's going to work. Tough sell.
Why would Facebook sue over this?
Because they want the data their button+js gets for them, and if they let you get away with not using the js, then anyone can get away with it. Better to shut it down early and painfully.