Cost of living where I a, in NA is def higher but I also know that my kids will get a better education overall. In fact, you have to pay a fortune in tuition fees for your kids to have a similar education to public schools here in NA.
I am not sure what is your situation but if you have more than 1 child, things start to get dicey cost-wise. I don't think there is a lot of disposable income to go around, and that's for a relative low cost of living city in Brazil like Recife. I guess if you are childless things would be different and I can see the fun times outweighing the rest. But for me it is a solid "no way".