Very true,
Running something on the open Internet is super annoying. If you properly subnet, or restrict to VPN, I can see it as useful but then you have a whole other host of problems to conquer.
If you're doing a static page or such, no issue, but the second you're hosting a webapp or need to have a database, yeah.