I mean, obviously I realize that some people will take what I write less serious if I don't correctly use apostrophes, so it's worth the effort. I also think the whole "don't split infinitives" thing is utter bollocks, but if you know that a section of your audience will get all triggered over it then it's probably best to avoid (even though it's silly).

My point isn't that you or I should stop writing apostrophes, but that you should stop being bothered by other people not always using the correctly (or rather, our perception of what's "correct").