Tracking server-side is fine, it is assumed the server logs contain a record of my visits and I have no problem with that. I think what most people object to is the third-party tracking that so many people use.

Company A tracking my visits to Company A's website = OK

Company A using Google Analytics to track my visits (while also enabling Google to track me across multiple sites) = Not OK

EDIT: (replying here as we've reached max comment depth) - I was unaware that it is possible to use Google Analytics server-side only (is this true?) but I hope my original point is still clear, DIY tracking is fine.