I think what would be helpful is to have an option to ignore all voting for purpose of sorting, so that you can vote if you want to, but if you don't like to, you can disable that option and it will display only in the chronological order (it could hide the voting options too if you wanted to, but I suppose it doesn't have to do).
chronological order would be a good option. I sometimes never get to the other pages when there are hundreds of comments.
Or make a comment necessary to downvote so hopefully there is an explanation. I have gotten downvotes and wonder why.
I reckon 90% of downvotes I made were accidental. The buttons are really tiny on a mobile phone!
I know the upvote button is small. I'm a noob so I don't have a downvote button so I can only guess they put it right next to the upvote one.