I've always thought down-votes could be removed without losing much. Rule breaking content can still be flagged and it removes one low effort kneejerk response option.