In addition, this isn't a case of, say, someone believing that people would be better off if they were occasionally hit with water balloons and thus deciding to throw them at random passerbys. (Whether that person would be justified in pelting everyone is subject to debate.) Instead, this is a case which only effects the people who chose to become his students. And how is he hurting them?
He isn't hurting their future prospects of getting a job (though I guess that maybe you could argue that he is diminishing his school's reputation or something like that). He isn't hurting their ability to learn--I think he's right that not using grades helps students learn better, and either way we generally give teachers the right to choose how to teach their students, since they're usually the most qualified to make that decision. The only possible way that he's hurting anyone is if later on one of his students get hired when they shouldn't have (or visa-versa), and arguably that decision isn't on him; if he's truly wrong about grades, then the rest of the students' grades should reflect that.
(Perhaps you could say that if some of the students wanted a grade, they should be given one, but I feel like he would be willing to give them feedback/change their grades if they asked him.)