You seem to be implying that morality is subjective, which I think is a dangerous belief. People may believe their actions are ethical, but that doesn’t necessarily make them correct.
> Haidt and Craig Joseph surveyed works on the roots of morality... From their review, they suggested that all individuals possess four "intuitive ethics", stemming from the process of human evolution as responses to adaptive challenges. They labelled these four ethics as suffering, hierarchy, reciprocity, and purity. According to Haidt and Joseph, each of the ethics formed a module, whose development was shaped by culture. They wrote that each module could "provide little more than flashes of affect when certain patterns are encountered in the social world", while a cultural learning process shaped each individual's response to these flashes. Morality diverges because different cultures utilize the four "building blocks" provided by the modules differently.
Perhaps these four "building blocks" -- suffering, hierarchy, reciprocity, and purity -- are sufficient for an objective basis upon which an intersubjective understanding of morality can be determined.
Then again, even the notion of objectivity is questionable and culturally-dependent...
There's a big logical leap between (for example) observing that many cultures proscribe lying, and saying that one shouldn't lie.
See Arthur Leff's article: http://digitalcommons.law.yale.edu/fss_papers/2814/
1. Some philosophers have a descriptive ethic (this is what people do) rather than a prescriptive ethic (this is what people should do). This is certainly interesting, but it isn't what most people mean when they talk about ethics--it doesn't give us any guidance on how to behave ethically.
2. Some philosophers try to work backward to a prescriptive ethical principle from a set of ethical prescriptions which are widely accepted but, importantly, unproven. For example one might start from the prescriptions "murder and lying are wrong, giving and learning are right" and attempt to come up with a prescriptive ethic that unites these prescriptions. But "murder and lying are wrong, giving and learning are right" aren't objective facts even though they're widely accepted, so any prescriptive ethic that comes from these prescriptions can't be objective either. It's not coincidence, for example, that in the west, the atheist/moral realist philosophers mysteriously often start from Judeo-Christian prescriptions.
Even the "Christian Post" will agree he's "partly right": https://www.christianpost.com/news/atheist-sam-harris-right-...
That said, I haven't read the book you mentioned, so there might be something in that book which isn't in his talks which would persuade me.