Sexual divergence is a game theoretical consequence of the differing costs two parents might have to form an offspring via sexual reproduction. Suppose a population of hermaphoditic sexually-reproducing species grows large enough to be comfortably self-sustaining and there is no difficulty in finding / exchanging gametes a sexual partner. At this point, every individual is equally capable of mothering or fathering the offspring, but one (say, mothering) ends up being more expensive because it requires more resource investment. Then a genetic quirk that causes a particular individual to preferentially spend more time fathering will be beneficial towards individual fitness, as the offspring/cost ratio is skewed in their favor. Eventually, one or more such genes catch on, and form the first steps of the evolution of the subpopulation with them into a male-type. And, over time, those better at specializing in being male-type will outcompete those who are less specialists at finding mothering partners, increasingly diverging this subpopulation towards maleness.
Now a population of mixed male-type and hermaphoditic-type individuals by definition can only skew male-type (in the sense that the average individual over the entire population somewhat prefers fathering over mothering). But each offspring has one male and one female parent; i.e. the population of parents does not skew in either direction. Run the math and you'll find that, counting hermaphodites as 1/2 in each category, there are more offspring per female-type than per male-type: those hermaphodites who prefer mothering are now the sexually fittest individuals in the group. And much like in the case with males above, a genetic preference towards mothering will catch on among the hermaphoditic population and begin skewing a subpopulation ever more female-like.
Eventually the male-type and female-type sexes fully diverge from each other, and the initial population of hermaphodites get out competed by the sexual specialists and fall solely into the fossil record.