I can apply your statement to other celebrity academics: scientists, astromoners, physicists, etc. And for all of those professions I have a hard time believing that any "celebrity" (i.e. Steven Hawking, Brian Cox, Noam Chomsky) that emerges from them produces a "toxic atmosphere of fraud" or undermines the work of other professionals in the field. But it may depend on who you consider a "celebrity".
That aside Latour was infamously an originator of the "Post Truth" and his support for utter nonsense. It is true that social aspects affecting what science is considered, achieves mindahare, is funded, and catches on and progresses. It is undeniable as seen by the derth of statistics and genetics papers from the USSR when both were declared bourgeoisie pseudosciences. It affecting what is actually right? Hell no. Although at least to his credit he realized what a monster he had created.
Philosophy isn't neccessarily bad but it isn't neccessarily good either. Given that it is often defined via its independence from the real world there are many spaces to explore. Which includes the potential to be absolute pseudouintellectual or antiintellectual nonsense. That makes it different from the other examples - they are verifiable and it is possible to call bullshit and outright prove them wrong.
"Celebrity philisopher" may just mean that it flatters preconceptions without the ability to be countered via hard proof that "no the moon is not and cannot be made of cheese".