This. A quick scan of the wikipedia page for diamond material properties suggests you are very correct. It appears very chemically inert, with some outstanding exceptions: "Resistant to acids, but dissolves irreversibly in hot steel"

https://en.wikipedia.org/wiki/Material_properties_of_diamond

Also, removed/liberated particles of Diamond from the workpiece which failed to fully chemically dissolve into the slurry would then contribute to the abrasive in the slurry. If the slurry abrasive was not also diamond, then that could lead to some serious scratch/gouging of the work surface.

Perhaps not insurmountable, but wow, that sounds like a stiff challenge, especially when accounting for cost.

I wonder if diamond would be machinable with a dry (plasma) etch instead? I am purely speculating here, this is far out of my wheelhouse. But SiO2 is already very chemically inert (though considerably softer vs diamond), but manufacturers regularly dry etch it.