But we don't have empirical evidence of black holes - we have circumstantial evidence of very dense blobs - they could be neutron stars or other things.
The reason they assume they are black holes is that if a neutron star was that big it would be a black hole - but if loop quantum gravity is right, then it would become a degenerate star instead and not a black hole.
Getting rid of black holes would help a lot BTW because they contradict a lot of well established theories, plus they can not be reconciled with relativity.