Under very weak assumptions (basically that an intelligent race would have the desire and ability to explore the galaxy via self-reproducing probes) the galaxy should be crawling with such probes. Even at sublight speeds---and even if probes were only sent between stars when the stars naturally moved closer than a light year from each other just from their orbital movements around the galaxy--in a few million years the probes would be ubiquitous.
What is extraordinary is the negative evidence we get from not observing these probes. It means that there has never been even one intelligent species who had the technology and desire to explore the galaxy. Or even that there wasn't a small minority of them who wanted too--or even one Elon-Musk-esque individual who wanted to.
Given that we can almost do this ourselves (heck, if our own Elon Musk tried, I wouldn't bet against him succeeding) Avi Loeb's hypothesis that one of these probes might have hit the earth isn't an extraordinary hypothesis at all. We should expect to find these probes, and maybe we haven't found them yet because nobody has bothered to look for them.
Bracewell-Von Neumann probes don't exist, as far as any can tell. You seem to be basing your credulity towards one unproven phenomenon on another unproven phenomenon. If anything, the fact that VN probes would in theory be easier than the sort of exotic physics breaking technology supposedly demonstrated by UFOs and yet the entire galaxy hasn't been consumed and assimilated as the math suggests should happen should cast even more doubt on UFOs.
You know.. all of the actually difficult (thus far impossible) criteria and complexity that would required for Von Neumann machines to work in practice.