I don't know what you mean by "not empirical", but if you mean that Darwin just wrote "hey, here's a cool idea" and didn't have empirical evidence for it then you're just flatly wrong. If you mean that now we don't have empirical evidence for it then either you've been terribly deceived by others or you're lying. (Probably the former. My condolences.)
Something can be a theory and also factual. And there can be a theory of something that is a fact. If you think "X is a theory!" is some sort of refutation of people who regard X as true, then you are 100% wrong about that.
The usual progression goes like this. Someone comes up with a theory (in the vernacular sense). Say, Darwin's theory of how living things got to be the way they are. Some theories are more or less correct from the outset. Some are just plain wrong. Most are somewhere in between. If the theory seems worth taking seriously, other people try to work out its consequences in more detail, and go looking for evidence for or against it, and refine the details. When the theory is all wrong, it will hopefully get knocked down as this happens. (Examples: "cold fusion", Lamarckian "inheritence of acquired characteristics".) When the theory is basically right, the wrong bits will get corrected (e.g., Darwin expected inheritance to be a sort of "mixing" process, which isn't really right and produces some wrong intuitions) and new ideas will be brought in (e.g., Darwin didn't know about genes), and as repeated investigation doesn't refute the underlying ideas it becomes increasingly implausible to reject them.
Today's understanding of evolution isn't identical to Darwin's. He made some mistakes and some wrong guesses and there was a lot he didn't know. But we have a lot more evidence than Darwin did that populations of living things evolve over time, that a major cause of the change they undergo is natural selection, that even very different-looking living things have common ancestors, etc.
Darwin had a "theory" in the vernacular sense: an idea that seemed to explain a lot of things and might or might not turn out to be right. It was a good enough idea that scientists after Darwin put a lot of effort into investigating and refining it. We now have a "theory" in the "grand scientific edifice" sense, and while details will continue -- ahahaha -- evolving, there's little scope for reasonable people aware of the evidence to doubt that its central ideas are facts.
[1] Even in the "vernacular" sense. It's true that sometimes "theory" means "grand scientific edifice, well evidenced and elaborated with mathematical sophistication", but it doesn't always, and when Darwin called evolution by natural selection a "theory" he didn't mean that.
There is nothing that we know with a higher degree of certainty then a scientific theory, such as the theory of evolution, the theory of general relativity, the theory of quantum mechanics, the theory of newtonian mechanics etc.
And that's not even counting the other achievements of the Manhattan Project, such as the first artificial nuclear reactor.
Please take a minute to consider this and realize that you should never repeat what you just said.
-- https://en.wikipedia.org/wiki/Level_of_support_for_evolution