I do meet with women, regularly, just never alone. My refusal to meet alone with women is hardly irrational. It is one of the safest and sanest choices a man can make in the current environment where a mere accusation is often enough to cause the loss of a job and future prospects.
I sincerely hope it never happens to you, but odds are high that it will if you acquire enough power at work. Not a popular view, I know, but still true.