I believe you're close to correct. According to a paper in 2013 by Graham Coop (an evolutionary biologist from UC Davis) every living person of European descent, if you trace their family trees back 1000 years, likely shares the same list of ancestors. And that list of ancestors is more or less all the people alive in Europe at the time who had descendants surviving to the present -- about 80% of the people alive at the time. Joseph Chang, a statistician from Yale, has also written a great deal on the subject.
Not the most detailed link, but illustrative: