We manage to do this for endangered wildlife too without anyone counting every single specimen; why shouldn’t we be able to estimate how rare a book is?
After that you are diving into forum posts etc to see if you can find anyone who has even mentioned owning a copy or having seen a copy.
I don't know what happens to some works. Supposedly thousands, tens of thousands, or sometimes apparently a million or more copies published and yet not a single copy surfaces for years.
Most rare books are rare because no one cared enough about them. Ie most rare books are rubbish.
1. Why are you here?
2. What is the purpose of this comment?