Regarding the number of onion addresses available you are wrong. Addresses are encoded in Base32 which means there are 32 characters available. So there are 32^16=1.208925819614629174706176×10^24 addresses available.
Not taken but available.
I agree with the fact that the crawler is really simplistic. But the project is new (2 months I think) and has to evolve. You can make a PR If you want to help me to improve it!
As a defense against the parent comment, though, this proves way too much. It doesn't matter how much k8s you throw at that, you're never going to so much as find your first site, if you're looking at the problem that way.
That's not really a relevant number here.
>Addresses are encoded in Base32 which means there are 32 characters available. So there are 32^16=1.208925819614629174706176×10^24 addresses available.
I sorta understand what you mean, technically it's 32 characters per position (5 bits), and 16 positions. In v2 .onion addresses, that is.
v3 ones [1] are 56 positions, but not all the bits are used for addressing, so the same formula wouldn't quite work to calculate real theoretical capacity. IIRC someone already made site which generates unlimited links to v3 addresses (without having them lead to anywhere, of course).
[1] https://trac.torproject.org/projects/tor/wiki/doc/NextGenOni...
V3 addresses are just ed25519 pub keys and a couple byte changes. You can use Go libraries like Bine [0] to generate as many V3 (or V2) addresses as you want from keys.
0 - https://godoc.org/github.com/cretz/bine/torutil#OnionService...
[1] https://metrics.torproject.org/hidserv-dir-onions-seen.html
edit: onion services, not addresses