The bats don’t need to be at all near the market, the theory is an intermediate host was in the market. A bat could infect the host population days prior or years prior. A long potential span of time.
It might be illustrative to look at MERS. They found camel populations >3,000 miles away from the outbreak with antibodies to MERS. One potential explanation is the virus doesn’t sicken camels and had been spreading in their population for a long time prior to transmission to humans.
Or we look to SARS. They did not find SARS in a bat population, but after 5 years of searching they did find a cave where all the building blocks existed. It was a couple provinces away from the outbreak. >500 miles.
So despite your beliefs, how many bats freely roaming the neighborhood may not actually be a much better probability analysis than flu viruses in Antarctica. /s