That's the massive downside to every implementation of bubble cursors / something similar I've yet seen. I'll label them "naive" implementations, because they're so mindlessly implemented: it selects the nearest clickable item, with no respect for any relation to movement or chaotic spots where you can select different things with single pixel movements.

Though I do see that this could be useful once they're more polished. I just don't get why everyone's drinking the same implementation kool-aid, rather than seeking to improve on it.