Nice trick. Couldn't you embed the query though, compare it to the embedding of the categories, then ship only categories that are close to it in the prompt to a smaller model?
But honestly, it only works for common knowledge that's already in the LLM. If the target document contains very niche or private information, then the hallucinated answer's embedding can be even farther away than the query's.
Even better is to search the corpus first with like naive BM25 / embedding search, aggregate over top N to get most representative categories, then have the LLM categorize in that set.