"""I say this as a person that knows vim well."""
These two phrases don't go together.
Would you say you knew how to play a piano well if you were still consciously thinking about where each note was?
If you don't use vim to the level that it's subconscious and "in your fingers" it might not seem all that useful...
Fwiw "ag" and the various plugins around it make your use case pretty trivial.