If it's a feature or integration test, why is it a problem to pull in the other packages?
If it's a unit test, why not mock out the other dependencies to focus on the narrow behaviour?
Go makes the latter both easier (implicit interface typing) and harder (lots of libraries only providing physical structs).
I have to admit I've grown nervous about tests that require you to crack open an object to inspect its internal state. I've often seen designs that "need" private variable inspection recomposed into a more functional style or broken into smaller, cooperating modules/objects.
Put another way: if the module's behaviour is so complex that you must inspect private state to assure behaviour, perhaps the module is too complex overall.
Of course YMMV, consult a doctor etc etc