I see this as the main tension in the extracting-class viewpoint. Assuming your language allows it, you can extract out a class that's still private to the current package, meaning that the public API is unchanged. But there are costs to extracting out that class.
I don't see why the class needs to be extracted in that case. If it makes the code easier to follow, do it; if it adds bloat and boilerplate, don't do it.
The key thing is the ability to expose stuff for testing only, not in the public API. Depending on the language, you might not need to extract a class at all -- just make some fields or methods package-private.
I guess if your language doesn't provide a way of doing that, extracting a class is a reasonable workaround because you can segregate your public API into high-level and low-level parts, and recommend that end users avoid the low-level parts. But that's not ideal; better to hide the low-level stuff if you can.