I feel like the key is knowing that your problem domain is fine with there always being garbage collection in the loop. Otherwise there's the risk that down the road you'll have to start tracing the code, rewriting things to reuse objects, etc etc.
Caching is often not such a bad idea regardless of GC.