I think programmers mainly care about memory leaks because of the potential negative consequences of using up too much memory -- programs crash or fail, or services frequently restart, or you have to provision additional costly server resources to keep things running well.

If your program continuously allocates memory that it doesn't free and doesn't serve a useful purpose, it doesn't really matter if it's, e.g., being referenced by some cache or just completely unreferenced -- it's still causing the same problem and has the same solution (that is, release the memory when you're done with it -- whether that's by calling "free" or clearing the reference, or whatever your memory management system has)