I've actually found the exact opposite. The library ecosystem is rich and mature. Haskell is, by default, "concurrency safe" because of referential transparency. You can safely "async" and compose almost any library in the Haskell ecosystem without worrying about shared memory etc underneath.

Also, a lot of the really common libraries like text, attoparsec (parsers), aeson, networking, etc are highly tuned for low latency and performance. Many use compiler rewrite rules and techniques called stream-fusion to compact a lot of the machine code away. Also aggressive inlining etc can be done.

I'm sure there are some memory-heavy or poorly optimized libraries out there but that's certainly not the norm. I've had no problems with the libraries off-the-rack.