Can't speak for GP but for me the kicker was that you cannot get any guarantees on the mutability of an object, and the unexpected quirks that could result from it. Example:

https://redmine.ruby-lang.org/issues/6037

You can (must) be pragmatic, of course; and assume that all will go well. But under the hood, it means a great many objects get pointlessly allocated and reallocated - or not, sometimes, with hard to find bugs occurring when a gem author overoptimizes their code to avoid making a few object copies, and ends up mutating your object's properties without you realizing it.

Things have improved since, with e.g. strings being frozen by default if you want them to be since a few versions ago. Even in this changelog, there's a least one point related to memory allocation that touches the performance consequences of needlessly allocating new objects.

Back then, I found Obj-C (and Swift, maybe?) more conforting in this respect, with most things being immutable unless you explicitly requested the mutable version. And I liked ARC a lot. (I haven't programmed much in recent years, so can't say what I'd use today if I were neck deep into code. Probably Swift.)

Another issue was that Ruby was then attracting a lot of end-users that had no formal software engineering skills. Which is fine for day to day tasks; not so much when they start distributing libraries. (Always vet your gems' authors and source code.)

I still love Ruby, mind you. It's beautifully expressive and it's still my preferred language for non-trivial "glue" tasks.