One thing I've noticed about myself, and I sense some of it in what you're saying, is that I really like the
easy part of learning new things. When you first start to learn something new, the "gains" come easily. You get this very satisfying feedback loop, going from knowing nothing to being able do some small thing quickly.
But then, the hard part, the part requiring discipline comes in. In the example of learning an instrument, maybe you learn your first pentatonic box on the guitar, and you get a quick hit of satisfaction when you can improvise something pleasant. But then you start to realize, as you learn more, that there's a huge amount of practice required to take you to the point where you can shred a solo.
This is where I often lose interest in some new hobby, personally. And the truth is, there's just not enough time in a life to truly master that many different things.
With that said, though, I think there can still be value in learning the basics of something new. It can tell your whether you enjoy that thing enough to keep investing in it, and you can often go quite far with basic skills. I'll never be a master woodworker, but I'm happy that I learned the first 50%, because that's enough to do some basic house repairs or build a shelf. I'll never be a great piano player, but I learned enough to be able to lay down some chords when I'm working on some music. I'm not going to be a full time drummer, but I can do a basic 4/4 beat when we're all drunk at the end of a jam session and switch instruments for shits and giggles.
I think it's just important to be realistic about how far basics take you and whether you're really going to invest the time it takes to make learning things worthwhile - or if, like you describe, that effort is feeding some other impulse that's not worth your time.
But it is ok to learn new things just because it's fun, and there's a lot of fun to be had being a generalist. So it's really (just like all things in life) just a balance that you need to figure out how you want to strike.