They start being less useful when you start treating them as "I can send them ill-specified stuff, ignore them for 10 minutes and merge their results", as things spiral out of control. Basically "vibe-coding" as a useful concept doesn't work for projects you need to iterate on, only for things you feel OK with throwing away eventually.
Augmenting the human intellect with LLMs? Usually a increase in productivity. Replacing human coworkers with LLMs? Good luck, have fun.