I've led a couple of great group trips to places I'm passionate about by emailing a group of friends and essentially saying "I'm going to go to X, here's what I think is amazing about X, and here are the proposed dates", and essentially getting folks to opt-in or not. Once the trip is locked in, there's certainly some discussion and folks say "I heard there's Y in X, let's do that!" and there's probably time to for that activity. But yeah, you need one master of the itinerary.
I do disagree somewhat about the cost splitting thing though - I've had several good experiences using SplitWise, although the groups have been relatively small - from 3 to 6 people or so. Maybe if you have more people, or weird family dynamics, or technophobes, it doesn't work out.