you only need to start with a handful of people. find one or two other people interested who will commit, and you can get it started regardless of who else shows up.

make sure you don't make it necessarily very CS-specific, either. business and art people might want to get involved, too, and would be quite useful. random other people might get a good learning experience on the entire process if they walk in with "a great idea".

The best way to attract a crowd for an event on college campuses is to offer free food.