I wouldn't advocate it as a rule of thumb but in my experience agencies end up producing things like http://boloco.com/ which if you started with Bootstrap you'll end up spending most of your time fighting against it.

As you pointed out, a CSS framework would certainly help but I'd argue in that case you might be better of starting with something else.