1. There's an agility to being able to change things yourself, at 2am if necessary. That's very important to startups.
2. There's a discipline to not accruing unreasonable technical debt of the type that will sink you two years in that you can't expect from someone who doesn't deeply care.
3. Managing devs generally requires understanding what's going on. Unless you're ready to hire a CTO with a track record, a non-technical person can't do that.
There's also a long tail-end of reasons. For example, prototyping and tinkering nights and weekends is "free," and can contribute a surprising amount of value. It can't be managed, though; you can't just pay other people to play around, and expect that to be focused in ways which might add value.
*Every* time I've personally seen the playbook of a non-tech founders for a tech startup, it didn't end well.