If we start slowly, focus not on delivering a working product ASAP but a nice codebase that's accessible for other developers to read, carefully split modules so each part of the browser is a mini project by itself, with clearly defined business logic, outward connections (GUI, APIs, Drivers, etc), it can be done.
Yes, it will be a lot of work. But with a tidy codebase and a welcoming community (at least for a programmer) that "a lot of work" will instead become a playground to hack whatever component they desire.
The problem is that people don't want another Linux-like hobby project, they want another Firefox