We are similar in size for our Angular2 application. We've also had very similar problems, especially setting up AoT compilation took pretty long and was a painful process.
I was fortunate to have some key help at times from Rob Wormald of the Angular team, giving me some key insights about making sure my classes, interfaces, etc. are all exported publicly so they can be statically analyzed, and making sure to split my webpack configs & tsconfigs for dev & prod builds since AOT compilation is too slow for typical dev workflows (although you also want to be able to do it on dev to debug AOT compilation related errors) - I ended up with 4 different webpack configs for sanity's sake.
That said, the CLI does support AOT compilation and has that support built into it now - if I were starting an application now, I would probably just use the CLI.