There is no solution where 100k loc is not going to be challenging to keep over time.
In my opinion, Terraform's big issue is that it was born as a declarative tool for managing infrastructure. Large configurations (IMO) necessarily ossify, because you don't have an imperative language that makes small progressive changes toleratable - it's a giant interrelated lump.
What's worse, when it grows, one needs to split it in different configurations, and one loses referential safety (resources will needed to be linked dynamically).
A, say, Chef project of equivalent size, can be changed with more ease, even if it's in a way, even less safe, because you have the flexibilty of a programming language (of course, configuration management frameworks like that have of different set of problems).
I'm really puzzled by the design choice of a declarative language. Having experience with configuration management, it's obvious to me that a declarative language is insufficient and destined to implode (and make projects implode). Look at the iterative constructs, for example, or the fact that some entities like modules have taken a long time to be more first class citizens (for example, we're stuck with old-style modules that are hard to migrate).
The puppet migration was definitely more painful, because of the entangled code.
There are inherent problems in the TF framework and the migraitons. 0.12 introduced for loops, and 0.13 added modules support to them. So a proper migration should convert deduplicate resources into lists of resources. This is painful for big models, since one needs to write scripts in order to convert resource associations in the statefile. And hope not to miss anything!
Due to the strictly declarative nature, it's also difficult to slowly move duplicated resources into lists, and handle both of them at the same time.
At this time, our time is stuck with a certain TF version, and can't move without spending considerable resources.
I'm puzzled by this comparison. I consider both of these to be primarily declarative languages. You declare the state you want puppet or chef to enforce, not how they get there.
E.G. https://puppet.com/blog/puppets-declarative-language-modelin...
To be more accurate, I'd say that Chef has a declarative structure supported by the imperative constructs of the underlying language, and this is what makes for me a big difference.
Consider the for loop as example. By the time it was added (v0.12), there was a (200 pages) commercial book available. And there are people in this discussion stuck at v0.11.
The difference in the declarative vs. imperative nature, as I see it now that the for loop is implemented in TF, is that it's embedded inside resources, that is, it fits strictly a declarative approach, and has limits. In Chef, you can place a for loop wherever you prefer.
Object instances is also another significant difference. It took a while for TF to be able to move (if I remember correctly) module instances around (that is, to promote them to "more" first class citizens), which made a big difference. In an imperative language, accessing/moving instances around is a core part of the language. In Chef, pretty much everything is global - both in the good and in the bad. But certainly the good part is that refactoring is way more flexible.
I think TF has always been plagued by repetition; in my view, this is inherent in the "more" declarative approach (since they're trying to embed imperative constructs in the language).