57 karma · joined December 23, 2014
certainly some of it is but not the lion's share - I have a much simpler (private) codebase which scales pretty similarly afaict.
the complexity of Maxtext feels more Serious Engineering ™ flavored, following Best Practices.
For a while I was using an FID variant for evaluation during training, but didn't find it very helpful vs just looking at output images.
there's a lot of room for improvement in conciseness of code. I would still be surprised if it was meaningfully possible to write a full-featured modern OS with one page of APL
When I say it feels like we spend a lot of time red teaming, that means I think we spend somewhere between 30 and 60% of research time trying to break things and see how they fail. This is fully compatible with not immediately implementing things - it's much less expensive to break something /before/ you build it.
The catch is that univalence is inconsistent with LEM at h-levels greater than -1, but assuming it is perfectly consistent for -1 types, which can be thought of as the "at most true" propositions of classical logic.
What the univalence axiom says is that you can treat types you have proven isomorphic as equal.