Core LaTeX is concerned with the flow of text in printed articles, and Knuth published the text layout algorithm “Breaking Paragraphs into Lines” http://www.eprg.org/G53DOC/pdfs/knuth-plass-breaking.pdf
Originally, HTML did the same thing, it was used to emulate the written page and layout text into a box. But not everyone wanted their site to look like a printed article, so CSS and new layout modes sprouted over the years. Today there are seven of them: https://developer.mozilla.org/en-US/docs/Web/CSS/Layout_mode
RectCut is fundamentally most similar in spirit to CSS flexbox, and fundamentally similar to a bunch of other box layout managers in other languages and UI toolkits, as you can see from the comments.
HTML includes a version of it, TeX and DTP apps use various versions of it.
Edit to add: I don’t know the details of TeX except that it uses a hierarchical “box model”. I suspect it’s fairly similar to HTML before flex-box. HTML flex feels like a refined version of Java AWT layout managers (a more successful evolution than Android’s painful variant).
Once you start adding constraints like min/max size and relative sizes it becomes more complex but still manageable. You’re still only reasoning about rectangles and their constraints.
Adding rendered text in the mix increases complexity immensely. Determining the size of rendered text is a very complex problem space in itself involving fonts, dpi, text flow, code points, LTR/RTL, ligatures and more. It’s not just rectangles anymore.