- traffic tiering and balancing based on request's perceived amount of work
- low latency data loading during our calc loop
- implementing a selector framework to unify all the stores used by the application
- redesigning the formula editor to make human friendly yet expressive enough for the most hardcore user (fun UX challenge)
We really ought to start blogging more about these things :D.
I work in this space. I don't envy this. Been through it more than once.
Did you also have challenges with circular references?
For circularity, we found that we could keep the UX dynamic by making a deep copy of the circular part of the DAG behind the scenes, asking the user to determine which variable in that path should be "resolved", hard coding that variable in the copy, then solving that variable to zero through a newton optimization. Once optimized (in parallel to main graph), it feeds back into the main DAG just like any other dependency.
Would be a silly approach in Excel, but not so much here.
Looks like you've traveled the enlightenment path to array languages! The first time I read Whitney's K language description was a singular mind-expanding moment.