I wrote the blog post. AMA.
Another question, have you tried using Accelerate framework to solve performance bottlenecks (or save yourself from having to write your own calc code)?
In principle, I could get rid of reference counting overhead by using value types or immutable data. I couldn't see a simple path to doing that without re-architecting everything (with no guarantee that the end result would not just have different performance issues.) For the moment, I'm awaiting compiler improvements before re-evaluating the tradeoffs. There is certainly room for the compiler to reason better on eliding retain/release. https://github.com/apple/swift/issues/58549
Yes, the code does use Accelerate where applicable. That is one component of the numeric evaluation. It addresses the lowest level of things like evaluate the sin function on every array element, or multiply there arrays element wise. Performance tuning is a game of whack-a-mole. There's always another bottleneck somewhere.
Can you say more about the parse tree impact on performance?
Is the expression left in a tree form and effectively "interpreted" from the AST for all points? Is that the critical path?
My basic question would be: why not go back to flex/bison/yacc/whatever via C-FFI? (But I think it would still be bad, since you'll want to get to a Swift data structure for your ops and those will still have the Arc issues)
I did investigate maintaining the flex/bison parser, since its generated state machine C code is more robust than my handwritten recursive descent parser when presented with pathological input. However, as you say, since I need a Swift data structure in the end, there is little to be gained and a lot of complication bridging via a C-FFI.