Ironically this is also what occams razor would demand from good Science, so you'd have a win win scenario, where you both create good software and good research, because you focus on the simplest most minimal approach that could possibly work.
Simplicity is a nice dream. The realities of research are very often stacked against it.
A Google search makes it look like Julia has a mechanism where you can extent the sets of overloads of a function or method outside the original module. The terminology is different (functions have methods instead of overloads in their speak). I don't see how that feature solves the problem in practice.
Check https://stackoverflow.com/questions/1801216/what-is-the-diff... and https://www.youtube.com/watch?v=kc9HwsxE1OY
I've seen research groups drown in their legacy code base.
The issue of juggling too many balls you describe is one you only have to begin with because the state of the art implementations are so shoddy to begin with.
Research suffers as much as everybody else from feature creep. Good experiments keep the number of new variables low.
And you say it yourself: good experiments change a single variable at a time. So how do you check that a series of potential improvements that you are making is sound?
Note that Simple != Easy or Naive
Hardcoded structures is potentially exactly the kind of simplicity needed.
What's not simple is a general "this solves everything and beyond" code-base with every imaginable feature and legacy capability.