The Python community has done an amazing job making a suite of scientific software. Numerical libraries and plotting are excellent in Python. Sympy and Sage are both very ambitious, and in my opinion, quite successful projects. So what’s the problem?
Longevity and stability. Make fun of it all you want, but old crusty FORTRAN code, and in the past 20 years C and C++, are the bedrock of serious scientific applications. Python usually isn’t. I’m not confident enough to say why with certainty, but maybe it has to do with the fact that Python APIs and type signatures (whether written out or not) are too handwavy and easy to change without detection.
In addition to this, I rarely find that people, scientists especially, are writing applications in Python. Python is too hard to deploy as an application sane people can use, is a mess to configure, and difficult to make efficient if your slow parts aren’t rewritten. Most scientists slop code around in Jupyter notebooks which bitrot and are forgotten about in 6 months.
The world is clearly in need of good numerical code and environments, even with Python’s amazing ecosystem. Julia has been popular among my savvier programming colleagues and it has attracted very high quality contributions, rivaling and exceeding Python in some cases.
Back to me: what do I do? I write my numerical applications in Common Lisp. It’s probably as unpopular of a choice as it gets in the scientific world. I use Lisp because:
- The language is standardized and my code won’t break when the compiler updates
- I can write very high performance floating point code, using SIMD/AVX, and readily inspect the assembly code at the function level
- I can easily bind to C and Fortran libraries
- I can deploy applications as static executables
- I can avoid garbage collection and heap allocations with careful programming
- I can write generic, extensible interfaces
- I can debug algorithms live—in the depths of stack traces—without the compile-printf-edit cycle, which is insanely useful when encountering tricky floating point accuracy bugs.
Clearly I’m a “programmer’s programmer” though, which may be a hint as to why Lisp remains unsuitable for scientist use. Most scientists don’t know or don’t care what a stack allocation is, and that’s fine, because the tools have evolved around them in such a way they can be hugely successful without that knowledge. It’s also remiss to not mention that scientists typically have 8+ years of training in their (non-software) discipline, and “ain’t nobody got time for [programming],” despite its growing importance and necessity.
The biggest negative of using Lisp is that a lot of work you’d take for granted in Python or Julia is not done. It took me several hours to get a certain complex 2^n dimensional matrix factorization routine working from LAPACK in Lisp, whereas in Python it would take me 30 seconds flat to look it up, read the documentation, import it, and call it. So it’s definitely the case that Lisp’s “startup overhead” is high. But at the end of the day, if I’m building an electromagnetic simulator, I feel some pre-baked math routines are the least of my concerns, and the overall code structure and quality is of the highest concern. Python is suboptimal in that category, in my opinion.
One good thing about college courses like these is that they (slowly) displace numerical programming courses that are MATLAB tutorials in disguise.