My students are shocked (horrified?) to learn that they're basically running 50-yr old Fortran code when they use scipy.minimize to train their fancy little neural nets.
I tried out most of the scripting languages out there (Ruby, Perl, Tcl, Groovy, R, and many more) and Python just seemed to click more and it has a whole lot less to worry about upfront than languages like C# and Java. In comparison to languages like C and C++, it's a godsend for those with typical automation needs.
In my eyes it seems like a pretty straightforward development. There have been plenty of other tools that may have made sense throughout history too. Matlab could have done this, but by that time nobody was going to build out massive libraries for something expensive and partly closed off.
A lot of numerical optimization code is this - it conforms to a strict 'C' ABI - taking (arrays of) simple floats and ints and outputting the same, so binding it to another higher level language is trivial, so rewriting it makes little sense. If the same algorithm were written in Java, most people would not want to bring in Java as a dependency for their Python/C++/whatever project, but since this is just a tiny C object file, it's happily integrated into everything.
They also tend to be very tricky to get right, I remember reading a paper where the author was adamant that changing the order of a multiply and an add (a mathematically invariant operation) would cause the algorithm to blow up due to the different scales of floating point values involved causing a major loss of precision. I'm sure there's tons of stories like this.
This is the sort of code which took PhD's who studied this exact topic years to get right, even though the actual code often looks unassuming, I would dread the day when I was required to touch it (but I never do - since it always does what it says on the tin)
You are thinking of real numbers, for example as all Cauchy sequences with fractions for coefficients. That is not what a CPU does.
I’m not even fond of Python, but I use it sometimes because things exist for it to use and because there are lots of developers around who can share the work on the code. If I write something in R, APL, Julia, OCaml, Forth, Scheme, or Lisp at work I’m going to be the only maintainer and will probably get a stern lecture about that. If I use Perl, Ruby, Java, or PHP, there are a few more people but it’d better be code only my team has to maintain. Go, Rust, C, C++, TypeScript, maybe JavaScript, and Python are safe in most of the company’s codebases but only C, C++, or Rust for the code that needs the most performance and the most stability of resource use.
Moreover, to the extent that they care about the “best” (or at least “better” within the scope of options with suitable ecosystems) language, “best”/“better” is highly subjective and shaped very much by familiarity
Then the whole ecosystem that Python has nowadays was built on NumPy, and sending data from one library to another was trivial because of it.
That's it.
Funny how it’s getting its time in the sun now.
You're setting your students up for failure if this is how you are teaching neural networks to them. You should switch to pytorch or at least something like tensorflow or jax or you are actively doing intellectual disservice to them and leading them to be actively noncompetitive both in AI academic paper writing/grad school and in the job market.
Similarly, use of sklearn considered harmful in a world where CuML/CuPY/Nvidia RAPIDS exists.
And also, knowledge of conda/pip/poetry considered harmful in a world where UV exists.
Teach MODERN tools please.
In 2030, pytorch and uv will be so 2025.