There are too many places to start a counter argument. It could start with what MLMs actually are, it could be that many businesses don't care about the talks and use scipy packages, it could be the cause and effect confusion of scipy packages existing and academic users using them--which relates to the Ruby part.
But seriously, I hope I'm wrong, but your post comes off as a bit paranoid and crazy.
Much of the success of Python these days is related to non-CompSci users. And no offense, but many of them are focused entirely on results which have very little concern for code quality, maintainability, or other software engineering concerns.
So now we have startups galore who want to be "AI" or "data science", and their PHBs gravitate to Python because Python is the machine learning language. Or so they've heard.
Ironically, and perhaps one thing the parent poster was alluding to, most AI/ML/data science companies actually end up doing very little of those and instead just doing normal software development. Maybe they have a small sprinkle of data analysis.
For better or worse (and honestly, probably often for better), people writing scientific Python care about solving a particular numerical problem. Code quality often matters less than just getting the right answer. Knowledge of a language or code quality is rarely the hard or important part.
Scientists generally spend time learning math and software engineers spend time learning good design practices. It shouldn’t be a great surprise what each group becomes good at. Time is finite.
I was around in the 1990s when Python started making waves in the scientific computing and research community. Python wasn't installed by default (we mostly used IRIX back then).
Python was excellent for "steering" low-level computations, to use the buzzword at the time.
That is, if you had a large body of existing C, Fortran, whatever code, you could turn them into Python extensions, let Python glue the components together, and use a Python program/script to organize the high-level data- and control-flow.
Swig was a popular tool to help automate that process. Swig 1.0 had support for Tcl, Perl, Guile and Python, back in 1996.
I used Swig to interface to an existing cheminformatics toolkit, and published my experience at https://www.drdobbs.com/cpp/making-c-extensions-more-pythoni... . Python made it very easy to convert the implicit object model of the underlying C toolkit into an explicit OO model in Python. Python's reference-counted garbage collection was also a good fit to the C library.
Together these helped simplify scientific research work quite a bit.
And that's without NumPy.
Even now I do a lot of non-numerical scientific software development, and rarely have need for NumPy. While I often use Python to steer components written in C or C++.
That is true.
The problem is, that code in Python of questionable quality (but real utility) is now turned into production code; and it is done so by building Python stuff around it. So now the insignificant implementation detail which was the language choice is now driving the entire project.
Also, a PhD data scientist/mathematician can get hired for a big salary at a company (especially fintech), and their code will drive critical business decisions. A regular software engineer is mandated to build a structure around that code to make it production worthy. In my experience there is no traction to say, "Great, you've learned how to answer this tough question; now let's implement your solution in a real language."
At my job I work both sides of the fence. Sometimes I write research code in Python/Julia/Matlab. Sometimes I write production code in C++/CUDA. The choice of production language is primarily due to performance needs rather than code quality, though.
I’ll repeat myself but slightly differently: the main reason Python is used over, say, Ruby is because Ruby lacks a comparatively good library for working with arrays. It subsequently didn’t develop all the ecosystem for ML and scientific computing (SciPy, Matplotlib, sklearn, skimage, Torch, etc). It’s basically never going to catch up. And when it comes down to it, Ruby is basically as ill-suited as Python, Clojure, Elixir, and most other languages when it comes to doing something like automatic differentiation on a GPU. And again, even so, Python is still the best choice among those because so many resources have been poured into PyTorch, Matplotlib, NumPy, etc.
My point is that their choice of Python (because of the libraries they use, or the training they got in school?) bleeds out and causes Python to be used in places where it doesn't fit so well.
If the libraries you mention were written for Ruby (and they absolutely could have been... probably with fewer defects and fewer lines of code), then we wouldn't be having this discussion.
There is no use of any of those libraries that someone could demonstrate which could not be equally or better demonstrated in one of the other languages you mention. Therefore, the only argument for using Python is that it's what those libraries are written in.
In fact, with a little effort an API could be written and wrapped around those libraries to make them available to other languages, decoupling the bigger projects from the library-specific logic. This is how it should be done at any reasonable company anyway. The data science wizards build their specific systems in whatever language they need, and that gets built into a minimal service. But things often start highly coupled, meaning the special code in Python gets business logic and persistence stuff built around it also in Python.
They use those libraries because there aren’t alternatives in the languages you prefer. Millions of man hours went into developing the Python ecosystem.
> If the libraries you mention were written for Ruby (and they absolutely could have been... probably with fewer defects and fewer lines of code), then we wouldn't be having this discussion.
This probably isn’t true because the core of NumPy is C. A theoretical NumRs would be the same. The issue with both languages is that they’re slow, so you have to resort to C wrappers. NumRs might be incrementally better at best. It wouldn’t be a revolution.
> There is no use of any of those libraries that someone could demonstrate which could not be equally or better demonstrated in one of the other languages you mention. Therefore, the only argument for using Python is that it's what those libraries are written in.
But no one wrote these libraries in Ruby. Someone had to do it, and no one did. Someone did it in Python. It would be impossible to justify recreating NumPy in Ruby today. If I want to do some simple image processing, Fourier transforms, simple optimization, and plotting in Python it’s trivial. If I want to do it in Ruby it is a ton of work.
> In fact, with a little effort an API could be written and wrapped around those libraries to make them available to other languages, decoupling the bigger projects from the library-specific logic. This is how it should be done at any reasonable company anyway. The data science wizards build their specific systems in whatever language they need, and that gets built into a minimal service. But things often start highly coupled, meaning the special code in Python gets business logic and persistence stuff built around it also in Python.
This isn’t a company scale problem. The Python numerical computing stack has developed over 20 years with enormous investment from companies and universities. Code could maybe be shared via an API, but someone would have to do it. It would have to be done very carefully to guarantee performance. If it were easy you’d expect to see significant code reuse between, for example, Ruby and Java web frameworks. I guess you could say this API does exist, and it’s called C FFI interfaces. Sane companies do use this, but it still sucks pretty bad.
Python’s lead isn’t unassailable, but any language that is just another C wrapper isn’t going to unseat it. Whatever does it will need to be compiled. I’d bet on it being dynamic with type inference, too, because static type checking almost never helps with correctness issues in numerical code while adding significant cognitive overhead. If it isn’t Julia, Julia is at least the blueprint.
I'm a scientist, and I code in Python, but my code is never directly incorporated into the production code base. (Similar story for the portions of my work involving electronics, mechanics, etc). A decent generalization is that once I solve a problem, and demonstrate a result using Python, the actual amount of code that needs to be understood and translated into the production language is usually a few tens of lines at most.
On the production side, the choice of language is just a tiny part of the entire process. They have frameworks, revision control systems, coding standards, review processes, the whole nine yards. Just using a "better" language isn't even an important step towards writing production code. If I wrote in the software department's favored language, they would still have to turn it into production code within their procedures.
The fact that a few lines of Python represents hundreds of lines of production code makes it more productive for me to think my way through problems and express the principles underlying the solution to the coders. I see no reason why the tool that's used for discovering a solution has to be the same tool as the one used for putting the solution into production. Laboratory chemists don't use industrial manufacturing processes when trying to make a milligram of a new compound.
But there seems to be an accepted rule that "if you're doing data science, you are a Python shop". And if you are trying to pitch a data science company to investors, they know Python for data science.
New companies start all the time using a technology because "that's what you use", without knowing why or why not. In fact, (and not that I idolize him, which I really do not), Paul Graham's http://www.paulgraham.com/avg.html essay talks about this. Most companies follow whatever is the accepted practice for doing things. It may "work", but it definitely may not be optimal. And the companies that choose a different path based on more consideration or more awareness may have an easier time.
I mean after all, there was a time when everyone wrote C (because C was so much better than COBOL or whatever). And believe it or not, there was a time when it was impossible to convince a company to try Java. Or as the old saying went, "nobody got fired for buying IBM". After all, it was the safe path. Substitute Oracle, or C, or C++, or Java, or Python, or PHP, or Windows (there was a time when people didn't trust Linux and couldn't imagine why you would ever try to run your business software on Linux machines).
Your reasoning makes sense, but in practice in the software world, it is not as well planned or thought out as you might expect.
The scientific computing and research community heavily used C/C++ for years/decades after the decline of Fortran.
Python came and solved their headaches by abstracting them away.
I agree with your points, but the only counterpoint I will make is that Python was created as a simple scripting language that could utilize lower language code. Basically it's a convenient wrapper for faster c code. So the fact that Python is populated with scientific libraries that are optimized in c is not purely luck. That said, like a lot of things in life, it's the combination of viability and luck that lead to success.
maybe people without the C background do not see that at all?
But I also take the Cython route when it's just one to a few functions that need optimization/parallelization... (Did you know you can instantly compile Cython in Jupyter to use with Python?). So I avoid writing anything else unless it's really necessary. (avoided* I haven't done that in a while)
But interestingly, Go is a very simple, straightforward language. It's got many of the same goals that Python had, and it meets those goals quite well. And it is blazingly fast. And it compiles so fast that it might as well be interpreted. And it is statically typed, something that is all the rage again these days and which Python pretends to offer but is really just noise and decoration.
I would go so far as to claim that anything built in Python can be built better in one of many other languages, even if you use Numpy or some other Python-specific language. And the way I would meet that challenge would be to isolate the use of the Python library and provide a minimal interface to that, and then call that Python program from another better language. Then I get the utility of the Python library from a language that scales better conceptually and performance-wise.
You may want to have a look at Gonum (https://www.gonum.org), and the Go HEP package developed by CERN (https://go-hep.org).
I was also surprised to see DSP and pretty sophisticated packages, although I never used them: https://awesome-go.com/science-and-data-analysis
And of course Go has Jupyter integration, it's almost like running a script thanks to its fast compilation time.
It seems like the people promoting Python for its ability to interact with C (to outsource time/memory constrained functions) are no familiar with the other uses of Python -- namely to build large systems in object oriented fashions.
A 10 or 100 or even 1000 line "script" in Python is just fine. Read it like a story, let it do the one job it needs to do, and go home.
But scale up to 10000 or 100000 lines, and you will certainly be forced to go OOP. After all, most Python modules use OO classes in cases where a module of functions would suffice... Now you are in the realm where Python is comparatively very poorly suited for the job.
It's a frog-in-the-pot boiling situation. It started out innocently enough with the frog choosing the nicest small body of water nearby, but now the frog is in pain and dying and unable to understand how they got into that situation.
There is absolutely nothing about Python which makes it especially suited to scientific programming or any other kind of programming. In fact, it is poorly designed for most software engineering goals, short of just simple one-file, few line scripts. But those could just as well have been written in many other languages.
Python just happened to be installed by default on most Linux systems, and it was more powerful than *sh. Just maybe one could argue that it was more accessible than perl because it had fewer $#!@$ symbols. If Python had not tried to offer OOP features (entirely built with duct tape, hotglue, and post-it notes) maybe Python projects wouldn't be such a mess. Simple procedural code would be better than the @decorator littered mess which is required to make Python have OO features approximating (but not fully supporting) other languages.
The amount of anger people have that things that, in fact, determine suitability for various applications are not the things which ought to do so in their own mental models of the world is... interesting.
This is a pointless conversation. You cannot know what you don't know. https://en.wiktionary.org/wiki/Blub_paradox
Yeah, sure. What makes you think that I haven't used a wide variety of programming languages covering every paradigm from unstructured imperative to procedural to functional (pure and impure) to OOP to relational to logic to...?
When I am amused at the anger people have about their idealized mental model of what should matter not being what actual does matter most to practical utility, I’m not speaking from ignorance of the different features of programming languages and the theoretical and practical benefits they bring. I am speaking from four decades of programming and understanding all that, as well as the human and social side of the activity.
Compared to Java and JS there is. The lack of operator overloading would make numpy matrix manipulations absolutely painful in those languages. It's just a chance that numpy wasn't built on top of ruby.
There are many versions of hype promoted for Python. Coal powered steam locomotives were great in their day, before there were better alternatives. But we don't irrationally hold the mentality that there is nothing better and why would you need anything else anyway?
Tell me any library which can only be built for Python?
Tell me any feature of Python which does not exist in one or more other languages?
Then I'll tell you at least some features which exist in several other languages but which do not in Python.
Python is inferior, and frankly anyone who is infatuated with it is ignorant of the other, better options.
Are you really making the "every programming language is Turing-computable" argument?
All your talk of features is missing an important point - Python is one of the few languages designed with insight from how to develop a programming language for non-programmers. In Python's case, experience drawn from ABC.
These are people for whom "public class java { public static void" is a barrier to entry.
For whom diagnosing segfaults is arcane knowledge.
And for whom hygienic macros sounds like a bathroom product.
I like Python because it's a language that my colleagues - most of whom are not software developers but who do some programming for their research - can productively use. Even with all the complexity of modern Python.
So no, there are zero features in Python which don't exist in other programming languages.
And it doesn't matter.
What better options do you suggest to a grad student in chemistry, biology, or physics?
We want a beginner language which has easy (and consistent) syntax, no requirement to specify datatypes (implicit), and can be interpreted (no compiling required).
In its most basic use, namely just writing procedural code with basic built-in data structures, Python is pretty ok. But it has some warts and gotchas, not the least of which is mutable default parameters in functions.
Now maybe the ultimate beginner doesn't use defaults for parameters, so it's not a problem (yet) for them. Same goes for the OO features. While the beginner may not write OO Python, using modules written by other people probably will require them to step into the Python version of OO. Now it gets messier.
Ruby is superior in the simple beginner case and the complex (OOP) case. It is superior and more consistent for functional cases as well, despite it not being designed as a functional language (it still has mutability risks all over the place as does Python, except for that default parameter razorblade).
The only beginner-unfriendly thing I can think of for Ruby is the optional parenthesis on function calls. I still think optional parens is a bad design choice, and I encourage people to always use them. It's usually a style choice (but not always, because sometimes it matters for how code is parsed), and I think it's better to know as a reader that foo.bar() is a function being called. foo.bar is unclear.
One small but significant feature Ruby provides is the ability to write more expressive function names with ? and !. A common pattern is to use foo? to indicate that the function will return a boolean. The alternative, without ?, is is_foo(). Likewise, ! usually (at least for standard libraries) indicates mutation.
Back to the point though, Python used simply in isolation is not terrible for a beginner. But inevitably outside modules will get used, and the misfeatures of Python will appear more frequently. And then a further eventuality is that this new programmer will now be a "python programmer", choosing Python for future bigger work because that's what they know. Why not start with a language that is also beginner friendly but also better for large, long-lived projects?
No, I am not.
As you must surely know, "simple" is not a well-defined concept. Forth is a simpler language than Python. Yet I am certainly not saying it's as good a language for beginners.
I'm saying that languages can be treated as a user interface, and improved based on feedback with the target audience. Python - unlike most other languages - incorporated insights from developing the ABC language as a teaching language for non-programmers.
Your "We want a beginner language" description is too broad as it also describes ABC, without including discussion of why Python succeeded where ABC did not.
> Ruby is superior in the simple beginner case
If you are basing things on your intuition and personal experience, then my intuition and experience says otherwise.
That's why these arguments are rather pointless.
> One small but significant feature Ruby provides
Yes, I get it - you like Ruby. But shrug some people like Fords while others prefer a Chevy.
And still my point is that Python explicitly incorporates design elements based on experience from teaching non-programmers to program. If it's part of Ruby, it's much less of an emphasis.
That's nothing at all to do with a language feature, which was your focus earlier.
That said, I think python is a very convenient wrapper for lower language utilizing scipy libraries. I also think it is great for quick prototyping. (It's been a couple years, but I even prototyped PySide Qt stuff in ipython notebooks, now Jupyter notebooks, back in the day).
Now, simply the popularity (perhaps created by luck), led to a huge ecosystem of scipy tools for the science community. That in-and-of-itself snowballed into a reason to use it. Could that have happened to a different language? Yes. but that doesn't invalidate that it DID happen to Python.
Perhaps your day job forces you to use the Python hammer for a screw problem; I'm sure that problem frustrates a lot of us working people, but I don't think it's inferior for the specific uses I outlined.
There's nothing wrong with using Python and its great libraries for some purposes. But none of those cases mean that Python is a good language. And frankly, the better the library is, the less Python language features you would use directly. The powerful libraries hide the failures of the parent language.
But as soon as you need to build something around your solution, unless you have another library which does the new thing you need (and encapsulates your solution neatly), then you will end up using real Python, on a larger scale. Then you will start disliking it.