It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language.
I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.
Its important to remember that most of the python ecosystem, isn't written in python. The functions are often thin wrappers/objects around the real computation, which is often written in a faster language, C/C++/Fortran.
Julia excels in composability, performance, ease of development. You don't need to recode your algorithms in another language due to the performance of Julia, as is needed in Python's case.
Generally speaking, I see Julia's biggest fault, time to first plot, being addressed far sooner than python being redesigned to have Julia's capabilities. For the record, I use python daily in the day job. And I use Julia there for analytics there, often with jupyter notebooks. Easy for users to consume and interact with.
I barfed at 1-based indexing for about a week, but now it is as natural as anything.
I would compare 0-based and 1-based indexing with whether you put semicolons at the end of each line or not. Either way doesn't really change the feel (semantics) of the language.
Also, fortan is 1-based, iirc, and a lot of numerical code is in fortan.
Oh, and many many beginning programmers and scientists have a hard time with 0-based indexing. Not sure why, but such you hear, so the choice is really not that odd.
It makes sense in certain context (and in languages like C that have a low-level mental model). For scientific computing at a higher level of abstraction where the mental model of a multidimensional array is a tensor, and not a memory offset location, zero-based indices really get in the way
The sensibility of the index choice is equal to the starting value of the index.
People should stop wasting time bikeshedding this insignificant detail, but for some reason it is to programmers like a red rag to a bull.
When you have to deal with a wide range of languages, stuff like this is small potatoes, compared to, say, indentation based structure. The latter can result in the completely non-obvious change of program flow due to a single errant space or tab.
Also have a look at https://github.com/giordano/StarWarsArrays.jl
I worry about someones ability to solve real problems in any language if they can't get their head around an +1/-1 when indexing into an array.
This. Think about what this signals to employers and interviewers if someone throws a hissyfit over this.
So this "very big elephant" is, in reality, a nothingburger.
For me, the very big elephant in the room is the semantic formatting. It has and continues to trip me up time and again. A touch of a space bar can change the flow of a python program. That is a huge asteroid of a problem.
For me I think the packaging ecosystem is bad, we need one package management tool like poetry built in. We need a built in typing system like typescript. Lastly we need to remove the GIL.
I’m pretty sure all of these are currently being addressed by the community.
I switch languages a lot and things like functools, itertools, dunder methods, list comprehensions, dict comprehensions are things I sorely miss especially in typescript. In particular list and dict comprehensions when used with care are a great deal easier to work with and reason about when transforming data.
I like to think that containers only exist because deploying a Python application is so %^#(&*# complicated that the easiest way to do is to deploy an entire runtime image. It's an absolute nightmare and travesty. So bad. So very very bad. https://xkcd.com/1987/
I'm not optimistic on TypeScript for Python. That'd be great if such a thing existed! I'm not optimistic on packaging or deployment. There is recent progress on GIL removal which is exciting! There is hope, but I'm proceeding with very cautious optimism.
Comprehensions are kinda great, but also hideous and backwards. Rust iterators are a significant improvement imho. The fact that no recent languages have chosen to copy Python's syntax for comprehensions is telling!
Oh, and I think the standard library API design is pretty poor. Filesystem has caused me immense pain and suffering. Runtime errors are the worst.
MyPy exists, Python officially supports type annotations.
I do think comprehensions are a weird feature for Python, particularly from the “one way to do it” perspective. And also because the language so strongly discourages FP patterns outside of comprehensions.
Overall I would say the language is a lot better than the ecosystem, and that it suffers a lot from having a bad packaging design. I’m not a fan, suffice it to say. It’s best if you can stick to the standard library.
Do you mean os.path, pathlib.Path, or something else? I use pathlib.Path all the time and never get filesystem errors anymore, it's really practical.
Not even close to "only thing". Dynamic language is a net loss of productivity once you reach a certain level of scale. Refactoring a codebase with millions of lines of code in a dynamic language is an absolute nightmare.
Opinions vary at what level of scale this happens. My personal experience is that once you hit just a few thousand lines of code that dynamic typing is a drag on productivity.
There's a reason things like TypeScript are increasingly popular.
I don't care what language first year CS students use. I care what languages I'm forced to deal with on a day-to-day basis!
nearly every dynamic language has trended towards this in the last 7-8 years. it's probably had the most profound effect in modern Javascript (typescript) and python. It's incredible how different production javascript and production python are compared to say, 2014 javascript or 2014 python.
JET.jl is a interesting project to do some error analysis through abstract interpretation: https://aviatesk.github.io/JET.jl/dev/jetanalysis/
more so than the goals of JET itself, the abstract interpretation framework it exposes should allow for TypeScript-like gradual typing.
This is important, imho, if Julia is ever to go out of a niche area because most of the rest of the world has already moved on to see the benefit that the improvement in application reliability that static analysis allows.
Python is popular because of the ML revolution. If ML didn't take off neither would Python's popularity. Is ML successful because of Python or despite Python? Well, the world is probably further along with Python than if it merely didn't exist. But if a different language that sucked less existed we would, imho, be further along than we are.
I'm not annoyed Python exists. I'm annoyed that its inertia is so monumental it's inhibiting further progress. We're at a local maximum and the cost to take the next step is really really really high. That's not Python's fault mind you, just the way things work.
Python was popular before because it's very nice language. People wanted to use it for science to, so they wrote very good scientific libraries for it.
R was very popular for non-neural-network ML some years ago, yet it wasn't picked up for NN, because R kind of sucks for general programming. As the joke goes, the best part of R is that is a language written by statisticians. The worse part of R is that is a language written by statisticians.
Python was growing at accelerated speed year on year well before neural networks.
Python was popular, including for scientific use, before the ML revolution; in fact, the only reason it is associated with the ML revolution in the first place is it's preexisting broad adoption in scientific computing.
Python and PHP are so big because of the languages themselves and their implementation, Js is a bit different in that regard I’ll admit.
Is Python better than assembly and C/C++ for ML? Absolutely. Is Python good? I don't think so. Other people might use that term, I do not. I think Python is a bad language that would be designed very differently it was built today. And we're stuck with its decisions because the inertia to change them is monumental.
It's not really the language's fault. As an industry we've learned a lot in the past 30 years! It would be a travesty if we hadn't made language design process! Unfortunately we haven't figured out how to effectively apply lessons to encumbered ecosystems. Case in point: the transition from Python 2 to Python 3 was an absolute disaster. And it only made tiny, incremental improvements!
I was recently writing code using Reactor/RxJava in Java 11 w/ Lombok. I don't think I've ever been so productive or lead a team as productive as when we were going ham on the functional/reactive Java. Now that I'm back in Python land, I am constantly frustrate on a daily basis with both the language and the runtime at every turn. Even with the asyncio we are working on, it feels like the absolute minimum viable product when compared to the java, node, or rust I have done.
There are some fantastic python enhancements that bridge some of the gaps like PEP 484 (Type Hints) and PEP 517 (holy crap an actual build systems that are not dogcrap) but it feels like the python community does not care.
I wrote a somewhat tongue-in-cheek rant blog post. https://www.forrestthewoods.com/blog/things-i-like-about-pyt...
I deny that hiring Python is hard beyond “hiring is hard”.
> You think you're getting a good programmer, because they know all the leetcode tricks,
Unless I want someone for a role that is very much like reproducing leetcode tricks, I don't think I would think someone is good for it because they are good at those. In fact, leetcode is mercilessly mocked as being almost completely irrelevant as a positive signal for hiring, though it may be useful as a filtering tool to reduce an overwhelming volume of applicants to a manageable one where high rates of both false negatives and false positives, but some slight advantage over just randomly discarding applicants, is tolerable.
That's...a complete topic switch and irrelevant. The discussion I was responding to is about the challenges facing whoever does have control.
Outside of that I interviewed several of my friends (I know them from a non-programming context, so I don't know their competency) who were predominantly python devs, and completely noped out of them for the same reasons (and these were my friends).
All I'm saying is that signal to noise for the common tests you give for 'problem solving aptitude and conceptual fundamentals', is much lower when you are hiring for a python position. You think you're hiring for those things, but you're actually hiring for leetcode-optimizers.
I mean, I'm not trying to do hire like that, and I think I have an interview that tries to test that effectively, but I have had to deal with the downstream effects of people who are doing hiring like this, and that has been a real problem for me.
If you instead hire for "engineering" positions, without caring about what languages the candidate knows, you can interview for their ability to solve practical programming exercises [1] in whatever language they are most familiar/comfortable with. Maybe this only works at FAANG-level hiring, but in these contexts, top tier candidates can get things done in any language, and that's really what matters, no? But more to your point, I've generally found candidates that pick Python (or Ruby/Perl/etc) can actually accomplish more–and therefore prove their capabilities–in the space of an interview simply because they're picking a more expressive language. Bad candidates will prove they are bad candidates no matter what language they choose.
1: Eg, reading/manipulating files, merging/filtering/sorting data, generating and processing multi-dimensional data structures, etc.
Python is one of the few languages that has a balance of ease of use, ecosystem, ubiquity, and useable type system. It's a fantastic glue language and it's extremely flexible.