I read somewhere on HackerNews that Python is kinda like a default language and rest such as Go, C, C++, Java are just for Optimization, once you figure out your solution.
IMO modern languages like Go and Java are pretty easy to get into and you can use them for a first implementation without really sacrificing development time relative to Python, as long as you have invested the time to learn those languages and the associated tools. (C++ is not like that unless you have made a very serious investment in setting it all up.)
I’m not trying to say that any of these languages are better/worse, just that they are differently suited for particular situations (program size, team experience, etc)
In mye experience Python programs are not more difficult to reason about than equivalent Java programs. To the contrary an over-reliance on certain design patterns and ubiquitous, inescapable OOP complicates Java code bases, while the static typing is so weak it affords little safety compared to e.g. Python.
Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing. (See the reference section here [1].) In other words you can get further with Python before passing the complexity threshold.
Bugs are also proportional to lines of code [2], which is another element that favours Python over more verbose languages like Java.
1. https://svese.dev/comparing-java-and-python-is-java-10x-more...
2. https://amartester.blogspot.com/2007/04/bugs-per-lines-of-co...
> Python is not maintanable above 50k to 100k lines of code and because of that people consider this code bases very large
Note that this is a speculation, not a conclusion, the article is not very thorough, and LOC is mostly used because it is convenient to measure, not because it is what we are trying to measure. I'd personally consider 50k a "medium" code base, and 10k is "small".
> Worth keeping in mind a that a Python program will be about half the LOC of a Java program doing the same thing.
LOC is a confounding variable.
> Bugs are also proportional to lines of code...
Not supported by the citation. The citation measures bugs per line of code, and finds that for 500 kloc of code, the average number of bugs will be somewhere between zero and 25,000. That's a very wide range.
To be clear, I'm not really trying to fight against Python or for Java here. I'm just giving my reasoning for why I might personally choose one or the other. I think that the idea that you would switch languages because Python is slow is actually far more situational. You might have ten reasons to choose Python or Java, runtime performance may only be one of those factors, it may not be heavily weighted, and in some cases, Python runtime performance can be extremely fast (I do a lot of NumPy stuff... it's great).
The relationship between programming language and code quality is, at best, a difficult relationship to study. It's hard to make any kind of direct statement like "using language X results in more bugs than using language Y" and back it up by evidence, even though we believe it to be true. Individual statistics which relate some variable to LOC is not useful in isolation.
See: https://arxiv.org/abs/1901.10220 (a reproduction of an earlier result, that invalidates many of the conclusions)
> while the static typing is so weak
Both of these are straight up false. Java can be written with static functions and single-depth inheritance trees just fine. Hell, the direction the language took for quite a few years now are pretty much this with records, pattern matching, etc.
And while Java is not Haskell, it has a moderately strong type system with quite a good generic implementation making even some more advanced functional patterns expressible.
Your claim on LOCs is also false (as well as most of this kind of claim) Java has at most a tiny bit of constant overhead, it won’t make it anywhere close to 2x.
The other issue with Python is that it uses indentation for scoping. Combine that with the fact that it is super easy to mess up indentation when you are moving code around via copy/paste and it is super easy to change the meaning (ie accidentally move a statement out of an if block).
Using indentation for scoping is great for small projects and for beginners. However, once you get to medium or large projects, having the extra redundancy of curly braces is reassuring.
Python is a great scripting language, and it's a great language for hackers that work on their own personal projects. It's not a great language for enterprise microservices on teams of like 20 people. The growing pains are real.
Not using a programming editor? I select the text and hit Tab or Shift+Tab, not exactly rocket surgery. The lack of redundant noise characters pays off every single day, where as odd indentation problems are a once-a-decade issue.
Copy/pasting large blocks, yeah, you can mess it up. But any editor worth using will let you select several lines of text and hit Tab or Shift-Tab to add/remove an indentation level, so fixing it only takes a couple seconds.
...or has equivalent functionality that doesn't require the mouse ;)
The ability to select text in this way has existed in Windows for at least 20 years.
Professionally, I'm at the whim of whatever environment my employer uses. I've used Linux (CentOS), Windows, and Macs. If I'm doing Python development, then it's pretty OS agnostic unless I'm using a Python module that compiles to a native binary, in which case Windows is certainly a nightmare.
I think you're spot on. Is some ways, it is a tooling problem, as type annotations require mypy (or some other type checker), and enforcement via e.g. CI.
Crucially, retrofitting them to a codebase is a difficult and tedious problem. On the other hand, it's relatively easy to do for new projects, but so is choosing a different language.
In my experience, this is because a lot of Python programmers are trying to write Java code in Python, carrying Java's awful and unreadable paradigms into a language that has no reason for them.
If you're doing data science things, it's hard to beat pandas/numpy. I get that those are popular in that community because the barrier to entry with Python is low. People who are just looking for a tool to solve immediate problems would do well to reach for Python.
The problems start when you try to write more complex things. Then you run into the weaknesses in tooling, testing, and performance. People would do well to skip Python and go right to another solution.
I say this having spent half of the last decade working on Python projects. If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
That's quite a statement. Go on, for most of the other languages, show me a web framework that's better than Django.
Python absolutely has deployment challenges, but the performance is Good Enough™ and the speed of modelling and maintaining and accessing databases in Django's ORM is so much better than anything else I've found. Its Admin interface too is superb for same-day turnarounds on little CRUD projects. Nothing comes close.
And I've written quite complex systems around Django. Ones that are multi-headed websites, APIs to physical hardware IO, ANPR, card readers, ID printers. Python hasn't let us down.
90% of the performance gains I've achieved are by optimizing/reducing large DB queries, or adding an index to the DB.
I think that’s subjective, so take this with a grain of salt. I do have a different opinion though. Also, my point isn’t to argue, but to encourage consideration.
I’ve used Django and Python quite extensively in the past and I would say that, for me, Phoenix/Elixir and Ruby on Rails are both better web frameworks than Django.
The reason I feel this way is that Phoenix/Elixir, for example, has much better dependency management and tooling than Python/Django (e.g., Mix is vastly superior to Pip, in my opinion).
Also, when I was using Django in the past, even the core team suggested using a different directory layout than is generated by default. This meant that every Django application I worked on had a completely different project layout. Both Phoenix and Rails are more opinionated regarding their project layouts. Some may consider this bad, but the benefit is I can go to any Phoenix or Rails project and instantly be productive because I know where all my controllers, models, views, templates, and contexts are going to be. I also like that Phoenix is much faster, more scalable, and makes better use of hardware resources than either Python or Ruby.
I think Python and Django are excellent, but I’d place them at #3 in my top-three list of web frameworks. That’s just my opinion though.
What I took issue with was the suggestion that "most other languages" have a better web framework. I'd wager most other languages don't even have a web framework. And even amongst those that do, there's some real dirt out there, and much worse examples of packaging, tooling, etc. Two Python frameworks, Django and FastAPI would be in my as-objective-as-possible top 10.
It was a silly thing to say.
The thing I want is a good monorepo solution for Python. There's at least 3 for JavaScript.
As much hate as JS gets, its package managers are also much better than Pip.
In order to manage complexity of Python as you do your big projects, you really need to do "enterprisey" patterns like DI. Even though it's rare for Python programmers to write like that.
Re: test runner: pytest is pretty good
Then if you need to do something like build a distributable archive for your app (say, to deploy it to lambda), you can use Bazel's dependency graph to get only the packages that the app transitively depends on.
-r package_a/requirements.txt
-r package_b/requirements.txt
Is bazel also a good choice for managing the virtual environments of my developers? Do you have any examples?
At my place of work we use some rules [2] to build virtual envs from Bazel dependencies. It's been a great stop-gap for us and allows us to use traditional Python tooling (e.g. PyCharm) with Bazel-managed dependences.
[1] https://github.com/aspect-build/rules_py [2] https://github.com/cedarai/rules_pyvenv
The built-in unittest module is essentially a clone of JUnit, so it can't be much worse, and most of the remaining limitations are fixed by pytest which people seem pretty happy with. What test frameworks are you comparing them to?
Is it really?
That's just like, your opinion, man.
> Tooling sucks.
Not in my experience. Lot of tools could be better, but I would not say there is a lack of un-sucky tooling at this point. Poetry and pytest in particular are largely excellent to work with.
> Dependency conflicts are common.
This has been a problem historically but it's leagues better today, especially with lockfiles.
> There's no test framework/runner worth a damn.
Um, what? Unittest (stdlib), Pytest, tox, nose, hypothesis, schemathesis, and a few other lesser known ones.
> Web frameworks are inferior to those in most other languages.
FastAPI is pretty amazing. Flask is well regarded as a good intro framework. Django I'm not a fan of personally, but many love it.
> If not for data science and academia keeping it alive, it'd be disappearing along with Perl.
And machine learning. And web dev. And sysadmin scripting.
Okay, so what solution? What do you recommend? I'm guessing if you're qualified to make statements like your first paragraph, you can bring your wealth of skill and experience to the table and show us what we should be using.
Unskilled developers can be very productive very quickly. Skilled developers can write some really beautiful code with it.
Has issues with the ecosystem - it's a headache with M1 and far too many Python projects are just broken. The latter is because it's used by scientists / students so is more a function of the community than the language.
I use it as a scripting language, which it excels in as long as you don't have many dependencies.
Go: fast and cgo is really powerful. The language itself is a little verbose for my tastes though - it makes the simple verbose. The package management story is weird (import the source code from github). Better than having the dependency hell of npm or Python though.
Python is extremely convenient when you are trying to be productive. It is a good general purpose tool, especially if you're doing anything web related. My project starts in seconds and requires no compilation. Stack traces and tooling for the language is far beyond most other popular languages IMO. The standard library is excellent, but also the quality of popular python libraries is unmatched by other languages in my experience.
Python is not super fast at doing some stuff. If you're doing that stuff all the time it may not be a good choice. But for most of what I do, python is really good enough. A user will not notice a few 10s of Ms for that web request, and usually that isn't even the scale anyway for this type of stuff.
It's not true for two reasons:
1. In practice writing a prototype in one language, then completely scrapping it and rewriting it in another language almost never happens for fairly obvious reasons.
2. The "slow bits" of a Python program are rarely neatly concentrated in a few parts of the program that you can rewrite in a fast language. The whole program is slow, so there's no small part that you can rewrite to make it all faster.
3. There are other languages that are just as easy to use as Python (easier even!) but are also fast enough that you don't need to rewrite anything in most cases. Typescript is probably the best option at the moment but there's also Dart, Lua and probably others I've forgotten.
Python is popular because it's seen as very beginner friendly so there's a mountain of content out there teaching people Python. Also the REPL makes it good for beginners and for science.
The very words such as bottleneck/hotspot indicate that it is common to get the most benefit by optimizing small parts of the code.
I had enough of the rewrite in C experience that since then, if the language doesn't come with a compiler on the box, either JIT or AOT, I only see it as scripting language for sysadmin work or learning purposes.
Also Vignette, remember it?
When we dediced to move away from TCL/C, after exhausting the performance options, J2EE was going to be the next version.
Due to a twist of fate of being Microsoft partners, we got invited to take part into .NET launch product line, and that Java rewrite became a .NET one instead.
The original founders, after going through these experiences, went out and created OutSystems based on the learnings, naturally in .NET and later a Java backend was added as well.
It's had its deficiencies as well, exacerbated due to lack of investment, but that is changing. These are being knocked out one by one, every release.