Hacker News Reader

Comment by avgcorrection | Hacker News Reader

Why does it make you wince?

From a non-python dev perspective: I always struggle with dependencies and versions. I have a script in front of me that I want to run, and am often just frustratingly brute-forcing commands to make it work.

Do I:

python? python3? pip install? pip3 install? python pip install? python pip3 install? python3 pip install? python3 pip3 install?

And then everyone mentions "oh just use venv" or "conda" or docker or... It just never ends or it never clearly explains what to do. And all I want to do is run a relatively simple script. Or course I would use a docker image to run a more complex app. But it's frustratingly hard to manage for people who don't use it often.

I'm from a java/Kotlin world where things aren't perfect either. But at least dependencies are clearly defined, and the only thing you need is Gradle that updates itself through a wrapper script if needed and pulls all dependencies. And a JVM/JDK which is also configured/installed quite clearly in a specific location.

Yeah, it's definitely painful. Over the years, I've probably spent in excess of 100 hours on understanding pip, venv, PYTHONPATH, poetry/pdm, pipfile, pyenv, ... still barely understand how it all works together.

https://peps.python.org/pep-0723/ might make life much simpler for single-script applications.

for simple scripts, this is the easiest and works.

  python3 -m venv venv
  source venv/bin/activate
  pip install whatever
  echo "source path/to/venv/bin/activate && python3 path/to/app.py" > myscript
  mv myscript /usr/local/bin

and just use myscript. I use this for a lot of little scripts

Why not just

Path/to/venv/bin/python path/to/app.py?

Neat trick though, gonna adopt it!

I haven't tried this myself, but you could probably save yourself the wrapper script and execute your script directly by adding a shebang pointing to the Python executable from your virtual environment.

printf '%s\n%s\n' "#!path/to/venv/bin/python3" "$(cat path/to/app.py)" > /usr/local/bin/myscript

I feel you. python itself has its warts but its mostly minor. The dependancy hell though.. god damn it confuses me every time. I don't understand why they can't get it together with one consistent dependency system with good ergonomics. ruby does it with gem. elixir handles things reasonably well with mix. rust does it REALLY well with cargo. hell even ocaml has opam.

but python? it between conda, pip, pip3, virtualenv and who knows what else. its hard to make something that I knwo will just work on everything.

To highlight another dynamically typed, interpreted language ecosystem, npm has been providing a remarkably smooth and uniformly adopted service for the JavaScript community.

Rye[1] is an all in one manager for python projects. Including the python versions and virtualenv, pip etc etc... It separates tool deps from app deps. Its all configured through a pyproject.toml config file.

Its still new but works well. I'm transitioning to it from an unholy mess of pyenv, pip installs and other manual hacks.

If you're starting a new python project that is more than just a straightforward script I'd use Rye from the get go.

[1]https://rye-up.com/

Part of the problem is that this used to require third-party tools, which gave rise to lots of different tools, but nowadays everything you need is included in Python itself.

The simplest way (in the sense of having the fewest components required) is this, using only built-in tools:

  $ python3 -m venv --upgrade-deps my-virtual-environment
  $ my-virtual-environment/bin/pip install whatever-third-party-package
  $ my-virtual-environment/bin/python3 my-script.py

This creates an isolated virtual environment in "my-virtual-environment". Use the pip and python3 binaries inside that directory to install packages and run scripts. Done. (The "activate" step suggested in other comments is just a convenience for setting your PATH to refer to the virtual environment implicitly, so that's completely optional.)

Note that Debian's python3 package leaves out some parts of the standard Python distribution, but you can install the python3-full package for a complete installation.

This is a fairly good answer but on windows, there’s no python3, so that’s your first command being broken.

Right, on Windows the recipe needs a few alterations but is essentially the same. This should work:

  > py -3 -m venv --upgrade-deps my-virtual-environment
  > my-virtual-environment\Scripts\pip install whatever-third-party-package
  > my-virtual-environment\Scripts\python my-script.py

This assumes that the Python launcher (py.exe) was installed when installing Python, which it is by default.

Py is not installed when you get it from windows store. Another gotcha to worry about.

The story about python on windows is a nightmare for new people for these reasons. And installing packages - while not as bad anymore - is a mixed bag due to fewer precompiled C extensions than on linux.

Paradoxically, microsoft have spent some effort on improving things. This has led to multiple incompatible choices which, effectively, has made the experience worse.

Sure. Add "Get the actual official Python from python.org, not whatever mess Microsoft has cooked up" as step 0.

If you do use Microsoft's Python distribution though, are you saying that it provides neither "python3" nor "py"? How do you use it then?

According to the docs[0] the Microsoft Store package should provide "python" and "python3" commands (effectively unsolving the problem solved by the Python launcher, I guess?) so either my original suggestion of "python3" or my subsequent suggestion of "py -3" should work, depending on which distribution was installed.

[0] https://docs.python.org/3.11/using/windows.html#the-microsof...

See this is the problem. You're already several "ah yes and you need to"s deep.

Not really. It's the same 3-step process in both cases.

You literally gave a different process on Windows to Linux.

It's the same process described with different syntax for different operating systems. It contains no "ah yes and you need to" and is not "deep". Here's a templated version of the same process, if you prefer, which works on both Windows and Linux:

1. [PATH TO YOUR PYTHON COMMAND IN WHATEVER PYTHON DISTRIBUTION YOU YOURSELF INSTALLED ON YOUR OS OF CHOICE] -m venv --upgrade-deps my-virtual-environment

2. [PATH TO PIP COMMAND INSIDE THE DIRECTORY CREATED BY THE FIRST COMMAND IN THE SYNTAX OF YOUR OS OF CHOICE] install whatever-third-party-package

3. [PATH TO PYTHON COMMAND INSIDE THE DIRECTORY CREATED BY THE FIRST COMMAND IN THE SYNTAX OF YOUR OS OF CHOICE] my-script.py

The fact that some parts of the command are the same doesn't excuse the other parts being pointlessly different. It's a paper cut for beginners. One of hundreds, which makes the whole Python UX completely awful.

Which languages do this much better? Honest question.

Rust, Go, Javascript/Typescript, Java, C#, Dart, ... I mean it's basically all of them except C/C++. And OCaml. OCaml is the worst.

Rust, Go and Deno are definitely part of a new breed of "developer tooling should just bloody work!" languages. Python is from the era of "eh, you figure it out" mixed with a little of "I'm making this up as I go along".

True, the last three build tooling as a part of the language from the start.

Java? I seem to recall build systems and tooling being more pain than Fortran. Not sure I believe that one. Certainly not my experience.

Thankfully, most distros/OSs don't ship with Python 2 anymore, so `python` will just link to `python3`.

Nope. Debian doesn't ship a `python` at all, which I think is the sensible option.

You should universally use `python3`. Except on Windows the official distribution is bonkers and has no `python3.exe`. Probably the easiest solution is to install it from the Microsoft store instead which does.

You seriously consider Gradle - a kitchen sink and a monstrosity scripted in obscure, extremely dynamic language - a better solution just because it has a... wrapper script?!

With Gradle, where do you define dependencies? All with version numbers in the `dependencies` block? In a separate file, declaring a hash map of versions and then a hash map of dependencies, and then putting the latter into `ext`, which magically propagates to just about everywhere in the build silently? In a separate file using some plugin that tries to replicate the lockfile functionality baked into most other dependency systems? In a separate TOML file, using the Catalog?

...and that's just about dependencies, I could go on for days about other Gradle features...

It's not any better than Python. You're just used to it. The same complexity is there in both Python and Kotlin, you just learned to cope with one better than the other.

(Gradle with Kotlin DSL and good IDE support is a little better, mostly because Kotlin devs are not as afraid of touching the build as when it's done in Groovy. It's still 2 orders of magnitude more complex than whatever you use for Python - unless you use zc.buildout or something like that.)

Personally, I wince at Python because, while I view it as a quick-and-dirty scripting language, some people write production code in Python. And here we are ten years later, stuck with code that reads a few metrics from the system, packages them into JSON, and sends them to an API for monitoring. The code takes 7 seconds at 100% CPU to resolve the imports, every time it runs.

You can write bad code in any language, I fail to see how that's Python's problem.

Installing python software is hard. Golang is much easier for and end user.

Go is what I reach for when I might otherwise write Python but need to actually, like, distribute my program.

Reply on news.ycombinator.com