Hacker News Reader

For a language that has "There should be one-- and preferably only one --obvious way to do it." as part of its core philosophy, this is very ironic.

Was thinking the same thing. When python was getting started it was up against Perl, which toon the opposite approach and there was a myriad of ways to do something in the language.

Package managers for languages were just starting to become a thing back then. (At least I think that is true...). Now days you wouldn’t dream of releasing a new language without some kind of standard package manager...

Dunno where I’m going with this comment but it is definitely interesting to see how much has changed since then. Hell even Microsoft has a package manager for it’s languages...

> Now days you wouldn’t dream of releasing a new language without some kind of standard package manager...

Isn't this one of the major criticisms of Go?

Yeah, and the exact same thing is happening.

My feeling is that a good package manager is the exception to the rule. Particularly around managing versions of the language runtime and the versions of packages available within projects using that runtime.

Rust is the exception here in that Cargo is a very nice piece of kit.

In terms of generic runtime version managers, ASDF is very nice cross language version manager for scoping versions to projects and switching between them easily.

> Rust is the exception here in that Cargo is a very nice piece of kit

Also Rustup, which is pretty key to Cargo remaining flexible going forward. There's a huge advantage to having the build/dependency management piece being highly coupled with the language runtime/compiler. Making cargo work with all versions of Rust would mean having to build in backwards compability and make it much more difficult to do projects like the new modules changes.

It's a key insight that there needs to be two tools surrounding a language, not just one. You need good dependency management and good version management. Node also has this right with nvm and npm/yarn.

Well, it's not like Go if feature packed in other areas people ask it to either...

That philosophy is pure and well intentioned. It will always fight with reality.

It's aspirational, not prescriptive.

Just consider urllib, urllib2, urllib3, and requests. Or getopt, optparse, argparse, and docopt. Or countless others.

I appreciate that the Python community continually strives for ever-more obvious and ergonomic ways of accomplishing tasks. While there is an ecosystem complexity cost to the solutions-that-weren't, it's no greater than the churn than I've seen in other popular languages.

The Zen of Python even pokes fun at the "one obvious way" by writing its three em-dashes three different ways in the same document.

>> The Zen of Python even pokes fun at the "one obvious way" by writing its three em-dashes three different ways in the same document.

I did not know this! Thanks :)

Pipenv solves a very real problem, but it has a lot of problems of its own:

- It's very slow: re-locking after updating 1 dependency often takes me ~1 minute.

- It has lots of bugs. To name a few in 11.6.9: clobbers comments in the Pipfile, inconsistently includes dependencies for other build environments in Pipfile.lock, stores the wrong index in Pipfile.lock for packages not on PyPI.

- They release multiple times per day, often breaking things in patch releases.

- Kenneth Reitz is quite unpleasant to deal with in GitHub issues, which I often have to because of the previous 2 issues.

From what I've heard, Pipenv "has been officially blessed" only insofar as its maintainer got commit access to PyPA's documentation and inserted a recommendation.

If what you've heard is true, it could be pretty toxic. Do you have any evidences to back up?

The recommendation was added by Jon Wayne Parrott:

https://github.com/pypa/python-packaging-user-guide/pull/443

He doesn't seem to be a pipenv maintainer:

https://github.com/pypa/pipenv/commits?author=jonparrott

I apologize for not meeting perfect standards of mood at all times. I have bipolar disorder, for what it's worth, and working on a packaging tool is very stress-inducing.

It’s not easy to develop for large communities, and even tougher to speak frankly about oneself.

Keep up the good work.

Also:

> From what I've heard, Pipenv "has been officially blessed" only insofar as its maintainer got commit access to PyPA's documentation and inserted a recommendation.

This is blatantly false.

Sharing your awesome Python program is its Achilles heel. I have 5 computers I work on a weekly basis. I stopped using Python and Haskell due to all of this.

Racket's exe are so simple and quick to share.

    raco exe foo.rkt

https://docs.racket-lang.org/raco/exe.html

(Edited tendon to heel)

Yes, packaging Python applications for distribution is yet another nightmare situation.

Back in the day, we used one of the various "freeze" applications (cxfreeze, bbfreeze, probably others) and some tools developed in-house to package and distribute them for various platforms.

It has been a while since I actually needed to do this, so I don't know what the current state of the art is.

I've thought about starting a "build an ecosystem for Racket" project where I document gaps in the extended ecosystem and review best-of-breed solutions from other languages as a kind of literature review / review of prior art.

Racket has such a great story for so many things that are painful in other environments. It's an under-appreciated language.

Would you be willing to collaborate on this? Racket was my first programming language and I miss it dearly. An excuse to write code in it would be great!

Oh my yes. Of course. PM me, and let’s figure out the right vehicle to get started.

The only issues that this really could address. Would be nice to have a "Wall of Shame" for these gaps.

A) Racket is so enjoyable everyone builds their own solutions

B) "The view that full-fledged problem solving almost always calls for language design" Which sadly falls onto deaf ears especially people from Python.

> Python, in its noble quest for backwards compatibility, has accumulated so many different ways of packaging, distributing and installing libraries and apps that it's rivaling Google's chat apps.

I'm not sure it's because of backwards compatibility. Perl has amazing backwards compatibility, but doesn't seem to suffer from this.

That's not said to pump up Perl or degrade Python, but because if you misidentify the problem, your proposes solution has a much lower likelihood of fixing it.

(Then again, maybe you mean something different by "backwards compatibility" than what I thought you meant)

As a Python dev it looks like I need to do some reading as I am still using virtualenv (it usually works fine but I have do encounter the occasional issue).

Can anyone give me a rundown of the benefits of other systems over virtualenv?

On a side note I hope Python isn't going to turn into a shambles like JavaScript - its certainly starting to look like it as far as package managers go.

JS isn't in shambles; in fact, their package manager situation is excellent: Use yarn, period. It could be better (yarn isn't default yet), but I would love for the Python ecosystem to get to: "Use pipenv and setup.cfg, period".

But as I said, it's too young. If you use pipenv, you still have to interact with pip, so that's two binaries whose commands you have to remember. And urgh, it's so slow. And it's kinda useless for libraries... pipenv doesn't publish stuff, pipfiles aren't appropriate for packages.

But we'll get there. One day. Hopefully it won't be too late.

So what are the benefits over virtualenv? You have given me the downside of another binary to manage. What do I get in return?

I've not given you anything other than a warning that it's too young.

Pipenv is promising. That means at some point I'm fairly certain it'll be and deserve to be the standard in Python packaging. In the mean time, it doesn't.

That still doesn't answer the question of what it will (eventually) give me over virtualenv.

> Pipenv [...] has been officially blessed as "The One True Way" of installing stuff by the Python documentation.

Oh? I don't see any mention of pipenv in the cpython git repo.

https://packaging.python.org/tutorials/managing-dependencies...

Nor in the Python Packaging Authority documentation!

Pipenv is mentioned in the PYPA Python Packaging User Guide tool recommendations:

> Application dependency management

> Use Pipenv to manage library dependencies when developing Python applications. See Managing Application Dependencies for more details on using pipenv.

https://packaging.python.org/guides/tool-recommendations/

You're quite right. I skimmed Installing Packages, which mentions it in passing is a tool for managing multiple venvs, and Packaging & Distributing Projects, which doesn't mention it at all.

> To those suggesting it, Docker is great but you're still dealing with a package manager inside Docker, so that's a moot point.

Package manager/dependency manager is must for any sort of mature mass production coding environment (e.g. cargo for Rust, Maven for Java etc.) Docker doesn't solve all that but at least you are doing it only once and replicate easily and predictably. And a lot of times you can reuse Docker images built by someone else.

If the "built by someone else" docker container contains a bunch of incomprehensible hacks or black magic, then you're really just handing yourself a nightmare down the road when it comes time to update that thing or apply security fixes.

Please forgive me for venting my frustration here, but my reaction to this is basically: oh great, yet another blinking tool to add to the list of misery that is the Python packaging experience. May as well check it out now before I have to do it in a hurry...

First of all it's not in Debian unstable. Ok, maybe it's just super new... so I'll install it with pip. A python3 -m pip install pipenv later and let's try it out!

$ cd ~/src/nexsan-exporter/nexsan-exporter $ pipenv install Creating a virtualenv for this project… Using /usr/bin/python3.6m (3.6.5) to create virtualenv… ⠋Running virtualenv with interpreter /usr/bin/python3.6m Using base prefix '/usr' New python executable in /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python3.6m Also creating executable in /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python Installing setuptools, pip, wheel...done.

Virtualenv location: /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG Installing dependencies from Pipfile.lock (ca72e7)… 0/0 — 00:00:00 To activate this project's virtualenv, run the following: $ pipenv shell

Cute emoji and nice colours but... ~/.local/share? For a directory that will end up containing arch-specific libraries? Ok, I guess no one gives a shit about this in the modern world, oh well. Wait... "python3.6m"? That isn't the Python interpreter I asked for... but it seems to be a hardlink to the same file as /usr/bin/python3.6 so I guess maybe this is intentional? Anyway let's check out the venv...

$ ls -l ~/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python3* lrwxrwxrwx 1 yrro yrro 10 Apr 30 17:06 /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python3 -> python3.6m lrwxrwxrwx 1 yrro yrro 10 Apr 30 17:06 /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python3.6 -> python3.6m -rwxr-xr-x 1 yrro yrro 4576440 Apr 30 17:06 /home/yrro/.local/share/virtualenvs/nexsan-exporter-Eq5p1XVG/bin/python3.6m

So this is the Python folk's what, fourteenth attempt to get this right, and they are still copying the python executable into the virtual environment instead of symlinking it in? This seems to be a regression from venv, which seemed to get this right!

Right, time to install my dependencies... according to 'pipenv' the right command for this is 'pipenv install -e .' Hm, I wonder exactly what the -e option does?

$ pipenv install --help

* no menition of -e in the output *

:sadface:

Oh well, let's just run it blind!

$ pipenv install -e . Installing -e .… ⠏ Error: An error occurred while installing -e .! Directory '.' is not installable. File 'setup.py' not found.

Maybe I screwed up and ran this from the wrong directory?

$ ls setup.py setup.py

Weird, what's going on here?

$ strace -f pipenv install -e . 2>&1 | grep setup\\.py stat("/home/yrro/src/nexsan-exporter/nexsan-exporter/setup.py", {st_mode=S_IFREG|0644, st_size=1769, ...}) = 0 stat("/home/yrro/src/nexsan-exporter/setup.py", 0x7ffc013a0200) = -1 ENOENT (No such file or directory) [pid 7365] stat("./setup.py", 0x7fff84176e30) = -1 ENOENT (No such file or directory) stat("/home/yrro/src/nexsan-exporter/setup.py", 0x7ffc013a0300) = -1 ENOENT (No such file or directory) write(2, "Directory '.' is not installable"..., 62Directory '.' is not installable. File 'setup.py' not found.

Oh FFS, I give up. I think I'll let pipenv pass me by for now.

You're doing God's work, man. Thank you for helping me avoid that headache. I'm sticking with pip

I wouldn't go that far. I would absolutely love for some tool, and pipenv as as fine as any other, to replace everything else. But right now it doesn't seem to be ready.

It's absolutely not ready, as I made clear in several comments :) I just said pipenv is the most promising one.

I agree it's frustrating. I'm planning to donate some of my time to the project to help the situation. I hope others can too.

Well I filed these as bugs this morning and got mixed feedback; the 'install -e .' but was caused by a stale Pipefile in the parent directory that I did not create, but at least removing it makes things work. Unfortunately upstream does not consider a stale copy of the python interpreter copied to dozens of directories on the filesystem that will never be patched to be an issue, so from my PoV pipenv gets a big raspberry and I'll stick with manually running 'python -m venv' which has gotten this right since Python 3.3!

What best practices do I teach to complete beginners to Python?

I am teaching adult beginners and it would be best to avoid the messy parts in the beginning.

I thought I could get by with just pip and briefly touching on virtualenv.

I sin myself by avoiding all the mess by cloning a full bare bones Lubuntu virtualization whenever I need a clean project.

Theoretically Docker should suffice but it is yet another layer of complexity.

There was Zope Buildout for years, that had declarative packaging format, used the cheese shop (biggest issue with conda) and isn't slow as molasses but it was and still is largely ignored by the larger community.

I just started using pipenv for a small personal project, and I like it. The usage is similar to git which I prefer over the weird bash activate mangling that a normal virtualenv does.