A Letter to /r/python
journal.kennethreitz.org
journal.kennethreitz.org
Also the pipenv maintainers were very rude about closing bugs. This has lead to a lot of reasonable criticism about the tool and how it’s maintained.
Despite marketing the tool heavily, Reitz is taking all of this personally and is blaming others or his mental issues.
Now, Reitz has posted this 3 times on /r/python in the past 2 days, and hosted a hangout while also saying he doesn’t have time for criticism.
And he and his PyPA buddies are all posting in all the threads and making passive-aggressive tweets.
To top all this off Reitz has now made and mods /r/positivepython in an attempt to imply the original is somehow not “positive”. And in my opinion to exert more power over the python community.
Before all this I really respected him, but this childish out burst will really make me think about how well design or endorsed his work is in the future.
The linked-to essay says: "There is a misnomer that Pipenv is not useful for libraries. That is false. Try using $ pipenv install -e . and see what it does when you lock. It augments setup.py. I, as a library developer, find that extremely useful."
I can't help but conclude that the documentation doesn't make that clear because the author doesn't think it's true.
I have just now read https://docs.pipenv.org/advanced/#pipfile-vs-setuppy on the topic and am still confused by the terminology. Eg, "Applications are different in that they usually are not depended on by other projects."
I am used to writing command-line applications which are used as dependencies for other projects.
But that is neither here nor there - no need to answer. Thanks for pointing out the nuance I did not recognize.
Pretty much nails it for me. Being up to date with new technologies and tools is important, and once it became officially endorsed I (and many others) had to run with it.
It was about 3 months I tried it last time, but at that time it was still slow, it ignored ~/.pip/pip.conf and also had no "proxy" CLI option like pip does.
I hope it have changed now but early on it also had some strange preference to environment variables rather than CLI options, which I find odd for an application aimed for command line.
I voiced my concerns about these issues, because why not? If I as a developer release something and intend to continue maintain it, I have a responsibility towards the users that depend on my creation. Old-fashioned stance perhaps...
https://www.reddit.com/r/Python/comments/8kjv8x/a_letter_to_...
> Pipenv — the officially recommended Python packaging tool from Python.org.
https://web.archive.org/web/20180228194710/https://docs.pipe...
If it isn't official, they did a very poor job of pointing that out.
The most relevant comment I found was by /u/bixmix:
> It's getting much worse - not better. The internal churn within pip also is not community driven as much as it is developer driven. It's like we didn't learn from the debacle of python3 vs 2.
I'm not a Pythonist so I had no idea there was such a huge debate over how to manage deps. At various points the past few years I've googled "bundler for python" and come up with different projects. I remember trying pipenv early on (didn't work with conda Python install). Then I revisited it months later--the project was much more developed--and it worked great. FWIW I like pipenv, it solves my largest gripe with Python development.
If you have as part of your community philosophy "there should be one—and preferably only one—obvious way to do it," it seems like you're going get a lot of stubborn tug-of-war situations (see 2 vs 3) where each side thinks their way is the superior way and there is less room to compromise for the sake of keeping the community together. That said, this pattern of debate is pretty universal across communities.
Ruby--even with it's MINASWAN motto--had it's own RVM vs rebenv drama back in 2011. Same sort of developer/community vs developer/community drama. Nowadays I think rbenv has a larger following.
http://www.rubyinside.com/rbenv-a-simple-new-ruby-version-ma...
To contrast this drama, look at what the NPM team had to say about Facebook's yarn: https://blog.npmjs.org/post/151660845210/hello-yarn
Maybe there is some inside baseball going on here when it comes to python foundation etc, but as an outsider looking in the vitriol seems totally unnecessary. I wouldn’t develop Python software w/o Pipenv.
We'll probably never know many projects have withered on the vine because of similar issues. Kudos to him for his open letter and willingness to deal with the negativity in a meaningful way. (It always kills me when people attack others for providing free fucking software.)
[0] - https://packaging.python.org/tutorials/managing-dependencies...
The community prides itself on being 'diverse' yet it's always the same names, over and over again. That's how you know who's in the club.
But on the other hand you could just do your own thing, write your own software and tools and ignore it.
I notice it also all the time.
I'm more upset about the whole pyproject.toml thing that means I have to migrate everything again, from Pipfile this time, even though I have barely finished migrating to it!
The long version of why it exists:
Historically, a Python package has been defined by the existence of the setup.py file. Executing this file with various command-line arguments is how packaging tasks are carried out: 'setup.py install' installs the package, 'setup.py sdist' builds a source distribution (.tar.gz) suitable for uploading to a package repository, etc., etc.
But this workflow has dependencies! Most packages will only depend on setuptools (which supplies the setup() function) and wheel (which supports building binary packages), so the packaging tooling assumes the presence of setuptools and wheel and you never have to specify them as explicit dependencies (in the old old days, setuptools used a drive-by installer file bundled into the package so people who didn't have setuptools could still install setuptools-based packages; nowadays, pip supplies it automatically).
But what if you have a workflow that requires something else? What if you use an alternative package manager, or package build tool, or really anything that isn't setuptools or wheel? How do you specify the toolchain you need to work with your package?
That is the thing pyproject.toml solves. Since it's not an executable Python file, it can't get stuck in a chicken-and-egg problem ("execute setup.py to find out what you need to execute setup.py"), and it can specify the necessary build system for a package in a standardized way.
pyproject.toml is not the place where you declare that dependency. This is why I am pushing back on you, because your comments all very very very strongly imply that you think pyproject.toml is a place where you would declare that dependency.
pyproject.toml is where you'd declare that you're using an alternate build/packaging toolchain. You would then use whatever mechanism that toolchain provides for specifying your dependencies (in the case of the default setuptools build/packaging toolchain: install_requires in setup.py).
https://github.com/sdispater/poetry/blob/master/pyproject.to...
But you've caught on to the key thing, which is that neither they, nor the default setuptools-based toolchain, have to do that. The pyproject.toml file exists to solve the chicken-and-egg problem of specifying that you use an alternative toolchain, when figuring that out might otherwise be dependent on your preferred toolchain already being present and configured.
The reason for the clichéd build you up, tear you down, repetition process throughout all of human history, is that a vast number of people will be rage monsters, petty, filled with envy and jealousy. The types that are prone to that ill behavior, will come pouring out of the woodword once you've acquired any consequential amount of success/money/fame/whatnot. These are not good people, they are not capable of reasoned discourse, their opinions do not matter, their hostile negativity should be ignored and disregarded accordingly (they're also, fortunately, extraordinarily easy to spot and separate out from the more reasoned criticism; their entirely unnecessary over-the-top negative behavior provides for easy filtering).
I mean, I have no idea if he's right, wrong, or sideways here, but what is clear is that he's released open source software and people are hating on him for it. It's free! You don't have to use it, do you? What gives anyone the right to throw hate towards an open source author?
Keep doing what you have got to do to make it through the day!
This will make you as happy as reading the New York times every 30 minutes to find out what Trump is up to, or being on Facebook or Instagram and making yourself envious of other people's lives.
Literally NOTHING matters in Python other than things on a yearly cycle. Don't worry be happy, and just unsubscribe to all things Python that happen at the Micro level and your appreciation of the language will increase.
Rather it’s bad because it’s yet another set of spec files and environment tools to learn, potentially further fracturing the community and taking attention away from other approaches.
To be fair, it’s not really a problem with pipenv, but rather with any “official” community endorsement of pipenv. Saying it’s a third party, optional tool and you’re free to have at it if it’s good for your project is fine. But the minute there’s any degree of officialness to it, it just creates yet another fissure in the anarchy of Python packaging.
My reaction is more malaise. Guess I’ll have to learn about this new packaging gadget too, to be current and signal how with-it I am. But in any case I have a choice, I’ll still just use conda and Docker for application & environment management, supplemented by pip and virtualenv when appropriate.
From my perspective, I’ve tended to prefer Docker for this. This allows me to version control environment components like environment variables in the form of the Dockerfile. I usually have a Makefile for my project which then uses multistage builds or other techniques to automatically detect when environment.yml (or package changes, like to setup.py) take place, requiring a rebuild of the container, so that downstream things like running tests, dropping into an interactive debugger in a full environment (with the right python environment, environment variables, etc.), all just become simple Make directives.
It doesn’t cover everything. Notably, for situations when there are many target environments, like testing a library I intend to distribute cross-platform, across Python 2/3, etc., I might still use a tool like appveyor rather than suffering the multiplicity of a bunch of Dockerfiles and Make commands to target multiple environments.
But for most applications I develop with an intention of targeting a small set of production or dev environments, it turns out to be better IMO to separate the Python environment & package definition entirely from the larger runtime environment (ENV vars, etc.), and use Docker for that part, with Make as the bridge.
Combined with making a library choose sensible defaults for ENV vars, copiously documenting them, and providing examples, this has worked well enough that I do not need or want to mix the external environment configuration with the app’s config files or package spec, in a “separation of concerns” sense.
Your mileage may vary, but this pattern has really worked well for me, and makes me skeptical of solutions that approach combining environment configuration with packaging as a first-class feature, rather than as a special case.
I guess more and more developers don't remember when all the coolest shit WASN'T free. You had to pay for compilers, editors, manuals, etc.
If someone writes a piece of software that you invest into your pipeline, understand that without some sort of contractual agreement, that person isn't OBLIGATED to maintain that code, much less give a shit about your feedback.
Is Ken smug?
Yes.
Is this post full of insufferable little whiny points?
Yes.
But is Ken going to have to do what the python community wants wrt pipenv?
Hell no.
Granted, a lot of this conflict could've been done differently and with better tact, but the gritty fact of the situation is that no matter what happens, it's not his fault for giving away the work. Even if the whole world depended on a module, it's still a voluntary action.
> My response to this comment is — first off, my fame, while certainly categorized under “cult of personality” is not necessarily accidental. It’s called marketing. I worked very hard at becoming well known within the Python community, and toiled away at it for years.
after quoting
>> If only these people didn't advertise themselves as the next best thing to sliced bread, collecting undeserved fame, jobs, influence, and as consequence harming very large number of people and technology in general, I wouldn't mind them doing whatever projects they like.
(w/o attribution, unfortunately), which shows that, if pipenv is a sub-par but popular solution (I did not ever use it), this latter quote may be a completely valid criticism.
I am diverting from my point: if Ken wants and encourages the python community to use pipenv, then he might have to obey that community, and he better does. But if he's releasing pipenv just to give back to the community and share his solution to his problems with others, then you're right. After all, if the participants have no regards for each other, that's a bunch of selfish people, not a community.
Reitz is (was?) a developer at Heroku and pushed it as the solution for paying customers.
It was OSS before that.
That's their problem, tbh.
When it came to Python he most certainly did “run the joint”
Heroku is not even a part of the equation of whether or not he's responsible for oss that he authored. No matter where he worked or any of that.
It's not his problem that people are using his software and have committed themselves to have expectations over something they have no right to try and control.
I've been a Microsoft developer my entire career of 20+ years and I remember starting out in the mid to late 90s after graduating from college and all of the tools you needed to learn you had to pay for. But because of both open source tools and the web, the need to develop "Windows Software" has declined precipitously.
Microsoft has had to make a lot of their tooling free, open source and cross platform (Visual Studio Code and .Net Core) and even thier proprietary stuff has had to get much cheaper for small developers. Thier online hosted version of TFS - Visual Studio Team Services - interops with everything.
I don't see myself leaving .Net for back end development. But, I don't have to pay the Windows tax anymore in terms of money or hardware resource requirements.
---------------------------------------
whatever dialog you're trying to start around
a blog post where you're talking to /r/python,
by sharing it here on ycombinator News,
is BETTER DONE by a text post on /r/python.
---------------------------------------
In fact someone (you? don't care) already shared your blog to r/python instead of posting your entry as a text post. so why are you sharing it here? r/Python is already having dialog on r/Python based on the blog post where a guy talks to r/Python.
this is really bad Rediquette, dude. not upvoting your transparent attempt at generating SEO points
The post is self-aggrandising and he has the audacity to think he knows what is best for everybody. There is also the real chance this tool will replace other tried and true tools and workflows with a more complicated one. It smacks of systemd vs upstart vs sysv
I'm not a huge fan, but how is he disrupting the workflows and toolchains? He wrote a tool, if you want to use it, that's cool, if not, that's cool too. It's not like he came to your house and forced you to use Pipenv instead of what you were already using.
If you want something to be mad at, it's the larger issue: We just started moving to Pipfile for dependencies and now we're switching again to some new project.toml file? The tool we use over this file is pretty much irrelevant, since all that matters is this file, and you can use any one of a myriad of tools to manage it and install your dependencies with.
If you don't like Pipenv, use something else that can install stuff from a Pipfile, but why are we making yet another standard to replace all previous ones (xkcd here)?
Written with literally no knowledge of this before:
This killed it for me.
So, you're saying you don't have enough career experience that this is surprising to you?
Megacorps are outsourcing their core processing systems to 3rd level subcontractors, and these are the systems that literally run the economy itself, and have been doing so for nearly 2 decades now if not longer.
Is this a just or fair situation? no (in my view) But hardly new or surprising.. and certainly not relevant to the point you're trying to make.