Issue: find a new home for repos including Requests | Hacker News Reader

Issue: find a new home for repos including Requests

github.com

162 pointsmaximilianroos82 comments

Good, I hope pipenv is also included.

Would prevent comments such as "I like the current implementation", "thanks for the feedback, but my ego says no" and "I have no idea" in some of the most important projects of the Python Community.

Or passive-aggressive closure of issues that are clearly affecting people and are issues.

https://github.com/pypa/pipenv/issues/966#issuecomment-34620...

Don't get me wrong, I appreciate requests especially, but a maintainer should be humble also and respect other people independent of how rockstar he is or feels.

I am very happy to see Organisations/people such as pocoo - Armin Ronacher and Sentry - David Cramer etc. are already stepping up to make sure these libs land in the right place and continue to be useful to the community.

Complementary article: https://vorpus.org/blog/why-im-not-collaborating-with-kennet...

While you have a point about behavior, let’s not forget that the library is _free_. Maintaining such projects is often a hard, thankless job.

The only obligation any maintainer has is to abide by whatever terms they release their code under. I don’t think it’s reasonable to expect professional support from any project maintained by a single individual.

I don't think that's the only obligation. The other major obligation of a maintainer is to not hurt or impede others in the progress.

Hurting others (which, to be clear, I'm not claiming has happened here) is pretty easy to do. For instance, someone opens a request, you say "Patches welcome," they spend time writing a long implementation, and you say "Never mind, I don't like this feature." You're wasting their time, and had you just not said anything, they would have been in a better place. Writing software with backdoors, or recklessly writing software in a way that makes it prone to vulnerabilities, also hurts people.

Impeding others is a softer form of this. There isn't the ability for five hundred Python dependency management projects to all exist and be successful. If you get your project listed as the standard recommendation https://packaging.python.org/guides/tool-recommendations/ , you are effectively promising that it is suitable for people who need a tool like it, and in turn that you have both the willingness and time to make it happen. You don't have to get it so listed. (You can also, of course, change your mind and de-list it.) You can absolutely say, this is a project for my own needs and I don't want to accept support requests at all. Then, at least, someone else can create a similar tool - either their own, or based on your open-source code (and abiding by your license). But if you're signed up to be the standard tool, you're impeding others who actually do have the time and bandwidth for improvements and would be willing to make a competing tool if you're not able to stand by your commitments.

Excellent points I had not considered.

If as you say, somebody released code saying that it was something they were willing to support, it’s reasonable to expect said author to stand by their statements.

That said, this seems like it would be a more implicit expectation than an explicit agreement in many cases. How could a community enforce good behavior in this regard?

Going by your previous example, would it be reasonable for the recommended list of libraries to be more carefully curated for people on such a way that only stable _and_ well supported libraries are put forward? Putting hard requirements on official recommendations seems like it’s be a fairly simple way to avoid such situations.

> The only obligation any maintainer has is to abide by whatever terms they release their code under.

That's not actually true, right? The obligation flows the other direction. People who receive the code are obligated to abide by the terms. The author does not have to do anything except follow the law. They are not bound by the terms under which they are distributing their code.

I worded that poorly. I meant “terms” in the general sense, as some sort of agreement between two parties, but you are absolutely correct.

True, I agree with how hard it is, but if users say a feature that is an integral part of the tool is broken, that should be priority or if you don't have time/interest just give it to people that can take care of it. I am not talking about a company asking for a new feature, I am asking for a bug that makes your popular PSF-recommended tool do unexpected stuff. (see the https://github.com/pypa/pipenv/issues/1554 as an example)

Earlier response to the vorpus criticism (presumably):

https://www.kennethreitz.org/essays/requests3-update

I have to be honest here, we as users of his libraries are not entitled to anything. Yes, there are people who do a great job at passing on the baton, but it is unfortunate to see folks who take the work of open source contributors for granted.

I guess this is good thing, but I can't help but think that there would have been better way to do this; the process seems pretty rapid haphazard first-come first-serve situation. Most importantly, PSF indicated interest for these in under 2 hours (which is pretty quick response) but there are already some repos that are being transferred to other people. Surely he could have waited at least one day for responses before actually starting to move things?

It's about as haphazard as anything else he does. See "To those that criticize the project..." on https://journal.kennethreitz.org/entry/r-python .

Oh I didnt knew he made pipenv AND requests - both are very fine projects trusted in production.

On Pipenv? Why are people so negative about it? If you don't like it - just use another tool. Its not like its M$ or systemD which both are a pain to get rid of if your organisation or distro decided to implement

Because whenever I use another tool, someone asks, "Why aren't you using pipenv? It's a very fine project."

> systemD

Honest question: Do people write systemd with incorrect capitalization out of spite, or because they just don't know better? It just irks me, like if someone was writing LaTeX as just Latex.

A lot of the complaints about Pipenv were that both its own docs and some of the material on the Python package index suggested that pipenv was a new “official” tool recommended by the Python core team, which isn’t the case.

Didn't it really suggest the recommendation was from the core team, and not PyPA?

Because PSF suggests it as go-to tool for env management

PyPA, not the PSF.

I have created a community for exactly this purpose:

https://www.codeshelter.co/

I'm not saying he should have used it, but there are options, rather than just posting "hey, who wants these crucial Python packages?"

It seems odd, especially some of his use of words, I hope he is ok.

To anyone reading this, who is also struggling as a free software maintainer: I'm sympathetic to your position, and I have a guide here on how to responsibly step back from your projects at the pace you need, be it today or next week or next year.

https://drewdevault.com/2018/12/04/How-to-abandon-a-FLOSS-pr...

If you need specific advice or a sympathetic ear, please feel free to reach out: sir@cmpwn.com. I know free software maintainership can be hard.

It's too difficult to write large/important free software projects alone. I've also written about my approach to building a healthy community of engaged contributors and co-maintainers to make sure your own investment & workload remains healthy, along with that of your collaborators:

https://drewdevault.com/2018/06/01/How-I-maintain-FOSS-proje...

This one is much more subjective, YMMV.

This is why I fear our dependence on 'authoritative versions' of libraries. We've already seen the perils of ownership transfers being used to embed vulnerabilities or outright malware, all under the pretense that the new versions are still authoritative and trusted.

A more mature approach seems to me to be to ensure forks are first-class, and that dependence is restricted to a specific fork: either by full name (@owner/repository), or - better yet - by Git coordinate (url/sha). This means an abandoned project can be picked up by any number of people.

The primary cost I can see is one of cohesion and divergent enhancement under the same library name. But in a way, we have ways to refer to that already. "Mordenkainen's Magic Missile" - the owner + name coordinate already has a natural representation. And perhaps once we've made that step, more progress can be made to help forks coordinate common changes and improvements where possible, or to split into more distinct artifacts where applicable.

Another view: It is an issue of libraries owned by individual developers. I commonly argue for organisations of some kind to take over ownership over "relevant" libraries. This of course is nothing which can be enforced.

Benefits are not only reduction of "bus factor" on a single developer, but also enables review procedures.

Depending on the project this could be a custom organisation, a "generic" organisation like ASF.

Libraries owned by a single developer for me are a notable risk, when taking them as dependency.

could you elaborate on how that is supposed to work? how would a dependence on a fork help with the maintainer transfer risk?

Let us say the current mechanism - handing a library from one owner to another - is used.

If the new owner acts in bad faith, what recourse do you have to maintain a non-abused version of the library? Can you replace the owner's authoritative builds with your own?

If you instead have no 'authoritative version' of a library, you lose some elegance to be sure; you can no longer say simply, 'let's use Magic Missile!' as there may be multiple forks of what could be labeled the original 'Magic Missile' library. But what you gain is the ability to maintain multiple versions of the library, and to be able to point out which versions are poorly maintained or even hazardous:

"We don't use Drizzt's Magic Missile, it was tailored pretty heavily towards his own projects."

"If I recall, Persophone's Magic Missile included a hidden cryptominer. Avoid at all costs."

so this is not a technical change but a change of presentation and perception. instead of one person or team having the authority, there is a community notion of who can be trusted.

this does exist. and it seems to happen when authority falls down or is not there to begin with. for example several community builds around distributions for the GPD pocket emerged, and some lived longer than others and some where more reliable than others.

in fact the emergence of linux distributions themselves follows this model. each distribution is packaging applications they did not create, and are hence not the owners of the authoritative builds. and we pick distributions based on which one does the best job.

of course distributions create their own authority on their collection, but the principle remains. they also may easily switch the source for their packages. i do remember a few examples where this has happened.

another area where this can be seen is with binary builds for projects where the original developers only release the source. it's the same dynamic. build your own, or pick a community contributed build by someone else. either way noone gets to use the authoritative builds because they don't exist. (or they exist only for one platform, in which case the situation is limited to those who are on different platforms for which there are no authoritative builds)

It's both a matter of perception and of tooling.

Consider the Java ecosystem. Dependency targets are specified by coordinates of both group and artifact, which satisfies the above separation fairly well, to the best of my knowledge.

Consider the NPM and rubygems ecosystems: they specify dependencies by the name of the library alone, implicitly enforcing the 'authorized artifact' model.

so you are saying that in java this works because i can choose the group where i get the artifact from, just like i can chose a different package repo for a package for my linux distribution.

but in NPM and ruby i am limited to one single global repo and thus have no choice but to get the one that is controlled by whoever owns the name.

Similar, yes. You can also choose repositories in Java, as well, which is closer to your Linux package repo. So there's just another tier in the hierarchy, there, basically.

You can run your own private artifact repositories for NPM and Ruby, and select which concrete artifacts they contain; but that's not the publicly hosted repositories. One option could be to centralize into a cluster of public repositories, which is closer to what the Java and Linux ecosystems have, and that may be a sufficient compromise in many cases. I think the layer of abstraction between repositories and concrete artifacts is still useful.

If there was a KeepAlive system for paying maintainers then this sort of thing might be less likely to happen.

Not sponsorship, not donations, not support subscriptions - those concepts are not the same a simple payment for nothing more than Keeping a project funded and Alive.

KeepAlive payments in which companies pay a minimum $100month fee to KeepAlive projects that they depend on.

How many companies have a dependency on Kenneth's requests module? Alot. Hundreds if not thousands. And if there was a self interest obligation for companies to make monthly KeepAlive payments then those companies would not be now wondering what it means to have so many projects that depend on Requests now without the project author.

UPDATE: having read https://journal.kennethreitz.org/entry/r-python maybe Kenneth would be exiting anyway. But who knows, maybe if companies actually paid to Keep their dependencies Alive then people like Kenneth might remain engaged.

Why was the name of this post adjusted to remove mention of Kenneth? I think that the fact that this is Kenneth Reitz is a major element of this story.

We often remove names from titles, because HN works better when the focus is on content rather than personalities.

https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

A nice fix would be if github.com/username could appear to the right of the title instead of just github.com. That's on our list, but not implemented yet.

But in this case, isn't a large percentage of the content predicated on the person generating the content?

For sure, but how much that's a good thing vs. a bad thing is not so clear. This is one of those cases (like https://news.ycombinator.com/item?id=20186280) where it's useful to have exactly one thing you're optimizing for. It helps you know what to do in situations that would otherwise be head-scratchers, and the consequences are often surprising yet rather easy to derive.

On HN, the one thing we're optimizing for is intellectual curiosity. That's merely one kind of curiosity and by no means the most powerful. Interest in personalities, especially celebrities or the infamous, tends to be fuelled by one of the higher-octane varieties—call it social curiosity. There's nothing wrong with social curiosity. It's human. But intellectual curiosities are smaller fish than that, and on HN we need to do things to prevent the bigger fish from eating them all up. Everything in our bag of moderation tricks is there to help with that.

I remember an email thread with you some years ago where I was complaining about someone/thing and you not only responded to the problem brilliantly but also smacked me deservedly for my part in letting the problem exist.

Your optimisation approach seems to me to be working out nicely.

Hopefully I've learned to be less smacky in the meantime!

Admirable values.

The new title (EDIT: "Issue: find a new home for repos") seems pretty generic to me, though. If I didn't open the story while it had the old title I would have probably missed it.

Edit note: At the time of writing, the article title did not include "Requests".

You're right, but from our perspective that's a feature not a bug. This story isn't lacking for attention. In cases of hot controversy, it's good to add a little resistance to the circuit. When a title doesn't immediately give away its meaning, it takes a bit more energy to 'overcome'. That energy needs to come from curiosity ("I wonder what that is?"), which is the circuitry we're hoping to activate here. Compare that feeling to "ooh, new development in hot topic X!" which is how one feels when a headline grabs attention and links it directly to something already charged in the mind. It may be a subtle difference, but these tiny initial conditions seem to have the largest effect on how a thread develops. This is related to the principle that on HN, it's good for users to have to work a little. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

But the content is specifically that repos by a notable maintainer are seeking new maintainers, not just random GitHub repos.

The most notable repo involved is "requests", an HTTP library more widely used than the one in Python's standard library (I'm under the impression it has become bundled with the Python standard install but am unable to find a citation for that).

How about changing the title too "Python requests library and other repos seeking new home"?

Ok, I've tacked Requests on above. That goes against my gut feeling about this, but let's try it and see.

for what it's worth, it made the title feel more deliberately chosen.

i'd generally expect titles to be close to the original title of the post. and the first version of the title just doesn't feel like a title that anyone would put onto a post, let alone an issue report in a tracker. with 'requests' added it feels more real and interesting.

Well, it was the title of the original article, and therefore the title that the site guidelines say we should use: "Please use the original title, unless it is misleading or linkbait." It was neither misleading or linkbait, which is why I don't feel good about having changed it. But that's fine.

https://news.ycombinator.com/newsguidelines.html

oh, i missed that. in that case i agree, the original should have been good enough. can't fix everything...

edit: i was thinking of this title "Issue: find a new home for repos" which is not the original one. somehow that "all" makes a difference, as does adding "request"

This story is meaningless without the person's name. Kenneth Reitz owns the #1 most-used Python package, `requests`.

Sure, but not every title needs to give away the meaning of a story. It can be frustrating when it doesn't because we're so conditioned to expect that—but interrupting such conditioning is part of the point of HN. We want something more than subsecond reflexes to be at work here.

If people encounter a title that doesn't fully yield to 300ms of parsing, that causes irritation, which is a good thing. Not good in itself—irritation is a bit painful—but good because it causes the rapid-reflexive subsystem to throw an error and pass control to the slower, reflective system that is able to process more information. That's the system we actually want to get activated. One's it's engaged there's a chance that it will remain engaged, which—this is the hypothesis anyway—will lead to higher-quality thoughts and comments. That's why I say it's good for HN users to have to work a little.

But it isn't giving away the "meaning" of the story - it's giving critical context that makes the story a story and not just some random "John Doe" on Github throwing away their personal projects to new maintainers.

I'd argue that if random John Doe is giving away their projects and that is on HN's front page, it's actually the better story. Then there's a bit of a puzzle: why? Why did people upvote this? What's interesting about it? And you have to work a little to find out. It excites curiosity, and if it doesn't, nothing will happen. That's a better starting point for HN than "new development in a hot story". Hotness and curiosity are at odds with one another.

i didn't recognize the name so it would not have made a difference for me, but having this posted on HN hinted that this is significant, and not some random little used projects.

I don't know how to distinguish that from

"You won't believe who's giving a way a repo

#2 will shock you"

Titles like that try to generate the same effect as hot-topicness by relying on gimmicky language. They activate the same reflexive responses as I described above—they just do it by form rather than content. That's even worse of course. At least with a genuine hot topic there's a feeling of satisfaction when you find out what the latest development is. It's like watching the next episode of a popular series. But titles that have only bait and no content are like feeling that anticipation, only to be tricked into watching another commercial. That works for getting views, but everyone hates it and has an icky feeling afterwards, and over time it builds up a strong residue of resentment.

That kind of title operates even less by intellectual curiosity than the kind I was talking above, so it's even worse for HN. They are the kind we edit and everyone agrees with the edit.

Without any mention of the person or specific repos, I for a split second thought that this article was about Github shutting down and all repos needing a new home lol

Ok, I've taken the word "all" out of the title above.

The author of event-stream on npm got so much flak for handing out the package when he couldn't maintain it anymore, yet I see no mention of any concerns here. Aren't the same risks of malicious takeover present, or is the python module environment somehow safer?

This is actually a much more responsible way to go about it than the event-stream issue, since it is being done in a public forum (Github issues). Event-stream was given away over an Email without an announcement [0].

Of course, the risks are still there for malicious takeover, but I think will be minimized due to the owner's notoriety and the amount of press this has already seen just on GitHub. I am pretty sure the maintainers of Pip will be keeping an eye on this over the next few months in case of intervening.

[0] https://github.com/dominictarr/event-stream/issues/116#issue...

If I recall, the flak wasn't around handing off the package. The flack was for the lack of due diligence in picking a new maintainer. The same risks are present for sure but at least the process is taking place in the open.

Yeah, it was this really. The author had largely abandoned the project, someone emailed him asking for it (who had no github presence whatsoever), and the author obliged.

Here, there's in theory the effort that the people being given the project have been more vetted and have to have some amount of github presence to get a repo.

Why can't requests be adopted by python?

It looks like it likely will be brought under the PSF umbrella.

I'm not sure what has prompted this, but Kenneth won't leave them to wither and die. He understands their importance and if he's handing them off, he'll make sure they are in good hands.

After the smear campaign and mass roasting he received less than 2 months ago, it's not hard to understand why any charitable intentions he had may have completely dried up

Context? I follow a lot of Python news, but never saw anything about this.

There was a bunch of hubbub around a preemptive promotion of the `pipenv` package to be _the_ package for managing dependencies in python.

There was also a period when development and releases was happening in a somewhat frenzied manner and some github issues were being handled / closed in an antagonistic fashion. I don't know how extensive these issues were, but I remember seeing a few examples that seemed to back up the claims.

At risk of potentially stepping into overly personal territory: As I understand it, Kenneth was unfortunately dealing with some mental health stuff during part of that time period. He posted a blog post that mentioned this.

Here's one related HN discussion: https://news.ycombinator.com/item?id=18612590

My bias: During my usage of pipenv, while it seemed very promising, I felt that it wasn't fully production ready and had some problematic warts.

> There was a bunch of hubbub around a preemptive promotion of the `pipenv` package to be _the_ package for managing dependencies in python.

AFAIK that never happened, and was only marketing from his side, and never an official stance from the PSF (or whoever would be the authoritative source). That's just one the many questionable things that has happened with him in the past.

I think there was language added to the front of some official python website (one of pypa's I think?) that gave that impression. It was later changed.

Yeah, it's true that the marketing around pypenv was definitely over the top.

My guess is that requests 3 is in development

> Therefore, if you or your organization are open to the idea of monetary exchange for ownership of a repo / pypi project, please email me

Am I somehow misinterpreting that, or was that really a blatant attempt to sell the repos and pypi ownership of some extremely popular Python libraries?

So it seems. Would that be a problem for some reason?

"Buy this and you will be able to put any code you want into any project that uses Requests and doesn't carefully audit the changes in new versions" is potentially a pretty huge problem, yes.

Similar issues have already happened with people buying or otherwise taking over popular NPM packages.

I mean this isn't that weird. I'm sure a lot of companies would like to buy 'control' of the project and move it in the direction that the original maintainers didn't necessarily want to. Plus being the company/org that maintains requests is pretty good PR.

There is absolutely an exchange of value here, no reason to assume that a change in maintainership means cash for malware.

IMO people should have a right to sell things they've made. I get that that is inconvenient to open source folks. And most of the time it will be fine. But, IMO, if this is a concern, someone should make it worth it to this owner not to sell it to a bad organization. It is just not good enough to say, "You can't sell this thing because it is inconvenient to me, someone with whom you have no contract and to whom you owe no duties."

What about the other maintainers of the code?

If you are the sole person that contributed code, that's fine, but what about codebases that had lots of code contributed by other people?

What’s being transferred here isn’t ownership of all the code, but control of the project and the IP associated with it.

> "Buy this and you will be able to put any code you want into any project that uses Requests and doesn't carefully audit the changes in new versions" is potentially a pretty huge problem, yes.

Would not taking money somehow obviate the issue?

This immediately follows:

> I am considering this an unlikely and rare situation. Only serious offers will be considered, and money won't influence maintainer-ship decisions.

> Only serious offers will be considered, and money won't influence maintainer-ship decisions

what will this mean for an async version of requests?

On a side note, there has been work on an async HTTP client in a separate project conducted by the Encode organization here: https://github.com/encode/http3

Someone more responsible can actually make it happen.

kennethreitz has not been responsible? `requests` is my favorite http library in all languages so far.

He has shown patterns of bad behaviors. One recent example of the top of my head :

- https://vorpus.org/blog/why-im-not-collaborating-with-kennet...

- and HN's thread https://news.ycombinator.com/item?id=19826680

Request has not been maintained by Reitz for some time now (meaning he hasn't contributed).

Take a look into pipenv issues and how he handles legitimate broken features such as --keep-outdated and you will know[1].

Requests I agree has been very stable.

Also as an extra reading regarding async support for requests take a look at this article by a Python core Developer(Nathaniel J. Smith):

"Why I'm not collaborating with Kenneth Reitz" https://vorpus.org/blog/why-im-not-collaborating-with-kennet...

[1] https://github.com/pypa/pipenv/issues/1554

why does genius coder like him still look for a job? can python foundation have some permanent fellowship for people like him?

Reply on news.ycombinator.com