woodruffw

WDYM by madness? I know what the flag is (and what it does), but people have strong opinions in opposite directions about what Python packaging should do around system environments.

It’s widely used, and there are still a lot of workflows that we hear about from users where they want/need to manually twiddle environments rather than having the “declarative” layer do it for them.

This is our first breaking release since March. We’re happy to answer any questions folks have about it.

Interesting that there’s no primary evidence.

But your point about importing heavy material is mislaid: the blocks are intentionally heavy, because they (apocryphally) served as ballast.

You’re probably referring to the Oven of Akhnai:

https://voices.sefaria.org/sheets/144163?lang=bi

(Talmudic interpretation doesn’t stress this story because it shows how much Jews love to argue, or because they can win arguments against God. It stresses the story because it demonstrates that the law has been received (“It is not in heaven”) and thus does not need to appeal to God. This, inter alia, is why nobody asks God for their opinion on eruvim.)

That wasn't clear to me :-). But even still: the eruv is a rabbinical interpretation, not something attributed to God.

It's happening for Actions security, at least. I don't know about their other services.

> PyPi and similar package managers should start publishing security warnings about the hosts of projects. That in turn can eventually lead to bans of packages from generally insecure places. Maybe if we start seeing 'WARNING: projects from github.com may contain malware!' after doing pip install XXX MS will start listening.

PyPI goes out of its way to not be an arbiter of package quality or security. PyPI really doesn't want people assuming those things based on presence, since it's (1) an open index, and (2) the resources needed to make those kinds of determinations at PyPI's scale are several orders of magnitude greater than what PyPI actually has access to.

(This is different from PyPI removing malware based on user reports, which does happen. But that's a reactive task and not one that comes with any sort of blanket guarantee.)

GitHub's response time on malicious repositories is often lackluster. However, from conversations with folks at GitHub, my suspicion is that this is because they're being starved for resources, not incompetency or maliciousness.

This (IMO) points to a perverse reality: things need to get worse before they can get better. In other words, Microsoft probably needs to feel more pain (in the form of negative revenue pressure) before they take their own platform responsibilities (vis a vis not distributing malware) seriously.

We say this play out recently with improvements to GitHub Actions security, I expect we'll see the same here.

(Edit: to be absolutely clear, I have first-hand experience that GitHub's security folks work extremely hard, and are often doing the kinds of invisible, thankless "deck-swabbing" work that nobody even thinks about. They're just under-resourced.)

If you're referring to the Belgian blocks[1], I don't think they're actually from Belgium specifically, just Europe in general. Although some might be from Belgium!

[1]: https://sohobroadway.org/soho-broadway-block-by-belgian-bloc...

You can look it up if you don’t believe me. Orthodox Judaism doesn’t have a supreme religious authority; it’s a general classification for ~hundreds of distinct sects and movements.

(Each movement has its own beis din, but there is not “high court” that unifies religious rulings across them. Which is why an eruv created by one group is often considered invalid by others.)

AFAIK the eruvim are not dictated by “high courts” (there is no “high court” at all), but are instead constructed and maintained by various Orthodox groups. Not all Orthodox groups agree about their propriety either, so the presence of an eruv doesn’t necessarily mean that it’s recognized.

(Also pedantic, but it’s not the entire city: there’s a Manhattan eruv that covers a large part of Manhattan, while Brooklyn, Queens, and the Bronx have separate smaller eruvim.)

> Religious infighting with 2 fundamentalist sects that somehow boiled down to some people digging a tunnel connecting 2 buildings because of... reasons?

The very short version is that both groups claim ownership of that building, and one of them wants to expand it significantly. Neither group is allowed to modify the building because of a court order, so the one that wants to modify it did it surreptitiously in an attempt to create facts on the ground that would modify the status quo.

(Obviously none of this is sensible, but the actual reasons are pretty quotidian.)

Python uses PEP 440 versioning, which isn’t semver. Semver is de facto common, however, and can be expressed as a subset of PEP 440.

I’m not sure what you mean by “change in dependencies”: every release has zero or more distributions (“files”), and each distribution in a release can specify its own dependencies. For example, a macOS-specific wheel might depend on something that Linux-specific wheels don’t need or vice versa.

> but I also hope you understand that it violates the principle of least surprise.

I think it’s fair to say that virtually everything about Python packaging violates POLA :-)

There are a lot of misconceptions here:

1. “Wheel” signatures (e.g. RECORD.jws) were specified but never actually implemented in any major packaging installer, to the best of my knowledge. In that sense it’s not really meaningful to ask whether they’re “still” supported, since nothing ever did support them unfortunately. The reasons for that are multifaceted, but a single easily identifiable reason is that they don’t solve the key/identity distribution problem.

2. The primary benefit of index-supplied hashing is for lockfiles. Specifically, they make lockfiles like uv.lock and pylock.toml useful/meaningful.

3. Is the source distribution problem, and is sort of fundamental to any packaging system that has any amount of dynamism in it. There’s a good argument to be made for reducing unnecessary dynamism (e.g. via wheels), but a nontrivial percent of Python users actually want this behavior.

Re: files: I think you’re getting confused about distribution files (sdists and wheels) versus the individual contents therein. You can’t modify distribution files on PyPI, you can only add new, unique ones. And this is now restricted to 14 days for each release.

It doesn’t. It’s really only relevant if you use version-specific APIs/ABIs in wheels containing machine code. Pure Python wheels don’t generally need this.

Very pedantic of me, but I figure it’s interesting to note: technically a release on PyPI can have zero files or even one or more wheels but no sdist. The former is a degenerate case that users don’t normally see, and the latter happens if the user chooses to only upload wheels (or their sdist upload fails for whatever reason).

(This doesn’t change your observations at all! Just as a demonstration of how Python packaging’s data model can be unintuitive.)

Yes, that would be a better setup and it’s exactly what PyPI is moving to[1]. This closes a gap in the legacy upload API, which evolved over time instead of being designed to avoid these kinds of problems.

[1]: https://peps.python.org/pep-0694/

Because uploads to PyPI are not atomic. They’re now capped within a 14 day window, but it would be extremely confusing to users to have the “release” hash of their dependencies change repeatedly.

(Also: we’d need to determine what it even means to hash a set of files. Do we order by canonicalized filename, by upload time, etc. Each of these has surprising implications!)

Edit: to be clear, it’s not ideal or fully intentional that uploads to PyPI are not atomic. But it’s the status quo and any change will take years.

Files are immutable on PyPI, releases are not (because releases are comprised of a set of files, and files are uploaded one by one).

This is unintuitive, but the TL;DR is that files will never change on PyPI, but (previously) a user could upload a new file to a release years after their last upload to that release. This has some legitimate use cases (like allowing people to support new Python versions without bumping a package’s version), but also makes introduces challenges around locking and release security that are elaborated in the thread linked by the blog post.

I agree this could probably be ratcheted down from 14 days over time, though.

I’m not sure what this has to do with TFA: Python does have hash-pinning. TFA is not about modifying existing files on the index (PyPI doesn’t allow that), but about adding new files to a pre-existing release. But that doesn’t change the hash of older distributions on that release.

The use case of uploading new wheels to an old release was actually (kind of) an accident: PyPI’s current upload API is stateless and originally there was only one file per release (sdists), so there was no need for a start-finish transition for releases.

(This will hopefully change pretty soon, with the “upload 2.0” work.)

You do not have to compare atrocities to make a point.

> But the original claim is that it is all statically checked in Rust

Where? The GP's comment says "prefers," which I didn't read to mean a blanket statement. I don't think anybody with more than passing experience in Rust (or C) would make such a claim.

> Some dependently types languages can prove this statically, also model checking can do this, etc. So it can be done statically as well, but not in Rust.

You're couching the part where it can't be done with full generality or can be done with full generality, but with punishing semantics. The appropriate comparison here is with other normal general purpose compiled languages.

> When the Einsatzgruppen were "cleansing" conquered territory they at least had the decency to do so as humanely as possible.

What a grossly offensive and patently false thing to say[1].

[1]: https://en.wikipedia.org/wiki/Babi_Yar

Yeah, that seems like a nice thing that Rust could offer. It strikes me as a weird thing to get hung up on, though, given that the norm in compiled languages - including C - is to express your bounds such that an optimizing compiler can (but won't necessarily) eliminate them.

(You mentioned WUFFS below, which is why I qualified with general-purpose! One thing that WUFFS does that I think Rust could add pretty easily is provable indexing, e.g. allow me to use a `u8` to index a `[u8; 256]` without having to widen to a `usize` first and hope that LLVM optimizes it back out.)

How would Rust perform static checks on dynamic bounds? That seems like an impossible order, i.e. not a reasonable evaluation criteria for any (general-purpose) language.

(I'm separately skeptical that it's the most important memory safety property; I suspect that a review of Chrome and Firefox 0days would show that UAFs and type confusion are, at least to attackers, equally if not more important.)

I don’t think a judge would be impressed by this. If anything it actually makes the legal case easier: there’s a legitimate use for a wiping feature on a phone (e.g. for theft or tampering), but actively producing false information demonstrates an intent to deceive or mislead.

I thought the standard for obtaining adverse/negative inference in a criminal case was higher than in a civil one, i.e. a judge will typically need more than just evidence that information was destroyed. But IANAL!

This would be under federal law, not Georgia law I believe. But even assuming it was under Georgia law: it hasn’t been entered into public evidence that the person in question knowingly destroyed anything. It’s entirely possible the phone had nothing of value on it; the prosecution is required to demonstrate that evidence was in fact destroyed or falsified.