What's up Python? Epic CPython commit, Django 5 and 2FA for PyPI
bitecode.dev
bitecode.dev
I heavily used Django since version 0.96 (2007) until 1.5 (2014) (with Python 3)! Then came a long break with lower-level infrastructure work: linux kernel, perf tuning, some C, some Go and zig.
Last week I picked up Django again. After 10 years (!!!) of not even looking at it. It felt like meeting a good old friend: they are the same, but older and more mature. Conversations are the same. But better.
Things change in the tech world, often for the worse. Django changes, as it matures, for the better. The website layout, tutorial, colors, excellent release notes, `./manage.py runserver` are all the same as I remember them.
Except today it's easier, since I am much wiser in deploying and maintaining things than I was a decade ago. :)
Part of the reason I took a break from tech in the first place was the relentless upgrade cycle. Thrilled that we've solved at least a few of the problems in sustainable ways.
It probably does help that we stick to Django 3.X as that's what's currently in Debian.
I do like the smaller frameworks like Flask or Bottle, but if I need a database or anything remotely more complex than answering a few API calls, then I don't see a reason to not pick Django.
PyPI announced orgs back in April, but it seems they still haven't figured out the details on pricing, etc. No telling when those will roll out, but I sure hope it's soon. I'm cynical, but the sequencing of work here very much feels like somebody at Google (or wherever) wanted to push a big open source security project to advance their personal promo case rather than thinking through the needs of serious project maintainers.
2FA was not created as a defense against password manager compromise. That is not its purpose. It protects against password reuse attacks and helps to protect against total compromise of people who have been phished.
Even better, a password manager can avoid giving up a TOTP code to a phisher in the first place because it is checking the domain.
If your password manager is compromised, you’ve got big problems regardless of 2FA tokens being in there or not.
The extremely marginal security benefit of storing the 2FA tokens separate from your password manager is just not even worth discussing in most scenarios. It exists, but doing that causes the additional risks of losing access to your 2FA token or having your 2FA code phished, both of which seem a lot more likely than your password manager being compromised. At least, as long as you’re using any halfway decent password manager.
Long term, the goal is to get rid of passwords and 2FA altogether by switching to Passkeys. Each Passkey will naturally be stored in a single place, since they can’t be split into multiple parts anyways.
That doesn't check for me:
- 2FA tokens being there -> total compromise
- 2FA tokens not being there -> no compromise of 2FA-protected accounts
Or did you mean something else?
> having your 2FA code phished
What would be a realistic scenario? If I'm using a password manager, it won't recognize the phishing domain, which means I won't get to the 2FA step.
A 1Password vault is fully encrypted and protected by several layers of security. The most important layer of protection: the 1Password vault is encrypted with a combination of your password and your Secret Key[0], which is a long key generated uniquely for each vault. Even with a weak password, the vault has very strong encryption because of the Secret Key, which you don't get the opportunity to mess up and make weak by accident. Without both, the Vault cannot be decrypted, and nothing is stored in plain text; everything is stored in the encrypted vault. An additional layer of security is that they can't even get the vault from 1Password's servers without both your password and a second factor, assuming 1Password hasn't been compromised, but this is not critical. Even if the attacker got their hands on the vault, the vault itself is very secure. No attacker is going to be able to brute force the encryption key.
The most likely way (probably the only realistic way) for a well-secured password manager to be compromised is for someone to gain access to your machine while your password manager is unlocked. A simple keylogger is not enough, since it won't capture your Secret Key unless this machine has been deeply compromised since the day you set up 1Password on it for the first time. But, even then... that would mean they already own your machine.
So, total access to your fully unlocked machine, with your password manager also unlocked. That's what password manager compromise means in this context, at least to me. Remote access or physical access, it doesn't matter. As I said in my previous comment, if they have access to your password manager, "you've got big problems", because they probably have access to a lot more than just your password manager. If they have access to your machine, and your password manager is unlocked at the same time, it's game over for virtually anyone at that point.
It doesn't matter if the 2FA tokens are in there or not. It doesn't even matter if the passwords are stored in there, although I'm sure they wouldn't complain about having access to the passwords. Most services will allow the threat actor to reset your 2FA token (and password) simply by requesting a reset email with a verification link. Since the threat actor already has access to your machine, they almost certainly have access to your email, which the vast majority of people leave signed in. The password manager contains the username you use for each service, which is all they need to start firing off reset emails.
A very few websites won't let you reset your 2FA token, of course, but it's much fewer than the number of websites with 2FA. Anything other than verification emails (or never letting you sign in again) is very expensive for a website operator. Plus, what are the odds that you're not already signed into those sensitive services on this compromised machine? They may not even need your 2FA for whatever they're trying to do here. They own your machine. In the absolute worst case scenario (for the attacker), they just leave a RAT (remote access trojan) on your machine and walk away. They would just wait for you to sign into whatever they need, while you're completely oblivious to the attack. The password manager is an irrelevance.
The thing is... very few people get compromised this way in the first place. It's not worth losing sleep over unless you need to protect some extremely important asset. Certificate Authorities lose sleep over these kinds of threat vectors when it comes to their root signing key, of course.
I suppose we could also say something something quantum computers? Maybe some three-letter government agency can unlock your encrypted vault by waving a magic wand over it? If that's the threat vector you're worried about, then I don't think storing the 2FA tokens in a separate app is likely to help very much, but I guess it's something.
Even in my first comment, I admitted that there can be a very marginal increase in security by keeping your 2FA tokens separate from your passwords, so it can be the correct thing for certain scenarios. But, it does present additional risks, especially for TOTP. For those scenarios, I would generally recommend a YubiKey and using U2F instead of a TOTP app on a phone. For your security to be better off by keeping 2FA tokens out of the password manager, I believe that you need to be implementing some extreme security practices all over the place. Otherwise, it won't matter. Your password manager should be an extremely secure place to store 2FA tokens. If it isn't, then you need to find a better password manager ASAP.
Perhaps there are some other ways a good password manager could be compromised that I haven't considered in this comment, but those methods seem like they would have to involve either serious design flaws in the encryption or a big wrench[1]. You can never be 100% sure about any particular implementation of encryption, but what are the odds that someone is going to burn a very expensive zero-day exploit on you specifically? If they would do that, why? If there is a single service, or a single certificate, that needs the utmost protection, then yes, you need to take unusual steps to guard it. But this does not apply to almost anyone.
> What would be a realistic scenario? If I'm using a password manager, it won't recognize the phishing domain, which means I won't get to the 2FA step.
Usually, someone receives an important-looking email that calls them to take action by clicking a link. They urgently click the link, and begin trying to sign in. If it is being done by a threat actor who has already compromised your password by another means, they would just skip straight to the 2FA token prompt.
But, considering how skeptical that person sounded of password managers in general, I wouldn't be surprised if they're the kind of person who avoids password managers for their "most important" accounts anyways. Instead, choosing to use (relatively weak) memorized password(s). So, then they get phished for their memorized password, then reach for their "secure" separate 2FA app, and a 2FA code gets phished that way too. Game over.
[0]: https://support.1password.com/secret-key-security/
Apologies for the wall of text, but I didn't have time to write a shorter explanation.
It’s not ideal, individual accounts seems like the only reasonable solution for legal and auditing reasons, but at least it’s possible to conveniently share users with 2FA enabled if you need to.
sites implementing 2fa don't make it easy to share the keys (because they shouldn't, that's bad!) but a shared totp key is better than no key.
Sites that offer TOTP as a second factor normally either have the seed printed out next to the QR code, or have a button (or link) to show it.
It's a major hassle to scan a QR code from the laptop screen without leaving the laptop, whereas copy-pasting the seed into the password manager is easy. Pasting it somewhere you can share with other people is just as easy since the seed is just a string of characters.
There's also the fact that TOTP is an open standard, so one could very easily implement a bit of software that translated the QR code into the seed, so there's really no point at all for websites to try to protect the seed from the user. The user owns the seed and the code.
TOTP and yubikey are excellent technologies that way. They allow two-factor authentication without breaking privacy.
Everyone within the sound of my voice: get a password manager. It sounds like a hassle but it makes your life infinitely better. It allows you to keep your life private and more secure than it was while providing more convenience than you had before.
I recommend KeepassXC. Open source, audited, fully featured, and can be paired with one of several different kinds of syncing technologies depending on your risk appetite.
I expect some people don't want to mix work accounts on their personal phone ("keep your life private"), and because smart phones are still not yet universal, even among developers.
I only installed KeepassXC two weeks ago to try it out because several people here on HN mentioned it, and because it was free software not connected to for-profit companies.
It is the only one I've tried, and I've only used it once, to see what it was like.
I think your historical comment omits something. When I made this complaint back in 2019 you replied at https://news.ycombinator.com/item?id=20058199 saying "I've forwarded this thread along to others working on PyPI as part of the OTF grant, and we'll be figuring out how best to explain using TOTP without being too mobile-centric."
That mobile-centric list hasn't changed, and I still don't have, nor want, a smart phone.
IMO, the most important thing about a pull request is to... actually be productive. I've worked with people in the past who would nit-pick my commit messages wanting me to waste hours of my time trying to use arcane Git commands. Any of which might and probably will clobber my work.
If you're doing Git reviews a good use of resources is to look for security problems, performance issues, and general engineering problems with someone's code, etc.
A BAD use of Git reviews is to spend the time making stylistic comments 'I would have written it like this, it looks much better', being overly pedantic about code formatting, or forcing your OCD on someone for their Git history. The reason people hate code reviews is somehow they're always done by this latter group of person.
If you're forcing people to waste time for silly reasons then you can count that they'll eventually leave your silly company. I know I have. I was once about to work at a company but saw that they literally used a committee of people to review every commit message wherein they would force every engineer to rewrite code for reasons that seemed to make astrology seem like a hard science. Consider not doing that.
It's not unreasonable (within the limits of sanity, of course), it's just a matter of respecting other people's time.
If you're not willing to bother with command line: the Git client in Jetbrains also lets you edit commit messages in a very straightforward way.
In a Gitflow repository I don't particularly like squashing feature branches since a sequence of commits allows the author to better document what they were thinking about. Squashing is fine in Trunk-based development since feature branches are usually less extensive.
I hate intermediary merges on feature branches because they tend to make up a large portion of the branch history and are harder to review than normal commits.
It's worth noting that that 2FA requirement will have (virtuous) knock-on effects: package uploads will require an API token instead of allowing a password, meaning one less place where a user can accidentally expose control over their entire account. For packages published through GitHub Actions, PyPI's Trusted Publishing goes a step further and removes the need for a shared API token entirely[1].
Edit: Thank you!
How many people do you think will bother to delete the global token after having used it, and then generate a scoped one? 1%? Probably much less than that.
(Note: even though PyPI calls them "user-scoped tokens," they have less access than a password does, since they can't manage the user's account itself. So, while not ideal, they are still a better choice than a user/pass combination.)
[1]: https://docs.pypi.org/trusted-publishers/creating-a-project-...
import datetime
class my_datetime(datetime.datetime):
@staticmethod
def utcnow():
return datetime.datetime.now(tz=datetime.timezone.utc).replace(tzinfo=None)
datetime.datetime = my_datetime
import my_patches
import datetime
print(datetime.datetime.utcnow())Don’t monkeypatch Python, m’kay?
from my_patches import monkey
monkey.patch_datetime_utcnow()
But if you're not writing gevent or pytest, you should probably avoid it. It's much better to explicitly import or create the modified version of the code inside your own namespace, like:
from helpers import my_datetime_utcnow
import datetime as dt
def my_datetime_utcnow(): return dt.datetime.whatever...
Why prefer to work with naive UTC datetimes, rather than explicit UTC datetimes?
What's wrong with plain `datetime.datetime.now(datetime.UTC)`?
user_over_30d_old = user.created_at < datetime.datetime.utcnow() - datetime.timedelta(30)
But in the future, I would need to do:
user_over_30d_old = user.created_at.replace(tzinfo=datetime.UTC) < datetime.datetime.now(datetime.UTC) - datetime.timedelta(30)
Which isn't any better than the solution I proposed above, of removing UTC from the now() datetime. Alternatively, I could modify my DAL code to promote datetimes from naive to timezoned in the database at the edge of the python layer, but that would have a lot of knock-on effects in things like isoformat() now appending timezone information, which has caused issues in other broken APIs that I use.
It should never be acceptable to break working software for anything else than critical security problems.
It recommends to use "hashlib" instead, which isn't API compatible to crypt, and if you load it on a new enough python... triggers a deprecation warning about "crypt" being deprecated. Oh, and it seems unmaintained.
Anyway, the PEP mentions that "crypt" is not secure, not thread-safe, not cross-platform and not useful for modifying the system password database... so it sounds like you really shouldn't use it for much of anything. What's your usecase?
> What's your usecase?
We need to store hashed passwords that are then used by third-party programs (like Apache or exim) to authenticate users, so we need to generate salts and hashes in formats compatible to them.
It works with passlib after some fiddling, it was just way more fiddly than I'd expect from python.
People get this wrong more often than they should. The link says things will get removed in 3.13, not just deprecated. So many people seem to think deprecate is just some fancy word for remove. I really don't get it. Remove means remove. Deprecate does not mean remove.
No idea which kind of tests are these "overall" though.
https://www.youtube.com/watch?v=HxSHIpEQRjs
Slides (pg. 31 performance):
https://github.com/brandtbucher/brandtbucher/blob/master/202...
This makes development to be insanely productive, I joke that to create a new feature in django is "pip install <new feature>", check out your options here: https://djangopackages.org/
requestss
requestss/requests
For example, I know that "kotlin-stdlib-jdk8" in Maven Central is the official package for Kotlin standard library, but what about "kotlin-stdlib-wasm-wasi", "kotlin-jdk-annotations" and "kotlin-native-compiler-embeddable"? Here it helps to know that they're all in the "org.jetbrains.kotlin" group.
1. Company develops package `company_secret_project_stuff` and publishes version 1 on their internal private PyPI instance.
2. They tell their employees to install it via `python3 -m pip install --extra-index-url https://pypi.intranet.company.com company_secret_project_stuff`
3. pip dutifully goes and installs the hacker's version of `company_secret_project_stuff` (version 999999) from the global PyPI index.
I know for a fact that my company is vulnerable to this. The only solution currently is for the company to also register `company_secret_project_stuff` on the global PyPI. But you can guess how happy they were about exposing internal package names. They opted to remain vulnerable instead (yes; stupid decision but I can kind of see their point).
Namespaces trivially fix this. You just register the `@company` namespace on the global PyPI index and then make sure all your private packages are in that namespace. Attackers can't publish packages with the same name as yours, and you don't need to make them all public.
One pip install only for private packages using the --index-url (NOT --extra-index-url) and then the other pip install for public packages no index modifiers needed.
Indeed, not a hill worth dying on either way but it’s a little wild the counter argument is “increased support costs” when, let’s be real, there is no significant increased support beyond the initial scope of work.
If core Python ever plans on consistency, introducing these aliases early is the main way for them to achieve it. Even if there is no direct plan for deprecation. Something something about the best time to plant a tree.
Deprecating often-used APIs like datetime.datetime.utcnow() because they're ugly? Sure, we'll do that, that won't cause anyone problems unless their code is wrong. (At least they didn't set the removal date to be +2 releases = +2 years, as they usually do.)
When there's no strong advocate for a module, implementing changes becomes a challenging task. However, modules with dedicated champions, such as datetime by Paul Ganssle or pathlib by Barney Gale, may undergo significant modifications after consideration and discussion.
Not everyone in the broader Python community will be pleased with these alterations, but they aren't made hastily. I suggest we show empathy towards those who willingly take on the responsibilities of being open-source maintainers, it's a fiery task.
While I've personally expressed dissatisfaction with the utcnow change on the Python discussion page, I also acknowledge that I'm not responsible for maintaining this module. Consequently, I've updated my code in a completely backwards compatible way: datetime.datetime.utcnow() -> datetime.datetime.now(datetime.UTC).replace(tzinfo=None).
I also rarely see methods removed at all, which is why the utcnow one sticks out so sharply for myself and others.
I can't reconcile this with the statement of "policy to eagerly deprecate and quickly remove things", but maybe you have some evidence?
Further there is a clear path discussed on the Python board of how to move any pure Python modules to PyPI for any part of the community that wish to maintain them. In the earlier years of Python it was not a serious option to ask everyone to use third party libraries, but now for almost all use cases it is a reasonable option.