Trying Out Rye
burakku.com
burakku.com
Homebrew is an interesting beast. I was debugging something architecture-related recently during one of my first forays into rust land and quickly learned that my brew-installed rust wasn’t going to cut it because I needed rustup. Meanwhile, I don’t think I’ve used brew-installed node since at least v12 because I use nvm to support and debug multiple projects with different node versions. Is this kind of thing just becoming par for the course?
For the pain with the Django project, I guess I’m also willing to cut Rye slack because the underlying goal seems admiral and sadly the problem the author encountered seems common in Python land in general.
Other tools are better for build or project dependencies.
Most Linux package managers are also not ideal for that purpose, incidentally.
I always want the latest version of git or ripgrep, no questions asked. Brew does that well. But for Rust, Node and others where you’re switching between versions, I prefer rustup, nvm etc.
Go is an exception though, I find I never want to go to a previous version of Go so brew is fine.
Which is a reasonable assumption to make, for example, if the language in question is also used by system utilities in a traditional Linux distribution. You really don't want yum invoking the wrong version of python.
For newer languages that change fast, though, we need something different. I don't think "every project brings in its own language runtime and the kitchen sink" is a sustainable situation, either. The dependency graph is crazy enough already.
Yep, this is the core reason: there needs to be a "system" distribution of Python, Ruby, etc. for packages written in those languages, and that distribution fundamentally needs to prioritize the interests of the package manager over "external" uses. This is particularly painful in the Python world, since using a "system" Python distribution for local development can break the system itself in all kinds of fun and exciting ways (usually beginning with a well-intentioned `sudo pip install ...`).
Homebrew suffers from this like other system package managers do, with the added pain that many macOS users expect Homebrew to do everything (whereas Linux users typically understand that they need `pyenv` or similar). PEP 668[1] helps a bit here by clarifying the responsibilities/expected behavior of system Python distributions, but at the cost of heartburn for some user scenarios (i.e. ones that depend on `pip install -U` and similar).
The problem here is that it's impossible to build a coherent distro if you allow arbitrary mixing of versions, without parallel installability. I don't think it can really be solved in the general case.
Yeah, I appreciate the honesty. I think it's just a straightforward "this is good enough for me, but I'm not going to hide the glaring red flags that will stop anybody from using this at work" approach, which I think is cool. Some people feel compelled to pretend that the choices they make for a weekend hobby project are exactly the same choices they would make with other people's livelihood on the line, and it really doesn't need to be the same standard.
Whatever that's supposed to mean.
(sometimes I try to make my meme* references four words long, as a nod to https://en.wikipedia.org/wiki/Chengyu )
* "ideas reproducing via people"?
Choosing anything, stating that out-loud and slowly forcing convergence would benefit all.
Lot's of options are great to explore the solution space, but it feels like we are way past that.
Probably a little early to officially bless Rye though anyway.
There isn't a lock file. How do you deal with dev dependencies? How do you work on a lib and its calling code without pushing it to pypa? Working with c deps?
All this stuff feels harder to do then it should be.
2) pip install . (optionally with -e)
There's a lot of compiled c code in things like numpy etc but it's mostly precompiled in wheel files so I guess I've dodged the authors issue (unless I've misunderstood something?)
Am I just lucky that the data landscape is different - would the author's issue happen with more or less any Django project?
Black Rye bundles Ruff for formatting (rye fmt).
Ruff Rye bundles Ruff for linting (rye lint).
> Ruff Rye bundles Ruff for linting (rye lint).
It took me a minute to realize that that wasn't some form of poetry or song lyric.
This would also largely invalidate the need to write the package manager in Rust. A Rust program managing Python packages adds an unreasonable barrier to entry and makes Python seem like a toy language if it can't manage its own packages.
Rust, Go and node (particularly in combination with Volta) show that combining package management and interpreter management into a close coupling can create a great developer experience.
For me (Rye author here) I wanted to see how close to that I can go. Unfortunately alone it’s quite a lot of work.
Decoupling the two while designing the interpreter manager CLI to forward some actions or parts of them to the package manager would be the best of both worlds. Users who can/want your interpreters use the merged CLI, others can still use the package manager separately with their favourite Python.
Which is why rye allows you to register any toolchain with it: https://rye-up.com/guide/toolchains/#registering-toolchains
The only reason it is awkward in Python is because of Python nonsense, not because it's a bad idea.
> makes Python seem like a toy language if it can't manage its own packages.
Well, quite. But plenty of people have to use Python so I don't see why we shouldn't make good tooling for it in better languages. The same thing is happening for JavaScript.
I really liked Hatch, but Rye - uniquely among standards-compliant new-gen package managers, afaict - has reasonable support for Python monorepos through its “workspaces”, letting you specify multiple local packages as editable installs.
I do believe Hatch is considering adding this at some point, but for now Rye is a good fit for monorepos.
Very fast, too. Armin and Astral just do great work.
Rye is the only game in town for monorepos atm, but the approach is far from definitive - I believe Armin has acknowledged as much. In particular we’re not completely convinced that the benefits of a single repo-wide interpreter are worth the trade-offs. Very interested to see where you land with it.
I am 100% bullish on some combination of PyBi, uv, or rye taking over the terror that is the ecosystem.
If you like the look of Rye, I suggest trying Hatch also.
Using virtualenv is just a really, really weird choice. The only reason anyone today would use virtualenv is because they want to support Python 2.X. Not that I think there's anything wrong with Python 2.X, in fact, I'd personally take it over 3.X any time... but I doubt this is what you had in mind.
On similar note: it's funny how you believe that installing from a Git repository is going to provide reproducible experience for everyone. Like, you haven't even considered the possibility that some time after you've done that another commit was added to that repository and things stopped working in the way they did before, did you?
Like... I mean, just give it some... a tiny bit, really, it doesn't require that much of critical thinking... and you'll see how your "advise" is ill-advised.
Flask and Sinatra contain code. They may not be "executables" by default but you will probably not read the source and they could obfuscate sending your local data to some malicious server or something. Also lots of python dependencies run code at install time (mostly to complie native dependencies but it is running python code that could be doing anything) so even before you run the test server for the first time just installing the dependencies may run code.
Open source is built on a web of trust. You could argue there's too much of it but I don't see a reason to declare rye one step too far.
Rye is kind of being used as the test bed for what uv will eventually be.
For me, rye (+ uv underneath) has perhaps the perfect workflow for an open source Python project. So I'm definitely using rye for that from now on -- instead of, say, poetry -- or hatchling directly, following the PyPA boilerplate[1].
You have a way of doing local development against any Python interpreter version. You have a way of tweaking dependencies and a straightforward sync/lock workflow. It all works atop "standard" PyPA infrastructure like pyproject.toml. You have a single command to build[1] project artifacts, like wheels. And you have a single command to publish new artifact versions to PyPI[2].
I think if you're doing local development on a project that is not meant to be published to PyPI, like a private Django project, then whether to use rye becomes more of a debate. For example, for a Django project I'm working on, I decided to just use uv directly, along with a Makefile. This is because during development of a Django project, I preferred to just use a plain requirements.txt (really, requirements.in) file, avoid the sync/lock workflow that rye imposes, and avoid the need to use something like "rye run". And rye's ability to build and package didn't solve a problem this project had, since the Django project wasn't being deployed via a PyPA packaging mechanism.
But this is probably also because the Python interpreter management problem, for me, is already handled by pyenv. I think if you're not already a pyenv user, rye is even more appealing because it handles "all" of the Python issues -- interpreters, requirements/dependencies, and packaging/publishing. (As well as a number of other standard dev-time issues besides, like testing, linting, and formatting.) But, in my case, I could hand venv management to uv, and then make dependency management part of a larger Makefile for my Django project, including custom linting, testing, and deployment steps. I wrote a little bit about my high level thoughts on Python packaging and dependency management, though this post was written before rye and uv were out.[4] If I were to update this post now, I'd say that you could swap uv for pyenv-virtualenv + pip-tools and be happier for it, and you could swap rye for the PyPA boilerplate. That is, you could upgrade to rye, or use it from the start, if you need packaging support, or if you like its dev-time workflow.
I'll also say, I found a little bug in how rye (+ hatch) interacted with my local git setup, and reported it to the rye team, and they helped me get to the bottom of it rather quickly.[5]
[1]: https://packaging.python.org/en/latest/tutorials/packaging-p...
[2]: https://rye-up.com/guide/commands/build/
[3]: https://rye-up.com/guide/commands/publish/
[4]: https://amontalenti.com/2022/10/09/python-packaging-and-zig
edit: found it https://github.com/pypa/hatch/pull/1317, already fixed
Nobody should have to "hope" that an article is about something, after reading a referring post's title.
Start flagging this, ingrates.
It sounds a little odd at first, but besides being the simplest to implement in Go, this also solves a problem for which Python needs tools like venv, pyvenv, virtualenv, ... that also this blog post mentions. The sheer number of these tools shows that this is a problem and that maybe that the solutions aren't optimal.
It also offloads most of package management (ie. pip like stuff) to just "go mod".
A specific binary per project also has other benefits in terms of securing the running environment, I think ... I'm still exploring this.
This is not yet in focus, but maybe starting from a totally different position will make some interesting or simpler tooling possible.
And in the same breath proceeds to describe Python ecosystem as hot garbage, no quotation marks needed.
Comeon, why do all these mental gymnastics trying to convince yourself your tools don't suck, when you already have all the evidence that they do?
As for Rye itself: yeah, no. pyproject.toml was supposed to do that... Of course it didn't work. And so did plenty of other "solutions" (Pipenv? Poetry?) that try to work around the defective basic components instead of building from scratch.
To fix this problem for real, the first stage needs to be the one where the people working on the fix understand the problem. As long as these people aren't willing to be honest with themselves and won't admit how bad their work has been so far, it's not going to get better. It will be more of the same band-aid.
/me close window
It's absolutely nuts that you'd repeatedly meet the same criticism, independently arrived at all over the place, and just conclude it's some statistical anomaly of bigotry and fools.
"Well, I don't have these problems, I just {{500 word article}}"