Ruff: Python linter and code formatter written in Rust
github.com
github.com
I don’t think experienced python folks realize how much the flexible tooling slows people down, and creates friction for adopters. Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I know this is kinda the n+1 issue where astral is just adding another. But it feels like a complete and well designed stack, not a box of parts encouraging me to build my own. Reminds me a bit of go tooling.
I am a convert. I’ll happily jump on their type checker and packager when ready.
My bugbear with that XKCD is that even if it's true, it's also how progress is made. By this point I'm so annoyed whenever anyone links it that I kinda just wish Randall would take it down.
Introducing n+1 solution/ standard may be the necessary cost of making progress, but it is possible to incur that cost without gaining anything
(at the extreme, trolling is fun activity with net negative on bigger scale)
The point is, if you are learning to code, you can skip nearly all the ecosystem tools. Their need will arise when the time comes.
I moved to poetry for things a bit ago until I found out about uv recently, it’s so fast I thought it didn’t even do what it asked a couple times.
Elementary OS has the nicest editor that other Distros have not snatched up yet not sure why. Its fast, and fits the bill where KATE and Sublime Text are for me.
I've been using Python for right about 20 years now. I've been through various prior iterations of the "standard" packaging tools. But I've never found myself wanting a type checker, formatter or linter; and aside from a brief period with Poetry (in which I used barely any of its functionality), I've had no interest in dependency management tools, either. What I have found myself wanting is fixes for the problems in Pip.
Different people work on different kinds of projects, and approach the language differently, and have radically different needs as a result.
Do you still write type hints, but not check them? Or just use dynamic typing as in Python 2?
Actually, I guess the first case is really type-hints-as-comments, so isn't substantially different from the second.
> I've been using Python for right about 20 years now
Either way, I've found it quite common among long-time Python users to stick with Python's traditional dynamic typing, whereas newcomers to the language tend to really dislike it in favour of strict static typing. (One exception is GvR himself, who seems to be very pro-type-checking.)
As the second group continue to add increasingly complex typing features to the language, I wonder if Python is becoming less desirable for those in the first group.
I occasionally write them as a form of documentation - when it would be more concise than explaining in prose.
There is nothing specifically 2.x-style about making use of dynamic typing - it's a core property of the language. While 3.0 introduced the annotation feature, it wasn't even specifically intended for typing (https://peps.python.org/pep-3107/) and standard library support wasn't added until 3.5 (https://docs.python.org/3/library/typing.html).
>As the second group continue to add increasingly complex typing features to the language, I wonder if Python is becoming less desirable for those in the first group.
Some new changes have been missteps, in my view. There's still nobody forcing my hand when it comes to typing, really; I just shake my head when I see people trying to make it do things it really wasn't designed for. On the other hand, there are definitely things I'd love to see improved about the language.
And type checking is so great for both preventing bugs (which it does all the time) and self-documenting code. Can't recommend them enough.
The more of this you can automate, the more you get to spend on "real work".
I have my own formatting preferences, but my desire to not have to think about them or go to meetings debating them is much much greater.
I have yet to find a single drawback of adopting mypy (or similars) that isn’t completely eclipsed by the benefits.
I guess that makes one of us.
Getting stuff working is much more important. I'd rather people concentrate on stuff like CI, Unit Tests and Deployments.
Those projects weren't even dumpster fires.
You can, genuinely, do without all of this stuff — but they're just helpful tools, not silver bullets or the only way to do things, but helpful.
I do use eslint/prettier btw, but other collegues can never seem to get this working so I've just given up on it and then fix the linter issues whenever I come across them.
Between this and your other comment*, you seem to be simultaneously expecting me to be more and also less literal in how I read your comments.
I am starting to suspect that this playing dumb is entirely disingenuous.
I guess you're going to continue to get surprised by how often you get criticised in this specific way, by many, many other people.
Hint: when someone doesn't understand you, you were in fact not obvious, no matter what you think — communication is a multiplayer game, not a single-player endeavour.
That's actually the biggest issue I've seen when a bit back joined a python-shop. Everyone there had mostly done python their whole careers. Fell in love with language during studies, then applied for jobs using that language. Stuck in the old ways, perhaps python felt nimble and nice 20 years ago compared to other things at the time, but with little progress (from its users) in using modern tooling it felt so arcane coming to this shop. And it wasn't the job's fault, it was a state of the art python stack, just a decade behind what other languages have.
It was nice to be able to write little extra linters or flake8 plugins in the language I was linting. Plus decades of effort had gone into making those plugins pretty great, finding the right separation of linting codes so that the right things could be linted/ignored per codebase.
I understand why they did it, and they have done it remarkably well, but "rewrite it from scratch" is almost never the best answer and part of me wonders if this result could have been even better achieved another way.
But that doesn't stop me from feeling that there were things lost in this process and wishing for a better outcome.
When this happens periodically (rather than, every other week) you also get the original GNU advantage which is you can revisit an old tool but knowing what you know now, for example today "everybody" has DVCS, and so ripgrep checks for the .gitignore file.
I don't think you could get flake8 to be as fast as Ruff, but I think you could improve it to be within an order of magnitude, and honestly that's plenty good enough. There are a lot of low hanging fruit, particularly around caching. I'd push back on there being no chance of being "as good as" Ruff, because "good" is not just speed. Ruff is _not complete_, if you want all the features and linters, you still need Flake8, or you have to sacrifice some. It's also not publicly extensible, and not written in Python, both of which are fair choices but I think deviate from the ideal scenario here.
(Pyright isn't written in Rust, but adequate performance is still a design goal)
If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
To be clear: there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge. My point is just that sometimes it's easier to start over than to fix the old, and that fact itself feels kind of sad to me.
Sad, but also liberating. There are surely more projects out there that would benefit from breaking with conventional wisdom to start over.
I don't think that's true. I think the old toolchains really were bad, partly because the Python community was uniquely resistant to building on what worked in other ecosystems, and uv in particular largely succeeds because it's the first tool in this space to finally just ignore the nonsense that had somehow become accepted wisdom in Python-land and apply a little bit of outside experience.
I mostly attribute it to backwards compatibility concerns (which I in turn attribute to the traumatic 3.x migration). PyPI continued to accept egg uploads until August 2023 (https://packaging.python.org/en/latest/discussions/package-f...). The easy_install program, along the direct command-line use of `setup.py` (as opposed to having Setuptools invoke it behind the scenes), have been deprecated since October 2021 (https://setuptools.pypa.io/en/latest/history.html#v58-3-0); but not only are they still supported, Setuptools can't even remove support for `setup.py test` (as they tried to in 72.0 and immediately reverted - https://github.com/pypa/setuptools/issues/4519) without causing major disruption to the ecosystem. One of the packages affected was Requests, which: a) is one of the most downloaded packages on PyPI; b) is pure Python (aside from dependencies) with nothing particular complicated about its metadata (I see no reason it couldn't be done in pyproject.toml and/or setup.cfg); c) wasn't even reliant on that interface to run tests (they use Tox).
(For that matter: Requests has been maintained by the PSF since mid-2019 (https://www.reddit.com/r/Python/comments/cgtp87/seems_like_r...) and the problem could have easily been avoided once the deprecation was announced, but nobody did anything about it. The project still defines most of its metadata in `setup.py`; `pyproject.toml` exists but is only used for pytest and isort config, while `setup.cfg` is used for flake8 config and some requirements metadata that's completely redundant with `setup.py`.)
A lot of it also just has to do with lack of attention and focus. https://peps.python.org/pep-0427/ (defining the wheel format) was proposed in September 2012, and accepted in February 2013. But Setuptools itself wasn't available as a wheel until November 2013 (https://setuptools.pypa.io/en/latest/history.html#id1646), and in 2017 there were still reports of people having outdated versions of Pip and not being able to install Setuptools from that wheel (https://setuptools.pypa.io/en/latest/history.html#v34-0-0). Setuptools relied on a separate package to actually make wheels until July of last year (https://setuptools.pypa.io/en/latest/history.html#v70-1-0) - an effort which was initially proposed in June 2018 (https://github.com/pypa/setuptools/issues/1386). It also took years to notice that Pip and Setuptools had separate implementations for the code that understands the "tags" in wheel filenames and factor it out into `packaging` (https://github.com/pypa/packaging/pull/156).
I hate to say it, but the development of Requests itself has stagnated and really needs some attention.
The most notable fiasco recently was the introduction of significant changes to TLS/SSL in version 2.32.0 [1][2], which caused widespread breaking issues and even led to a security vulnerability.
Attempts to address these problems in versions .2 and .3 introduced new major issues which still exists in current version [3].
A patch to resolve the new issue was provided by one of the core members of the Requests project as early as June 2024 [4], but for some reason, nothing has been done about it, despite repeated pushes from the community.
If you check the commit history, updates have been sparse lately, even though there are still many unresolved issues.
[1] https://github.com/psf/requests/issues/6655
[2] https://github.com/psf/requests/pull/6667
I think the impact of Rust (and Go) shouldn't be underestimated here. Prior to these languages if you wanted fast runtime you were either using C or C++ with all their associated footguns, tricky build systems, and poor support for exposing and consuming libraries. Or you were using something like Java or C# which meant managing another runtime just for tools, which is especially bad for something like UV which manages runtimes - imagine if you were using a python tool to manage your Java versions and a Java tool to manage you Python versions, it would be a mess!
With both Go and Rust you have fast languages without memory safety issues that compile to a single, easily-installable binary and have a broad ecosystem of libraries available that mean that half the time you don't even need to write any difficult code and can just glue some modules together, and the other half of the time you only need to write the code you actually want to write and not all the support code for general pupose stuff like reading config files, file watching, etc.
Given the state of python packaging and tooling, I'd say that consolidating the tooling is a big win in and of itself, and with the tremendous speed wins on top...
Don't you have "decades of effort" to resolve this wonder?
The strong push to always contribute to the existing projects rather than starting new ones in the Python community is what caused the previous subpar state of tooling. I think Astral is also picking a good middle-ground where they still aim to build as much as possible on existing Python standards (and help contributing to new ones), so it's still building on what has been done in the past.
Having a clean process to rewrite from is what made it so fast. They knew exactly what outcome they wanted and they had access to the internal implementations on how to do it.
All that effort was not wasted at all.
Any opinions about the current state of the art type checker?
Code Name is Red Knot: https://github.com/astral-sh/ruff/discussions/12143
Main tradeoff is this only works if your codebase/core dependencies are typed. For a while that was not true and we used pylint + pyright. Eventually most of our code is typed and we added type stubs for our main untyped dependencies.
edit: Also on pylint, it did work well mostly. tensorflow was main library that created most false positives. Other thing I found awkward was occasionally pylint produces non-deterministic lints on my codebase.
Besides re-adding features that Microsoft makes exclusive to pylance, it tweaks a number of features that IMO makes pyright work better out the box:
https://docs.basedpyright.com/latest/benefits-over-pyright/p...
You can use an extension fork which swaps it with NetCoreDbg maintained by Samsung instead. This is what VSCodium uses.
Also note that both of these essentially consume the debugger API exposed by the runtime itself, you can easily make one yourself in a weekend but no one bothered because there isn't much need for another one.
For typechecker, I also vouch for Pyright which is what we use for all our django backends at work.
Just be aware that you will have hard time to typecheck part of your code where you rely heavily on django's magic (magic strings, auto-generated properties/methods, etc...).
In these cases, it's sometimes better to avoid these features entirely or accept that some part of your code will not be typechecked.
I've used ruff/uv for sure, but I never pay attention to Astral who is behind it.
The comment where you literally say "I have never heard of astral"? Gee, I wonder.
From the comment I read, quoted directly:
>I have never heard of astral though.
If you meant "I only learned about Astral thanks to this post", then I pointed out how you might have found out by yourself before.
I've used it a few times since then and now I'm a big proponent of using only ruff. I think most of its value comes from:
1. Being fast (or at least fast enough that it's not annoying). 2. Replaces the linting/formatting combo of multiple tools, reducing the cognitive load for the developer.
Anyway, big fan.
For example, with Pip you simply pass the `--python` option to have it install for a different environment - you can create your venvs `--without-pip`, and share a copy of Pip across all of them. If you use Pipx, you can expose and use Pipx's vendored Pip for this, as I described in a recent blog post (https://zahlman.github.io/posts/2025/01/07/python-packaging-...).
Twine and Build don't care about your project's environment at all - Twine uploads a specified file, and Build by default creates a new environment for the build (and even withough build isolation, it only requires your build environment to contain the build backend, and the build environment doesn't have to be the one you use for the rest of development). Setuptools, similarly, either gets installed by the build frontend into an isolated build environment, or can reside in a separate dedicated build environment. It doesn't actually operate on your project's environment - it only operates on your project's source tree.
then they don't need your dev tools.
>won't know what a venv is but do have to write python every now and again
In practical terms, you have to understand what a venv is in order to be a Python developer, for the same reason you have to understand environment variables to do anything serious in the shell. Learning the fundamentals is a one-time cost.
Language processing like compiling or linting is just one of the many aspect where rust can really play out it's awesome strength.
- The insanely long times are only for the initial build. Further builds are incremental and more tolerable.
- Compilation errors can be checked using 'cargo check'. It avoids the code generation step that's part of a build. I find myself doing it way more often than builds. So it's a time saver depending on how frequently you use it.
- You can extend the incremental build mentioned above using sccache [1]. At the minimum, it allows you to share the build cache between all your local projects. So it saves time if your projects or other builds share a lot of common libraries (that's very common in Rust, though). But sccache can go further by using online build caches (for example, using S3) that can be shared between hosts. Finally, sccache also supports distributed builds if you have a few machines sitting idle. (This is like distcc with extra features).
Now, whether a tree-sitter-esque ecosystem that is designed for source modeling could catch on like t-s did is a more interesting question
I'm not that fan of Ruff because to me it doesn't make any sense to add a rust dependency to "python" and it blows my mind that people are so keen to accept the ugly formatting inherited from Black just because "it gives consistency to the team, so I don't have to think, and we don't have to discuss code style"...
But all of that personal opinion set aside, what triggers my initial statement is that so many persons are so excited to run to use Ruff because... "It is so fast"... when I'm quite sure most of them would never notice any real speed difference with their modest codebase.
In almost all the codebases I have seen, proprietary and OSS, pylint or any other linter was quasi-instant. There are probably a few huge codebases that would see a speed benefit of a few seconds to use Ruff, but their owner would probably better have a look at their code to understand how their reached such a monster!
On the contrary, within only a year or so of coding by a small team, this ecosystem takes single digit seconds or less, while the traditional tools are taking minutes and more.
Particularly in analytics or scientific computing we've seen minutes versus under a second.
Having the tools not require additional/parallel Python dependency management is a plus.
Note that we watched these tools for a long time, and jumped in when rye converged with uv. It was promising to see fewer ways to do it.
// More on that here: https://news.ycombinator.com/item?id=41309072
Know what I care about more than Black making my own code look less pretty? It making my coworker’s code look less horrid. (And funnily, I’m certain everyone on the team thinks exactly that same thing.)
If it did affect people equally, it would be great. Unfortunately, spaces for indentation is an accessibility issue and the Black maintainers are hostile to making this configurable. Normally I am in agreement about minimising configurability, but this isn’t a matter of taste, it is making the Python ecosystem more difficult for some disabled people to participate in.
https://github.com/psf/black/issues/2798
Fortunately, Ruff makes this configurable, so you don’t have to choose between autoformatting and accessibility any more.
I do have a question, though -- can accessibility tools infer tabs based on visual layout or some other mechanism? It seems like one option is for accessibility tools to transform code into tabs internally, which would make them instantly compatible with the vast majority of existing code that uses spaces. What are the practical barriers to making that happen? And is there a good article that discusses all this in a clear, even-handed fashion?
parser.add_argument(
"--stdout",
choices = ("true", "false"),
default = "true",
help = "log data to stdout"
)
parser.add_argument(
"--stdout", choices=("true", "false"), default="true", help="log data to stdout"
)
I also like to have spaces around the "=" in my keyword=arguments, except for very short and simple calls.
PEP 8 says "Don’t use spaces around the = sign when used to indicate a keyword argument" so black is following that recommendation.
However, everywhere else in the PEP (assignment like 'x = 5', and annotations like 'class Spam: foo: int = 4' or 'def spam(foo: int = 4)', there are spaces on either side of the equals sign.
That irritates me every time I have to use black.
parser.add_argument(
"--stdout",
choices=("true", "false"),
default="true",
help="log data to stdout",
)
Formatters _are_ a compromise. They make your coworkers' code nicer, and your code worse.
One, as I learned, could be resolved by a simple use of a terminal ",".
The other is how it removes spaces from around "=" for keyword arguments, but not for other uses of "=".
I can't provide much more as I rarely use black. As a single developer, I don't have to worry much about that sort of compromise. ;)
That I think it's wrong and ugly is an entirely different point.
PEP 8 specifically says it it not universal:
> Many projects have their own coding style guidelines. In the event of any conflicts, such project-specific guides take precedence for that project. ...
> However, know when to be inconsistent – sometimes style guide recommendations just aren’t applicable. When in doubt, use your best judgment. ...
> Some other good reasons to ignore a particular guideline:
> When applying the guideline would make the code less readable, even for someone who is used to reading code that follows this PEP.
I think always omitting spaces there makes it less readable, even for someone who is used to reading PEP 8.
That makes me more compliant to PEP 8 than black. ;)
My point stands - I do not think those spaces are superfluous. Consider the following:
a = 4
def foo(i: int):
return "A" * i
class Spam:
foo = 4
bar: int = 6
def eggs(self, n: int = 5):
return foo(i=n)
For this one case of a short function with simple names, okay, I don't always use spaces.
But otherwise I think the lack of spaces makes it the code harder to read, and thus "uglier".
The rest of us just follow the PEP8 style guide and move on
I think that rule is ugly. I explained why.
If you don't like the thread, move on.
There are other issues with what you said, but the biggest one is: you have some strongly worded criticism for a project that has set a new bar for usability and consistency in Python code quality tooling. These tools are developed by humans like you and distributed to you for free with no obligation to use them. No matter how I look at your comment, I don't see how it's helping.
I see precommit hooks as where you avoid the low-hanging fruit, like “is this code actually parsable?”
(Unlike most formatters which are instant, dotnet format takes at least half a minute, because it performs a build just to format your code.)
But the main thing I like about ruff is that it is a single tool with consistent configuration, as opposed to a combination of several different tools for formatting and linting that each have their own special way of configuring them and marking exceptions to lint rules.
I work on a fairly modest code base (22k lines of python according to sloccount), and I'm seeing a significant decrease in runtime between `pylint src/ tests/` with default parameters and `ruff check src/ tests/` with the majority of checks enabled. More specifically, I'm seeing a decrease from 18 second to less than a tenth of a second with a hot FS and having deleted the ruff cache between runs.