Hacker News Reader

Comment by danpalmer | Hacker News Reader

Having written a bunch of Flake8 plugins, including custom plugins for internal use at a company, and using 5-10 popular plugins on every Python project I work on... rewriting the entire linting ecosystem in one monolithic Rust tool doesn't feel like the best solution. There's no good story for building an ecosystem around this yet, and I think that's a big hurdle to overcome.

Ruff is fast, sure, but the benchmarking seems a little disingenuous, as I believe their number includes caching, but doesn't necessarily include caching for other tools. In fairness, not all the other tools have caching, but it is common to run them through pre-commit and therefore only on the current git diff, which speeds them up by orders of magnitude.

View on HN

> Ruff is fast, sure, but the benchmarking seems a little disingenuous, as I believe their number includes caching, but doesn't necessarily include caching for other tools.

It looks like the ruff benchmark is run with the `--no-cache` arg: https://github.com/charliermarsh/ruff/blob/main/CONTRIBUTING...

There's no reason your custom bespoke plugins couldn't be called by ruff as necessary. It's silly to burden the happy path of 99% of users who just need common sense python linting with those edge cases and custom needs.

That could be done in one of two ways:

- Supporting flake8 plugins, using the existing community, and sacrificing the speed.

- Requiring new plugins, in Python or another scripting language, sacrificing the community progress and goodwill, and sacrificing some of the speed.

Neither of these options is good. The Python linting ecosystem is a mature one with a lot of investment into the existing tools, and rather than try to speed those up (which could be done in a number of ways), Ruff started from scratch.

It doesn't feel like the right decision for an ecosystem that is as community focused as Python, and the engineering reasons feel like a toss up at best.

???

There is no downside to adding your bespoke flake8 plugins. For people that don't use them (99% of people) they get the benefit of blazing speed. For your custom plugins you live with the tradeoff of slower execution to do those AST passes with flake8/python tools. Even if ruff didn't exist you would still be burdened with your slow flake8 plugins speed. There is zero downside to you and only upside to people that aren't you.

Kinda just sounds like you're grousing because somebody moved the cheese.

It's not just "custom" plugins, it's all third-party plugins though right? If someone wants to publish a new linter for something, right now they can, and others can use it easily, but Ruff centralises that and makes it harder.

You're right that it will probably still be faster overall because "most" linting will be done with Ruff and any extras would be done externally, but now you've either got 2 tools when you had 1 before, or you've got to shell out to Python which adds overhead, or you've got to rewrite plugins in a Ruff-compatible format, or something else.

> Kinda just sounds like you're grousing because somebody moved the cheese.

I'm just disappointed that someone looked at slow linting and decided the answer was their own new tool, rather than participating in the existing community. Now the effort has forked, it'll take more work overall in the community, and we were already lacking engineering resource.

I'm disappointed the flake8 community hasn't prioritized performance and has led to python linting being much less widely utilized. I'm looking forward to tools like ruff giving much faster and more usable linting.

That's what esbuild did for their plugins—which made them useless since it kills any performance gains of switching to esbuild in the first place

Yes the point isn't esbuild will have magic pixie dust that makes nodejs AST processing magically faster--that's objectively impossible. The point is you can migrate to the new system over time without losing critical plugin functionality right now. It's not to live in a steady state with old slow nodejs plugins.

Having now switched several projects from webpack+babel to esbuild, with a bunch of plugins to replace various bits of resolution magic, SCSS compilation, and more, the performance gains are consistently astronomical. Simple builds go from taking several seconds to consistently completing in an unnoticeable amounts of time. Full rebuilds are still significantly faster than incremental (--watch) builds used to be.

Setting this up takes a bit more work than for webpack, and it's easier to run into limits on what's reasonably possible, but I don't intend to ever go back.

Reply on news.ycombinator.com