Thanks for the explanation! Makes sense. ai.gov would be a better fit.
52 karma · joined September 5, 2022
If you're nearby Munich, feel free to visit us in person :)
How does it look about long-term sustainability? Looking at the git history, ~97% of bcachefs commits are yours. What happens if you step back, burn out, or can't continue for any reason? Is there a fallback plan? A community or team that could realistically take over?
For anyone evaluating this for production use in a company, that's the question that matters most. A filesystem isn't a library you can swap out — you're locked in for years. The technical quality can be excellent and it still won't pass a risk assessment if it depends on a single person.
Quote from webpage: "The COW filesystem for Linux that won't eat your data"
Quote from webpage: "It's the job of the filesystem to never lose your data: anything that can be repaired, will be."
Quote July 2025: "I've been digging through the bug tracker and polling users to see what bugs are still outstanding, and - it's not much. So, the experimental label is coming off in 6.18."
I was a big fan of bcachefs and was looking forward to deploying it across ~100 machines in production. Unfortunately, the removal from the mainline kernel has seriously undermined its credibility for use within a company environment.
A filesystem needs time to mature, and that's fine — but the official webpage should clearly display a warning that this is still experimental and that its long-term support situation is uncertain. People evaluating it for production use deserve to know what they're getting into.
Would have loved to use it.
Are you interested in autonomous developing and finding creative solutions on your own? Then join our Wahtari team as Frontend Developer (m/f/d) and take an active role in shaping the future of our hardand software platform for machine vision tasks. You will work in a highly focused, independent and enthusiastic team, where you will get to play your dev skills in an highly motivated environment. Your responsibilities include the whole lifecycle of software products such as design, development, testing, deployment, maintenance and improvement. Also, utilize your expertise to solve scalability issues and to expand Wahtari’s product portfolio.
I try to avoid python as much as possible, because I mainly work with Go & C++ and multi-threading with those languages is just better (imho). Bringing python a step forward and making it future proof might be a good thing... Even if this means to break some things? Not sure if dismissing the GIL is the right step, but there is a big performance gap to fix. Or maybe the AI community must move to a better suited language? Having python code in production just feels so wrong. Especially if a rewrite in another language shows the performance gap.
Quote: "In PyTorch, Python is commonly used to orchestrate ~8 GPUs and ~64 CPU threads, growing to 4k GPUs and 32k CPU threads for big models. While the heavy lifting is done outside of Python, the speed of GPUs makes even just the orchestration in Python not scalable. We often end up with 72 processes in place of one because of the GIL. Logging, debugging, and performance tuning are orders-of-magnitude more difficult in this regime, continuously causing lower developer productivity."
Quote: "We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers."