Hacker News Reader

Comment by hijodelsol | Hacker News Reader

This is a serious risk for the open source ecosystem and particularly the scientific ecosystem that over the last years has adopted many of these technologies. Having their future depend on a cap-ex heavy company that is currently (based on reporting) spending approx. 2.5 dollars to make a dollar of revenue and must have hypergrowth in the next years or perish is less than ideal. This should discourage anybody doing serious work to adopt more of the upcoming Astral technologies like ty and pyx. Hopefully, ruff and uv are large enough to be forked should (when) the time comes.

View on HN

On the flip side, I'm not sure I ever saw a revenue plan or exit strategy for Astral other than acquihire. And most plausible bidders are unfortunate in one way or another.

They could have joined projects like the Linux Foundation which try to not depend on any single donor, even though complete independence from big tech is not possible. I don't know the motivation behind Astral's approach, but this acquisition does leave a weird taste behind about how serious they were about truly open source software. Time will tell, I guess. (Edit: typo)

> I don't know the motivation behind Astral's approach, but this acquisition does leave a weird taste behind about how serious they were about truly open source software.

It was because Astral was VC funded.

https://astral.sh/blog/announcing-astral-the-company-behind-...

Astral was building a private package hosting system for enterprise customers. That was their stated approach to becoming profitable, while continuing to fund their open source work.

Private package hosting sounds like a commodity that would be hard to differentiate.

Yeah you'd think so but somehow JFrog (makers of Artifactory) made half a billion dollars last year. I don't really understand that. Conda also makes an implausible amount of money.

From my understanding there are a lot of companies that need their own package repositories, for a variety of reasons. I listened to a couple podcasts where Charlie Marsh outlined their plans for pyx, and why they felt their entry into that market would be profitable. My guess is that OpenAI just dangled way more money in their faces than what they were likely to get from pyx.

Having a private package index gives you a central place where all employees can install from, without having to screen what each person is installing. Also, if I remember right, there are some large AI and ML focused packages that benefit from an index that's tuned to your specific hardware and workflows.

Private artifact repositories also help to mitigate supply chain risk since you can host all of your screened packages and don't have to worry about something getting removed from mvn-central, PyPI, NPM, etc.

Plus the obvious need for a place to host proprietary internal libraries.

We have some kind of simple pip repo that is private where I work. What would astral bring to the table?

How many people use that simple pip repo daily? If the number is not in the high hundreds, or a few thousands; maybe nothing. But once you get up there, any kind of better coordination layer is useful enough to pay money to a third party for, unless maintaining a layer over pip is your core competency.

Close to a thousand I’m sure.

I mean that was a thing at one point but I feel like it is baked into github/gitlab etc now

> a lot of companies that need their own package repositories

Every company needs its own package repository. You need to be able to control what is running on your environment. Supply-chain risk is very, very real and affects anybody selling software for a living.

This is besides the point that in the real world, not every risk is addressed, at least in part because available resources are diverted to address larger risks.

Makes sense to me.

Most of the companies that spend $$$$ with them can't use public registries for production/production-adjacent workloads due to regulations and, secondarily a desire to mitigate supply chain risk.

Artifactory is a drop-in replacement for every kind of repository they'll need to work with, and it has a nice UI. They also support "pass-through" repositories that mirror the public repositories with the customization options these customers like to have. It also has image/artifact scanning, which cybersecurity teams love to use in their remediation reporting.

It's also relatively easy to spin up and scale. I don't work there, but I had to use Artifactory for a demo I built, and getting it up and running took very little time, even without AI assistance.

Yeah I mean I understand the demand. My previous company used Artifactory. I just don't understand why nobody has made a free option. It's so simple it seems like it would be a no brainer open source project.

Like, nobody really pays for web servers - there are too many good free options. They're far more complex than Artifactory.

I guess it's just that it's a product that only really appeals to private companies?

Both Artifactory and Sonatype have somewhat restricted open-source options, which is part of their "get a foot in the door" product-driven sales strategy.

There are no competing open-source projects because such projects would need to provide more value than Artifactory/Sonatype OSS, which are both already huge projects, just to be considered.

JFrog has a free version. It's called the JFrog Container Registry. Lots of features are missing and you can't use the Artifactory API that it ships with, but it's there.

There are also several free registries out there: Quay, Harbor, and Docker's own distribution. They all have paid versions, of course.

A commodity yes, but could be wrapped in to work very nicely with the latest and greatest in python tooling. Remember, the only 2 ways to make money are by bundling and unbundling. This seems like a pretty easy bundling story.

It's also a crowded and super mature space space between JFrog (Artifactory) and Sonatype (Nexus). They already support private PyPI repositories and are super locked in at pretty much every enterprise-level company out there.

There’s always room for improvement…

I've used JFrog Artifactory before and I wish I didn't.

Ditto for nexus

With the goodwill and mindshare they earned, it does not feel impossible.

Perhaps OpenAI is aiming for a more compelling suit of things for penetrating enterprise (I'm just speculating as I go here).

that was never going to work, let's be honest

What would be the added value against JFrog or Nexus, for example?

i mean ofc but like you can self-host pypi and the "Docker Hub" model isn't like VC-expected level returns especially as ECR and GHCR and the other repos exist

My hope would be that this eventually pushes pip to adopt a similar feature-set and performance improvements. It's always a better story when the built-in tool is adequate instead of having to pick something. And yes UV is rust but it's pretty clear that Python could provide something within 2-5x the speed.

The problem is funding.

There seems to be a pervasive believe that the Python tooling and interpreter suck and are slow because the maintainers don’t care, or aren’t capable.

The actual problem is that there isn’t enough money to develop all of these systems properly.

Google says that Astral had 15 team members. Or course, it’s so hard to make these projections. But it wouldn’t shock me if uv and ruff are each individually multi-million dollar pieces of software.

If you’d like to invest a million dollars to improve pip, or work for free for 3 years to do it yourself, I’m not sure if anyone would object.

pip isn't exactly a "built-in" tool. Beyond the python distribution having a stub module that downloads pip for you.

`ensurepip` does not "download pip for you". It bootstraps pip from a wheel included in a standard library sub-folder, (running pip's own code from within that wheel, using Python's built-in `zipimport` functionality).

That bootstrapping process just installs the wheel's contents, no Internet connection required. (Pip does, of course, download pip for you when you run its self-upgrade — since the standard library wheel will usually be out of date).

These tools are open source, if they lock them down the community will just fork them.

This might be true for uv and ruff, and hopefully that will happen. But pyx is a platform with associated hosting and if successful would lock people into the Astral ecosystem, even if the code itself was open source.

Nice idea in theory, in practice is how many folks down in Nebraska are going to show up.

as someone who works in the python tooling space I think you underestimate the number of people who would be willing to do this. i would personally help maintain a community fork of ruff if it got to the point where one was needed, though I draw the line at moving to nebraska first.

It is proven by the amount of projects that eventually falled by the wayside after the first wave of volunteers run out of steam to keep it going post-fork.

that is a fair point, but I also believe that that happens when the project gets superseded by something better. I do not think ruff or uv will die because people went back to earlier solutions, if openai does kill them and the community fork runs out of steam it will be because someone made an even better tool, possibly incorporating the lessons learnt from astral's efforts.

isn't that's the point of open source software? like when Oracle bought Sun. someone forked mysql and created mariadb.

Indeed, and how much has it kept the pace versus the alternatives?

Also, the survivors are the exception, not the rule.

"Someone" was the original mysql creator. It was a brilliant way to get Oracle cash then havr his own company. Getting a cake, and eat it too.

I never adopted them, keep using mostly Python written stuff.

Either pay for the product, or use stuff that isn't dependent on VC money, this is always how it ends.

There are ways to independently fund open source projects, though. I have previously contributed to the Python Software Foundation and to individual open source maintainers through GitHub donations (which are not dependent on GitHub, as there are many alternatives). Projects like the Linux Foundation exist, too. And government funding, especially for scientific endeavors or where software is used to fulfill critical state tasks, is an option, too. I refuse to subject to the hypercommercialization of software and still believe in the principles behind open source.

Which is why I mentioned "....use stuff that isn't dependent on VC money...".

> I never adopted them, keep using mostly Python written stuff.

Maybe you use non-transitive pure Python dependencies, but it's likely that your tools and dependencies still rely on stuff in Rust or C (e.g.: py-cryptography and Python itself respectively).

I use mostly the batteries, given that the only purpose I have for Python, since version 1.6, is UNIX scripting tasks, beyond shell.

As mentioned multiple times, since my experience with Tcl and continuously rewriting stuff in C, I tend to avoid languages that don't come with JIT, or AOT, in the reference tooling.

I tend to work with Java, .NET, node, C++, for application code.

Naturally AI now changes that, still I tend to focus on approaches that are more classical Python with pip, venv, stuff written in C or C++ that is around for years.

> This is a serious risk for the open source ecosystem and particularly the scientific ecosystem that over the last years has adopted many of these technologies.

At worst, it's just Anaconda II AI Boogaloo. The ecosystems will evolve and overcome, or will die and different ecosystems rise to meet the need going forward.

I anticipate OpenAI will get bored and ignore Astral's tools. Software entropy will do its thing and we will remember an actively developed uv as the good old days until something similar to cargo gets adopted as part of Python's standard distribution.

Would single maintainers of critical open source projects be a better situation?

Are you not aware of foundations?

The issue is lack of money not lack of legal structure.

Consider ffmpeg. You can donate via https://www.ffmpeg.org/spi.html

How much money do they make from donations? I don't know but "In practice we frequently payed for travel and hardware."

Translation: nothing at all.

If such a fundamental project that is a revenue driver for so many companies, including midas-level rich companies like Google, can't even pay decent salaries for core devs from donations, then open source model doesn't work in terms of funding the work even at the smallest possible levels of "pay a reasonable market rate for devs".

You either get people who just work for free or businesses built around free work by providing something in addition to free software (which is hard to pull off, as we've seen with Bun and Astral and Deno and Node).

Google contributed tons of developer hours for things like bug fixes, without which the project might not be where it is today.

There are examples of foundations or other similar entities paying developers, like Linux, SQLite, even Zig.

Maybe the difference is some projects rely on core contributors more because external contributions are more restricted in some way.

But sure, the entire open source model doesn't work, lol

There's a wide gap between the arguments "the open source model doesn't work" and "the open source model failed to produce anything as good as uv after a couple decades of python tooling churn". The latter is why people are understandably unsure of where things go from here.

Seems like you're responding to the wrong person. The person I replied to said the open source model doesn't work. Nobody said the thing in your second quote.

I get the point you're making, but the way you introduced it isn't conducive to productive conversation.

I don't agree. If anything, I'd argue that the comments above were a lot less conducive to productive conversation than mine ("Would single maintainers of critical open source projects be a better situation?", "Are you not aware of foundations?", "But sure, the entire open source model doesn't work, lol").

The entire context of this subthread is whether or not the model that Astral was using was reasonable or not compared to an open source approach. From your initial comment, you've been touting alternatives, and the comment I responded to was giving specific examples of where you think the model worked. I don't think you've provided much evidence that there was a good alternative here, and when you're taking an opinionated stance, a productive conversation will sometimes involve people pointing out flaws they perceive in your arguments.

As opposed to Pip, which is obviously free and sustainable forever.

And takes a lot closer to "forever" to download all of your dependencies

I don't know how to search for that report, can you share it?

Reply on news.ycombinator.com