Hacker News Reader

Comment by takluyver | Hacker News Reader

I agree, though it's worth noting that while volunteers can maintain the software and administer the indexes, they also rely on infrastructure provided by big corporations. E.g. the Python Package Index runs on a CDN provided by fastly, which serves hundreds of TB per day. I very much doubt the non-profit Python Software Foundation could afford that bandwidth if it wasn't an in-kind donation.

View on HN

It's a couple petabytes, Michael. What could it cost, $10?

Seriously though, Fastly's donation of their CDN service is generous and eases the burden on the PSF, but if push came to shove they could definitely afford the bandwidth. In 2018 they had a net income of half a million.

Hi, PyPI maintainer and PSF director here.

There's absolutely no way the PSF could afford PyPI's bandwidth out of pocket. Last I checked our "bill" from Fastly would be close to $1.5M/month.

Also given that PyPI is critical infrastructure for millions of people and software projects, anything cheaper would not really cut it.

As of March PyPi was pushing out 300TB a day through its CDN. Ignoring the "off the shelf price" of $0.12/gb and assuming they negotiate a bulk discount driving them down to $0.05/gb that's still $15,000 a day (or just shy of $5.5 million a year). Their net income in 2018 would cover less than 10% of that bill.

Well.... they couldn't afford Fast.ly, or Akamai, or Brightcove, but they could afford any tier-2 or tier-3 CDN.

For example CDN77, would start you off at 0.016/gb if you have more than 100TB per month, without any negotiation.

If you have 300TB per day, you surely can negotiate sub-cent pricing somewhere.

No, the costs are manageable.

I think the charitable/community projects setup for developers, need to act like non-profits in other sectors and actively seek out donations.

I assume there's some difference in e.g. speed or reliability between Fastly (8 c/GB) and CDN77 (1.6 c/GB)? Even if you can negotiate it to 0.8 c/GB, you're still talking about a huge piece of the PSF's budget. They would have to either cut other expenditure (e.g. making grants) dramatically, or find a lot of new income (continuously, not just a one-off donation drive). And PyPI's bandwidth is growing rapidly [1].

If PyPI didn't have sponsor providing bandwidth, I'd guess it would implement some form of rate limiting and encourage people to mirror/cache packages much more to reduce load. I don't think it would die completely, but it would be less convenient and still cost the PSF a fair bit of money.

[1] https://twitter.com/di_codes/status/1235707819955032069

The PSF's tax returns are published on python.org, and 'revenue less expenses' for 2018 was just under $280k. IANA accountant, so maybe that's the wrong line to look at.

Hi, I'm a PSF Director and the PSF's Treasure as of this year. For transparency, our tax returns as of 2018 (and soon 2019) are up on https://www.python.org/psf/records/

You are right that this donation does not show up in our tax filings because they provide it to any OSS project.

Also, it's not income that you would put in your 990s. Similarly, the donor cannot deduct the expense of the donation.

Correct on both points. (I do consider it a potential liability though.)

Yes, that's a good point. The Perl repo and services are also relying on donations from various companies. That said, it's easier to switch donors than it is to switch repositories.

If necessary, it seems like it'd be easy for any of these package registries to ~~blackmail~~ encourage big companies into donating infrastructure.

"If you don't support us, we might accidentally forget to audit our packages and feed malware into your build pipelines. It'd truly be a shame..."

Reply on news.ycombinator.com