Hacker News Reader

Comment by film42 | Hacker News Reader

> The modern Wikipedia hosts 11–12 times as many pages as it did in 2005, but the WMF is spending 33 times as much on hosting...

Wikipedia also keeps "revision history" and a "talk" section for each page. Those original pages created before 2005 can still be modified, potentially adding to the overall cost. Breaking news stories can cause massive amounts of churn which I imagine has increased with the site's popularity [1]. So really 11x pages = 33x higher hosting costs doesn't seem unrealistic considering how much metadata is associated with each page. That's not to say there isn't a problem, but "page count" might not be the best metric. I wonder what average number of revisions per page looks like over the last 12 years.

[1] https://xtools.wmflabs.org/articleinfo/en.wikipedia.org/Talk...

View on HN

Since 2005, hosting has gotten much cheaper, and techniques for storing extremely large amounts of data have gotten much more efficient. But as far as I know, MediaWiki is no better positioned to begin taking advantage of those facts. It's still a PHP app accompanied by a huge, vertically scaling MySQL. There's no telling what might work better for Wikipedia, but there's plenty of avenues for experimentation. Maybe revision history could be more effectively stored in Amazon S3-like storage, instead of MySQL. Maybe not, but there's many such possible optimizations I doubt have ever seriously been considered.

I think the savings would be in having an efficient core with lots of caching, similar to the Stackoverflow architecture.

Push as much off to the CDN as possible, and dynamically reduce the cache time as edit velocity increases on a page (with cache time increasing again as edits fall off).

You can do even better than that with something read-mostly like Wikipedia.

Have a bunch of caches (or a cache hierarchy) which request pages on first use and then cache them indefinitely. Then when a page actually changes, you push notify all the caches to immediately evict that page.

Sounds error-prone if the notification ever fails to reach some subset of the caches. A long expiration time (e.g. several hours) is probably almost as good, but also "self-healing".

Giving out the wrong page for several hours is obviously better than giving out the wrong page forever, but it's still broken.

Better would be to re-request the page after several hours and compare it to the cached copy. Then if they're different, update the cache, but also generate an alert to notify the administrator that something "impossible" happened and there is a bug somewhere that needs to be fixed.

You're right, page count is not the best metric. Instead you can just look up the hosting costs in their yearly report and find out it's a pittance compared to all their other spending and declining in relation.

Could you link me to that?

https://upload.wikimedia.org/wikipedia/foundation/4/43/Wikim...

Page 3. In 2016 they spent $2M in hosting vs. e.g. $2.2M in "travel and conferences" and $3.6M in "donations processing expenses".

Well, shit. That’s horrendous, half of the money is just going to salaries.

Given that the entire point of WMF is that they pay people to do stuff, it would be alarming if salary wasn’t their largest expense. Why would anyone donate if not to pay for them to work on stuff?

Most of what they do is host the work of others, who are not paid. It seems more than a little obscene that a corps 300 employees are splitting 31 million, for the task of overseeing a 2mil hosting operation.

They spend more on processing donations than hosting. They give away about 5x as much as they do on hosting. What value are these people adding beyond the cost of hosting the repository built by people who are not paid?

I mean, that's what the complaint is about. People think when they donate it's to ensure the Wikipedia keeps running and is maintained. In reality most of that money is feeding an insatiable beast that has very little to do with "running Wikipedia".

> Most of what they do is host the work of others, who are not paid.

Have you actually read their annual report lately? I don't follow what they do at this point, but the last time I checked many of their contributors were paid.

Unless they’re one of the 300 employees splitting that salary, winning awards, or under the ~3mil of specialist services....

If the latter, then they still account for a fraction of the budget.

Bull. Shit. An operation which could be run for about 5 mil is running an order of magnitude higher for no discernible change in service.

The vast majority of people who write Wikipedia articles are unpaid. We are all volunteers, writing with our own time. In fact, paid editing must be disclosed: https://en.wikipedia.org/wiki/Wikipedia:Paid-contribution_di....

Having office space in SF seems to be...unnecessary? Can someone explain the need for this expense?

Operating Leases

The Foundation has a seven-year noncancelable operating lease for its San Francisco location. The lease expires on September 30, 2017.

Minimum rent payments under operating leases are recognized on a straight-line basis over the term of the lease including any periods of free rent. Rental expense for operating leases for the years ended June 30, 2016 and 2015 was $1,341,844 and $1,291,452, respectively.

Future minimum lease payments under noncancelable operating leases as of June 30, 2016 are as follows:

Year ending June 30:

2017 $ 1,306,986 2018 329,565

Total minimum lease payments $ 1,636,551

Half of expenses being wages and salaries isn't clearly out of line and is probably fairly typical for a lot of non-profits. Also, if you look at what the senior people are being paid, it doesn't really seem particularly excessive--low six figures.

There's a difference between a nonprofit and a charity.

There seems to be an obscene amount of profit compared to expenditures related to the actual site. Charity hardly even enters it; looking at those financials, nooooo one will accuse them of charity, barring their award activities of course.

Your comment mentioned salary specifically, not profit. Nobody should accuse mediawiki of charity because that's not what they claim to be. They are simply a company whose stated goal is not to make a profit. In that case it makes sense their salaries are high if we assume a positive correlation between salary and work completed.

Reply on news.ycombinator.com