Wikimedia Foundation's runaway spending growth
en.wikipedia.org
en.wikipedia.org
For the record, I prefer that this thread have the same title as my essay. The current title misses several important aspects of my essay, such as transparency and the dependency on revenues always increasing.
Wikipedia also keeps "revision history" and a "talk" section for each page. Those original pages created before 2005 can still be modified, potentially adding to the overall cost. Breaking news stories can cause massive amounts of churn which I imagine has increased with the site's popularity [1]. So really 11x pages = 33x higher hosting costs doesn't seem unrealistic considering how much metadata is associated with each page. That's not to say there isn't a problem, but "page count" might not be the best metric. I wonder what average number of revisions per page looks like over the last 12 years.
[1] https://xtools.wmflabs.org/articleinfo/en.wikipedia.org/Talk...
Push as much off to the CDN as possible, and dynamically reduce the cache time as edit velocity increases on a page (with cache time increasing again as edits fall off).
Have a bunch of caches (or a cache hierarchy) which request pages on first use and then cache them indefinitely. Then when a page actually changes, you push notify all the caches to immediately evict that page.
Better would be to re-request the page after several hours and compare it to the cached copy. Then if they're different, update the cache, but also generate an alert to notify the administrator that something "impossible" happened and there is a bug somewhere that needs to be fixed.
Page 3. In 2016 they spent $2M in hosting vs. e.g. $2.2M in "travel and conferences" and $3.6M in "donations processing expenses".
They spend more on processing donations than hosting. They give away about 5x as much as they do on hosting. What value are these people adding beyond the cost of hosting the repository built by people who are not paid?
Have you actually read their annual report lately? I don't follow what they do at this point, but the last time I checked many of their contributors were paid.
If the latter, then they still account for a fraction of the budget.
Bull. Shit. An operation which could be run for about 5 mil is running an order of magnitude higher for no discernible change in service.
Operating Leases
The Foundation has a seven-year noncancelable operating lease for its San Francisco location. The lease expires on September 30, 2017.
Minimum rent payments under operating leases are recognized on a straight-line basis over the term of the lease including any periods of free rent. Rental expense for operating leases for the years ended June 30, 2016 and 2015 was $1,341,844 and $1,291,452, respectively.
Future minimum lease payments under noncancelable operating leases as of June 30, 2016 are as follows:
Year ending June 30:
2017 $ 1,306,986 2018 329,565
Total minimum lease payments $ 1,636,551
My salary was in the 100k to 150k range. That is not a princely sum for software developers in the Bay Area, but it is a lot of money, especially when you consider that it came primarily from donations of $50 or less that donors could have spent elsewhere, on other causes. I was humbled by that fact and made a point of reflecting on it daily to make sure I was doing my best to maximize value for donors and users.
Causes me to immediately stop listening and start doubting everything you said and claim, questioning your motive.
It is an emotional appeal. Which 1) insults my intelligence, thinking I'm stupid an unaware enough to fall for that. And one or more of the following 2) you're too stupid or poor at arguing that you don't know you're doing it 3) the facts don't support your side so you try to rile up the ignorant and gulliabl to win 4) you don't care about argument / have other motives like fame, clicks, distraction from other issues.
Now that sentence does not make any sense. Author must have forgotten it when he changed the title
"I don't know about all dem der hosting costs you speak of, but I sure as hell don't like that cancer"
I think this is much more closer to the actual point he's making.
Also I'm not sure that condescending the author when they've actually put a lot of effort into sighting their claims and spelt out what their point is clearly and coherently, is necessary.
Whether you agree with me or not, I would ask everyone to keep an eye out for any factual errors so that I can correct them.
For me, it echoes the tendency of every angry child on the internet to call things they don't like "cancer." Edginess of this kind does not speak to me.
I think we're just going to have to agree to disagree about this.
[1] - http://www.comscore.com/Insights/Rankings
I've worked with a bunch of sites farther down that list that are spending an order of magnitude more than that.
If the site was repeatedly down or slow people would complain even more. Wikipedia is truly global and that adds to costs.
This is probably the best point in the entire essay, if they reduced spending funding would essentially become obsolete. Kind of clashes with the begging banners which have made me donate to WMF the last couple of years.
[1] http://gawker.com/5192786/jimmy-wales-definitely-not-getting... [2] http://www.hoovers.com/company-information/cs/revenue-financ...
As was Animats.
For a proposal like this, I would expect to see a breakdown of where the money has gone and realistic projections of futures expenses. Simply providing alarmist statements based upon linear extrapolations (after being adjusted for inflation alone) then proposing an even more alarmist buyout (when the only thing of tangible value is the domain name) is not enough when proposing potentially crippling changes to funding.
The Foundation’s financial statements are online: https://en.wikipedia.org/wiki/User:Guy_Macon/Wikipedia_has_C...
Just to take one example, Sue Gardner, Wikimedia Foundation executive director from 2007 to 2014, received $100,000 pay rise and was secretly kept on as a "special advisor" after we were all told that she stepped down in 2014. This was kept secret for years, until it turned up buried in a required financial disclosure.
I also would like a breakdown of where the money has gone and realistic projections of futures expenses. What you see is my best effort to provide that. If anyone has more information, I will be glad to add it to my essay.
This sort of comment is really pernicious. If you bring up such an accusation, you should really be prepared to back it up.
It's possible that they'd already cut down on the spending by the time you joined. After checking back when I would have seen the case I mentioned, it was in fact 2010 and not 2011.
Also is it possible that the foundation has different practices in different countries? I was in Australia at the time for reference.
Out of curiosity, how common were these local community hackathons? Even flying economy it seems a bit excessive to be flying people across the world for that...
Here is a detailed breakdown of the costs associated with the most recent hackathon, and some of the things that came out of it: https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2017/Repo...
I'm not even a tall person, but economy is tight enough that my whole body hurts 1 hour into a ride. It's impossible for me to find a sleep position and watching last year's mediocre movies isn't a plus to me.
For a rare trip, I can deal with it to save money. But for frequent trips, I could see myself being quite pissed off if I had to take business trips regularly and sit in economy each time.
Really, if you're spending a lot of time in $place, be it an office, a plane, a house - whatever, you want it to be as comfortable as possible. Business class gets cheaper the more you use it, and even then for a business with revenue well into the tens of millions the price difference on a single airfare is negligible. You might pay a few hundred extra on a ticket, but in return, you get a happier and more comfortable employee.
It's in the same basket as every other 'quality of life' improvement a business could make like nice desks, social events, fancy coffee machines, free food etc. If it fits into their budget, and offers a benefit to their employees that makes them more productive and more inclined to stay - then I'd say it's money well spent.
Thanks. Curious, because this comes up on HN now and then: Do you make your research public? I understand this might not just be in your hands though.
Specifically, Google's changes to how they display Wikipedia derived info have driven traffic down. Links that used to go to Wikipedia often now link to Google's own sites with copies of the data.
https://searchengineland.com/wikipedia-confirms-they-are-ste...
Both of your links seem to confirm declining page views.
It also questions what the money is for. As far as I can tell, Wikipedia doesn't pay for good content, those contributors are unpaid.
The money is largely for the site and underpinnings. And we just established that views are being drawn away from that...
You have things like the comments system, Flow or Structured Discussions, which is a step back from the current system and yet still has money being spent on it.
The WMF's assets increased by $20,000,000 over the last year, up to reserves of now $90,000,000.
I'm not in a position to judge the effectiveness of their spending, but this completely antiquated paragraph about the waterfall model of software engineering makes me doubt the author's judgement.
No, you are missing the point. I did not imply that Waterfall if better than Agile (I have managed several Agile projects and really like it when it is done correctly). What my point is is that the WMF is failing to do things that Agile requires -- things that have been around so long that Waterfall required them. Nothing about Agile allows you to build software in secret with no input from the people who will be expected to use it and then throw it over the wall.
It would again stress that this is not the fault of the developers. Their management forbids them from talking to the ordinary Wikipedia editors who will use the software.
I would also note that focusing on hosting costs pretty much ignores the point of my essay, which is the exponential growth of spending and the dependence on future exponential growth in revenue.
And all this is against a backdrop of continuing to present themselves as a scrappy do-gooder surviving on fundraising and volunteer efforts.
The threat Macon identifies is that the bottom will drop out of the exponential growth in revenue and spending, leaving them no option but to sell out to Google, Facebook, or another party that will simply buy and monetize them, destroying the basic value that Wikipedia does still offer.
I don't know how worried to be, but I was genuinely shocked to find out how much money they take in and how many staff they have, without fundamentally being much different than I remember from a decade ago.
wikipedia is great for mainstream stuff, but even technologies with vibrant communities often lack a wikipedia page. and i'm sure the problem extends well past tech
See https://www.reddit.com/r/wikipedia/comments/74ba05/slug/dnxd...
Do you mind mentioning some of them?
But it does point out that there's a lot of inconsistency about the quality and depth of articles. Lots of articles are pretty good but then I'll hit some important tech topic that has a pretty bad entry. Things like biographies are also wildly uneven depending upon how currently well-known a person is.
The Wikimedia Foundation doesn't write articles or decide what topics to include; the Wikipedia community does that, at no cost to the Wikimedia Foundation. Anyone who wants to contribute can join the community at any time.
Also keep in mind that Wikipedia does not allow original research, so if a topic doesn't have much reliable third party coverage, then the article wouldn't be able to include a lot of details.
I'm not sure if that all requires this level of growth, but its use and importance have grown exponentially. I can't think of many places that deserve those funds more, really.
Re: "It's also become a target for parties that want to manipulate it, but has been surprisingly resilient to that" not a penny of the budget deals with parties that want to manipulate Wikipedia. That is dealt with by unpaid volunteers like me. --~~~~
The same is true for Google. Are they negligent for not running the whole thing out of a Starbucks?
You mention that the WMF has a year's budget in reserve. For a non-profit, this is generally considered a bad thing except in certain forms like building an endowment, because hoarding cash is just a way for a non-profit to actually profit and keep it. The previous ED of wikimedia actually called this out as a problem because they don't have the governance structure to prevent "log-rolling, self-dealing, and other corrupt practices".
I think the question is if they actually need a $65mil operating budget -- or one that's growing 15%+ per year.
Especially since they're giving away a sixth of their budget in awards and grants! At that point, they're taking on a secondary role as a financial institution funding projects in addition to their primary one. If they cut this, they'd need only $55 million and would have almost two years of reserve.
(We can also talk about if salary creeping up from a third to half the budget over the last ~5 years is appropriate. But that's a smaller issue.)
Also, look at how much they spend for nothing https://meta.wikimedia.org/wiki/Wikimedia_Foundation_salarie...