Wikipedia's in Trouble (2019)

Or the category thing - yes, it has circular references which breaks the nice theory behind it, but what practical issues to the readers or editors does this present?

The syntax is messy, yes and it's a problem. But converting it to a markdown/HTML is downright absurd. I think the current approach of improving it step by step is the most reasonable. Wikipedia is a project which can afford to sustain its own custom syntax (because of history, compatibility concerns and wikipedia's special needs).

I think the biggest issue isn't technical in nature - it's the actual task of building the encyclopedia - to decide together on its scope, decide what is notable and what isn't. To fight against brigading from interest groups, fight against attempts to game their rules. All of these are intrinsically difficult tasks and of course wikipedia has made many mistakes. But even then it still fares pretty well in my book.

The really serious problems are organizational in nature: the overgrowth of bureaucracy, the jungle of policies and "sort-of" policies that can be weaponized, the emergence of an inner clique skilled in navigating the quagmire, and the resulting impossibly high barrier to entry for new contributors.

The experience of contributing to Wikipedia these days is inherently adversarial. People come and write something genuinely useful or at least do so in good faith, and then realize they unleashed on themselves a whole lengthy process when they have to defend what they did on numerous talk pages over an extended period of time, or succumb to deletionists. Even if they prevail, the experience is hardly pleasant, so they eventually leave and never look back.

An artefact of the above is also that the editorial base suffers from an overrepresentation of people with vested interests, as they will always be the most highly-motivated to stay. There are of course many great editors too but the way things are going, the project is not sustainable, and its quality is being compromised due to an organizational failure.

What Wikipedia needs is not Markdown (not that I have anything against it) but courage, fresh air, and less of the siege mentality.

Wikipedia predates markdown by several years. The only truly “universal” markup language I know of at the time was bbcode.

Even if markdown has been designed, it’s lacking support for templates and tables that wikis make heavy use of.

Every wiki software I’ve used has had a custom language for one reason or another. Usually good reasons.

-wikitext unparsable - wikitext is a bit insane but there exists a parser called parsoid. If we couldn't parse it it would be impossible to make a visual editor

- most pages are redirects: not sure what the problem is.

-boilerplate text - there is a template system to have repeated text in only one place. Not sure what the issue is.

- he goes on a rant about no search without giving much context. There is a search feature based on elastic search (older versions were based on lucene directly). In my opinion its a pretty decent search engine (especially compared to most sites that make their own search). I'm not sure what the actual complaint is.

-complaints about wikidata - this is more political than technical, however "If wikidata was a company, it would not exist anymore, and you wouldn't have heard of it." seems patently false. Wikidata is pretty popular even outside of academia, and is used quite extensively.

- category tree being a graph not a tree - that's kind of unfortunate but what exactly is the problem here. Its a problem on commons, but i've never really seen how its an issue in practise on wikipedia. Complex categorization is being taken over by wikidata anyways.

Template ecosystem is complex - it could certainly be better, but the complexity here is a trade-off allowing more flexibility and allowing the system to evolve.

Inclusionist vs deletionist: no comment

UI design: perhaps a fair point here, although i do kind of like the stability of the current design. Most of the modern web sucks imho.

Moral failure: i dont want wikipedia to fix the world. That's not its role. Its job is to document, not to partake.

Viz editor not being enabled by default: i agree, although i think it was pushed too hard in early days when there was still kinks, but its long past time now.

To be clear, i definitely dont think wikipedia is perfect, i just disagree with some of these specific criticisms.

Freebase, DBpedia, and many others (me) have tried, but the reality is that the markup language is poorly defined and the only path that is really tested is the one that ends up rendering HTML.

If you feed HTML from Wikipedia into a web parser that supports the DOM (say Beautiful Soup) you can generally parse out what you want pretty effectively. Once I switched from the "markup rabbithole" to "parsing standard HTML" I was able to turn my MediaWiki extractor into a Flickr extractor in about 15 minutes.

What matters to Wikipedia visitors is content quality (and managed depth) and reliability. All the rest is froth. Layout? Are you kidding?

Could it be better? Hell yes. It'd be great to see many articles, written by committee, visited by a professional editor with years of proven experience -in that category-. The Foundation's got the money to pay one per category. Each has to visit 5 articles per day. Each they finish is locked and marke 'pro-edited in (the year)'. Make that fact a search filter. ('Administrators' choosing? Brrrr.)

Then there's Wikibestia.org. Every year, copy a limit of 1 million articles there. Chosen how? Poll the visitors, it's theirs! 'Wikibestia ... the people's choice!' 50 net downvotes? Guillotine! Big contest, media advertising, whatever.

Articles that can only be read by subject-matter experts completely miss the point. They're just there for vanity or whatever. Flag them, give them one year to move to Wikiexpertia, while versions for non-experts are prepared (or not), then delete.

I think the biggest problem he mentioned is the "exclusionists." There's no harm in including a page on every local elected official, every Nobel prize winner, even every band that has a reference or two somewhere. Most likely it will remain a little article, but so what? The history of small local bands, for example, would be very interesting 100, 200 years from now. (Wouldn't it be fun to read about some town's local musical groups in 18th century America?)

You can put bounties or some kind of incentive for people to take on the conversion, but I think if the new format is better, a number of people will feel strongly enough about it to want to convert each article when it's updated. And even if 12 years from now some articles aren't converted, so what?

But that's fine. It'll create more innovation as people ditch it and create better tools. The one part of the article I disagree with is where they say it's "unparseable", which obviously it's not. It's parsed millions of times a day. Even by third party tools that harvest it for semantic data.

That means it can be transformed and used to seed other databases using non-terrible description languages like Markdown.

It's untrue that they haven't added features over the years: they have, but they do it at a glacial pace. Consider one of the recent ones: when you hover over a link, a box pops up and shows a preview. They used to render math on the client-side, but now, they just convert to SVG: as a result, it loads instantly, and renders consistently. The WYSIWIG editor rollout was slow, because people felt that it would attract low-quality low-commitment edits. They first released it as an optional feature, and then turned it on by default, when they were confident that it worked as intended. Oh, and my favorite? Allowing an article to start with a lower-cased word (say iOS); I remember that there were a bunch of redirects just to correct for this deficiency.

Yes, it is a giant pain to edit some pages in that arcane syntax, but nothing else even comes close in terms of features.

Yes, there are an enormous number of templates, but in practice, an infrequent contributor just finds a page that uses a similar template, and copies it out.

Yes, there are lots of bots, and they try very hard to guard against spam, without making you sign up or even solve a CAPTCHA to edit. Plenty of bot edits are "good" edits: they revert rage-rewrites, rage-deletions, and all kinds of malicious user behavior.

What you don't understand about redirects is that, the good ones can't be automated. It's not a string-matching problem. Yes, they could automate /some/ of the redirects, and they try. I've personally never run into a typo-redirect in recent years.

Yes, it can get political at times, and it's _very_ difficult to have objective guidelines about which pages are worthy of existing. Politicians' pages often get locked, when there's an upcoming election, and this means that you need an account to edit. Again, MW has lots of great features.

Wikipedia is aging, and nobody can deny that, but who would want to do the thankless work of parsing the markup and porting it to another system, AND correct the breakages? What commercial value does it have, and who's going to fund it?

this reminds me of the library in the david brin uplift books, where the myth is that it contains perfect knowledge but in fact there are sinister memory holes

(not saying it's sinister in this case)

To be more accurate, those are the editors on the English Wikipedia (the largest one). The French Wikipedia, for instance, have this visual editor enabled.