Wikipedia needs an IDE, not a WYSIWYG editor
medium.com
medium.com
To be fair, I think this exaggerates the role of the editing UI -- if the editing UI was that bad, then people wouldn't have contributed in the first place. The more common narrative for reduced participation is the growth of cultural issues which make it less rewarding to participate (e.g. http://www.technologyreview.com/featuredstory/520446/the-dec... ).
(Edit: Not trying to be negative -- the suggestions in the article sound cool anyway.)
While bios are the biggest area, there's a bunch of other areas where I've learned not to pay attention to Wikipedia because it's too spammy: anything to do with tourism (full of booster and hotel spam), any company except very high-profile ones, many non-profit organizations, etc. It's good when writing about people/things that no longer exist, though; the articles about 19th-century French companies don't read like they were written by a PR agency.
For anybody who does an occasional 5-10 edits per year. The admins leave no moment unused to point out that you're doing something wrong (most likely based on Guideline CA 13, where CA stands for cryptic abbreviation₁) and in a lot of cases you're left to the god will of a single person.
The whole appeal system needs to be streamlined, it can't be that there a multiple entry point's for a deleted article, solely because it was deleted for different reasons.
It's the bureaucrats that drive people away, compared to Wikipedia the German tax forms/appeals are outright user friendly by now.
₁ I see no good reasons why abbreviations should not be expanded by default in discussion pages, makes it look way less intimidating to the casual user.
[For Wiki-Admin Eyes Only] If there any admins reading here: can someone please undelete GitLab? https://en.wikipedia.org/wiki/Wikipedia:Requests_for_undelet...
Thinking about those kind of issues I think they could improve the friendliness of the process by sending helpful / apologetic emails along the lines of "Dear Mr X, Sorry we had to take down your article because we need ... blah blah... If you can find the references just click this and do this. Best, Wikipedia team." Rather than the current situation where the article just disappears and there are somewhat cryptic comments under the talk tab. I don't think it would be terribly hard to implement.
https://en.wikipedia.org/wiki/User_talk:Mark_Arsten#GitLab_....
feel free to add them.
I don't understand were wikipedia's fixation (other than the name and origin) comes from to handle everything as a wiki page, were any kind of ticketing system might be way more transparent.
>I don't understand were wikipedia's fixation (other than the name and origin) comes from to handle everything as a wiki page
I think that's where a lot of problems come from and I think it's a tech issue that they built a wiki system and used it for everything and it's hard to change now it's ingrained. My guess with your appeal and the links is no one looked at it because it's not like an email system where someone gets it in their inbox, it's more that some one has to check on that page and maybe Bob thinks Joe is doing it or visa versa, perhaps.
I had just started contributing to wikipedia, a couple hours daily, then after about one week an admin decided one of my article title was typographically wrong, after a couple silent reverts from him I got to his personal discussion page to ask why and he brandished some arcane internal rules. I checked and he was wrong so I told him so but he refused to acknowledge this and suddenly I was faced with a bunch of people picking on me. I was followed everywhere i contributed and all my edits were reverted by seemingly fans of said admin who also showed up to gang vote against me in all attempt at resolving conflict.
So I went to the public wikipedia space to ask for help and discovered said admin had silently modified the internal global wikipedia policy to reflect his view and it's all downhill from there. I started receiving threats and at this point I was spending 4 hours daily on wikipedia but barely contributed a thing to the content anymore. After a bunch of ineffective bans I worked around, all my contribution were deleted on ground that I had no right to add content while banned.
Then came the messages from people experiencing the same situation who supported me because I stood up, I got tipped that a bunch of admins were discussing me on a secret page and escalated the issue until it reached jimbo wales himself who dismissed the whole thing as he said he fully trusted his admins. Then I discovered that other people went through the same process before me, I wrote an article on the importance of newcomers for a healthy wikipedia long term and the bias of the power structure and a weakness of wikipedia that would allow a few individuals to ruin the whole project. For this I received a permanent ban from all wikimedia project and a request from the top of the power structure to delete all my contribution to wikipedia side projects, wikitravel just deleted everything no question asked.
So i did the same thing others did before me I replaced a key article of wikipedia with this very story that happened to and reverted my change so now this story would live hidden in a page history inside wikipedia and stopped contributing to wikipedia to do something worthy of my time instead.
back then wikipedia was already a totally different thing that what it pretended to be and imho the problem wikipedia has cannot be solved with a better editor.
Edit: more articles identifying interpersonal friction as the leading factor deterring would-be editors [3,4]. At this point I'm suspicious of the author's due diligence in studying the problem, making the proposed solution more dubious.
[1] Suh, Bongwon, et al. "The singularity is not near: slowing growth of Wikipedia." Proceedings of the 5th International Symposium on Wikis and Open Collaboration. ACM, 2009.
[2] Thom-Santelli, Jennifer, Dan R. Cosley, and Geri Gay. "What's mine is mine: territoriality in collaborative authoring." Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2009.
[3] Kittur, Aniket, and Robert E. Kraut. "Harnessing the wisdom of crowds in wikipedia: quality through coordination." Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 2008.
[4] Halfaker, Aaron, Aniket Kittur, and John Riedl. "Don't bite the newbies: how reverts affect the quantity and quality of Wikipedia work." Proceedings of the 7th international symposium on wikis and open collaboration. ACM, 2011.
That's exactly what killed my one and only attempt to substantially improve an article (not just typo-fixing and stuff):
The two guys with a lot of edits on that page couldn't stand someone else editing "their" article.
Sure, they tried to bring up superficial disagreements about the content, but that could have been resolved trivially. It was very obvious that they didn't want someone else on their lawn.
Unless Wikipedia fixes exactly that ownership problem, specifically, I doubt they can improve their attraction and retention rate of contributors.
And that is a social problem. I bet it will never be fixed.
If you go in and blam an entire page, no doubt the original authors will cry foul; it probably took them a fair amount of time, even if what they wrote wasn't particularly good. It's also fairly rude without at least giving them a chance to improve sections.
Incremental editing will also give you a far stronger case when they get arsey, particularly if your inclusions are better than what was there already.
The key is trying to work with the grumpy people and reassure them that you're not trying to take their article away from them.
After the second or third edit, incorporating their feedback, someone called it "vandalism" and reverted everything.
Wikipedia is broken.
I used to see more great collaborative experiences, now I see even incrementalism and conciliatory pleas get rebuffed.
I wouldn't think much of my (anecdotal) experiences, but with all these articles claiming the same thing, I'm willing to accept this might be a wider trend.
In my view, that right there is the problem with Wikipedia. Just because someone puts up content on a "commons" site does not give them sole ownership or even a moral claim to resist changes proposed by the those who come after them.
Why should the average Wikipedia user not have the chance to clearly see the differences of opinion (in terms of value of information) right on the article page itself? The answer is most likely because the Wikipedia UI is not designed for lucid presentation of such differences.
Wikipedia's (subconscious?) striving to be an authoritative source of information is in direct conflict with very nature of information (and human knowledge) to be non-self-consistent and ever evolving.
FTFY
To me, this only furthers and enhances problems with ownership. If my first approach to "collaborative editing" is "give the original writers a chance to improve" rather than "collaborate" and "edit", then it only enhances the perception or "reality" that that article belongs to them.
This was typical of the paper encyclopedias. The initial creation of the Encyclopedia Brittanica was a huge job. The periodic maintenance update was a far smaller one.
I would contend that should be leading to more edits. Few people have the depth of knowledge of expertise to contribute to 'US Foreign Policy' article, but masses can contribute to an article about their home town.
For example, the article on my town doesn't even explain its unusal name. At least half of its residents could make that particular edit... but have no incentive to do so.
What proportion could/would actually cite a source for the edit, though? It's likely not that hard to find a source: the local library probably has some local-history books that explain the source of the name. But in my experience the proportion of people willing to open a book before making an edit is smallish. There are some local "edit-a-thons" trying to improve that, though. I've never been to one myself, but from what I can tell they tend to be organized at libraries or universities, and the organizers have already pulled out some relevant reference materials, so people can more easily start making cited improvements to articles about their local area.
Sociological issues among established Wikipedia editors do contribute do lower retention of desirable new editors, but like Animats I suspect Wikipedia's comprehensive coverage is a significant factor (perhaps more so). It's simply less appealing to do curatorial work than to make large additions to major articles like "Genetics" or "France".
Which factor contributes more to Wikipedia's active editors drop-off, sociological issues or encyclopedic completeness? Has anyone looked at that? I'd be interested to see an analysis of how the trends in the notorious graph at http://commons.wikimedia.org/wiki/File:Enwp_retention_vs_act... correlate with word count and reference count in important articles.
There just aren't incentives to contribute to Wikipedia aside from ego; its too hard to do it out of a sense of communitarian sharing. And its that same ego that makes the bureaucracy so awful.
However, Wikipedia feels that the editor is a big enough problem that they have spent a lot of time and money on their Visual Editor. That time and money could have been better spent.
Wikipedia would be greatly improved if they just issued a lifetime ban to anyone with more than 100 edits. They are all small-minded despots whose only intention is carving out a little fiefdom of pages and "protecting"/manipulating them by any means possible. Usually with a political agenda in mind.
It might be an unavoidable symptom of shifting priorities from addition to revision.
When I've worked on joint drafts, having many collaborators tackle a blank page is incredible. Our ability to rapidly churn out rough but readable pages is always stronger than I expect. Each new edit, even sentences in broken English, are putting ideas on the page, contributing value, and so more collaborators is almost always for the better.
There always comes a turning point during drafting, though, where additional collaborators suddenly become a liability. You always want a few different eyes for editing. But when you have enough collaborators polishing a mostly finished product, it will kill that piece (or add months to drafting).
My most optimistic read: maybe this is ok. Polishing is harder, not everyone is good at it. Maybe we should expect more edits to be rejected in articles that are 95% of where they need to be.
Pessimistically, though: Maybe each new collaborator just increases the risk of bikeshedding and petty squabbles. A couple bad edits create presumptions in editors, driving them to revert even high quality additions as their heuristic subconsciously shifts from "new stuff useful" to "new stuff harmful." The most stubborn or fanatical will drive out reasonable high quality editors, leading to a quality death spiral.
I don't know, it's probably somewhere in between.
The Wikimedia Foundation did a great deal of research, including professional usability testing, and reached the (I believe fairly self-evident!) conclusion that a visual editing environment was sorely needed. You can go look up videos of regular people attempting to contribute to Wikipedia if you don't believe me[1].
The difficulty of the task, the particularities of hardcore Wikipedia editors, and the importance of preserving the record that Wikipedia's edits represent makes the project have a very real amount of essential complexity. If you think this is easy then you have not done much work with complex rich text editing on the web[2]. People in this thread suggesting Wikipedia use an off-the-shelf OSS rich text editor don't understand the requirements of the problem.
The team at the Wikimedia Foundation has done an exceptional job given the task at hand, especially given their relatively small amount of funding (relative to other sites Wikipedia's size). The reasons the VisualEditor isn't enabled by default have to do with getting from a 97% solution to a 99.9% solution[3], and based on the work I've seen out of the Wikimedia engineering team, I'm sure they will get there.
1. http://commons.wikimedia.org/wiki/File:Wiki_feel_stupid_v2.o... and http://usability.wikimedia.org/wiki/Usability_and_Experience...
1. Or really any kind of rich text editing.
2. In terms of never, ever causing a problem with the underlying wikitext.
Oh, the arbitrary causation!
" Thing is, editors hated its bugginess so much that the roll-out was reverted shortly afterwards."
to:
" Their solution to this was a WYSIWYG editor, which failed for the basic reason that it denies the fact that Wikipedia is a program. "
Something tells me he might be making it up as he goes along! I wonder how successful Microsoft Word would have been with markdown editor.
[0] http://www.cs.rmit.edu.au/adcs2010/proceedings/pdf/paper%204...
https://www.mediawiki.org/wiki/Parsoid
This is the bidirectional conversion engine between wikitext and HTML+RDFa that powers VisualEditor and several other tools. It tracks source range to DOM structure correspondence as proposed in the post. At this point, the IDE is basically a user interface and performance problem. The conversion is readily available through a REST interface, but on the largest articles parsing from modified wikitext to HTML can take around 10 seconds. Most of that time is spent in the expansion of the myriad of citation templates that we like people to add. It is possible to speed this up to something more usable for an IDE, but it's not trivial.
You might also be interested in this blog post about some of the challenges we encountered while building Parsoid: http://blog.wikimedia.org/2013/03/04/parsoid-how-wikipedia-c...
First, it relies on the function Wikitext->HTML being injective. But isn't it trivial to create two different Wikitexts that compile to the same HTML? Whitespace is just the start of this story.
Second, apparently the template language is Turing-complete. Let's say I write a prime sieve in order to generate a page that lists the first 100 prime numbers. What would it then mean to edit "31" to change it to "30"?
(With apologies for not yet having read the things you kindly linked to.)
You can't. Using recursion (even primitive) is discouraged, there are even automated safeguards that make it hard (but I don't remember what they are exactly).
But I agree with you in general, I think making a visual editor that deals with templates correctly is not an easy task.
Yes, and Parsoid works around this by preserving some metadata about wikitext in HTML (such as information about whitespace around syntax elements) and, since this preservation isn't perfect, only reserializing HTML→wikitext where the content was changed during editing.
> Second, apparently the template language is Turing-complete. Let's say I write a prime sieve in order to generate a page that lists the first 100 prime numbers. What would it then mean to edit "31" to change it to "30"?
Assuming we're talking about VisualEditor, you currently just can't do that (you can only delete the entire template inclusion and replace it with normal text, or edit template parameters).
(As a nitpick, wikitext is not Turing-complete (there is no loop or recursion construct, recursion is explicitly checked for and causes an error), you can only write complicated algorithms by manually unrolling enough loops. However, for some time now you can also write templates in Lua, which is a proper Turing-complete programming language, see <https://www.mediawiki.org/wiki/Extension:Scribunto>.)
There is more to compliance than a simple feature comparison though, most of which can only be identified by large-scale testing. Each night, we have been running tests on a sample of 160k articles from 16 languages to check our progress. In this test setup, 99.99995% of articles round-trip perfectly from wikitext to HTML and back. Currently the focus is on visual diffing to identify remaining rendering differences.
There is still a good amount of work left until Parsoid is ready to replace the PHP parser, but most of this is actually not relevant to you if all you'd like to do is extract semantic information.
Regarding the paper: The authors correctly describe some of the issues inherent in wikitext parsing. Its conclusions are however based on strong assumptions about the implementation strategy. For example, they do not consider the option of flattening a PEG parse tree back to tokens in order to implement context-sensitive and generally unbalanced parts of the syntax. Similarly, the analysis of the parsing complexity seems to assume a lack of transclusion limits.
Unfortunately I wouldn't say it is very easy to use - the Wikipedia editor is much easier. It's also completely unclear which parts of Wikidata data are used in Wikipedia, and where.
That means the Wikidata data is usually much worse than the Wikipedia data, even if it theoretically easier for machines to use.
I'm not sure what you mean when you say that the editor of Wikidata isn't easy to use, I think its interface is pretty straightforward.
(English Wikipedians in general are very "territorial" in my experience, so to speak, and rarely appreciate efforts that put some piece of "their" articles beyond their "jurisdiction", like Wikidata does. Editors from other language versions are a lot more welcoming.)
I agree it's pretty easy to add a Freebase ID to a Wikidata ID. Unfortunately this isn't really that valuable, because it's easy to a machine to work it out too.
Eh, I can't really agree with this. VisualEditor rollout was indeed premature (critical features like template editing were developed literally days before it), but a year has passed and it's gone a long way since then. Haters still hate it, but personally I find it more pleasant to use than wikitext for many tasks.
This was a really interesting read, though. I still this there's value in the WYSIWYG paradigm, even if we have to fall back to "IDE mode" for infoboxes and such (right now if you double-click a template, you're shown a key-value table to fill in; there's no live preview…). After all, most of the content of Wikipedia is text with some light formatting.
Except they aren't. WYSIWYG is an abstraction over the code layer that works in the terms of "bold", "italics" and "underline".
Did you get a chance to check out fellow HN homepage link Paperman [1][2]? It offers exactly what you propose: Double-clicking on the results frame highlights the relevant line in the source, and vice versa. This could easily be adapted to match your suggestion.
[1] http://paperman.patricklorio.com/ [2] https://news.ycombinator.com/item?id=8507310
Basically, in future why assume we have one definitive repository of objective fact? The notion is ridiculous. What I believe will happen is a git-ification of Wikipedia, so you can fork the whole thing, run your own when you disagree with the direction, and pull in changes from those you trust. If you're going to attempt to fix this class of problems this is the only way forward.
http://www.foxnews.com/tech/2013/09/06/wikistorming-colleges...
I also considered the decline is that it's kinda 'complete' it's not like the old days where it was easy to contribute something important and meaningful.