Vital Wikipedia Articles | Hacker News Reader

Vital Wikipedia Articles

en.wikipedia.org

436 pointssoheilpro77 comments

Open articleView on HN

I would argue that List of Common Misconceptions[0] belongs in any list of important or vital Wikipedia-articles.

[0] https://en.wikipedia.org/wiki/List_of_common_misconceptions

You're not alone in arguing for the importance of this page :) https://m.xkcd.com/843/

Ugh, the world has me figured out, but at least it approves.

obligatory link to lesswrong: https://www.lesswrong.com/

Second only to the List of Cognitive Biases:

https://en.wikipedia.org/wiki/List_of_cognitive_biases

really... you think you know them all but you don't. Everyone needs to understand the IKEA effect for example

The problem I have with these sort of lists is that they try to be comprehensive and add a lot of second order biases that tend to drown the basic/ fundamental ones. Take your example for IKEA effect: it is in fact a consequence of loss aversion. Once you invest emotion, time or money into something, you assign more value to it, because you do not want "lose" your initial investment.

Is that again a consequence of "sunk cost" fallacy?

Keep in mind that certain biases are censored against e.g. Apex Fallacy.

The Wikipedia page on that was deleted due to a lack of reliable sources (https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...). There still seem to be no reliable sources. (https://scholar.google.com/scholar?hl=en&as_sdt=0%2C6&q=%22a...)

I'll admit I don't see the point of many of the other cognitive biases, but at least they were coined or studied in some peer-reviewed papers.

I can't believe there's a wikipedia page consisting solely of things that I bring up during party conversations to seem interesting. Thanks for showing me this.

This list is awesome. Along with [1], it could make a great starting point for learning about new topics but also quickly gaining knowledge on what topics are considered important in a discipline. For example, if you want to learn about philosophy but don't have a good understanding of what topics are even available, just scroll down in the list!

I think it could also serve as a list for topics a well-rounded person should know something about (of course, this is highly personal / debatable etc.). Or at least heard of. Perfect way to learn something every day!

Great submission, bookmarked, thanks!

[1] https://meta.wikimedia.org/wiki/List_of_articles_every_Wikip...

Totally agree. I just read Earth [1] for the first time and it was really fascinating. Don't forget to click on 'Level 1' in the top bar as well, to see the top 10 most important articles.

[1] https://en.wikipedia.org/wiki/Earth

I had a different reaction. I understand the reasons for having this sort of list internally, but some of what's included and left out is really odd to me, especially when it comes to biographical entries. I worry that these lists will kind of reify a very superficial approach to certain areas.

I'm not sure how they came up with these lists, but it would seem better to me to somehow quantitatively organize them, by numbers of edits or some index of controversy or something. That way there would be a more direct relationship with the reason for having the list in the first place.

As it is these lists remind me a lot of the controversies over Wikipedia when they started giving editors more and more power. It seems to reflect some preconceptions on the part of the Wikipedia editors more than anything else.

First, I don't understand what you mean by "having this sort of list internally". Wikipedia is completely open, so what does "internally" even mean?

Second, I don't understand your desire. Do you want more controversial topics? The goal of an encyclopedia is not to list/explain controversial topics[+], or am I mistaken somehow?

Third, to be concrete: who should come off the list and who should be on the list if you don't agree?

Fourth: if you don't agree on the list and can't come up with replacements, give us exactly the definitions that should be used to warrant inclusion. My guess is that these definitions / rules / decisions are so difficult to make in the first place, that agreeing on those rules is as difficult as agreeing on the people / topics that are included in the list.

[+] Definitely not to mark them as somehow more important than other subjects that are less controversial.

1. By "internally" I meant primarily for the purpose of Wikipedia itself, as opposed to closed. I admit the choice of term was poor or misleading.

2. I don't want more controversial topics, I just meant that I can see how Wikipedia would want to track or flag topics that are especially controversial. Controversiality (if that's even a thing) is just an example. I just meant I could see how Wikipedia would want to flag topics based on some quantitative criterion.

3. That's a big topic and I'd rather not delve into it. Some might be on such a list if I made one, but others would not, and still others that aren't on the list would be there. I think my underlying concern is that there shouldn't be a list at all. If it's important enough to be on Wikipedia, it is important.

4. Again, I wouldn't have a list of "importance." I would have a discussion of which topics should be flagged, and why--what the criteria are for deciding something should be flagged--and flag based on those criteria. My sense is it should be based on something objective and quantifiable.

Well it's impossible to have a list of any '1000 best of' that will be agreed by everyone. If you want to delve into the subject, it was a puplar subject in literature some time ago: https://en.wikipedia.org/wiki/Western_canon#Debate

I know it's possible to download the entire Wikipedia database, but does anyone know of a way to download every article in this list? Preferably as a torrent.

Not a torrent or a full solution but applying the regex /wiki/(?!.\:)[aA-zZ0-9%()_] on the source should select all the articles (along with some generic wikipedia links matched at the bottom), then batch adding "https://en.wikipedia.org" to the beginning of each line gives full urls.

Here's one such list: https://hastebin.com/terugezeda

wget has an option (-i) to download links line-by-line from a text file but is sadly making a mess of the images, using

  wget --span-hosts --convert-links --adjust-extension --page-requisites --no-host-directories --no-parent --wait=1 --reject="robots.txt" -i wget.txt

  wget -H -k -E -p -nH -np -w 1 -R "robots.txt" -i wget.txt

for short.

Maybe someone has a better idea for the last step

edit: shorthand version

I'd recommend inliner for the last step: https://www.npmjs.com/package/inliner

I was about to say that a torrent is hardly necessary. How big could a 1000 mostly-text files get? Pretty big as it turns out. Downloading a dozen random entries from that list, the sizes seem average around 2 MB, and that's including only the small images on each page (not the big picture you get when you click on an image). So 1000 entries at 2 MB each would be 2 GB.

Picking apart just one page (the Jane Austen entry), the plain ASCII text with no markup is only 88 KB. The 19 small images, plus some tiny buttons and logos, are 536 KB, and the markup (HTML, CSS, and whatnot) is 497 KB. I was surprised that Wikipedia, in terms of page weight, is mostly images and markup. (Not complaining, of course. Wikipedia is one of the few big sites on the web that doesn't throw in gratuitous and irrelevant images and videos.)

Are you including page history? Are you allowed to re-distribute without page history?

Also, you probably know this but there is a prize for compressing bits of Wikipedia. Here's the wikipedia page about it: https://en.wikipedia.org/wiki/Hutter_Prize

http://prize.hutter1.net/

About the test data: http://mattmahoney.net/dc/textdata.html

And discussion on HN: https://news.ycombinator.com/item?id=7405129

I can offer one option that's in the ballpark of what you're looking for:

https://wiki.kiwix.org/wiki/Content_in_all_languages

Kiwix is a standalone viewer (Available as desktop, phone or web server [including a Sandstorm.io package which I put together]) for archived websites of various sorts, Wikipedia being the flagship. This link lists all available things, and it's countless. However if you ctrl-f for "physics" (and keep searching until you hit the language you want) you'll see that they have subsets of Wikipedia available that cater to many interests. Physics, basketball, "for schools", history, etc.

All content packages are indeed available as torrents.

Please use the API. DO not scrape wikipedia via the website.

What you're looking for is:

https://en.wikipedia.org/wiki/Special:Export

You can start with the index page and collect all the page titles you're interested in, and then use the special:export API to download XML (probably other formats too) of all those pages.

Level 5 has 31 video game designers (under Artists, musicians, and composers -> Game and toy designers), including Shigeru Miyamoto as the only one also included in Level 4 (under Businesspeople). (Mario is the only fictional character from games included in Level 4.)

Notable omissions: Richard Stallman (Linus Torvalds is in), GNU (Linux is in), GPL, Free Software (Open-source software is in Level 4), Rust (Assembly, C, Java and Javascript are even in Level 4), Deep Learning, Hacker News (Reddit is in).

> Rust (Assembly, C, Java and Javascript are even in Level 4)

The languages in parentheses basically run the world, Rust is barely used in comparison, it's not surprising it doesn't get the same attention.

>GNU, GPL, Free Software

I love these things as much as anyone else on HN but realistically they are just semantic differences between already included pages. A line has to be drawn somewhere, and since they are mostly just minor differences (which, to be fair, have large implications... but the differences themselves are still quite minor), I don't think it's fair to say they should be included.

>Hacker news (Reddit is in)

Isn't reddit one of the top 10 websites by DAU? HN is intentionally niche.

Is is possible to see these ratings/labels/classes on an article? For instance, can I tell https://en.wikipedia.org/wiki/Charles_Dickens is B-class from the page itself? I assume the lock icon at the top right may have some correlation with the assessed quality, but it doesn't seem consistent across classes.

Article ratings below "Good Article" are considered only relevant to the coordination of Wikipedia writers and editors, so these are only shown in the top templates in its Talk Page.

For example,

https://en.wikipedia.org/wiki/Talk:Charles_Dickens

You'll see:

--- This article is of interest to the following WikiProjects:

* WikiProject Biography / Arts and Entertainment (Rated B-class)

* WikiProject England (Rated B-class, Top-importance)

* WikiProject Children's literature (Rated B-class, Top-importance)

* WikiProject Hampshire (Rated B-class, Top-importance)

* WikiProject Journalism (Rated B-class, High-importance)

* WikiProject London (Rated B-class, Top-importance)

* WikiProject Kent (Rated B-class, Top-importance)

--- This article has been reviewed by the Version 1.0 Editorial Team.

* C - This article has been rated as C-Class on the quality scale.

* B? -

This article has not yet been checked against the criteria for B-Class status:

Referencing and citation: not checked

Coverage and accuracy: not checked

Structure: not checked

Grammar and style: not checked

Supporting materials: not checked

Accessibility: not checked

To fill out this checklist, please add the following code to the template call assessing the article against each criterion.

* High - This article has been rated as High-importance on the importance scale.

The article's class is on its talk page.[1]

If you have an account and don't want to leave the article to find out the class, you could enable the metadata gadget. [2]

[1] https://en.wikipedia.org/wiki/Talk:Charles_Dickens [2] https://en.wikipedia.org/wiki/Wikipedia:Metadata_gadget

The Gadgets section has an Appearance setting to "display an assessment of an article's quality in its page header".

https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsec...

The lock icon is a different issue, it is used in a subset of controversial or high-profile articles to limit the types of editors that can edit it (from requiring editors to be logged-in to a temporary full edit block during disputes).

Well there goes my productivity for the day.

Superb list. Should read ! A great example of a « knowledge tree », could be useful to find a root concept for an idea you want to explore. Any other tool to find « first-principle » roots of any topic ?

I wonder how these pages change in content and tone across languages..

Level 1 with 10 articles is available in 32 languages. There are some differences among the languages I checked, but they are mostly just using different representative articles for the same general categories. E.g. French has "culture" instead of "human" and Chinese has "culture" instead of "philosophy" and Catalan has "geography" instead of "Earth" and "society" instead of "human".

What does the process of selecting these articles look like?

Generally you can find this kind of stuff in the Talk pages for articles.

https://en.wikipedia.org/wiki/Wikipedia_talk:Vital_articles#...

See “What makes an article vital?” and “How are articles selected?”

https://en.wikipedia.org/wiki/Wikipedia_talk:Vital_articles/...

Surprisingly ”University” is only level four. I’d say it definitely deserves a place on level three, being not just a subconcept of ”school”, but an institution responsible for the majority of scientific research.

Edit: Okay, these are definitely the weirdest downvotes I’ve ever got on HN.

I guess it is pointless to argue about what topics are the most relevant. This "vital articles" list seems completely arbitraty and, frankly, not very important.

So few A-class articles in the list.

One problem there is Wikipedia culture. Suppose you're a bona-fide content-area expert, fully capable of accurately conveying your knowledge in clear language that most people can understand. You might want to concentrate on creating the article.

Instead ... if the topic is not obscure and unpopular ... odds are high that you'll wind up jousting with dozens of inspired but unqualified challengers. The first time two of the paragraphs you've spent hours crafting disappear on someone's whim, you're probably going to stay gone.

"Anyone can edit"? In many cases, no. Take the 'Start-class' article 'performing arts', for example: how many people are truly qualified to summarize that enormous topic?

A-quality doesn't emerge from a melange of bits and pieces.

What I want to emphasize is that majority of articles represent western democratic, capitalistic, liberal culture. Reading these articles you won’t learn anything extra that you were tought in school. On one hand it is ammusig how english wikipedia became a western culture mirror. On the other hand it is sad that you can not get insights into other cultures without western culture filter.

>...it is sad that you can not get insights into other cultures without western culture filter.

I don't understand this sentiment. Why can't you? There are quite a few other entry points out there created by non-westerners; why complain that an index created by westerners for the English Wikipedia has a western bend? How could it conceivably be different?

They could include Lewis Dartnell

wikipedia is now under the effective control of limited number of entrenched editors, mostly subscribing to western establishment's "liberal" ideology, with almost absolute power over content. a prime example of this bias, is the article about british empire. comparison of that article with articles about other brutish regimes(ussr, mao's china, etc) is telling, even though it's atrocities far exceeds any other regime, in terms of both quantity and extent.

While I agree with that assessment, I don't really think it has much to do with Wikipedia itself, it's just how things are in the entire Western culture. History is written by the victors and all that.

Consequently, it feels a bit unfair to put this on Wikipedia editors. Kinda like blaming a random restaurant manager for forcing waiters to rely on tips. Yes, if they have enough profits, they can pay people more and fix a small part of the problem. The general issue remains systemic though.

> Consequently, it feels a bit unfair to put this on Wikipedia editors. Kinda like blaming a random restaurant manager for forcing waiters to rely on tips. Yes, if they have enough profits, they can pay people more and fix a small part of the problem. The general issue remains systemic though.

Systemic yet oddly restricted to a particular area.

Can you provide some links to said atrocities by the British empire? (I'm genuinely curios)

There probably are some Wikipedia articles you could link to.

Edit: just a few examples

* https://en.wikipedia.org/wiki/British_concentration_camps

* https://en.wikipedia.org/wiki/Famine_in_India

* https://en.wikipedia.org/wiki/Slavery_in_Britain#Enslaved_Af...

I was looking this up just now. It seems 29 million died in India alone under British rule due to famine. [1] However there was widespread crop failure in the late 19th century. [2] I don't know how much control they had over what and how every set of deaths occurred, so it's not yet clear to me you can just directly compare number of deaths between empires without taking into account the surrounding context.

[1] https://www.quora.com/How-many-people-did-the-British-empire...

[2] https://en.wikipedia.org/wiki/British_Empire#East_India_Comp...

The Bengal Famine is a well-studied case where, AIUI, the current consensus is that imperial policies contributed significantly to the suffering and death toll:

https://en.wikipedia.org/wiki/Bengal_famine_of_1943

While it wouldn't be wise to attribute all those deaths to the British Empire, it's important to note that British agricultural practices led to widespread famines and crop-failures. Diversion of food crop land to grow Opium and Indigo led to famines[1]. And the risk of crop-failures was often beared by the cultivator[2].

1. https://en.m.wikipedia.org/wiki/Irrigation_in_India#History

2. http://barrett.dyson.cornell.edu/NEUDC/paper_364.pdf

>so it's not yet clear to me you can just directly compare number of deaths between empires without taking into account the surrounding context.

This is true, but perhaps in a way you weren't anticipating; this point was made decades ago by Herbert Marcuse in response to people attempting to compare, for instance, Nazi death tolls with Soviet ones. This mode of argumentation reduces concrete matters of policy and motivation (quality) into mere numbers to be thrown about (quantity). But authoritarianism is not a quantitative matter - a nation with harsh laws under which only few are convicted is still an authoritarian nation.

And yet such nuance is never afforded to the other empires mentioned in GP's comment. "Communism killed 100 million" etc. being widely held views.

This is a good start:

https://www.independent.co.uk/news/uk/home-news/worst-atroci...

As an Australian, this came to mind.

100 years of massacring Indigenous Australians.

https://en.wikipedia.org/wiki/List_of_massacres_of_Indigenou...

This is probably one of the best examples of modern white-washing of our history ..

I did not get your comment and I would like to understand it better. Isn't "whitewashing" the process of forgetting about the contributions or tribulations of non-white people in our history? Acknowledging atrocities committed by western civilizations seems like the opposite of that (and I do think we should hold ourselves to a very high standard and being aware of these atrocities is part of that).

What I mean is that Australians sweep the details under the rug about the genocide of the First Nation people that occurred in order to make their modern nation .. its not at all near as well acknowledged as it should be, and Australia continues to build tourist traps on sites of mass murder as if its nothing.

Australians prefer ignorance over acknowledgement, generally. Its not something we should be proud about .. in fact, there is a lot of shame in being an Australian with NO CLUE about the civilization that was swept away to make way for White Australia. Australia's first inhabitants were far more advanced than we care to admit.

EDIT: some things all Australians should know about the first people:

* They have been on the continent for at least 65,000 years * The current biosphere is a result of their management of the land and wildlife for the duration of those 65,000 years * They used cryptographic techniques to keep the peace between tribes * They have the oldest confirmed oral history of any human society, ever - scientifically proven to stretch back 40,000 years. (We in the west can't keep our oral traditions sane for 100 years, by comparison..) * They had a version of the Hippocratic Oath long before European medicine * Their medical culture had an understanding of antibiotic and antiseptic materials in their environment and were using them to great effect at a time when European medicine still believed in miasma theory

.. and there is much, much more. Alas, the hubris around what has been done to the First Nation people (see for example White Australia policies continuing into the 1980's... forced castrations... forced re-settlement of children... etc.) has resulted in a general ignorance of the situation among current Australian culture...

Of particular recent relevance are the opium trade and the opium wars, which led to the creation of Hong Kong.

https://en.m.wikipedia.org/wiki/Opium_Wars

Whenever Wikipedia is mentioned in hackernews, it's always the same comment. Have you got anything new to add, or any substantial argument to support your position? As it stands it's a pretty serious accusation and a pretty thin argument.

I've rarely seen this comment on hackernews about wikipedia. I think you may have been thrown off by the use of "liberal" to think it's a right-wing critique. Incidentally, the complaints I'm used to are that "they keep deleting the thing that I'm involved with that I think is notable" "they won't let me edit myself even though I'm an expert on myself" and "weird editors squat articles."

This is a "wikipedia is completely western and whiggish" critique.

Do you mean to imply that "conservative" western ideology would be more truthful about British empire?

That's not what GP means by "liberal"; "liberal" in this sense refers to the belief and support of the current "democratic" establishment, the right to private property, the idea of human rights, and freedoms defined as matters of separation between people (i.e. the assumption of the person or people attempting to take them away rather than a positive conception in human community). In this view, both US 'liberals' and US 'conservatives' are both liberal. Thus, what GP means by 'non-liberal' includes the critics of liberalism, most notably ones on the left.

You can read more about the philosophical notion of liberalism and its critics at the Stanford Encyclopedia of Philosophy: https://plato.stanford.edu/entries/liberalism/

Thank you.

I think you mean “English-language Wikipedia”. Try switching to one of the other languages and then putting the article through Google Translate. They’re not the same article semantically! Each language has its own set of editors putting its own spin on things. It’s only to be expected that each language’s editorial policy will be dominated by the cultural hegemony of that language’s speakers, if one exists.

(For example, even without government interference, you would expect “Taiwan” to have different first-sentence descriptions in the Chinese–Simplified, Chinese–Traditional, and English Wikipedias.)

> For example, even without government interference, you would expect “Taiwan” to have different first-sentence descriptions in the Chinese–Simplified, Chinese–Traditional, and English Wikipedias.)

I agree with your sentiment, but that's not the best example. There is one Chinese Wikipedia, and articles are machine-transliterated between simplified and traditional forms (plus a few more forms using region-specific vocabulary).

Well, besides Mandarin there are also articles about Taiwan in Cantonese, Eastern Min, Gan, Hakka, Literary Chinese, Southern Min, and Wu. They all seem to be able to agree that it's an island in the Pacific, but differ in the degree to which they mention that the Republic of China controls it. The Hakka article prominently displays the national flag, while the Southern Min one doesn't mention it in the introductory paragraph.

At least one Wikipedia should maintain close to facts approach. If you prefer authoritarian "rewrite history" approach then you can consult Russian Wiki, where they rewrote all political and history pages, and I assume Chinese Wiki is the same.

you can consult Russian Wiki, where they rewrote all political and history pages

Can you link some specific examples?

En: https://en.wikipedia.org/wiki/War_in_Donbass Ru: https://ru.wikipedia.org/wiki/%D0%92%D0%BE%D0%BE%D1%80%D1%83...

En: https://en.wikipedia.org/wiki/Russian_military_intervention_... Ru: none, lol

En: https://en.wikipedia.org/wiki/Annexation_of_Crimea_by_the_Ru... Ru: https://ru.wikipedia.org/wiki/%D0%9F%D1%80%D0%B8%D1%81%D0%BE...

En: https://en.wikipedia.org/wiki/Russo-Georgian_War Ru: https://ru.wikipedia.org/wiki/%D0%92%D0%BE%D0%BE%D1%80%D1%83...

For several years they vandalized Kievan Rus' page (in Ru segment) renaming it to Ancient Rus'. For some reason they stopped doing it now , but all changes are still in history. https://ru.wikipedia.org/wiki/%D0%9A%D0%B8%D0%B5%D0%B2%D1%81...

En: https://en.wikipedia.org/wiki/Malaysia_Airlines_Flight_17 Ru: https://ru.wikipedia.org/wiki/Катастрофа_Boeing_777_в_Донецк...

Basically any mention of unfavorable or outright bad mention about current country of a dictator or about any preceding country from which they claim inheritance is censored or obfuscated or buried in a flood of "alternative" opinions.

1) Some freak repeatedly vandalized a Wikipedia page. Editors reverted the vandalism, perpetrator stopped trying. If I understood all of this correctly, why is it not simply Wikipedia working as intended?

2) So, Russia and USA & Britain are involved in some recent and ongoing armed conflicts (all of them near Russian borders, I'd note). There's some difference in coverage of said conflicts between Russian and English Wikipedias, sure (though I skimmed the linked articles and didn't immediately notice blatant censorship on either side). And you cite that as Russians rewriting "all political and history pages"? Seriously?

3) The "lol" one is the coolest. "Ukrainian armed forces: 232,000 … 15,000 defected to Russia." Didn't see the figure before. If Russian Wikipedia was a worthy piece of pro-Russian propaganda, of course it would also have this figure in a prominent box. Alas, Russian Wikipedia is worse than English one at almost anything, including pro-Russian propaganda.

“Adolph Hitler” is the 6th most visited article in the last 90 days. Wow.

I guess human drama needs a bad guy.

That article immediately follows World War II and World War I and D-Day was June 6.

I'm surprised by William Shakespeare coming in at #16.

> Vital articles sorted by number of views in the past 90 days (as of 17 July 2014)

Sex is the eight.

I don't see anything wrong with that :)

The list seems awesome, I'll definitely go through it all.

Reply on news.ycombinator.com