A stroll through the archives of Editor & Publisher
niemanlab.org
niemanlab.org
Google actually scanned a huge number of newspaper microfilm and microfiche back in 2010-11: https://news.google.com/newspapers
It's an amazing resource, but it's hidden away and researchers within Google have trouble accessing it even if they knew it existed. I spent months with lawyers to get access to the original files for research and at one point they told me they were going to delete it! Whaaaaa!?!
It was something like 6 PB, which seriously, to Google isn't much, but the team that "owned" the data wasn't using it and to them it was just an expense. Ugh. People don't care about history.
(Each * here representing some unknown third party getting access to your email address, phone number, blood type and sexual preferences.)
New York State funded an effort to scan, but not digitize historical newspapers, and while the microfilm is stored in the state archives, the online versions are hosted by an eccentric guy who digitizes the microfilm as a hobby. The guy puts everything online, but makes it difficult to work with in a variety of ways.
According to the site, it specifically avoids "major" periodicals:
"A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media."
It's a century's worth of NYTimes front pages. An amazing long term dataset that is fun to flip through to answer infinite questions you might have about both how the last 100 years really went down and also about the evolution of presenting information to readers.
https://static01.nyt.com/images/<year>/<month>/<day>/nytfrontpage/scan.pdf
If you do find more, please let me know.
I was hoping scientific American or wsj etc would have them but couldn’t find any.
The Washington Post was recently caught scrubbing an unflattering quote from a 2019 profile of Kamala Harris:
https://reason.com/2021/01/22/the-washington-post-memory-hol...
Personally I had not seen anything from him stating he believed the security holes were fixed on a broad scale, or on any scale. But his recent video in November 2020 suggests he no longer had any substantial concerns regarding voting machine security: https://www.youtube.com/watch?v=cMz_sTgoydQ&t=521s
For COVID, back then, it was "tech bros are afraid of the flu (and maybe also racist)" and then "go hug a Chinese person", and then "closing flights from Wuhan is racist (even though China itself was doing it domestically)". It was funny to see the escalating tsunami of wrongness coming at the media, but of course, they would never admit wrong-doing.
For Kamala Harris, well she was (is?) hugely unpopular. She has terrible political baggage, has been personally responsible for terrible systemic racism, and the news coverage reflected that quite accurately up to a very specific point. Tulsi torpedoed her early on in the debates by bringing all this up.
However, when Biden said his VP choice would be a woman of color, there was an immediately 180 in coverage and retroactive editing to make it look like she was an exceptional candidate who has always championed racial issues and is a regular everyday person just like all of us.
That said, when you drop out of a race or an election completes, the oppo stops. That's not really a surprising change.
This is a myth. Wuhan Tianhe International Airport was completely shut down. Domestic and international flights stopped at the same time.
Niall Ferguson of the Hoover Institution falsely claimed that China kept allowing international flights to keep taking off after it stopped domestic flights, and his claim has since gone viral. Even Trump has repeated it several times on national television.
The irony is that the airport was shut down so quickly that foreign governments didn't have time to get their citizens out of Wuhan. They had to negotiate with the Chinese government to allow specially chartered evacuation flights to take off from the city. But Niall Ferguson looked at a flight tracker that showed scheduled (not actual) flights, and concluded that international flights were still taking off from Wuhan, and then wrote an Op-Ed about it. The myth has never died, despite repeated debunking.
Is that material still available anywhere? It was really extensive IIRC.
https://news.google.com/newspapers
It's fun to browse but I'm sad that the project seemed to just sputter out. At one time I thought it would grow to make historical newspapers searchable with coverage comparable to the books searchable through Google Books.
Sad they haven't made more of it really. Thanks for the pointer!
Alexa APIs, incidentally, were one of the first AWS products along with ECS (e-commerce service) and SQS in 2004.
As a side note, I'm loving the 90s tabloid layouts.
Well said, agreed! I should add Wikipedia.
At a first glance, they seem reasonable.
- 43% Direct support to websites. Keeping the Wikimedia websites online is about more than just servers. It also includes ongoing engineering improvements, product development, design and research, and legal support.
- 32% Direct support to communities. The Wikimedia projects exist thanks to the communities that create and maintain them. We strengthen these communities through grants, projects, trainings, tools to augment contributor capacity, and support for the legal defense of editors.
- 32% Direct support to communities The Wikimedia projects exist thanks to the communities that create and maintain them. We strengthen these communities through grants, projects, trainings, tools to augment contributor capacity, and support for the legal defense of editors.
- 13% Administration and governance. We manage funds and resources responsibly to recruit and support skilled, passionate staff who advance our communities and values.
- 12% Fundraising. Wikimedia is sustained by donations. Millions of remarkable individuals and institutions ensure that we have the necessary resources to continue our global mission.
[1] https://wikimediafoundation.org/support/where-your-money-goe...
As for archive.org they offer basically no transparency as to which sites are excluded from their archiving (and as far as I know they will remove all the content of a site once the owner asks them to).
"years and years ago", this was in 2019
The running one of the biggest technical and community concerns on the planet takes a lot more then serving content from a computer somewhere. Salaries, community management, improvements to the wiki software—these things cost money too and are no less important then responding to HTTP requests.
But lighting up an empty building doesn't turn into Wikipedia.
Most internet organizations do that (free, charity or commercial). Good people are expensive. You need sysadmins, programmers, some graphics artist, probably more than a few lawyers etc.
And I wouldn't want them to use the cheapest, possible people that they can find for those roles. And asking good people to work for free or cheap, is just as shitty.
And a lot of free software and opensource foundation spend most of the money organizing conferences, so that people can meet in person and have presentations and working groups etc.. But when Wikipedia does the same its somehow wrong ?
They already do that with their editors.
> And a lot of free software and opensource foundation spend most of the money organizing conferences
I would avoid donating to any such organisation myself.
Are you now pivoting to saying Wikipedia doesn't spend enough rewarding contributors, from having previously complained about having a celebration with contributors in Accra?
I strongly encourage you to go to one of the events run by the kind of organisation you're complaining about. I'm not exaggerating when I say Mozfest is life changing for many people. Open Knowledge Foundation events have been career-defining for me and many people I've mentored in the UK, Kenya and South Africa. I've never been to a Wikipedia event but I know about them and I think you're imagining something vastly different than what happens. These are events that are about building communities, networks, and opportunities for open collaboration. If you don't like how these organisations do it, sorry, because they are the ones doing it successfully.
They are volunteers. They do as much works as they want or don't want to do. Once you are the size of the Wikipedia there are tasks that need to be done, and you can't really relay on volunteers to do it in their free time.
> I would avoid donating to any such organisation myself.
Its your money.
I am just saying that its normal and expected in free and opensource communities to do that sort of things. Organize conferences, meetups, public awareness, handle the legal stuff and hire the core stuff that makes sure thing are running 24/7.
The Internet Archive has proved itself so many times over, and is so underfunded, that it deserves all the money it can possibly raise - which will allow them to solve more of the problems they face.
> and managing that community takes a lot of the resources, obviously
I do not think that parties in Accra using donation funds is an integral part of "managing that community".
The Internet Archive has proved that they are not to be trusted multiple times. Such as when they decided to not be transparent, to retroactively remove content if asked by the site owner or if robots.txt started banning crawlers, when they started "lending" e-books with DRM, or when they decided to do the whole "National Emergency Library" thing and publishers sue it over it (the lawers and possibly said publishers if they win the ruling - which is likely - will be paid via the donation money).
Well, that's why the opinion is unhelpful.
Wikipedia runs the largest decentralised and communal knowledge curation project in history. Making it and keeping it great takes a lot of free labour, all over the world. They have to organise communities at many many different levels, and often that involves getting a bunch of people in a room together - people who aren't getting paid - for a few days to make a ton of decisions and design and implement things. It involves recruiting new people to help and teaching them the organisational processes and skills, and nurturing new people up through the organisation, and resolving comflicts. All of that is absolutely necessary to make something like wikipedia work and barely scratches the surface.
A party for the people who worked their asses off for free is really an incredibly cheap way to reward people.
As for whether parties are important for wikipedia, anyone interested in this topic can read the links that I posted earlier. There was a debate whether they are helpful or not if I remember correctly.
Many wikipedians will disagree with you.
> They produce something incredible
The volunteers do.
> But it's not helpful for you to encourage others to do that.
Again, my desire was to inform people as to not regret their decision to donate, unlike me.
These are the events you're talking about. They hardly seem to be lavish events.
https://meta.wikimedia.org/wiki/Wikipedia_20/Events/OFWA
https://meta.wikimedia.org/wiki/Wikipedia_20/Events/Ghana
These are people working to enhance the encyclopedia. Here's one example: https://twitter.com/NanaYawBotar/status/1356678087560343567?...
Parties serve a purpose. We are social mammals.
If everyone who works for a charity has to work for at most minimum wage, then I don't think sites like Wikipedia or the Internet Archive would be as high quality as they are.
I doubt they'd do as well fundraising if your average donator knew that Wikimedia was pulling in $100m a year, is paying it's directors and above $200-400k a year, and the vast majority of their dollars are not spent on "servers and power" like their ads imply.
Personally, I'm fine with their budget, but I do wonder if their fundraisers are as ethical as they could be.
Honestly you should compare what WMP pays with other similar sized global organisations.
They should choose between the donors and the billionaires then. (regardless, they are a tech company, they could employ people remotely)
This still does not address the issue of the dishonest fundraising though (pretending that they need your help despite making 1/10th of their wage and not saying up-front on what they spend the money on). I am sure that you would be upset if you gave money to a beggar that you later see getting in his Mercedes, or if you donated money to an organization for the homeless but then realized that 90% of the money goes for researching new pie recipes. It honestly feels like that.
Why? They have absolutely no reason to do that. They have a mission, they are raising money for that mission, and doing so very successfully.
> (regardless, they are a tech company, they could employ people remotely)
Could, and they know that, and they make hiring decisions based on their expertise and knowledge, and have deciided the organisation is better off hiring some people in SF.
> This still does not address the issue of the dishonest fundraising though (pretending that they need your help despite making 1/10th of their wage and not saying up-front on what they spend the money on)
I understand that you had a misunderstanding about what wikipedia is, how their community management works, and that you didn't make use of the public, publicised resources that make their spending completely transparent. That doesn't make them dishonest, it makes you lazy.
From the banners one might conclude that they are facing imminent shut down and getting by in an unheated offices.
no way I'm giving them money especially after reading the threads here and people attack h_anna for exposing the truth.
Nobody has exposed anything, these are all public facts published and actively publicised by Wikipedia itself. They literally actively try to expand participation in local groups, and small parties for the people who gave their time is a gesture of thanks to those people, not some ostentatious overspend.
six digit salaries is small to you?
For the people running an organisation the size and importance of wikipedia? 6 figures is a laughable minimum. The legal and social responsibility, the vast and varied expertise required, the personal social networks you put on the line. We need the best people in those jobs to keep that organisation existing at all, let alone functioning at the level is has and does. Recruiting those people is competing against the rest of the world to hire them, and they are already taking a huge cut in remuneration because they want to do good.
As a former Internet Archive employee, but not speaking for them here:
That's inconsistent. If the Archive were as meek about deferring to traditional-rightsholder supremacy as you want with regard to the "Emergency Library" of digital books, the web archive might not exist at all, or could only include material with explicit prior permissions – shrinking it to a tiny fraction of its size.
When the Archive started crawling & storing websites, there was no clear legal right to do so. (The 1996 DMCA, which if read a certain way, immunizes some such activities as "caching", wasn't even law when the IA started in 1995 - but its immunity also requires the prompt retroactive removals you find objectionable!)
There was, from the start, a colorable argument, based on 'fair use' & the historical role & respect given libraries by our law & culture, that this should be legal, and could be legal if the facts were interpreted a certain way.
But on the letter-of-the-law, there was an immense risk rightsholders could sue - like the publishers have now with regard to pandemic book-lending.
It was only by demonstrating the immense value of such an archive, & repeatedly making the case for its legitimacy via such demonstration and reasoning in courts & legislatures & culture, that the right of such an archive to exist has now been firmly established. The fact that lawyers & courts themselves have found it so essential has been part of the success. And, now that it is a familiar activity, it continues to gain reinforcement by being assumed-as-legitimate when new laws/policies are drafted, because those now avoid language that could be inadvertently interpreted to prohibit something obviously good & existing.
Still, having an automated exclusion procedure (via 'robots.txt'), and generally respecting credible rightsholder takedown requests, is essential to capping the legal risks of such a large archive.
And the Archive's approach on book-lending, including during the "Emergency Library" program, has been broadly similar. An urgent need & technological opportunity arose before laws & explicit "ask first" processes could accommodate the situation. Culture & precedent suggested extraordinary, but temporary, adaptation could plausibly be legal & would be net-beneficial for society. (Which is better: no access to library-loaned physical books while in 'lockdown', or tracked time-limited access to temporarily-created digital copies, smaller-in-number than the count locked in closed libraries? Does technological format-shifting in response to an emergency, with minimal impact on rightsholders' revenues, fit 'fair use'?)
Still, the legal risks were capped by setting the program to be of limited duration, & having a policy of respecting any explicit book-exclusion requests from rightsholders.
If you're really so afraid of rightsholder damages judgements, sure, the Archive isn't your best donation target. (I think the risks to the Internet Archive from the latest lawsuit are limited to having a bad precedent established, not bankruptcy.) But know that every historical website you can access is there because the Archive was willing to take some risks establishing new rights/precedents, & your preferred policy of fewer-exclusions would mean yet more legal risks. And lots of people donate to good causes specifically so that they can defend themselves, legally, or push new cases, legally, for broader benefits.
> Still, having an automated exclusion procedure (via 'robots.txt')
I am talking about retroactive removal, not exclusion. That being said I heard that they recently stopped doing that.
> If the Archive were as meek about deferring to traditional-rightsholder supremacy as you want with regard to the "Emergency Library" of digital books
Another organization could be created for this. By having IA do this it sets at risk the rest of the archive (as well as the donations given to it).
> Which is better
Not supporting DRM is.
Thanks for bringing up that thread. I have a lot of reservations about giving money away to such "non-profit" organizations before but I get an idea of whats going on behind the scenes now.
I certainly will no longer be donating to the EFF and Wiki organizations knowing non-profit organizations possibly funding parties and luxurious corporate life styles. I want the money I give them to stretch as much as possible. I'm not against parties either but when you spend millions citing conferences and travel (even some large for-profits dont spend this type of money) citing you are non-profit to mismanage my money then I simply won't stand for it.
My trust in local non-profit organizations were already pretty shaky but I somehow trusted these large organizations because I thought they would be more frugal and we would have more transparency. I was clearly wrong.
You simply have no idea how the money is being spent and its very difficult to dictate how they should better allocate resources either.
Apparently most people think that frugality would be the default in these organizations but once you give them a credit card, they won't think twice about over spending on stuff that has marginal benefits like luxury company cars, chartered planes, expensive dinners etc.
I will still donate to causes I believe in but I am going to now ask for receipts and will enforce strict frugality. Parties should be limited to the office with a dozen Dominos pizzas and everybody should bring their own soft drinks & cups from now on.
I like them, but they seriously need a change in management.
They opened themselves up to millions of dollars (if not billions of dollars) of liability for no good reason. (Covid doesn't give you a pass to give away someone else's property without permission.)
Similar to you, I'm not going to donate to an org that is so reckless with donations. If they would admit it was an error in judgement and come to a quick settlement I might become a donor again.
Even still, the damage has been done -- it'll take a long time for publishers to trust the Internet Archive again.