The “Million Dollar Homepage” as a Decaying Digital Artifact
lil.law.harvard.edu
lil.law.harvard.edu
I can't currently access the article at https://lil.law.harvard.edu/blog/2017/07/21/a-million-squand...
[Insert joke about irony here.]
Looking at the million dollar homepage, many of the links were never valid:
http://paid & reserved/
http:// paid and reserved - accent designer clothing/
http://reserved for edna moran/
http://paid & reserved for paul tarquinio/ (1200 pixels)
http://pending order/
These links are all shown in plain red ("link to unreachable or entirely empty pages") in the "visualization of link rot," so it looks like the authors didn't account for invalid URLs.
>In a 2003 experiment, Fetterly et al. discovered that about one link out of every 200 disappeared each week from the Internet. McCown et al 2005 discovered that half of the URLs cited in D-Lib Magazine articles were no longer accessible 10 years after publication [the irony!], and other studies have shown link rot in academic literature to be even worse (Spinellis, 2003, Lawrence et al., 2001). Nelson and Allen (2002) examined link rot in digital libraries and found that about 3% of the objects were no longer accessible after one year. Bruce Schneier remarks that one friend experienced 50% linkrot in one of his pages over less than 9 years (not that the situation was any better in 1998), and that his own blog posts link to news articles that go dead in days2; Vitorio checks bookmarks from 1997, finding that hand-checking indicates a total link rot of 91% with only half of the dead available in sources like the Internet Archive; the Internet Archive itself has estimated the average lifespan of a Web page at 100 days. A Science study looked at articles in prestigious journals; they didn’t use many Internet links, but when they did, 2 years later ~13% were dead3. The French company Linterweb studied external links on the French Wikipedia before setting up their cache of French external links, and found - back in 2008 - already 5% were dead. (The English Wikipedia has seen a 2010-2011 spike from a few thousand dead links to ~110,000 out of ~17.5m live links.) The dismal studies just go on and on and on (and on). Even in a highly stable, funded, curated environment, link rot happens anyway. For example, about 11% of Arab Spring-related tweets were gone within a year (even though Twitter is - currently - still around).
I know that few people likely keep locally saved Web pages in near line accessibility. But if you only build the tool, I would happily mount the nearly 5 terabytes of ULTRASCSI drives I've got in storage, which were largely Squid cache snapshots, from my office WAN proxy box.
I'm more inclined to use Apache Lucene.
Both because of the abundance of people doing similar extraction from comparable data stores, but also because Lucene is a part of the Hitachi data suites which are my business choice.. Allowing I ever return to business.
I mention Hitachi only because by anointing Lucene in a Tier 1 product, if I encounter the need to provide a audit that is relied upon to attest any sensitive data is purged from the cache, I'm sure that it will be acceptable in the format of such a high end system report.
Of course that's window dressing,but my aim is to get big businesses to contribute more old data... just who might aggressively retain data for their own reasons, compliance and security the usual needs,
Well those donors are who need the reassurance of a more vendor.
I wrote above the delightful ideas this prompted me to forget about dinner last night, so I will stop here.
But I cannot help but feel in my gut this is a real and viable project that can snowball and be a huge thing.
Oh, sorry, I misled you, I have to add what just now came to my mind : anything that encourages big companies to give into really old data archives, is just when they are ripe for selling new bulk capacity. The need for refreshing the media is so overlooked by businesses that otherwise have great practices. So you can nudge even the least hungry prospect customers towards your sales team.
Can't a vendor jump on this one?
Edit format
I don't understand, unless maybe I was gushing somewhat. I do that. I'm female. And despite decades writing programs mainly for my personal research, I have a non technical, traditional media business, corporate experience of computing.
So when I look around here, I am both very happy to not see references to big ticket kit, and the feeling of independence is fresh without question.
But I cannot recall any acknowledgement of the tooling that corporations have habitually to hand. That GitLab controversy I thought hardly touched the issues of providing the level of service that major enterprises require and do obtain from the market, instead of using open source software for singularly critical infrastructure, without any analysis of the alternatives. It would be very boring to discuss storage products, its hard to get real information about them, even, despite the influence of a host of new companies. But the debate didn't touch a single advantage or aspect of the options exist.
I'm in total darkness trying to guess what I spoke improperly. I thought I might prompt someone who sells hardware support for say Ceph clusters, to comment, using the idea to gain awareness their own business.
But a silent, cold, zero, feels badly in need of some sort of company that commiserates the comment I wrote. I can totally grok if the Shut Up Woman reflex is one I have prompted you to feel. My bad, I usually have a lid on the gush. Is the idea so awful? I only know that the commission on storage systems is still enormous, and I think in particular younger HN types could make sheer bank, in a EDS sales setting, where real understanding of technology is, well, imperfect. It seems like the HN world doesn't acknowledge the corporate world much. That's a pity, if true. Below the household name corporations are so many in real need of serious minds, and I know personally that the salaries are not as bad as they are made to look at with statistics when debated here, I mean you think a typical character here is going to be a bottom decile performer, or top centile producer, compared against the big wobbly world of corporate computer departments?
Edit horrid auto correction
There are some other handy tools like youtube-dl, which let you archive not just youtube videos, but many different types of media content. Including soundcloud and bandcamp.
If you're curious, take a look at the discussions at the following links:
- https://en.wikipedia.org/wiki/Wikipedia:Archive.is_RFC
- https://en.wikipedia.org/wiki/Wikipedia:Archive.is_RFC_2
- https://en.wikipedia.org/wiki/Wikipedia:Archive.is_RFC_3
- https://en.wikipedia.org/wiki/Wikipedia:Archive.is_RFC_4
I'm also not sure what the current status of automatically archiving links is though, but as you can see, the idea has been attempted before.
In Wikipedia's usual frustrating manner, it's unclear to me what was even going on to trigger those RFCs or why people thought it was a problem. For some reason they were upset with links to archive.is. But why? Was archive.is replacing working links with archive links, or something?
Edit: From what I can tell, the archive.is bot was doing the same thing the archive.org bot Animats mentioned was doing. It's just archive.is didn't follow Wikipedia's policies and procedures.
/*
FILE ARCHIVED ON 5:47:20 Aug 6, 2015 AND RETRIEVED FROM THE
INTERNET ARCHIVE ON 20:45:17 Aug 24, 2015.
JAVASCRIPT APPENDED BY WAYBACK MACHINE, COPYRIGHT INTERNET ARCHIVE.
ALL OTHER CONTENT MAY ALSO BE PROTECTED BY COPYRIGHT (17 U.S.C.
SECTION 108(a)(3)).
*/
curl -v http://www.milliondollarhomepage.com/index_files/widgets.js
/*
FILE ARCHIVED ON 5:47:20 Aug 6, 2015 AND RETRIEVED FROM THE
INTERNET ARCHIVE ON 20:45:19 Aug 24, 2015.
JAVASCRIPT APPENDED BY WAYBACK MACHINE, COPYRIGHT INTERNET ARCHIVE.
ALL OTHER CONTENT MAY ALSO BE PROTECTED BY COPYRIGHT (17 U.S.C.
SECTION 108(a)(3)).
*/
They then just copied it all as-is back into some web hosting.
The point I'm trying to make is that the Internet is a different kind of thing than a library, it is more dynamic (which is good and bad) and generally a more living and lived-in type of medium. MDH was a money making scheme. Clever? Maybe. Intended to be long-lasting? Doubtful.
It's a cool article, nonetheless.
I say "needlessly" because the actual cost of storage, CPU, and networks are dropping so quickly that hosting a page forever should asymptotically approach a fixed cost that isn't much more than the first few years.
So, on the other hand, if someone isn't willing to sponsor the cheap cost of hosting a site, maybe that site and its content isn't worth preserving?
Im really wishing for one of the big cloud providers to offer a one time payment guarenteed for life "digital locker". A single e2e encrypted service to store my digital assets. Essentially an online scrapbook that you could add to and journal your whole life in without any prying eyes or advertisement.
As someone who's been involved in internet hosting since the days of dial-up, that's just not true. Or rather, it's not the whole picture. Hardware breaks and becomes unsupported. Dynamic pages run on software that rots. Is a hosting provider expected to still support pages that run on SSI, CGI with Perl that requires libraries that haven't been updated in 15 years and are full of security holes?
Who's going to pay to move websites to new hardware? For the trouble of ensuring backups are working properly? For reacting to legal problems?
Hosting involves a lot more than sticking a .html page behind an internet connection. Software rots. Webpages rot. Someone has to be paid to provide upkeep.
They drank the Cool-Aid, so to speak.
The article uses the Million Dollar Home Page as an example of an interesting historical artifact that tells us a lot about the internet of 2005 in terms of design, the context in which the page was created, etc.
However, Million Dollar Home Page is also used as an example of the complexity of archiving internet content, i.e. archiving a page in a complete way must also involve archiving the pages it links to, otherwise the functionality of the archived page will decay over the years. This has important implications for its usefulness as a historical artifact.
Most of us remember what the internet was like when the Million Dollar Home Page was created, but many years from now, it will be challenging for people studying the history of the internet to really know what it was like back then unless we archive things in a way that preserves functionality.
so, in the end, even if we managed to keep that page up, it would still be of very little interest to the pople of the future.
the proper way to do things is to document everything. by good historians.
I have a huge soft spot for projects where you can get the most happening because you are not required to jump a known hurdle to usefully contribute.
Overnight I was fretting with the necessity of sorting out any residual legal issues that might attach to digging out old cache dumps.
But forgetting the fact that the very same companies are commonly invested in just the tech to sort out the problem,.
Problem being one which can present all kinds of ways. Do you have any chance of finding adult content in your cache? Do you care about how much you are seen surfing the competition's websites? Will URLs reveal that anonymous forum login from 2002, slagging your rivals benchmarking? Did you put Squid on your intranet or webmaster, without https because your predecessor thought it was on the non routeing private range? Did you use DNS in any way to point to document resources that are accessible to users via the proxy server and Squid?
Anyhow lots of data protection suites have been used to purge archives and remove any trace of activity or files best kept private.
That's how the latest HDS kit is sold : cluster FS with hyperconverged local nodes crunching security, audit and search results.
I'm sure I'm only wishing on a prayer that you can find great troves for redecorating the empty space that the WWW once pointed to.
But imagine that we really could fill enough gaps in the dead link forests!
Even just attempting could be a superb way to promote your storage products and bless your customers who offer up the raw stores with lots of great ways to engage regular journalists with the subject.
I totally live best effort projects, and especially the ambitious culturally interesting ones.
So does Joe Public.
I'm in.
Where do we start?
(if I have the time, I'm quite serious about this, my profession is advertising not online but the traditional way of doing it. I have just got my head back from exploding with the multitude of ways to sell, promote, demo, boast, reminisce, predict, forecast, warn people about how society will crumble without personal all flash arrays... Last one might be pushing it a bit, but I see a fantastic deal of business on the back of the idea. It's beautiful because everyone is invited to participate, no vendor or company is locked out. So the public is not getting the boring official line and canned quotes. This is real people showing the technology but showing the extent to which we discard valuable culture. For thousands of years mankind grew as our means of recording documentation and thought and expression grew. Look now how easily we will throw it all away!!!! I know we can do better.
It's crazy how many copycats came out, very unoriginal thinking going on.
Definitely not copied from anotheer commenter ;)
I keep a list of things that follow this pattern, and yeah, altcoins/tokens are another. It really just becomes a catalogue of what's been popular/happening these past couple decades.
Any successful thing on the internet that hits the media will sooner or later be copied (usually sooner). Remember 'awesomenessreminders.com', the story had barely been told or the first copycats were up. And that was the HN audience doing the copying.
I was quite busy for the next few months as orders kept rushing in!
I've also had a few competitors pop up & the script was pirated, including some selling it on ebay, they even forgot to remove my contact email, so their customers were coming to me for support.
Eventually, the fad died down, but it got a lot of business out of those 800 pixels.
Thanks Alex, if you're reading this, for the original idea & letting my ad on your site.
In this case, the library didn't fail. The ecosystem around it did.
It hardly seems fair to blame a billboard being in disrepair if the company it advertised no longer exists.
Even though its with a business we're not doing now, my business partner and I are on there.
Edit: don't think it deserves a downvote - is it not an interesting question? I bet there are loads of serial entrepreneurs on both
On the million dollar homepage, she has the face just under the Union flag.
I'm also not a big fan of redirections that cross failure domains, just because they make the web slower and less reliable.
Fun, well, that's a matter of taste so I won't debate you there.
But "fairly straightforward"? Let's not get crazy, here. IMO, ICOs are doing to cryptocurrency what derivatives did to traditional investing. Abstraction, indirection, and confusion, all to make a buck...
I've read a lot about ICOs and I still don't understand what's going on. I can't even tell what people are actually buying (like EOS, where it explicitly says the tokens are worthless and not convertible). An even bigger question is why people are throwing so much money on them when it's so unclear what you're getting.
It's a good example of a double pyramid scheme. The value of Ethereum is itself going up; it was worthless 4 years ago but now has a market cap of $20B. And then you can "spend" Ethereum to get additional tokens out of an ICO, which means that you're buying into another pyramid scheme with tokens which themselves have appreciated. The huge dollar values, after the double conversion, generate great news stories, which inspire more people to buy into Ethereum, which prop its value up even more, which mean that the companies who did ICOs can convert their cryptocurrency to dollars at better rates and inflate their dollar values even more, which inspires more ICOs at bigger valuations, and so on.
Lest this sound too dodgy, remember that all currency is itself a pyramid scheme: it's inherently worthless, and worth only what peoples' expectations of its future valuation will be. So the endgame for cryptocurrencies is either they run out of hard-currency buyers, in which case their value comes crashing down and everyone who bought in loses everything, or they replace the dollar as the world reserve currency and everyone who didn't buy in loses everything. I think the latter option is unlikely, but also underpriced, in the sense that it's more likely than most people give it credit for.
Here's a thought experiment... in the US, work full time while getting paid in Bitcoin. Make all transactions in Bitcoin. At the end of the year you'll need to file a tax return. Your final options are: 1. Buy some USD to pay your taxes. 2. Go to jail.
Demand for fiat currencies is ultimately underpinned by them being the only instrument accepted for payment of taxes.
You're driving at an important point that I want to take a bit further. Ultimately, the "backing" of a fiat currency comes from the barrel of a gun. People continue to use U.S. dollars because the U.S. has the strongest military on earth right now. Same throughout history - the backing of the British pound came from the British Navy, and when the British military collapsed post-WW2, the pound sterling didn't last much longer.
A major adoption driver for cryptocurrencies has been the weakness of central governments. We've seen this most with China, where wealthy Chinese businessmen are eager to take money out of the country in any way possible. But the most likely tipping point that would drive widespread adoption of cryptocurrencies would be if the perception became widespread that the U.S. government could no longer guarantee physical security; if, for example, we "lost" the war on terrorism. You could easily imagine people "hiring" personal drones or robots for protection with BitCoin if this came to pass, to fill the vacuum of the state security apparatus.
A pyramid scheme is one where there is a continued and ongoing transfer of wealth from new scheme joiners to earlier scheme joiners, which works great until the supply of new joiners dries up - then the thing collapses.
A new currency could work (in theory) absolutely fine with no further issuance, and no new joiners beyond the first round of adopters - not at all the same as a pyramid scheme.
I always wondered why he didn't allow changing the pixels/URLs and reselling them. Should have been easy to set up and an ongoing revenue stream.
X Money From VCs => Y (discounts to users/deals with businesses to "start using my product") => High sales => Repeat (Series A/B...)
Reddit in April did a twist on the limited pixel idea with https://reddit.com/r/place
Eventually the last bit of 1000 pixels went for 37k.
If the kid had extended the page another million pixels, you could have measured the peak interest and the subsequent decline.
Cards Against Humanity raises $100,000 to dig 'tremendous hole' Makers of party game livestream a backhoe digging a gigantic hole somewhere in the US, saying ‘as long as money keeps coming in, we’ll keep digging’
Source: https://en.m.wikipedia.org/wiki/Lukas_Podolski
[1] http://www.thespoiler.co.uk/index.php/2009/02/04/fc-koln-fin...
[3] http://www.fourfourtwo.com/news/schumacher-helps-fund-podols...
>Most of our political leaders are not engineers or scientists and do not listen to engineers or scientists. Today a letter from Einstein would get lost in the White House mail room
but then supports a candidate that denies climate change and replaces scientists in government positions with company stooges. Never understood Thiel.
Also, my attempt at getting rich online: http://schraitle.flat-line.net/cash/index.html (I made a whole 26 cents in the end)
You didn't notice because the people who created them didn't, either.
http://wiki.twit.tv/wiki/TWiT_Brick_House
I couldn't quickly find information about how much they cost or he raised, though googling reveals multiple companies that will sell engraved bricks to power your fundraiser.
"Million Dollar Cat Billboard project sells 10 000 “squares” (places on a billboard) $100 dollars each to make world’s first ever cat billboard and put it up in 10 cities around the globe for a month. To proudly show your cat to the world you need to buy at least one square. But of course you can buy as many of them as you wish as long as they are available."
"Identifiers for the 21st century" https://doi.org/10.1371/journal.pbio.2001414
note/claimer/disclaimer: Although I am included as an author I do not write that well.
Also I wonder how Word got around to me about things like this in the days of MySpace and yahoo as my internet.
News aggregation sites have been around for awhile.
It's all in the marketing!
Edit: looks like the million dollar homepage is probably a bit more complicated than 2d knapsack, somewhat like tetris.
Another edit: looks like the only non-rectangular space sold on the million dollar homepage was the last space (as specified in the top linked article). So perhaps 2d knapsack is appropriate. It's late here.
I actually have seen the server that it was on. The hosting company provided free hosting to it for several years (at least 8 IIRC). I think that at some point the server died and Alex Tew decided to move elsewhere.
At one point, he had the million dollar pixel lotto which was the same idea except at two dollars per pixel and one person who clicked on one of the pixels would get half the money that came in.
The canvas was partitoned into blocks of 10x10 pixels that was white until sold. You would make a payment for X number of blocks through 2Checkout or PayPal. After transferring the money, you would send an e-mail to Alex Tew with the image, coordinates of wanted blocks, URL and TITLE/ALT text. He would then confirm the payment and upload your banner within 48 hours.
So the entire operation was manual with orders placed by e-mail.
For the record, in 2005 I told him about Million Dollar Homepage, and he was angry that someone could get rich off such a stupid idea when so many others fail.
That provides me with a digital copy, and it is automatically sync with my Kobo reader.
1 million pixels for only a dollar each!
That guy made a nice bundle off the idea, it got picked up and hyped by the media so much I'm sure the companies that bought in got some ROI, or at least some publicity. Such was the extent of the dot com bubble that this sort of nonsense could happen and everyone cheered...
It still rode a wave of media hype and naivety.
I've often encountered a sort of "reverse link rot" where a cited article is in another language or obscure journal or reference material that was never digitized and is inaccessible.
Perma.cc was created by the Harvard Law School Library specifically to try to solve that issue, since so many citation links were becoming useless.
And realistically it's something that after the initial craze cools, can run on quite cheap hosting.