Searching the Creative Internet
crawshaw.io
crawshaw.io
This caused a a shift away from creating content for content's sake. 15 years ago, someone may have posted an in-depth technical article for no other reason than to share knowledge. Now, that article would be posted on Medium.com in a gimped form, as a lead-in to their 10-hour video course on that topic.
The early period of Twitter briefly brought the "weird web" back, but once brands and businesses descended upon it, it became all about retweets and follower counts. Quality nosedived as a result.
The internet didn’t stop being interesting, HN is a great example of this in action. The blog article about rejected Disney princesses is likely the most interesting piece anyone of us read on the internet today.
Only Google didn’t deliver it, a HN link to an article about the creative net did. I say google, but really, any of the gape keepers is equally guilty.
Interestingly I think all the gate keepers are at a place that yahoo, AOL and others where when google disrupted them.
So maybe, just maybe, we’ll see the dawn of something better soon.
This sounds really hard, and for me I don't know where I'd start, but I also didn't create google or pagerank which at it's dawn also seemed like magic, so I have faith this could be done.
It's the sort of mission to which I could dedicate my life, until I too was corrupted by the success and lure of endless wealth, at which point today-me sincerely hopes some other smart oblivious kids would come along and displace me.
So I too hope to see something better in my lifetime.
Take the results of a big search engine, and programmatically filter out everything that contains something from the Adblock filter list. Not just getting rid of the ads, but ignore the entire page if it contains advertising. Iterate through enough results until you actually get the first 50 or 100 hits not containing any advertising, and return those as the basic search result.
There would be collateral damage of informative pages that attempted to good-faith "monetize", and you'd miss out on the stackoverflow results etc, but I would hope this would surface many of those super informative sites of yore. That is assuming they're even still indexed.
That makes me kinda sad, like a cool drawing getting thrown away that no one saw. These pages would sit out there, only with inbound links, but never a search result.
These ideas are half the battle, I think the other half is the curation, which is a little more abstract, but possibly machine-learnable to some degree?
At a more general level, I miss the days in which I could type a search query like `intitle:"index of" mp3 mb` and get actual, unfiltered results. I've toyed with the idea of indexing the web myself and use simple filters, but I think I'll wait until someone here gets funding for it instead.
I personally miss the prevalence of technical and personal web logs of the late 90s. That's not to say that they aren't still around, there's simply more alternatives to shift through and many of them do focus on marketing themselves for visibility (which seems to take something away from the feeling of the older net IMO). A large focus on centralization certainly seems to have shifted the broader tone online, though there's still plenty of gems floating out there.
While most of what I could still locate dates only back to the mid-2000s , consider https://web.archive.org/web/20050122063815/http://matt.might... , around that time blogspot was pretty active and well ranked for search engines at the time, freestanding wordpress instances were fairly popular after individuals moved past writing html posts manually and uploading them over ftp to a simple host, phpbb instances were available for most topics in the mid 2000s. The key thing is that once you move before the easy availability of php based webhosts, then you get into the territory of hand edited plain html (or microsoft frontpage) and very very little to no-javascript.
Lots of interesting stuff about hardware and software. Minimal fluff, maximal content.
By trimming the top 1000 results, I was able to find some truly random websites that just happened to have a deep archive of interviews of a band I was looking up. All interviews were from '80s magazines, pretty much all defunct now.
Had I stuck with Google/Bing, every top search result would either an eCommerce store, Spotify artist page, Allmusic, or a snarky Pitchfork/Vice piece about them from the 2000s.
For this search, Spotify, Pitchfork and Vice were at the top of the search results because they are SEO-optimized This means that Google/Bing search will show the links from the domains that perform best under their page rank algorithms. Since Pitchfork and Vice are domains with a high number of backlinks and lots of active traffic, those were the ones that rank the best.
Given that so much of web content is just rehashing what was orgiinally reported/said somewhere else, finding niche content is going to be harder and harder.
Oo oo I know this one. They have those so that they can report discoveries about the nature of the universe to the taxpayers who are paying them to make those discoveries.
Heck, I'd go even further and create a JavaScript-and-third-party-cookie-free search engine. Block any domains that host tracking pixel images too.
Bring back the old web!
My first-run result is here http://kakapo.susa.net:8080/cfs/ - I think the results are promising, even on a tiny index (around 1M pages, I think).
It is a question of search / discovery mechanisms. Mostly this kind of query is “satisfied” by things like Twitter. But I wish there were a good blog / discussion search engine. Those died a long time ago. As you say the results I am looking for only show up on lower pages in Google. Maybe there is a better search engine for that I don’t know about.
I'm usually disappointed from twitter search(unless it's for finding smart people, and endlessly browsing through their stream), how do you manage to get so much out of it?
Specific threshold/blacklist TBD/subjective, and you'd get false positives (people posting truly original content on their Facebook page or Blogspot blog). But by and large, a lot of the truly "labor of love" content out there is done by folks both savvy and invested enough to set up their own domain name, and would pop out if you just filter out all the corporate domains.
And turn it into the mainstream content the author doesn't like?
I mostly don't get this reminiscing about the past state of a technology. When I see it I always suspect someone is remembering their youth in a candid way.
If you were to stumble across Uncle Brand's online presence, you'd likely be amazed with what an interesting and humble and cultured and well traveled person Uncle Brand is. And is that so wrong? He's only creating what he's seen other people do after all. Only giving you what you want. Why not put your best self out there?
And maybe Uncle Brand has a little something unique he does, that one emoji he always uses or that obsession with ramen. And maybe he starts attracting an audience. He's reliable! He's relatable! He's authentic! He's safe!
But an audience is something you have to maintain, something you have to grow. The audience didn't come for Uncle Brand the man; they came for Uncle Brand the brand. So he starts refining his brand, churning out more content, gets a better camera for his photos. He's got more resources now and can ape what big brands do.
In some ways, the internet became too real, too tied to the real world. You can even make real money on the ol' www! But when this happened, rather than the internet liberating us from the old, we just recreated the old incentives and shallowness and commercialism. But shittier and more random. Youtube celebrities are mostly just shittier celebrities. Instagram is mostly just shittier magazine and food and travel photography. Internet journalism is mostly just shittier journalism. So much online content is pre-internet content just pushed on a new channel.
And what's so wrong with getting real? Uncle Brand is all in. He's a souper star! The Martha Stewart of ramen. By now, Uncle Brand is using his brand to hawk stuff too. He's using his brand to hawk other people's brands. Promote, promote, promote. Sell, sell, sell.
The internet defies generalization. There are certainly great communities and forums and subcultures and people creating amazing stuff out there today. More than ever even. But the internet as many people experience today is indeed quite different from what I original loved. It feels like all the incentives are wrong; platform incentives resold to creators and users as their own.
I don't want to be a brand. I don't want what old world is selling: the celebrities, the popularity contests, the consumption, the fear of judgement. I just want to create awesome stuff and have fun and try something new. And I want connect with people who are doing the same!
Thank God for Wikipedia, it is a miracle. Long may it stay antifragile!
And long may NPR use its supporters' money to produce consistently archive-worthy content!
Donations to Wikimedia foundation or your location public radio station make great "solstice" gifts, if you're into that sort of thing.
So yeah, I just turned this comment into an ad, because let's not throw out the baby...
And yes, I'm old enough to remember the "good old days." There is every bit as much signal now. And every bit as much more noise.
And yes, Google's hegemony is a threat to the capital-I Internet's antifragility. (I just got Taleb's book, can you tell?) Guess who powers the analytics for crawshaw.io? It's all of a piece, people. Walk the talk.
And yes, I have a little Shakespeare site that's "better" than the top-ranked ones in many ways, but I accept that if I wanted those top spots---like any other top spots---I would have to sweat and hustle and fight for them. I don't, and no contrarian anti-Google is going to hand them to me.
I'm all for the better thing. Blue sky, every day! Step away from the machine! But c'mon, it's 2018, let's not dis NPR and Wikipedia! We're fighting the good fight!
EDIT Also, to eksemplar's point that "The blog article about rejected Disney princesses is likely the most interesting piece anyone of us read on the internet today." I picked this piece at random and it was so amazing. I emailed it to my wife who's an artist. Man. Thanks for that alone, ye OP!