When Yahoo Ruled the Valley: Stories of the Original ‘Surfers’
nytimes.com
nytimes.com
We used to have "Porn Whackers"* at Xoom who would go through user sites that had been reported/complained about/whatever and decide whether the shady stuff that people put on their free pages was
1) kiddy porn (freeze the account, send directly to the FBI)
2) legal, but simply in bad taste (do nothing)
3) copyrighted software, PDFs, that sort of thing (delete)
4) copyrighted porn (initially delete, but see below)
At some point it dawned on us that we were throwing away money by simply deleting the sites in category 4. I wrote a redirect and we cut a deal with Playboy to display ads for their website instead of a regular 404 page when a visitor landed on a nabbed-for-copyright-infringement freebie site.
Needless to say, the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms. Either that or you found something else to do. Plus there's that magical feeling of being one of the first humans ever to see a guy's tied-up nuts being stomped on by a crazy woman in high heels. We never did figure out whether that was a nasty domestic dispute or some sort of fetish thingy.
Eventually Xoom got bought by GE, who had the bright idea (pause not) to make the spam emails "opt-in". Way to take a moneymaking idea and turn it into a turd, guys. Eventually they realized that they'd killed the goose that laid the sort-of-golden eggs and we all sought other employment.
I miss SF from back then. My apartment in the Mission was $650/month and the 21 Club was still the best dive on Earth. The Excite guys were especially hysterical, they'd show up to the colocation facility so zonked on heroin that servers which were literally on fire gave them pause in terms of how to deal with the situation. Plus the amount of pretty girls around was obscene. Combined with all the free booze, it wasn't a half bad scene. Ah well, nothing good ever lasts.
* Yes, the obvious thing occurred to us. Nobody cared.
* * There may have been a 5th category, "legal and in good taste", but out of 17 million users and 20TB of sites, I never recalled seeing one that fit this description. There's a reason we talked about categorizing Porn & Warez.
From the skills I learned in that chatroom I would wind up making sites for a fee (I think I charged $30, felt like a lot of money but wow must have been a steal for the business folks). Years later I've been working in SF, recognizing the locations you describe and envying the rent.
A website on overfishing I hosted on xoom as a project for fifth grade science class got me an A, so there's one site that was legal and in (in my teacher's opinion) good taste!
This all reminds me how important free things are to kids who want to learn things but can't spend any money (even $5 for a SquareSpace is prohibitively expensive).
The customer service people at Xoom were great ones, many of the more colorful characters I've ever met. Ed went off to start a security firm, Bryan moved to LA to star in gay porn, and I dated a crazy pink haired girl who ended up in SF by hopping trains all the way from North Carolina. It was a good group of people, by and large. As previously noted, many Xoom ops people eventually ended up at Google, and I like to think it was a net positive there, too.
Of course we also made mushroom-and-peanut-butter sandwiches in the break room and piped the heat exchanger from the "machine room" into the rest of the building (an old bank building on Montgomery), but overall it was a good group.
https://www.youtube.com/watch?v=LTJvdGcb7Fs
I seem to recall that basically all lists of top search terms are filtered. Any raw list would be at least half dirty.
Remember, they were only bothering to whack copyright infringing porn, or at least that's all I remember there being time for. Otherwise it's just Rule 34 in action.
Your comment that the "the pornwhackers were awesome to drink with. When you stare into the abyss for 8 hours a day, you develop some excellent coping mechanisms" reminded me of an amazing article published on The Verge and written by Catherine Buni and Soraya Chemaly. [0] (The article was featured here on HN 91 days ago.[1])
The article examines the psychological cost of content moderation and reading it helped me appreciate even more the difficult work content moderators perform to make the web a better place. (Hats off to dang, sctb and others whose user handles I don't know.)
Anyway, here's a teaser from "The Secret Rules of the Internet" in case you missed it the first time it came around.
> Mora-Blanco sat next to Misty Ewing-Davis, who, having been
> on the job a few months, counted as an old hand. On the
> table before them was a single piece of paper, folded in
> half to show a bullet-point list of instructions: Remove
> videos of animal abuse. Remove videos showing blood. Remove
> visible nudity. Remove pornography. Mora-Blanco recalls her
> teammates were a "mish-mash" of men and women; gay and
> straight; slightly tipped toward white, but also Indian,
> African-American, and Filipino. Most of them were friends,
> friends of friends, or family. They talked and made jokes,
> trying to make sense of the rules. "You have to find humor,"
> she remembers. "Otherwise it’s just painful."
>
> Videos arrived on their screens in a never-ending queue.
> After watching a couple seconds apiece, SQUAD members
> clicked one of four buttons that appeared in the upper right
> hand corner of their screens: "Approve" — let the video
> stand; "Racy" — mark video as 18-plus; "Reject" — remove
> video without penalty; "Strike" — remove video with a
> penalty to the account. Click, click, click. But that day
> Mora-Blanco came across something that stopped her in her
> tracks.
>
> "Oh, God," she said.
>
> Mora-Blanco won’t describe what she saw that morning. For
> everyone’s sake, she says, she won’t conjure the
> staggeringly violent images which, she recalls, involved a
> toddler and a dimly lit hotel room.
[1] https://news.ycombinator.com/item?id=11500234
EDIT: formatting
It's crazy to me to think that YouTube was relearning many of the same things the pornwhackers had figured out ten years earlier. I guess there is an underbelly that Silicon Valley really, really doesn't want to talk about. When it mattered, though, YouTube did the right thing, even though it could have cost them the business. Little decisions like those can, in a small way, change the course of history.
I don't think that sort of flashpoint ever happened at Xoom, because we didn't have any policies against bad taste. For all I know, Damien and his crew were gleefully promoting protest sites the whole time.
If you ever have a chance to read Ted Rall's piece "A Cog In Satan's Bulldozer", do it. He touches on bits of it here and there in http://comiczine-fa.com/interviews/ted-rall, but the real thing is mindbendingly great -- truly, almost all cynics are disillusioned idealists. When I worked for google from 2003-2004, it was as close as I've seen to a corporation with a pervasive conscience (people really did ask whether a purchasing decision was "evil"). It seems like even their ideals have slipped.
I went back to school after a while. Now I design clinical trials for kids with cancer. It doesn't pay as much but at least when I die I won't say "gee, I sure sold a lot of ads!"
Despite the hype, the Web just isn't all that impressive as an information resource. The static, publicly accessible HTML text on the Web is roughly equivalent in size to a million books. The UC Berkeley Library has 8 million volumes, and the average quality of the Berkeley library content is much, much higher! If 10% of the material on the Web is ``useful,'' then that means there are about 100,000 useful book-equivalents on the Web, which is the size of good public library. The actual figure for ``useful'' is probably more like 1%, which is 10,000 books, or half the size of an average mall bookstore.
http://people.ischool.berkeley.edu/~hal/Papers/japan/index.h...
I'm leaning strongly toward curation via tagging, using standard ontologies, and vetted through designated / qualified experts for top content. A mix of crowdsourcing and expertise seems most viable.
Quasi-related -- a rant on Pocket suggesting improved "tags" functionality (I swear there's another, better, though I cannot find it in the usual places).
Also youtube is an amazing resource that books can't compete with.
Tags and ontologies also can't compete with search, Bigquery and data lakes.
Currently, the predominant business model for commercial search engines is advertising. The goals of the advertising business model do not always correspond to providing quality search to users. For example, in our prototype search engine one of the top results for cellular phone is "The Effect of Cellular Phone Use Upon Driver Attention", a study which explains in great detail the distractions and risk associated with conversing on a cell phone while driving.... It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media [Bagdikian 83], we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.
Including in my tagging schemas for Pocket are quality indicators. I've been using an arbitrarily selected ranking scale of "a-1, a-2, ... a-5" to indicate quality (higher is better), and also tagging items with keywords indicating negative attributes (usually "bullshit", though others apply).
Being able to group classifications by reputation of author or publisher is also a key element. While there are hazards in argument from authority, there are frequently benefits both to recognising expertise and identifying non-credible or impeached sources. If a source routinely pursues some objective rather than the truth, you might tend to question its value.
PageRank essentially relies on the implicit reputation vetting of links, using link sources as a basis for quality assessment. It's useful but also limited. And as noted, the bias toward advertising and specific gain (see also Celine's 2nd Law, from Robert Anton Wilson's "Illuminatus" trilogy) raises issues.
I've been doing significant amounts of research, much of it online, but much also tracking down published book and article sources (most of those through illicit download sites -- really the only way the information can be obtained reasonably), and what I'm finding is that the noise associated with online data is just boggling.
Books, it turns out, compete exceptionally well with YouTube. Ultimately they compliment one another. Sometimes I want to read, sometimes I want a lecture or demo. Both have their place.
Excessive data ultimately isn't illuminating.
Yes, online search can be useful at finding out, say, what's happening right here, now (transit applications are a huge one), or within a given social circle, but both of those are domains in which an exclusion scope is set which rules out virtually all available results -- you're looking for information about a specific location and time, or concerning a specific group of people.
When it comes to general information, "reliable sources" dwindle. If you're looking for some established authority or text, you're left with the specific sources. If it's news, typically you're going to want some vetted news source -- and yes, that includes informally vetted sources on Twitter or YouTube. One of the most annoying things about both sources are the crap leaches who instantly show up to clog any trending keyword topic with crap content. That shit's got to get stomped hard.
For new and developing information, ways to broadly pre-filter and bubble up quality content are an interesting problem and one I'm actively working on. Community rather than broadcast methods strike me as more viable.
Also, I wouldn't imagine there to be any sort of IP to such an idea, as the sites listed belong to other people/companies.
Yahoo had a form that allowed you to submit your site to be added to the directory. You included the URL and categories you would like to be listed in. It would then be evaluated and added by a Yahoo employee. It was one of the first things I would do after creating a new web site in the early 90s.
So in a way, with something like the way Netflix replaced explicit star-ratings based recommendations with viewing-history based recommendations, we are still looking for curation but in a subtler, more algorithmic way... same with Google PageRank and Facebook Newsfeed for that matter. Maybe algorithmic filtering is just curation at scale.
1. Deleting from history any "poison" content -- generally, stupid shit -- "that corner of YouTube" stuff, helps.
2. I've largely abandoned logged-in use of YouTube for numerous reasons. Its app is all but unusable due to pre-roll ads. (Tasteful banners I can stomach.). Routinely purging all history, and/or resetting device ID helps.
3. Purposeful and directed use of YouTube, or, frankly, any other service, is best. Search for what you're interested in and read/view that. Much of my use is autodidactic and informational. There's actually a wealth of information online now, if you can filter out the crud.
4. Being able to blacklist the stupid shit channels would help a hell of a lot. If I could personally banish to the depths of Hades any channel that posted stupid crap, things would be markedly improved.
#3 is my usual mode for initial watching, but I'm prone to go with suggestions from that when I'm watching. Sometimes, even with best intentions, it gets messy.
Number 4, that would be most excellent. :D
(I've also noticed that, well fed and pruned, YouTube's suggestions actually tend toward useful these days, though that's with exceptionally aggressive pruning.)
Yet another option is to use youtube-dl to download and play content locally. I do this pretty much exclusively on desktop, and find it quite helpful.
So nowadays what you'd previously look up in something like dmoz you now just search for in Google.
When I'm doing market research or looking for certain types of information, being aware of forums like HN is more productive.
Long term the movement of everything behind walled gardens like Facebook will make it worse.
Huh, what? There are estimated to be more than a trillion sites out there, from what I've read.
The number of domains is definitely closer to a billion than a trillion. This number can be verified pretty easily (unlike the number of web pages).
https://www.washingtonpost.com/news/the-intersect/wp/2015/01...
[0] This PDF of an article by P. Frana (p.31) is the only reference I can find to this image[2] I clearly remember. https://instructure-uploads.s3.amazonaws.com/account_1770000...