A weird epic ramble about Etsy homepages from the middle 2000’s
twitter.com
twitter.com
That was a small part of the thread, but I mention it because of what he said about the power of human curation -- that the curator picked out these great treasuries from the sea of shirtless dudes. Because if you have the same frontpage for everyone, it's feasible to have an expert just pick that instead of needing some weird flaky NN that's expensive to develop, train, and run, and produces bland (or disturbing) recommendations anyways.
(Even algorithmically-driven sites like HN benefit from that, like dang manually emailing people to resubmit interesting stories.)
There is a lot to be said about the death of the shared experience, I wish I had the energy and writing talent to actually write a book about it because I think it deserves one. It's shaping our world from our politics to our culture. There are many positives and negatives about it.
Compare that to, the other day I uploaded my first TikTok about some obscure bit of Lord of the Rings appendices minutiae[1] and got 1500 views. Not going to be the next John Green here, but it felt cool to just try something on a whim and have some success. The algorithm spread that viewership out and made things more "equal", I guess. Plus, it made the "niche lord of the rings content" genre possible, since it wouldn't merit 1/15th of all TikTok viewership in the "TikTok has a fixed Etsy-style FYP with 15 videos" alternate universe. I wonder what the most popular 15 videos on TikTok are right now...
That's all to say, you should write that book (or essay!), I'd read it :-)
[1] Middle-third-age Gondor looks a lot like a mirrored Tang dynasty China, and has other similarities
The person arguing the affirmative called out something I’d not fully appreciated: binge watching of shows has killed what we used to call water cooler conversations. Prime time TV was a great provider of shared experiences most weeknights. Sunday evenings would be some just-released-to-TV blockbuster. And we’d roll into the office the next day and “omg! Did you see X last night?!” Of course they did, there was only like 3 channels back then (I’m in Australia, don’t laugh ;). But binge watching, catch-ups, streaming in all its form has sufficiently time shifted everything to the point we have low guarantees about having a shared experience. Or maybe I only managed to consume 2 episodes, and you watched them all. But you cant remember exactly about… “episode 2, is that the one… oh no that’s episode 3! Never mind”. We’re not able to have the same kind of free flowing conversations we used to. They also pointed out that sport is one of the last remaining shared in thine experiences that provide that offer similar opportunities. But it carries with it the factional tribalism the can push us away from someone as much as it can unite us.
That was used as a jumping point to suggest this lack of shared experience has slowly broken our sense of togetherness and community. That in turn has fostered the kind of hostility to others that seems prevalent online.
I found that a bit of a bridge too far. Though at the same time so much of the rest of the rationale actually resonated with my own experience, I’ve just not found the destination we’re at quite as bleak.
The rest of us are just left out. Even if we get time to watch one or two episodes, we can't talk to the ones who binged it all, because they can't remember what happened in which episode. The only way to prevent spoilers is not to communicate at all.
I like the Disney+ way of dumping 1-3 episodes straight away and then an episode every week, that allows the water cooler talk and fan theories to develop.
Looking back from a personal standpoint, communal watching and discussion of those shared cultural phenomenons really marked my life. The most emblematic recent examples would be Lost and Westworld I think.
But it's funny your comment about TV blockbusters, it shows it's been going on for much longer and we didn't realize it. My mother use to tell me often about her childhood in the 50s, how every week-end they would show the big blockbuster westerns with John Wayne on TV for a whole afternoon, and it's what everyone would watch and discuss afterwards. And it's just now that I make the connection with this phenomenon.
Looking back at my childhood, I can't remember TV being anything but trash. (I think it varies wildly from country to country.) There were no blockbusters on TV, and Cinema played the role of providing that shared cultural experience.
Now Cinema is dead and we're going to streaming. Who knows where it will end, but maybe there will be room for that experience to continue in another form.
Furthermore, as streaming options proliferate, I assume that more people are hopping between services. Episodic releases don't totally solve that problem--but it does add some friction. Someone can't now signup for a service to watch the new hotness lots of people are talking about, binge watch, and then cancel their subscription.
Other stuff can still happen, eg. a colleague enthusiastically sharing something they learned or a discovery they made ("did you know how to properly use a paperclip?!").
But yeah, there's significantly less "things you probably also saw since yesterday and have an opinion about" stuff.
There's nothing more civically uniting (in my city) than taking about the Ravens.
If you can get over the geek, middlebrow dismissal of "sports ball" and the whole CTE thing, live local sports on TV crosses racial and class boundaries more than anything else I've witnessed in my 18 years living here.
One thing that helps me get interested in sports I don't otherwise care about is to think of them like a board-game, thinking through the math of various possibilities and optimal strategies.
Another thing, if you live in a state that's amenable, is to bet small amounts on the outcomes of games. Betting $20 on a game makes it way more entertaining (for me anyway), and when I lose, it was worth the money just to enjoy the game more.
The last thing that helps is to think of it as a purely physical performance. Going to watch live sports can help with this - but for people who haven't spent much time in the sporting world, it's hard to understand how big, how fast, how strong, how coordinated, professional athletes are, and it's possible to just enjoy watching people running and jumping and throwing the same way you'd enjoy watching people dance or play music or esports or whatever, just people with exceptional skills doing exceptional things.
At this point in my life, I emphatically do not enjoy sports for their own sake, I pay attention to them for the social aspect and for the reasons mentioned above, but I couldn't care less about the outcomes of anything. It's a far cry from how I felt about them as a young person, but it's evidence, perhaps, that not being interested in them doesn't mean I can't still connect with people over them.
Church and community groups played a much larger role. Sport was still as popular. You might “go see a show” but it seemed like a much bigger deal than going to the movies is today.
My earlier comment wasn’t about people having something to talk about. It was that people were able to talk about an experience they both shared. Regularly. Like every week, if not every day (about watched they watched the previous night).
I think maybe the other observation here is that before TV people were more likely to actually experience things together with others, and not just talk about them after the fact. Way out of my wheelhouse here though as I’m no historian.
I'm not sure that's true. The studio system in Hollywood churned out a huge number of movies.
Today (and really for a long time) my sense has been that going to the movies tends to be about younger people on dates, getting together with friends for a fairly low effort, not too expensive activity. And, probably for this reason, from what I can tell cinema attendance has sprung back better than a lot of people expected.
But certainly the asynchronous watching has an effect too. "Everyone" watched Seinfeld etc. on Thursday night. Today, even for shows that drop weekly, most people are just out of the habit of watching shows as soon as they come out.
There's kind of still tentpole film releases, but Covid put a serious damper on that for a few years, and globally the market is splitting with China and India developing their own gigantic film industries equal to Hollywood.
Can you share the name of this podcast please? Sounds very interesting.
I miss that experience when reading novels. Best I can do is go online and find some people who read the same book 3, 8, 20 years ago. That's not the same.
Customised feeds took this away from us, disconnecting us even further. I guess the Metaverse (or anything alike product) will try to exploit this and reconnect users in the virtual reality. Being connected is okay, but the context will be controlled by someone else.
And even when it is shared on a wider scale l, it is filtered thru your filter bubble and social bubble.
And now every joke starts to be more of an inside joke.
It will be interesting to see how shared experiences evolve. We used to be limited by geography and subsequently cultures were geographic in origin. As geography becomes less of a limiting factor to how we share experiences, cultures and sub cultures will co-exist with the geographical cultures as well.
I feel like we are in a transitional phase. Marketing works most efficiently when everyone has the same shared experiences. When marketers and widget makers have to increasingly create for niche sub cultures, the profits are less. Instead of making one ad for the super bowl, now marketers have to make an ad for each social platform. Instead of making one style iphone cover, there now has to be custom or curated designs for all the markets.
I think eventually the profits will win out and or people will recognize the detrimental effects of lack of shared experiences, and somehow technology and society will coalesce around a broader shared experience environment.
A hurdle to this happening is the fact that people love being part of sub cultures or "scenes". It may take a while.
It is getting increasingly harder to socialize with people in general. There are so many references that the conversation has to stop, everyone has to view the video/tik tok/post, then the conversation can move forward again. Professional and educational settings I do not think are immune either. People are becoming so much more specialized in their fields that it is harder to identify with others. This would be less of a hinderance to socializing if everyone had a larger portion of shared experiences.
This makes so much more small things viable.
Look at the custom keyboard scene. There are now multiple companies and people making their living selling keycaps for keyboards!
This sure was not viable before.
I think since 2010 so many shared experiences coming from the internet that it actually became rare to learn new stuff from conversations.
That's a huge pet peeve of mine. All my guitar playing friends watch the same fucking videos on Youtube, so they just repeat what the people say on the videos and there's no opinion anymore. Everyone watches Tom Scott videos and whenever I mention "Autotune" they go "They actually use Melodyne because someone on a Tom Scott video said Autotune is just an effect and every non-T-Pain vocal is Melodyne". Autotune can actually just auto tune, very happy for my Bavarian buddies for their success, but sometimes Autotune is just Autotune. I must have heard this about 10 times already.
On Hacker News there's always those little memes... XKCD 927 (Jesus I wish it would die), Betteridge law of headlines, Gell-Mann Amnesia. It's the same things over and over.
Even conspiracy theories have become homogenised, and now there's no new crazy cuckoo theory coming out of nowhere anymore that you can laugh about with your friends.
Before that the only shared experiences were things like the Friends last episode or news, or sports, or things like that, and that was MORE than enough.
It's great that the new generations are learning that people can know about different things and share experiences orally rather than via a screen.
Sorry for the rant.
Dudes a creep.
I kinda expect old forums (and old friends!) to have the same old lingo from before and be set in their ways. But the crazy thing to me is that this happens with new venues and new people too.
I not-so-recently moved countries, and also moved cities, and it's definitely crazy how even completely new people still talk about the samey internet things as the other city. I did the same 15 years ago, and it was completely different.
I'll check your link for fun stuff later!
Well, and the really old ones that are just him writing passive aggressive comics about his ex-gf.
Overused crap that kills discussions, overused by uncreative people.
The results of this are evident everywhere. Fashion, music, web design, food, academic writing. In any discipline or artistic endeavor, it's like the whole world decided this is the one best way to do things and nothing outside of that norm receives the nourishment (capital, time, attention, thought, support) that it needs to develop.
One of the last examples I remember is about 18 months ago I noticed everyone, everywhere, as if coordinated by some unseen entity, started replying "oof" to everything. Maybe this really is just a variant of the Bader-Meinhof phenomenon (another one of "those little memes" you mentioned, heh).
Maybe I should start participating in Star Trek discussions!
I watched that episode in the last year and didn't make the meme connection until I was reading someone mention it somewhere. It would be interesting to have a modern take on that episode. The thing with memes is they don't necessarily convey an idea or a point that furthers a discussion. They're often just a way to relate something in the current context with a past idea (for a laugh), so it almost seems that a language based on memes would just end up stagnating culture.
Maybe but I think it is just more social signalling, to either show you are part of the opposition or to highlight the comment for like minded people. Similar to other comments : Yikes, Wrong side of history, a bad look, etc. Most of these comments are made on highly political threads, highly controversial topics such as system-d don't seem to evoke these kind of comments. That leads me to think that they are more emotional in nature than analytical.
I’m seeing lots of “r00d” these days
You would have hated it, then, in the days of only one or two TV channels, and in towns where there weren't many newspapers.
Which I suppose is a gloomy point: people have always been like this so it's unlikely to change.
All my friends, acquaintances and groups that I see daily are also kinda new, seeing I moved countries and barely see old pals, so I doubt it's about being stuck in a group.
And it's not really about hearing the same things as from N years ago. It's more about hearing the same information from 10 different people. Before I would hear a piece of information from one or two, despite having access to pretty much the same number of people. There is definitely more information online, but people are looking at the same things! Things weren’t as homogeneous as they are now.
I see the echo chamber effect in a lot of these communities. You need one of the 5 approved amps for X style, or are you even a guitarist? These arguments amuse me more than anything.
But is everybody really like that? If I'm watching YouTube videos on guitar gear I'm trying to get a feel for something, and then I try stuff. Sometimes I agree, sometimes I don't, but at the end of the day I choose what I like.
I hadn't played in quite a few years and am recently back at it. Back then, I certainly owned things the Internet community collectively agreed sucked. Now that I'm back at it, some of this gear is now considered good to great, and I have to laugh about it. No regrets for having my own opinions.
I don't have many friends who play though, so perhaps being all on my own shelters me from the monotony. Or perhaps you need to find different people?
In those internet communities there’s people who don’t play this game but they barely post. In real life, there’s also the people who don’t care, but then again they won’t really talk about it as a hobby.
But the people who actually had some interesting insights 10 years ago are now only parroting the “mainstream” guitar influencers. Which themselves are often parroting things from 10 years ago.
I’m not complaining, btw: it is what it is. It’s just that it is homogeneous and boring!
For instance, most of my friend group watched CGP Grey for a while, and we don't debate how to choose a movie or restaurant anymore, we just choose one quickly via either approval or veto voting. Or online someone mentioned XKCD 927 and then we can discuss the finer points of the new standard instead of just how it's going to replace all the others (for instance, instead of saying Matrix is just the chat protocol to unite everything else, talk about how it's an eventually consistent graph replication protocol and how that differs from XMPP's point-to-point system).
It feels like this shared culture and background can serve the same purpose as like shared education in math, literature, and history - to provide ground knowledge and references as a jumping off point to discuss more in depth, or with more nuance.
But the things I'm complaining about don't really become part of "shared culture". They're more like temporary "memes" that people use to show off they're interesting. Which is perfectly fine, we all do it! But the issue for me is that those little trivia tidbits that people are using are repeated so often that they start to irk and people look deeply uninteresting. :(
One of the parts of this that you didn't mention is politics, which irks me in particular. I can barely bring up any political topic without incurring a strong emotional reaction that causes the other to close off their mind - and that includes with people that I mostly agree with!
There's even this politics-like part of homogenized programming subculture - strong emotional reactions to things like webtech, Electron, C++, Rust, Windows, Apple - and you see the same "arguments" over and over (because I don't believe that there's really "critical thought with the intent to convince" happening - just talking points regurgitated over and over).
In the particular case of HN, many of these memes just straight-up break the guidelines, and many users don't care, because they'll upvote or vouch low-quality content that fits their worldview. (I'm guilty of this myself, even when I'm specifically watching myself to try to avoid it)
This kinda thing definitely seeped into politics. Especially with Facebook memes, even my parents couldn't escape those :(
And yep, programmers in 2022 are definitely not a group known for originality! Even (we) the dissenters walk in step...
It's only about the "money shot" the punch line and that's it.
I've had existential crosses about this and wondered if I had an original thought. Even having an existential crisis seems trite.
I'm not sure the alternative of everybody having unique ideas would be preferable though because most of them would almost by necessity suck.
Funny, that's another pet peeve with Hacker News. Posts that are phrased as if there's only two ways things could go and they're black or white, and completely unrealistic to anyone spending more than two seconds thinking about it.
The alternative is the one I presented (watching an actual variety of stuff), not some unrealistic crap.
Also let me flesh out my "alternative", which is starts with suggesting that perhaps people aren't being too unoriginal and that them changing their behaviour would be for the worse. Maybe the true alternative is changing your own behaviour so you're exposed to people culturally different than yourself just like the person who you first responded to.
>> "What a shocking bad hat!' was the phrase that was next in vogue"
-- Extraordinary Popular Delusions and The Madness of Crowds, 1841
Memetic culture is older than you think. That book covers a lot of different types of group behavior that could be called "memes".
Think about the emperor's new clothes, but instead of the kid shouting his message in the town square, the kid privately sent the message to each of the same people. Not the same result, is it?
I'd love to work at those. As opposed to the monstrosities I often see in some unicorns and well-funded places.
Spent a minute trying to figure out if you were trying to make a pun (mono or sites, or idk) before realizing that’s how it’s spelled. I gotta get some more sleep.
I look for dev teams of 3-5 for this type of company.
Made my day
> “he was busy with the lasers and the raccoons”
Now I'm kind of curious what sort of content ends up on Pinterest that has to be moderated out, because for some reason I assumed it would be categorically different from the type of content that has to be moderated out on a site like Reddit, but now I'm thinking it's probably pretty similar.
That said, I think there's something foundational about having moments like these in your career. Startups really are zany-land, or at least were, and the days of Flash and PHP were /full/ of weird things you'd have to deal with operationally thanks to completely absent conceptions of security. Some of the strangest things I've dealt with technically all were related to operating services built in PHP. I know it's not really en vogue to say, especially with how bad Javascript is as a language, but building services and sites for the web is MASSIVELY better now than it was in the past. The late 90s, early 2000s was fraught with ridiculous security issues caused nearly entirely by the entire ecosystem of languages, frameworks, and the people using them being ignorant that security is even a thing. NPM package security issues aside, the current situation in the web is so much of an improvement over the past it's hard to fathom.
That’s not really what happens. Most users of a website are trying to use it as intended. It’s 1% of users who get a kick out of trolling a website with weird content. The difference is that they have a lot of free time and their content can be gathered randomly from Google and posted with little effort. Compare that to actual sellers, who have to actually make something and photograph it, and you can see why the low-effort spam content fills the site up despite being a relatively small number of users doing it.
What I like about the story is that it makes my own experiences seem normal. At some point in life, we all run around trying to fix things we don't fully understand.
This made me laugh a lot harder than it should have!
site still has a "we just opened" banner
https://web.archive.org/web/20050714023957/http://www.etsy.c...
It got nuked. The woman that set it up still owns it, but there’s nothing there.
Luckily, it can still be found in the wayback machine.
It's pretty much my only "impressive" job of my career, in terms of name dropping at parties. But in terms of technology and business it was a madhouse. I remember thinking "Nothing like a business was happening at Esty." when they fired me. (I was going to quit, but my girlfriend at the time convinced me not to, and then they fired me a week or two later anyway. Long story.)
I always figured someone would eventually show up and want to interview me about it, but that's never happened so far.
I've forgotten most of the wild stuff.
All the business logic was in stored procedures in the DB so as you can imagine it was always on fire. Upgrading didn't help because our growth rate was ridiculous.
The frontend PHP guy was a newbie to programming. He barely knew what he was doing but he could nail those beautiful UX designs pixel-perfect. The guy's probably a millionaire now.
The backend sysadmin, DBA dude was a total Master-Blaster. If he didn't like it he would literally throw a fit and scream at the CEO and shut things down until he got his way. He used to lie to the CEO all the time if he didn't want to implement something he'd just say "it's impossible".
They built a skateboard half-pipe in the office even though none of us skated.
It was pretty nuts.
I found out later on that they hired me because my resume was the first one on craigslist that had Python on it. Then during the interview they found out I used Devorak layout on my laptop and that's why they hired me.
The "middle layer" they had me write was pretty much useless. It basically passed requests from the webservers to the DB. Over 80% of the API was empty methods, literally just a docstring, with all the processing happening in a decorator.
I implemented a cache in there (early use of Memcache. Glyph hisself wrote a Twisted interface to it in front of mine eyes! They were like "We want to hire a networking consultant. What about this guy?" Me: "You want to hire Glyph and I get to hang out with him and learn from him!? Yes. Yes, please." So that was nice.)
The thing is they didn't turn on the cache. It just sat there. The investors pushed this Sid person onto us to try to get some semblance of adult supervision. ... I've told this story before, Sid wanted some information on long-running requests, and I had to switch it (the cache) on in prod myself to show them that the required logging was already there.
There there was the other Sid who wanted to rewrite everything in Java. (First Sid wanted to rewrite in C++.) "Project Phoenix" he called it, that didn't go over well.
(Recall that the problem wasn't the implementation language of the server code, it was that all of the business logic was in stored procedures in the DB so rewriting the server code was never going to help.)
That's all I got this morning.
I barely can’t rembeber the perfectly written API, but I still laugh over some of the wild shit I saw over the years on companies that started with a single php file and a mysql db, and when you go to have beers with your teammates its a 100% safe topic to just have laughs all the way long.
While there was excitement and fun, we objectively were doing a terrible job, and that felt terrible.
Rob and Chris were simply incompetent, and succeeded only despite themselves. Haim at least knew his stuff, he was just unprofessional (screaming at Rob, etc.) (Edit to add: Rob was a great UX designer, all those beautiful layouts and pages were his work. He was incompetent as a CEO.)
Working there became so frustrating I was going to quit. (It's the only time I didn't quit a job that was obviously bullshit. I don't blame my girlfriend for convincing me to stay on, it was my choice, but i recognize that it was the wrong choice.)
We wasted so much investor money.
The thing that bothered me the most was that we used our users as the QA dept. Chris would push to prod, things would break, and he would scramble to fix them. The users thought he was their hero fixing problems and didn't realize he was the one causing the problems in the first place!
And then, in the end, the whole thing was gobbled up by (I think) the Waltons? And now it's a caricature of what it aspired to be.
Despite the "success" of the company I see it as a cautionary tale.
I had a very similar experience in a few startups/scaleups.
They were incredibly successful despite the horrible tech and horrible environment for development.
IME the reasons for that are often due a culture problem stemming from the top, so there's no way to really fix the problems as an employee.
One of those companies still has one of the founders in a "tech director" or something like that. Really great guy, great person, but he must be writing his 3th or 4th ORM now, and I'm not even joking. The first few iterations were in PHP but the new one was going to be in Java. Each time the backend needed a new feature that was difficult to implement on top of the previous ORM, he would retreat to design a new version of the backend (including a homegrown ORM) and gets green light from the CTO. He and the team are clearly not really good at producing a good design, and also not good at finishing, considering the messes left behind on every rewrite-everything implementation. When I worked there there were three "official" ways of talking to the DB.
The frontend framework was replaced a couple times (jQuery, Vue, another one which I won't mention since nobody uses it), even though the home page didn't really change for a few years. It was an e-commerce-like site but not even the items in the homepage would change (it started changing while I was there, though!).
The dev headcount was >400 IIRC, so it's not surprising that they had so many rewrites.
I got burnout pretty fast from all that. Adding a single button to a page was a two-to-four-weeks ordeal. My manager was a bit of an asshole, who said the problem was me and I should go back doing therapy and taking antidepressants (like I disclosed I had done a few years ago) instead of looking for a less stressful job.
Kinda saddens me because it's a company I liked, although I couldn't stand the tech culture.
I hope you've found better place(s) to work since then. :)
I still have the weird stories to tell, but glad I'm out of there.
I hope you're also in a great place!
That was pretty much early 2000s web development.
The original poster also made a thread about the middleware [1], and there's a video about it too [2]
I've never met Dan McKinley
> ... adding a Python middle layer.
Yup, that POS was my baby. "Emid" the Etsy MIDdle layer.
> They had asked a Twisted consultant what to do, and the Twisted consultant said they needed a Twisted middle layer (go figure).
That's inaccurate.
Glyph was the Twisted "consultant" (for those who don't know, he's one of the main authors of Twisted.)
But he was called in later, the middle layer was not his idea, I had already written most of it by the time we hired him. I do not know why they wanted to hire him. Pretty much the only thing he did was write the Twisted Memcache interface, and that took him about 20 minutes? He live-coded it in front of me, less than 100 LoC.
The thing was, Haim (the sysadmin & DBA) blamed all problems on the middle layer as a scapegoat. It wasn't actually the problem (how could it be? It did almost nothing!) but since no one else knew that it was easy to blame it. People talked about "the Emid Problem" but it was really the DB on fire all day every day because Haim wouldn't let any business logic escape his control (as stored procs in the DB.)
> The initial release of this resulted in two full weeks of downtime, and an infamous incident where one of the investors had to drive to Secaucus to physically remove the other engineering founder from the cage.
I don't remember that particular incident but then they wouldn't have told me. Haim's meltdowns in the data center were legendary though.
All that stuff about teams must have been after I got fired. I was the whole Emid team when I was there.
> people were understandably really mad at this middle layer, but nobody agreed on the reason to be mad
Right, because it wasn't actually the middle layer that was causing the problems. It wasn't helping, but it wasn't hurting either. (For one thing, it wasn't like it was spewing tracebacks or anything like that. It worked fine, it was just pointless.)
It was very frustrating having my work blamed for problem it wasn't causing, and then being unable to get anybody to listen. (Rob didn't have the knowledge to evaluate the technical details, Haim was lying to him, and Chris was sitting next to Haim playing along with it.)
Anyway...
> The theory of the drop-in team was that the existing middle layer was bad because although it used Twisted, it still used a threadpool. (Twisted is a reactor loop like nodejs, but in Python.) For Twisted acolytes, this situation was heretical.
Again, this didn't matter because the middle layer wasn't the bottleneck.
> The middle layer didn’t really do anything. It was consultant-provided speed holes which (it was believed) would make things faster by (counterintuitively) adding network hops.
Yep. I told them this at the time. (Again, it wasn't "consultant-provided", the three founders hired me to make this thing. Glyph came later and was only there for a week or so IIRC.)
> The middle layer just received RPC and invoked postgres stored procedures. Which, if you have a superficial understanding of things, seems like the kind of boilerplate you can replace with an abstraction.
Yep. Like I did: a single decorator handled 85% of all the API methods, the method bodies were just single docstrings to satisfy syntax. Not even a "pass" statement!
> Then they proceeded to discover that here in reality every single existing endpoint did things differently.
That would be the other 15% of the API, yes.
The rest of that thread is beyond my time, but it sounds horrible. Godspeed.
> So over the course of a few weeks we frantically rewrote the drop-in replacement to use a threadpool instead, exactly like the original heretical one.
> Leaving us with literally the same code as the thing it was dropped in to replace, plus a ridiculous declarative framework, plus some tests. It was around this time that everyone got fired (but not me).
Okay, that is fucking funny.
I regret the waste, but this makes me feel a little bit better about the whole mess.
That's generally true about the overall stability, the DB popped constantly.
One reason Emid got grief was because it started barfing every time sprocs, table schemas, views, etc were changed and nobody ever really managed to resolve this. The DBA's would update the text of a sproc without telling anyone and Emid would start throwing exceptions. Certainly a set of fixable problems, but nobody ever really got a good mental model of when it needed to be HUP'd.
"Emid uses psycopg, ergo we have to rewrite the whole thing" was one of the insane justifications tossed around for the rewrite project. Never made any sense.
The business issue motivating removing the middle layer altogether was just that to do anything you needed to change three things in three different programming languages (maybe four if it involved JS or Flash), and this was foolish.
Haim (DB) and Chris (Front-end) were sitting next to each other hammering out the API together, I was mostly working remote. They would tell me the API updates/changes over IRC, I'd update the Emid code, and we'd kick the servers to restart.
> Never made any sense.
Nope. :)
(In hindsight, given what I know now, rewriting the whole thing in Erlang would maybe have been a good idea!)
During the period between Rob's CEO tenures he and Sid #1 built a Google Analytics alternative together, which was in fact written in C. We very narrowly avoided acquiring this company.
The thread seemed to mostly touch on some organisational disaster topics and a little bit of ‘how can we make this system work better for sellers’. I didn’t really see it as requiring automating the system so much as making it a bit less horrifying.
It would have made more sense for the employee to spend their day finding good collections and adding them to a list. Then whenever someone loads a page a random item from the list is used. Old items would expire from the list after a week or two. This lets the employee do their job 9-5 and spreads out the load on the people who get featured on the front page.
It's also happened in the brick and mortar world, when American ideas took over Europe.
For this to change, the American age would have to end, but I'm not sure if whatever comes next will be an improvement.
... wanting to wake up every hour, 24 hours a day, to sift through pictures of shirtless men to find some beautiful furniture to put up on the frontpage?
But in startup land, where you have limited resources, somehow the ticket to make that somewhat automated by queuing up a bunch of these things at once wasn’t high enough of a priority.
Honestly the best way to have gotten that fixed was to put the developers into rotation. I bet somebody would have cooked something up after the first shift!
"The magic in the system was the person yanking 24 beautiful treasuries a day out of the sea of shirtless dudes."
That's defeatist logic. Sure, scaling with a single person doing it is likely impossible, but all of this reads to me as a bunch of technologists being utterly unable to integrate the human element into their systems. The conclusion is that a human aesthetic sense was needed -- first to filter dickpics and second to put together a gestalt page.
I can imagine ways to scale that, but it hinges on trusting the human element -- something technologists and investors are utterly incapable of.
I hate it when you don't realise the thread ends and twitter shows you a full sized login banner where pressing back doesn't bring you back to the thread but instead goes back to previous website.
https://nitter.net/mcfunley/status/1550219636464967680 is making it easier but I'm like this is 100% an article that could feature in any tech magazine or blog.
1, I believe Twitter downranks link tweets in the algorithmic feed. If your following is on Twitter, you need to meet them where they are, but you can't meet them with a link. I've seen some people work around this by combining a Twitter thread that summarizes the post with a link at the end of the thread to the actual post, but then you need to write twice (or 1.5x, I guess) the words.
2, thread-tweeting can be lower barrier than blogging. The tech side (hosting a blog and associated yak care) a little, but mostly the writing. Writing a big post all at once is a thing and easy to get discouraged by, where on Twitter, the lack of an edit button forces you to go forward and not second-guess yourself. It's also more ephemeral, less feeling of "this needs to be perfect since it will live on my website forever".
2.5, There's a retro-tech thread tweeter (who I'll avoid naming since they've requested to not be linked from HN due to this recurring complaint) who's said they specifically find it helpful for ADHD since they can tweet out bursts of little updates, work on something else, then come back. Personally I also have ADHD and trying to write a twitter thread is my personal hell, but to each their own :-)
My jumbled up mess of a brain would never allow me to get a coherent 20+ tweet chain out without typing it out first, splitting it to tweet sized chunks, editing, re-editing and then copy&pasting it to a thread manually.
The story begins while I was still working in the financial industry in lower Manhattan, watching Visual Studio whitescreen for a half hour as it often did and questioning my life choices. 2/
One day while that was happening a woman I knew DM’d me a link and said “I want everything on this website.” 3/
I don’t know what she was trying to link me to because the site was broken, and it served me this error page. “Hey, sick error page,” I remember thinking. 4/
This kept happening though, and eventually I came to be dimly aware that there was a new site called Etsy with a reputation for being full of beautiful stuff. 5/
That reputation came from the homepage, which at the time was a grid of items that looked like this 6/
The homepage grids changed periodically, and they were always interesting. The items would be beautiful individually, or beautiful together, or it’d contain some oblique joke. They’d be a rainbow of color, or they’d be the same shade of eggshell white, whatever. 7/
It was this really wonderful handcrafted thing about handcrafted things, from the pre-personalization era. 8/
Anyway I became familiar enough with this via osmosis to recognize the name Etsy when I saw it on the python.org jobs page, so I jumped and applied. 9/
There were a few red flags in my Etsy interview which I was not conditioned to notice or emphasize at the time. It was scheduled for 3PM, but didn’t start until 3:30 because my interviewer was “really hung over.” 10/
The interview was also about Lisp. There was this Lisp moment in the middle 2000s. You had to be there. 11/
So after joining the company I eventually learned that the homepages were sourced from a feature called Treasury. Treasuries were a way for Etsy users to create collections, roughly the same as Pinterest these days. 12/
Treasuries were selected and promoted to homepages via human cron. There was one poor employee living the Leonardo Da Vinci lifestyle, waking up every hour around the clock to turn over the homepage. 13/
When I started she’d been at it for a while, and it took another six or eight months for that to end 14/
Obviously this is not the kind of thing a human being should be asked to endure for a startup. But you have to understand, there was a problem of approximately this magnitude everywhere you looked. 15/
------- So the first version of Etsy (2005) was PHP talking to Postgres, written by someone learning PHP as he was doing it, which was all fine. Postgres was kind of an idiosyncratic choice for the time, but it didn’t matter yet and anyway that’s a different story. -------
The p95 of search in that era was several minutes. The site was down for a full hour each week or so, and not on purpose. The human tragedy of someone waking up every hour to change the homepage felt kind of contained I guess 16/
So like many of the frontend features of Etsy in those years, the Treasury was built in Flash by a visionary named Jared Tarbell, the “fourth founder.” He was more of a visual artist working in Flash as his medium than an engineer. 17/
He lived in New Mexico and was very difficult to get ahold of. Once I talked to him and he told me he was working on etching intricate patterns into rocks with a laser, so he could drop them in the desert for future raccoon civilizations to wonder about. 18/
Search ranking was a big problem for us, and we figured we could improve this if we knew things like “this or that listing is popular in popular treasuries.” 19/
Beyond that, the homepage was the same for everyone looking at it. We thought maybe we could iterate away from that to something less artificially scarce. Getting an item on the homepage was a big deal for sellers, but the math of the situation made it unattainable for most. 20/
The Flash architecture really held those ambitions back. There were only 250 Treasuries live on the site at any given time, which was a function of the RTMP server backend it was using. This was some kind of Flash chat server dingus. 21/
We didn’t even have access to it. There was a contractor managing a Windows box someplace. And there was no database backing it, so as far as we knew once Treasuries were off the site they were lost forever. 22/
In our heads this beautiful, wonderful thing feeding the public perception of Etsy was a small scale tragedy in terms of data loss and lost opportunity. 23/
Back in the constant chaos of daily goings on on the engineering team, we were trying to get our arms around things like security vulnerabilities. We hired some pen testers for the first time, and they found problems everywhere. 24/
Flash was one of the systemic problems. Everywhere we had Flash, we had XSS. And there were worse issues with it, for example, you could impersonate the Etsy CEO in chat with users without authenticating if you wanted. 26/
There was an internal Flash endpoint you could use to exfiltrate all of our sales data, and in fact there was a whole other company that had figured this out and was selling our sales insights to Etsy sellers. 28/
Anyway we realized that, oh god, we are going to need to get our arms around this Flash stuff. And Jared had his hands full with the raccoons and the lasers, so we tried hiring somebody. 29/
We hired this guy that I guess I will call Steve. That’s not his real name. We let Jared interview Steve all by himself, which was a thing we never did again. 30/
Steve worked there for about a month, or a month and a half tops. A couple of vignettes I can remember about Steve: one day he freaked out at me and called me a pseudo-intellectual. Touché. Totally fair. 31/
Steve had this intricate conspiracy theory he believed in centered around Bertrand Russell, which I never figured out. 33/
There was one day when the site was down—the new CEO ran into the eng room yelling at everybody. Everyone was hair on fire freaking out about it. And then suddenly, we all just stopped as we gradually realized that Steve … 34/
… was at his desk watching an Infowars video at maximum volume. He’d been cranking the volume that whole time because he had his headphones on, but they were not plugged in. 35/
One day, I was walking back to the office after lunch and I see Steve coming out of the building. I hid, crouching between two parked cars, so he wouldn’t see me. 36/
When I got up to the office, the head of ops handed me Steve’s laptop. “Here you go, you’ve got to figure this Flash stuff out. We just kicked Steve out.” 37/
The first thing I did was make sure there were neither physical nor virtual booby traps on the laptop. After establishing that it wasn’t going to explode or send my passwords to Steve, I opened up Adobe tools and started fucking around 38/
I quickly surmised that he’d gotten nowhere, and was just watching Infowars videos that entire time. But luckily I got in touch with Jared who delivered a tarball of the Flash source (it wasn’t in source control anywhere) 39/
Somewhere in that tarball I found a set of credentials for an ftp server in a comment. After connecting to it, I do an ls and find a giant pile of binary files, format unknown. 40/
However the file timestamps are a clue: this is the schedule of when the treasuries turn over. It’s the treasury data! 41/
This happened on a Wednesday or a Thursday, and I barely slept through the weekend as I tried to reverse engineer and recover all of this data. Which remember, we all believed was going to be a gold mine that would change everything. 42/
I never figured out what the file format was, but I opened it up in hexl-mode in emacs and defocused my eyes and could figure out the basic size of the records and overall structure. 43/
By Sunday I had figured it out just enough to be able to write a Python script that downloaded all of the data and grepped through it to dump out the Etsy listing ID’s that were in them along with comment counts and other metadata. 44/
And I had reverse engineered Jared’s ranking formula, so I could emit the most popular (and therefore best) treasuries of all time. I hit enter on this script and let it crank for hours. 45/
When it was finally done, after days and days of effort I opened up MacOS finder and flipped through the results. It was almost entirely men with their shirts off, which surprised me at the time. 46/
I went back to the office and never mentioned it. I don’t think anyone even asked what happened with the whole treasury data thing, because again, the entire building was on fire. 47/
“Yeah, I don’t know, the treasury data thing didn’t pan out.” 48/
There apparently was this entire phenomenon of sellers putting up sexy pictures of themselves, and then people making treasuries out of them. But engineers and leadership never knew about this because the moderators were just filtering that stuff out. 49/
A few years later, we rebuilt the treasury and let people have as many treasuries as they wanted. And it turned out that what most of them wanted was to make treasuries full of penises and vote them to the top. 50/
This generated an entire protracted incident where I spent two weeks working on “class DickFilter” - and then pretty much the day that I was ready to deploy this the hivemind decided it wasn’t funny anymore and the meme just kind of dissolved. 51/
I guess it’s a testament to the power of compartmentalization that I rebuilt the entire treasury and didn’t forsee this crisis that was 100% predictable on the basis of my own experience. 52/
Evolution probably weeded out everyone that would have dwelled on the Shirtless Men Result too much and given up back on the Serengeti or something. 53/
There are probably several morals here I guess, but the main one for me was never to overlook the human in the loop. The magic in the system was the person yanking 24 beautiful treasuries a day out of the sea of shirtless dudes. 54/
I still think about this saga a lot, for example when someone suggests in a legal filing that you could find bots on Twitter using AI without worrying about the human labeling. 55/55
I could see Twitter gaining foothold as a "public forum" before, but it is now turning into another user-hostile monetization platform. This is of course understandable, since investors have poured billions into it. I guess I can only dream it will lead to its demise and to be replaced by a truly public service, like Wikipedia.