Etsy CTO on Its Conservatively Crafty Tech Philosophy
techcrunch.com
techcrunch.com
Given that they've produced some impressive things, Etsy always seemed to be the example of a shop that was overdoing it - but I had actually assumed they were overcomplicating it with 20 engineers or so. Looking at their github now, and hearing those recent numbers, it feels much worse. That level of work is great, but for a company that does what Etsy does, much of it seems like overkill. (Managing 1000 instances should be the same as managing 100, this is where automation kicks in. Architecture enables that.)
Ultimately, that's a tremendously large amount of engineers, to where I can't really mentally assign what all but about 20 of them might do. Let's be generous and say 30 - or even 40 :) All of that eats into the bottom line (and also makes adding more staff harder, as your tech becomes more and more custom).
Really, at the basic level, it's a CRUD-based storefront website. Yes, it has scaling needs. But it's not really novel in that regard.
Yet, they've built more automation around their automation (lots of things in the Chef ecosystem) and probably made things a lot more custom.
I can understand some of this for say, multiplayer gaming or video streaming infrastructure maybe, but I would suspect Etsy could have been run a lot easier with less wheel reinvention.
Startups should really strive, IMHO, to write as little of software as possible, that's outside of their line of business. When wheels are square, build better wheels sure, but what I see from that number looks like MASSIVE levels of maintaince costs for being that custom.
So anyway, thanks for producing things that other people use and that are useful - YES - but from the startup accounting / efficiency perspective, I wouldn't want to see my company run that way.
I think that mpdehaan2's point was that (from the outside looking in, at least) almost everything Etsy's doing seems to be a minor variation on a solved problem. That was definitely not the case for Amazon.
At the scale of Etsy, they don't get any benefit from AWS - they're certainly big enough to justify the cost of an operations team, at which point AWS is just an expensive exercise in vendor-lockin.
It really surprises me how many people think that being more self-reliant and understanding/controlling more of your dependencies, is a bad thing. It's not always possible, but why wouldn't you, if you had the resources to do so?
The only time I got close to an answer on stuff like this was when I asked why the company behind a large tv/broadcast network insisted on using "vendors" for every little piece of technology - they would pay for hosted forums and battle through the issues with integration (of which there were many) rather than picking up an existing forum product and customising it as necessary.
The only answer I was ever given: they want someone to sue when it goes to shit.
Amazon didn't invent virtual infrastructure that adjusts dynamically based on load.
The throat to choke argument is not sufficient anymore for a lot of enterprises though because they are sick of their vendors giving poor results or ones not aligned to their own goals in the product lines. I haven't exactly heard of anyone getting actually sued, but you can make a little money back sometimes if a vendor doesn't meet their SLA.
I started advocating mostly open source solutions partly because using vendors is like taking drugs and eventually they'll start dictating your architecture rather than you pick your architecture, which is likely to conflict with a competing vendor's architecture. All of this talking and demoing is also costly, but these type of evaluators are fairly easy to find compared to talented / competent developers.
But really, companies that already have solid lines of business that can only grow a bit are far more concerned about loss of revenue than trying to grow n% faster in most parts of the company.
Sure, you could run this in the cloud and not worry about physical machines, but that will probably be more expensive. As companies grow, it becomes cheaper to bring lots of things in house, rather than outsourcing them.
As companies grow, small % differences can result in large total impact, at which point having engineers eek out small gains may make a lot of sense, e.g. think of Google's results around latency, or small improvements in recommendation engines that drive higher sales.
I question what the current cost structure looks like. As a public company, my guess is that investors will do the same as well.
In a world where Instagram deals with orders of magnitude more customers with a whopping five engineers, it's a pretty high bar to show why so many resources are necessary for this particular operation.
I'm sure the Etsy guys are sensitive to that as well, but from the outside it's hard to believe there isn't a better way to use resources more efficiently.
Not the same problem domain.
edit: Instagam has not that many customers. Has many users. That's a real diference.
That said, the resource difference between these two companies is considerably large, and the tech in question (e-commerce) is not exactly an area of unknown innovation. From a technical standpoint, that's sufficiently a solved problem.
They've written StatsD (Node), a custom deployment framework, an A/B testing framework and a bunch of other technology software (https://github.com/etsy) that could need dedicated engineers to build and maintain. But all of this would pale compared to the needs of the business itself.
At sufficiently large scale, any CRUD-based storefront website becomes an entire supply chain management system. I suspect there are many ways to slice up an SCM, but from a functional level, you'll need subsystems for each part of the domain: managing customer orders, warehouses, logistics, payments, offers & promotions, developer facing APIs, accounting & book-keeping, ..
These systems can be built using a service-oriented architecture, but Etsy is famous for promoting the large monolithic app architecture (http://highscalability.com/blog/2014/7/28/the-great-microser...), which would remove the need for a lot of SOA plumbing work and communication overhead.
Apart from the core business logic, they'll also need a whole bunch of QAs, Android and iOS developers, designers, DevOps folks, database admins, security specialists and so on. All told, they could probably run a leaner ship than the 200-odd engineers they now have, but I wonder how much more leaner.
One small correction - In this case they also do not have warehouses since items are shipped from craftspeople. I also would expect mobile apps could be done by a team of a handful of people.
As for SOA vs monolithic apps (highscalability link currently is a 404), it's largely horizontal scaling and a load balancer. That in itself shouldn't change architectual choices.
Every large(-ish) company I know has pretty strict rules about the languages, frameworks, and tools used for the main products.
The more flexible of these do allow prototyping using a different toolsets. The ones that have a developer audience always experiment with new environments.
New tools and new processed get adopted over time, but in many cases the cycle they go through emulated pretty closely the adoption cycle seen outside. Even if the new shinny toy syndrome is very strong in this field, this makes a lot of sense, as nobody really wants to throw away everything. (_While writing this I got to think about the numerous rewrites eBay has seen over years_).
> The company owns and operates its hardware and networks in its own datacenter. Part of the reason for avoiding the cloud is control, but not necessarily just from a security perspective as you might think. The real reason is Etsy personnel want to get under the hood and understand every aspect of every piece of technology the company controls.
I continue to be very curious about how polarizing the cloud remains.