Hacker News Reader

Comment by shebson | Hacker News Reader

I generally agree with your post, but I think there is a critical difference between your argument and that of the blog post. Of course teams are more productive with technologies they know, but that isn't necessarily an arbitrarily-defined "boring" technology.

To pick on one specific example in the post: Node.js is popular enough that there are lots of teams and engineers that are most comfortable and productive working with it. For these teams, choosing Node certainly wouldn't cost an innovation token, while deciding to build some service in Python, Ruby or PHP (if we take at face value that this is more "boring") may end up being more costly.

View on HN

> For these teams, choosing Node certainly wouldn't cost an innovation token, while deciding to build some service in Python, Ruby or PHP (if we take at face value that this is more "boring") may end up being more costly.

It absolutely does if it is only one team in the organization. If the entire organization is using PHP and, let's say, you acqui-hire a team based on NodeJS, unless they are doing something absolutely fundamentally different they should learn PHP and push code in your existing infrastructure. This way you have one way to deploy, one type of application server to support, one set of gotchas relevant to your domain, one set of QA tools etc. Building good products is about far more than just shipping the product, it's also about the cost of long term support. Because what you are doing is fundamentally automation, the less you have to manage the more benefit of the automation you are getting, the more you can forget about it and focus on shipping other things.

What you are describing is pretty much definitionally local optimization and is exactly what you shouldn't do in large engineering organizations.

Depends on your increment of isolation. This is, in theory, why microservices with APIs mean that it really doesn't matter. As long as there are sufficient hire-able engineers who know that technology, it can be used.

> This is, in theory, why microservices with APIs mean that it really doesn't matter.

No it still really does matter, because if your company needs to deploy those microservices in different ways then you need more people to support the deployment infrastructure. If you need to test those microservices in more ways you need more people to support the testing infrastructure. For engineers it's the amount of friction felt when trying to move around and work on new and different problems in your company because they're one of five people who knows how the hell the Java infrastructure works.

If you have an existing testing and deployment infrastructure etc. your team gets those for free and doesn't need to reinvent (and support) those wheels.

> As long as there are sufficient hire-able engineers who know that technology, it can be used.

Yikes. Hiring and firing engineers is pretty hugely expensive and something you want to help avoid having to do.

> No it still really does matter, because if your company needs to deploy those microservices in different ways then you need more people to support the deployment infrastructure.

This is what PaaSes make a non-problem.

I should know, I worked on Cloud Foundry Buildpacks.

Here's how to deploy the PHP app:

    cf push your-php-app

And the Nodejs app:

    cf push your-nodejs-app

And hell, why not a Ruby app too:

    cf push your-ruby-app

And let's not forget that Python microservice:

    cf push python-code-works-the-same-way

We also kept up with the cool kids:

    cf push your-go-code-with-a-godeps-file

And for the "boring" crowd:

    cf push your-java-app-too

In general, these are all intended to Just Work™.

You know how making these surprisingly unalike systems deploy identically is really hard?

So does Heroku, from whom a large body of Cloud Foundry buildpacks code is derived. So did we, when we found issues specific to making code that assumes a connected environment work in a disconnected environment.

The point is, if you're doing this all by hand, you're doing it wrong. You should rent or install a PaaS and move along to the part where you create value instead of inventing a cool-sounding wheel.

> The point is, if you're doing this all by hand, you're doing it wrong. You should rent or install a PaaS and move along to the part where you create value instead of inventing a cool-sounding wheel.

That makes a lot of sense for small organizations, but I'm sorry PaaSs absolutely do not scale to the needs of many organizations.

> In general, these are all intended to Just Work™.

My emphasis added. When they don't Just Work it's really nice to own that infrastructure and be able to fix it yourself. It's also nice to be able to tailor things more specifically to your needs. Again, I agree that owning that infrastructure is not the right solution for organizations of all sizes, but neither are PaaSs.

> That makes a lot of sense for small organizations, but I'm sorry PaaSs absolutely do not scale to the needs of many organizations.

Outside of the giants who rolled their own because there was nothing around in the early 2000s, who?

> When they don't Just Work it's really nice to own that infrastructure and be able to fix it yourself.

Cloud Foundry is specifically designed to run either in the public cloud, the private cloud, or both. You can get it hosted it from Pivotal or IBM, amongst others.

The work of my peers and I made that possible.

> It's also nice to be able to tailor things more specifically to your needs.

Cloud Foundry is opensource and the IP belongs to an independent foundation. I am personally aware of at least two companies who have private forks of buildpacks because that suited their extremely precise requirements. It took them about two developer days, tops.

And their modified buildpacks also Just Work™, because they're based on a robust design that Just Works™.

That's really cool, I did not know about that. You were talking about PaaS and Heroku, which I don't think is the same as forking an OS project and owning it yourself, which is fine for what I was talking about. I don't think it's difficult to name companies for whom Heroku is not appropriate. Regardless, it's all about where you draw the line, gradations not black and white.

I still stick to my main point: your organization gets a massive benefit by all using the same toolset. If you are using Cloud Foundry, I'd still suggest the whole company stick with one language, one deployment infrastructure etc.

To be clear, if you're google I'm not suggesting the entire company all be forced to use one language or something. In that case your company is likely working on products that are different enough that it makes sense to do away with some global optimization. Some judgment is obviously required. But if you're in the sub-500 range (which the vast majority are) it makes a lot of sense to really optimize globally with your toolset, even if deployment infrastructure is relatively easy to setup.

PS I love that you are using the phrase Just Works - the company I work for is called Justworks :)

I mentioned Heroku for two reasons. First, they are the pioneers of public PaaSes. Second, several Cloud Foundry buildpacks are extensions of Heroku's buildpacks.

I think that the nice thing about something like CF is that a whole range of problems just goes away. On the other hand, as Weinberg observed, when you solve the worst problem, the second worst problem gets a promotion :)

Cloud Foundry doesn't get much buzz on HN. But I'm a one-eyed bigoted fan, so I mention it whenever I can. I'm actually a Pivotal Labs employee, my main work is agile consulting. But I've seen enough gigantoglobomegacorps who are choking on their own impossibly heavyweight deployment/ops mechanisms that I am a bit of a bore about talking up Cloud Foundry.

It definitely sounds awesome. We've got a pretty simple deployment system at the moment and so it's a solved problem for us, but when it starts to breakdown we'll definitely take a look at CF.

No worries.

Sorry for being snooty up above. When you work hard on a product it's easy to become emotionally invested.

> When you work hard on a product it's easy to become emotionally invested.

Know that feeling :)

I haven't worked in a specifically microservice environment, but I am currently working in a company that has quite diverse technology choices. One of the problems we have is that we're pretty small, and sometimes there is a lot more work that needs to be done on one component than another, in those cases you can't really have people who are relying on the work help.

E.g. we use erlang/cowboy for our web server and when there are bottlenecks changing that, they pretty much fall on two people who know erlang well enough to work on it.

It seems like it would have been better if the web server was written in something a larger chunk of the engineering org could modify so that when people needed changes to it for their project, they could make the changes, get them code reviewed by the maintainer, and get it shipped.

The other concern I have with microservices is that doing any wide-ranging changes is hard. Maybe it's always hard, but microservices seem like they would exacerbate the problem. I feel like I already run into this issue at my current job where we mostly have small (~10k) codebases, where people don't really want to make changes that will require making changes to more than two of them at once.

> It absolutely does if it is only one team in the organization. If the entire organization is using PHP and, let's say, you acqui-hire a team based on NodeJS, unless they are doing something absolutely fundamentally different they should learn PHP and push code in your existing infrastructure.

Substitute PHP with Java, and you've described the situation at my company exactly. The acquiring company had a legacy Java application and a lot of automation invested in making that platform work. The acquired company was a NodeJS shop that was using it long before this article or the comments in this thread would advise (this was pre-npm days). To give you an idea of the numbers, the acquired team was 4 engineers as compared to the 100 engineers of the acquiring company (50/50 split with an off-shore development team). I won't say which side of that divide I was on or go into the full year of culture shock that we went through, but fast forwarding these past 4+ years and now the bulk of the company's main product has been re-written in Node and developers are significantly more productive. Features that used to take months to push out in complex releases using a convoluted process of branching, meetings and tons of arguments are now delivered continually using the Github flow with little-to-no drama and far fewer production bugs/downtime. Our customers have never been happier with us and developers have never been happier to work here. All of this came from the fact that the CMO who advocated for the acquisition supported the small team of 4 in every effort to pervade the small team's technologies and practices across the larger organization. Having been in organizations that performed at a much higher level, he recognized just how much opportunity there was for improvement and recognized that the team of 4 had the vision to create the necessary blueprint for the rest of the organization to follow. It wasn't easy, and most of the developers who were here at the beginning of the shift are no longer part of the company. But it worked...and while a sample size of one is hardly conclusive, I have a hard time agreeing with your point having seen it play out so well in the real world.

I've been on both sides of this very important scenario that truly does matter for this industry (the failure rate of acquisitions is hardly as well known as start-up failures when the amount of dollars wasted - oftentimes publicly traded - is probably larger on these sunk costs), and the VAST MAJORITY of acquisitions result in the larger company overriding the smaller with basically just existing customers sticking around out of little choice and almost everyone disappearing (Palm and HP, anyone?). Do you think a company of 4 being acquired would be able to as dramatically affect a company of 200 engineers? How about 2000? What if the engineers aren't even in charge of the platforms they're required to use? (It's literally defined by what your customers want, for example, in a hosted software shipping company)

I am not doubting that your scenario happens, but the possibility of changing a team of 100 (likely pretty jaded) engineers while certainly difficult is not necessarily what people think of when we're thinking acqui-hire. In fact, I'm barely entering my second decade as an engineer and I'm starting to think that surviving an acquisition intact and with career advancement somehow is probably far more lucky than hitting a start-up lottery jackpot in the first place.

There's gotta be a sort of trend of engineers that have gotten acquired so many times that their specialty now is to be able to scale / re-focus technology stacks and integrate and operationalize them better for other companies. Start-up companies typically want to see engineers that have a history of building stuff fast, growing rapidly, and the usual stuff that people get glory for as engineers. Established companies really aren't as picky. There's so many companies getting acquired you'd think that there's a niche for transitioning software over by now at least as contracting gigs.

> Features that used to take months to push out in complex releases using a convoluted process of branching, meetings and tons of arguments are now delivered continually using the Github flow with little-to-no drama and far fewer production bugs/downtime.

I have a really hard time believing that Java was the culprit and Node the savior rather than the organizational stuff you mention...

It was most certainly the organizational stuff that was the main problem. But if, as the poster I replied to suggested, the small team and simply started to submit Java code instead of their Node code, that organizational stuff wouldn't have changed.

Much like a change of location can help break someone's self-destructive habits, the change of platform helped break a lot of the toxic organizational habits that had built up over the years. The shift could have been to many other platforms. And if the platform had been something other than Java, a shift to Java could have improved the situation as well. The important part was that the new mindset and practices around more frequent/frictionless development and delivery.

I do think that it's easier to have that mindset when you use Node rather than Java, but Java has gotten better in this regard over the past few years.

You're talking about an organization that has an existing infrastructure that is bad. This thread is about an organization that has an existing infrastructure that is good but not 100% optimal for NewProjectX, and whether or not it makes sense to use a a better fitting technology for NewProjectX.

Yep, what's "innovative" for one person or team is "boring" for another, and vice versa. NodeJS / PHP is a good example.

Reply on news.ycombinator.com