Hacker News Reader

Comment by aaronbrethorst | Hacker News Reader

Every time I spin up a new project, I try to answer the following question honestly:

    "Am I using this project as an excuse to learn
    some new technology, or am I trying to solve a problem?"

Trying to learn some new technology? Awesome, I get to use one new thing. Since I already understand every other variable in my stack, I'll have a much easier time pinning down those 'unknown unknown' issues that invariably crop up over time.

Trying to solve a problem? I'm going to use what I already know. For web stuff, this'll be a super-boring, totally standard Rails web app that generates HTML (ugh, right? How last century), or maybe a JSON API if I'm trying to consume its output in a native app. For mobile stuff, this'll be an Objective-C iOS app.

Waffling about it and saying 'well, I am trying to solve a problem, and I think maybe a new whiz-bang technology is the best way to do it' is the simplest path to failing miserably. I've watched incredibly well-funded startups with smart people fail miserably at delivering a solution on-time because an engineer was able to convince the powers that be that a buzzword-laden architecture was the way to go.

You don't know what the 'right' solution is unless you understand the tools and technology you'll use to deliver that solution. Anything else is just cargo-culting.

View on HN

Comments here are geared against picking a technology just because it is brand new and exciting, but sometimes you need to pick up something that is just different from what you or your team know well.

In a project I worked on once, we went with "what we knew" (standard normalized SQL schema) to build an analytics engine. The problem with "going with what you know" is you are likely to badly reinvent well-established patterns. If we had stop for a minute and learnt about star schemas, the project could have ended in a much better shape than it did, and maybe the effective time to release would had been shortened.

BTW, learning "new things" is almost always useful but isn't always precisely exciting. Data warehousing is one example :-).

Becoming proficient with a selected set of technologies is still a good idea, but I'm willing to learn and try new things all the time. First thing I ask myself is if a problem was already solved by someone else, and how.

FWIW, maybe the approach in this case is traditional normalization to get the product out the door. When it becomes untenable, hire someone for whom star schemas are boring.

Put another way, the star schema was known to be a better approach only after the fact. Had your team researched “exotic” (unfamiliar) approaches early on, there is no guarantee you would have landed on a star schema.

Eehhh... I knew about star schemas already, although I've never used one. I do think my cursory knowledge would have caused me to research them when starting a project like that though. Obviously it's all got to be a balance, but I think that the OP is probably a little bit too conservative.

Especially if you are working for other people, we are paid to innovate, and we are paid to learn stuff until it is boring. We've got to stick up for ourselves and learn on the job when we can. One new thing a project sounds great to me.

That's a great approach. By restricting yourself to one new thing, you can evaluate it in isolation.

Then, when you make your decision about that new thing, you'll know why you like/dislike it. Your decision won't be clouded by arbitrary things like libraries inter-dependencies.

> I'm going to use what I already know.

Using what you already know may not always be the best approach, because of "When your only tool is a hammer every problem looks like a nail" phenomenon. I would do my research and use what makes sense and what is best in the long run, despite of me being experienced or not in the technology. When you're a software engineer with years of experience under your belt, picking up the next one will not be a big challenge.

Yes, I think this is where experience comes into it. Knowing when it will likely pay off in the long run. As a junior its difficult to differentiate between hype and genuine usefulness.

I generally agree with your post, but I think there is a critical difference between your argument and that of the blog post. Of course teams are more productive with technologies they know, but that isn't necessarily an arbitrarily-defined "boring" technology.

To pick on one specific example in the post: Node.js is popular enough that there are lots of teams and engineers that are most comfortable and productive working with it. For these teams, choosing Node certainly wouldn't cost an innovation token, while deciding to build some service in Python, Ruby or PHP (if we take at face value that this is more "boring") may end up being more costly.

Yep, what's "innovative" for one person or team is "boring" for another, and vice versa. NodeJS / PHP is a good example.

> For these teams, choosing Node certainly wouldn't cost an innovation token, while deciding to build some service in Python, Ruby or PHP (if we take at face value that this is more "boring") may end up being more costly.

It absolutely does if it is only one team in the organization. If the entire organization is using PHP and, let's say, you acqui-hire a team based on NodeJS, unless they are doing something absolutely fundamentally different they should learn PHP and push code in your existing infrastructure. This way you have one way to deploy, one type of application server to support, one set of gotchas relevant to your domain, one set of QA tools etc. Building good products is about far more than just shipping the product, it's also about the cost of long term support. Because what you are doing is fundamentally automation, the less you have to manage the more benefit of the automation you are getting, the more you can forget about it and focus on shipping other things.

What you are describing is pretty much definitionally local optimization and is exactly what you shouldn't do in large engineering organizations.

Depends on your increment of isolation. This is, in theory, why microservices with APIs mean that it really doesn't matter. As long as there are sufficient hire-able engineers who know that technology, it can be used.

> This is, in theory, why microservices with APIs mean that it really doesn't matter.

No it still really does matter, because if your company needs to deploy those microservices in different ways then you need more people to support the deployment infrastructure. If you need to test those microservices in more ways you need more people to support the testing infrastructure. For engineers it's the amount of friction felt when trying to move around and work on new and different problems in your company because they're one of five people who knows how the hell the Java infrastructure works.

If you have an existing testing and deployment infrastructure etc. your team gets those for free and doesn't need to reinvent (and support) those wheels.

> As long as there are sufficient hire-able engineers who know that technology, it can be used.

Yikes. Hiring and firing engineers is pretty hugely expensive and something you want to help avoid having to do.

> No it still really does matter, because if your company needs to deploy those microservices in different ways then you need more people to support the deployment infrastructure.

This is what PaaSes make a non-problem.

I should know, I worked on Cloud Foundry Buildpacks.

Here's how to deploy the PHP app:

    cf push your-php-app

And the Nodejs app:

    cf push your-nodejs-app

And hell, why not a Ruby app too:

    cf push your-ruby-app

And let's not forget that Python microservice:

    cf push python-code-works-the-same-way

We also kept up with the cool kids:

    cf push your-go-code-with-a-godeps-file

And for the "boring" crowd:

    cf push your-java-app-too

In general, these are all intended to Just Work™.

You know how making these surprisingly unalike systems deploy identically is really hard?

So does Heroku, from whom a large body of Cloud Foundry buildpacks code is derived. So did we, when we found issues specific to making code that assumes a connected environment work in a disconnected environment.

The point is, if you're doing this all by hand, you're doing it wrong. You should rent or install a PaaS and move along to the part where you create value instead of inventing a cool-sounding wheel.

> The point is, if you're doing this all by hand, you're doing it wrong. You should rent or install a PaaS and move along to the part where you create value instead of inventing a cool-sounding wheel.

That makes a lot of sense for small organizations, but I'm sorry PaaSs absolutely do not scale to the needs of many organizations.

> In general, these are all intended to Just Work™.

My emphasis added. When they don't Just Work it's really nice to own that infrastructure and be able to fix it yourself. It's also nice to be able to tailor things more specifically to your needs. Again, I agree that owning that infrastructure is not the right solution for organizations of all sizes, but neither are PaaSs.

> That makes a lot of sense for small organizations, but I'm sorry PaaSs absolutely do not scale to the needs of many organizations.

Outside of the giants who rolled their own because there was nothing around in the early 2000s, who?

> When they don't Just Work it's really nice to own that infrastructure and be able to fix it yourself.

Cloud Foundry is specifically designed to run either in the public cloud, the private cloud, or both. You can get it hosted it from Pivotal or IBM, amongst others.

The work of my peers and I made that possible.

> It's also nice to be able to tailor things more specifically to your needs.

Cloud Foundry is opensource and the IP belongs to an independent foundation. I am personally aware of at least two companies who have private forks of buildpacks because that suited their extremely precise requirements. It took them about two developer days, tops.

And their modified buildpacks also Just Work™, because they're based on a robust design that Just Works™.

That's really cool, I did not know about that. You were talking about PaaS and Heroku, which I don't think is the same as forking an OS project and owning it yourself, which is fine for what I was talking about. I don't think it's difficult to name companies for whom Heroku is not appropriate. Regardless, it's all about where you draw the line, gradations not black and white.

I still stick to my main point: your organization gets a massive benefit by all using the same toolset. If you are using Cloud Foundry, I'd still suggest the whole company stick with one language, one deployment infrastructure etc.

To be clear, if you're google I'm not suggesting the entire company all be forced to use one language or something. In that case your company is likely working on products that are different enough that it makes sense to do away with some global optimization. Some judgment is obviously required. But if you're in the sub-500 range (which the vast majority are) it makes a lot of sense to really optimize globally with your toolset, even if deployment infrastructure is relatively easy to setup.

PS I love that you are using the phrase Just Works - the company I work for is called Justworks :)

I mentioned Heroku for two reasons. First, they are the pioneers of public PaaSes. Second, several Cloud Foundry buildpacks are extensions of Heroku's buildpacks.

I think that the nice thing about something like CF is that a whole range of problems just goes away. On the other hand, as Weinberg observed, when you solve the worst problem, the second worst problem gets a promotion :)

Cloud Foundry doesn't get much buzz on HN. But I'm a one-eyed bigoted fan, so I mention it whenever I can. I'm actually a Pivotal Labs employee, my main work is agile consulting. But I've seen enough gigantoglobomegacorps who are choking on their own impossibly heavyweight deployment/ops mechanisms that I am a bit of a bore about talking up Cloud Foundry.

It definitely sounds awesome. We've got a pretty simple deployment system at the moment and so it's a solved problem for us, but when it starts to breakdown we'll definitely take a look at CF.

No worries.

Sorry for being snooty up above. When you work hard on a product it's easy to become emotionally invested.

> When you work hard on a product it's easy to become emotionally invested.

Know that feeling :)

I haven't worked in a specifically microservice environment, but I am currently working in a company that has quite diverse technology choices. One of the problems we have is that we're pretty small, and sometimes there is a lot more work that needs to be done on one component than another, in those cases you can't really have people who are relying on the work help.

E.g. we use erlang/cowboy for our web server and when there are bottlenecks changing that, they pretty much fall on two people who know erlang well enough to work on it.

It seems like it would have been better if the web server was written in something a larger chunk of the engineering org could modify so that when people needed changes to it for their project, they could make the changes, get them code reviewed by the maintainer, and get it shipped.

The other concern I have with microservices is that doing any wide-ranging changes is hard. Maybe it's always hard, but microservices seem like they would exacerbate the problem. I feel like I already run into this issue at my current job where we mostly have small (~10k) codebases, where people don't really want to make changes that will require making changes to more than two of them at once.

> It absolutely does if it is only one team in the organization. If the entire organization is using PHP and, let's say, you acqui-hire a team based on NodeJS, unless they are doing something absolutely fundamentally different they should learn PHP and push code in your existing infrastructure.

Substitute PHP with Java, and you've described the situation at my company exactly. The acquiring company had a legacy Java application and a lot of automation invested in making that platform work. The acquired company was a NodeJS shop that was using it long before this article or the comments in this thread would advise (this was pre-npm days). To give you an idea of the numbers, the acquired team was 4 engineers as compared to the 100 engineers of the acquiring company (50/50 split with an off-shore development team). I won't say which side of that divide I was on or go into the full year of culture shock that we went through, but fast forwarding these past 4+ years and now the bulk of the company's main product has been re-written in Node and developers are significantly more productive. Features that used to take months to push out in complex releases using a convoluted process of branching, meetings and tons of arguments are now delivered continually using the Github flow with little-to-no drama and far fewer production bugs/downtime. Our customers have never been happier with us and developers have never been happier to work here. All of this came from the fact that the CMO who advocated for the acquisition supported the small team of 4 in every effort to pervade the small team's technologies and practices across the larger organization. Having been in organizations that performed at a much higher level, he recognized just how much opportunity there was for improvement and recognized that the team of 4 had the vision to create the necessary blueprint for the rest of the organization to follow. It wasn't easy, and most of the developers who were here at the beginning of the shift are no longer part of the company. But it worked...and while a sample size of one is hardly conclusive, I have a hard time agreeing with your point having seen it play out so well in the real world.

I've been on both sides of this very important scenario that truly does matter for this industry (the failure rate of acquisitions is hardly as well known as start-up failures when the amount of dollars wasted - oftentimes publicly traded - is probably larger on these sunk costs), and the VAST MAJORITY of acquisitions result in the larger company overriding the smaller with basically just existing customers sticking around out of little choice and almost everyone disappearing (Palm and HP, anyone?). Do you think a company of 4 being acquired would be able to as dramatically affect a company of 200 engineers? How about 2000? What if the engineers aren't even in charge of the platforms they're required to use? (It's literally defined by what your customers want, for example, in a hosted software shipping company)

I am not doubting that your scenario happens, but the possibility of changing a team of 100 (likely pretty jaded) engineers while certainly difficult is not necessarily what people think of when we're thinking acqui-hire. In fact, I'm barely entering my second decade as an engineer and I'm starting to think that surviving an acquisition intact and with career advancement somehow is probably far more lucky than hitting a start-up lottery jackpot in the first place.

There's gotta be a sort of trend of engineers that have gotten acquired so many times that their specialty now is to be able to scale / re-focus technology stacks and integrate and operationalize them better for other companies. Start-up companies typically want to see engineers that have a history of building stuff fast, growing rapidly, and the usual stuff that people get glory for as engineers. Established companies really aren't as picky. There's so many companies getting acquired you'd think that there's a niche for transitioning software over by now at least as contracting gigs.

> Features that used to take months to push out in complex releases using a convoluted process of branching, meetings and tons of arguments are now delivered continually using the Github flow with little-to-no drama and far fewer production bugs/downtime.

I have a really hard time believing that Java was the culprit and Node the savior rather than the organizational stuff you mention...

It was most certainly the organizational stuff that was the main problem. But if, as the poster I replied to suggested, the small team and simply started to submit Java code instead of their Node code, that organizational stuff wouldn't have changed.

Much like a change of location can help break someone's self-destructive habits, the change of platform helped break a lot of the toxic organizational habits that had built up over the years. The shift could have been to many other platforms. And if the platform had been something other than Java, a shift to Java could have improved the situation as well. The important part was that the new mindset and practices around more frequent/frictionless development and delivery.

I do think that it's easier to have that mindset when you use Node rather than Java, but Java has gotten better in this regard over the past few years.

You're talking about an organization that has an existing infrastructure that is bad. This thread is about an organization that has an existing infrastructure that is good but not 100% optimal for NewProjectX, and whether or not it makes sense to use a a better fitting technology for NewProjectX.

I do the same thing and usually end up trying to solve a problem using WordPress which is probably cringe-worthy to a lot of people. Most of the things I come up with are basically content-publishing so it works really well for hacking around and making something fun.

My latest creation is a instagram-style feed of the beers a few friends of mine have been enjoying recently.

It works just fine and none of my non-technical friends have cared that they add beers in their mobile browser rather than via an app or something.

> "Am I using this project as an excuse to learn some new technology, or am I trying to solve a problem?"

How about both?

You may well eventually solve the problem, but not on a timeframe that will be useful to anyone besides yourself.

Usefulness to yourself is sometimes the only thing that matters. I'm not trying to be snarky here: your own self-development is not anyone's interest, except yours.

If you're a PHP developer wanting to start some Python, your boss would love you to continue working with legacy codebase using PHP instead of migrating parts of it to Python. At the expense of your future career prospects, of course.

I'm talking more about delivering value and less about self-improvement. Both have a time and a place, but your boss is unlikely to look kindly on you saying "I'm interested in learning Python, so I've decided to rewrite our 1mm user web app in Python instead of fixing the bugs you asked me to, because I think I could make more money with that than PHP."

Actually, as a boss I frequently do look kindly on that. I'd much prefer to have that built into assumptions and explicit upfront.

Sometimes I may say "can we do that on our 1K user side project instead?" but I think that if you are managing developmental resources and you are not allowing them learn new things, you are not following a strategy that is likely to work out long term.

> I'd much prefer to have that built into assumptions and explicit upfront.

How do you build unbounded learning time and an unguessable set of work items into a schedule?

Easy, you spec out doing it "conventionally" and pad with a reasonable first guess to do something unconventional.

Oh we need that in 2 weeks? And its about one weeks work the old fashioned way? Well try something else fun for a bit less than a week. Sometimes you win, sometimes you lose.

Sometimes you can do both in parallel. In the real world there's always wall clock time delays imposed by whatever. So when you're stuck waiting for whatever, learn as much as you can about XYZ till the main line is unblocked.

You adjust the schedule as required and don't do it for projects with tight deadlines.

The truth is if you don't ensure your developers career development is baked into your schedules then you get a combination of high turnover, unmotivated engineers, and engineers who learn on the clock "in secret" by making decisions that are best for them rather than best for the company.

All of these cause far more problems and are harder to account for than being upfront in the first place.

> and engineers who learn on the clock "in secret" by making decisions that are best for them rather than best for the company

Yes, and possibly not even consciously. Without that constant reminder of the pitfalls and learning curve of new technology, it's easy to convince yourself it's all upsides, or at the least undervalue the downsides.

Woot you got minions now!

Not always.

I run an in house database with a Django front end.

Started off with the Django Admin application, which is great for getting things up and running quickly. Its definitely been worth learning more and more of Django, as forcing everything into the Admin gets pretty hacky, and basically generates a lot of technical debt.

Spending a bit of time and learning class based views for example (over the function based views) has paid off, as it leads to far more concise code, and basically less code to mange over the long term.

Today for example I know enough about Django and how the Admin application is built from it's components that I am adding features I wouldn't have assumed were possible a couple of years ago. (Stack Overflow hasn't got a decent solution to the question of a redirecting to a confirm page on save - I'll write one up assuming I get it working).

How much can you learn if you're not trying to solve a real problem, though? My personal experience is that I can dink around with a new technology all I want, and not learn half as much as when I'm trying to apply it to do something real.

Sorry, I was more vague on that initially than I should've been. I've answered this elsewhere, but: what I was referring to was delivering value on behalf of someone else vs. using a given problem as an opportunity to learn something new. Delivering value (e.g. adding a new feature to your product, fixing a bug in a shipping product, identifying and attacking a market opportunity) shouldn't be seen as an excuse to learn a new technology unless you have absolutely no pressure to ship that work on any sort of schedule at any presumed level of acceptable quality.

I'm the same way: I need a real problem to work whether I'm applying skills I know or skills I want to learn. I just don't ever promise anyone (except myself) anything when I'm applying technologies I don't yet know to the problem.

(ugh, right? How last century)

Its not really about a "last century" thing, "last century" most apps need not multiple front-ends like today, unless you are 100% you won't need multiple front-ends, doing a json api from the start simplifies things a lot.

I was being a smartass :)

    unless you are 100% you won't need
    multiple front-ends, doing a json
    api from the start simplifies things
    a lot.

This also presumes that I already know Backbone, Ember, React, Knockout, Batman, Angular, or whichever new JS single page app frameworks have appeared in the last 30 seconds.

Would you agree that it's easier[1] to write a web app that emits HTML than it is to write a web app that emits JSON which is consumed by a single page web app written in Javascript?

[1] Where I define "easier" in this context to mean requiring less wall clock time to get something showing up on-screen and doing something useful.

You don't have to use a JS MVC framework just because you built your core logic behind a JSON API. You can just as easily have a server side rendering app talk to the JSON API instead of the directly to the datastores. You get to stick with 'what you know' while still maintaining flexibility for later.

yes, but it will more complex and take up more time, which I think is what he is talking about.

Last century most apps absolutely did need multiple front-ends. Don't you remember the days when there was a Windows version, a Mac version, a Java version, if you were lucky a Linux version, and maybe a web version for the bleeding-edge developers?

Relatively fewer apps were network-enabled, but even then, there were a bunch of technologies like CORBA, DCOM, and RMI to marshal communications. There were also a bunch of custom binary protocols; back then, it was common for software developers to work on all levels of the stack, and the culture of library re-use was not as entrenched (at least in the enterprise) as it is today.

Great advice. Also extends to teams, 'What do we all know well enough to execute?' should trump 'What would be fun?' every time. I've worked in situations where someone saying 'Oh I decided to write this in Clojure, even though I'm the only one here who knows it and we're running out of cash' cost significant time and resources to fix (the fix was rewriting the project in plain ol' Python myself). It just isn't a sensible risk to take.

Reply on news.ycombinator.com