Engineers without borders, silos, and vendor walls | Hacker News Reader

Engineers without borders, silos, and vendor walls

rachelbythebay.com

61 pointsWill_Do16 comments

Even though I'm strongly biased towards agreeing with Rachel here, I'm going to throw this out here anyway: in case of 3), you could technically always fire the three subpar vendors and replace them with their better competitors (assuming there are any). Replacing a vendor may be easier than replacing an in-house team, but then again, it may not.

But with vendors come different issues. They can go out of business, or get acquihired. If their product is in any way not aligned to your business needs - and it almost never is - then you have to either throw a lot of money at them so they customize their service for you, or you can alter your business to fit the vendor's offerings (and keep altering it as they iterate on their product). Both options are terribly inefficient compared to just asking your engineers to tweak something in your own codebase.

(And then your country's government and the US government decide to unfriend each other, and suddenly 95% of services you depend on and their competitors can't service you anymore. There's that risk too. See e.g. Venezuela.)

I viscerally hate the idea of a business being assembled purely from third-party services, reducing itself into a pile of contracts. I really dislike the way how our economies start turning into NPM. But everyone keeps saying there are good business reasons for that. So maybe they are. I don't have a formed rational opinion on which way is better yet.

This could at least in part explain some common patterns:

- Non-IT companies overall have terrible IT systems because as soon as they have two systems they need some sort of integration which the vendors are likely to botch and the resident "IT person" isn't qualified to build and maintain.

- "Nobody was ever fired for choosing $vendor" actually makes sense when integration is a black hole for time and money. At least a single vendor is likely to have already taken care of integrating with their own products.

- When people start looking at which parts of an IT infrastructure to improve it's usually the oldest and most botched-together piece. But it is nearly impossible to replace because of all the botched-together custom integration code which was cobbled together by half a dozen barely-qualified staff.

> resident "IT person" isn't qualified

More and more, I think such a phrase should read, "resident management requested too many features without sufficient budget".

> code which was cobbled together by half a dozen barely-qualified staff

Idem, "code copy pasted to satisfy excessive requirements of pointy haired boss" is the more informative statement here

> code which was cobbled together by half a dozen barely-qualified staff

So long as each part of the system can export and import CSV, or similar, then anyone with Excel and enough time can become a Systems Integration Expert

Very frequently, with the amount of churn that one sees today, you can't even get to the end of a deployment with the same people that you started a project with.

I cherish my customers that have company lifers on the other side. Dealing with people that actually know what they are doing and have history and context to draw on is such a luxury.

Multi vendor is even worse than described. I worked for one of the vendors. The incentives make you fight tooth and nail for suboptimal technical decisions that direct money to you rather than the other vendors.

We won the internal bid for a functionality module that ended up being a standalone service called by system X. Vendor X had lost that bid. They convinced the bank that it is absolutely necessary for us to expose a JDBC driver as an interface.

Their engineers were actually pretty cool and we agreed to implement something which made more sense. Unfortunately this decision was blocked. We ended up exposing a simple RPC API via a JDBC driver. I think it had over a thousand methods, almost all of them empty.

This has always been the engineer's problem with sales people. You have sales teams visit you from 4 vendors and they all claim that their product will best solve your problem. This means that at least 3 of them are not being truthful.

Also: most people here probably experienced having to rush some features or deliver facades, because sales team managed to sell something that doesn't even exist in the product, and it needs to be demoed next month.

Given that, consider that when a salesperson from some vendor visits you, they're not likely to be accurate in describing their wares either.

Ah yes the "I heard someone say in some internal meeting that this might be possible to do, so I'll sell it" move.

Yes. Also, "I base my understanding of the product on the bullshit some other sales team wrote for the company page two pivots ago" trick. Or, "a thousand or a billion, it's all the same, who counts all these zeros (and what's that 'scalability' word you keep using?)" move.

"I base my understanding of the product on the bullshit some other sales team wrote for the company page two pivots ago"

is slightly more forgivable than the other things, particularly if you change "sales team wrote" to "marketing team wrote". In a company with more than one or two products it gets very hard to keep track of what a company actually sells, and the marketing material is at least theoretically a customer-consumable list of that.

I was following it up until the vendor FUD. For most of the code your part of the orgs wants to run, chances are, you aren't hiring the world's best dev/prod/designers to dedicate their lives to say your website's anti-spam comment forms -- they're going on the differentiating stuff! For the boring whizbuzz widget, the top vendors in the field will already have centuries/millennia worth of worker hours & experience behind it. And, even if you picked a wrong vendor, should be waaay less political to replace them.

One phrase that I've increasingly come to love is "torpedo in the hull" (https://allaboutstevejobs.com/blog/category/steve-jobs-histo...). It's tough seeing senior folks tolerate vendor-outsourcable projects going to weird internal special projects & artisinaly-managed OSS. We know most folks switch jobs every couple of years now, including those behind that fancy internal project, and inheriting unnecessary code & ops is such a drain. Why encourage planting timebombs?

There are trade-offs to everything, and a lot depends on a deal you have with your vendors. But if anything, it's the vendors that are the actual "torpedo in the hull" - a foreign body in your system, that you have very little control over and that can explode at any moment, leaving you with a vendor-shaped hole in your company. It's manageable when the third party is a commodity vendor - you can replace your printing house or blog host every other week at almost no noticeable difference. But in the world of software services, everyone is doing their damnest to not be a commodity, to make themselves unique. That uniqueness means a larger hole when they go down (or get acquihired), and also stronger ability to extract money from you. A codebase you own won't one day tell you that, starting next year, they're discontinuing the service package that was critical to your business, and now you have to buy two separate service packages plus a custom integration fee just to retain the functionality you grew dependent on.

Right -- this is about commodity stuff. Agreed, asking a vendor to own your non-commodity bits means not owning your core speciality, which adds even higher levels of risks.

A nice thing about vendor-shaped holes in commodity areas is the holes are clear and plenty of vendors happy to quickly fill in. Comparing that to changing some internal project that gets its tendrils into who knows where with who knows what servers & services is night & day. I much rather figure out a Hubspot-to-Wordpress-and-Marketo migration or Zenefits-to-Gusto migration than some weird internal Elixir, Java, & InfluxDB Zeit blog spread over a bunch of AWS services that were primarily developed by two microservices devs who had both left the company to their next great thing over a year ago.

A different perspective from a dev shop owner.

Our clients do outsource custom development primarily due to cost-saving reasons. Being non-tech companies it is very difficult to justify high IT project costs. Competing for qualified talent is no fun as well which leaves them with internal IT whose primary job is to keep existing things running and that's it.

For them, it is a choice of:

A) Outsource development of the 3 projects and actually have the 3 projects completed and bring business value

B) Hire 3 teams of engineers, managers, UX, whatever, and keep them salaried

C) Do not have the 3 projects at all

The budget often allows for initial development only, making option B non-viable.

This is a good example of Coase's "The Nature of the Firm" [1] as applied to services.

(The question, though, is why the three badly-performing services aren't replaced.)

[1] https://en.wikipedia.org/wiki/The_Nature_of_the_Firm

Reply on news.ycombinator.com