I Failed to Transform the Enterprise
anonyfox.com
anonyfox.com
I will never, ever, ever, do in-house software development ever again. Non-software companies treat their inhouse developers like crap. They don't get it. I don't know when non-software businesses thought it was a good idea to move software developers in house. Most of the company don't want them there. Or deal with them begrudgingly. It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating. And they are the first on the chopping block when things turn down.
In my experience, if you are recruited by a company that doesn't sell software or software services you should tread very carefully, ask lots of questions. Sometimes you'll find out a 100 year old furniture company only has a dev team because the grandson thought it was a good idea. And then you run very far away.
Edit: I’m confused then: if robots have software therefore it’s a software company, then tons of companies are software companies. But so many of those products, the software is just a hidden detail that no customer ever thinks about or cares about. Where do you draw the line?
(There is an interesting discussion to be had about where the boundary between "software" and "hardware" companies is, but an environment with a high level of wilful misreading isn't one of them)
Appliances are an obvious example. But also things like cameras to a large degree. Or auto infotainment systems until Apple and Google software was integrated. (Embedded is sort of a different category that is functionally essentially part of the hardware.)
The software output at such a company is confused at best. Usually with some exec pitching digital transformation without definition. No one wants to deal with the tech team, pay and expectations suck, getting hardware can be a years long process, and eventually someone pulls the plug.
By that definition google is not a tech company, their product is selling advertising space. Tech backend just enables that.
Or Netflix. Certainly not selling tech or software, they rent movies.
For most companies building the physical product is a loss leader, what they’re selling is their custom software.
- companies for which software is the sold product (or its support, hosting...)
- companies selling a service or a product that requires inhouse made software
It could be argued that this second one could be split in two as the software may not be delivered directly to the client but the border is a bit blurry when it is a website/app platform like netflix.
So generally, IT vs product development.
I have been on IT projects (internal development) tied to revenue. It was pretty sweet while it lasted.
The problem is most consultants are not good at what they do. It why the sales guys for the consulting company are not the devs. Most the time consultants come in for 3-6-12 months, cause some trouble and then just leave the mess to the company to address later. There are some good consultants but most are just people that couldn't work in house because they are not up to par for long term engagements of products that need to stand the test of time to add legitimate value.
> It's hard to quantify their revenue so they are treated as an expense not cost saving or revenue generating.
This depends on the business, most the time, unless you make software you are an expense. The company makes their money in different departments and making money is the name of the game. It's takes a different set of skills (albeit better/stronger, industry knowledge and communication skills) to be an in-house developer vs a contractor.
The challenge here is that under this model the different departments get credit for the revenue they generate. What their relationship is with the in-house dev team is very much something that one should be aware of if they choose to be in that situation. It's not too dissimilar a situation to every single 'change management' model that has executive commitment as the number one success factor.
/r/oddlyspecific :)
Me: So what is it you do here. The job description says you are looking for a developer?
Them: Oh, we make and sell boxes.
Me: Like the cardboard boxes you get delivered?
Them: Yes, exactly!
Me: Okay, so where does software fit in that? Are you aware of development cycles? {Other interview questions I can't remember}
Them: Oh, we don't know the technical part too much. We want to automate some of it.
Me: ...
Software developers at software companies/tech companies generate profits, so they are treated as such.
Software developers at more "traditional" businesses (retail, industry, manufacturing, etc) are often seen as an expense and something that should be treated like any other expense - try to minimize it.
> The amount of customization by far exceeded the initial offering of the services, and had to be done within the constraints of each individual technology stack. > > This accumulated in hundreds of partially overlapping systems, wired together with brittle ad-hoc integrations.
They had a software solution and were officially spending zero dollars on software development. You can't beat that. Never mind the huge amounts of time they spent keeping it working. All of that money showed up in other places on the bottom line. Maybe a lot of it was IT, but a lot of the expense was probably people whose job title had nothing to do with technology spending hours doing repetitive and error-prone integration work with third-party systems, emailing around spreadsheets and CSV files, manually tracking down and fixing bad data, and answering the phone for confused customers. As soon as you start employing software developers to create a more efficient system, it looks like you're spending 100x more for software, on a trajectory to take a very, very long time to achieve parity with the existing system.
I do think this is valid, but feel I should add: that willingness needs to go both ways. You were willing to put in the extra work, and your employer was willing to "get it".
There exist organizations that will never "get it" despite the extra work, and these are the places others are talking about when they say "run away".
I experienced both variations of this earlier in my career. I was successful the first time, and tried to apply that success in a larger environment.
The lack of organizational willingness to "get it" in that second environment is what motivated me to move to a place where building software is the primary reason for existence. I do not intend to look back.
Managerial debt can happen anywhere, including tech firms. It's where you're dealing with people who don't understand what recent technology can do. Maybe they got it 10-50 years ago. Maybe they were very good at it 10-50 years ago. But today they're behind the curve and unwilling or unable to understand that a working modern solution is possible.
They may also be risk averse for perfectly rational reasons, in which case you need to sell damage management security promises and fall backs with any upgrades.
The bonus issue is a very real problem. If there's pushback from that level it's often because those people can see dollars moving from them to the newcomers. And they'll do almost anything they can to prevent that.
These are very difficult issues to deal with, because they're usually hidden behind spurious excuses and rationales and passive blocking actions.
If you break down what most vendors and consultants do, in terms of either dollars or hours, you'll usually find that they mostly do sales, PR, client relationship, etc. Projects are designed and managed to make the right people look good to the right people, manage expectations, etc.
They'll spend massive time giving non-tech executive buyers easy, fun choices.. let them feel strategic, visionary and empowered. They'll communicate the hell out of every piece of progress. Whoever hired them also has a massive incentive to buy into and sell onwards, that the project is a massive success.
IE... if your job is to make a clients' Salesforce instance do a thing, 80% you your job is stuff other than making a clients' Salesforce instance do the thing.
Internal dev teams will rarely have this pageantry. Meanwhile, no one outside that team knows what they do, how well they do it or such.
They spend 2x more on selling than they do on R&D.
So when that Oracle Sales Guy is giving you the pretty talk, and taking you to lunch remember that all that money your sending to Oracle is literally mostly going to him! The product makers just get 'a cut'.
Now consider how much of high tech is just that, sales and marketing, conferences and luncheons.
That's a staggering waste for a developed economy.
We are very inefficient with these things, we need better approach to IT.
I question the wisdom of somebody who started refactoring and replacing systems right after massive layoffs in an industry that was crippled by the pandemic.
Businesses in survival mode can't spend resources like this. The author points out the executives became enraged when the found this out and I agree with the executives.
Well wisdom is gained from experience and it seems the author learned a few lessons.
You agree with their 'enraged' state? You think it was justified? Even with hindsight, or just at that moment in time?
> refactoring the old things we produced into the new technology stack.
You don't see how that could fix things?
As a rule, when things are broken it's the best time to go out fixing them. But yeah, this has a non-zero cost that maybe can destroy the company.
1. I worked with a single internal guy to build a web-based ordering system, in the days where people faxed in orders. This replaced weeks of data entry, dozens of part time data entry people, and enabled a big change in materials ordering and production (orders were all in a database within a few hours, vs literally weeks of keyed-in faxes from before).
The one guy that was in charge of all this suddenly had a lot more power internally, and it pissed off a lot of other folks who suddenly had less influence.
2. More recently, worked with a team of external contractors, and internal folks were resistant to help any of us set up external dev systems. They all worked in the physical offices, and had no need for 'run a dev system on a laptop'. One of our guys spent a lot of time (and kept getting chastised by the client's mgt) for getting and documenting a build process for external contractors to use. The pandemic hits, office buildings are shut down, and internal folks have a much harder time figuring out... how to do anything from home.
In both cases, under-the-radar skunkworks projects ended up having large (and in one case transformative) impact and value, but were ignored or fought by those with entrenched power.
For decades now, per Alistair Cockburn, I assumed people just really, really hate change.
There's some research which shows our selfish little brains have an immunological type reaction to change. Anything that threatens our internal stable equilibrium is squashed like an invader. No matter how much we say we want to change. No matter how much we may need to change.
While wholly correct, turns out Cockburn was an optimist.
We now know that some fraction of people would rather watch the whole world burn than give up one iota of power.
"When there just is no interest in actually improving things, I should leave it at that. "
Yep, if you can't convince them to care about and drive the improvement themselves, you're gonna have a hard time if driving it becomes the most important thing to you.
This is all very vague, so who knows.
But at the very least, I would guess the OP's attitude of I-know-better-than-everything-that-came-before arrogance plus ignorance of why things were the way they were made big failures inevitable.
It's a real limitation -- though a common one -- to need everyone to see things from your perspective, rather than being able to understand their perspective.
Generally its great, but not a panacea, and while it would have been improvements it would have suffered fun new problems.
Source: been the young guy who thought if only they listened, and then they did.
This is why.
The reason companies like this use consulting companies is because the VPs want total control. Internal IT is purely a cost center, and unless your company (read: the business) is already on board with the product lifecycle op model, there is no point in trying to do any of this from within IT. Consultants work at the VP level and in a large enough company, we find a technical area (usually data as it has the biggest business returns) to modernize, then leverage success there build out the business case for the rest of the company (data is a good place to start building API gateways). And selfishly, the business is willing to pay higher hourly rates than internal IT (which we generally leave to the offshore MSPs).
When I sell transformational solutions to a company, I know it won’t be successful without top-down buy-in. Consultants are also better positioned to do these transformations because we do them all the time; and the exercises around stakeholder management and consensus building are extremely important. They’re the kinds of things internal teams are never resourced or trained for, and they don’t happen quickly.
Plus thanks to those political games there are lots of soft skills to learn as well.
But he then outlines all his failings at the bottom, which I agree with, so he clearly knows better. It's it possible that this tone deaf attitude was a primary culprit in the downfall of the organization he and his handpicked successor built?
This is one reason technologists can make bad managers. They don't think about the politics or don't want to play it.
That might be true or not, but you can't operate with that mindset and expect people to come along with you. Whether or not it's true, you need to presume positive intent. Perhaps they see aspects of their jobs that won't automate well that you do not. Is that stubborn self interest, or standing up to a naive new leader who doesn't get it?
Every family member who ever turned into a whiny baby while trying to do free IT for them saw themselves as the victim, or the person under threat. That's the kind of person with power in most organizations, and you have to find some way to meet them in the middle without compromising everything to get anything done. It's not enough to be right.
I doubt whether this extends to the politicians / executives enacting those policies. They will certainly frame it like that, but for a lot of them, their underlying thinking is going to be much more self-serving and about manipulating the narrative.
I'd wager a SPA is a hammer/nail reaction here.
Thankfully, the depth of mismanagement here was pretty exceptional in my experience. In most cases, even if I'm personally skeptical of the need for an SPA I can see a reasonable argument in favor. The cartoonish straight out admitting that the sole purpose was to be more "modern" with literally zero regard for actually serving any user needs is thankfully not something I've run across too often.
This is just how company politics works. Those executives wouldn't be in their positions if they didn't look out for themselves first. Also it is turtles most of the way down. This experience should be a good lesson.
Only the larger of non-tech companies can get away long term with internally built tech. This is because only they have the money to build and maintain teams and build and maintain the technical expertise.
Sometimes, it still takes a massive bite in the ass for these giants to wisen up. For example, see Home Depot and their credit card scandal and then their subsequent investment in self checkout, online sales, and lockers. They have done really well for themselves.
Smaller companies eventually either get swallowed up or destroyed by bigger companies, by disruptive new players, or get a new lease on life due to the creation of a service layer below them.
Nowadays those new disruptive players are often tech plays. Also the new service layer is often tech. One example of a tech service layer is Shopify. AirBnB and Uber are examples of industries getting swallowed up by tech. Expedia is another example of disruption from the previous generation. Home Depot swallowing up more of the construction supply industry is an example of bigger players crushing smaller players. Bye OSH.
Do not encroach upon my power, you cretin! It doesn’t matter if the business dies because of my stupidity, as long as I go down captain of this ship!
Edit: It's a legitimate solution. I've survived 3 reorgs at my company because I could code. I read up on the new incoming system and became effective.
As annoying as that is, I kind of feel you should be above that. You have all right to be annoyed and skeptical, but not to sabotage the effort.
> The only system not affected by the complete meltdown was: our new one, which was designed to withstand outages from backend systems below.
I don't understand, how can their SPA frontend still be functional when the backend is totally down? You can display things to the user, but you cannot do anything meaningful from it?
New things go with old things. The people who maintain a product ideally should own the improvement or replacement of that product. That's how you get everyone on board. And if you can't manage to do that, then you'll probably end up in some kind war.
Yet still somehow customers buy the products and (more often than not) the company delivers a product that is half working enough to be tolerated and used.
“Nobody gives a hoot about profit”.
If I had to guess Rust vs Java/C/Perl?
So you’re a developer AND you’re nominating executives? Something about this story or the author stinks.
How could you possibly know that they were "afraid"? More likely, a group of people just had an abrupt re-org with little-to-no input and now had a new poorly-defined internal "digital" group that failed to provide clear direction on how it was going to replace an existing 3rd-party process. If worked suddenly ground to a halt and the new department was the only thing that changed, maybe existing leadership wasn't the problem?
> Without any real support from any other executive, our poor VP was tasked the impossible: digitalize the enterprise... but without any decision making power outside of his own department.
What about the other poor VP's? How did this new digital VP engage with them, individually and as a leadership team, to ensure that this new department had a well-structured process by which it provided value to the company? Did this new department try to lead the agenda, or simply steer it? You should be aiming to provide a service, not expect service from others.
> Executives went full enrage when they heard about what we did there
Well, yeah? This reinforces my opinion that the new department completely failed to engage the company. I'm sure the executive leadership team had more important things it wanted accomplished. How did adding Phoenix LiveView to an existing mess of a stack add value to the company in a way that aligns with existing objectives? Was the digital VP even consulted on this decision?
It's also possible leadership at the top, a CEO/founder, didn't know how to manage an executive leadership team and the dysfunction went too deep for a VP to fix. If the leadership team isn't meeting regularly and on the same page and collaborating with the CEO to determine a roadmap, then something is very wrong. You can't fix bad communication between leadership stakeholders that can't decide amongst themselves what priorities are.
> they wouldn't even come up with such ideas on thir own
You're telling me that a group of people that had been successfully making demands of 3rd-party contractors and companies couldn't do the same of an internal team? I don't buy it. How did they get to such a tangled mess in the first place?
> there are sometimes no other options than doing it as we did
Maybe so, but how did you bring along other stakeholders in this decision-making process? I wouldn't tolerate this attitude from a contractor/freelancer, or a full-time employee. A big part of leadership is convincing others of the value of a certain path, and failure to do that isn't the fault of the other party.
> Without toplevel commitment, best efforts and even results don't matter.
Yes. In the vacuum of clear leadership direction, other people will find opportunity to fill the empty void with work. But separate leaders working in isolation, even creating one-hit-wonders, will never be successful as a team collaborating on even a mediocre agenda.
> When there just is no interest in actually improving things, I should leave it at that.
I don't believe this for a second. Either there was complete absence of top-level leadership, or the new department wasn't structured to service the business and failed to build the confidence of other stakeholders that it could deliver value. I see a lot of blaming the other guy, and very little other actual introspection other than "without including all people affected by this".