What Silicon Valley gets about engineers that traditional companies do not
blog.pragmaticengineer.com
blog.pragmaticengineer.com
One of the things I worry about is that even at companies that are doing "Agile transformations" and adopting methodologies like Scrum, in practice have "Product Owners" who are there to give instructions via Jira tickets. Other roles like "Business Analysts" are there to ensure that lowly developers never have to worry about things like understanding the business themselves.
So I get funny looks when I suggest that engineers should go talk to people in the business, and write design docs. Silicon Valley engineering practices can look very non-Agile to a lot people from traditional companies that are doing Agile.
The ironic thing about Agile is that it has a cottage industry of "scrum consultants", "scrum masters", project owners, etc. which advocate ostensibly for letting developers self-organize but which also has a vested interest for career and self-preservation reasons in inserting itself above software engineers in the decision making process.
I see this position as a critical component of any agile strategy that seeks to maximize scrum master productivity, while still allowing scrum masters all the autonomy they need for their day-to-day duties.
his holiness will make his presence known in due time
suppose I should have guessed there were already layers above and beyond the scrum master
it's like the leaning tower of bullshit
(Credit card required. Rebilled at $29.95 per month until cancellation. Minimum period of 12 months. T&Cs apple.)
We plan 8 hours a day of work for developers, but they spend 4 hours in meetings, 2 hours fielding questions and just 2 actually working.
When applied to in house engineering teams, it is largely ineffective if it doesn’t come with the same kind of freedom that highly paid software consultants get. This is manifested concretely in how many companies have corrupted agile methodologies by focusing too much on the perfecting practices (or “ceremonies”) rather than focusing on the foundational principles.
It's regretful that it caught on so well and has (incorrectly) evolved into a concrete process in most places rather than an ideology.
I suggested instead engineers get looped earlier into the process with stakeholders. The director said it was a bad idea as we couldn’t possibly hope to understand the business side of things.
Very glad I got out
The article and the conversation here suggests that the Silicon Valley model is a good way to give developers more leverage and also to get more out of them.
That model is generally far from cooperatives and still has plenty of hierarchy.
Getting 'organised' and into cooperatives might still work, I don't know? I'm just saying that the available evidence here points to a very different direction.
(Cooperatives have been tried, and there are some software companies inside and outside of Silicon Valley that work along similar principles. But by-and-large they aren't the companies eating the world.)
Matt Levine's Money Stuff often harps on the theme of investment banks being run like workers' cooperatives in practice.
I don't think a company's purpose is necessary to eat the world but that's going into opinions so I will refrain.
It asked why couldn't development firms be organized like how doctors or lawyers often organize themselves, as partnerships with the PMs, etc like nurses or paralegals.
In some jurisdictions, there are limits that make it harder for outsiders to employ doctors or lawyers and sell their services like you would employ programmers.
(In eg Germany, that even applies to pharmacists.)
Just because anyone can be an engineer, it doesn't follow that we can't form partnerships. I could be wrong, but those doctors and lawyers who are in partnerships don't seem to be using that structure due to lack of employment opportunities under other structures.
Another argument I heard was that partnerships are required in certain fields due to liability concerns so that risk can be contained to individual partners. But this only explains why they must, not why we can't.
A big part of the regulation of doctors and lawyers is the part that keeps the competition out. Google or your favourite startup can't just train up a few neural networks and start dispensing legal or medical advice; even if that advice was much better than what you'd get from your median human lawyer or doctor.
The organisation into partnerships might be more about who's excluded, than what the people in the club are allowed to do?
You are right about liability concerns playing a role, too. And then there's also tax issues.
I agree. I just mention it because the companies that 'eat the world' will be the ones people have heard of. And those are not collectives.
Accepting the value of that behavior requires a great deal of personal and organizational maturity, especially when you want to try something out quickly and your developers refuse.
Sample phrases include: "Why did the customer ask for that - it sounds like our planned implementation won't satisfy their goals", or
"Won't this change mean we operate at a loss to subsidize your other company, which will affect staff bonuses".
I don't get it. Can you explain?
Staff at company A have negotiated a profit share / performance bonus.
Boss directs company A to provide services “below cost” to company B. Company A now has no profit to share; it has been funnelled elsewhere to avoid paying the staff their bonus.
It has slimy tax advantages too, for the company.
If you're getting profit sharing, you should be able to see the books, and have controlling shares, too... IMO.
I’ve never considered it from that angle.
Brilliant.
I do think SV puts more trust in their engineering teams to produce polished software, whereas other companies will burn through tons of dev resources over nit-picking or implementing speculative edge-cases.
The devs still tend to have to ask a lot of questions to get anywhere useful but in these situations the P* roles at least give them some traction to work from and get movement.
If you've got good business analysts, this is what they're supposed to be doing.
I think if you have great product people, great business analysts, and great developers, the system of PO-Analyst-Developer can work. The problem is that BAs and product people aren't generally making a ton of money, so the great ones will move on to more lucrative positions, and great developers get bored quickly if they're just given a list of JIRA tickets. So pretty soon, one of the three things breaks down then all hell breaks loose. The product folks don't have a clear business vision. The analysts are soft technically and don't know what questions to ask. The developers are either building a space shuttle when they need a bike, or they don't know how to build the bike in the first place.
The best engineers I've ever worked with and for were constantly asking "why" and would get incredibly frustrated in this "Product Owner runs the backlog" type of organization.
Edit: details. Our devops allies itself with management and kept “business secrets” from developers. It was abbhorant to witness. I eventually introduced functionality into the devops tools and openned it to developers and got bullied (2nd edit: by my own team and management, i was soft bullied: silence treatments, secret meetings without me, lies, etc. i was junior. My own confession: i did the work without discussing it with the team after being told “fuck the devs” by a senior.) to quit.
However, because developers often don't actually want to interface with customers and just want to build cool tech, and charming socialites are very glad to act as that interface, this usually works. As a secondary effect they (non-intentionally) obfuscate the inner workings of the company from the devs who would otherwise (as the GP points out) realise how much they are being (ab)used and turn the screws on the upper management.
If you look at things that are actually hard, nobody keeps you from getting the info. You're just not going to understand that graduate seminar in ergodic theory, though you're welcome to attend.
The reason they keep you from the info is of course you'll see the naked emperor.
Science is "easy" because we can model it, to a degree.
Non-science is so hard that we can't even model it, it's that complicated. So instead we try to voodoo our way around it, rely on simple heuristics, history, etc.
But the man who invented 1-800-GOT-JUNK and plastered the phone number on the side of every one of his bins was not some better of Richard Feynman. He understood one thing really well: If you solve a problem, have simple messaging, and unabashedly self-promote you will win.
The lawyers, and accountants, and technologists, will all come and help you with the complicated stuff in exchange for a piece of that mountain of cash.
I think one has to differentiate between "complex" and "complicated". Complex stuff requires Big Brains. Business is not complex, all its processes are simple and fairly well-understood; but it is complicated by the interaction of so many factors.
That said, I do always try to bring along "business people" in engineering challenges and considerations, for which you don't need a full engineering degree. I do appreciate when I get the same treatment.
And what kind of "high level physics" do engineers know? A typical masters track doesn't cover the hard maths like in GR, sounds more arrogant than anything... kind of proving my point ;)
If you have ever been in a relationship, you know how irrational people are: ergo, ability in logic and mathematics plays little role.
Went to MIT in engineering/science, and I would do a mental face-palm when I heard other engineers try to ‘sell’ what they are working on.
It’s the “curse of knowledge” where you forget how to relay ideas because you forget your axioms have been updated but the person you are talking with has not had theirs updated.
This is to say some people (in all fields) can sell ideas, and some can’t. Just as some can do high level mathematics, and some can’t. Those curves overlap in unexpected ways.
This goes a little further beyond this point I was making, but I also see collective decision making like this as being more fair. Everyone at the buisness is affected by these decisions, so everyone should share in the decision making process. Natural leaders and experts should become apparent and people will listen to them if they don't attempt to bludgeon them with their credentials.
For me, the engineer is actually the only person who can do those business functions properly, if the business is solving new problems (rather than a cash cow). I tolerate people who aren't engineers, because I often can't avoid them, but once you get more than a few of these people in your business, things get awkward for the actual engineers.
I also tend to think of engineering as more than just coding. An essential part of it is communicating. You can't do that without technical skills, there's just so much friction.
Aka "Wagile".
"We have a standup every morning, where you're required to report on how many of your assigned Jira tickets you completed yesterday and get berated for any incomplete ones no matter the cause! No excuses! We've committed to these deadlines and budgets, that you know nothing about having not been consulted on anything."
When I worked for a large multinational in SF, 100M+ users we operated very much in the “SV” style the article describes.
I got to see how our satellite offices in other parts of the world operate; EVERYTHING was in calendar, work was tracked in 15 minute intervals, meeting about meetings, and follow up meetings after the meeting, and a post mortum after. One project manager for every 3 engineers. Dealing with them I felt like I was looking some sort of Project Managers Gone Wild porn set
Then I moved away from the BA and found out most places are like this, it’s real bad
Double talk.
Plus, most implementations of Scrum are bullshit. Scrum does not recognize the role of a "project manager". There's the product owner, the scrum master and the development team. That's all.
The scrum master only exists to guarantee the process is followed. The scrum master is just a scrum evangelist, not a real leader.
I still can’t figure out how line managers and architects fit into a well run scrum environment. Seems they would be losing a lot of power.
You can read the official Scrum guide here, it's a few pages long:
https://www.scrumguides.org/docs/scrumguide/v2017/2017-Scrum...
I think Scrum exists only to expand the adoption of Scrum. That is at least the reason behind the role of "Scrum master".
By maximizing adoption, the authors can personally benefit and become rich and famous.
FAQ
Q: What are you working on?
A: Look at the issue tracker
Q: Are you working on X?
A: Look at the issue tracker
Q: Are you still working on X?
A: Look at the issue tracker
Q: Is X finished?
A: Look at the issue tracker
Q: Did QA look at X?
A: Look at the issue trackerThen wait for 30 min, do the same. Then wait 3 hours, do the same.
What will happen? they will forget about it because it is irrelevant to them.
Therefore the scrum ceremonies are just bullshit and a waste of time.
Quod erat demonstrandum. Fuck scrum.
I never wanted to bang my head against the monitor harder than in that moment.
https://www.questionpro.com/blog/nps-email/
(I don't know anything about this company and am not endorsing them, just noting it as a reference.)
I've seen this at many companies and its one of my biggest pet peeves. Almost nobody uses it correctly. They don't net anything. They take the average. It's not clear at all why it's any different from any other score. I think most people think its some sort of collection or measurement thing rather than a calculation on the data. I hate it. Reeks of innumeracy.
Not that it really matters if you use average vs. nps...
I do agree that if this process is used to score political points though, then it loses all value. Maybe a compromise is to capture the summary of these conversations somewhere. But honestly if an org has become this political.... I would start looking elsewhere.
How many of those recordings are ever played back? None, in my experience.
Public slack threads are pretty great since you can follow the decision making process when solving problems. Great slack threads may even include data on what the author tried/didn’t try etc.
I agree with you that video recordings are for the most part useless. Until we find a way to get high quality automatic transcription of videos and add them to a good search index, they will have limited use.
Slack conversations often contain more data. It’s also easier for me to follow a human conversation than a poorly written summary with many missing details.
If you have < 5 customers, then this works fine.
If you have > 10,000 customers, then this works fine, because you can rely on analytics.
If you have 100 - 500 customers, then it kind of breaks. You can't rely on analytics because those customers are too important, and you can't rely on 1-1 interaction with engineers because there are too many customers to consult on each change.
I've gotten more than funny looks. They get downright mean or irritated, because to them it sounds like you're trying to take their job away.
Remember, these are the people who bring the requirements to the engineers.They have people skills! Can't you understand that? What the hell is wrong with you people?! https://www.youtube.com/watch?v=fcIMIyQnOso
This is of course because the thing that traditional companies call "Agile" is really just the same business-as-usual top-down manager-controlled garbage software process they've always used but with the cool-sounding name "Agile" slapped on it.
Whereas what SV companies do is much closer to what Agile is really about, regardless of what they call it.
I was a developer brought into a team along with codebase. They would allow their own devs to go full SV-style, free. But when it comes to me; "please check with PO, create a jira subtask for this"
any notion that engineers should not be on customer calls, or driving engineering specs is the opposite of what i would consider the best implementation of agile.
Having more engineers in decision making positions certainly helps, but I think it boils down to whether the organization is set up for trust.
A good product owners facilitates the communication between business people and engineers (or designers, or others). They don't dictate instructions, they regularly spend time with business people to understand their needs and what they care about, work with engineers and designers to craft solutions that are the best suited to what the business needs. When the relationship works well engineers are shielded from constant interruptions, and can rely on one person to answer their business questions instead of trying to have discussions between all the stakeholders.
As an engineer I was myself very skeptical of the role in the past but then worked with 2 awesome product owners for a few years and it was a fantastic experience.
All backlog was generated by having two analysts or their boss meet with the product owner, a direct marketing exec, writing down what she wanted, then meet with team to give marching orders.
Being a direct marketing exec, all she cared about was “engagement”, so our product was mainly just feeds of pet advice, tips and tricks along with a game (that the bored analysts played all day).
The app also allowed booking store services such as grooming, vet visits, and boarding. But the implementation of the service bookings was so poor it took over a dozen steps with lengthy spinners for network requests. Our metrics showed less than one third of service booking attempts completed.
I pointed out that since the company was terrified of Amazon, that store services were a key differentiator and margin producer and we could make booking services far faster and easier with some design changes.
The response was, that’s not in our backlog. So next build after completing my work, and helping other devs complete theirs, I worked over a weekend and cached some booking data in the app to eliminate spinners on entry into the booking system.
I proudly showed it to the analysts that Monday and they nearly had heart attacks. That wasn’t in the backlog!
All they cared about was making the product owner happy (she was a teller, apparently) and all my efforts did was make me unpopular with them and their boss.
I think it's caused by
1. When you have very deep management chains you start to have lots of people with opinions on what you should do, how you should do it, who you should do it with. Even if on average most of them aren't micromanagers, it only takes a few.
2. As you grow you start pulling in more people with experience outside of SV who bring in culture with them. Note that "SV" is also not monolithic here. Companies like Microsoft (ok, technically not SV) are farther to the "traditional" side of the spectrum.
3. "Horizontals"
As you grow I think reducing engineer autonomy is unavoidable, and the goal is merely to stick to reducing it and not eliminating it.
Also, I think dealing with those large scale issues is what separates a staff/principal dev from a senior dev.
This isn't true in my experience. I'd say it's more like a binary search than a fully-connected network. I talk to my immediate team first. If they can't help, I dig through the company-wide documentation and codebase, then message people who have done or considered something similar to what I'm trying to do; they either give me the information I need, or direct me to someone more likely to have it. Each hop gets me closer to an answer, and I typically end up having to talk to 1-3 engineers rather than the O(500) who work in my organization.
Part of the reason this works is because there are a handful of very senior engineers who have a good sense of what's going on across a big part of the company. Complex requests often get routed through one of these people, and they always seem to know who to talk to. I guess that's a form of hierarchy, but without the "command-and-control" aspect. I see it as an occasional escape hatch rather than a primary means of problem-solving.
In my anecdotal experience, the “product” people generally have lesser context than the engineers, and often rely on engineers to do their jobs. The good ones make a serious effort to get at least a vague understanding of how the product works and are hungry to hear ideas on how to deliver more value.
However most PMs will not be this way. Their primary instinct is to interface with the product hierarchy and explain to engineers what the “higher ups” want. They exercise this power by vetoing ideas that don’t fit into this narrative, even if these ideas would improve the product. Engineers then mostly don’t really share everything that they’re working on with the apathetic PM; the motivated ones build things anyways risking rebukes while others will do just what product asks for.
The only scalable solution really, is to create processes and career paths for engineers who are inclined to product design to become PMs, or at least spend part of their time as PMs.
At Service Oriented B2B companies, there is another class of people who become excellent PMs: the Customer Service Managers, who have dealt with Customers and their issues, and have a deep understanding of the problems and frustrations faced by users of these systems.
Teams have very limited scope in exchange for enormous autonomy. “Inmates run the asylum” as my former manager put it.
It never means “let’s do things in a coordinated and mutually beneficial way”. It always means “you will do the thing that I want you to do, or the way I want you to do it”.
Weasel word, if ever there was one.
Do you mind expanding on what you mean by "Horizontals" here?
Not after the infamous need-to-know memo.
Team docs visibility defaulted to "only to people I explicitly share with" for almost a year now.
It's only a question of time until code branches follow suit.
I couldn't find that memo, could you please link to a copy?
The audacity of that email was that it made accessing "need-to-know" documents internally (!!!) a fireable offense.
In a kafkaesque move, "need-to-know" was not determined by labeling or access restrictions. Therefore, you can't know you are accessing a "need-to-know" document until you do.
In short, this made using internal search engine fireable. It made clicking any link on the mail list fireable.
That was a year and a half ago.
[1]https://www.buzzfeednews.com/article/carolineodonovan/google...
This is a real experience in a FAANG company - we were developing a new product in enterprise space targeted to small enterprise IT admins who may not be highly trained or experienced.
One of tiny feature was the ability to download and self manage security certificates (upload again if any issue with security), it took literally weeks to define this feature by the engg and UX team. From tiny things like how to name the downloaded security files.. should we have the app name in the file or not - otherwise how would IT admin locate the file when they search, should file have time stamp because later we discovered that these d**n things expire, is there a 'standard' to begin with that defines the naming convention of these things, and so on..
All this because? The detailed level of spec is never written by product managers. Which I think is a very foolish culture, particularly in enterprise space.
This follows my experience. Even at a FAANG, product managers aren't going to get every detail and edge cases are going to pop-up, so requirements are going to come from a back-and-forth between product and eng throughout the process. That back-and-forth is the key, not expecting engineers to figure out all the requirements, nor from product managers either.
A subtlety complex feature took a couple sprints to iterate on stakeholder feedback and fully define the business need. The story doesn't seem that bad to me.
There was no "stakeholder" feedback as there is seldom an owner for these kind of decisions. If PM is the 'stakeholder' then these folks would have asked her "why are you not defining this?"
I've seen it too many times where the non-technical manager gets to make technical decisions and refuses to listen to the engineers.
I have worked at a good number of places, including a FAAMG.
The freedom that was given to me directly correlates to company size & seniority.
When I was younger, I didn't get much autonomy, why? because I was a naive arrogant prick. I would dream up solutions to problems that didn't exist. I would make existing problems worse by trying out new things.
When a company gets larger, it will bump into problems, missed features, deadlines or massive bugs. New procedures will get be implemented to make sure that those don't happen again. Ticketing is a good thing, so long as its developer lead. It allows delegation of tasks, and simple tracking of state. Its an artform that should be taught.
I was at a start up for two years that was bought out by a FAAMG. We were working on cutting edge computer vision research. We had a tight process, tracked with ticketing. We had complete autonomy.
We were chucked into SV "culture" only to find that is somehow managed to make cutting edge research both more chaotic and boring. Ironically I have much less freedom than I did when I worked at a large financial newspaper.
What it came down to, per the article, was that everyone in the co. respected that the engineering team was capable of delivering results and solving problems.
But i think per your question about JIRA, it gives you autonomy if: - anyone on the team can add a ticket - the team is trusted to make small changes in scope or spec to tickets in response to things they encounter as they work - there is little to no ceremony around accepting tickets as done - the team is trusted to check their own work first, and then collaborate with QA / stakeholders / etc. on releases
Also, if the R&D project is a team project involving multiple people, what is a reasonable update/sprint cadence?
1. Most tickets are opened by engineers themselves, and don't contain detailed specifications, but just notes to provide context.
2. The purpose of the ticket is to provide something to link to, and a place to put information as it emerges. So commits and PRs usually link to a ticket, as do engineering and product management planning documents, reports etc. Some tickets are terse and quickly closed, others could develop a long history, with status changes, links to other tickets and documents, ownership by different engineers over time etc.
3. We organize development into sprints, but a Scrum master would be disgusted by our process. Sprints are just short-terms planning horizons; we ship as soon as the code is ready and close tickets when the code is in production. The purpose of a sprint is to batch up all the low-level planning and prioritization so most of the time we're focused on execution.
4. High-level planning is all in GDocs, spreadsheets and Photoshop/Figma. Product management doesn't create tickets unless they are reporting a bug, and even that was pretty rare. Engineering management gets a lot of information about what's going on by observing Jira; this reduces the need for meetings to communicate status. (Doesn't eliminate meetings, of course, but it does seem to make them more interesting.)
Basically, it boils down to, you have to keep status somewhere, and Jira does that pretty well.
Any engineer could open any ticket for another engineer or team. You could also set the priority to how urgent it was. The receiving engineer could adjust the priority according to their own work of course.
Basically all work that you do was in the tracker. You could even add things for yourself, so you had a central place for your or anyones todo list.
Best work environment I ever worked as a software developer. There was never something blocking that some manager had to resolve, because you would just open a ticket an assign it to the person or team you think is relevant. All discussion was done in the ticket.
the business goals were translated into "epic" tickets, after that it was entirely up to the engineers.
An example of this would be something like:
We need an API that allows secure access to service x. It must have authentication, respond with an answer inside x milliseconds, It must work with a developer admin interface (see ticket z)
Ticket z would be assigned to the web team, and we would work closely on an interface between the API backend and the dev interface.
After that, my team would layout and make a high level design. We'd break that design down into individual "sub epics" and let each dev create and work on tickets as they see fit.
Technically I don't think JIRA by itself means no autonomy, it's just how it's used. But JIRA can expose a ton of metrics to upper management. And I think that management influencing how JIRA is used probably indicates lower autonomy.
For instance, story points and velocity being tracked by management might be problematic. Is the team tracking everything with story points? Otherwise certain types of work could be hidden, and slow down velocity. Is management using those metrics for team or individual performance reviews?
Jira doesn't intrinsically take away engineers' autonomy but in many cases it's the most visible manifestation of company policies/procedures that do.
I once worked at a place where Jira was without doubt a positive force within the company. There was an Altassian-trained Jira specialist working there who collaborated setting up each Jira project, and provided ongoing advice/assistanece to users (both PMs and devs). This was in a company of around 40 devs, 8 QAs, and 6 PMs, running perhaps 10-12 concurrent projects (almost all websites/webapps). It worked _really really_ well. (And I've spent th3e last 5-6 years looking for excuses/opportunities to poach that specialist...)
The key takeaways I learned from her was to identify and get leadership agreement right up front about whether a particular project was going to be "agile" or "waterfall" - because if you are going to be agile, Jira wants to be set up as a high level todo list of features/functions. If _any_ of the Jira tickets start to decree (from a PM/BA/manager) _how_ a feature or function is to be accomplished, you're fooling yourself and should have set it up as a waterfall project full of disposable-worker-bee style tickets.
I'm sure my takeaway was only scratching the surface of the deep understanding she had about project planning and managements styles, but that has work OK for me over the years. Either provide Jira tickets for features, and let the devs create their own sub tickets to document how they're doing things (and to get discussion/signoff on thing they ask for input on), or have the PM/BAs document a project in minute detail and fill each dev's schedule with sprints full of tickets representing 2 hours work or less.
I guess my point here is "Don't blame Jira. It's widely criticised flaws are largely a problem with the way it's used rather than a fundamental flaw in the tool. Poor project management results in shitty Jira use. Great project management will shine even if using postit notes stuck to devs laptops every day."
I worked on a team that had a very finely tuned Jira workflow. Then one day we discovered that all our customizations had been eliminated. The company had decided that it wanted all configurations to be based on a company standard, and no deviations would be allowed. We weren't even warned ahead of time.
Is it IT? Then they will treat you like a cog and consider you just a cost to the company. Is it Product? Then they will consider you valuable and critical to the company's success.
Because these days every business has to be investing into software products, and the ones that don't recognize that the software is key to all their products are the ones that are dead folks walking. They're also great targets for SaaS consultant vultures.
(And if they did, I'd immediately be looking for a different cookware supplier.)
But I'd still be wary of working for them if their software engineers were in IT.
I think that's the point. Cookware's just a further-from-tech example of it.
I think the only other role that gets ignored as much is the Executive Assistant(s). Much like sysadmin, good ones make sure everything runs smoothly, and should be prized like gold.
Being viewed as a value multiplier (like good instances of the above should be) is a fine goal in life.
1. It's not a slam dunk that software development is an asset to a business. Entire books have been written on how to keep software projects from eating your business alive. Many software businesses fail.
2. The hardware people notice that they're being treated as second class citizens, and wander away. Remember, the best leave first, including the people who fully understand how your product works.
What types of things do you say and why do you feel they may be red flags?
Just don't talk negatively about your old job. "I'm looking to move because I want to have more impact". "The work I'm doing now is not core to company's success and I want to have more impact". Etc.
I never looked down on someone who was doing boring work if that is what they were supposed to be doing. I'm looking for people who can communicate well and have strong core fundamentals. If you can demonstrate those things then you're doing well.
What kind of red flags are you worried about?
This would tell me that you're really engaged, up to date on the latest methods, and motivated to make things around you better.
For example, if you couldn't remove a bad pattern because it was too deeply entrenched for you to handle it yourself, and you couldn't get anyone more senior interested: that's expected for a junior developer and probably wouldn't be held against you.
Politics. I was competent, precocious, energetic, participated in many meetings and liaised with lots of teams with which people in my position don't usually liaise, bailed out some people in other parts of the business with unexpected skill sets and initiative, and made the manager look good.
Meanwhile, the one job I was actually fired - not laid off - from, I was sitting very close to the money train.
Waiting for it...
> As a result, these competitive types will also try to remove anyone who isn’t an “ally” regardless of whether these people provide value to the business.
Ah, there it is. I had to learn this lesson the hard way, and let me tell you, the kool-aid is no longer so sweet. The money train is often just a mobile coliseum. There's no escaping the gladiator aspects role. It's disappointing.
For me being unimportant enough that money spent on a scrum master doesn't make sense is the right work-life balance.
Duh! Companies don't put all the engineers in one cost center. It's either by teams/product/business unit/function
Not directed at you, just some general comments.
As someone hinted at in another comment, it seems a better (less dichotomous) framing is:
- tech first (developer lead, selling software or hardware),
- tech enabled (developers are a key collaborator with the business, e.g., logistics, manufacturing, ecomm, marketing, really every company should be), or
- tech as a cost center (the “traditional” company in the article).
What about the trifecta of product, engineering, and design advocated for by Marty Cagan?
I think the same argument that’s being made for giving developers more weight in the development process could be argued equally as well from the side of design.
Edit for formatting.
Funnily enough I work at a software company which is bootstrapped and where cashflow matters, I'd love to run things like a fully funded SV startup but I can't. We just can't afford it.
I agree with the premise of the article that more engineers need to be in powerful positions, but some domains are just too complicated and require substantial domain knowledge to make "product" decisions. It's the collaboration between product and engineering that's important, not switching one for the other. The biggest problem I have with the article is that it presents the problem as too binary with only 2 possible outcomes - someone being in charge.
Tech companies put engineers in charge because the engineers are the domain experts, not because they figured out some secret sauce.
One problem I fought was that a lot of people expected handholding from me, or wanted me in their meeting to "feel important." There really is value to having your manager act as a gatekeeper.
Another problem I had was that less ambitious people in other departments (support, qa,) would throw their work over the wall to me. I eventually had to get involved with triage to train the other managers to push back on this.
I can relate to this. One way to look at this is that dev-team is where majority of IP is created and not in teams like support/qa/docs/PMs. Over the period of time, they are also the reservoir of domain knowledge.
This often results into nuisances like:
1. Instead of reading product docs people will ask questions directly over slack (e.g Do we support Windows 2008? Do we have feature X?)
2. Adding dev team in a slack channels in which sales can directly ask questions which ideally should be answered by Product Managers.
But, there are absolutely companies maintaining low-growth Clunky Java Accounting Application for Insurance Companies #574 for whom this approach is contraindicated. Those companies can hire relatively entry-level, average graduates, at commensurate salaries, and assign them highly structured JIRA tickets.
Would they get better talent if their fronds tended more toward the SV model of compensation and autonomy described in the article? Absolutely. But in terms of marginal productivity or ROI, it's far from clear that it makes much of a difference to the financial performance of Cluky Java Accounting Application for Insurance Companies #574. That means massively overpaying for skills, or skill levels, that aren't really needed, or aren't needed enough to matter.
No, it's not the kind of software development any of us want to do, and I think the audience for this post are for the most part in no danger of having to. But I think it's important to be honest about the fact that this sort of dimly-lit, cubicle-dwelling sweatshop software work _is_ a high percentage of software development occurring on the planet. For that type of company, following the "SV" philosophy would arguably constitute a dereliction of fiduciary duty.
My view may be somewhat coloured by my exposure to medium to enterprise-sized companies in the telecom world, where a lot of the work is tedious, unimaginative and formulaic, using unexciting but established technology stacks, and so forth. But it still needs to be done, and there are companies who darken the skies with people to do it.
Lastly, I incorporate by reference everything that has been said elsewhere in this thread about the nature of large organisations and the poor scalability of engineering-led company process. Most software work, by volume, doesn't consist of skunkworks innovation or the development of something new. Most software work is incremental, in many cases strictly maintenance mode, and caters to many stakeholders. Large organisations and big capital serve an entirely different purpose that is realised at points further along the technology and product lifecycle. In this comment, I merely chose to focus on (1) the problems of empowering "average" talent this way and (2) the poor prospectus for doing so.
Agile is not always the right approach. For highly regulated or safety critical industries, it's quite likely to be the wrong approach.
(On the other hand, Waterfall for B2C mobile apps or casual games is also almost always the wrong approach too...)
Their business processes and products are often sclerotic, and so are vulnerable to disruption by smaller, more nimble startups. That's the central premise of virtually all B2B software from the Valley. But they certainly aren't going to die.
In fact, large companies and enormous capital are required to take things from a "disruptive innovation" to a household name, to say nothing of providing an exit for SV startups. So, they're a vital part of the biosphere and they're not going anywhere.
I think this is becoming less and less true, unless one is competing with the tech giants directly. In the past enormous capital investment was required to build out the minimum viable capacity needed to serve larger customers or market segments. Today you can rent that capacity. In the future more and more types of capacity can become rentable, such as logistics, compliance, manufacturing, etc.
It was very much a factory in the way the article says, but I feel like it has to be because of the nature of the work. I did talk to customers directly on larger projects but since I was building on top of internal systems and processes my role was mainly implementation/glue coding. Most of my frustrations were with our internal dev and IT who built and maintained the systems - over time they became more walled off, not less, which definitely impacted our work. And unfortunately as we were more 'client facing' any outages came back to bite us, not them, which was an annoying organisational dysfunction. They simply weren't incentivised to care as much as we were.
You can almost directly correlate the "traditional" approach to seeing engineering as a cost center, while startups/S.V. style companies see engineering as a profit center, and they treat their employees accordingly.
On the contrary, companies where tech is an afterthought-- well why would you empower your engineers to make business critical decisions if you're running a hospital or a sports team.
I get a lot of satisfaction from that. Of course that means that as a team lead/manager, I have to have a lot of self discipline to allow my team members to do the same and not try to solve every problem.
I was at Skype during the eBay years and it was similar. Lots of autonomy. Then Skype got sold, Silverlake instituted Scrum training for everyone, and well, look what happened. Product owners took over; the engineers became Jira-ticket minions.
That said, I do understand the problem with giving engineers too much freedom, if there's not enough maturity on the business side. I've been at places like that, too. Engineers gone wild. Just burning money.
I see too many startups adopting Agile/Scrum for lack of a better clue on how to run software development. These days I refer people to Basecamp's Shape Up for "just enough" structure.
I don't think it's about SV companies against traditional companies, but more about companies that intrinsically value technology versus those that do not.
We've had to smack it down pretty hard in a couple of cases, and pretty strictly enforce the triangle when communicating with non-engineering departments.
What's worked for us is to not restrict communication, but to hold our engineers accountable to what they say they'll get done and when. If something slips and it's not for a technical or personal reason (i.e. some risk materializing or needing to take unplanned time off), but rather because someone floated in and asked for something that wasn't scoped in, that comes up in a review and reflects poorly on the engineer and the asking party. On the other hand, if the engineer is able to get all of their scoped work done and also get the unplanned work done, that reflects well on the engineer and potentially the asking party. In our experience, approaching these situations in this way also has the benefit of preserving and encouraging individual autonomy.
If you don't let your engineers understand the business problem they're supposed to solve, you're wasting valuable resources.
http://www.paulgraham.com/yahoo.html
A quote: Hacker culture often seems kind of irresponsible. That's why people proposing to destroy it use phrases like "adult supervision." That was the phrase they used at Yahoo. But there are worse things than seeming irresponsible. Losing, for example.
In particular, how this impact long term maintenance vs. the desire to build new interesting things? (e.g., see https://themaintainers.org/). And what about people who just enjoy the act of coding — of wizzing through tickets and gathering points?
For another potential cost, take the example given of the Like button in the piece. I'd be curious what Justin Rosenstein now thinks of the example of the Like button. The author says "The business impact of this button is well in the billions: allowing Facebook to (re-) target ads, and "track" users outside of the Facebook site. [...] Everyone is better off for it: the people with the idea and the business." Does everyone here include the users and world?
Perhaps if we had better metrics, this SV would work for everything. But in the current world, with profit, data, and growth metrics being incentivized, it may have real societal costs.
That's not coding. Those people enjoy closing tickets and gathering points. That's fine, but they should try videogames, it's a similar experience but 100x more entertaining. Shame it generally doesn't pay.
Full disclosure - I’m the technical co-founder.
THis process is not required for your average company where IT is a cost center and CRUD-type apps generator.
The choice between a company where (a) you work on assigned tickets, but standard business hours and weekends off and X days of guaranteed holiday per year versus (b) a high autonomy, high pay but if the company needs you to pull a 90-hour week then you do it without asking - it's a tradeoff. Whether you get to write your own JIRA tickets is just one of many dimensions here.
One of those two options is not compatible, for example, with having responsibilities outside of work such as childcare.
High autonomy is higher variance. Some of that may be good - innovation. Some of that may be bad - customer or market failures. The outcomes are money consequences to a business, and given business has some to downside tolerance to that. In some businesses tech innovations have huge realizable upside.
Maybe a way to see the autonmy tradeofff is in terms of manufacturing complexity. A business whose products reach customers via endpoints on hosted service is different than one whose products are tractors off an assembly line. Maybe that's understood as how capital-intensive it is to bring a marginal capability to market.
I'll say further than in an ideal organization, with respect to autonomy, the job of management is allocating attention and distributing context, and that management and staff alike are serving a well articulated mission through the vehicle of the business organization. In a given instance, we all can take a compass reading on "true north" for business.
A non ideal business, for starters, has a self serving and self interested control structure that views autonomy in relation to power.
Both SV and traditional companies can be ideal or not. I think the difference comes down a culture of thinking about risk - reward, and the collective, cultural engagement with mission.
I wonder if there's data (survey? perhaps correlated with compensation?) to back up and establish magnitude and trend/diffusion or refute any or all?
Also would be interesting to look at those characteristics for non-employer open source projects and government. There would surely be lots of variation across projects and departments/governments, as there would be across companies, though I'd guess for non-employer open source 1, 2, 3, 5 would be very high, 4, 6, 7 unclear or lacking, and for government possibly even lower than traditional companies across the board? Just speculating, again, curious about data, or more speculation. :)
Hard to address in a comment, easier to check for some big red flags.
What percentage of people involved have lived it before? If 0%, then I’m not sure it’s even possible. Similar to trying copy Disneyland. You could read a novel describing it but if no one‘s actually been there it’ll be a rough ride.
What’s the plan for anyone leading people? Some people freak out when people are newly crossing the triangle below them, more so if they don’t know why it’s important or how to adapt to it.
What are the checks to make sure your org is not just choosing a fancy name (I.e. digital transformation) and merely mapping old processes back onto it but with more overhead?
It seems like maybe that’s the mistake is to think about this as a process difference when really it’s a systemic and cultural difference.
Take the example of a stakeholder talking to a new hire. If stakeholders do it to ‘do the new process’ it’s not there yet. If they do it reflexively because they expect they may get useful feedback, then that’s a good sign.
In an SV-style company that develops a SaaS product, engineers can make all sorts of decisions, since they are solving a problem they are intimately familiar with.
What about other industries that require specialized knowledge or regulatory bureaucracy, like finances or healthcare?
there's so much stuff happening
It’s balanced by the CoL adjustment. In short, SV engineers are not better off, and haven’t been for the past decade. This is apparent by the observed exodus in the light of Covid forcing a shift to primarily remote work. Engineers are trying to reclaim their wage advantage by moving to cheaper locations, and companies are fighting back by reducing compensation accordingly.
I want the greatest share of the returns of my own labor I can get. If that results in very high salary, great, it’s earned. If I take a risk at a place where that translates to much lower salary, that’s fine too - as long as it’s very objectively meritocratic.
If I am making $200k per year as a top, top performer, while my employer is paying far more mediocre performers $175k (perhaps based of geolocation), or where both of us are bringing in ~ millions of revenue, that’s super unacceptable.
As time goes on, the surplus of my labor productivity that a big corporate employer can capture beyond my total compensation should be decreasing heavily - and expert software labor is one class that has the negotiation power to actually push that issue.
So I’d flip your question on its head. Why do we pretend that rent-seeking executives deserve the surplus revenue generated by rare talented engineers? Why (with complete sincerity) aren’t many, many, many more software engineers paid on the order of what Hollywood actors are making, while executive salaries simply have to come way, way down in these lines of business?
In other words, for small cap companies, yes, the ROI of athletes to their franchises is relatively higher than engineers to their companies. But for most of the SP500 this is not true, and there are lots of software-heavy companies with much higher market cap to the extent that even with 10k or 100k employees, I doubt an athlete’s ROI for a small sports franchise is proportional to the ROI of employees in the large company.
For example, virtually any engineer at a FAANG company should be making top pro athlete money (not considering endorsements). There are literally thousands of engineers at Amazon and hundreds at Netflix that have this ROI on the soaring revenues of these megacorps, to a much greater degree than athletes have on their teams.
Some actual numbers:
In 2019, Google's gross revenue was ~$162b. ~$76b went to "costs of revenue", ~$26b went to R&D, and ~$11b to stock-based compensation. It's unclear what % of costs of revenue is attributable to "salaries of software engineers", though. Let's approach it another way. Google employs 30k+ SWEs, the vast majority of which are in the US. The median outlay probably falls somewhere in the L4 - L5 range, though there's a fat tail at the top. If we want to be conservative, we can just ignore the tail and say ~300k/dev. $300k * 30k = $9b (this is probably low).
Google's net income in 2019 was $34b. So you could multiply SWE comp at Google by not-quite-5 and zero out Google's net income.
What do top pro athletes make? Wikipedia says the top pro athletes earn mid-high eight-figures/year.
There's some headroom to pay engineers more, but it's definitely not "top pro athlete" more. Engineers can and do make that kind of money by founding & growing successful startups, not by working for FAANG (possibly with a very small # of exceptions at the top).
The only problem is that there is no leverage or scale because the software is coupled to physical goods, which means no big profits to pay above average salary and no career progression.
And that is the point where I don't agree with the article. I don't see the correlation between salary and autonomy.
This keeps everything focused on the important usually externally facing issues.
But - I'm curious what costs folks have seen in making this switch to giving engineers more autonomy and expecting them to be problem solvers for the wider business rather than code-factory workers. My hunch is that it's usually worth it - but what are the risks?
It's easy to mention a couple of anecdotes about some engineer who made an improvement or feature that represented millions of dollars for the company bottom line. But what about people going down strange rabbit holes, engaging in resume-driven development or following the allure of cool tech that isn't really needed for the use case (e.g. "let's use Cassandra for this!"), Not Invented Here syndrome, premature optimization, spinning up unnecessary microservices because that's the way to get promoted, building a fancy internal system that internal stakeholders don't actually need or want, etc. It's really really hard to simultaneously see the business big picture and also be down in the technical details, and very easy for engineers to fall victim to the Dunning-Kruger effect and go off in the wrong direction. Perhaps the few people who are able to do this well are able to compensate for all the waste, but I don't think we can assume that it leads to good results for the business in every case.
As for risks, the biggest problem I've seen from this cultural change in a company is the disharmony it creates between the people who enjoy task-driven work and just working their 9-5 vs. the people who enjoy creative problem solving. It usually works better to have that culture from the start and also why it is so hard to change a company's culture to this way of working because it disrupts a way being that almost everyone in the company is used to.
Software engineers are among the highest-paid people in our company.
This is because they can bring some of the highest leverage through coding and problem solving.
We want to expose them to the business, so while they are doing their "normal" work, they can also find more impactful opportunities for the business."
A vision needs to flow top-down, but if you want your engineers to make helpful suggestions, they need to be involved on the business and customer side from day 1.
What the article compares: SVC-like vs traditional can be more accurately described as non-software employment versus software employment. It isn't that silicon valley has some magical solution that nobody else has, but instead is incentivized to leverage expectations common to all other industries. Again, this is impossible to see if your experience is limited to writing/managing software, but is clear as day otherwise.
The key reason why this is different for other industries is that the expectations are higher for employees where the investment per employee is higher. This is even true of lower paid and lower educated professions like truck driver or construction equipment operator compared to software engineer. In all other industries there exists some form of licensing or certification followed by some form of formal training such as a broker/agent relationship and then there is continuing education or periodic re-certification. These processes exist because of regulation. They take time and money away from the profession, which is an investment into each employee. With that level of expense more is expected in return by the employer.
The software profession has none of that. There is no investment in a software employee aside from keeping them happy enough to retain them. As a result its hard to think anything more of them than as a tool, like a janitor. This where you get the churn the article accurately describes as death by JIRA. SVC-like companies invest more in employees with higher salaries and thus expect more in return, which is not entirely the same as a formal professional process, but is a step in solving for this gap.
As for plain as day examples look at the common expectations for engineers at engineering firms, lawyers at law firms, and even truck drivers and police officers. There are all kinds of administrative and documentation expectations that are impossible to find in most software developers. Comparatively most software developers at the traditional companies seem largely illiterate. As an example most of these guys are scared to death of writing original software without a billion packages from NPM or Maven and are almost utterly incapable of writing formal documentation.
Do you work somewhere that has a large management apparatus? These behaviors are sort of bare minimum expectation in smaller companies. It sounds like you perceived developers as a group to be mostly anti-social curmudgeons.
If you want someone to critically think, they have to have business context and in many cases a vested interest in what they're building.
This crops up quite a bit when your engineering is primarily outsourced/overseas and your work involves an issue that might be a primarily US based problem.
How can you expect people to critically think when they don't have context around the issue?
It’s not so cut and dried as that, now is it. The pressure from the expectation and need to bring leveraged value can be detrimental. sometimes, and highly dependent on the person, this is life destroying. factory work is fine for most people, i believe. even engineers.
Oil, Pharma, Banking have all had great times when employees there had lots of autonomy and made bank.
True with SV Engineers are the main type of employee so probably do better than at firms where engineers are helping other divisions make money.
If you have worked for a few years in this industry, you would know this.
Every engineer knows it doesn't work that way (but many who charge by the hour w/ consulting may be happy to take the job for a while).
I'd summarize the problem as lack of real engineering leaders. It's as if programming/engineering is looked down on or not taken seriously enough.
- Biotech (in Silicon Valley to boot!)
Regarding autonomy--usually a high degree of autonomy is found in two places: 1) at larger companies, a very senior, trusted individual who has proven himself to be competent and capable of making big decisions; 2) at startups, a very junior individual scratching an itch and loading up on technical debt.
For the most part solutions are prescribed or narrowly constrained--99.999% of engineers are going to use the same services architecture, libraries, and compute infra every other engineer is using.
Regarding curiosity and problem solving: I guess I have a different definition of "curiosity" and "problem solving." What I've seen in this industry ain't either, generally: it's mainly driven by personalities at the individual or corporate level, e.g., "everyone" doing what Google does or following Fowler, and empirically unsupported anecdotes/fads (e.g. TDD, agile, etc.). The vast majority of engineering work in any established company is to use an existing tool or infra to implement a generally well-defined need. Most of the interesting details are already solved, and the implementation work, while also interesting, isn't really solving the problem, per se.