What to do when tech jobs go bad
medium.com
medium.com
I've contracted with a dozen startups and small businesses. Most of these companies will find a young developer charging between $20/hr and $50/hr and have them start coding on their new app/website idea and feel wildly optimistic when they see the first 80% of the project come together.
Then it's time for the edge cases, and because there aren't any senior devs on the project, handling those edge cases introduces bugs. Weeks and months pass, and the project that looked like it was a surefire success descends into development hell.
But the company never fires the developer. The developer actually quits from being over stressed. Then the company goes looking for a new developer, who tells them they'd be happy to build their software for $20/hr-$50/hr. The new developer comes on, looks at the codebase, decides all of it is crap, and recommends they start over.
The new developer finishes 80% of the project in the first week.... rinse and repeat.
The company should have hired a developer or firm with a record of success, with actual, running software out in the wild. How they achieved that may or may not line up with the ideas in the OP, but at least the company would know exactly what they're paying for.
Recently, I was implementing Pivotal Cloud Foundry for a client. Everyone was assuming PCF would deliver incremental and linear amounts of value during implementation. I had to keep telling folks, "No, it doesn't work that way. PaaS is only valuable at the very end of the implementation, when all the technology is ready and the team has been trained."
it may be linear progress from "lines of code" or some other metric, but there's a point (or usually a few) at which peoples' perspectives on the project changes.
I've hit this a couple times on recent projects, and it's usually tied to some new UI stuff hitting people. Usually, they've seen all the various pieces, but until they can click-click-click through end to end on a process, they don't care, or get frustrated and mentally tune out of that part of the project.
It's part of why I maintain that it's much better to be 5-10% over every estimate along the way but maintain your velocity indefinitely. Better for people to be a little disappointed in you now than to drop a giant bombshell on them later. It's not a popular opinion, but it does work if you have a team with the fortitude to stick with it.
If you're lucky and know that this is the case. If you think you're actually "90% done", you're in for a world of hurt.
I get that feeling when anyone cheerfully exclaims that we're 90% done.
I'm starting to believe there's a similar rule for how many copies of MMM you see on the bookshelf. I've seen as many as 5 stacked together, and that place was whackadoodle.
I now use the same metric to decide when to quit a job and move on. I'm now at a startup, and keeping a close eye on its inevitable transformation...
We really need to end this craziness.
It was a terrible place to work if you cared about what you did at all as you never got to finish anything. They shuffled you around like cards, different teams, different buildings, different projects, basically rearranged things based on the whims of those currently with the clout as a way for them to increase their clout.
key things that help - focusing on extensibility/abstraction in areas of code that will require ongoing changes. This is where you most need experienced and talented engineers. - maintain testing discipline - conservative dependency management and picking the right battles between DIY and open-source - creating periodic blocks of time for engineers to refactor software or architecture.
After that project I intuited that projects had a maximum complexity after which the wheels fall off, so to keep a project growing past that point you have to remove accidental complexity before adding new intrinsic complexity. For the limited sample size I have direct or indirect access to, this really seems to hold. And in fact someone told me that RPI teaches something very close to this in a required class for one of their Masters programs in CS.
Abstractions do not fix this problem. They put it off until the next performance emergency happens. It's an avoidance tactic and kind of paints you into a corner. Ultimately you're not looking for an abstraction (I mean, it is, but so many bad things are also abstractions that I hate to use the word for this situation). You're looking to model the problems actually being solved. You're looking for the truth.
Abstract stuff is "just" more abstract and adds a new concept on top of the old. Abstraction makes the coder feel all grown up and maintain the daydream: "once it's done building the features will be so much easier."
Automation is the part where we say, "this is repetitive and hard to keep track of - so make the computer do it." Automation creates leverage. And sometimes you need a truly novel abstraction to automate successfully, but in most instances you only need the same old 70's era programming constructs wired up a bit differently. Which does lead to dropping one abstraction in favor of another, as you allude to.
Even if you get into, "well we need to Optimize it, we can't just do the simple and straightforward thing" you can start automating the trivial, repetitive optimizations and get into the business of generating code. And that can be abstract, but it doesn't have to be - you don't have to incorporate a whole suite of compiler checks or a runtime model, the "compiler" can be a FSM that emits source code - but it'll be automated, and you can add the interface and checks to it that are most relevant to the problem, and make the output look human-readable to some degree.
Edit: another benefit of this approach is that your initial coding environment truly is "implementation detail" since it's the starting point and you just add the leverage you need from that point rather than feeling obligated to use the officially branded abstractions.
I often wonder what happened to being professional. If someone ever says they prioritize working for actual money over being passionate about changing the world while basking in the vision of some VIP on the board that's like admitting murder. I see this mindset more and more in non-tech jobs as well.
When the company succeeded, due in part to the systems I built and designed from scratch, I was unceremoniously asked to leave a few weeks before the transaction closed for many millions. I was asked to leave because as we drew nearer to a sale of the company, I had the nerve to ask for the company to put on paper how I would be a part of that transaction, since I had been verbally promised that "a rising tide floats all boats" and I would be rewarded handsomely for my work. Mistake #2 of 1000 then, accepting a verbal agreement over one in writing.
In the end, when I asked about getting it in writing so close to the sale, I was asked to leave. When I said, "but, you told me I was to think of this like a family", I was told, "we thought you were smarter than that".
That episode did a lot of damage, and I never saw anything meaningful in terms of compensation- the owners got fabulously wealthy.
I made many mistakes then, and I could have pursued legal action, but it was all too painful.
Since then I always try to insulate my feelings and remember that in a professional setting I can always do my best and be professional, but to always remember that to others, it really can be "just business". It's important to keep some distance and avoid emotional abuse at work, both doing it and receiving it.
It's not to say we can't be friendly, but just to be professional, honest and ethical in the workplace, always.
Edit: I meant to mention, that in this job, I was working for the "changing the world part", but when I saw the imminent sale coming and I had nothing, felt I had to ask if I would be a part of it. Back then, it was a mistake for me as well, to assume that I was going to do good and get handsomely rewarded and expect that the two would come together. I think it can happen, but I learned the hard way how to compartmentalize changing the world and getting paid with simply doing necessary work to get paid and survive.
I do also feel that the founders in that story should be named and shamed. There is absolutely no excuse for that kind of behavior. They should never be able to do that to anyone ever again.
If those type of employers could get "named and shamed" based on someones verbal only, then it would be a dangerous society we live in. On the other hand, if it was written down, there would have been no problem in the first place.
The company is the one that made the verbal agreement, which they could have written down at any time. The company is the one that abused it's power, and most importantly, the trust of the employee. To me, that is an incredibly heinous act, which should come with draconian punishments.
You don't know if the company did anything.
And my draconian punishment remark was directed at founders who fuck over their employees in general. Violating that trust should carry extremely heavy penalties, to the point where you are much worse off than if you didn't.
“Manson Family” Or “mafia family” Or “Addams Family”.
Take that vacation. Take that time off. Ask for that raise.
Tomorrow someone you've never met (and may never meet) might circle your name on a list of employees to be laid off. Never knowing what you did or didn't contribute. Or how dedicated you are.
Was on a project where we had multiple applications. The team had a new employee code an entire system as a prototype but then it was decided to put it into production. Nobody ever reviewed this code, and that engineer told me himself that the code was crap. He left the project to get away from this code and I inherited a mess, with nobody to ask questions from.
To make matters worse, the entire company relied on this system and there were about 20 regular users of this system. I was responsible for handling all of their feature requests and support which was often. Many bugs I fixed were extremely challenging edge cases, and there was zero appreciation for the complexity of many of these.
My manager was telling others at the company the code was solid, and it's "just python code" which can be figured out.
I notified the company I was leaving and they begged me to stay. Even offered to "release" me and hire me at a higher salary under another manager. The only issue was I would be required to support this same project.
My manager refused to give me a release date so I had to call HR and tell them I needed to leave at any cost. Pretty sure it's a disaster there since I left.
At first, the goal is “build a prototype,” then it’s “make this an MVP,” and then finally “what’s left to get this to production?”
I don’t blame founders for this; I blame the programmers, especially if they’re consultants. This is a symptom of failure to manage expectations of stakeholders.
As a developer, your job is to show the stakeholders what the tradeoffs of different decisions are, both in terms of immediate time to results, and technical debt. If you hear “make a prototype,” you better ask for a projected lifetime of the prototype, and discuss with the stakeholders the tradeoffs of quickly building a prototype, making sure they understand it could mean more work later to productionize it, in exchange for less work up front.
Also as a developer, you should realize that nothing is ever a prototype. Don’t use “it’s a prototype” as an excuse for writing unmaintanable and insecure code, because eventually someone is going to want to add some features to it. Good luck telling the stakeholders after three months of building a prototype that you need to start from scratch because the code is awful. This is especially difficult if the prototype looks like a fully functioning product (with minimal features) to the stakeholders, but you as a developer know that under the hood it’s completelty unmaintainable.
It’s better to take the time up front to put in place whatever you need to write quality code, than it is to take shortcuts and write bad code. A good developer will be able to explain this to a stakeholder, instead of caving under pressure to rush a prototype.
Therefore, if they say “ship it!”, they are either wrong or missing information.
If they are missing information, the developer is at fault for misinforming them of the tradeoffs necessary for decision making.
If they are wrong, then either the developer has cited irrelevant or incorrect tradeoffs, or the developer made valid points but the stakeholders understand the tradeoffs and are willing to accept them.
You're trying really hard to bring this back to always being the developer's fault, and you're ignoring that, many times, management just doesn't listen. You can explain until you're blue in the face, but if they don't want to listen, nothing will make them do so.
And ignoring that, almost all of the time, the programmer has very little power in these sorts of transactions.
But you cannot blame the developer. Many app are prototypes to get them over a certain point to get funding. What happens is the technical debt is never paid and you are forced to scale the prototype while adding new features.
I understand product owners and businesses wants estimates but software engineering is not making a car. Forcing programmers to estimate their every task makes for a very bad work environment. They are not robots.
It requires more project management, but it's much easier to quarantine problems and explain them to stakeholders.
My rule of thumb is 4 hours. If you think it will take longer than 4 hours, it should be broken down. Ideally, most tasks will have estimates of 1 hour.
That way, when a 2-hour task requires 2 weeks to complete, you can point to the specific task and its challenges. It provides an easy explanation for why the release was late, why the requirements were too ambitious, etc.
Engineers do not need to be managed down to the hour. Good grief.
The micro-tasks I'm describing go into a project management system. They are either assigned to or picked by developers who complete and document the micro-tasks on their own. Because the tasks are so small, there is almost no management of developers required.
Finally, it's trivial to find out where project estimates went wrong, because any hidden challenges are well-quarantined in the micro-tasks. If the task isn't well documented, it's still pretty easy to go to the developer and find out why it took longer than estimated.
If this isn't micromanagement, what would you define as micromanagement?
I think some managers just don’t have enough to do so they make work for themselves (and everyone else). Like a border collie with no sheep to herd, they will start herding ducks, children, etc.
If development was so easy that a manager who doesn't understand programming can break developer tasks down into small chunks - we'd live in a completely different universe!
In reality, management breaking anything down into smaller tasks, instead of people who will actually do the work, breaking things down, is going to lead to ZERO architecture and a shit codebase, 100% of the time.
That's if you're lucky and the project is a clean slate.
Existing codebases are full of code smells that'd take days to comprehend, days to chase down and comprehend what the actual requirements were/are because those are never properly documented, and then re-write that 'micro-task' so that the next developer doesn't have to spend their days in hell.
Oh, this'd need to now be tested, there'd be new bugs, and those would need to be fixed also.
So instead, in the real world - your micro-task gets handed down, the developer doesn't bother understanding what the existing codebase is doing, they just patch it until it works, contributing to the unmaintainable piece of garbage that all developers with any self respect left in the world hate.
Since only the brave/crazy ones will say 'this micro task will take 2 weeks instead of 2 days because the existing code is garbage', those brave ones will get replaced by those who will say 'yes, 2 days'. Your project will need more and more time per task, and more and more developers doing maintenance, but the managers will continue reporting successful micro-tasks completed on time!
I've just described corporate programming and how you're directly contributing to the hell that it is.
More precisely, a no-development-management-knowledge manager’s worldview.
While obviously development experience can be a source of development management knowledge, there's no reason it should be the only source, and development managers ought to be screened for job skills as much as developers are.
That's why McDonalds works, and software development is a perpetual disaster - McDonalds understand that to manage even something as simple as flipping burgers, you need to have done it yourself first!
It's the arrogance of business school management that's responsible for a great deal of turmoil across many areas of everyday life. Just imagine going to school for something that's not difficult, to 'learn' how to boss people around. For good pay!
Isn't that the dream of every non-creative, lazy, half-wit you never want managing anybody ever? Yes... yes it is...
This is a recent phenomena on a mass scale and it isn't going to last - getting high rank without having earned it has always been a complete disaster, just look at any point in history.
This is just another way of saying management is incompetent. There's no project management strategy that is going to overcome incompetent project managers.
For the engineers, and yours, peace of mind and progress -- please spend some time with stellar execution managers that understand software development is a people problem, not a project management problem.
And I agree with you: talented people, be they managers or developers or janitors, will be able to overcome any process or planning shortfalls.
But there's no difference between what I'm describing - a project manager breaking down features into bite-size bits of work - and a developer taking on a single feature and writing down TODOs in code, on post-its, on a white-board, or in the comments of the project management software.
The difference is one of documentation and communication to stakeholders - which is the PRIMARY problem with software estimation. There are ALWAYS unknowns in software. A team leader needs to be able to go to his boss when something is falling behind and say, "We didn't anticipate this. This is the reason it's taking longer".
This is FAR superior to going to the developer and asking them why feature X is taking so long and having them compile a report from Notepad/post-its/white board scribbles. With my process, you can just pull up the project management software and find the tasks that went way over, and the developer stays almost completely insulated from management.
I'm sorry, but you haven't really worked with or had training from real-world engineering managers.
This is the worst kind of condescending ignorance, it betrays an inability for civility that I wouldn't tolerate in a colleague.
It is not your fault if your leadership demands detailed answers as to why X is off by Y days. All you're doing is cascading that mistrust/poor management skills to your engineers. I understand what you're doing is the most reliable way to do it, but there is a better way. This is why I mentioned its a people problem.
The better way is to not break down things into hourly tasks but weekly sprints as most teams do. Let your engineers break that down into how much ever granularity they're comfortable with. If something goes off track, build communication trust that helps them keep everyone in the loop and work towards a common goal.
As a product person, I've had the fortune of working in two of the big 4 with some of the best engineers there is - so maybe that contributes to my worldview and it might be terribly biased.
Thanks for explaining :)
However, as padobson notes, there's an overhead associated with breaking complex things down into smaller units. There's a point at which it becomes counterproductive.
I feel like I have, over the years, sometimes crossed that line.
(And, other times, not gotten close enough to that line and been surprised by weeks/months of schedule disintegration.)
The only thing that makes small estimates seem better is the worst case tends to be less time, but overall they don't help.
Giving single numbers for an estimate with uncertainty is bad for all parties.
Additionally, with experience one can get better at giving estimates with less uncertainty for well known tasks and recognize the uncertainty for other tasks.
AKA Y is much higher when you deal with low hour estimates.
This is most noticeable when you have lot's of tasks. If you think 200 tiny tasks may take 1 month or just over 1 year that's an almost useless estimate.
I wrote down in detail all of the tasks involved just to help me estimate like I always did. By this time we were transitioning company wide to scrum.
Every task I put in was scrutinized by the contract “scrum master” as it wasn’t part of the MVP even though I knew it wouldn’t affect the schedule since I had prior work I could base it on.
The rest of the dev team had the same issue. I got tired of fighting and the scrum master was getting in our way. I told them to do what they thought was right and run it by me first if it was anything major. None of the things we were trying to do were customer facing, they were all backend architectural things.
I tried to fight the good fight but then had the epiphany - I didn’t have to. I’m the dev lead, I do the code reviews and no one higher than me will be looking at the code. I told them not to be detailed in the how within our project tracking system.
Another senior dev told me the same advice you realized. Now we just create more generic stories. He’s stared to go story by story and wants to know how it meets a business objective.
Our stuff is extremely customer facing and time critical. I welcomed Scrum over Agile but a bad Scrum Master can kill the benefits.
If you don't know a given approach will work, sometimes that's best to timebox some experimentation time (say an afternoon), but then you can also say to your PM/EM 'ok, we need a little time to work out how long it's going to take', and decisions can be made accordingly as to priorities.
Very little of being a software engineer is about writing code. That's just typing. The skill is being able to come up with a coherant plan to solve a problem. That's why architects don't tend to write a lot of code.
I use the term discovery phase to describe this. You give the discovery phase a certain amount of time to complete, it should be from a few days to 2 weeks. The deliverable at the end of discovery phase is a document. The document explains what experiments were run and what we should do next.
But I know plenty of people who need constant monitoring and handholding.
This also goes well with the rule mentioned in the article: always have at least 2 people working on a project.
It doesn't make things faster but instead takes a lot of mental energy that could be better spent on solving the problem.
Frankly if I'm going to spend a couple days on one aspect, I'd rather just implement that aspect.
I prefer to identify the risky parts and do them first, and refine estimates along the way. TDD helps because it keeps components isolated, so you don't have to build "in order".
Waterfall development with adequate time for planning/discovery up front does for sure give better estimates -- but those estimates are very expensive to get and are usually not all that reliable anyway, especially for multi-month projects.
In other words, "anything useful".
My impression has always been that the true cost of a dev task is only revealed when solving it. That's why they're hard to estimate. If they were easy to estimate, they would be trivial to implement and therefore could be largely automated. And for the lack of work, we'd aim for the next larger goal (which is again hard to achieve and estimate).
This seems to be a concept so hard to grasp for non technical managers.
By definition tasks aren’t that similar to other tasks, otherwise we would be a super crappy software shop that can’t provide semi-generic solutions.
Dev team’s mission is to solve unique problems to the stack. Discovery is a major time consuming task by definition, and can’t be properly estimated, as it’s an information problem.
The problem with this is that the time required to complete all the pieces is almost never equal to the sum of their estimated durations.
Do you mean bank holidays? Cause regardless of how you feel about your company, you should always take those. Rest and disconnect from work are important, especially in environments where you really like the work or the job, and the risk of overheating is much greater.
That being said, your point about bank holidays is a good one. Never give your company free labor -- they'll never give you free capital.
Realistically, tasks should be grouped into roughly day-sized estimates for any long project with an equally large QA pool possibly run in parallel.
Granularity only gets you so far before the returns diminish to the point of worthlessness. Imho anyway.
It's also the realization that I can't/won't fully plan out someone's personal roadmap for them. They need to understand what the team is working on, what the business priorities are and make their own tradeoffs. I set high level priorities of course, but I think people should have control/ownership over their day to day tasks.
> Normalizing and glorifying drinking alcohol.
Sure, alcohol is not healthy, and peer pressure to drink is bad.
But I've found such events to lower some barriers and help connect with people I would usually hardly talk to short of hi and bye. And I think this may affect work productivity as well as personal happiness in a positive way.
After all, "optional one pint of beer after work, once a month" is regular.
"Two or three pints a day, starting at 3pm" is also regular - but sixty times as much.
I'm not saying never have alcohol at company events, far from it. But don't have every event be centered around it. If you're going to do company events, you need to have a variety of different things, so that, while not every event is everyone's cup of tea, people don't feel isolated at every event, either.
One thing I disagree with (and I know this puts me in the minority) is code review.
Code review for trusted engineers is a waste of time. You're only going to find those bugs that are very hard to see. And the odds of you catching them in code review are very low. Also, if you're a kind of fast moving startup (like my company is) there's so much new code coming through that it is a serious slowdown to review it all. At least at the level where you'd need to if you're going to catch the things that the original author missed.
I only make juniors and new hires do code reviews. I'll also ask for code reviews if an engineer is taking on a task that's totally new for them. But otherwise we've totally cut them out and been fine.
As you said, it's completely different for junior engineers where it ends up being more like 90% bugs/tech debt, but the code reviews aren't all useless
It's truthful and not "whiney", as the more sociopathic among us are quick to sneer.
However, an underlying thread is needed to weave these criticisms into a meaningful critique. Mismanagement is a theme, but, alone, is insufficient for making the piece persuasive or insightful.
I've seen this so many times and it almost always ends in failure. Rewrites as well. Engineers can't resist improving a system while rewriting it (adding caching, redundancy, etc). They neglect proving core functionality is intact before making improvements. They also often neglect the thousands of man hours it took to find all the bugs in the system.
I've been building ML systems for a while and I've even seen engineers coming up with their own search/optimization algorithms. This always fails. It's hard enough to get standard techniques to work, since your data usually differs (slightly or severely) from what the technique was validated on (standard, cleaned data sets used for journal papers).
> Solution: Start over. The sunken cost fallacy is real. You’ve made a mistake, and that’s ok, you can take those learnings and build something better. Don’t settle for chaos.
Seems like a smart solution, and yet I almost never hear “throw it out and start over” as an option for dealing with unforeseen problems.
Perhaps as programmers we should take some lessons from writers, who will often throw out all their work from the past day/week/month, because once it goes into production (a published book), you can’t change it. In software, you can change it, yes. But if it doesn’t add anything useful, complicates existing things, or will slow development in the future, there is a clear reason to just drop it and move onto the next approach.
One way to make “just drop it” a more palatable option is to give yourself a grace period from the outset. Something like “if we can’t get this to work in a week, we’ll drop it and reevaluate.” This is when it helps to have at least two people (engineer + manager, or engineer x2) because it eliminates “developer pride” as a variable in decision making and holds someone accountable for making that decision after a week. As a manager, often the most important decisions are not what to build, but what not to build.
This is one I've hit a number of times, my solution in the past has been to 'go rogue' and get something done, people will be bothered (usually someone unofficially says it's a good idea), if it works out often enough, we'll start 'deciding' to do more things if for no other reason than to take some credit.
While reading, I was trying to come up with a cohesive idea as to why these things happen, or what single thing could prevent them. The best I could come up with is having a good 'Director of Development' or maybe Project Lead. I don't know what the title should be because I haven't really worked for one. I've worked with great team leads and companies with fine CEOs. CTO's are rarely close enough to the problems being solved and often just not available. I myself worked as a Director of Development for a small startup that eventually ran out of runway. The dev's said they enjoyed working together but I don't think it was long enough or going through the many pivots to test out how that would have been.
Most places I've been have business people that plan projects. Then there are product owners and PMs with teams, design/UX, front-end, back-end, SRE/devops, sometimes even multiple teams with overlapping roles to execute. This I know, doesn't work for long.
Startups that grow and promote internally to fill such a role, may or may not work--depends on the individual. What's really needed is good alignment and balance of business goals and employee happiness. Two things which are both true: startups need to iterate fast to find that magic point; employees are a company's greatest asset. You can't forsake one for the other.
Despite the quoted item, I couldn't see how to understand the (incorrect?) use of downvoting comments as disagreement rather than to discourage unwanted behaviour which doesn't contribute to a discussion. In any case, a comment as to why is always helpful.
Basically, I did see a bunch of adversity, but artificial adversity. Adversity that doesn't make anyone stronger, or better, or even feeling accomplished. Just a bunch of bullshit that one had to wade through. And if you're in the position in your career where you have alternatives, why on earth would you decide to stay in that environment?
I agree that my run through of the article seemed to point to problems more than solutions, but I have always felt that the old chestnut 'don't bring me problems, bring me solutions' was a bit shortsighted.
Yes, it would be nice if every problem had a solution provided in a gift wrapped box. Yet, it is dangerous to pretend that problems do not exist simply because we do not know how to solve it.
Now, how to do this without become overwhelmed and despondent in the face of these problems is another issue.
At first i tried to tackle those issues but it seems that nobody cares or doesnt want to accept that we have such issues in the first place. Im just happy others see those issues too and try to solve them at least by rising awerness.
Im also sure that if i tried to bring this article to discussion i would be fired next day because of "playing primadonna" one of our managers use to say...