Big company tale: six months for a list and a button
rachelbythebay.com
rachelbythebay.com
It took a bunch of teams of senior devs and other people 1 year to translate some simple excel sheet with formulas into a little Java powered site on top of a database. The cost was a few $100k and the end result was something most people here on HN can do in an afternoon (max a few days), but tech was never the issue; it was the process & politics between the IT dep (which was in another country (Germany), the head office, the hired consultants (I was one of them) etc. My job was to steer the tech people at a high level (they had their own project manager etc; that was not my job) and the consultants (from another company) that did the implementation kept disagreeing with my approach and everyone else kept disagreeing with everything else (like the color of a button). In the end it was implemented like I wanted it in the first place (and they took the credit) and I heard, much later, from one of the seniors that their PM told them to disagree with me to pump up the final bill. Which worked; luckily I never worked 'with' them again after.
This explains a lot of my experiences now
Sure they have an unending stream of work, but you rather still do far less work for far more money or rather; be able to bill far more hours for the same end result is preferable for profits. The art is then to still have a ravingly enthusiastic client at the end of the project. It's an art. Or a scam. As a techie I find it the latter but I admire how they keep the client happy, time and time again.
In reality, I have not doubt a half decent motivated engineer can replace a group of these unmotivated developers, which I have seen done and did my self in the past.
The real reason unmotivated engineers are there and things work this way in a company, in my experience, is bad management. Good people leave, who ever is left does what ever they can, those who care, suffer.
Comment: So, everything went smoothly?
This isn’t just something that happens at BigCorps unfortunately. Any company can have an anti-innovation, anti-hacker culture like this.
How did they survive being so inefficient? Huge gov. contracts, where everything takes time.
I mean, perfect place if you just enjoy a steady paycheck, while working on side-projects and personal stuff - but even that gets boring, real quick.
Have multiple stories on waiting for resources, such as images and text that took weeks to get and then were not used as they didn't follow the guidlines given ('text can be max x char long', 'image resolutions need to be w times v').
We have all but given up on trying to involve them, as it will slow everything down to a crawl if we do.
I used to work at a big tech company, but now I’m one of a few cloud devs at a startup. One day we realized we needed a dashboard, so I spun up a server and put a single binary on it that saves data to SQLite and renders HTML server-side. Dumb stuff. Took 2 days, problem solved, been humming along for a year, no plans to rewrite it.
Now let’s say another dev decided that they wanted to build another dashboard to monitor a service they just built. Obviously I’m gonna insist that they add on to the stuff I already built. I show them where to add the function (literally one function that makes an HTTP request and returns an error), it takes them 10 minutes instead of 2 days, and doesn’t add another entry point for attackers.
But now let’s say our company now has 100 devs. There’s now more status info than can fit on a single page. If you don’t do this right, the dashboard communication/management overhead at some point grows larger than the cost to develop a new dashboard.
What the dashboard team in OP’s article should’ve done is to create a self-service solution for making a new dashboard: here’s a big template, fill in whatever health checks you need in one of N languages, and tell us when you’re done and we’ll spot check the code and deploy it for you. That way they maintain control of all the dashboards, and teams get their dashboards way faster.
(Am I missing a downside, other than that the dashboard team will need to downsize?)
> Am I missing a downside
I don't think so, I'm just wondering, what to do if the dashboard team creates a buggy & hard to use, worse than nothing, dashboard self service, and everyone is supposed/required to use it
> 100 devs. ... If you don’t do this right, the dashboard communication/management overhead at some point grows larger than the cost to develop a new dashboard.
That's an interesting point!
That sounds like the situation in the OP's article. Maybe it would be best to address that as a political problem ("don't make us use this thing") instead of a technical one.
This seems quite interesting to me. I haven't really got chances to interact with mailing lists, is this really the case?
Often these initial questions are very unspecific and often as a maintainer you only get the question, answer it and then never see this person again. Time wasted.
When someone already wrote code and want to contribute it you know that the person already invested some time and is seriously about implementing some stuff. Here the likelihood that your time is wasted is much less.
So, when you don't have XP with X tech, but you wrote your hacky solution
and the person who has XP with X tech tells you that other_solution is more maintainble, then probably...
He/She's right.
On top of this, I’ve come to appreciate that maintainability is very much context sensitive. What’s unmaintable on an expensive team of engineers can be totally maintainable on a team of cheap interns. Is the solution maintainable by the engineers better? arguably. But the solution maintainable by the interns may well be good enough.
So, yeah. I try to be responsive to the needs and capabilities of teams when they request help from me in an organization.
There was a piece few days ago about picking co-founder using the army way.
Beside being a well written piece, it made clear one thing.
The role of leaders in an organisation is to get stuff done, despite rules, regulations, hierarchy and all the whistle and bells that an organisation need to create in order to exist and sustain itself.
(Which doesn't means disregard rules, it means find a way to make stuff happens in the context of the rules.)
In the article there are dozen of things that are just expected given the conditions. But there are also a lot of stuff that the author could have done differently in my opinion.
For instance figure out the resources that was suppose to create the application, pass by its office, offer its a coffee and communicate clearly why the things is important and how important it is and what is the goal of the project and finally reassure the resources by putting in whatever work tracking system all the details.
These kind of human work and relationships building is fundamental.
It seems easy to you just ask another department "please do this" and when naturally things takes too long to look reasonable complaining on the internet.
There is a world of difference between:
1. Some random guy wants a boring dashboard and it is not very clear the reason and the why.
2. That cool engineer has a real problem that really bother it and its whole team and I can easily fix it.
Yeah it is not a scalable way to solve problems but dumping work on some oscure Jira board is not scalable neither.
You are assuming that just because the manager can swing by at author's desk, their team is also at the same location. Not necessarily true. They could be sitting across the world, in a another dysfunctional team, they have a bi-weekly dysfunctional meeting and nothing functional comes out of it. Even when engineers are in the same place, some of those can be very bureaucratic/defensive/lazy/CYA type because bigco allows such people to exist in the shadows. Based on past experience, I'm only slightly exaggerating.
> These kind of human work and relationships building is fundamental.
Having said that I said, I also agree with this in the current context - that personal connection, when established, speeds things up. But then again, this doesn't always work out/is possible due to reasons mentioned above.
But even a chat message or an introduction does wonders.
My point was that, from the tale, I don't think the author did enough to actually accomplish the task and that the complaints on the internet public square is hardly justified.
Said so, I find it a well written piece that may be useful to younger engineers approaching these kinds of dynamics.
I would find extremely interesting a similar article comparing when stuff goes well and when they move at high speed highlighting the differences of approaches and results.
I suppose that then the dashboard team would have replied that same afternoon: "Done, here: __", end of story. Without needing any coffee.
And if they (the dashboard team) didn't see how the dashboard was important, they'd kept asking for clarifications until they knew, and then clearly said if they'd do it or not
I have to agree. I’ve always assumed that everyone else knows who this person is, and that she’s some sort of FAANG superstar, but each time I read these pieces, I see someone exhibiting entitlement and always looking for external blame.
I read this particular post thinking about the very first task I was given after switching from an advertising career (making Flash sites with ActionScript) to a software-service company, where I needed to construct industry-grade tools with Js.
My new tech boss gave me the task of building a dashboard that would hook into a secure database and display real-time stats using D3 to track things like number of online users and response latency. This was unlike anything I’d built before, and starting with zero knowledge I had a working dashboard about a month later.
In other words, not rocket-science, and not beyond the capabilities of anyone with even a small amount of frontend/backend experience to build themselves - or in this case spec properly so that another team could understand what was needed, and be able to build it quickly.
In the case of things going right with a lot of effort, there’s the case where things went right because of that effort. If you write stories about those, you can be accused of being self-aggrandizing - but they do give people practical options to consider in similar situations.
In the case of things going wrong, there’s the case where you tried to correct things, but were not successful. There’s value in exploring what you could do better - but that’s not a story. The story is the people and archetypes and systems and so forth. In that case, you can be accused of seeking to blame others.
The other stories could be where you did something wrong (but which at the time seemed like they were good and necessary), but despite that the outcome was successful, or because of that, the outcome was unsuccessful. If you’ve read enough of the back catalog, these do exist.
I like that these are presented as stories, and that they are representative of situations people will recognize and also find themselves in. If you’ve seen them, it’s validating that others perceive the challenges the same. If you haven’t, you’re forewarned about things that may come up.
I see what you mean but keep in mind this is where she(?) vents out, so that should make some things show more than others.
Both specifically communicated to people still working there mentioning to look up and leak videos.
Certainly doesn't seem like someone I would want to work with either.
Oh noes! Should we cancel her for that?
I feel like you're making a storm in a glass of water.
>Certainly doesn't seem like someone I would want to work with either.
But you don't work with her ... so what's the real issue here?
https://rachelbythebay.com/w/2020/03/03/emoji/
This one stands out as "you are all dumb and I am so smart".
Then also that person does not understand that one should keep hers "edgy" stuff for group of her besties. Or at least understand that other people might not be in the same context.
So for the original story posted - well I don't trust the narration. It just seems there was lack of communication.
Yeah, we've probably all felt frustrated at work due to managers, coworkers, bureaucracy or something else, and sometimes we want to vent. You don't do that in the company slack, though. That's just juvenile and stupid.
This person really doesn't sound very pleasant to work with, and I don't feel inclined to believe the retelling in the OP. There are always two sides to a story.
> For instance figure out the resources that was suppose to create the application, pass by its office, offer its a coffee and communicate clearly why the things is important and how important it is and what is the goal of the project and finally reassure the resources by putting in whatever work tracking system all the details.
That's far not always possible. An example from my previous job was a situation where the upper management wanted to enforce some changes in order to try and shine in front of the CEO. Changes which no one further down, myself and my direct manager included, agreed with. We warned the upper management that this is a terrible idea and that a lot of people would quit - in a similar fashion - in the office over a coffee (actually over a zoom call, considering the pandemic had started). The response we got was "If they wanna quit, fine, f*ck them, we'll find new people". Needless to say we announced our departure the next morning with 0 room for discussion. And same happened to the rest of the team over the coming 6 months. What I'm saying is that you are right in principle but this does not always work.
As a industry we talk a lot about agility but stories like that are more the norm than the exception. It is good to see an outside perspective on what is expected. Typical customers swallow our behaviors due to the lack of understanding but an IT ops like her just knows what is possible.
And most of the time, nobody learns from the experience when it happens to others, or even to themselves. "Learning from failure" is one of the most overused and most underutilized mottos in tech companies.
This is a story about broken parts of a large company and how something got done in spite of it, there are lots of these on HN.. They are cathartic for the author and readers who experience similar scenarios. Perhaps you were expecting some sort of moral to the story but some things are just broken and you have to do it yourself, still I'm not sure why you are picking on this author in particular.
> stuff that the author could have done differently [...] pass by its office, offer its a coffee and communicate clearly why the things is important and how important it is and what is the goal of the project [...] These kind of human work and relationships building is fundamental.
This cuts both ways - and I believe the recipient of a task (as the recipient of many) has a greater responsibility in understanding the requirements and keeping dialogue open when they do not understand something. But this is not the problem here, in this story the person tasked and the person originating the task are too far removed from each other due to corporate layers.
If this work was important, then why wait on another team who give you a "no telling when" reponse? Get the team together, and assign someone to do it same week. Don't do half-assed follow ups then complain when the dashboard team (which has already shown you it's not their priority) doesn't do anything about it.
If it isn't important, then go focus on the important stuff.
> January 29, early: there's this team that nominally owns dashboards, and they got wind of us wanting a dashboard. They want to be the ones to do it, so we meet with them to convey the request.
> January 29, late: asked "dashboard team" manager if they had been able to get the network stuff talking to our server yet via chat. No reply.
Am I the only one to think this is completely unreasoanble to message the other team later that day? I mean it's not as if I am sitting and dangling my legs waiting for a Karen to come, drop everything I do and do her dashboard. They might have had stuff planned for weeks/months in advance...
PM will adjust when appropriate; they do care. But, I can’t just build every little thing an internal customer wants because I have revenue-producing/cost-saving updates to build and deploy. Make your case that this lost+button saves more money than something else on my backlog, and we’ll probably squeeze it in.
This is a mid-sized enterprise software company and I manage one of our SRE teams.
I have no problem with another team being too busy to take on work in my area, provided they don’t actively try to take on work in my area and then pocket veto it.
Although that might still be a time-saver for the backlogged "dashboard team".
The new dashboard team manager, who appeared in the end, seemed to be a bit more "get something done" minded though
And sure, everyone has different priorities, and may not understand the urgency of tasks in the same way. And it's only human to want to promise to do things for other people, and therefore make them happy. But promises by themselves aren't enough to get work done: when I'm in the workplace, I don't want wishes and happy thoughts, I want people to deliver. If someone tells me to hold off on doing a one-day project, because they're going to do it for me, and rest assured, it's on the roadmap, even though we won't tell you when we're going to do it, because quite probably, the answer will be something like "never, as long as we have more fun things to do" --- well then, at the very least I expect to be told as much, so I can avoid throwing my project into that particular black hole. If they can't, or won't, communicate realistic priorities and deadlines, then they might be great to eat lunch with, but I definitely don't want to work with them on a project.
Office jobs, in my opinion, tend to play host to microcultures which value politeness, sympathy, and relationship building so highly that these virtues start to displace and eclipse the honesty that is the cornerstone of serious, professional work. I come across this theme over and over in these kind of "story-rants", a theme that in the telling can come off as rudeness or abrasiveness on the author's behalf. Those who are keen to interpret it as such, should keep in mind that this says as much about them, and their own opinionated view of how the workplace should operate, as it does the authors of those stories.
the author is a big favourite of HN. they're some early-100 or so employee of Google. many of their articles reach top-5 in HN
> turns out, no, wait, the button is there, but it doesn't ask for confirmation (as we had asked)
This could have been a much better article if it had laid out the exact requirements in the beginning instead of saying "we just need a list and a button". And while it sounds like the dashboard team really dropped the ball with a lot of things, it also sounds like they may have just not have had good a good specification of what they needed to deliver. If you tell front-end devs to deliver a page that just does this thing (and glosses over the details) they're going to spend time interpreting it how they want. A CSS bulldozer sounds excessive, but why not.
I think the company velocity would have been greatly improved if they had handed off a formal spec, broken out the chunks of work, and put them in an issue tracker.
If you aren’t aware of how there are teams trying to “own” turf, and prevent alternatives (even one-offs), and also how the entire company tries to funnel anything that matches a keyword to that team, even when it is the most tentative match, then you aren’t aware of the challenges faced to navigate them.
You see one path (going with the flow), but don’t see what happens when you don’t follow it.
Not going with the flow is definitely valuable - it’s something any good senior person should have in their tool belt. And if you read Rachel’s stories, there are many examples of not going with the flow (and comments in the HN posts about how she’d be more effective if she didn’t go against the flow).
The challenge is that you can’t always go against the flow either. It’s celebrated if you cut through some red tape - but at some point you’ll just get a reputation of being contrary. Even if you don’t, it’s tiring to have to be the one trying to course-correct the world. Either way, you have to choose your battles. And a button on a dashboard probably isn’t worth using your capital on…
There’s a degree to which you can just go out and talk to people and build relationships. You’d be mistaken if you think that developing these relationships (as well as a reputation for solving real problems, which puts you on the right foot with many strong engineers) is an avenue that wasn’t explored.
As someone else pointed out, a request for an internal tool is going to be sorted behind something that is revenue-generating.
The thing the author doesn't really realize in this article is that building consensus and making sure everyone agrees with the priority of the project is part of the job.
It kinda sounds like he had pet project that others disliked so they tried to kill it by claiming the "dashboard team" was handling it. Then he attempted to vicariously micromanage the project from another team.
How were other services at “big company” handling alerting? (And why would you rely on a manual process for altering anyway)
This story left me confused more than anything.