Hacker News Reader

Comment by itamarst | Hacker News Reader

The extra terrible thing is that this is completely counter-productive. E.g. see this takedown of crunch time by someone who works in the game industry: http://www.igda.org/?page=crunchsixlessons

But this is a problem outside the game industry too, though it sounds like they're even worse than usual. If you're a software engineer stuck in one of these jobs, you need to realize working fewer hours is good for you and your employer. More here: https://codewithoutrules.com/2016/08/18/productive-programme...

View on HN

You're assuming, incorrectly, that management (especially upper management) is worried about producing quality products. They sort of are, but for any of the people who mange to survive in those positions, that's a secondary (or even tertiary) concern - their primary concern is survival. The only way to survive has little to do with creating a great product and everything to do with making sure the blame never falls on you.

Well said! I'm immediately reminded of the paradox of the 4th down punt in American Football. Put simply, head coaches choose to punt far more often than they statistically should; but, they do so anyway because it ensures they won't take the blame for the "unorthodox" decision.

In this case, it's management giving the impression that they are "doing something." We see the same thing with, "can we add more developers to speed this up?" The answer is almost universally "no" and, presumably, any manager in the modern era should have read (or at least be familiar with) the over 40 year-old book, "The Mythical Man Month." But, from a senior management position, there aren't many levers to pull--and when the heat is on, they have to be seen to "do something."

One interesting source on the topic (there are many others):

https://www.nytimes.com/2014/09/05/upshot/4th-down-when-to-g...

Speaking to senior management about being "seen" doing something here. Maybe you should pull up your bootstraps and ask your team what they need, use your authority to knock down any blockers the team identifies, listen to them, work with them in the trenches, and in a way become their slave, ready to move on anything they ask of you. But hey, that's just what I'd do in the situation and be danmed if someone tried to tell me I wasn't doing something.

I don't know how much that would help. If senior management were capable of contributing at this level, they'd be an architect or product owner. They're not, so they cannot help.

A certain design and a certain set of requirements is what defines the time table. You can influence it to a very small degree by adding more people, but that's it. Nothing anybody can do will make it faster, unless they propose a better more efficient design or way of doing it. Or reduce the requirements.

Note that it isn't the case that anybody who goes into management is automatically survival-oriented as opposed to doing-good-work-oriented. It's just that the ones who prioritise their own survival tend to survive longer than those who do not. It's sort of a combination of the anthropic principle and tragedy of the commons - management is the way it is, because if it weren't, it wouldn't be where it is. Meanwhile, if your company was structured in a way that rewarded striving to produce quality product using humane practises, you will (paradoxically) make less money over the long term than sweatshops. Sadly I'm not smart enough to come up with a system that fixes this, except perhaps to run companies like democracies and do away with monarch-style CEOs. But then my employee-focused democracy would be outperformed by the money-focused kingdom next door.

UBI would help these situations. Those managers would probably invest less time in the survival mode stuff if they knew there was always an option out. Further, their staff wouldn’t need the work so desperately so the amount of stress they felt would be “capped”.

The flip side, though, is that intrinsic measures of software quality are terrible. Most boil the ocean looking to define what makes something good in the small, and completely fail to generalize into the large.

Worse, too many are interested in building a general solution when they haven't even done a single concrete solution. I have yet to find a proud developer that said, "I used the boring choices for this problem and got a solid solution."

This is akin to expecting an artisan to give better results for a standard kitchen than a builder would get from buying Ikea.

And don't take that as some sort of value judgement either way. Builders are there own to of Craftsman. Different doesn't need a hierarchy.

My gut for why this is? In a fiercely competitive field, leverage from success is the primary driver. And you get more leverage and success by being different. The upside is huge. Naturally, so is the downside. Survivor bias kicks in and people think they succeed from wiser choices, so they are incapable of reflecting and making less risky choices as they age as an org. Which requires an influx of young risk takers that have not hit a downside yet.

This is difficult to read. I'm sure you have some good points in there.

I'll blame writing on my phone while on the bus. With my stop coming up. :)

That said, probably nothing too deep. I certainly wasn't aiming for it.

My main points:

  * Most of us in engineering focus on intrinsic measurements of our code's quality.  These don't generalize well to product quality
  * Nothing wrong with being a builder, instead of an artisan.  Especially when it comes to building.
  * Most people that succeeded fall to survivor bias and think they know why they succeeded.  They are usually wrong.

There's also probably a portion of managers that cause the crunch by being incompetent up front. Not managing scope, escalations, resourcing, decisions etc leads to tight deadlines and 'sudden' urgency to meet earlier expectations that were never managed or adjusted along the way.

Yup, it's an endemic problem that has no sign of going away due to the constant influx of young people willing to take it on the nose to get into the industry.

Ironically if you look at Gamasutra's yearly survey you see a huge cliff of experience right at the ~3 year mark as those people get burned out and the cycle repeats.

I think for a lot of these people, they have never worked in any other company and maybe dont even realize work can be done another way. I’d love to ask an actual employee at one of these places: Do you realize crunch time is rare or nonexistent for most other companies? If so, do you stay where you are purely “because games?”

I remember joining a particularly toxic company and mentoring one of the junior guys there. It was his first and only job out of school, and he thought work was simply supposed to be toxic—like this was normal.

Sadly I think it's just that they're young and don't have a lot of responsibilities yet so it's easier to take on the brunt of it.

I know before I went into the industry it was at a point where crunch was widely known. Heck, some people bragged about it like a badge of honor/rite of passage.

A lot of it was maturing and realizing that there are more important things in life then giving every waking hour to a company that may throw you out on the door as soon as the title is shipped(or even sooner).

I mentioned it a little further down the thread but I saw 3 different divorces at the last studio I worked at, largely due to spouses just not being there for each other due to the long hours.

What gets me is that in almost no other industry (besides maybe investment banking) would this mentality even make sense. “Boss had me putting in roof shingles for 16 hours straight with no overtime—it was awesome!” “We had crunch time in the garage for six months where I never saw my family. I’ve finally made it as a real mechanic!” Could you imagine hearing any of that?

Having been in construction for a while, the only difference in attitude is they get overtime. It was well known that you work the first 40 hours just to get to the big money of overtime. There was plenty of bragging about how many hours you worked and how few days it took to frame a house (our record was 2.5 days on a small cookie cutter house we built a dozen times before).

It was also known that you could get a union jobs that paid a little more per hour, but because there was no overtime you made less money (you were not allowed to work overtime, not to be confused with they would illegally cheat your out of your overtime). At the time I couldn't have paid my bills on union a union paycheck, but after overtime I could pay my bills.

Most of the industries that have been that bad have been somewhat curtailed by unionization.

Hollywood is the clearest parallel to the Game Industry, and most of their abuse is stopped by unions, where they exist. The current direct parallel in Hollywood right now seems to be the Special Effects/CGI/computer animation shops which currently have an average lifespan of two years, aren't strongly unionized, and see a similar turnover/crunch as the Games Industry.

https://danluu.com/wat/

I'd hazard to say (and a lot of this is from some undergrad time in an architecture program) that's an additional factor with younger devs.

Undergrad college teaches you to lie to yourself about your productivity.

In the sense that time is equated to value (rather than work product being value). Which I think infects a lot of things.

If the game can't be delivered on schedule, and butts-in-seats time is perceived as valuable by a company, then instituting death march crunch hours seems reasonable.

On the other hand if a company values actual results, and a game can't be delivered on schedule, then it has to make a very different decision: push the schedule or increase people assigned to the project.

I'd argue that the end result of crunch is usually pushing the schedule anyway (because either more work can't get done, or sleep deprived work has more bugs), but I can see how the two perspectives would look very different to management. And explain a lot of dysfunction.

> has no sign of going away due to the constant influx of young people willing to take it

Why is the onus to change the industry on the inexperienced folk (who likely don't know any better) rather than the bosses/leadership who theoretically get paid the big bucks to figure out how to properly run their organizations?

Because the bosses/leadership are getting paid the big bucks to keep things as they are. If they felt the need to change things, they would have already. As such, they've felt just fine trading their humanity for money.

The big bucks dudes(to be clear here we're talking about publishers, not developers) are incentivized to keep the status quo. They get a constant stream of new talent that works at sub-par rates.

Very few studios are independent at this point so it's rare for any of them to have leverage. Since all the capital is organized around the big publishers unless you have a breakout success(which can be harder than the startup space) they can dictate terms and conditions.

There used to be a fun clause where upon studio bankruptcy the code/assets would revert to the publisher. Seems reasonable right?

Well what would happen is the publisher would start denying milestones for frivolous things, usually at peak headcount ~4mo from ship. This very quickly puts the studio in the red and they fold since they can't sustain a lawsuit and employ 100+ people while not getting paid. The publisher would then get source+assets, re-hire the team that was suddenly out of a job at 80% rate and ship the game with no royalty clause to the original studio + get IP. That's the kind of exploitation you see in that industry.

It's very similar to many of the abuses the Hollywood studio system has done over the decades. Hollywood seems to make it clear that the only working answer to making that somewhat sustainable for the talent, for the creative people, for all the people that aren't overpaid executive sharks and don't want to be entirely chewed up and spit out, is to unionize.

Yeah, there's a lot of parallels. In my mind it was always closer to the shenanigans that went on in the music/record industry because there's very little unionization in gamedev.

That's fucking sick game the publishers played. Thank goodness there are multiple ways better ways to fund a game studio right now instead of getting roped into one of these toxic relationships.

Observing that the root of the problem is an eternal influx of young people who think the only fun programming job is in gaming isn't putting the onus on them. You can't solve the problem if you refuse to understand it because the real problem offends you. I'm perfectly happy to put the onus on the businesses abusing these people, and to suggest even some regulation may be called for.

However, on the flip side, it is vital that we, as in, we programmers and people posting on HN, understand the problem properly so we can address it to the extent we can. I've done what I can on HN and places like /r/programming to discourage people from monomaniacally focusing on gaming, thinking that the only other thing anybody ever does in programming is relentlessly program useless CRUD apps for banks or something. The truth is that while not every programming job is good, there's a lot more interesting jobs out there than just gaming, and a lot of gaming jobs suuuuuuccccckkkk, and the truth that you need to tell young people thinking about getting into programming is that your odds of landing an interesting job are actually way better if they do not monomaniacally focus on being a "games programmer". The reality of the games industry is that not only are you likely to go on crunch time for months at a crack, you're likely to be doing it on some licensed Disney crap game, not even the AAA game you're dreaming of.

(I have no problem with people taking risks. If they insist on rolling the dice for the games industry, hoping they can make it, more power to them. But I want people to take risks with as much information as possible.)

As discussed on a previous thread -- I like making games, but I hate the gaming industry.

So today I write games in my spare time while I turn one of the giant stone capstans on an enterprise Java installation to keep food and a roof. It's not sexy work by any means, but my team is great and I get to punch out after eight hours.

Yeah. I want to not hate work. I'd even like to enjoy it with some frequency. But it's not really my primary source of entertainment, fulfillment, or enjoyment, and while there are some artistic types here who will legitimately look at that and be horrified, it doesn't particularly bother me. I am large, I contain multitudes, and they don't all have to live at work.

What are some industries to look for work in if you’re tired of working on CRUD and you want to work on some kind of visual art? VFX, gaming, etc, seem to be the only things I can think of.

What is it about programming the visual stuff that you like? For me I realized what I enjoyed about graphic algorithm is that you are constantly dealing with high level concepts like perceptional models as well as having to make a performant implementation using various approximations and what not. It wasn't necessarily the "art" part of it. Sticking to visual work:

* Someone has to create the tools that the games and movie industry has to use. Companies like Adobe and Autodesk that create 3D rendering and modeling software need tons of programmers familiar in graphics programming at all levels. Office source software like Blender is actively developed.

* Vehicle manufacturing companies like Honda, Ford, Boeing, Airbus, BMW all use tons of visualization and engineering software when designing their vehicles.

* Architectural firms use all kinds of rendering packages to visualize their buildings. VR is becoming a part of this and perhaps a place to get in early and become an expert in.

* Companies like Sony and Samsung that create panels deal with so applying human perceptual models to get colors and contrast correctly.

* Nikon, Canon, Fuji all offer hardware and software to the medial field and have to process X-ray, MRI info etc. Even in their consumer-level cameras lots of graphics know how is needed to process imaging data in their video streams.

* All of the tools mentioned above will have a user interface that has to be carefully and creatively designed to be used by creative people, analytical people etc. This is interesting in and of itself to me.

Things like video codecs and digital signal processing tickle the parts of my brain as game programming did.

Lots of good stuff here, I'll also add that most UIs are hardware accelerated so tons of stuff in that space.

If performance constraints are your thing embedded systems are almost a drop-in as a engine dev.

Wow, thanks for all the suggestions.

> What is it about programming the visual stuff that you like?

I like that at the end of it, I get to see someone interact with my application. Your example of GFX tooling sounds like something I would like, but I probably lack the experience for. I feel like the math bar for being a game developer is a lot lower than being the implementor of tools for game developers. I don't have a very strong math background. Game development seems appealing because of all the tooling and resources for learning.

> * Vehicle manufacturing companies like Honda, Ford, Boeing, Airbus, BMW all use tons of visualization and engineering software when designing their vehicles.

I have a friend who works in vehicle UIs, maybe I should ask him if his company is hiring. That being said, I don't even know how to drive and I feel like that might be a problem.

> * Architectural firms use all kinds of rendering packages to visualize their buildings. VR is becoming a part of this and perhaps a place to get in early and become an expert in.

I applied for a job once with a start up working on an architectural design application. I didn't get the job, but it did seem very interesting. Unfortunately, VR makes me nauseous.

> * Companies like Sony and Samsung that create panels deal with so applying human perceptual models to get colors and contrast correctly.

That sounds interesting but like something I don't have the domain knowledge for.

> * All of the tools mentioned above will have a user interface that has to be carefully and creatively designed to be used by creative people, analytical people etc. This is interesting in and of itself to me.

That doesn't seem very different from working on CRUD, to be honest. Occasionally it's interesting but mostly you're just glueing together widgets from a toolkit.

I guess my problem isn't that I think gaming is the only interesting place to work, it's that I have only an undergrad degree in CS / digital art from a non-top tier university. When I interview for these super interesting jobs, I don't meet a lot of people like me. There are ton of people like me working on making CRUD though. Gaming seems like there isn't a terribly high barrier to entry. That, and I love games.

The interface stuff is hard to gauge on the surface but don't write it off just yet. Many interfaces require lots of hardware acceleration to work right even though they look simple (for example Photoshop or Lightroom). Video editing software has to use all kinds of tricks to show real time updates usually involving some kind of fast approximations upfront with better and better ones blending in. This is often straight, hardcore programming to get the memory and performance of the rendering at a good place and not really applying PhD level stuff. Often times the topics you touch on will be years ahead of the game stuff. Game graphic techniques are often times things that we've been doing in other software for years and finally possible in a game(eg tone-mapping, HDR, physically based lighting, etc).

Then there are interfaces like this:

* http://www.musictech.net/wp-content/uploads/2016/07/Caption-...

Things used by creative people often don't have existing widgets toolkits to draw from and entire new UIs have to be created.

Domain knowledge can be acquired along the way. Not just for games, but in general you never want to be "Just a programmer". You want to be "X that can program" or a "a programmer that knows X". You need enough domain knowledge to be useful to experts.

For cars, I don't didn't mean the car's UI itself but rather that industrial manufacturing uses tons of software with large graphics components. Things like SolidWorks or other kinds of things that analyze the physical shapes of things have a lot of just straight hardcore programming in them.

Edit: as for education, go get some! MIT open courseware has tons of calculus and engineering classes for free. The text books can be found online often times for a few bucks. The digital signal processing course was actually fun.

I know that there are some companies around me that make UIs for various devices. Alpine and Panasonic do nav and in-flight entertainment devices, for example.

Wow. That's exactly how long I lasted in the video game industry. 3 years.

I think the answer is obviously some kind of labor organization, akin to the various creative "guilds" that already dominate the entertainment industry. Everyone else who works in entertainment already figured out that the execs will run you into the ground without this protection.

> due to the constant influx of young people willing to take it on the nose to get into the industry.

I mean, I guess. I'm in my late 30s and I just got sucked in to a project that was behind schedule by miles to try to meet an impossible deadline. The expectation seems to be that the 90 hour week I had last week is going to be repeated again this week and, most likely, also next week, come hell or high water, and any sort of common sense or evidence about how short-sighed this effort is be damned. I like my job, for the most part, but this has me considering my options.

My point is, I see this as an upper management issue. I can't fault people, including myself, for doing what we have to do to keep the paychecks rolling in, and especially when the power dynamics are as uneven as they are. You are right that younger people (and, also, let's not forget our friends in this industry on work visas) are less capable of dealing with the issue than veterans. But, from my point of view, it's not us who are wrong. Instead, it's the management chain who is unaccountable to their bad planning, poor decision making, etc... who create this situation.

That's why people are calling for a guild. A way so that developers can push back against this, and not have to worry about being fired.

I'd love to have something of a union to back us up, I honestly don't see any way to improve the situation otherwise.

I think a lot of the HN crowd has enough experience and clout to keep from getting exploited, but I'd personally love to be a part of a large union where I can help mentor junior engineers and get help in turn from more senior ones.

I think beyond shared negotiation, what a union could really help with is managing younger engineers' careers. I stuck around in a bad first job way too long because I just didn't have enough experience or exposure to successful engineers to realize how crappy my job really was.

To be fair, there are a lot of senior developers telling upper management what they want to hear, including by saying nothing and charging once more into the breach.

Yeah, and that's exactly how I ended up in my situation. I was pulled off of one top-priority project to another, magically higher, top-priority project because a so-called senior engineer wrote some checks he couldn't cash.

Young people are looking forward to overwork themselves, because older people heroize overworking yourself. Often by bragging how much they worked and exaggerating or leaving out down times or that late night coding was actually 70% beer drinking, often claiming that crunch is necessary or price for good product.

And oftentimes by using crunch as argument for why the person who works 90 hours a week should have more say in things.

When discussing crunch itself, crunch is talked about as bad. In any other context, it is praised and heroized.

Agree 100%; but there are a lot of self-deluded workaholics who don’t understand why everyone else can’t (or won’t) work like them — and these people tend to set the culture.

You have no idea how difficult it is to bring these ideas to executives on the business side. They wrote this idea of “40 hour weeks are more productive” off decades ago because it didn’t suit their needs (people in executive positions tend to get there by working longer hours than others, so this becomes the metric they push as critical for success.) “The Mythical Man Month” is something everyone reads simply so they can ignore it because “well, that applies to normal people...”

I've worked in crunch mode for an extended period in the games industry and at a startup, and the startup was much worse to me but to be fair at the startup I was the only programmer, and during the gaming crunch period (I forget, but think it was somewhere between 6-12 months) I was on a core team of about a half dozen programmers and a dozen artists and I was almost never the choke point for progress.

That was a hell of a sentence.

I think management people needs to know, lots of problem solving work, like programming, can happens in "Background Thread". i.e You are thinking / solving the problem without realising it.

Making them sitting in front of their desk for 20 hours a day doesn't really help.

That is assuming all programming work are 95% Thinking / Reading / Finding out answers then writing. I am not sure if the same applies to Game programming though.

> But this is a problem outside the game industry too

There is definitely a parallel in film and animation as well. Underpaid, worn out talented people are legion in production and post-production.

> “You’d get a lot of people coming right out of school, going, ‘Oh I really want to prove myself, and I really want to make sure that they see that I’m contributing,’(...) They either worked themselves out and would get sick or would become bitter.”

Part of this happens because management does its job well in the first place, that is stretching resources to the max. But let's not forget it happens with the tacit consent of the burned out crew. Unrealistic optimism, the excitement of being part of something big, the belief that this "crunch" is only temporary are also to blame. All that is understandable and unfortunately, these seem to be a part of the creative process.

> “[People who] take a lot of pride in this product are the people who are going to kill themselves. And those are the people you really don’t want killing themselves because they have the most value in the company.”

Perhaps more transparency is needed on the part of the crew or communication/education with their peers—especially with freshly graduated ones. One of the issues is that everybody knows this is toxic yet tends to accept the situation assuming this must be how production works.

I just wanted to say how comprehensive and enlightening your first reference was: It gives clear empirical / historical evidence of why working too many hours will ultimately kill your progress.

http://www.igda.org/?page=crunchsixlessons

Reply on news.ycombinator.com