Ask HN: Should I quit the field of software development? | Hacker News Reader

Ask HN: Should I quit the field of software development?

155 pointssage76151 comments

I have been in the field for a few years, mostly doing backend development.

I'll preface this by saying this field has never been my passion, just a profession I like.

I have been having an extremely difficult time with getting and clearing interviews. Screening rounds have become at least 2-3 hour long algorithm solving sessions which can range from dumb (requiring me to know some syntax that I am not allowed to look up) to extremely demanding.

System design rounds require me to solve problems that the top engineers in the world spend months on. One question I was asked was "How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale. In fact most of my work experience is kinda vanilla and boring, so I generally have nothing much to draw on or much to talk about.

Also, I'm not talking about FAANGs here. I never even applied to them. I apply to basic CRUD development roles, half of them entry level, and even there, the grilling has become intense, if I even get an interview.

I haven't even applied to jobs for weeks now, because in interviews I feel like I'm getting humiliated and I want to apologize and end them half-way.

I'm sorry if this is incoherent.

I'm sure the problem is me, since other people are doing fine. I need to draw a line and figure out some good indicators of when I should quit this industry, and anyone here who could give a few pointers on that, I would be thankful. I don't want to throw more time and resources into an industry where baseline expectations are something I can't/won't reach.

View on HN

I'd like to give you a different view on the interview / question you mentioned. While it's certainly possible the interviewers are running the process badly (many are), pretend they have good intentions and answer to that. The tweet question is basically: can you think about performance.

Let's say you don't know anything about queues or distributed systems. You can say you never worked at that scale. You can also say what you think would be the number of followers (millions+), what would be the time needed for a write you a database (~millisecond each) and why that won't match 3 seconds, so you know how much time you have per follower. You can talk about the problems (disk persistence is slow, network is slow, keeping more in memory is better), that having many processes / machines working at the same time will likely be the way to go. You can try to pull the interviewer into discussion about it.

You don't need to know the answer (let's be honest, the interviewer didn't know it in details either) to actually talk about why the problem exists and what you understand about it.

> You can also say what you think would be the number of followers (millions+), what would be the time needed for a write you a database (~millisecond each) and why that won't match 3 seconds, so you know how much time you have per follower.

While we are on the topic I am really curious to know how they solve it actually.

Twitter has changed this approach few times I guess, earlier it used to be simply insert tweet into a collection of tweets, and then when you load use timeline, look up the people they follow and find/merge those tweets. But it's going to create a lots of load on systems. Another approach is to maintain a cache of user's timeline(mailbox of tweets), when user posts a tweet, lookup all the people who follow that user, and insert the tweet into each or their timeline cache. results have be pre-computed, so less load. Both approaches fails when you have folks with lots of followers, so may be they use a hybrid of these approaches. this is Discussed in detail in "Designing Data-Intensive applications" book.

> Both approaches fails when you have folks with lots of followers

The traditional mailbox architectural model can be improved via a few methods though to increase scale.

1. The entire tweet doesn’t need to be duplicated for each follower, only a lookup reference (“ID”) to the tweet.

2. Unlike in a traditional mailbox sense, each user’s timeline is a bounded collection. So rather than maintaining every tweet for every followed user in the timeline, it’s capped to the X most recent.

My guess would be lots of distributed queues, some resources dedicated to heavy lifting of few vs random noise of everyone else, and storage layer which allows lots of append traffic without blocking. Maybe some storage special casing like "randos get their tweets serialised on your timeline, but because you're following Taylor Swift we'll just query that common feed in parallel rather than copying it".

They've got lots of engineering blog content, there may be a better answer here https://blog.twitter.com/engineering/en_us/topics/infrastruc...

I don't work at twitter, but one thing you can know for sure is that the tweet needs to be distributed to multiple machines to meet this performance requirement. A single machine probably could probably send three million trivial messages in three seconds, but if you mix in any business logic it's not gonna happen.

What I'd do is have a shared highfanout queue. Once you have more than say 10k followers your tweets go to the high fanout cluster. You'd have hundreds or low thousands of machines, each of which serves a slice of consumers. When the tweet is sent, you write it to this queue. Each consumer is pulling from one of the shards. If you have a thousand workers that means each worker only needs to send a thousand messages in three seconds, which is very doable.

Only about 180k twitter users have more than 20k followers. If each users tweets every 200 seconds, which seems like a high estimate, then that implies a load of about 1kwps for this system, which seems doable, especially if you have a small intermediate layer of distributors which consolidates the write load.

That's just my sketch.

Right. Seems very doable to verbally sketch it if you have ever seen something like a fan out.

If you only worked in single Wordpress servers, I could see it a big leap.

To be fair, having worked in a twitter-like domain is a valid selection criterion for any internet company planning to handle lots of traffic. It is possible that working on wordpress servers would not prepare you very well for such a job. Being unable to even articulate a basic strategy for handling a problem like this would be something I would consider a yellow flag for a potential teammate. It would make me question whether the person would be able to help me in my work.

If someone's resume was only wordpress sites, and they somehow made it into a technical interview for a team that managed twitter-like systems, I'd consider that a critical failure of the selection process. As an interviewer, I would try and make that experience as painless as possible for the candidate, and if not totally shocked by their abilities, have very direct conversations about the shortcomings of the selection process with whoever brought them in in the first place.

> I'd consider that a critical failure of the selection process.

Me too. Unfortunately, I have no control over recruiters or the questionable candidates they sometimes choose to send in. I just have to do my best to ascertain whether the person is a good potential teammate.

Yes, and my point is to not take that critical failure out on the candidate.

I wouldn't "take it out" on the candidate. I always treat my candidates with due respect and courtesy. I would no-hire them, though, if they were not a match for the team.

Follow up question for a little clarification. Would you write the tweet into the queue once or append a message to the queue for each follower containing the tweet id?

Seems to me you’d need to do the latter right? That way you ensure you process each follower at least once, but each worker doesn’t need to be aware of others it’s just pulling messages from the queue.

I don't think so. Maybe for the active users, but maybe not even then. Most twitter users are inactive and so you wouldn't want to fan that kind of data out so widely. You'd either have to pay enormous gc costs or enormous storage costs for someone like Donald Trump (0.5 GB per tweet assuming 64 bit tweet IDs). But I think I would just have a sharded in memory cache of recent tweets for the big dogs and if someone loads their feed you would do lookups into the sharded db for each subscription. You'd then cache the feed for the user so you don't have to do that lookup very often.

If you were going to write, you'd write into the cache, rather than persisting anything to disk. That's my guess at least.

I can’t speak to how it’s actually solved, but I can give a sketch from my experience. The game is generally to (1) consider denormalizing data and (2) rethink when you have the computer do work.

Denormalizing data refers to moving away from a model where you store one and only one copy of each ‘tweet’ in something like a relational database, to a model where you might actually store a separate copy of each tweet for each follower. That is kind of an extreme example of denormalization, but it’s a good way to illustrate how you could make it near-instant for any user to load their twitter homepage. If you stored the interesting tweets of every person I follow in a table just for me, you would make ‘reads’ (loading my homepage) incredibly cheap, but writes (someone with a lot of followers tweeting) very expensive.

Those trade offs exist everywhere in a system like this, which is (2), you get to decide when you do work. If celebrities with a million followers tweet an average of once a second, but people load their feeds a hundred thousand times a second, it is entirely acceptable to do 10000x more work for a celebrity tweet posting. This is actually the same concept behind using indexes in relational databases, but done more explicitly.

My personal answer to this question would probably start somewhat space-innefficient. I would take each tweet and put it into an event processing queue which writes a reference to it into each followers feed. This sounds nasty, but it scales with the number of writes, not reads, and we have less writes, and it scales linearly with the follower count of the writer. I would then improve efficiency by thinking about dormant accounts, caching, and maybe doing a bit more work on read.

The other requirement worth thinking about is the 3 seconds. How many followers are going to be watching for that 3 seconds to matter. And how much slack can be in that? Could 1.5 million followers be sleeping and not need the tweet for a few hours or only when they next sync.

If it's a pub/sub model, you wouldn't care at all about how many were watching right? You'd just publish.

And for something like Twitter, you'd probably Publish but then also log to some kind of "Notifications" store, so if a user did care but was not actively watching, on their next subscription they'd receive the messages they'd missed.

The plot twist in this system is blocking, muting, and locked accounts, which make denormalized timelines a lot harder. You basically need a very fast side system that can be queried for visibility, especially because eventually consistent privacy features (i.e., running a "account became locked" feature on a similar queue to "fan out tweets" in your model) are not treated lightly by customers.

One trick to problems like this is to solve it differently for celebrities, head queries, etc., than for less commonly accessed data. Essentially have two different systems and combine the data because designing a single system that meets both scale requirements is harder.

As of 2017, pretty sure Redis is a large part of it:

https://blog.twitter.com/engineering/en_us/topics/infrastruc...

You also don't need to know the answer in order to ask lots of clarifying questions. From the question alone, without an greater understanding of the landscape, I suspect the question is unsolvable.

I really enjoyed reading: Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems Book by Martin Kleppmann

Can't recommend it enough and it works through a lot of this talking exactly about twitter.

I work at scale that is similar to twitter, just different concerns. Billions and billions of events where the results are expected in seconds or less. This is a beautiful example of how to ace a design interview. While we prefer folks with experience at scale, we are happy to bring on candidates lacking that experience when they can break down a problem like op did. I recently hired a dev who did not know about not trusting the network and duplicate delivery of packets and who was running into these issues at her own work (but was not senior enough to be "allowed to work on it"). However, she did break down our similar interview question. Even though she did not "ace" it, we can see her potential. Hired.

With over 20 years experience in the industry, and some products I have worked on visible in SuperBowl advertisements, I can honestly say during interviews I can count to potato.

Most recently during a live coding challenge I forgot to you, you know, occasionally run and test the code I was working on, and had one hour to complete.

I can also anecdotally report code challenges seems to have become a lot more common in interviews recently.

And yet, we are still lucky to be in an industry with so many opportunities which pay so well. After several months of interviewing, and failing pretty much every code challenge, while being a programmer over 40, I finally got hired. And can happily report things are going well once again.

My advice is train interviewing. Train not just solving programming puzzles, but also doing it under the gun. That's not easy to train for real, but try to get as close as possible.

And there is no harm in keeping your options open regarding a career change. If not the interview process, but the work itself ever starts to really make you unhappy, it might be time to switch careers.

> I can also anecdotally report code challenges seems to have become a lot more common in interviews recently.

At least these are halfway representative of the job. It's a lot better than being told you're two inches tall, in a blender, and the blender's going to turn on in five seconds, what do you do.

Pleeeeease tell me you’ve actually been asked this question in an interview

No, but it's an apocryphal Google interview question. One of these days I'm actually going to weigh my head (another Google question).

I've gotten a logic riddle that involves scissors and rope, asking what my super power would be, and favorite movie. One that was at least based in math was how many ways are there for a knight to move from the lower left hand corner of a chess board to the upper right without moving back (no left or down moves). Not the DP-solution, though, a closed-form equation.

Get in a pool at the deep end with some floaties and some iron ankle weights. Add floaties and weights until you are at equilibrium at your neck line. Hold your breath and add some weights slowly to your hands until exactly when you start to sink. Drown, but include a request in your will to have them remove your head at autopsy. Have a friend, loved one, or coroner weigh it.

I fly out. If they're going to make a completely unreal scenario, I'll use a completely unreal solution :)

i think that is the correct answer, reverse the blades and then jump over them so the air current forces you out like one of those indoor skydiving places

Try to dismount the blade from the axle (they're designed to be easily separable, at least in the model I own) and just have the axle rotate and do nothing?

It took me a long time to understand that interviewers are often less competent and giving them than they are at coding.

Typically an interview question should be hard enough to test the limits of any candidate. The goal is to see how someone thinks and how honest they are when faced with a technical challenge, not what they were able to memorize.

With that in mind, take heart if the interviews have been sometimes off the mark. Most interviewers were just pulled from some normal day to ask some questions and often unconsciously see it as an opportunity to show off a bit. They have the keys and you need them.

That said, if you do not feel passionate, I would recommend playing around with some other areas like frontend or ML at home, on your own time, and see if any of them click.

In addition, I mainly have run interviews as a 1st or 2nd round screener, and I suggest to continue with interview process even if someone kinda blows a question as long as it seems like they're not lying about their experience.

For interviewing experienced candidates I'm often trying to come up with a problem the team has faced and turn it into a series of questions. When it doesn't work, I feel like there's a 50:50 chance that I didn't phrase it well/didn't include enough info (but unless I happened to interview several candidates that week and we just don't have time) I am fine with letting a second screener or the whole team talk to a candidate.

But I guess OP is probably referring more to final interviews. But for me at least, phone screens really are ok to get things wrong, in case there is anyone out there with phone screen jitters.

I've seen no evidence that most interviewers would be any better at coding.

Sorry for going offtopic

"How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?"

Looks like the interviewer has atleast read the first chapter of "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems"

If you read and understand the above book, I am reasonably sure that you can crack most of the system design interviews.

And never again need that knowledge on the job you apply for.

I would agree with you if we were talking about leetcode style data structure/algorithm questions, but system design is almost always relevant. I found 'Designing Data-Intensive Applications' to be a useful book beyond interviewing.

Even if you are not working on a product that requires supporting millions or billions of reads / writes, knowing what is overkill and what isn't, for your project's use case is still useful

Nah, you use similar knowledge all the time in lots of different systems in different fields. You’re not designing a production system in an interview scenario; you are demonstrating critical thinking, showing that you understand how to break down a problem and sketch out a solution, and can clearly and thoroughly explore the different constraints that need to be considered. These are general purpose skills.

Why do you think system design knowledge would not be used on the job? New systems are built all the time. Existing systems are extended/refactored all the time.

System design interviews IME are very applicable/productive actually. Especially ones that put you in a non-greenfield scenario.

Getting good at those requires studying topics that actually are relevant to the day to day job. The Kleppmann book referenced above is one of the bibles IMO. A software book that's actually worth the paper it's printed on.

Thought the same. He probably saw those JOIN vs Queue vs Hybrid designs and thought that he understood the entire system. Probably never even touched a system of that scale.

I feel ambivalent about this kind of question. You might argue that it's required general knowledge, but relying on having read any one particular book seems arbitrary and a "gotcha" question.

I could be missing something about US software culture - is this considered a must-read or "celebrity" book over there?

It's well-recommended in tech forum/enthusiast circles. It actually is a good book and worth reading if you're a backend dev.

But the % of devs in the US that have read it is probably below 2-3%. Most of us (devs, not HN users) don't read or do code after work.

It’s interesting how much of a bubble we can be in, that everyone’s coding/studying in their spare time. I do wonder what it actually looks like for the broader tech community.

I don't have any advice, but it's not just you.

This is my passion — I've been doing it since I was a kid and am ten years into my post-college career. I've been interviewing non-stop for six months and have been rejected at the end of every interview loop.

I'd leave the industry if there was anything else I was good at. Practically, if I do get an offer I'll just try to stay at the company as long as I can. I'm great on the job, but terrible in the interview (apparently).

> I've been interviewing non-stop for six months

Is this just market conditions right now? I'm guessing landing a job any time from 2010-2019 was easier?

I think it's partly market conditions, but also I think the interview process changed a lot in the past five years and I didn't change with it.

There are plenty of jobs out there, but a lot of competition. One place said they received over 1,000 applications for one role.

Can you expand on what you didn't change in the last five years? Or in other words: what should I focus on.

I don't want to assume anything about the parent background. When I see things like "1000 applicants" I assume these are likely to be either generic roles, or well publicised roles. Even then, most of those 1000 people are unlikely to pass phone screens.

If you go for the long tail of jobs which need specific backgrounds or unusual combinations of tools, or are simply in "unpopular" or non-technical domains, there is much less competition. It doesn't take much to put yourself ahead of other candidates, but to do this well it's a long play and it make take a few moves to get into the job you really want.

Asking what you should focus on requires more context about you - what's your background and what job do you want? Are you currently failing to get jobs (and do you know why)?

I don't think anything has changed in the last five years in terms of what a whiteboard "structured" interview is. It depends on the company. It's well known that lots of places just crib from leetcode.

> There are plenty of jobs out there, but a lot of competition. One place said they received over 1,000 applications for one role.

I don't get it. (Tech) companies out there are paying a lot of money to hire developers right now. How is this possible if they receive over 1,000 applications? If that were true they were paying peanuts because at least 1 in those 1,000 applicants will say 'Yes'.

Because it's not 1k qualified people applying to the job. Also, retaining engineers for more than one or two years is a problem our industry has.

Software Engineering jobs get applications from people all over the place. Our industry is "open" compared to similarly paid professions.

You wouldn't apply to a finance job without a relevant degree or experience, you need to go med school or pharmacy school for medical fields, you need to go to law school or at least pass the bar exam for law jobs.

Meanwhile I work with devs who studied Comp Sci, random liberal arts, other types of engineering, and some who dropped out of college entirely.

I think it's a net good that we don't gate keep the profession but a downside is that everyone applies to every job. For example, bootcamp grads are told to blindly apply to jobs. So are college seniors.

I wonder if you’re suffering with imposter syndrome - the feeling you’re not cut out for the job and everyone else is so much better. I see you’ve asked similar questions to this since 2016. It’s something most of us face in our careers and really sucks when it happens.

Look at some positives. You’ve been working in the field for a few years so you can clearly do the job. You’re applying during a pandemic and the market is tough right now. You’re clearly interested in tech or you probably wouldn’t be hanging around on HN on a Sunday, and this in my mind already puts you above many others!

So what can you do? Apply for jobs that don’t require algorithmic/systems interview tests. They do exist, I’ve worked for a couple! Or, game the system by studying interview questions (Crack the coding interview, competitive programming sites) so you’re more prepared. Of course you could also look for another job in a different industry, but I don’t think you really want to and are just a bit anxious with interviews. Finally, consider moving into SDET. A background as a developer makes the transition quick, you’re immediately competitive compared to those without dev backgrounds, and salaries are similar. They don’t tend to have the same interview questions.

> I see you’ve asked similar questions to this since 2016

That's good context. And to OP's point,

> I have been in the field for a few years...this field has never been my passion, just a profession I like.

I've been programming professionally for 10+ years, 20+ when you add in college and what I did growing up. That, and computers are a hobby of mine, and I definitely have a passion for them, and I know a lot of stuff beyond my day-to-day work. I've worked for multiple FAANGs, at other big name's you'd recognize, and startups you wouldn't. I get past phone screens probably 80%-90% of the time and get offers 33%-50% of the time.

There are a lot of people like me in the industry. We're not a majority, but it's hard to compete with us, it's not helpful to compare yourself to us, but it's hard because that's what interviewers compare you against.

> Apply for jobs that don’t require algorithmic/systems interview tests.

Yup. Companies also go easier on system design when someone has ~4 years of experience.

> Crack the coding interview

I can't remember which one of these I skimmed, but I didn't find any of it super helpful. I might be the wrong audience.

> competitive programming sites

Not the "competitive" ones, but if you haven't done 50 medium questions with some easy ones thrown in on HackerRank before doing a round of interviews, you haven't prepared. I'd actually avoid the hard questions because they usually involve a twist on one particular algorithm that will take a while to implement, so you'd never actually see it in an interview.

The important thing is to know your way around heaps, trees, tries, linked lists, graphs, etc.

I found this book helpful, but I read it specifically because I'm better at coding than algorithms. It's not great for knowing how to combine standard data structures to solve problems or how to traverse a tree.

http://mimoza.marmara.edu.tr/~msakalli/cse706_12/SkienaTheAl...

> We're not a majority, but it's hard to compete with us, it's not helpful to compare yourself to us, but it's hard because that's what interviewers compare you against.

This makes me so sad. I've prepped hard on leetcode for 6 years (since freshman year of college) and I haven't had much to show for it but a total compensation less than Stanford new grads despite two years of tenure. I wonder if there's any point of even living most days.

Don’t be so hard on yourself mate, you aren’t defined by your job or salary. We should try not to constantly compare ourselves to other people like this, there is always somebody better off or more successful.

Based on nothing but these two comments I’d much rather hang out with you than I would the commenter above you. So there’s one thing you win at.

Yo buddy, no need to compare yourself with others. You should compare yourself now with your past self six years ago: are you a better person now?

If you've spent 6 years doing leetcode and you're still bad at it, then perhaps realise that such problem solving games are not for everyone. Like sudoku. I prefer the word game where you find the word in a grid and I hate crosswords.

We're all different and that's what makes us unique and interesting.

> You should compare yourself now with your past self six years ago: are you a better person now?

Kinda? I had more potential then, I could have done Facebook after sophomore year and had my career set for life. Instead I failed that interview and my career has been treading water because I had to join Amazon out of undergrad.

Its apparent that I"m not good at these things...the problem is my happiness and financial security are related to them and I'm not doing well!

> I had to join Amazon out of undergrad.

You're suffering for a lack of perspective or gratitude. If you're a millennial, realize that you were probably told that you could do anything (and thusly everything), and the effort it would take was trivialized... Both are incorrect. You most likely can do 1 thing super well and it's going to be a shit ton of work if you want to compete against those equally competent.

In what way? I grew up Asian and went to a state school, I got the message that I was incapable of accomplishing anything from an early age!

> state school

5 of the top 10 CS programs are at state schools.

> I got the message that I was incapable of accomplishing anything from an early age

Apparently you took it to heart and can't recognize what you accomplished.

> a total compensation less than Stanford new grads despite two years of tenure.

> I failed that interview and my career has been treading water because I had to join Amazon out of undergrad.

WTF? You have an engineering(?) job at Amazon and are complaining you might have done better if you got the internship(?) at Facebook? And that with two years of experience, you make about as much as a new grad? Is this a humble brag?

You're probably in the upper 25% of tech for your experience and the upper 1% in the developed world for your age group.

Is the meat of your complaint that you'd be making more money if you did a stint at Facebook? Don't think it "sets your career for life." Big companies can even hurt carers long-term because you're not exposed to as much. Facebook on your resume will get you calls from Google recruiters and calls back from wherever you apply, but it won't inherently help you pass future interviews.

> I wonder if there's any point of even living most days.

This means you're valuing the wrongs things w/ the wrong weights. Compensation is to help you live more comfortably. If your pursuit of it is making you less comfortable (than the compensation received) you're doing it wrong.

I went through something similar, and ended up failing a bunch of coding interviews. Also I read David Graeber’s book on bull shit jobs, which was really cathartic for me.

I ended up quitting my job, selling my house, and now I’m looking for some property to buy so I can subsistence farm.

This is really awesome - but I feel like it's also an extreme that few are willing to take. One thing which is an argument for software development is that the relatively good salary should enable you to save up and take a sabbatical every 4-5 years. To me, that is absolutely worth doing this job, even if I don't always like it (also, there's also almost nothing I would be qualified to do, at the moment, to be fair).

For sure, and it’s definitely something I never would have planned or expected for myself. It was also partly influenced by COVID, and further compounded by pretty rough break up.

But the funny thing is, I really feel more comfortable and confident than when I was “succeeding”. I used to feel so guilty about a one line code fix taking weeks to deploy, or I used to feel so guilty and terrible for not paying attention during scrum and refinement and all those meetings. I always felt like I was working too hard and not hard enough.

Farming has this reputation of just interminable toil and physical labor, combined with high costs for agro chemicals. But then you read about farmers like Gabe Brown or Chris Trump, idk.

It only takes three nonlinearities to create chaotic, non predictable behavior.

I never had anything to do with farming but I can imagine its appeal. Living from the fat of the land and not caring about planning and preplanning and retrospectives and all that.

Best of luck!

Will look up Gabe Brown & Chris Trump.

About the three nonlinearities, I'm afraid I'm missing some context - never read up about chaos theory...

If nothing else I can imagine the hard work, hours and constant threat of bankruptcy of farming would make me very happy to get back behind that keyboard again!

> I always felt like I was working too hard and not hard enough.

Wow, you've succinctly described how I feel about my dev career spot on.

Not sure if its the exact same sentiment, but I've started to describe my work as "hard in all the wrong ways".

It's true that a lot of software jobs are in fact BS jobs. But is this a conversation that the industry is ready to have? Probably not.

I've read several stories that start exactly with what you're doing (quitting job, selling everything, moving...) where the author didn't stick to the plan but gained a lot from the journey anyway and ended up back in the workforce, except they were more productive and happier than before.

I think it takes a certain amount of reflection to reach a mental state where you become the person you want to be. That is crucial to avoid subversion of the self to the corporation. Once you get there, you can have a more symmetric relationship with your employer. Both parties end up better off. It's a shame that society often doesn't have the "speed bumps" which allow people to do this without going through radical ups and downs, but it is harder to change society than it is to change yourself. I hope you get what you need out of your journey.

Perhaps, but I doubt that I’ll ever be able to go back to a 9-5. Thank you for the kind words, they really do mean a lot.

Were I to ask these questions in interview, I think I'd be looking for the candidate to have a bash, do some napkin math, come up with some contraints/failure modes and to communicate their assumptions well.

> How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?

In this example, I'd be looking for you to have fun, ask a bunch of questions and to make some sensible decisions based on the information you have.

To be honest though, it does seem strange to ask for a "basic CRUD development role". Admittedly, if I had an experienced candidate apply for an entry level position, I would be a little leery. Why do you think you're entry level despite a few years experience?

I'd maybe take a break from interviewing, and use the time you're currently spending on interview prep on building something as a personal project. If you enjoy it, then hang in there. If not, maybe you should stay at your current gig (you don't mention anything about this) and figure out what you do enjoy.

P.S. I'd also stay of Hacker News for a while. While I love it, I will admit that there's a higher degree of obsession here than in wider industry.

Sometimes, the interviewer will purposely give you a question beyond your reach. There are a few reasons.

- To zero in on what you already know.

- To see how you react in a situation where you don't have all the information.

What is your attitude when you are in this situation? Do you shut down? Walk out? Or are you able to form open ended questions to get the missing information?

In any type of job, you're always in situations where you don't have all the answers, and need to work with others with a positive attitude. I don't know how you are going to turn this around, but maybe looking at the situation from another point of view is the first step.

Just the fact that you are asking the questions tells me there is hope.

I ask questions beyond the job scope in interviews, but I when the applicant looks nervous, I tell them that's what I'm doing. I'm often pleasantly surprised at the answers and it's one of the more useful aspects of the interview for me.

I would much appreciate the info that that's what was happening. The context helps to calibrate correctly.

I'm sure it would make you less nervous (which is why I tell them), but would it actually help? Either the role is above your skill level, and it won't matter how you answer the questions, or it isn't and doing your best will be good enough.

Either way, there's no need to be nervous. IMO, you should focus on just answering the questions as best you can.

Helps as is in is this person looking for a specific bit of trivia that they'll write me off for not knowing or is it just an exercise to see how I reason my way through unfamiliar territory.

The fact there are so many hidden requirements you're trying to tease out is where the nervousness comes from.

One of my best interview experiences was when I was asked a few relatively simple questions, looked over a class of their code and describe what it does and some mistakes that might be in there, and went over a code sample I brought in.

Then he said, "Okay, I know you can do the job, now I'm just curious how much you really know."

And then the really hard, low level questions came out. I said I didn't know a few times, and struggled even when I sorta knew the answers, and he took it in stride and played the role of teacher and taught me a few things that I still remember today. I didn't feel pressure like I did in other interviews, despite being much harder questions than I get asked elsewhere.

If you do it that way, I'm cool with it. If you start right out the gate with the hard questions, and you make no indication that you're not expecting right answers, I have no choice but to assume I better get them right, or else (especially considering I haven't moved on to the next stage of interviews before for getting one question wrong, or saying I didn't know for one of the questions, or the HR person was looking for a particular keyword I didn't say, quite a few times).

[these are my opinions and not those of any of my current or past employers].

I feel for ya. And I think while passion for software is really great to have, given how our interviews are structured it won't necessarily help you clear them. Our interviews are now all about process. There is such a huge supply of entry level talent (crazy compensation and low barrier to entry) that companies are looking for a standardized way of measuring candidates. This means preparing for interviews will have nothing to do with your role (unless you are widely regarded as a specialist/expert in your field). The good news is since this is standardized it can be hacked by a little trick called preparation. I can assure almost every question in a tech interview can either be found leetcode or in the system design primer. All you need is a month of preparation (or make monthly interviews a hobby like going to the gym so you stay sharp). It is all about mastery and mastery needs consistent practise.

Instead of thinking you cannot do it, I'd encourage you think in a different way. What is keeping you in this field (money? Friends? Domain?) Is it enticing enough for you to put in the effort to stay with the pack for the rewards?

Honestly, it’s not you.

These days I contract so it never comes up - but I used to resist requests to help out with hiring other than putting in a good word if I know someone personally. I hated interviewing applicants as much as sitting interviews myself.

There’s not a great way to find out in fifteen to thirty minutes if someone is competent or a good fit so we resort to asking these nonsense questions because it’s the done thing. It’s a bit like Agile. There’s no good reason to do it, so we just do it until someone comes up with something else to do.

I do pretty well at interviews these days if only because I’m relaxed (contracts are usually short-to-mid term so I have a very “it’s just one gig” attitude and tend to crack jokes and ask about the team rather than the project - I’m pretty sure I can do the work, I’m not sure we’ll get along 8 hours a day so that’s more important for me to find out).

To your question, if you’re thinking about throwing in the towel - I wouldn’t if it’s just because of a problem of interviewing.

You have experience and are therefore clearly capable - but life’s short and as you’ve said it’s not your passion you might be happier doing something else.

So the best answer I could give you brings you back to where you started I’m afraid:

Only if you want to.

I was in the same boat. I decided to try the research software engineer path (not scientific computing, they are different). The work here is reasonable: Build a maintainable piece of software, no need to scale for billion users or sth like that.

The downside is that the jobs are contract-based, as is every academic jobs.

So I'm thinking of switching to a more practical, less competitive driven field as well.

So your first mistake I believe is to think the interviewer is just testing you on a narrow exercise.

Approach it differently: most of the time they are rooting for you! Hey you might be their future colleague. That in mind, focus on tackling the exercise showing how you think about it. That's mainly what they want to see and if you are transparent with them about the fact you don't know well this layer so it might be a little bit handwavy it is fine, manage expectations and when you guessed or rebuilt something existing right it might even play in your advantage.

On the passion side of it, I have to be blunt, it is a self fulfilling prophecy here. If you are not interested by the field and related fields, the lack of curiosity will impede your growth. You won't get the next interesting wave, you'll stay on mainly boring things and this will reinforce your feeling that the field is boring. Computer engineering is a non stop learning endeavor.

For the system design interview I would blame your recruiter for not preparing you for what to expect in the interview. Larger companies like FAANGs actually do a very good job at that. The Kleppmann book others have mentioned is of course excellent for these interviews, but here are a couple of other resources for a crash course:

1. https://github.com/donnemartin/system-design-primer

2. https://www.educative.io/courses/grokking-the-system-design-...

Most interviews unfortunately have little to do with the actual work and require significant practice to do well.

> One question I was asked was "How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale.

If you're technically competent overall, you should be able to learn enough to make your way through this from a few hours of reading and watching youtube videos. That's what I did when I was in the same situation, and I passed the architecture interviews at multiple FANGs.

Same for the coding questions. You need to spend time reading interview books and doing leetcode.

> If you're technically competent overall, you should be able to learn enough to make your way through this from a few hours of reading and watching youtube videos.

I actually did read up a bit on it later. This may not be what twitter is doing right now, but in the videos i saw, this 3 second thing is not possible. You can batch process the tweets and use websockets or whatever to send them, or just let users get them when they log in to their accounts. That's what i read anyway.

I don't care to look much deeper unless I'm actually working at Twitter, and it seems that's my problem.

The biggest hurdle for me to get the resume selected. How do people do that!?

It's easiest when the recruiter contacts you. So Linkedin profile, email on Github, etc.

Starting out is hard, the first job really is the hardest, and I doubt recruiters are looking that hard.

Once you have several recognizable, well-regarded companies on your resume (or Linkedin profile), even sending in your resume through an online submission will normally get you a callback. That, and you'll meet at these places, so the next time you're looking for a job, you check with friends to see if they're happy where they're at and if they'll refer you.

I'm starting out. How can I make recruiter contact me?

This doesn’t address your question, but: don’t be afraid to push back during an interview. Too often, interviewers get away with questions that have no relevance to that particular role. Also, it’s ok to turn down an interview request if it doesn’t meet your criteria. This can be hard especially if one isn’t employed currently, but it is better to set your expectations beforehand, rather than waste hours/days.

Interviews, and in fact, most first meetings between people, are often shibboleths: a perfunctory test of similitude to prevent outsiders from getting in and causing unnecessary angst by thinking differently. (In Biblical times, apparently, the word "shibboleth" could not be pronounced by The Other Tribe, who pronounced it "sibboleth", and were denied entry or worse, killed by the guards).

If you're OK with this, then interviews can be an enjoyable game with rather predictable behaviours (and even predictable questions, probably taken from "Cracking the Coding Interview" or equivalent online course). You just have to learn to jump through the hoops. Just look at leetcode or google, "how to pass tech interviews" and pay up for a course on it. Just remember to not roll your eyes during the interview. (The real issue arises if you start hating this process or the thinking behind this in the industry).

You could get lucky and find employers / interviewers who assess general intelligence, or better, find employment through your network of friends, ex-colleagues, and mentors, which would bypass this process.

All the best!

Keep doing your best. I am not often involved in interviews but when I am I look more at how an applicant handles the situation more so than looking for a technically correct answer. Well except for the questions that cover fundamental knowledge. I expect those to be answered correctly. Regarding the tweet scenario - if you don’t know a specific answer tell them that but also reach back into your experience and describe to the interviewers of a similar challenge that you succeed with and how you solved that. The floor is yours to convey your strengths and passion. With that said I would personally hate to have to go looking for work. I am a sysadmin who honestly wears too many hats to be an expert in any particular field. Most of my knowledge is institutional and my expertise is very focused on the areas I need to know. Another thing I have learned during interviews is to look for someone that comes across as genuine and caring. That goes a long way in a team environment. As far as quitting the industry only you can answer that. To me passion is key to both excelling at and enjoying your profession.

I don’t think I hate the field . I have my niche corner that I love doing. I tried to get out there and do something else and had the exact same burnout feeling you are having about tech interviews.

Taking a break from this all for now. I cannot afford to spend a few months monkeying around leetcode to work on stuff that I know I will never need in the job. I’d rather spend that time with my family.

Wouldn't the higher salary translate to retiring younger = more time with family?

I used to think so, when I was single and working more than I should. But, I have a one year old at home now, and I’d give anything to reduce stress elsewhere and spend more time with him. He is growing so fast right in front of our eyes, and I do not want to miss these moments in our lives.

Of course, I’m not talking about slacking to the job to use family time. I’d like to work in a low stress environment and shut work off when I need to. Interview prep demands me to spend a lot of hours on toy problems in leetcode which of course I can’t do at my work time. So, I decided to not do it, atleast now.

On HN there have been numerous interview horror stories that I just didn't understand. Recently though, I've been through one, so I think I understand what you have been through a bit better.

Interviews are done by people, trying to size you up. Even though their question would seem to be humilitating you can still try to make the best of them. As others have already posted, getting through the interview is a skill on itself and I have seen interviewees that passed full score and have been terrible to work with and vice versa.

The real question is: is this the only profession you care about? If so, it's time to learn to game the interview. If not, do feel free to give a shot at the other profession. It may benefit from your experience as a software developer.

As many have posted on HN before, interdisciplinary employees have higher employment value.

Anyway, good luck with whatever you decide.

https://github.com/poteto/hiring-without-whiteboards

no don't quit. Just table this thought for a 1 year and do the following in the meantime.

1. Grind leetcode 2. Grokking system design github

focus hard on these for 1 yr and try again. If you are still having hard time then yes quit by all means.

Recently i interviewed a guy that failed all the questions. During interview.

I know he can do it. So we hired him anyway. He turns out to be really good and started contribution with in days.

Corporations really needs to know what kind of problems they are trying to solve. And see if the candidate is right fit for the job.

What signals did you use to determine he could do it if he failed all the questions?

One of the most important things to discover as an interviewer is a candidates thought process. The actual solution is often not that interesting, but how the candidate rose to the challenge, what questions they asked of us to improve their understanding, and how they explained their solution tells us a lot about their work attitude. While it would be great to present them with problems from our everyday work, that often requires too much background knowledge to cover in an interview.

The aim with these questions is to discover what it would be like to work with the candidate.

The interviewers are not necessarily waiting for you to memorize or come up with a working solution, but for you to prove that you can be a self-thinking, communicating member of the team. Especially if you apply for a CRUD-app development role.

To provide a perspective from the other side of the table: I conducted a good share of job interviews for my company and would likely pose a comparable question to the applicant mostly to determine a) their ability to grasp a problem and to reflect on the degree of their understanding of it, b) test their behavior in situations that are over their head and c) test their ability to come up with and discuss solutions based on reasoning and the facts given.

For a) I would expect the applicant to identify what information is missing and either to communicate assumptions for the white spots on the map or to pose relevant questions. Regarding b), I would expect the applicant to openly state if they feel that the challenge is above their head _but_ try to give me a solution nonetheless. Which brings me to c): I would not expect a working solution; not even a complete one. I would like to see whether the applicant can work on given input and outline the reasoning behind the suggested solution in a coherent and understandable way based on that input.

My bottom line for you would be, that if you think what I said might apply to the interviews you experienced, you should try to re-frame such questions for yourself. Unless you see yourself unfit to adapt your approach (which I do not think, as ASKing-HN is proof for existent self-reflection), I see no reason in what you wrote that indicates that you should leave the industry.

As someone who might quit the software industry myself (after 30+ years) very soon, I can give you a lot of reasons why that might be a reasonable decision ... but the interview process, as bad as it often is, isn't one of them. There are still places that haven't succumbed to the mania for interview processes that are supposedly meant to minimize false positives and/or eliminate bias but have really become more of a hazing ritual. You just need to keep trying until you find one. It took me a long time to find my first programming job, when I had neither degree nor (obviously) experience, but I kept at it and eventually got my foot in the door for what turned out to be a pretty good career.

It might also help to remind yourself that often the goal of these exercises is not actually to extract a technical answer but to observe how you think through a new problem, ask clarifying questions, handle disagreement, etc. Just stay calm, stay personable, ask questions, be willing to say "I don't know" if that's the case, and remember that another person's hasty judgment is a reflection on them rather than on you.

How come you might quit the industry?

In a nutshell, because I feel like I can. I've done this for a long time, often at high intensity, and I'm feeling like it's time to do something else. Almost everyone will get to this point sooner or later, no matter how passionate they were when younger. The relevant part is that pursuing one's possibly-non-remunerative dreams becomes a lot easier when you're sitting on years' worth of income from working in tech. Quitting too early foregoes that opportunity.

I feel your pain.

One thing I would suggest is that programming/software engineering is a broad church. Not everything is large scale backend work. I personally get a lot of satisfaction out of frontend development, and the expectations there seem to be a lot less about hypothetical google-scale nonsense that will never happen in the role, and more "do you know the tech?". Despite what people say/think modern JavaScript/typescript is good to work with and in modern web apps that are 10s of thousands of lines of code there is often a surprising amount of actual "engineering" required.

That said, sometimes just knowing some of the basics might be enough compared to other candidates. E.g. for the tweets in 3 seconds thing, you could talk a lot about horizontal Vs vertical scale and what might mean etc etc and go from there rather than a "sorry I have no idea". You don't always need to be "correct" but just show you are aware of the hypothetical solutions to their hypothetical questions.

Good luck. Hang on in there. I appreciate this can be high stakes for tou, while posting an answer here in a comment is zero stakes.

I don't have recruiting experience in the field on either side in the past 5 years and likely am in a different country anyway, but from what you wrote I'd tend to believe that your selection of companies you applied to was a bit unlucky/wrong. Try smaller companies that aren't 100% IT focused so they have no pressure to hire rock stars to fix their issues with their 800 mediocre software engineers.

> I'll preface this by saying this field has never been my passion, just a profession I like.

You need to realize you're competing with people for whom programming really is a passion.

Also, perhaps more important, ask yourself if, at any particular time, the programming job market is a seller's or a buyer's market. If it's a seller's market (meaning favoring the programmers), after a brief interview employers will take you on as an intern for a week and see if you can produce results. If it's a buyer's market, the employer will ask hypothetical questions having little to do with actual programming, in a process that may seem to be intentional discouragement. From your description, your interview fell into the latter category.

The intern approach addresses a well-known problem in the field -- interviews don't efficiently identify productive programmers, they identify whiz kids. This is why the intern arrangement has become more popular over the years.

But the bottom line: if find yourself asking whether you really want to be a programmer, chances are, you don't.

This type of interview is the equivalent of the SAT. They don't measure absolute talent. It doesn't mean you are a bad programmer or even a bad systems design person. It doesn't mean you might not be a good manage, executive or anything else either. What they measure are skills in a finite domain.

Fortunately, these interviews are coach-able so it might be worth paying for a coach. It will be one part learning things (likely again) and learning how to demonstrate "you know what they want you to know" and if you don't that they know you would know how to figure it out and are motivated to.

These types of interviews are kind of an "abberation" too. The traditional way to get a job is to ask questions of your interviewer and be genuinely interested. Look up the social and psychological aspects of interviewing.

I suggest sticking with it.

If you do enjoy development day to day, and can get your assignments done, those are the actual skills you need.

Interviewing is another game entirely, and it sounds like you've had a lot of unfortunate interviews that were more about the interviewer stroking their own ego than finding competent programmers who can do the work.

There are jobs that don't have that kind of crazy running the interviews.

Spend a few hours brushing up on Fermi approximation and mapping problems to known algorithms / data structures / tools, since they're useful skills anyway and will help your interviewing technique.

Apply to lots of places, do lots of interviews, try to answer the questions as best you can, and don't worry about it when you hit interviews where they ask nothing relevant to your day-to-day work.

Good luck.

You are lucky software development is not your passion — see the post about BS jobs. I still code because I am passionate about coding. I am also passionate about sailing. I once asked a fellow sailor (who was really amazing at sailing) why they didn't sail for a living? They said working at it for money would totally ruin their passion for sailing.

"How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?" I really don't care, and neither should you — that's a bad gig, but I really liked reading the stories about your interview experiences. Bring a humble piece of your CRUD code to the interview, and wow them. If you still like software development, keep doing it as long as it is a good gig.

> Bring a humble piece of your CRUD code to the interview

Nobody ever looks at my github. I don't know why this myth is running around that you need to have an active github.

I put all my work that I did on my startup there. It was 99% me since I was the only engineer. I frequently invite interviewers to look it up esp if they are looking for someone who has worked in that tech stack.

Never has anyone looked at it, asked me anything about it.

The people I have actually seen get jobs were those who

1) Applied in a spray and pray fashion. Mass spamming. 100+ applications a day.

2) Ground leetcode like their life depended on it.

All those guys with amazing projects and great github profiles struggled more than those who simply did leetcode all day.

So thanks for the inputs.

I got that system design book, but i feel like algorithms and systems design together would require 3-4 months of preparation at least, and I am sure to lose my visa by then.

Some people asked me about my passion, I don't want to reveal that but it's something that normal people would consider 100x more competitive than software, and honestly, I don't feel that. It feels like I'm closer to making it in that than software.

In my passion, to get hired you get judged exactly on what you do, so it's trivial for me to get hired. Everyone asks for your portfolio, I have a good one, and that itself is a major foot in the door.

I am not sure if I will continue in the field, but I am sure what little interest I have in it is waning daily.

Rule #1 of software industry: Never judge the work nature by the interview. Cracking interviews is a different skill which is completely different from what you need to solve real problems for real people.

If you quit now, it means the gatekeepers have won and I don’t think these gatekeepers as a good enough reason for someone to quit. As to when to know to quit: there will come a time when you will burn out. When writing code doesn’t excite you as much as it did before. When you find yourself in this zone for say 3 weeks in a row, take a break and try something else. There’s a fifty fifty chance you’ll comeback and about the same odds that you’ll find something else interesting too.

The interview process for software engineering has been broken for some time. Don’t give up.

> I have been in the field for a few years, mostly doing backend development.

Have you reached out to former colleagues to see if their current companies are hiring and if they'd refer you?

I'm over 10 years in and it's worse for me. When I was less experienced I had problems with technical interviews and got rejected for any mistake or slower-than-expected answer. Now I have the experience and ace the technical interviews, but get rejected because of salary or culture fit (i.e. the interviewer doesn't like me).

You can't win in this field. Hiring is so random that the main advice everyone is offering is to apply everywhere and pray. I regret choosing it.

The job market is exceptionally tight right now, so don't take it personally. I doubt anybody is giving good answers to stupid questions.

The quizzes and puzzles style of interviewing is terrible. Don't stress about it. Just be yourself. A good answer to most questions is, "I'd look on stack overflow and some blog posts to see what other people are doing." If they don't like it then you probably wouldn't enjoy working there anyway.

You must have been popular in school. I grew up playing with rubik's cube and tavern puzzles. Programming interviews are designed to find people like me.

I like puzzles and quizzes plenty but it has nothing to do being a good programmer.

Try applying for roles outside of tech companies if you haven't already. It can potentially be a lot less stressful while still being fulfilling and paying well.

I apply to insurance companies, banks etc, roles which are not glamorous.

Same deal there, in fact I just bombed another test. Had to do a whole bunch of things to a graph (4 part thing) in 45 minutes.

One look at the question and i knew I was toast.

> System design rounds require me to solve problems that the top engineers in the world spend months on

Usually with these they just want to see how you think. Nobody's really expecting a complete answer, much less something you could go implement and have it solve the problem flawlessly from the get-go. They just want to watch the gears turn; don't be so hard on yourself.

> How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?".

On the receive side, sample the current time, subtract 2 seconds, and adjust the message time stamp to that.

The recipient of a message from a celebrity has no idea when it was actually sent; they are not engaged in a real-time chat with that celebrity via another channel.

What are you passionate about? Find out what that is and see if they can use engineering at all. You’d be surprised at what you will learn to do the thing you love. We all have things about engineering that we don’t necessarily like but that isn’t any reason to give up. Keep trying, I hear the market is hard right now due to some type of pandemic.

The challenge of quitting software is clearing stating what other job you would like to do. I just recently discovered there are many police officers who were former lawyers that quit the law out of frustration and enjoy policing so much more.

Sure do something else for a while. In my experience, skills transfer well by experience. It's experience that counts.

Sounds like you're pushing a rock uphill and you need a change. Good luck.

You'll have to, soon or late.

Sooner or later you'll start facing ageism. It is rampant. The interviewers will look younger, the interviews scarcer, the technology stack will move to a new shiny thing that is a rehash of ideas of previous fads of old, people will start using arguments like "fitting into the culture",...

I read somewhere a long ago that software developers have an half-life of 6 years. If you're long past that, better really start acquiring other skills.

I'm a 44 year old software engineer. I've done other things before, but I've gotten an offer every place I've interviewed.

This simply isn't true. It might be true in silicon valley or some other hipster magnets, but there's definitely a market for those of us who have been around for decades and been solving problems for decades.

I am based in the North East, also over 40, and my experience both of you are correct. Agism is absolutely real, but opportunities still exist.

What can be done about it?

I'm just starting out in my career, but I find the prospect of future me being devalued because he's a couple years older terrible, and I don't understand why others seem okay with it. Aging is going a second per second for everyone, isn't it?

The point is that there's really nothing to worry about. There are plenty of jobs for those of us not in our 20s.

Agism is illegal already, and yet we still deal with it.

As a young developer I watched great old developers, fantastic at their job and with super deep experience let go first.

My advice is make sure you save a lot. And be prepared to either retire early, switch careers, or start your own business/consult once you have over 10 years of experience. Connect with people, keep in touch, and make sure you stay current with new technologies.

As an outsider to the industry, I have always wondered what the situation is like in smaller companies or for starting your own business to serve local clients.

Part of the reason why I ask is that most of the places that I have worked for respect experience more than youth, and couldn't care less if the firms that they were contracting work out to employed experienced developers in their 20's or their 50's.

As an outsider to the industry, I have always wondered what the situation is like in smaller companies or for starting your own business to serve local clients.

Small companies, typically startups, are diverse. Some like hiring diamonds in the rough, which can include old developers. Other are obsessed with youth.

Starting your own business or consulting are standard and popular options for old developers.

As an industry insider, people complain as loudly as people from any industry. But just look at the actual unemployment and pay rates per industry. Few other industries are better.

Do you stay current?

I see resumes for similar aged people who know cold fusion and Delphi, and those that know JavaScript and Elixer.

I suspect interviews go very differently for the two groups.

I'm also a software engineer in my 40s who has not had trouble getting jobs. My answer is yes, I definitely keep current. Right now I'm working on Gatsby in TypeScript. 10 years ago it was Phonegap and native iOS. 20 years ago it was PHP and a bit of Mac software in Objective-C. By staying current I can negate the main objection that employers have to older developers, which is that our skills are outdated and we're unwilling to learn new things. It's more fun too!

Absolutely. C++, C, Java, Python, learning Rust now. Working on another MS just for fun.

I don't do web dev so I don't give a shit about javascript.

Yah it doesn't mean you need to know everything, but you have to know something current or I think life gets hard for you..

Advice I give to anyone (with career options) that is considering this field is that serious work in this field requires significant professional level learning and intellectual abilities, yet the profession as a whole is effectively treated as "skilled labor". If you are serious about SE as a career, then within in 10 years, do one of the following:

1 - Start your own sofware development company. Do the startup thing.

2 - Start your own software consultancy business.

3 - Make an informed choice of a niche and become an expert.

Outside of the above, at most you can look forward to (in general) is either transitioning to management, or a continuation of what your are describing: a glorified blue-collar career. (White collar professionals with experience seeking employment do -not- suffer the same indignities that is the common experience of skilled workers in this field.)

Management is fine if that turns you on, but you will definitely not scratch your geek itches as a manager, so the question is: is wasting your intellectual abilities as a young person worth a "payoff" of ending up in a management role? I mean, if you're going to end up in middle management, wouldn't it be smarter to just cut the chase, get that MBA, and start your management career properly. Or is it perhaps worth a fun first 10 years of "free soda", less structured work environment, and the adolescent culture of software development? Your 30+ year old self will likely tell your younger self "this was a mistake". Seriously consider the fact that this is a field where inexperienced workers routinely dismiss experienced practitioners. The technical domain ignorance pervasive in the field is rather significant and one of the more surprising aspects of this industry. Knowledge and experience can actually be a liability in this industry and high level fakers abound.

There are many other possibilities, of course, but these type of opportunities (such as making a pivot to writing books on software development, etc.) are more about the individual concerned -- their drive, ambition, work ethics, social skill, etc. - than about software engineering in particular. But so are the 3 options listed above.

TLDR:

For certain individuals, career choice is incidental. They possess the necessary skills, ambition, drive, discipline to succeed regardless of the industry. For the overwhelming others, a career choice is a strategic choice. Software development as a career is a poor choice for the latter. If you are intellectually capable, but not a go-getter, pick another career that requires brain power.

You're saying that riding the train of being a very solid generalist doesn't work past a certain age?

It may not be exciting, but don't you think there are lots of 45-55 year old Java backend engineers that get paid pretty decently? (140k-200k)

Great post.

Just that you're asking this question strongly suggests you should not.

I worry about the horde of people who never ask. How do we cull their numbers?

in the same boat

I don't know if you should quit or not, but I do think you're mostly just psyching yourself out here. I can't see your actual interviews, but I'm just guessing a little based on your example.

On the 3 second tweet time question, I think you're just getting anxious at the thought that they might expect you to replicate thousands of engineer-months of work in a 20-min interview session. Nobody sane expects you to do that. If I was asking that question, I know nobody's gonna build Twitter in 20 minutes - what I want to know is how you think about and attack the problem.

I've never worked with anything Twitter-scale, but I might attack it like: How many followers are we talking about here? Okay, let's just start with a simple, dumb system and build it up. Simplest thing I can think of is a basic CRUD application backed by a Relational DB. SQL joins to get the tweets from your followers. You'd have to make a new request and new query manually to load new tweets. You could auto-poll, but that'll get real hard on the DB with even a modest number of users trying to refresh semi-rapidly. Maybe we can have a system where a single DB is still the source of truth, but there's a parallel system to load new tweets faster for those watching. Could maybe use websockets or long-polling for web clients. Let's build some separate servers for that websockets traffic. I'm pretty sure that as long as there isn't that much overall traffic to any one watcher, we can handle a good number of connections on one server, so we don't need a ton of them. Maybe creating a new tweet still adds it to the DB normally, but also triggers a background job, so the tweeter still sees fast response time. So we've got a background job going, and a bunch of realtime connection servers that are maintaining connections for specific users. We've gotta connect them somehow, send a message from the background worker to the realtime servers to send the tweet to connected clients that are following that user. Now let's think about how that would work...

Basically, show that you can examine the problem and come up with a few possible solutions and think about the trade-offs of them. I want to hire the engineer who can do that for any problem set in front of them. It's much harder to justify hiring the one who just has a blank stare for anything they don't have experience with already.

And for the record, that question isn't inherently bad IMO. But it would be a bad interviewer to just ask it and stare at the poor nervous candidate who has no idea what you want. A better interviewer would make it more clear what they're looking for, and help them down the first few steps of what I just went through above until they get the idea. Ex: So let's say Twitter 1.0 is a basic CRUD DB app. What might the DB schema look like? What's going to happen in terms of DB queries when we try to render a timeline for a user? What's going to happen if they all start hitting refresh every 10 seconds? If we want users to be able to see tweets faster than that, what might we do?

The goal behind these questions is not for you to find the right answer, unless the interview is about a specific topic where you should know the right answer.

The goal is to have information on your thought process, how you solve problems, and how you communicate.

- Assumptions you have

- Your ability to clarify ambiguities

- Your interviewing skills: with coworkers, clients, reports, managers, applicants, you spend a lot of time interviewing people to extract meaning, facts, context, information.

- How you frame a problem

- How you behave in unfamiliar waters: you will build things you can't find the answer to on Stack Overflow or in a blog post.

- Your awareness of tradeoffs: do you make decisions being aware of the tradeoffs involved, do you have magical thinking, or cult of the tool/framework.

>"How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale.

Would you not like to tackle that problem and work on systems that handle that kind of scale? One argument against this is "who cares, they're not twitter so why are they talking about hypotheticals. They should interview me on what I'm going to do on the job". This is very valid, but there also is a counterpoint to this: this might stem from the desire of the interviewer not to be biased. If they quizz you on a problem they have been working on for the last year, they'll probably get frustrated by the fact you can't find the right answer to the questions. Or worse, they'd be impressed if you happen to find the exact solution they have converged to. You'll appear to be much much "smarter" than you are, and the expectations will be set high since you found the answer in a few minutes rather than a year. This is dangerous.

>I'm sorry if this is incoherent.

It's very coherent and understandable.

>I haven't even applied to jobs for weeks now, because in interviews I feel like I'm getting humiliated and I want to apologize and end them half-way.

We need to reframe this. If you put yourself in the shoes of whoever is interviewing you, they are trying to vet you because they care about their team, and they want to hire someone who will raise the bar of the team. From another perspective: if you were already in the team, wouldn't you want whoever is in charge of hiring to be an effective gatekeeper? Would you want a person who is impressed by an applicant spitting a few buzzwords and frameworks? That applicant then lands the job, gets assigned to your team, and becomes the reason you apprehend going to work? They may have been burned by past "bad" hires and have decided to tighten it up. People holding a team back, undermining decisions, commiserating while never explicitly voicing their concerns. This is a problem that must be prevented from happening, or dealt with swiftly if it happens.

Yes, they're not FAANGs but it's precisely that! They may be a small team in which you as a person are a large percentage of the workforce. If you are one in a team of 10, you represent 10% of the workforce. If you were at a FAANG, you'd represent a much smaller addition.

Furthermore, if it is a small team, it will hopefully grow, and you will take on more responsibilities, and you'll eventually hire people. They need to be very, very, disciplined about hiring people who will shape the future of the company.

It can be frustrating if you look at it from your perspective of someone who has a lot to offer and feeling humiliated, but that feeling is a very mutable reality once you factor in these different perspectives. Maybe even enjoy the interview, enjoy interacting with smart people, ask questions to learn new things, discover things you didn't know were a thing, and be glad for that team they have someone shielding them. You can ask them for feedback.

It is a fit that didn't happen between two pieces at a specific time and place. An impedance mismatch. If either of these changes, the fit could happen.

>I'm sure the problem is me, since other people are doing fine.

If one approaches it from a frame of rejection or humiliation, it can be extremely hard. This is like intimate relations. There are more productive and effective ways to approach these in terms of learning what you can from that interaction, wishing someone the best and to find someone who does it for them, fixing what is to be fixed, maintaining what is not to be "fixed", being courteous.

>* I need to draw a line and figure out some good indicators of when I should quit this industry, and anyone here who could give a few pointers on that, I would be thankful. I don't want to throw more time and resources into an industry where baseline expectations are something I can't/won't reach.*

OK, are you quitting the industry because you have a bad experience with the hiring process? If so, this can be fixed, and then if you want to quit, you can quit while in a good place.

Now, if even after that you still want to quit and you can't stand the "industry" anymore, you could amortize the experience you have and use it in your real passion. What's your real passion if you don't mind me asking?

A bunch of people will tell you you just have impostor syndrome and 10x programmers are a myth and don’t believe everything you read about Silicon Valley and blah blah blah.

I dunno guys, maybe some of these people really should try to do something else! It is no shameful thing to leave the field of software development. We’ve had a huge influx of people and interest to our field in recent years, is it really so crazy to think that some of those people and some of that interest is not well-placed?

For what's its worth this was my passion and I'd leave the field if I could, for a number of reasons. Just realize though that if you leave theres a high chance of a pay cut.

There must be something else thats blocking your way. I did a round of interviews with top notch distributed systems teams that deal with the kind of scale youre describing. All FANGs or similar ranked companies, with 400-600k / year offers. I have no experience in backend programming of distributed systems and every interview I started with "I don't know much about DS: Ive only read a couple books, but here's how I'd solve this problem". Got offers from every company, some chased me for a few months after. Perhaps the trick here was my credentials (in an unrelated field), ability to bullshit my way thru and the skill to be likeable, which is really finding something in common with interviewers.

There is a lot to be said for the entirety of your question, but i'll try to focus on this since it stood out to me (plus the interest of time):

>System design rounds require me to solve problems that the top engineers in the world spend months on. One question I was asked was "How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale. In fact most of my work experience is kinda vanilla and boring, so I generally have nothing much to draw on or much to talk about.

System Design rounds are going to be difficult. The idea isn't to give a correct, complete answer. It's to see how well you can reason about a hazy problem based on your current knowledge. Bram Cohen who wrote BitTorrent has this to say:

"My suggestion for learning software architecture is to practice. Obviously you can't practice it by doing hundreds of projects, because each one of them takes too long, but you can easily design a hundred architectures for problems which only exist on paper, and where you strive to just get the solution to work on paper. Start by modifying the requirements of a problem you're working on. What if the amount of bandwidth or CPU was a hundredth what it currently is? What if it were a thousand times? A million? What if you had a thousand times as much data? A million? A billion? What if the users were untrusted and you had to either prevent them from damaging the system or have a means of fixing things when they did? It doesn't matter if these scenarios are totally unrealistic, what matters is that they're different and that when you try to find architectures for handling them you take the inputs just as seriously as if you were about to start writing a system with those requirements for work. Try to find as many different approaches as you can, and come up with scenarios in which the stranger ones would be better." [1]

Secondly, "vanilla and boring" is a moving target. That question up there would for the most part be vanilla to senior backend developers, a little more than trivia and a waste of time.

You cannot say "I never handled that kinda scale". This is the wrong mindset to apply to any sort of problem solving. Start with what you know and get to the point where you have a list of knowns and unknowns. And for heavens sake, you have a massively powerful search engine. "Writing scalable systems" should be a query in your head right after you think "I don't know how to scale".

Talk about your vanilla and boring work. If you think it would make you feel better, focus on how it impacted the customers or whoever. It's perfectly fine to work on non-world changing things.

Lastly:

>I'll preface this by saying this field has never been my passion, just a profession I like.

Probably a healthy mindset, considering the amount of people in this field who chose it as a means of escapism from the hardships of every day life and convinced themselves in the process of coping with trauma that it's a passion. Passion isn't a requirement for good work, but some time spent learning the tech and being analytical in thinking about problems goes a long way.

And to answer your Q, ideally you should only quit if you don't like the day to day work and related reasons. Interviews don't represent this at all. It's mostly idiots on an ego trip.

[1] https://bramcohen.livejournal.com/4563.html

Reply on news.ycombinator.com