Everyone should read support emails | Hacker News Reader

Everyone should read support emails

medium.com

504 pointsstuffhq209 comments

This is why we regularly take developers to customers. It is eye-opening to see on-site that your mobile app with tiny but well-designed buttons doesn't work for registering container positions when the user is in a shaky 90 metric tons weighing machine, handling 30 metric ton containers.

We write software for container terminal and other logistical actors, and seeing the software being used by real users is so incredibly important when designing new screens and workflows, it's baffling to me that developers aren't taken on-site more often.

I think that's valuable, but depends on the organization. At a company I worked at engineering used to go on customer visits, but the "customers" were the executives and managers who make buying decisions and put forth requirements. However, these were not the folks who actually used the product. Their opinion was important (gotta sell it) but it was NOTHING like what we saw in support.

The the few times end users were there, they clearly did not give their honest opinion in front of their bosses. And really even folks giving feedback who use a product are poor at doing son.

But support is where the rubber hits the road and folks actually encounter real issues that they can't solve on their own and create real pain points that will come up.

In my example there were still a lot of issues we'd take back to engineering and they'd say something that amounted to "but they said they don't use it that way" and it was a real chore to get engineering to understand the difference between what an executive asks for / some of the requirements they were given, and what the real user does / needs / asks for. Understandably engineering resources were sometimes irked by this, and support often took the hit politically because of it. It was one of the reasons I got out of support despite getting along with the engineering teams really well.

At the last company I worked at they brought glad handers to take care of the execs while I the "low level tech" went around to people's cubes and spent 2-3 days talking to them, understanding their requirements (usually with NO meetings, just 1-1 after talking with their managers and leads.)

This got me the best feedback I have ever gotten, and I regularly implemented fixes and changes that made the C-levels rave about the product (because their entire staff was so surprised at someone giving a shit.)

I was quite impressed by my head of product's subterfuge, she was incredibly talented.

At one previous employer, I got into some heat for being that "low level tech" that went directly to users' cubes and talking to them directly instead of waiting for stuff to filter up and then back down chains of command. The users were really excited to see some features they had needed for years finally implemented and all it cost them was buying a bored college intern lunch a couple times. The managers were so unhappy with me though for breaking protocol.

That job taught me a lot about what to avoid in a corporate culture, because of how dysfunctional that was that the more actual work I did the worse my reviews got and the more my managers seemed to hate/resent me being on the project. It was a situation I hope never to repeat in my career.

Yeah, going directly to the end user is almost certainly best. The amount of times you find people spend 2 hours of their days on what would be literally a 5 minute fix (my favourite being the select with a thousand options that wasn’t alphabetically sorted) is mind blowing.

And all that just to preserve some egos. Though to be fair, in some cases users do request crazy stuff, you just shouldn’t listen to them and look at what they actually do.

I love finding those high impact, straightforward changes! I support dozens of spreadsheets at work and am the person people call when one breaks.

* Conditional formatting cut a 2 hour weekly job down to under 5 minutes.

* The intern, who was hacking away and learning VBA, spent around 20 hours and was able to automate 4 hours per day of manually retyping info between Oracle and a couple spreadsheets. Would have been about 2 hours if he knew VBA going in, but he learned a ton.

* 2 hours standardizing a spreadsheet format and making a custom macro to shift values by x rows and y columns turned a monthly 2 hour, tedious update task into 2 minutes.

Every company needs one person who can make simple changes to inefficient business processes and save a ton of time.

I feel like there are a couple of obvious problems that exist:

- Knowing that such a person exists - Knowing that the fix is simple - Knowing that they will have time to look at a thing quickly

I'm a person who will answer any question on any topic if asked, and help anyone with anything if I have the ability to do so. That's because I enjoy doing stuff in general and my long projects aren't meaningfully going to suffer. Counterpoint, the only people pestering me on this are friends / close colleagues and it wouldn't scale.

I reckon that the net benefit of being willing to do that to the company is positive, but honestly I'd do it anyway because sometimes 2 hours hacking on a powershell script is a welcome distraction!

Similar to the "users request crazy stuff", managers of users request crazy stuff based on what they think their users need. Stuff that goes up and down two very different reporting chains (up the users' then back down yours) is almost always going to be some weird telephone game crazily divorced from reality.

"How is the user actually using this software?" is sometimes a weirdly hard question to answer correctly in reporting chains.

Then when those reporting chains get siloed due to budgets and in-fighting and over-complicated "change management procedures" (due to often over-zealous risk aversion, maybe to avoid losing budgets), it's far too easy to find yourself not developing for the actual users but for some ideal "manager-user" that doesn't actually use the app day-to-day (or maybe even at all) and instead believes the map is the territory. (Those that think the issue tracker about the app is the status of the usage of the app, or worse that PowerPoint screenshot demos of the app are all that matters and tells them everything about how the app functions.)

I remember back in my sys-admin days, when a vendor would come in, we would 'kidnap' the devs/engineers and keep them in our office, while he C-levels/Directors spent time 'discussing business'.

At the same time we were giving the devs/engineers a taste of what the requirements should REALLY be, or what the problems REALLY were. That gave them a far better idea of what our pain-points were.

It happened that devs/engineers, while in the presence of theirs and the customers' C-levels/Directors, they only smiled, nodded and agreed, which is kinda dangerous as they were forced to agree on insane, intangible, and unfeasible requirements.

Yeah, absolutely, we see that all the time as well. We have the benefit that usually we only have to do sales at the executive level, and for everything else we deal with the operations people, which means that if we go for an on-site visit we usually won't meet higher management than operational team leads.

We learned that what executives say and what operation does is not the same thing, which is why we sell our software we say: "We'll fix your problems, but be warned: we won't do it your way", and as long as that expectation is maintained we have very happy customers.

I've worked on several projects where the actual needs would get distorted between the customers and the developers in a perverse game of telephone. Each producer or manager feeling the need to reinterpret as they saw fit.

My most recent job the head of the company had a great comment. One of the first things he told me "Our end users are not smart, no matter what anyone tells you. They can barely use a computer. Remember that."

He wasn't wrong ;)

I HATE that attitude. It may occasionally be right (depends on your product/intended use) but it was the catch all ‘do as I say’ excuse one of the executives used at a previous job.

Have a suggestion on how something could be done better? Your way is too complicated and our users are dumb as posts. So we can’t do it. It’s not worth even thinking about.

Of course if THEY want to do the complicated hard to understand thing, it will be fine.

Just one of MANY examples.

Why do the job if you have actual contempt for your users?

>Why do the job if you have actual contempt for your users?

It's not contempt, it's understanding who is at the end of the keyboard.

In this case our customers are fairly capable, but the software is really for their end customers who are in a niche industry that is AMAZINGLY behind the times in terms of technology, and the people they hire who actually end up doing a lot of interacting with the software are honestly barely high school grads and are surprisingly incapable / startled by anything out of a very specific routine.

> Why do the job if you have actual contempt for your users?

To... To get paid, isn't that obvious? Sometimes you have to do what you have to do.

I mean, yes, in retail or the service industry, but it should be possible to find a job in IT where you can at least tolerate the end users (or just never see them altogether).

Seconding MBCook here; I believe this advice is nonsense.

Truth is, most people are smart enough. If this is a product for internal use, then they might be also extremely motivated, as it happens when handling some piece of software becomes a key element of keeping a job.

Taking the approach of "our users are dumb" makes sense only when you're not thinking about providing value for those users, and instead you're worrying your product could make the wrong first impression (and probably not on users, but the managers), and thus not get bought. It's a valid priority for business, but it would serve everyone better if people were up-front about it.

>Truth is, most people are smart enough

I agree, if we assume they're going to try. It's not a perfect world and IMO most people really aren't trying / putting in the effort on the things we think they should. It's not fair, but if they're not going to try, they're still the person behind the keyboard.

> It's not a perfect world and IMO most people really aren't trying / putting in the effort on the things we think they should

That's a managerial problem not a technical one, so any technical solution like dumbing down the user interface will almost certainly fail.

If your user interface is so simple that understanding it doesn’t take any effort at all, that’s a pretty great thing.

I imagine the hypothetical lazy user thinking ‘I cannot figure out a way to convince my boss that I do not understand this UI...’

> If your user interface is so simple that understanding it doesn’t take any effort at all, that’s a pretty great thing.

Nah, that's an indicator the software is most likely not very useful. Like in code, in user interfaces too there's both essential and accidental complexity. Plenty of the latter to improve on, but reducing essential complexity === reducing features === reducing utility of the software.

> I imagine the hypothetical lazy user thinking ‘I cannot figure out a way to convince my boss that I do not understand this UI...’

I imagine said lazy user asking the boss to buy a training course. If that software is so important for the company, a training course is both useful and cheaper than hiring someone who maybe can learn the software themselves in place of said lazy user. Training courses tend to provide ample opportunity to slack off too, so that's a double win for the lazy user :).

(That said, from personal observations of my wife's and mine, people seem to be split on that somewhat halfway - one half will jump at the opportunity to learn something new and progress at their workplace or in their career, the other half will roll their eyes and ask "why should I bother"? That latter group responds to "because it's that or your job" argument, though.

Is this a question of redesigning your visits to incorporate your real stakeholders? There are many companies with management and workers at polar opposites like this. I've seen this tackled by proposing that you offer some on-site support sessions and bring support and engineering along for the event.

Focus on making management at the company feel like they are doing something for their deployment and get more forthright feedback because they can't be in as much control. Establish some direct follow ups, if needed, to primarily tackle the support issues, but ensure the folks who were on-site are part of that conversation. Present a summary of the event back, including action items which you take from the event alongside their pet problems/requests.

I work at a company where most of our contracts are for larger enterprise businesses and, as a result, we have a training component to our launches.

As much as possible we try to bring a person from Product (dev + design + PM) to be involved in the launch project. The benefit here is that the product team gets direct access to the users. And they get "early access" to the challenges our customers face: in the early days of using a new product, the issues shine very bright compared to later when the users have found a suitable workaround. This has and will certainly continue to be essential to building empathy between the people who build the product and those who use it.

At a company I worked at engineering used to go on customer visits, but the "customers" were the executives and managers who make buying decisions and put forth requirements.

Developers should meet the customer organisation to talk to users. If you're not talking to users you're not getting the most useful feedback.

Where I work we often supply software to other software companies, and our UX team goes and visits their customers to get insights about how our products are used.

Agreed.

I've seen many a time where the customer interface contacts believed whatever bullshit that they were peddling internally, and the information we had was essentially completely wrong.

Vendors walk a fine line with these issues. You have to keep the bullshitters happy as they control the money, but you need to get shit done to keep the golden goose producing when sponsors get promoted/fired/held accountable.

> The the few times end users were there, they clearly did not give their honest opinion in front of their bosses. And really even folks giving feedback who use a product are poor at doing son.

Actually I worked over 2 years at a company developing in-house tools. For almost all tools/projects I talked directly to the end-users. 2 observations:

1) users become greedy when it comes to features (especially those users in supervisor/manager positions)

2) there's a tipping point with amount of desired features when users actually start preferring to use the newly developed software instead of a previous solutions

I think when you say users become "greedy" it's that they don't truly understand the depth of the problem they want to solve. It's not that they are in a true definition sense of "greedy", but ignorant about the cost vs benefit analysis. If if was explained to them it would take 20,000 man hours to solve their issue, and then it would put them out of a job, most of them would clam up.

I worked at a first-response call-center company where after each shift, they would go through the records that they processed that shift and manually count the number of high-priority incidents and various other metrics. They did this after each shift, taking about 15 minutes. For the last three years.

After my field visit I took it to the PO and we implemented these metrics in one sprint. With a single click they could print all import metrics about the last shift.

Sometimes when you automate this stuff away, you lose the value of manually reviewing the data. Just seeing a metric isn't the same as reading through the logs and seeing what actually happened.

Once I got two developers to use their software on the live system. As a result, they fixed a few bugs which didn't appear on the dev systems as those were being run with much smaller data sets.

When normal users had to wait 20 seconds for a page, they didn't like it, but it was completely possible that the system just was that slow. But when the developers had to wait 20 seconds they used the time to look up what took so long and found minor bugs which caused the slowdown and fixed them right away.

As a result, the developers asked to be given more opportunities to work with the live system.

shaky 90 metric tons weighing machine, handling 30 metric ton containers

Also gives an immediate answer to all these "should we fail gracefully or crash loudly?" arguments.

Yeah, that sounds like the most fun place to crash loudly ever.

Just a small failsafe to ensure nobody is nearby.

It took me quite a while to unbind the commonly used abstraction of containers with the real objects :P

I think one of the ways working at a startup benefited me the most was having to cover the support lines when our support guy was out. I got to interact directly with many customers and learn how they _really_ used the product, what was confusing, what wasn't, etc.

This is what UX Researchers are for.

Agreed, as a developer, I'm often asking to see the production environment for the users I'm building for. Even if it's a new application it's usually replacing something existing and understanding that is huge.

sounds like you make apps to help logistical actors to drop their phones.

Hi, Support Engineering Manager here, who has forgone Dev for a career in Support.

In a small company (<10), yes, do this, everyone will level up, and you'll get extreme customer focus.

As companies grow 2 things that I think are really more valuable:

- Support Engineers, These are folks that have commit access and can write fixes, who are focused directly on customers. Don't distract product from new features, Support Engineers can fix low hanging bugs, (button states weird/form is weird, error messages wrong, etc, etc), and then funnel harder or product-changing bugs to product. For every 10 devs, I think you should have 1 Support Engineer.

- LISTENING to your support team. I always see companies and people doing the "uhm, oh yeah, we should totally all jump in and do support to better understand it" but really, it's like, trust and LISTEN to your support team, ask them what the most important thing to them is, and actually prioritize it.

It was extremely demoralizing when "holier than thou" devs came to do support and kicked and screamed for their week rotation. Sure, thats indicative of bad culture, but I promise you there are at least a few devs on your team who are this way right now. Let them be devs, instill a culture of trust and teamwork in different areas, and your company can scale and move fast.

Some caveats and related advice, having been on the product side and two different customer-facing sides:

Integrate support engineers at least part-time into the product engineering teams. Support engineers who don't know what product is doing can duplicate or complicate product work.

Invest in paying and training the support engineers. Don't pay them well and they'll either transfer to product engineering once they get competent or jump ship to someone else. Poorly trained support engineers who "have commit access and can write fixes" will invariably introduce new bugs that will block product work.

"Don't distract product from new features" is a great goal, but recognize that it's a goal, not a law. There will be fires that product engineering will have to walk into and put out alongside support engineering, especially if your product is more than an small-team app, exponentially if the problem comes about due to systemic design problems on the product side.

It will be tempting to outsource support engineering to save money for in-housing product. Don't. If anything, outsource low-hanging product work before any support work. In-region support engineers make everything about firefighting smoother and customers in every geo will appreciate it.

Incorporate support pain into your dev process. That's not bug metrics—measure the _pain_ your product causes customers. Feature requests are pain. Insufficient docs are pain. Licensing issues are pain. You can insulate product from customers, but don't insulate them from the pain the product causes customers. Use it to help guide the roadmap and you'll find low-hanging fruit that product never considers because, chances are, they aren't interacting with painful parts of the product (otherwise they'd already be addressing it).

I love this extra perspective to my comment!

The only thing I'd add:

> who "have commit access and can write fixes" will invariably introduce new bugs that will block product work.

All fixes by Support Engineers should go through the normal CI pipeline and Code Review process. This is designed to exactly stave off this kind of effect. But yes, I still agree with your points! I love HN <3

This is excellent advice. I will add to it:

If in a small company you are spending a certain percentage of time/money/effort on support (probably 10-15%), there is no reason for that percentage to be different in a large company.

Go through that again with me, please. You've been spending 10-15% of your time listening to customers and providing good service. When you grow, why would you want to provide worse service?

And yet, this seems to be the established thinking (oh, we can't possibly provide direct E-mail support at this scale, let's hide our contact information and make customers jump through hoops just to contact us, and let's keep our support team small so that we can make more money).

Your customers will report lots of issues, and yes, some will be bogus. But most will point to real problems, either with your software, your hardware, your usability, your processes/procedures, or the limitations of any of the above. If people write to you, it means they took efforts to contact you: do not just discard their reports as "bogus".

Another way to look at this is that every bug report is worth its weight in gold, because for every customer who reports a bug, there are 10 others who do not, and just stop using your app/service/whatever.

Also, if you ever extend/automate/scale your support, make really really sure that there is ALWAYS a way for at least some user reports to make their way to developers, designers and architects. Otherwise many valid bug reports will get stonewalled and lost. See the recent Apple FaceTime ("facepalm") fiasco for a good example of this.

An organization with 5 customers can babysit each one to make sure they’re happy in 10-15% of their time. An organization with 1000 customers couldn’t do that even with 100% of their time. The only way to maintain a fixed allocation of customer support funding is to make it less high-touch as you grow.

I respectfully disagree. Note that 10-15% of resources of a 5-customer organization is very different from 10-15% of resources of a 1000-customer organization, assuming your customers actually pay you money (generally recommended).

This means that even assuming you get no benefits of scale (and you should be, your software/product should improve as you learn and support load should fall), there is no reason why you can't provide the same level of support as your organization grows.

The problem is that people assume for some reason that a larger organization can't dedicate 10-15% of their resources to support. This is why whe get crappy support or no way to contact the organization at all.

This is all about scaling. You want to do 10x the customers with 2x the employees, so that the business finally starts to make money. The service level that took 10-15% of your resources back then would take the majority of your resources now; the same % of your resources now is much less hours per customer.

There shouldn't be a linear increase in support requests with a larger customer base.

As your product scales to more customers, the percentage of support requests generated should be going down as you improve your product and support documentation based on existing customer support requests.

This ("shouldn't be a linear increase in support requests") is true some of the time, but not all of the time in all lines of business. Make a thing foolproof, and the world will go on generating greater fools, you will see a greater variety in weird requests or bizarre expectations, etc.

The number of requests may scale linearly, but the number of actual issues won’t.

But that's my point, isn't it? If you want to do 10x the customers with 2x the employees, you will be cutting into your support resources, and your customers will suffer.

This is exactly why we (as customers) all have to suffer bad or no support. And sadly, we put up with it, mostly.

There are serious overhead costs to running a company 10 times as large. If you also have to hire 10x the employees, you'll end up needing to charge more than most customers are willing to pay. (Customers who are willing to pay out the nose generally do get good customer service, even from organizations that are normally bad at it, up to and including teams of engineers deployed to the customer's office to help out.)

If you have 200 times as much revenue, why don't you have 200 times as many employee-hours?

I think most companies want economics of scale, but even if they don’t:

Finding 10 good engineers that care about your customers is difficult but doable. Finding 2000 is much much more difficult.

Trusting 10 engineers to communicate with customers without promising things they shouldn’t, mentioning upcoming features that hurt current sales, have generally good communication skills, is again difficult but doable. Asking that of 2000 people....

That highly depends on how much money you're getting from each customer

as someone who did support for years and now does engineering I think this is so correct. Support is listening to customers, not only that they are listening to a LOT.

I think it's valuable for engineers to talk to customers but they and product should also actually listen to support - they are probably telling you a large majority of the pain points your customers are experiencing.

also - and another major thing - make sure you have a way for support folks to become engineers, product managers, designers, etc... in my (clearly biased) opinion this often results in an engineering / product department with a deep understanding of your customers needs in its bones

Can you share anything about what you've seen work as far as collecting and broadcasting experience from support to other teams?

1. Invite support to product/eng meetings where you are prioritizing. Add them to the agenda by default so it's natural to expect them to speak. This is huge and helpful.

2. Consider baking in "Support gets a MINIMUM of X things" on priority. Tough conversations will be had, but support/product/engineering will learn how to get what each other need. This one falls apart and "when the going gets tough". Support gets the short end of the stick usually. (That's why I advocate for a Support Engineer, so those things don't even need to leave support, they just get fixed and when support speaks, the teams know it's big.)

3. Make sure whomever is heading support knows what other stakeholders need to see, so they can deliver. Some would call this "data driven" but in some orgs, "data" is just 3 angry users.

4. We currently use this term "Stable Counterparts" and that's vital, figure out who should regularly be talking to support and about what, and once that relationship forms, things get smoother. Until then, support will be yelling into a void.

Thanks for this rundown! I'm especially attuned to your (3) and (4); we've run into a lot of CS teams creating "liaison" roles with sales/product/engineering, and even in some cases with finance teams re: categorizing causes for churn and such.

I once joined our support team to talk with one of our clients (I'd written a new bit of functionality and they wanted me there in case they would ask technical questions). It was one of the most eye-opening experiences ever. Our client talk about their clients abusing our software in ways that we as a development team had never even thought about.

EDIT:

On the other hand, I don't think it should be done to create a sense of urgency for developers. They are not going to work better if they also have to worry about each and every emotion that's being displayed in support emails like the example in the article. That's just creating a sense of responsibility in your company by appealing to emotions and guilt.

We do retail / light industrial automation stuff.

Every so often (rarely but say once a year or so), we bring some members of dev team along on an install. Its always very very eye-opening for developers to see what 'problems' look like in the field.

Further, something devs miss about installs is the lack of infrastructure you have to deal with (on a new build-out for example). So saying 'Just download latest version X and install on each of the clients' will take 5x longer than it does in comfy office environment since it means: (a) Go to hotel or tether phone and download 1GB file over 3G, (b) Put on USB stick, (c) go to each client, disable all USB restrictions and install new software, re-enable all restrictions (d) do all this while power is being turned off / on because electricians are still working.

Could you elaborate on the kinds of abuse you saw?

> Could you elaborate on the kinds of abuse you saw?

In a very broad sense, sure. I don't want this account to be traceable to where I work now, so some details will be changed or omitted while keeping the kind of abuse the same.

We make a website where our clients can make timeslots available to the general public. The length of a timeslot is dependent on how many people the person booking slot intends to bring, as well as some other factors. Whether a person can book at a certain time therefore (partially) depends on how many people they tell our clients they'll bring.

We also made it so that the booker can change how many people will be coming along (for example because of sickness), or, if they are delayed, by how much, so our clients can more fully use all of their available resources.

However, how bookers were often using it, is by saying they'd come alone (if you have a very short timeslot, there's a better chance it's still available), and then change the number of people to how many were actually coming along. Or they'd just book a slot at the start of the day, and then delay it until they were placed on the time they actually intended to come. In both of these cases, they were abusing features which were genuinely needed for the product to work to their own advantage. It was slightly naive of us as developers not to take this sort of behaviour into account.

Fully agreed. I'll go a bit further: Everyone, especially engineers and PMs, should once in a while actually reply to support emails. As CTO of my previous company, I still found time for large amounts of customer support, and so did one of my cofounders.

It's definitely important IMO for key people to be in contact with the userbase. Get a better look at what people are asking for, feel more directly responsible for both CS failures and successes, etc. And it overall increases the quality of support (rather than put people in CS who have no idea what's going on aside from the script they follow).

Key quote from the article:

> My experience with everyone reading support emails is, that everyone feel an increased responsibility and a sense of urgency to eliminate whatever emails hits your support inbox.

I am always sad when I see people who think doing customer support is beneath them. It's a red flag, IMO.

I'd like to add, shadow your customers a year after deploying your product. Engineers often blame the user, but users often end up using peculiar workflows to get around ui issues in your app, specific bugs, or other issues. Also if the user doesn't understand part of your product a year later, you probably should be improving that experience Either within the app or externally by improving documentation and providing training.

After some burn in time, is it really the user who is doing something dumb?

While reading the article, I got a feeling that reading support emails is only one step away from answering support emails - and here you are, suggesting just that.

I've seen further steps down this road - if there are engineers already answering support emails, then why do we need support people at all. After all, engineers have better knowledge of technical details and can make changes themselves. No need for intermediaries.

I understand the reasons for having engineers read (or maybe even sometimes answer) support emails, but still I'd be extremely wary of again working for a company that walks this path.

The reason lower-tier support personnel are employed is because someone reasoned that the engineers' time is too valuable. If 90 % of incoming support requests will be escalated to developers, it makes little sense to have extra people whose main task is to create SAP tickets or press "forward" in Outlook.

Another extra perk of making devs handle the support is that they also have the chance to fix the underlying problem. Remember that for an engineer, fixing broken software is always less of a burden than correspondence with customers.

For our on-prem enterprise software, we have two levels before a ticket reaches the dev team. Level one handles RTFM-type answers, and asks for log files (and then provide RTFM answers or engage a dev). Level two handles demos, implementation and architecture guidance, and reproducing more complex bugs. Dev handles the most complex issues that trickle down, and backup when staffing is short (mainly only around holiday weekends).

> Remember that for an engineer, fixing broken software is always less of a burden than correspondence with customers.

Don’t be so sure. Just because you’ve identified the issue or a good fix doesn’t mean the ticket won’t go in the backlog to die a slow death before you’re allowed to fix it.

Best way to make me improve software is to force me to use it.

> While reading the article, I got a feeling that reading support emails is only one step away from answering support emails - and here you are, suggesting just that.

I didn’t get that impression. The article mentioned reading these emails for 30m a week. Going from there to doing tech support yourself as a CTO is...

It seems to me that a CTO doing tech support either understand something I really don’t, or really fails to understand something I do. Superficially, it sounds like the sort of low-level digging around you get from engineers that got the CTO title but never got comfortable with the transition.

Obviously I wouldn't recommend people do full time support if it's not their role, and the larger a company gets the less practical it is especially in the executive roles.

But the attitude displayed in some of the replies is exactly what I'm talking about when I say some people think support is beneath them.

interesting ! Would you answer those support tickets as a CTO (I mean, with the title in the signature) or as a random support agent ?

question also works for PMs / devs / co-founders / ...

Never signed anything other than with a name.

Even if they do not reply, they should read every support email that comes in for the product/feature they worked/are working on. It is the best way to put themselves in the shoes of the user, and make better software.

I know someone will respond that they don't have time read all of the support emails, but my response would be that getting so many support emails is itself a data point.

At Basecamp, everyone in the company does a workday in support every month or two. It's a fascinating experience to take part in: https://github.com/basecamp/handbook/blob/master/our-rituals...

I worked on a software project for 10 years where we did this, with 5-25 developers in the rotation over time (we grew). It was unpopular with many of them, but I loved it, and found it incredibly valuable. Even the people who thought it was useful learned valuable context they didn't realize they learned (e.g., how helpful our own documentation was when they needed it).

I really like the idea of doing a day of support a month. When I did it every day of the week it drove me crazy and made me really cynical though.

This is one of my favorite things about Amazon's support system. Support has direct access to PMs, developers, customer facing solutions architects and all of these orgs have close connections to the customers as a result. While not everyone has access to the support cases, customer sentiment is conveyed as part of this process and features or bug fixes are quickly roadmapped as a result of customer pain.

The fact that there's someone at the other end of the support channel who actively cares and understands the problem and the customer's voice is being conveyed company wide creates a better experience for everyone.

I have reported and watched maybe half a dozen bugs get fixed on Amazon, starting in 2003 I think. Never even experienced a reply from any other company when trying to report problems over the years.

I think a handful of companies are getting more proactive with security reporting, but everyone still treats the quality of their services and front line support system as an afterthought.

The customers are testers, especially in "break early, break often" scenarios. I wouldn't call them testers to their face, but... operationally, they're part of the loop. Treat them as such. If you're not going to pay for much testing up front, at least treat the real testers (customers/endusers) as part of a process, not as part of a problem.

If, when I reported an issue, and it was determined not to be PEBKAC... loop me in on updates, or followup with questions. I'm happy to try to reproduce issues, or give more details, or whatnot. I'm a user/customer - I want the product/service, and I want it to be better.

Yeah, it's honestly disheartening, but it's great seeing Amazon realize the value that support can play. Even looking at the hiring requirements for some of the positions can give you some insight into why it works as well -- support teams which require it have strict development requirements, and consequently the support engineers 1) know the problem space and can infer what you're trying to do 2) know what it feels like to be blocked by something completely out of your control and feel like you're writing into the aether

It would be great for companies as a whole to start realizing the value that you can bring to your existing customers through this experience, and to recognize that this is another face of your company.

> This is one of my favorite things about Amazon's support system. Support has direct access to PMs, developers, customer facing solutions architects and all of these orgs have close connections to the customers as a result.

This only works if they know how to convey the relevant issues to the right developers properly.

I had some large files on my Amazon Drive (the consumer product) which could never be downloaded via the desktop app, web browser, odrive, or rclone. I had to endure multiple instances of support going through the flowchart, and some promises of conveying my issue to the engineering team only to be met with disappointment.

I never managed to recover those files in the end.

> This only works if they know how to convey the relevant issues to the right developers properly.

100% this. I don't know that it was like this across the company, but the way customer-facing issues were escalated to my group was pretty awful and almost always mishandled. Customer tickets landing in the same queues as host/service auto-generated tickets, bad handoffs and prioritization, devs with no support experience asking for way too much information from non-technical people just trying to use the service. I suppose the first-line support org is to blame in part. Though it wasn't one of the hotter products they offer, it was definitely old enough and nearly foundational to have warranted several more subject matter experts on the support side. Seeing them tout themselves as one of the most customer-centric companies out there was flabbergasting when it came to that org.

(Source, used to eat bananas)

I had to look up what your "used to eat bananas" reference meant: https://nypost.com/2017/05/23/amazon-gave-away-too-many-free...

Interesting story!

"We can't give devs access to support, because [ZenDesk, FreshDesk, etc] costs 9 gazillion dollars per user"

For that exact reason I've had a weekly "emails and chill" date with a supporter. Brought coffee for both of us. I flicked through incoming support emails for 30-40 minutes....

We did the same thing with a small team. Did wonders for our response time and for refining our support processes.

What's your support volume like right now? Just email, or also chat?

ZenDesk and other help-desk software have to be the highest reward-to-effort SaaS startups ever built (from a founders point of view).

Simple CRUDs with non-complex infrastructure... that cost a boatload per client.

It really depends. The per seat cost for ZenDesk is pretty low; if you have an efficient process, it can be a lot of effort for them. My work's support people are very efficient, so we were processing tons of tickets and bogging down their infra. We ended up building our own ticketing to get the performance we need.

On the topic of reading support email; I think it's important that developers regularly have tickets escalated to them, and regularly hear about the trending issues; but that doesn't mean they need to spend a day in the queue. Processing support email is a skill that needs training and practice to do well; it's not something an engineer can do once a month or a few times a year with good results. Of course, this may depend on the scale of your support queue and of your team; there's a big difference in a few emails per developer per day and thousands per day.

"My work's support people are very efficient, so we were processing tons of tickets and bogging down their infra"

That kind of says it all... There should be no reason support tickets should bog down their infrastructure. We're not talking about millions of real-time transactions here, even in the most popular consumer products. I should expect perfection because it's simply the most trivial product you can build, technology-wise.

At least FreshDesk has an "occasional agents" option (users who are paid per every day they logged in).

https://support.freshdesk.com/support/solutions/articles/227...

Cost is one issue. Another is that these tools are designed mainly for triaging conversation volume and aren't really designed for PMs, salesfolks, or engineers to extract what they need from historical conversations.

Companies that want to get every team engaged with customer feedback can: (1) Invest in their own data extraction and pipeline to push conversational info into cross-team dashboard. (2) Perform or outsource a bunch of research and report on "voice-of-customer" to the rest of the org. (3) Use active feedback reporting tools like ProductBoard or NomNom to organize (one kind of) customer feedback. (4) Try passive conversation analysis from new tools like frame.ai (that's me!) or scopeAI

Then it is a product development priority to pick something else, just as you wouldn't pick a bug tracker, version control system, or language runtime that wasn't affordable to let every developer use.

Exactly. That's why it's helpful to pull the data into a separate UI where everyone can get access to it.

Our product www.fwdeveryone.com is designed for this, if anyone is interested. We make it easy to share specific threads within private repositories for your business for these kinds of use cases.

For most of these products the api is relatively easy to use to pull conversational data.

Or use ScopeAI :) (Biased since I work there)

Every member of IT where I work (758 people) have licensed ServiceNow account.

What's volume licensing like at that scale?

Support engineer here. For anyone considering a career in support:

- It's harder than development. Green fields are easier.

- It's pretty much a thankless job.

- You are sort of the janitors of the IT world. Not much respect.

+ The money can be pretty good. Because of the above, management usually rewards good support engineers.

+ You are welcomed by those who don't want to do support.

+ You stay fresh, learning other people's ideas all the time. This contributes to longevity.

All things considered, it's been a good career move for me. YMMV.

> You stay fresh, learning other people's ideas all the time. This contributes to longevity.

I recently moved from a dev/support combo role to more of a support one, and this really stands out to me. I'm supporting a half dozen solutions currently, and each one consists of a pile of different flavours of the month. I'm learning a ton of new technology to support the solutions, much more so than if I'd stayed on any one dev team.

Very good list, although I'd disagree about pay. I'd add:

+ it's one of the easier routes to becoming a literal expert in how to deploy the software.

Pay for level 1 positions is indeed not great, but the money is pretty good in companies that value support enough to do it in-house and once you've learned enough to be senior/backline to a whole crew of juniors.

I do agree that the pay does not necessarily reflect the fact that the job is often harder than mere software development: you're basically debugging other people's software in realtime, often while angry people yell at you.

Since the article author appears to be the submitter: you should quit Medium. It's not helping your content when a full-page popup obscures it and entices your users away to a sign-up page, then keeps nagging them with popup nagbars on either end of the page.

If you need help migrating to another platform, please shoot me an email.

This extension helps Medium become readable again. https://makemediumreadable.com/

Not the OP, and I don’t disagree re: the nightmare Medium has (predictably) become. But I’d love to know what platform would you suggest?

I self-host my Jekyll-based blog, by compiling each push on builds.sr.ht and rsync'ing it onto a static web server. Basically roll-your-own GitHub pages. I also wrote a tool which converts your Medium posts into a Jekyll blog.

https://git.sr.ht/~sircmpwn/unmediumify

Have you considered building https://cdn.sr.ht? I do a similar process for some sites and use AWS CloudFront to essentially get globally deployed static sites.

Since this is a workflow you've encountered might be something that people would pay for. I definitely like a CloudFront alternative that wasn't as overpriced.

The day Bill Gates answered a support call

https://blogs.msdn.microsoft.com/oldnewthing/20091123-00/?p=...

I'm very skeptic that the customer would believe it was actually Bill Gates that took his call.

I could imagine it if it was a grandma or grandpa (or similar, you get the idea)... but then she wouldn't know who Bill Gates was anyways.

Bill could be all nice, calm, and helpful because he is not sitting there every day taking calls from end users having to actually work according to some metrics and deal with aggravation.

It was 1989 though.

I get a kick out of stories like that but I would wonder about this:

>Bill Gates is being taken on a guided tour of the product support department's new office building, and during his visit, he asks one of the people manning the phones, "Mind if I take this call?"

In particular "Mind if I take this call?".

Mind?

The entire idea of the head of a corporation (such as Microsoft in particular) asking permission in that way as if the employee who works for him would have some reason to object or be offended. As if he is butting in front of him in the line at a store or taking his last 10 minutes on a jetski.

Presumably he also took over the employee's desk, computer, chair, telephone headset, photos of wife and children, and so on.

"Mind if I..." would be a normal, polite thing to say in such a situation.

It's not the employee's desk. It is Microsoft's desk (and Bill is defacto Microsoft) He wasn't in the employee's house. He is using the desk that Microsoft provided for him. He is not 'another guy doing tech support asking to sit in and do a job'.

Unless it is a part of the company culture you signed on for, relationships between employees and employers are never black and white, "We bought this desk and phone, so we reserve the right to come over and just snag it out of your hands whenever we want"

Basic human decency always enters the equation

I'm going to be annoyed if a coworker swipes company gear I need to do my job effectively, or gets a headset I'm using all sweaty. And my desk certainly has personal effects which do not belong to the company - which could be anything from family pictures to keyboards to extra monitors.

Maybe there's no reason for the employee to object or get offended, but that also means there's no harm in asking permission since the answer will be yes. And there's harm in not asking permission, even though the answer will be yes. You have a couple of choices:

You can treat your employees like peers to get along with and respect. Even if it's more optical illusion than reality, this can make for happier - and thus more productive - employees.

You can treat your employees like minions to boss around. Not exactly the most motivating of environments for your employees. They could try and make their unappreciative boss a little happier... or they could browse facebook a bit more. And maybe steal a few pens to stick it to the man. Maybe not a big deal for a once-off interaction, but even that sets an example for your managers and their not so once-off interactions with their "minions"...

It's called being polite.

In a certain context it can also be considered patronizing.

Let's imagine someone is doing heavy labor and digging ditches. More appropriate to just say 'let me do that for you'. Rather than 'mind if I do that for you'.

Bill must know it's not a great job to take support calls constantly and deal with the aggravation. No doubt that there are people (like with any job) that like the job and are fine with it. But those are most likely not people that Bill associates with, is friends with, or respects. The contrast between the parties is where the 'patronizing' comes in.

Another example is a Physician in a hospital. If someone is cleaning bed pans and the Physicians says 'let me do that for you' it means one thing. If they say 'mind if I do that for you' it means another thing.

Or it's showing genuine respect for your employee's job, autonomy and skillset, and implicitly acknowledging that you probably aren't going to do as good a job as they would.

"Mind if I take this call?" could easily be met with "would you mind taking the one afterwards? I know who's dialling in and we're just wrapping up a long-running issue"

If a user is willing to message you, you can be sure many hundred other users are thinking/experiencing the same thing.

I remember support commenting that lots of users seemed concerned about security (messaging to ask if their files are kept after) and we were removing them, plus it was written in the privacy and terms page.

Finally we put in plain writing that we didn't keep any files, right on the landing page... 40% lift in conversions.

Support is a better signal than exit surveys.

That would require developers to think of users as people instead of cattle, and then they might have to accept that the decisions they made because "developer time is worth more than user time" have an actual real-world cost on actual real-world people.

In my experience, such time pressure comes from management, not the developers themselves. This is evidenced by the painstaking effort a dev will go to to make their ultimate experience on a side project.

In other words, users are often sidelined by new feature requests (for other users) rather than a selfish conservation of time by developers.

The priority game is as yet unsolved, unless you can point me to contrary evidence (which I would be eternally grateful for).

> In my experience, such time pressure comes from management

In my experience, open source developers are just as bad, if not worse, when it comes to their treatment of users.

Yes, but open source developers are in their vast majority unpaid volunteers gifting their labor to other users, so these users can either accept the gift as it is or not accept and move on. They are not entitled to other people's work and time for free.

>>> such dev behavior is driven by managers

>> open source people, who don’t answer to managers, act similarly. Therefore, it seems unlikely to be due to managerial influence.

> well open source is a gift of labor, people shouldn’t look a gift horse in the mouth

What you say may be true, but also irrelevant to the discussion being had.

Point is: even absent management pressure developers choose not to care about users.

> "developer time is worth more than user time"

To an extent this is true, in the sense that one must consider the opportunity cost. When working on feature A or fixing bug B, I cannot be working on feature X or fixing bug Y. Also, some features might require a disproportionate amount of developer time compared to the time the user saves.

Just the other day I dismissed a feature request from a user which would have saved that user an hour or so doing a particular task. The problem was that it would have taken me over a week to implement and test (it required complex changes), and the user only had to do this task once a month at most. So for now, I couldn't justify the development time vs user time benefit.

That being said, my primary motivation is to enable users, letting them be more efficient and doing things they otherwise couldn't. I love adding features which allows users to spend minutes rather than hours doing a task, or even entirely automating the process. But I must be mindful of the opportunity cost.

Take it one further: everyone should do support, not just read the emails. 1 day per month or so should do the job, it puts everybody in a good position to appreciate that if they don't do their work properly the support people end up taking the heat. So do not just read the emails, answer them, make it work for the end-user and spot the dysfunctional bits in your organization first hand.

> Take it one further: everyone should do support, not just read the emails.

That's a really fast way to burn out your entire team.

Devs don't necessarily have good communication or people skills. They haven't been trained to respond politely when someone sends them a screaming, all-caps e-mail that their product is garbage. That's my job as the Support Engineer.

The gp comment mentioned once a month doing support to provide empathy for Support Engineers. If doing an activity once a month is so bad as to cause burn-out, perhaps us Support Engineers are the real suckers?

I would argue a once a month support engineer stint / shadowing would have more benefits than drawbacks.

While that stereotype may be true for some developers, I don't find it holds for most devs I know and work with. People incapable of dealing with emotion probably don't make great employees to begin with.

The emotional response is probably the most egregious that a developer can do, but I stand by my assertion that developers aren't trained in communications. I work with some developers who write amazing code, but if you read their emails, you'd think they were written by a ten year-old. The responses are terse, lack proper punctuation, and usually rife with misspellings. And that's fine: their job is to write code, after all. But it doesn't look good if that raw response goes out to the customer.

At one of the companies I've worked for, the tech team can be directed a ticket or whatever to look at, but they respond to the customer service team, not the customer.

When I started at my current place, I had some great learning experiences answering the support phone after the regular support staff had gone home for the day. Officially our support ended at 1600, but sometimes I'd hear the phone ring 5-6 times in a row, so I figured they probably had an urgent issue and picked it up.

Since I was still quite new, this introduced me to areas of the program I didn't know very well yet, as well as learning just how the users used our program. This knowledge has been invaluable when improving the program.

I also had several instances of me going "wait why did you click there before going here?" with a reply along the lines of "oh if I don't do that I'll get an error message, let me show you", bugs that had never been reported yet being present for years.

The flip side of this, of course, is that not everyone wants to be involved in support and that sharing around the support burden can lead to a loss of focus within the organization. Often everybody doing everything is indistinguishable from everybody doing nothing.

At my last job the support team was a thin abstraction layer that seemed to pass almost everything directly through to my dev team. For a while I enjoyed it -- it is satisfying to see how my efforts could directly help customers, undertand how they were using the product, etc. The type of software I was working on meant that technical configuration problems were showstoppers and people hailed me as a hero whenever I stepped in and resolved their issues.

The downsides were insidious and slow to show themselves: lack of time to invest in fixing problems in a systematic way rather than helping customers one-by-one because of time consumed by support, inability to focus on feature work due to support-related interruptions up to 2-3 times per day, the sentiment from leadership that we weren't delivering new features fast enough because of our invisible support labor, the feeling of being "always on" because we had customers with high-urgency tickets around the globe, etc.

After two years of that, I was staring burnout in the face. I told my leadership I didn't want to work this way but nothing changed. When a month went by where I only made 3 commits to source despite feeling overwhelmingly busy I knew something had to change, if not in the org then in myself. I told my manager I simply would not participate in support any more and was deleting Slack from my phone, that if they wanted a good customer experience they needed to make the necessary investments. In my mind this ultimatum was the first step toward quitting my job. But an amazing thing happened once I started putting boundaries in place: leadership started talking about the need for better systems, better documentation, the need to prioritize work that would make the product easier to use and support. My productivity on feature work skyrocketed and my product managers were happy because we were over-delivering for their bosses after under-delivering for so long. I was given a large raise a few months later, the kind people normally have to change jobs to get.

For myself, I learned it's important not to enable dysfunctional process just because I can excel, for a while, in that process. I learned it's important to set boundaries in my work. I learned that focus is sacred and should be protected. If you are a leader in your company thinking about diffusing the support burden, carefully consider the productivity cost to your most expensive/valuable employees that comes with repeated direct exposure to customers.

So much this. I'm working for a company that used to have Support Engineers and now does not, and I now get bombarded directly with requests from customers, and I haven't been able to make any effective new development happen for months as a result.

Those Support Engineers are invaluable. They can investigate those requests, decide which ones are legitimate (we get a lot of either one-off minor freak things happening due to network glitches or customers just interacting with the software in really crazy ways and we don't have time to baby proof the software -- I should note that this is internal software, so we only have a couple hundred users), and for the things that do happen from time to time, like a customer accidentally closed something out and needs to reopen it but we don't normally allow that to happen so it doesn't get abused, we provide tools so the support engineer can fix those things without involving the rest of the dev team.

Once those things are taken care of, everything that's left, assuming the support engineer thinks it's important enough, can then be passed along to us. We used to only get a handful of those a month.

Now that there's no Support Engineer I get these requests all the time, and I'm getting burnt out, as I'm effectively working two jobs now instead of one (actually four, as I'm also an Architect now and the director of the phone systems, without the promotion, and basically have four bosses now, who don't communicate with each other and each have their own high priority requests I'm expected to do for each of them in addition to dev and support work, but that's another story....we lost a lot of people and there's almost no efforts being made to replace them).

You hit the nail on the head right here. Helping support is literally reward-less work.

Agree 100%. The trouble is, you can't ask everyone to read everything, and it's not always affordable or even possible to give everyone access to the data. Our goal at frame.ai is to help you understand and act on your chats and emails -- everywhere they happen -- by drawing your attention to the conversations that warrant it.

Data are normalized and unified from sources including Zendesk, Intercom, Slack, and Service Cloud. Destinations include manual export, with triggered alert and warehouse sync under development.

In the middle is a layer of enrichment and search-based dashboard prototyping. Enrichment includes "sentiment moments" (wins, issues, risks), conversation cleanup via elastic tagging, and auto-tagging. You can see a peek at some of our research in this area at this blog post: https://blog.frame.ai/learning-more-with-less-1e618a5aa160

Importantly, there's no per-seat charge. We think everyone in the company who can have access should be able to explore customer conversations, visualize them, and export for further analysis and presentation.

One great thing my company does is put a prominent link on the intranet homepage where anyone in the company can schedule a 1 hour shadowing session with customer support. You don't need a reason and within a few hours you've got time scheduled within the next few days. I think the whole process was actually driven by support as they wanted the rest of company to know what they do and see what they see.

So, does anyone actually do it?

We have a policy that if a bug generates more than 20 support calls in a day, the developer who introduced the bug has to spend the next day in the call center answering support calls. It's not designed as a punishment. It's designed as an eye opener to the effect it has when we don't write proper tests or don't take proper care in making changes.

Does it "work"?, because it really does sound like punishment.

I'm an advocate for developers reading/handling support things, but at the same time the skills that make a good developer are not necessarily the same as those that make a good support person. Having some rota/schedule makes sense, but it seems like a full day of doing support isn't going to make a developer happy, primarily because they want to be a developer - not a support-person.

I started my career in technical support and looking back it was one of the greatest learning experiences.

The Support Team knows the product better than anyone in the company which is extremely valuable if/when you get promoted to another position within the company.

I did the same. Worked in support, moved on to web development. It teaches you so much about understanding and communicating with customers.

I had some conversations recently where I offered my opinion.

"I think this might be a bit complex for the given users who are using this."

"But they want all this information, we're doing it."

A few weeks later end customers are all confused and they're up in arms because they can't figure out how things work because the product threw the kitchen sink at them on one or two pages...

Now its impossible to email many companies I find. You click on "Contact" and get Twitter, Instagram, Facebook... but not email :(

A few weeks ago I tried to make a complaint to the national mail service, because they decided the pickup point closest to me was about half an hour from my house, rather than the pickup point that was literally around the corner.

They had an online complaint form, but you were only able to use it on workdays between 0900 and 1600.

The Social Security website has limited operating hours.

https://imgur.com/KYSOqUn

> They had an online complaint form, but you were only able to use it on workdays between 0900 and 1600.

Was it an interactive form with immediate feedback? I'm trying to figure out a way to justify hours on a feedback form.

They promised feedback within a couple of hours. Even so, I see no reason to limit it in this manner, just tell them you'll to to get back the same day or the first workday...

Oh yeah, that's ridiculous (they won't be getting back to you in a couple of hours after 4 pm). I'm sure it's a result of a institutional momentum, and nothing to do with actually capabilities.

> Now its impossible to email many companies I find. You click on "Contact" and get Twitter, Instagram, Facebook... but not email :(

I did the website for a nonprofit I set up recently.

From past experience leaving a catchall "contact" email address around, we decided to omit a public email address this time because we didn't want it to be harvested and subscribed to random newsletters.

But there must be way to deal with it. A form (where you can enter your return email). CAPTCHAs.

At Amazon, our team did a day of shadowing tech support calls and it was extremely illuminating. You think you've designed your product to work a certain way, but your message may not be communicated very well to your customers. If it's possible, I would try to get on a support call or support a customer to gain that empathy to really understand what challenges people using your products are having.

At Re:amaze, we don't hire for customer support personnel. Our culture is that everyone that we currently have on staff, be it engineers, co-founders, marketing - all work on customer support. Granted, we're a small team right now that makes it quite possible to do so, but it really keeps things super lean. We're also a chat and helpdesk platform, so it helps to have everyone use and understand the product.

There's also plenty that can be learned from answering support - from discovering bugs to fixing them to understanding feature requests to satisfying customers (and the joy that you get from a happy customer), that it's our main onboarding tool for our hires across all departments. Sometimes support can take up a couple hours for a team member, but it's totally worth it. Not to mention our customers absolutely love our helpfulness and responsiveness in support and is one of the main reasons we've had plenty of businesses switch over from our competitors.

That doesn't work at scale. Imagine your marketing team fielding password reset failures all day.

For a small company though, sounds like a fine way of doing things as long as people don't get bitter about the ones who actively avoid doing the support.

By all means make e.g. Zendesk tickets available on a Slack channel, so people can dip their toes in and see how things are looking.

Don't spam everyone with every support email. You need a ticketing system at least to coordinate work, and prioritization and assignment to get workflow. Don't spam everyone with new tickets in the ticketing system either.

Having a rota of developers who either work directly with support, or on diagnosing and addressing issues that have come from support, is also a good idea I think. Problem areas will filter through from this.

Having all developers always interruptible by support will just slow things down. Interruptions from support often blow away half a day or more worth of development time, from context switching, checking out specific branches etc.

How do you prevent this from becoming another stream of continuous disruptions that prevent knowledge workers from having enough sustained, concentrated time to develop big picture solutions?

Obviously you want to prioritize work to address what adds value for the customer. But can you really do that if everyone has to be constantly disrupting their work to scan incoming streams of idiosyncratic support requests? That doesn’t make sense to me.

I’m all for cultivating empathy for the customer and keeping people informed, but why does that need to be based on a constant inflowing stream of requests, as opposed to a monthly customer-focused all-hands meeting or something?

This is literally what the 'management' types should be doing, and then prioritizing this work along with feature work.

The support org should have enough staffing to be able to 'train up' and start tackling more and more issues themselves. Even if they're not writing the software, they can often work to get reproducables or get you most of the way there "null pointer deref of x caused this bug" etc.

They have it timeboxed to 30m a week (month?). That should prevent constant interruptions.

It’s not like they’re making everyone true combo support/dev people. Just giving devs an occasional taste.

One of my proudest moments in my career was when I took this to the next level and set myself up as the go-between between support and dev. We were building internal productivity tools so we could help the support folks clear things up more quickly, and give them the administrative resources they needed to do so. It was all about empathy with the user: the end user, and the support user on the admin side.

It was not a glamorous job at all but it had a meaningful impact, one that many people gloss over, and I feel the same when reading this. The best thing you can do is care about your users and the people who deal with your users.

We did this at Loom (W12) and it was one of the most powerful things we could have done as a team. Internal comms became much more streamlined as a result and it was usually apparent to everyone what we needed to work on next.

This works both ways. Once I took a job with a large software house who put me in support for my first 3 months (to learn the system :-) It was the worst software I had ever seen. Customers were calling in with problems that had been solved by everyone else 10 years prior. We had over 5,000 open bug tickets. And most of the programmers who had written them were still there, spewing out technical debt faster than ever (now that they were agile lol).

I couldn't even make it the 3 months. They didn't either. Their customers' lawyers were more agile than they were.

I often listen in to support calls when I'm on Skype with my business partner (I'm the dev, he's the sales / support / biz owner); this isn't planned, but happens because he's the only one who can take the calls. There isn't a single call I haven't learned something from, or (more positively) haven't had an idea on how to improve the product from. I've also learned a lot of respect for him as someone who not only keeps his cool with a (admittedly!) sometimes grumpy dev, but oftentimes grumpy customers.

I built my whole company around this idea. For years, I was the front line support - IMO, its the best thing you can do to improve not only your product but your approach to life. That and having kids.

How did having kids improve your approach to life? (This is for my own comparisons sake since having a kid also changed my life and how I think)

One thing I always liked to do now is imagine the person who's angry or annoying me is just a 2 or 3 year old. It's much more fun and takes the edge off the situation whatever it might be...

When simple (the internet bank) was first getting started, I was in their public beta. Support was handled by the developers. All the engineers had to do rotations on support.

As an engineer, I think I would hate that being a required part of the job. But also as an engineer who works in the banking/transaction processing industry, the support was damn amazing.

A ton of things about the company went downhill as they grew and especially after they were acquired, but the support turning completely crappy was the worst by far.

This is a great idea. Read a few customer emails and you will never again have a dispute over whether a GUI is simple enough to understand. Whatever you do, it won't be.

Excellent policy. A couple of other operational "eye openers" that have worked for me:

- If you run a business targeting the enterprise, have everyone on the management team go on a sales call once a quarter. Often the CFO will not actually understand what the product is, much less how customers think of it.

- If you have sales people making cold calls have management listen in on a few. Again, you'll see how people think of your message.

I loved my time at Twilio partly because Jeff made it a point to make sure everyone, no matter their role, understood what was happening in support. Many engineers, product managers, and even sales people would take support tickets to get a better understanding about what the customers were struggling with.

Every company needs to do more to understand what the customers are struggling with and the easiest way to do that is to get on the front lines.

Once upon a time, most growth came from creating supply chains for things people already knew they needed. Then, it was about creating products that made people's lives better in ways they didn't expect. Now, increasingly, it's about getting an audience together and adapting to what they need as it changes over time. Staying close to the customer is getting more and more important.

When I was at Square the design team would shadow support staff to learn about the kind of problems people are facing and more importantly how people talk about your products, i.e. what they call things, how they try to explain what's gone wrong, what they did to solve it or how they ended up with the problem to begin with.

It's the closest thing you can come to true realistic user-testing.

That's not just the closest thing to realistic user-testing, production IS the user testing phase :P

Yup although the sucessful interections with your product is as important for context of where it goes bad.

Sadly in many companies that’s as close as they’ll ever be allowed to get.

Actual reports from actual users are priceless to build what provides value actually worth paying for.

I've met devs considering good code and well engineered systems as their ultimate work goals. While myself, only ever saw software as tools towards any actual value keeping the business afloat.

Does that make me a weird software engineer? Or one that should look for some better suited role? Still wondering...

It doesn't make you weird, but unfortunately may put you in the minority of developers. There is a balance to be struck. Business can drive really bad technical decisions (debt) due to time/people constraints that take decades to remove. This may be needed simply to get the company going or stay going (see most startup MVPs). Those decisions need to be intentional and strategic, otherwise it becomes the norm, and your entire company gets bogged down in supporting "legacy" systems. On the other extreme, a "perfect" solution can never be delivered so it's kind of hard to sell.

My anecdotal experience is that these trade offs are rarely consciously made, instead made by who ever has political power at the time.

Good point. I may add I constantly try to learn and understand best software or engineering practice while working hard to also stay aware of whats most useful for the business. As you say " There is a balance to be struck"!

Like trying to know enough to deliver value while avoiding shooting yourself in the foot and also not wasting time on what's only seen as a black box by outsiders?

Starting out in tech support 100% made me a better developer. I've seen the user vs developer tension from the side of the user who deals with broken things every day, hoping that the dev team will eventually fix them. I know how to talk to users now and find out what they really need. And I know how to talk to developers and managers about prioritizing issues.

Bill Gates used to take Microsoft support phone calls once in a while, to get a sense of what was really bothering the customers.

As a user experience designer the first thing I ask for is access to any and all data available, including support chat system and access to support folks so I can quickly gauge the general level of issues being experienced. It’s just one way to identify possible issues, far in advance of jumping to make any UI changes.

> and access to support folks

This is key. Support people is the ones that talk with customers, and know what's going on. Their experience is key.

In most companies this people is low-pay look-down people, gets tired quickly etc.

> Unless you are a soulless robot, the above statement will probably trigger more emotions and requirements for actions than:

Is that not an appeal to emotion and thus incentivizes customers to embellish their issues with emotional stories to get expedited treatment?

I created a tool for smaller companies/individual developers that allows users to submit feedback that gets pushed directly into Github issues: https://bugbucket.io/

I agree so much that I built an entire company around it.

The insight and nuance that comes from reading support conversations is second to none but more importantly emphasizing with your user enables you to focus more on building intuitive products.

Where I work dev team also does the L3 support: we have L1 who can change the password, reboot the computer, we have L2, who know the basics of our app and its surroundings, but if an issue is not obvious to them - here we come!

why does l1 even exist

Who says the ability to "reboot the computer" is avail to the user? (ie. remote RDC connection or a locked down retail POS terminal).

Also, the "real purpose" of L1 tech is to categorize and prioritize incoming calls for further L2 / L3 processing. Knowing "who" needs to handle the call next, and its priority, is usually critical to getting real problems resolved timely.

Because most people have very basic questions, and you don't want to waste dev time because someone don't know the basics of their OS.

Password changes, mouse cables pulled out, icons disappearing from the desktop after windows update and other stuff like that.

They are prob able to hire a cheap on call firm to handle it.

Interesting SaaS idea...a company that plugs into whatever CRM you use (Zendesk, Freshdesk, Salesforce, etc.) and automatically compiles a summary of tickets that is sent to certain people or teams within the company.

We do this at Firebase! Actually right now I am about to start my week as "Support Sheriff" which means that I am the escalation point for support cases. Everyone on my team does this about once a quarter.

Great job on your site design. I have not looked at the source, but it loads quickly, works without JS, and all the elements are accessible with keyboard hints.

I think support needs a communication channel with development too. Both teams need to grasp the challenges of the other.

Most of the support emails are really dumb. Instead of reading product description or help they simply write you email and ask you. 98% of users get through intuitive design. Example1: Customer didn't know how to switch camera back/front and the product used the same icon and approach as Apple. Example2: Customer denied access to camera and asked at support why camera doesn't work.

These sound like legitimate user experience problems. I have fallen into the trap of example 2 at least once, not because I didn't know why the camera didn't work, but I didn't know how to resolve it at the time.

One can't design for 100%. Designing for those 2% simply isn't worth and doesn't bring any cash. PS: user did not read camera access usage description, clicked deny and then asked why it doesn't work. This is pure stupidity.

more we are closer to customers in any way possible, we will have a great future as we get to know what we our customers want not what they need.

We need to understand customers the way we understand our program. deep dive and debug their issues and come out with a solution that is win-win.

Great idea if a product of a culture.

Terrible idea if people are 'forced' to do it.

Be careful, it can be demoralizing for engineers working on the product as you can get the feeling that nothing works.

I would presume most competent engineers have a pretty good grasp of what is or isn't working. If something doesn't, they probably knew about it already, or at least aren't surprised.

And if they are surprised that's the entire point.

All 3 employees.

In companies people must share support emails to improve their product, that's real.

There are better ways to understanding your customer than encouraging your entire team to give attention to inbound support.

Care to elaborate?

You have your folks on the CS frontline knowledge share their experiences with the company

Everyone should also get paid more then. Also, if 33% of your job is answering support emails, should we still call it a "Software Engineer" position? How about being very clear about these expectations as-part of the job interview process?

Reply on news.ycombinator.com