Backlog size is inversely proportional to how often we talk to customers
bitbytebit.substack.com
bitbytebit.substack.com
I hate the strategy of just taking every idea you hear and throwing it into a ticket. You just end up with this giant icebox of stuff you'll never do. If a big new prospect demands one of the ideas that's in the icebox be implemented immediately in order to close a deal, you're probably still not going to pull it out of the icebox, because you don't remember that it's there. Instead, you'll just create a new ticket for it, and eventually when going through the icebox, someone will go "hey, I think we built this already" and close as dupe.
Instead, I strongly prefer to have tickets that at least have some possibility of getting done in the short to medium term, and store other ideas elsewhere. Engineering keeps a list of tech debt that they'd like to address. PMs keep one list per project of possible improvements. For potential new features/products, they write PRDs but don't immediately turn them into a bunch of tickets.
Ultimately I think the giant backlog of stuff that mostly won't get addressed is a sign of weak PMs who are afraid to say no and like to fall back to the comfortable answer that is, "sounds interesting, I'll write a ticket for it."
In short: maintain a list of long term refactors that everyone's gradually working on, without necessarily showing it in the ticket tracking system.
JIRA + a tree of tickets is so bad for project planning I recommend using the backlog as a diplomatic tool and a polite way of letting things get lost. It is is better to use advanced scheduling tools, like a spiral-bound notebook or sticky notes, rather than a backlog. The backlog is for management and politics.
I hate this approach. I totally get it, to be clear. I've done it. But I hate it. It's a sign of a really bad culture if you have to placate people in this way.
If your senior leadership is clearly communicating priorities, you should be able to say, "Look, this is an interesting idea, but as you know this quarter we're very focused on X, and this just doesn't align with that. The best route to get this done is to give your feedback in the next planning process and see if you can push Y to be a priority, then we can definitely talk about getting this on the roadmap."
A huge part of the job of a PM is to say no. If people aren't able to take a no on feature requests, there's a problem somewhere. Maybe it's with the way you're delivering it, maybe it's with the overall culture or maybe it's with the person making the request. Whatever the issue is, though, it's better to deal with it and create an environment where people understand that sometimes their requests will get declined for valid reasons.
Get a stable product and then wait till someone else does something innovative and copy that.
What do you use and what are its top features?
If you just drop every customer email complaining about a supposed problem into a ticket in your company’s private queue, no one but internal employees will ever see it (and 99% of those won’t ever notice it or read it) but if you give your users a place to vent and discuss their ideas (with moderation of course) you can have a pretty powerful new idea/roadmap tool.
But if they're around anyway, they can keep it in whatever personal backlog they have instead.
How should I manage my personal backlog? There’s over a 100 post-it’s on my desk. Should I create another backlog for longer-term ideas or maybe hire a personal PO?
> Should I create another backlog for longer-term ideas
IMO yes. That way you can easily review it before meetings, and come ready to said meetings with your most important items. I think this is quite a personal thing, so this kind of workflow may not work for you. But I found it almost impossible to work without such a list.
It's work to keep a backlog not a total disaster, but that doesn't mean the answer is "tool x" or whatever. It's just work. Ugly annoying work. Ie, work.
Seemed super effective.
Why? Why have two places for the same thing? It's just an awkward tagging system. When would you search this but not search tickets?
Alternatively, just don't write them down. If you don't get a benefit from keeping the idea, don't write it down.
Mid-option, I think linear does this by default, cancel backlog items older than X. Not been touched or moved? Probably not relevant. Plus you can always search through cancelled tickets.
A ticket is a different level of detail. Takes more time, thought and effort to put together a good ticket. If it's for something that's not going to happen for 12 months, then there will have been enough change in the product/org/etc. that the original ticket won't work as written anyway.
A doc with a bunch of categorized feature ideas has enough info that you don't lose any good ideas, and then when it's actually time to attack one of those categories, then you can do the work of building out actual, ready-for-implementation tickets.
> Alternatively, just don't write them down. If you don't get a benefit from keeping the idea, don't write it down.
Totally agree with this. A lot of stuff isn't worth writing down. If it's way out in the future and really important, it'll come up again anyway.
A ticket-system like JIRA is the same as an e-mailclient to me. You sort by activity, having the most active tickets on top. You can flag/mark them, you can put them in different boxes or just use tags,... All are just means to create order in chaos.
But I would never think to move my old e-mail to a different 'system' than my normal e-mail(client). I never think of old e-mails or drafts as something I need to clean up.
I'll frequently have an idea and toss it into a backlog ticket. Maybe just as a title to start. When a customer mentions that problem/feature, I can add that info in the backlog even if it doesn't immediately become a priority.
When the issue does become something worth working on, I'll usually add more details but the various evidence accumulated over 6 months of letting the item "bake" is invaluable.
You can then just tag them.
This means that a search can ignore them, or it can bring them up while you're looking at features/etc.
All that administration overhead is why the ticket based system doesn't get this information. If it takes 30 seconds to create (and properly organise/categorise) a ticket then that's too slow. I want to be able to write tickets as fast as someone can say them, and move/reorder them with my usual copy paste / text editing tools.
For example, ctrl shift v is how you paste unformatted in ms teams. You’d think by that logic, ctrl shift c would be copy but no it starts a call. There is no built in way to eliminate that and when I ask ms people they say “oh use powertoys and blah blah I had to do the same”.
There are multiple times this has been raised on the forums
https://answers.microsoft.com/en-us/msteams/forum/all/turn-o...
Do you want to put things like this on a different list, basically a “won’t fix” list? Why even write them down? Just to make people think you are listening?
This is across 8-10 jobs in multiple industries over close to 20 years. Some places have been better or worse than others but I can't think of any place where people routinely were creating, organizing, and categorizing tickets this quickly, at least not tickets that had any value to anyone other than them as a reminder to do one very specific thing.
Sure, for tickets that you deliver. But when we're talking about managing a backlog there might be 50 potential tickets that never get implemented (or even scoped) for every one that does. And you don't really want those to ever become a ticket at all because that ends up being far too much product work, precisely because it takes ~20 minutes for each ticket.
Put another way: you need a way to track high-level work for prioritisation before putting too much into scoping and detailing what implementing it would involve.
The trick is to have each "ticket" be basically a bullet point in a list (where lists are grouped by headings/subheading in a single document). Maybe some are 3-4 lines, but most will be a single line bullet point. That way you don't have to search it (which doesn't tend to work very well as you won't know what you're looking for), you can actually browse the entire document.
But it's worth reading the comment in context. If your problem is that old tickets are a nuisance because they're almost always irrelevant, just have them auto cancel and when you get that notification rescue the 2 per year that you truly want to keep.
If you don't have that problem, keep them.
> Why search through cancelled tickets which are resolved to find and unresolved issue?
Why would resolved tickets be cancelled?
Also, why search through two different systems?
> when you get that notification rescue the 2 per year that you truly want to keep.
That makes even less sense. Looking at notifications and making a decision on closing/reopening is a form of review. If you have time to review, do it properly on your own schedule and in batches rather than be beholden to some arbitrary autoclose timeline with yet another focus-destroying trickle of notifictions
It was when it was created, it's not now. And it is then practically removed.
Again you can build whatever workflow you want, tag stuff or move things if they're super important to you. Subscribe to them if they're vital. Or don't auto close if that doesn't work for you.
I don't think that having a tiny number of issues that are
* Never looked at
* Never prioritised
* Never searched for
* Not subscribed to by anyone that cares about them
* Somehow super important to not have a "cancelled" status on them
Should define your workflow to this degree that you start keeping multiple lists of work. I'm not sure I've ever come across an issue worth leaving like this. Can you tell me about an issue that fits this description?
> It's not now
You don't know that, in the process you yourself proposed that determination was done when you get an autoclose notification
A ticket is something where there is an understanding that we want to do it if time and resources permit, something where it would be okay for someone to grab it and do it.
On the other hand, an idea or suggestion may easily be something that should not ever be done, that drives the product in an inappropriate direction or adds an ability that we don't really want to exist for various reasons; and that is conceptually fundamentally different and should be kept in a separate place from the "big todo list".
If you want it to be that, sure. They're really just notes with IDs and tags. What you're describing is a curated backlog / upcoming work.
> On the other hand, an idea or suggestion may easily be something that should not ever be done
Ok? So either say why and cancel the ticket or delete it if you don't want a record of it. I don't get the value of writing it down in a different place so that you have to search in two places.
Usually when customers give feedback it's to address an immediate need they have, so they'll suggest adding a button/option/toggle/whatever. And that's fine, but if you do that for every piece of feedback, you'll end up with an unusable mess. Especially because it's usually very easy to add that one extra button/option/toggle.
You usually require someone to go through all the feedback and try to address the root cause of what they're asking for. Sometimes the extra option is unavoidable, sometimes it leads to redesigning some system to be more generic to support what the customer wants + more.
I do think the feedback needs to be linked to the work somehow, not hidden away in a separate PM tool or whatever. Ideally retaining the raw feedback, because sometimes when summarizing it, things get lost.
There is the issue of dupes but it avoids the other issue, which is that if you hard decline too many requests from right after they are submitted, stakeholders may stop giving you feedback, which is a bad thing. So it is an unhappy middle ground (which is I feel the normal place which product management occupies. I could setup a small shop in the unhappy middle ground and sell souvenirs.)
Maybe you don't want raw feedback dumped directly into your development board, so you might not just add extra columns to it, but another parallel board whose "done" column flows into the dev board's backlog seems like a reasonable approach.
I agree that you need some system to track feedback. I'm not convinced a board is the optimal choice, because (some?) feedback is kind of timeless. In my mind, it is never "done". Feedback can apply to future features that haven't been thought of yet when the feedback was originally processed, so it's valuable to keep it around in some sort of tag based system. You probably want a way to easily search it as well when designing new features.
In my experience it's really better to keep requests in a lightweight PM tool until they have garnered enough interest to write out as a ticket. The tool I use is Dynalist, really just a glorified todo app, where you can sort/collapse things that go together but there are a million tools like it. Only when there is some certainty it will ever be built does it get a proper ticket for the devs to look at.
But if you find yourself constantly fixing things that were specified and should actually work you got a problam with code quality. And if you constantly have to add small new things there is a problem with your customers clarity and a bigger meeting might be in order.
If someone wants to save an idea for a later iteration, they can do that however they want, but they don't get to clutter the team backlog with it.
It also mirrors my best experiences with well-managed backlogs.
This necessarily means that the backlog probably grows.
And you are doing both them and yourself a favor by making this clear. You might lose a couple of customers. But you would have lost them anyway and unhappy customers can also be annoying for everybody and slow you down, give you less than favorable reviews, etc.
Suppose the idea is likely impactful but complicated to implement. In that case, you might want to think about putting those ideas together to argue for refactoring that would enable more impactful ideas.
If the idea would change the kind of service being offered, then it should inform strategic product thinking.
If the idea is relevant but unlikely to have an impact because it's too low level and about code quality, maybe you want to think of automating or including it in the linting process.
I think the problem here is that the tool, Jira tickets, isn’t suited for the use that you make of it: communicating with clients and defining product strategy. The problem is that there isn’t a good tool for that.
- It provides an obvious place to put some ideas about then that can be collected over time. Often thoughts from significantly different points in time can provide valuable perspective. Even if most of the time they are just out of date.
- It allows tracking value over time. For example adding a quick note that this would have been helpful to customer 123. If you only file tickets when you are ready to implement them it is easy to miss the common friction problems.
So basically it is a way to log interest over time. But they are also easy to filter out so low cost to keep them around.
But I'm sure that you could also manage these "feature requests" in a separate system. But that often adds friction.
Therefore the giant "icebox" is out-of-sight but the social factor of "there is a ticket for my idea" across the company is also fulfilled. works pretty well so far!
When your company has 5 projects, do you count Project Z's backlog against Project X's backlog? No you don't, even though they are both in the same database. The reason you're OK with this is because your teams' views have been adjusted so that they only see their own project. So they each see what they consider to be their own backlog. Use the same approach for low-priority requests: put them in the same database, but don't elevate them to the team's backlog view.
If someone has an idea or a request you should not suppress it entirely. That will only encourage people to add yet another system of logging to your organization. Just because a request is not urgent or can't be put on the roadmap yet does not mean it's not important.
And why throw away what can be logged for later archeology? "Oh hey the answer to this new problem is right here in this old request we never got around to."
This gives us a natural filtering mechanism for community input. We regularly sort this by number of upvotes. It also lets our users submit some off the wall ideas, and gives a way for low friction community feedback to be captured. Finally, it lets community members easily be notified when a feature is delivered by subscribing to the issue.
We also have a separate, private list of commits to customers. (The commit is public and is added to a milestone, but the customer is private.)
We also group issues by theme[1]. It's not perfect but it's an attempt to make sure we don't run into the duplicates issue you mention. It allows the eng team to address multiple issues in the same area of the code at once.
I understand what you are saying about clutter, but the low cost public feedback and tight loop between issue and code is worth some issue creep, IMO.
0: https://github.com/FusionAuth/fusionauth-issues/issues
1: https://github.com/FusionAuth/fusionauth-issues/issues?q=is%...
Still think there's value in having an open roadmap and feature list that customers and users can submit too, regardless of industry. I wouldn't recommend you use GitHub, though :).
(/S this isnt actually a good idea but may be practical depending on the org)
As for my first attempt to guess the power structure: vote = 1 + sum(direct_reports.vote)
You could have a suite i guess.. maybe look for overlap?
Satisfaction? Vote = vote * staff turnover for voters role.
Greed? Vote = vote * voters team profit
Top down control? Vote = vote * voters grade
Reverse disfunction? Vote = vote * number of complaints raised by voter
Cross functional impact: vote = vote * num departments voting for this
My question marks reflect i have no idea what the real optimisation is when you select a defined metric / score.
In reality (as someone who wants to make things hollisticially better in a sustainable way) i look for the overlap of several things:
Who do people have on speed dial to fix issues? Will this reduce how many calls they get? Ignoring what official processes say.
Will this impact the bottom line?
Will it increase customer satisfaction?
What will be the impact on staff turnover? I prefer to reduce this but at least there needs to be a mitigation if its negative. Key questions are: will people better understand personal their impact on Cost and quality? Will this make success and failure clearer outcomes for the individual?
Then finally will this be a step towards or away from governance principals of org? E.g. does this cause party A to have needless influence or judgement over party B (at all levels).. basically will this enable individual agency at all levels.
I always justify it on paper in terms of governace efficiency/efficacy, profit and quality.
They are not actual features, they are a information collection on how a idea might be not possible or a placeholder for a better solution not yet in the making.
Not all tickets have to become a feature in software- some can become wikipedia pages on requirements/conditions of the project.
This was part of the wisdom of the ancient agile way: a box of index cards for your backlog. The physicality of it gave clear indication when it was getting silly. Furthermore, regular backlog pruning was a necessary activity because the box would get full!
I do think it's important to organize all feature requests, pain points, feedback etc. though but it needs to be separate of delivery management and it should be thematic (assuming there's a lot of feedback). That's the point of CustomerIQ (I'm a founder), to give that a home but also quantify themes over time. These would inform the backlog/possible make it in, but keeps delivery tools cleans.
For old PM the backlog is a memory prosthesis of all the old product conversations (a graveyard of broken dreams).
New PM will look at backlog in horror "what even is this I don't understand it", declare bankruptcy, "clear all that crap out".
Especially when all the tickets in the backlog just have 1-2 sentences and can't possibly be understood without the context that lives in the head of the original PM who wrote them.
---
"Redo John's fix for frobulator service"
Definition of done: The fix is implemented correctly.
At one of my jobs, they used Asana when I started. It was too full of backlogged issues, so we moved over to Jira. Then Jira got too full. A month before I was laid off, one of my coworkers said, "Maybe we should try out Asana."
And it’s not so easy as “find out what will make your customers lives better and make that your next feature”. What if you have more than one customer, and the features that will make their lives better are unrelated and each is large?
You should always listen to your customers. You should almost never build what they ask you for.
A customer will ask you for a feature like “put a button on the second screen that invokes this other tool”. Inherent in that request is not only a UX implementation, but also a statement of a problem that they have, and a muddled business process that they use at the current time.
You need to figure out what their business problem is, but throw away the ideas for UX implementation and anything process specific.
Then build the minimum feature that solves their business problem, in the most economical way for you, and consistent with good UX design. Bonus points for building lots of integration points that allow your customers to extend your software while isolating your customers business processes from your code.
> Every team I’ve worked on that maintains a backlog, got most of their backlog from customers.
Or maybe you aren't the one who calls Bob, but someone from Sales, or Customer Support is the one who calls Bob, so now you don't just have a back-log, you've got two panicked people who are sure Bob is going to take his business elsewhere.
First, Bob's easy. Bob cares about all of those little things but most of the time, Bob's not stupid. He's fully aware what's important and what's not, the reason he brings you the whole list is because he thinks that will get your attention better than it did when he brought up his "critical feature" that got sidelined. If you don't sell to thousands and thousands of customers, but hundreds with a tens of really important ones, and Bob's one of them, the problem is failing to apply empathy to the Bob problem. Personally, I love these guys because they're usually very easy to placate. Usually Bob's list has 100 things on it, 99 of which are solved with the same four lines of code, one of which is a horrible pain in the ass. Usually at the beginning of the conversation, all 100 are important, but after you finish 99 of them, sometimes Bob doesn't even notice the one that's missing, he's too shocked that not only did one thing get fixed, but his list is gone! Call Bob in a week to follow up; if you haven't been neglecting him too long, ... he's got one or two bullets or nothing at all.
But for all the others, yeah, the backlog will grow ... initially. If you keep following up, and keep reaching out, though -- you'll find your customers are on your back less because they know you know what you need to do and they know you're working on it. Over time, it becomes easier to temper expectations, you have fewer emergencies, things get scheduled more consistently and the backlog ... eventually starts to shrink again. Your customers not only have limits on the things they want/need out of your application, they also complain less and less feverishly about the things that do irritate them because it's "just the one button that's wonky" and not a thousand little paper-cuts of things that just don't seem to work like they should.
And yes, always listen to the customers/never build what they ask for ... except ... that's only good advice in very specific circumstances. If you know your customer's needs better than your customer does than that's the best advice you can take. For everyone else, "always listen to your customers until you understand it precisely on their terms" then start asking them questions to understand why it is they're asking for the thing they're asking for. The problem with the "the customer doesn't know what they need" premise is that if you dismiss what they're asking for and the thing you produce doesn't fit "the thing they were trying to accomplish by asking for that" everybody loses. Sometimes, hell often, you're better off giving the customer exactly what they asked for than missing the mark by even a small amount even if you provide something substantially better. What you end up with is "a cool feature that nobody wants because your customer really did just want the 'print to PDF' button because of some archaic fax workflow you were unaware of" (or insert some other weird example).
I'm not poking fun/chiding here -- I give the same advice to others all the time, but it's easy to forget that "you can't tell the customer what they want if you don't talk to your customers" :)
Everything else you write is spot-on, too. There's one issue with the way their their platform presents data to our customers that's been killing us - to the point that we've lost sales, and have consequently stopped using a major product feature. I've explained the issue to everyone from our customer reps, to sales engineers, to two successive SVPs of that product area. Everyone gets it; everyone says "oh, yeah: we need to fix that". It's been five years, and I've given up hope that it'll be fixed.
They've created another annoyance, which speaks to bad practice. Their reporting tool doesn't cover an edge case that we'd like to address. It's no problem, no one can think of everything; I'll use their data API to build a custom report for myself. Except... their API doesn't expose the one field I'd need to make my report possible. That data isn't a security concern, and is being used elsewhere in their standard reports. From that (and a couple of other examples), their report system clearly doesn't use their data API. Dogfood your APIs, people! (Five years of feedback, and that's never been fixed, either.) For the love of God, dogfood.
And, if you're going to solicit feedback from your customers, do something about it, or else explain why you won't. Don't solicit information, nod about it, promise fixes, then not. Don't then do the exact same thing again, with more senior people, a year later. I've built and maintained enough software to know that software is hard. I'm pretty sympathetic towards engineers, and fairly forgiving about software problems. At some point, however, these problems cease to be software problems, and I'm much less sympathetic about that.
So, everyone else reading this should take it from "Bob": this guy's advice is how to maintain your customers' faith in your product and your company.
But yes, customer trust is based on both listening and delivering results that help them.
I've seen two approaches to this, ProductBoard and Jira Product Discovery. ProductBoard has a CRM approach where you make a note of every customer interaction (call, meeting, email, support ticket, etc), and later you can break down the customer interaction into needs and wishes. So initially all you have is a list of customer interactions.
Jira Product Discovery starts with an idea. So if you want to keep track of everything you hear from your customers, you're forced to immediately break this down into a long list of all wishes and needs.
Benefit of Jira Product Discovery is that it keeps this long list of ideas out of the developer backlog, but it's yet another long list. You can connect Jira Product Discovery Ideas later to Jira epics and stories, so delivery is easier to track.
ProductBoard is the better product discovery tool if you want to analyze product priorities based on customer conversations. It has tools to aggregate and analyze the items mentions in these conversations.
To do this you need complete observability of all customer interactions, i.e., meeting/call notes, presales notes, forum posts, zendesk support tickets, services interactions, etc. Productboard goes quite far in enabling you to bring all these channels together so you can build a data driven roadmap. Jira Product Discovery on the other hand is quite thin on this, more focussed on handover to the development teams, showing roadmaps, and tracking progress. Hopefully they'll add actual product discovery in the future.
One additional benefit of a tool: you get insights into discussions other PMs have with customers. It's good to be aware of requirements customers have on your product area that they may have communicated to other PMs working on other parts of the product.
I think maybe the implication is that if your backlog is big, it's a "smell" that you're over-planning. I don't think that's true, but I guess the sentiment is good.
Your point about "directly informed and refined by customer feedback" is well-taken, but in my experience a large backlog is rarely that, but more a dumping ground that seems daunting. Of course it depends on who is managing it and how, but more often than not I've seen PMs succumb to big backlogs rather than make it a well-vetted list maintained proactively like you have stated.
By the way, I enjoyed the article! It's a good share, and appreciate the perspective.
Dumping details into something that we can’t understand yet seems like it smacks of a way to manage anxiety. Kind of like an inverse form of procrastination.
It’s easy to believe that we can just think our way out of problems, but sometimes we need to build small things, solicit feedback and iterate. And that can be uncomfortable if there’s something impeding that.
I wish Apple, Google, and every other company on the planet did this. Every Single Day, Multiple Times A Day, I run into UI issues in software (and non-software) that IMO would be mind numbingly obvious to fix if anyone had bother to actually watch a user. And, on top of not watching, it's often impossible to give feedback. Ever try to give Apple feedback? AFAICT it goes onto a black box, if you can find it. Google? They'll only take feedback in certain parts of their apps making it impossible to actually report an issue.
1. Recency bias & opportunity cost: When you're intentionally not working on a problem space given higher priorities, you still want to collect incoming feedback. This aids future work and if the feedback already exists, bump it up the priority list. When the team kicks off projects, you'll want to assemble as many data points and a core source will be scanning through your backlog.
2. Reactive development: If I chose to bypass the discipline of logging feedback (which I'd love to do from an energy conservation perspective), I'd find myself working on the most recent and lowest-hanging fruitful tasks, neglecting the broken windows that have long existed.
3. Team knowledgebase: If there's a single point of responsibility to collate feedback and deliver solutions, then I think the OP's point can stand as it's viscerally stored and probably just more efficient to have a fire and motion strategy.
When there's a team involved, there needs to be a shared corpus to asynchronously log and retrieve data points. Duplication is better than no data and can provide insights when written from different perspectives. There's no way about it, this backlog will quickly get big.
This can be taxing and messy but dealing with complex systems and people is messy. For well-oiled teams, it's necessary to have good housekeeping of your backlog. This includes archiving irrelevant tasks, de-duping tasks, regularly prioritising and ensuring you're making the best use of your tool.
What can be helpful for organization is to deem everything initially as "for consideration" and have a small "up next" & "bugs" column that should contain no more than 5 items each.
The tool itself is insignificant compared to good backlog maintenance.
What might be missing is a facade on top of your exhaustive backlog that surfaces comprehensible information that allows you to dive deeper (eg search, and see similar tickets) when necessary.
This needs a gigantic "YMMV, depends on the type of product and customer, etc" attached to it.
I used to be the PM for a large and complex set of developer tools. The only reason a very large backlog would be filled with unvalidated assumptions is if someone filled the backlog with unvalidated assumptions.
In my experience, a very large backlog can also be filled with validated assumptions, and the way one does this is by talking to more customers.
I can understand an early-stage product having things in the backlog that are unvalidated that probably came from ideation and spitballing sessions in the early stages of planning.
But I would strongly disagree with the premise that "Backlog size is inversely proportional" is a reliable rule of thumb.
I agree that it is critical to make sure the things a team is building provide actual value and will benefit real customers. I think this would be better framed as "Talking to customers is critical to ensure you're building the right things".
Account spoofing is a really easy way to test for issues in production, but it also means you have to trust your support and dev teams with production data. In some industries that is a huge problem for customers. The last startup I worked with gave devs no access to production data at all (customers were all large law firms who wouldn't have bought in of it meant giving access to data). This should really be seen as a last resort; it's much, much better from a security standpoint to work assuming you can't ever access customer data. Your software will be higher quality if you do.
If they had some sort of account spoofing they could have realized what was going on. In many apps there are no problems giving support staff full access to customer accounts. Furthermore, in many apps the team is so tiny that everybody could sit around the same table and that solves many issues with trust, even if everybody is working from home.
However the more you want hide from the support staff and the more it gets difficult to have a meaningful spoofing mechanism. Maybe the support staff might be fed random numbers instead of the actual amount of data, but even the number of operations on an account might be telling.
By the way, after I walked in the bank and made a person change my street address I'm still receiving some communications with the old address. Too many databases.
This doesn't track, at all.
The more you talk to customers, the more useless tickets for non-issues are created.
What you don't want it one person going in, trying to play it as a salesperson, misunderstanding half of what is said, attaching too much weight to bikeshedding and random offhand remarks, not asking whys, almost mirroring everything customer says and promising to deliver it, etc.
Treating it as smarmy sales might be how an entrenched company that already has a semi-credible product can close some enterprise deals, but it's death for a startup that wants to understand the problems, and figure out a good solution they can build.
"Talking to customers" is great for understanding customer problems, but know your business objectives first. Else you'll waste time gather feedback in areas you won't work on anyways.
E.g. You discover a business opportunity to build a simpler confabulator. Step one is to identify the different possible outcomes for customers that would indicate you had successfully built something simpler. Identifying what that means is actually hard. Maybe lots of people say they want something simpler, but to some people that means "less data to manually input" and to other people it means "better guidance in the configuration process". So there's value in this step on its own - it helps clarify your thinking and gives you ways to know if you're successful. And THEN you can identify solutions to help achieve those customer outcomes. Maybe for the "better guidance" one you have possible solutions of improving the UI or integrating an LLM to answer questions. But since you've tied it to a real outcome for customers, you can do a better job evaluating those options.
It's hard work but the process does a much better job leading to coherent solutions vs. "bucket of features".
If your backlog is massive, people aren't having hard conversations about what work will ACTUALLY get done in the next 12-18 months.
Title: FizzBuzz copilot
Description: ...
Cost: $50,000-$150,000
Potential Value: $3000-$5000Maybe you have one whose job is to convince new people to be customers. Maybe you have one who helps your customers with the parts of your product that are so broken or unintuitive that people have to ask a human for help when they get stuck. Maybe you have one whose job is to keep your customers happy enough to renew their contracts.
I can guarantee these people are frustrated up to their eyeballs that nobody gives any weight to the feedback they have about the product. Their whole job is to understand the customer experience, and feature development is planned by teams of people who don't even use the product themselves, never mind do they know how thousands of people use it.
You're sitting on a gold mine of insight about your customers. Ask the people you hired to talk to them what they need.
Executing a customer interview discovers the why behind feature requests.
If you leave it at the Sales and CS level you can end up with requests like I want to export to Excel. Sure, build that feature, but you risk missing the bigger picture.
Why does the customer want to do that? What’s their pain/problem/opportunity/JTBD?
You may uncover it is to integrate with another system, to give their boss a report, or to do some data analysis.
As a PM your job is to not build what customers are asking for, it is to solve the underlying need in line with your product vision and company strategy.
For example, a backlog is a priority queue. A priority queue can only be long if work is added more frequently than it is removed.
Work can be removed if it is either completed or abandoned.
Work can be added when users request features, users find bugs, product owners predict features will be useful, or the dev team adds technical improvements.
Talking to users will increase the bugs identified and the user requested features.
So by these relationships, talking to customers will directly increase the size of the backlog.
And the overall backlog length may be large due to many factors unrelated to talking to customers: slow development, never deleting out of date work, adding too many technical tasks, adding too much unvalidated vision work, etc.
Does anyone know of any books, blogs or youtubers that bring this kind of logical system level thinking to software work management?
Backlog _priority_ gets smarter with customer input.
Founders have an initial product pitch and guess at what they have to build first, and they and any hires might have been stuffing tasks into the box, but they might still be trying to get prospective customers to talk with them.
There might be a bit of a Catch-22, if you need to have some ideas and/or something tangible before prospectives will talk with you, even if what you've thought of or done so far turns out not to be what they need.
They were a typical "we make products/digital experiences for brands/companies" kind of shop with some very big brands. Most companies their size that I worked for -- including one with extremely similar kinds of customers -- had about 100 employees, of them 70 developer/design/ui/ux, of those 70, 60 were developers, 10 were split amongst UX/UI/Design in different ratios depending on need.
At the company that did it right, half of the 70 were UX/UI/Design. Developers didn't get involved in the product until after project managers and UX had completed most of their work, Design/UI/Development would follow with development laying down the first line of code after everyone else is about 1/3 of the way through. Though we had fewer developers than any place I've ever worked at (even more stark considering the output was much more than we produced at other shops), we had the highest number of Senior/Dark Wizard developers than any place I've ever been, too. The polish of the products we produced was magnificent and we were well known for extremely well designed products. You saw many of them if you followed the second Obama election (those Surface large arcade-like tables were really popular that year -- my company did the app for at least one -- I think two -- of the 24-hour news channels) or bought a few major consumer brands.
This was also the only place I've ever worked with that regularly assembled test panels of customers for direct UX testing (we had someone on staff "that's all they did" was user studies of this nature) -- all paid for by the customer and the results were pretty incredible most of the time.
A dedicated UX researcher! What a rare creature, most UX teams have to fight tooth and nail to get one of those.
The best I can tell is that the company was laid out specifically focused on industrial/product/digital design. It built excellent, polished products[1] and the clients that were attracted to the company wanted that polished product. We positioned ourselves as a partner in executing on that vision but the reality is that we were taking a few sentences of an idea and creating it (with a bunch of hostility from the partner along the way). If the client wanted the polished product, we gave them an RFP that included everything required to get there and many of the things on that RFP came from bruises we received from previous work, so we were ready with "Here's a perfect example of why that's a must-have." And some of these bruises were incredible: I know of at least one project that was over a year past-due owing to a customer's constant design adjustments.
They did a lot right. The design pricing and QA was double what I recall seeing at previous employers. Development cost (equating to time) was a bit more, on average, than previous employers[2]. The development budget didn't necessarily suffer due to design being too large, but we weren't consistent enough with change-orders to handle the fickle customers we often ended up with. Hiring was impossible -- to get the people necessary to build these products they had open positions in nearly everything and hired anyone who was exceptional whether or not they had work for them (we were paid salary). This sort of worked, but after losing a huge contract for a product in a language we wrote almost nothing else in, we took a 25% hit to staffing affecting everyone who specialized in that language exclusively. It was a very demanding job but I met some amazing people there -- I remember a certified genius hardware guy who when I asked about his experience learning "C++" he said "I learned from Bjarn Straustrup" -- delivered factually, deadpan, no hint/intention to brag -- so much so that I took it as "read the books/code/self-taught as he was an autodidact" but he clarified "my employer paid to have my team trained by him (I think in the 90s)." WTF?!
[0] You have to be careful, here -- we used a couple of providers and I can only remember Shiffrin Heyworth and I can't find them any longer. Shame, too, it's even harder to find these kinds of companies if you want to participate as a panelist (most are scams) -- they had real secret shopper programs, expert studies (flying panel participants with specific, narrow, subject matter expertise), 3-5 hour paid catered studies (with a 20% chance of being handed food/a cheque and sent on your way) -- all design/product/marketing focus -- it was a lot of fun.
[1] "Products" was a range, but mostly digital, however we did a few interiors of the early Microsoft stores. They spun off a few companies from products they partnered with third parties to design/develop, as well.
[2] Adjusting for "specialization tax" -- I worked for a company that wrote software add-ons for Skype for Business/Lync/etc ... there were very few of us ... my hourly billing rate made me wonder where my corner office on the top floor was hiding at. Though my dataset is small, it's still about twice what my time has been valued at to a third-party.
Yikes. I would not allow admins to change customer data in production by spoofing the user’s identity. There are too many ways for that to go wrong. Instead, copy the user data into a test environment and mess around with it there as much as you want.
Also most companies have a backoffice that should provide access to the information and possibility to apply actions to the state. So just giving access to this information in a more limited way that mimicks the customer's view is a no brainer for early-stage.
* Small company; two-sided marketplace with some hundreds of thousands on one side, and some tens of thousands on the other, so rather small scale.
It's just horrible to investigate a customer issue and is a frustrating experience both for the devs and the customers.
"Hello, <customer>, do you mind doing the exact same operation you were complaining about right now?", then it's shifting through microservice logging hell.
#1. Customer reported problems/desires, with a focus on a particular customer's pain, their viewpoint on solutions, etc.
A long back log of #1 customer pain points makes sense to me, especially if you can link new ones to related old ones, and review them in order of urgency, recency, magnitude of customer pain and number of related incidents.
Groups of related pain points could even be triaged as rationale's for potential new adjacent products, as apposed to potential feature work. Without getting into any design work.
#2. Feature work tickets, filled out with rich info on what wide-spread or critical customer problem has been selected to be solved. How it fits into the business plan, product plan, development schedule and design plan etc.
This seems like a list that should be kept relatively small. A backlog of #2 feature to-do's suggest a lot of wasted initial/pre design work, on problems that never reached a critical level or product fit. Also designs without serious development plans quickly turn into useless busy work.
This separation also allows list #1 to be managed by customer relationship roles, and list #2 by product planning roles. With coordination as problems get prioritized for product planning treatment.
--
Spit balling and looking for feedback! I have not encountered this problem before, but am scaling up staff for a product suite with many more potential directions than we could built out without ruthlessly organized prioritization.
When people can't even make features that make sense to users, generating ideas to go in the backlog is a terrible idea.
If you want to understand your users problems and how your product addresses them properly I cannot recommend the book Mental Models by Indi Young enough: https://www.amazon.com/Mental-Models-Aligning-Strategy-Behav...
Both can be good, depending on what you wish. If you want to organically and steadily grow the company, you choose the former. If you want venture capital money and cash out fast, you choose the later.
Not sure what he means by low level reusability? Instead of writing a high level component top to bottom, let's say a dialog, write some primitives that can be combined to build that high level component and others?
At a certain scale it needs to go away though, or be very tightly controlled/audited like you say.
Not just the ACLs you've established internally for which personnel can do what, but ensuring that you have express permission of that particular customer to be looking at that particular data of theirs, on that occasion.
On one hand, I have a backoffice that gives access to the staff to everything related to the customers, but on the other hand I'm supposed to believe that being able to 'impersonate' a customer is this highway to hell? As long as an audit log is held about who did what, this can go a long way forward.
Are you Series A? B? A decacorn? Sure, implement whatever distributed tracing dashboard analytics data lake serverless microservice monitoring solution you can dream of. But this is good advice for early stage.
- Looks at github... :-/
But if you go back to original Agile Manifesto, this article is really hitting a lot of the key principles.
Seems like Agile, or MVP, anything that someone writes down as some really valid bullet points, then grows into some consulting industry that ruins it.
If something is going in, something's coming out, either to a sprint or deleted. And we can call it "Burn It To Ground" never to be brought up again. Or some process like that.
You know, this is an observation I've made sub-consciously a thousand times and I really appreciate you putting that into words. In my experience, frequent check-backs with the customer -- and planning those "detours of development to make something visible" or "prioritizing this with a fake back-end to make something visible" (which we may do anyway if we have front-end/back-end working simultaneously) -- I find the most useful parts of working this way have nothing to do with the usual Agile Suspects[1].
My observations come from the lens of being employed by a few companies where I was a Senior developer with one or two Mid/Junior levels designing all or a major part of a product for a third-party (usually IoT or fully digital product someone paid us to design/develop). This may not apply to "I make a SaaS product I sell to customers" because of the closeness of working with them, but some of them likely do:
(1) Willingness of the customer to engage in regular communication is (anecdotally -- my observed) biggest factor to success. Ya know what, sometimes it doesn't even matter, either. I did three jobs for an oil company everybody's heard of, paid for via consulting dollars given to them by Microsoft. They met with us about five times for three products with the vision being to roll out one of them to "every employee except for the clerks" and the rest back-end to the complete appropriate environment. All three were met with praise and brought us future work. Not one of the three was deployed beyond test capacity (of the something-ungodly-like "150,000 user licenses" they paid for, we logged all of ten, it was sad).
(2) If they are unwilling, or you suspect they are dis-engaged, meet with "your contact over at the company" and have an off-the-record (but you know it's not) careful conversation to discover the reason for the dis-engagement. We were straight-up told "Oh, no, they do this all the time. They have to spend the consulting dollars." Emphasis provided verbally but never explained. It was said as though it could have been finished off with "or they will be fired" -- despite that being absurd; that was the level of intensity. Sometimes, though, you find out that you're not listening (or in a larger forum, they're not willing to tell you what the elephant in the room is) and it might be the only opportunity you have to fix it. A past product I botched pretty well had this issue. I had a customer become disengaged on a product I was working -- up to the point of actually shopping the product to another company half-way through. There's a lot lost in the detail, but it came down to them paying us to develop a product with the assumption that they'd have total control over the layout of the data in the database (despite not mentioning it, nor it being specified in any useful way in our contract). Unfortunately, the product had about one way it could be laid out in the database -- and it wasn't a way this unsophisticated customer would be able to just run a few simple SELECT queries on. Or, that's what I thought the problem was. Once I rang up their "one terribly abused IT guy", and exchanged a few war stories from my work history, I said "What's up with Tom trying to force this ridiculous schema down our throat?" He laughed and said "The money they budgeted for the project covers everything except the admin tool and Tom knows he'll never grasp your Schema well enough to write it himself[2]." They'd been telling us they'd pay a second invoice for the Administration tool and this was confirmation that was out, which I'd already scoffed at but this was absolute confirmation. I wish I could say we did the right thing with the information and it all worked out but, unfortunately (and this was out of my control), nothing was done with it.
(3) Frequent communication leads to higher engagement which leads to the feeling of partial ownership. I saw this at two shops -- one where we had almost no competitors developing against the APIs we were experts at -- we were expensive, so customers became invested in the success of the final product. They were engaged because "if it failed, they might not have jobs." But the other shop was unique. We provided a service that, on the surface, looks a lot like a whole lot of other things, including things sold by Microsoft. But the product was designed targeting three, specific, narrow verticals. These verticals, for at least two huge brands, had nothing that really met their needs so our pitch was "we'll make a version of our product designed for your company with you." And we did. We responded to every little detail asked down to ensuring the buttons/other things looked like every other internal web app they used, branded in a manner that completely hid that it was a SaaS product we sold with another name or that there was another company involved. But we knew every single big brand had a department that does this even if they hire a whole other company to manage the majority of it. So we ensured the contracts allowed for us to re-brand and re-sell it to every one of their competitors (and because of the product, it ends up benefiting them if the other company is as successful as they are). Fast forward a few years, and those two companies have been purchased three times over, almost ended up becoming one, and each time the question of "Should we get rid of Super-App?" wasn't just "No", but met with stories from the department head of "How it was a short meeting after they heard from me and (the three other folks we built it with)." They continue to pay the yearly renewals and work with us to develop out more things every few months. We've re-done this with the other verticals to the point of "we've captured all of the low hanging fruit in that vertical" to "we are a very competitive product in that vertical" but where we're deployed, our customers love the product. Those early customers feel actual loss when people talk about replacing it because they feel like they had a part in its creation.
(4) Frequent communication means you actually understand what your customers are saying. The biggest problem with talking to customers is they ask for things that have nothing to do with what they want to accomplish. "Can I save that to a PDF file?" "Why?" "I want to print it in a consistent format?" "I can put a print button on the page that will do that?" Customers get hung up on specifics of what they want driven by limits of what they believe things can do. If I had a dollar for every time a customer was surprised it was easy to do something "that, I'm fairly certain I could make work in IE 6 with a little effort." That's not a "har har, the customer iz stupiDZ", it's "that's not their job", "that's mine." But they're trying to translate a little of what they're asking into "Software Development Shop" because to explain it to you in their language, you better know a whole lot about manufacturing headlight bulbs (or whatever), and you're trying to translate everything into headlight bulbs. Over time you understand each other's languages and know when to say "... wait a minute, are you trying to ...?" with results that work out a little better than Clippy.
[0] Let's pick the topics: "Anything in the category of project management, scrum, agile", and the "Customer's Don't Know What They Want" (aka Listening to the Customer Results in Features They Don't Use(tm)), the "Startup Advice Doesn't Apply Outside of Startups" ... but that has nothing to do with what I wanted to say.
[1] Building the product this way ensures that the customer and you are on the same page with what the end result should actually be. Nothing explains how it's going to work than a UI that kind of does some of the things the working thing will do.
[2] I did inquire why "internal IT Dynamo", who I knew was a programmer from outside circles, wasn't being forced to write it. He said "I told them I'd quit." It sucked because I was so proud of this design -- it performed brilliantly despite being 6th normal form for 3 tables and it actually had to be 6th normal form to do what I was trying to do. Even better, because it's such a bitch working with that data, I put instead of update/insert/delete triggers on a set of views so "if you wanted to screw with it in SQL Management Studio", go ahead, all enforced by simple constraints, 1/4 the code that was ultimately required by the customer's insisted-upon design but -- importantly -- didn't grow table data exponentially for every configuration added resulting in the whole system falling down after setting up about ten configurable products.
There are countless other articles claiming that customers don't know what they want. And that once you implement a feature they asked for, they aren't going to use it.
So, which way is it?