Domain knowledge or a lack thereof (2013) | Hacker News Reader

Domain knowledge or a lack thereof (2013)

jacquesmattheij.com

105 pointsggr234230 comments

The worst in my days were the very large projects. Besides many other problems. The system's funcional design was written up by analysts talking to the client's middle managers. Neither of those groups had a grasp on how things realy got done on the workfloor beyond the most basic sunny day scenarios. Actual developers and actual users were not allowed to talk to each other as many times the new system would obviously involve some redundancy/displacements in the workforce which needed to be kept under covers and some completely made up at salrs pre-existing system capabilities of the supplier such conversations would 'leak'.

Needless to say these projects typically crashed and burned irrecoverably at first contact with production.

Very first software project I ever worked on professionally gave me a great opportunity to learn this lesson - that managers can not tell you how a task is actually done; you must go and look at how it is actually performed.

I was tasked with automating a billing process which, I was told by the boss, just involved copying some data from one database over to another database.

Come to look at the actual databases. One of them is denominated in US Dollars, the other in Pounds Sterling.

Boss says ‘I’m sure you can just use the current exchange rate’. I decide to go and check how the manual process is currently done.

When I ask the person who has been manually carrying out the task what exchange rate they use, they say ‘I take the current rate each morning and add a bit to be safe, because the actual charge will go out in a few days and the rate might change so I don’t want to risk the actual amount billed being higher that day’ (I can’t remember the exact reasoning but it made some kind of sense given the state of international electronic banking at the time).

How much is ‘a bit?’ It varies depending on how volatile the currency was looking. They were basically hedging. Boss had no idea that the manual process involved this human judgement step.

Taught me to always go and investigate in detail how the manual process is carried out. Where’s the bit where a human being is ‘adding a bit’?

Ask "dumb" questions.

The best lesson I have learnt working in product development (both software and physical) is to ask "dumb" questions. The vast majority of the time they aren't dumb. What seems to you, on fist pass, to be a dumb question is almost certainly an insightful one. The person, your customer/client, will be pleased to hear you asking it.

Every answer to a question is a jigsaw piece, ask a question that fills in the next gap until you have the full picture. Each answer gives you a new edge in the picture to ask a question about.

Sometimes the "expert", your customer, who you are asking won't know, and that's just as valuable as it gives you insight into them and the new field you are working in.

Don't stop asking questions, start with them, then Google and read anything and everything you can.

An enormous part of new product development is learning about the field, then applying your learnt knowledge from elsewhere to it.

I can still hear the words of my first IT manager from 15 years ago, in his French Israeli accent: "I can tell how smart someone is by the questions they ask."

Wasn't quite sold on his assertion back then. Now, I cannot agree more. Doesn't apply to just tech, but all facets of life.

Thats how I hire :)

I tell them what we do and listen to questions. If there are any ofc.

I think this needs to be caveated. There are "dumb" questions and then there are actually dumb questions.

A "dumb" question that challenges some implicit assumption that nobody actually has addressed but maybe some people were also wondering is good. Telling people to ask "dumb" questions IME empowers people who can't differentiate between that and something easily googleable or that belies a fundamental lack of knowledge to ask time-wasting actually-dumb questions. You don't want that.

Domain knowledge is also the thing upper management conveniently ignores when it comes to fire and hire (I intentionally changed the order ;)), off-shoring or we-do-projects-but-are-not-a-software-shop ignorance practices. The little percentage in money you gain, you dramatically loose when delivering on this fresh people.

If you have a domain competent group, your requirements/stories/tasks are faster written, better developed, better tested and quicker delivered. In addition to hitting the market need. If not, you will see money burning.

Unless the market is big, most employers give zero incentive to accumulate domain knowledge. Between learning the new hot framework and accruing domain knowledge, one is clearly better for the average dev's career.

The advice is great in a world where people remotely cared about good software and are willing to pay for it. Not such great advice for reality.

yup. theres way too much stock put on "best practices" driven by the biggest corps as if whatever they're doing must be good.

Agile and MVP development is my favourite example. "race as fast as possible with no documentation to build the shittiest piece of software that bamboozles your customers into continuing to pay you because otherwise they wouldnt have the patience to wait for good engineering"

It "works" only in a market client-vendor relationship, but the "works" part has nothing to do with leading towards good software development.

Any employer can increase the amount of domain knowledge in their software development teams at any time. All it takes it giving them the time of the employees who use the product. Ergo, the actual problem is that is too expensive for most profit driven organizations.

In my experience software engineers aren’t trained on the domain nearly enough during onboarding. Deep domain knowledge is critical in every part of the software design process.

This is a reason why I like to use Python for more things than is strictly necessary. There's a ton of domain experts writing Python for their non-SWE jobs. Contrast with e.g. JS and Ruby which are more SWE/webdev-only languages.

I like to think it makes knowledge transfer at least a little bit easier across the user-developer membrane.

You should try VBScript, Access and Excel, true troves of shadow IT lie hidden on those shores.

Seems like it should be completely obvious, but when I worked at $SOCIAL_MEDIA_CO there were several Engineers who refused to use the product but claimed this had no effect on their work.

Sometimes it is. Sometimes it isn't. I figured that out after spending some time writing software for a highly domain-heavy software company - think half the employees are physics or chemistry PhDs, top brass is old-school classical engineering but also familiar with modern software methodologies.

Since I was building components primarily doing data exchange or other generic software work, unrelated to the domains the company was targeting, I thought I could get away with minimum, surface-level domain knowledge. And technically, this was true - my work never really required understanding any of the complex non-software stuff the PhDs were discussing. But on a more fuzzy, emotional level, it was a different story. Being unable to understand the point of half of the conversations I hear, or even keep track of the acronyms and terminology used, was very draining and demoralizing. And sometimes it would, indeed, make me guess badly about which parts of my work were most important, as I didn't fully know the path connecting my purely software work in the bowels of our product, to the value the software generated for the customers.

So my advice is, domain knowledge matters even if you're separated from it by half a dozen layers of abstraction and only doing pure computer stuff.

drug dealers shall not get high on their own supply.

(2013) with a discussion at the time: https://news.ycombinator.com/item?id=5263486 (110 comments)

This person appeared to make hardware manufacturing applications (like CNC machines). In my experience with software like CRM there's a nuance to which domain knowledge is important (but I presume also financial software, and to a lesser extent ERP).

My experience is more with CRM. In these cases, domain knowledge is still important, but it's more about the curiosities around how a specific sales/marketing/customer support team is structured - how do they divide up leads? Do they compete with each other within the team or are bonuses given to teams? Usually these decisions are informed by domain knowledge. E.g. the customer support of a boutique expensive watch brand would be more of a concierge service (with tickets requiring immediate attention - a delay in response could represent a big loss in sales), while for a mass manufacture cheaper line it would have more of a volume of tickets and human response times might differ. Escalations would therefore need to be programmed differently. To give another example, I can imagine seasonality having a huge impact on how a financial team want their software programmed, and that is informed by domain knowledge.

But knowing how watches are made wouldn't help me with understanding 'domain knowledge'. I need to understand how they are sold. The domain that I need to understand is 'how does this sales team work' not 'how does this product work'.

I guess what I'm saying is the right kind of domain knowledge to match the task.

I'd argue domain knowledge becomes less and less important as the size of the organization grows. Eventually, it gets to the point where what you are working on is just a tiny piece of the overall work, it just doesn't matter. Also, at that larger size, your input doesn't matter very much either, that's what the BA team is there for.

The best organizations will work to leverage the productivity of individual devs over just throwing bodies at the problem, but that doesn't happen everywhere.

Domain knowledge should be available in the team, not necessarily every member of the team. Together with good communication this solves the issue on a 80/20 basis.

The exception is of course if you work solo, in which case it does indeed become a full stack problem of sorts. Individuals who can pull this of can make significant impact.

I agree with the article, domain knowledge is important and under-appreciated. It's strange that companies hire SW engineers mostly on knowledge of the tools rather than understanding the problem domain. It's a wrong Taylorist idea that programmers should focus on how and not on why.

> I believe that a lack of domain knowledge is the root cause of a lot of very bad software that gets developed and I think that it is up to computer programmers and their managers to deal with this.

This is probably the best thesis statement I have seen regarding S/W engineering in a long time.

At IKEA, we have something called "Front Days" that non-retail co-workers can sign up and work in the store for a few days. I did my first one three weeks ago, and it was a great way to get out from a desk and see the reality of what we're all working towards.

Every major retailer from McD to Starbucks does this.

While it would be ideal if all the software devs writing say a General Ledger application were accountants, the reality is that will not be the case.

I call domains where expertise lies outside of the software development team - “external domains”.

For these, programmers, testers and managers need additional skills in requirements elicitation and capture, and more importantly skills in modeling a system (real or imagined) based on those requirements.

In a software world dominated by technical requirements, how to determine and model functional requirements in an external domain can be what makes a programmer a great one.

> While it would be ideal if all the software devs writing say a General Ledger application were accountants, the reality is that will not be the case.

I don't think this is necessary at all. However, to do a good job here, it's critical that the people building the software understand the principles of how the domain works and how people interact with it.

To take this literal example: I'm not an accountant or bookkeeper but I've spent 15 years of my career working with people in this domain. Even when I encounter a new part of the domain I'm not familiar with, I find it easy to put together the need from my existing knowledge and figure out what we need to do.

Take an example where I didn't invest in understanding a (financial) domain: my team inherited a report generation script that a founder was running for Finance. Initially, we just tried to expand it to handle more transaction items. Because we didn't understand how this report was being used, we skipped issues, caused issues when we changed things elsewhere, etc. Our Finance team was pissed because we were making their lives harder with our total lack of understanding. The answer? Spend time with the Finance team making sure every engineer understood what this report contained, what it was being used for, why it was important.

> In a software world dominated by technical requirements, how to determine and model functional requirements in an external domain can be what makes a programmer a great one.

I agree here, I just think that this understanding needs to go a little deeper than just modelling the domain. I feel it's important for a team that works in a domain to really get it. I think this looks like being able to explain the core of the domain and how it works in practice to a layperson.

From a selfish dev side learning too much domain will lock you into to a company.

I prefer not to have any vendor lock in so I focus my skills on the parts that are transferable to other companies.

Domain knowledge to me is primarily a BA responsibility. The bits that a dev needs to know are converting that into the implementation.

FYI this also probably comes from someone who hasn't had a very niche and difficult domain. The most complex domains I've worked in would have required decades of training and experience to understand. My job was to know enough to to my job well.

100% agree.

And some domains are extremely complex, requiring an extensive learning process. This is why large consulting-led projects fail, because there isn’t enough time taken to truly understand the domain.

I loved your idea. We have to be experts in our own field. The person who tries to learn everything actually learns nothing.

But don’t fall into the trap, as a developer, of mistaking your hastily acquired superficial domain knowledge for actual expertise.

One particular Dunning Kruger trap that devs can fall into is fixating on the most complex parts of the domain that they understand in any depth, and the edge cases that complexity suggests, at the expense of addressing the concerns of the actual experts, that are most relevant to the task at hand.

This can lead to software centering domain concepts that are esoteric and obscure at the expense of the concepts needed to meet the actual requirements. Or embedding rules that to experts are more like guidelines.

Like: it’s great that you’ve developed a deep interest in and love for heraldry, the system of colors and tinctures, and the way flags can be described using a blazon - but it’s probably okay to just have an SVG url for an image of the flag in the database, not a full blown heraldic DSL.

Reply on news.ycombinator.com