The empathy gap, or why good-seeming ideas fail

It gets better with practice as you develop more accurate intuition. Eventually, most new ideas that you try will work, but it takes a lot of failures to get to that level.

Yep I think you hit another nail on the head (likely the bigger nail).

There is a whole 'nother category of ideas I wasn't thinking about which are the ideas that seem promising but just like way too much effort to even test.

Those ideas just get completely lost when it's hard to test ideas.

Maybe using empathy for logical problems is akin to using GPU for computations. Maybe there are some problems it just solves well even though it wasn’t necessarily designed/evolved for that initially. See also people who solve problems in their dreams/subconscious/downtime.

When learning, extending, or fixing software, I generally work by building an understanding of software's principles, implementation, and runtime behavior, so I can reason about the current state and the effects of potential changes. Though sometimes I tweak code to see the effects and better understand the problem (shh we don't talk about tweaking things until they "work on my machine" and we're done).

Understanding runtime behavior is easier in software with high debuggability and transparency (which is orthogonal or even opposed to performance, since logging slows programs down and optimizations impede debugging). Bryan Cantrill's "Platform as a Reflection of Values" talk (https://vimeo.com/230142234) has a list of software values.

I think "observability" is another good word for what I want, but it tends to be thrown around in cloud-scale software rather than the human-scale software I prefer. Perhaps "empathy" is a workable word for developing an intuition (and less so for building a mental model).

To paraphrase the Lady Jessica, perhaps:

All great problems of ML can never be solved; they can only be outgrown.

EDIT: mentat learning>>machine learning??!!

Thanks for the write-up! The "empathize with the model" line is great. :)

I think the problem is really that people (in ML especially) come up with solutions before they've properly framed the problem that the solution is trying to solve. We should be finding solutions to actual problems, observed by looking at the data and past model performance on that data.

And once we've got a good problem, there's usually more than one way to address it. Your initial 'good idea' is typically just one, and often not the easiest to implement or evaluate.

So, when faced with an observable problem, I formulate a number of possible causes (hypotheses), and then list two or three ways that I might solve the problem if that hypothesis is correct. I then pick the /simplest/ solution for each hypothesis, implement them, and see how each one does. Some will have more effect than others; this helps update my prior on which hypothesis might really be driving the problem, at which point I might consider revisting some of the more complicated solutions. Assuming it hasn't already been fixed!

You deserve some credit for raising the question, but in some sense I'm not sure what you'd expect?

I don't mean that in a hostile way, I just mean that the overlap between reality and good ideas is very low almost by necessity? This is why we have science, after all.

In fact, I think the idea that people who stumble on things that work somehow do so because they're smarter or more talented than those who don't is somewhat dangerous. Certainly, bad ideas are bad. But at some point you reach a set of ideas that are all equally probable given existing information, and you're literally at the mercy of luck, almost by definition. Maybe there's some bit of information here or there that can tilt the scales one way or another, and maybe some are better at that than others, or just have access to additional information through experience or resources, but at some point you are just left with the need for raw experimentation.

I think the only way to do this really is through as much information as possible, which is why I think transparency in data is key -- so you can make use of what others have done already as much as possible.

Maybe there's some way of bootstrapping to a sort of meta-modeling? What I mean is, maybe you can develop a model/AI system that learns how to learn and becomes sort of auto-developing, but it seems like we're not quite there yet.