The main questions I'm curious to answer are (1) why do so many good-seeming ideas fail? and (2) what should we do about it? So I'm curious to know y'alls answers on those questions too.
I don't mean that in a hostile way, I just mean that the overlap between reality and good ideas is very low almost by necessity? This is why we have science, after all.
In fact, I think the idea that people who stumble on things that work somehow do so because they're smarter or more talented than those who don't is somewhat dangerous. Certainly, bad ideas are bad. But at some point you reach a set of ideas that are all equally probable given existing information, and you're literally at the mercy of luck, almost by definition. Maybe there's some bit of information here or there that can tilt the scales one way or another, and maybe some are better at that than others, or just have access to additional information through experience or resources, but at some point you are just left with the need for raw experimentation.
I think the only way to do this really is through as much information as possible, which is why I think transparency in data is key -- so you can make use of what others have done already as much as possible.
Maybe there's some way of bootstrapping to a sort of meta-modeling? What I mean is, maybe you can develop a model/AI system that learns how to learn and becomes sort of auto-developing, but it seems like we're not quite there yet.
Yeah I guess in any field the low-hanging fruit will be yanked. One situation we could end up in is where we feel like we have more possibilities but counter-intuitively they don't work (otherwise, by definition I guess we could say, they would be low-hanging fruit that had already been yanked). This is the situation you may be describing. Let's call this situation A.
Alternatively, we could end up in a place where there's no more low-hanging fruit and we also can't even come up with ideas that seem promising. I guess at that point the field will just kind of die and people will move on to other pursuits. Let's call this situation B.
I guess a field should hit A at some point and may continue to progress to B eventually or splinter into sub-fields.
Yeah so from a very meta perspective I guess I should expect that things that I expect to work should fail. But at the same time I can't ignore the fact that I honestly felt surprised for at least a few of my ideas that they failed (others felt more like a gamble so it's no surprise they failed). So in some sense I should expect to be surprised but I am still trying to get to the bottom of why I felt surprised in the individual situations...
Your point does raise some other psychological questions. Like maybe we're wired to delusionally believe our likely-to-fail ideas are likely-to-succeed. If we get rid of those delusional beliefs, will we become unmotivated to try new ideas, or will we just seek out actually promising ideas?
I think the problem is really that people (in ML especially) come up with solutions before they've properly framed the problem that the solution is trying to solve. We should be finding solutions to actual problems, observed by looking at the data and past model performance on that data.
And once we've got a good problem, there's usually more than one way to address it. Your initial 'good idea' is typically just one, and often not the easiest to implement or evaluate.
So, when faced with an observable problem, I formulate a number of possible causes (hypotheses), and then list two or three ways that I might solve the problem if that hypothesis is correct. I then pick the /simplest/ solution for each hypothesis, implement them, and see how each one does. Some will have more effect than others; this helps update my prior on which hypothesis might really be driving the problem, at which point I might consider revisting some of the more complicated solutions. Assuming it hasn't already been fixed!