How long is "long enough" to deem that a line of experimentation "didn't work out?" There is no period long enough. Sometimes a negative result is quite useful down the road. Some research just comes together when the right things are learned and tried. So any notion of a "dead end" is really just a tentative assessment, frozen in time.
It sounds like you're saying "don't bother exploring dead ends, either do it right the first time or give up," which is a lot like saying "I just want a penthouse apartment, stop wasting time building a foundation."
Not really. The point of big scientific research is that at the moment a project is being conceived, no one has any idea where it will go and whether it will bring anything useful. Yet in time, it always brings breakthroughs.
I'll put it bluntly: we need more trust. The right way to do science is to keep throwing money at it, no questions asked[0]. IMO the reason we've seen so much progress during last two wars was not because war per se stimulates research, but because during a conflict the military has infinite budget and it starts throwing more money at research that can be used, so scientists can do whatever they want without justifying it for general public[1]. Free of money&management issues, creative minds could focus on doing fundamental research in whatever they felt like doing[2].
Come to think of it, science is best done in the same conditions the programming (or any creative endeavour) is - when you're free to think, don't feel the pressure of looming deadlines, don't have to deal with managerial bullshit, and when you have enough money to not have to think about it (both in life and in cost of work materials).
So yes, I advocate some kind of almost blind trust. Just throw some part of taxpayer money at R&D, no questions asked. It's a long-time investment.
[0] - Of course there's a need to build in a system to limit the amount of parasites who will feed on free money; but they ultimately cannot be avoided, we can only try to limit their numbers.
[1] - I think it was Feynman who pointed out that the problem with science in the 90's was that scientists suddenly found themselves accountable to general public and struggled with explaining why their research was useful.
[2] - If that last paragraph rings bad to you, please note that I'm skipping over a lot of points related to creativity needing to be free of trivial bullshit, intrinsic motivation beating monetary rewards/prestige in performance, etc.
Look at the Large Hadron Collider. Do you suppose they scrub from their data stores all tests that fail to produce a "working" result? Are you suggesting that trying and failing produces irrelevant knowledge?
Why?