I've done a fair share of both. I think the perception accomplishment was just around the next turn (but you stopped too early) is often an illusion. Refer back to No Free Lunch; a general ability to know "how far" you are from a solution in problem-space you don't know would be a good start on the universally optimal search algorithm (which we agreed doesn't exist).
For a specific example, I've slaved away at game dev (in my free time not professionally) for more than two decades - and never released a game. I did an unreasonably deep dive developing some "technologies" which in the end turned out to be net negative contributors to my overall project.
It turns out there are ideas which you can sink years into only to exhaustively prove they don't work. There are others which appeared to work only because corporations threw many man-years at the task.
--- ON THE OTHER HAND ---
I think the amount of time and effort you put into each project over time has a compounding effect. That is, the person who stops before finding a solution doesn't know whether they stopped 15 minutes away from figuring it out or a lifetime away. As a result, they may be less inclined to believe they could solve the next one and put less effort there, and so on.
The person who puts in just a bit more effort (read: stubbornness) initially may find a serendipitous payoff and be more inclined to believe they have special luck or skill to solve the next problem. (Throughout this comment I've been using the word 'solution' to stand for both actual problem solving and the unspecified 'accomplishment' of the original post, as for the purposes of describing this as a search-optimization problem it's the same.)
A real-world example:
In college I built a single board computer on my own as a fun project. For a display I bought a portable DVD player (back when 7 inch LCDs were difficult to come by cheaply by other means) to harvest the screen from it, but the screen driver board from the cheap electronic device died shortly after the conversion.
If I had money or lab resources I might have just replaced the screen. Having already spent money I didn't have to buy this screen in the first place, and feeling deeply offended by the idea that a thing could just "break" and there's nothing to be done, I started troubleshooting in the dark.
I decoded part numbers and looked up chip datasheets. I noticed there was no power getting to the +5V pins of some chips, and had the audacious idea: what would happen if I force-fed power into the line? So I directly connected +5V power and got... nothing.
Long story short, I wasn't successful until I deduced the existence of and tracked down ALL of the tiny, individual switchmode power supply circuits randomly distributed across the board and soldered tiny wires to inject the missing voltage. In all I had to supply +3.3V, +5V, +12V, -12V, and 300VAC the latter of which I hacked in using a standalone CFL driver sold for PC case modding.
At no point did I ever identify a missing "turn power on" signal, a hierarchy of dependency between the power supply voltages, or any other eureka moment or root cause. The darn thing was just dead and stayed "mostly dead" almost up until I found and connected the last missing voltage. That is to say: I worked on this miserable thing every night for a week or two with little indication I was making progress (and not just silently frying more chips) until one night it all worked.
That's what I mean by my theory about how persistence can separate people into can/can't: at any point before that if I would have stopped I would have thought I couldn't get it to work.
To answer your question how can you tilt your own balance more towards persisting with projects: I can't give you a magic bullet. But I think it's instructive to describe how I feel when I'm being like that:
I'm pissed off! It stopped being fun ages ago. I'm probably slightly dehydrated because I forgot to drink water hours ago. It's cold and I'm up past midnight, and someone is probably mad at me for not going to bed, and I don't care.
I don't think that's necessarily a great way to be all the time. But I just wanted to make clear the kind of motivation for continuing something after the newness has worn off is maybe a different source. Instead of asking how to get more of the initial kind of motivation, perhaps you should look for what it is that will fill in the second part of the bridge for you.