Most projects have some good idea buried within them. Often instead of giving up you can figure out what the good idea is, and refocus on that. But there are cases where you should give up on an idea: for example, when some external change has made it obsolete.
It's harder to say when to give up on the people. Probably some people are not meant to be startup founders-- either because they lack energy, or because they're hopelessly impractical, or because their minds are more suited to other work. It's hard to judge this for oneself. Sometimes it's even hard for me as a neutral observer with a lot of practice to judge.
In practice most dying startups get garbage-collected by money constraints. If you can't convince customers or investors to give you money, eventually you have to give up and get a job. It may be that test is the best we can hope for.
I'm not saying you throw it out, just switch the majority of your effort elsewhere. Keep your demo/beta or whatever running, and if people start using it, you'll see that and can move back to it. Further investment is all you should worry about.
- you have no more passion (go find a more interesting project) - you run out of money (go work for a few months and save up some seed capital, and find a more interesting project) - you become seriously unbalanced (emotionally, spiritually) for prolonged period of time (say, 1year). - you have serious health problems
You, as the entrepreneur, better be damned excited about your idea and your team and your product. So excited, in fact, that you talk about it all the damned time to everyone. Of course, talking is rare...you don't talk to many people during that phase, because you're busy working. But nonetheless, if your girlfriend isn't sick of hearing about what you're working on, then you might not be passionate enough about it to pull it off.
Death of a startup is due to cumulative losses, not one factor. If you'd like to build a better scenario I'll throw some change in. ;)
Might want the check out "The Dip: Knowing when to Quit and When to Stick" by Seth Godin...
One short measure you can decide on is uptake. If there is little or no uptake you know you are in trouble. But I agree there is a lack of information. Startups are designed to stumble all the time. Is keiretsus` startup stumbling? Is the lack of a compelling idea holding it back? Is the product something that "customers want"?
Is it an Artix? ~ http://www.paulgraham.com/bronze.html