In practice I update when there is something I want in a more recent release. This happens more often with nightly because there are more bugs to be fixed and feature changes, but it isn't that often.
Anyways, that's not an exhaustive list. Not wanting to deal with rustup might be another reason so you use your distro rustc packages instead.
1. Search the bug tracker for the message/symptoms. If you find the bug (more likely on older nightly's) stop here.
2. Create a minimal test case by creating a copy of the code and then deleting large portions of it until you have a dozen or so lines and no dependencies that reproduce it and are all needed to reproduce it.
2.5. If using an old version of rustc, `rustup override set nightly` to use a recent nightly on this directory and see if the bug is gone/changed. This takes about 30 seconds (compilation is instantaneous since I've already deleted all but a dozen lines of code).
3. Search the bug tracker again now that we have a better idea what is going on.
4. If there really is nothing, report it.
The time savings by bailing out early at 1. greatly exceed the time cost introduced by 2.5.
If you're using nightly you should be using rustup... but since rustup is so easy to use that's a very very low cost.
I never experience that on stable, and it’s been my primary reason for not recommending Rocket for so long, even though that it’s an excellent library in all other regards. (I believe it’s working with stable in the master branch at this point, and just waiting on a release of 0.5).