Long term amphetamine usage has been shown to potentially be toxic to the brain, especially at high doses. It also increases risk for high blood pressure, heart attack, and renal failure for those with pre-existing conditions or who are at-risk for other reasons: genetic conditions, obesity, etc.
Most of us (myself included) will take long breaks from their medication. I don't take it while on vacation, I take most weekends off of it, etc. I take it when I need to be at my best for work and I take it if I am on vacation and feeling like my emotional reactivity is too high, etc.
Ritalin is actually shown to mitigate the effects of amphetamine toxicity, as well, surprisingly, because while it also work on dopamine it uses a different strategy. Instead of causing your brain to produce it (thus potentially frying the cells that do so), it simply prevents the brain from re-absorbing it, much like an SSRI does with serotonin. So, I've heard of some people switching back and forth between amphetamines and Ritalin, though I have not done so myself.
Lastly, there are other drugs such as Cymbalta and other SSRI/SSNRIs that seem do help some people. SSRIs have a whole slew of side effects (especially weight and sexual side effects), and, while SSNRIs tend to have far fewer side effects (Cymbalta seems to have barely any for most people), they are still a fairly new class of drug comparatively so I could understand someone being cautious.
All that being said, the risks are worth it for me. Even if it took 10 years off my life, I'd rather have the remainder of my life be productive, happy, and have healthy relationships than continue to be the shit show I was at times in the past.
That being said, as I mention in another comment, there are valid reasons to be cautious with ADHD medication. And as for the stigma, I am very careful about who I talk to about my ADHD in real life. A few of my coworkers know, my boss knows, and a couple of my friends and family members. I discovered very quickly that ADHD itself has a pretty bad stigma, let alone the medication. Everyone thinks you just want to get high. So, I just don't talk about it anymore, even though I so often want to explain to someone that the reason I did something rude (like interrupt them 5 times in 10 minutes) is because I'm ADHD and my medication has worn off, ha.