Mom's in the medical field. She attended a seminar led by a psychologist/psychiatrist husband/wife team. The husband found, as an adult, that he did benefit from the medication (Adderall), but at a much lower dose than is considered typical. 2.5 mg, IIRC.
Apparently, some people are more sensitive -- or, whatever term you want to use -- to the medication; for them, a "standard" dose is too much.
If you still have some tablets left and are unsure how your doctor might receive such a notion, you could use a pill cutter to create your own lower dose. If/As you've already been prescribed a higher dose, and given that the medication is not essential to life function and you've already safely been able to discontinue that higher dose, I don't see how trying a lower dose would present any additional risk.
However, I'm not a doctor, you should talk to your doctor, etc. Just... some of them seem to be pretty narrow-minded including about the possible efficacy of lower doses.