Firstly, all of your experiences are affected by your consumption of external substances. Even hydration affects experience, as does sugar and cornsyrup and the fumes from your automobile.
My real issue is that your perceptions of reality are a construct shaped by evolution and culture, and while they probabalistically correspond to aspects of reality that is also true in a drugged up state; the difference is mostly quantitative, albeit with such difference in quantity as to appear qualitative. We're patching together fragments of sensory input with processes that fill in the blanks and delay event streams so that things appear to happen in orders that are useful for us to react to.
We delay our perception of light to better match our perception of sound, so that people talking have lip movements that correspond to the sounds they're speaking in the moment. This applies to other events as well, and researchers have hacked the percieved casaulity of the event stream through conditioning; first the subject repeatedly presses a button and hears a sound a moment later that is offset by an artificial delay. After the subject is accustomed to this, the artificial delay is removed and the subject experiences the sound before they press the button.
As a side note it obviously isn't clever to have all visual input delayed, which is why your brainstem gets a separate stream of visual data it can respond to quickly. When the neural pathway that feeds data into your percieved consciousness is damaged, your brainstem can keep responding to data and catching objects that "you" can't see or speak about -- this condition is known as blindsight.
Your eyes are much less sensitive to color in your peripheral vision than they are to colors in your central field of view, yet you don't experience that color differently. By the time you experience color, the values in your periphery have been "corrected" to match more reliable data. There is noise in that process, and guesstimation.
If I experience the sky as a different color than you do, is one of us experiencing the "true" color of the sky? Does consensus reality have anything to do with truth, other than by usually massaging out some errors by looking at averages?
Moreover, can a color even be true? A color corresponds to some frequency of a light particle, or whatever the fuck a light particle really is in the fundamentally unknowable base reality that our conscious processes are implemented on top of, but a color is not that vibration or that particle, a color is an experience. [Edit: i.e. a representation; do not mistake the pointing finger for the moon, nor the pointer for the register.]
I'm not claiming that sober experiences don't generally correspond more to base reality than do experiences under the influence of entheogens (I only feel the need to make this explicit because some folks in the same drug fan clubs I'm in do make such claims), but they still aren't one-to-one. You aren't experiencing reality, only a useful model of reality. After a standard disclaimer that all statements about reality are implicitly probabalistic and unprovable (including that one) we might venture that some experiences are more or less "true" than others depending on how much we think they correspond to reality, but none of our experiences are strictly true in this sense. The only way I can see arguing for a strictly true human experience is to judge the experience entirely apart from reality; if the phenomenon was experienced it was true in this sense, but obviously we're talking about subjective rather than objective truth at that point and drugged-truth is just as valid as sober-truth. Words are mushy.
If Alice is on drugs and doesn't see a ghost, and Bob is sober but hallucinating a ghost while his brain is in a "natural" state, is Bob seeing what is true while Alice is not? If you think this example is utterly ridiculous and nobody has ever hallucinated something on that scale while sober, I would check the pop-sci book The Man Who Mistook His Wife for a Hat by Oliver Sacks. Sober perceptions can be super trippy, even if they aren't for most people most of the time.