We can take "safe" as meaning "gets you high, and doesn't kill you" not as "absent from all conceivable known and unknown long-term health effects". Also, "safe" can simply be taken to mean as "the sought-after mushroom, properly identified" (believed to the user to be safe) and not some other, unknown mushroom mistaken for it.
The risk of using the wrong item is separate from the risks inherent in the genuine item; in that context, it makes sense to use "safe" for the genuine item as a rhetoric device.
Of course, none of these mushrooms are safe; there are unsafe doses of their ingredient.
Nothing is safe without reference to dose; not water, not salt, ...