Marijuana is the dried leaves and flowers of the Cannabis sativa or Cannabis indica plant. I cannot imagine how the term is racist. But the fact thay it makes you think of 420, that’s just you.
https://en.wikipedia.org/wiki/Marijuana_(word)#Early_use_of_...
It's definitely suspicious at least. If you really want you can view it as more cynically exploiting existing prejudice than prejudiced in and of itself, but I am not sure that is much of an exoneration.
We're all aware of previously uncontroversial words that are taboo today. The reverse can happen as well.