For me, the downsides manifest as drarticlsly increased anxiety (I naturally have next to none), extremely poor sleep - I sleep but the sleep is so low quality it begins to feel like my brain barely works right, and the obvious one, the effects on your motivation. I naturally tend towards ADHD style dopamine chasing, and weed makes that about 100x worse. Instead of getting my work done, I will procrastinate with any number of cheap dopamine hits such as video games, internet sleuthing, etc.