Slavery started to fade away through the 1800s. This wasn't because people suddenly realised the slaves weren't having fun, or suddenly after millennia of religion someone decided to take the 'be nice to people' dogma seriously. Neither of those things take centuries to figure out.
The change was the industrial revolution that let relatively ordinary people create enormous amounts of wealth if left with their freedom. Slavery is a great example of liberating people to do things being much more impressive than bans. Banning slavery was a big step that sped the whole process up by a few decades, but the writing was on the wall once it became obvious just how much wealth a motivated free person could create.
> let people do heroin for example. would be very impressive.
Yeah, it would. Countries should have some faith in their people to figure out what a mistake looks like without the police arresting people. I'd applaud any government that can get to that level of enlightenment.