At least here in Portugal, it means you still can't sell or buy it. It's just not a crime to possess small amounts or smoke it (I'm not sure if it's still punishable with a fine or not).
It's still a good step, since it doesn't get you a criminal record nor it clogs the courts, but it's not anything near to actually legalizing and regulating the sale and consumption.
Short version? The penalty for a violation is a fine (like a parking ticket) and not a jail term.
It still illegal in order to comply with UN and other international treaties, but in practice it's not enforced.
A parking violation isn't a crime, either.
The terminology is fuzzy, for sure. Here's my perspective: the Netherlands is a good example of the difference. It's not a crime to buy, sell or use marijuana there, but it is illegal to produce it. Marijuana is thus "decriminalized, but not legalized".