It's interesting that this misconception made it into a clearly knowledgeable article. Containers have state on the writeable layer that is persisted between container stops and starts.
I usually start mine with —-rm so they’re removed on shutdown.
I’ve seen people apply security updates via ‘apt update; apt upgrade’ within a running container. Guess what happens when that container is eventually destroyed?
Honestly I think there is a lot to be said for making the writable layer of a container read-only. It makes sure that things like logging, if you care about them, go somewhere safe, or if you don't, get turned off explicitly. And also prevents gotchas like "oops wrote important data to /var/lib/notavolume when I meant to write to /var/lib/therightvolume" that show up at the worst times.