I understand it as that too. But even then, I dislike it.
Yes, "shipped, but terrible software" is far more valuable than "perfect software that's not available". But that's a goal.
The "move fast and break things" is a means to that goal. One of many ways to achieve this. In this, "move fast" is evident: who would ever want to move slowly if moving fast has the exact same trade-offs?
"Break things" is the part that I truly dislike. No. We don't "break things" for our users, our colleagues or our future-selves (ie tech debt).
In other situations, the "move fast and break things" implies a preferred leaning towards low-quality in the famous "Speed / Cost / Quality Trade-off". Where, by decreasing quality, we gain speed (while keeping cost the same?).
This fallacy has long been debunked, with actual research and data to back it up¹: Software Engineering projects that focus on quality, actually gain speed! Even without reading research or papers, this makes sense: we all know how much time is wasted on fixing regressions, muddling through technical debt, rewriting unreadable/-manageable/-extensible code etc. A clean, well-maintained, neatly tested, up-to-date codebase allows one to add features much faster than that horrible ball of mud that accumulated hacks and debt and bugs for decades.
¹e.g. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations is a nice starting point with lots of references to research and data on this.