A story is not marked as [flagged] as soon as someone flags it, and flags have influence even if the submission is not marked as [flagged].

See dang's comment in this thread: https://news.ycombinator.com/item?id=13741276