There were enough passengers, and flying the Concorde actually became profitable for the airlines once they figured out they just needed to charge through the nose for it. This was despite prodigious fuel consumption and that fuel becoming much more expensive after the oil crisis.

The main problems were that the requirement to only fly supersonic over water massively limited the possible routes it could fly, and that actually flying in a Concorde was not very comfortable (cramped, tiny windows, hot, vibration etc). Boom promises to tackle both of these, which will open it up to far more routes.