Because labels or dimensions are not stored in as a value but as a row identifier in most implementations. That results in having to scan the entire row space and look at every row name and see if it matches the lookup.

Storing labels in a row based system (like SQL) allows querying by value, not column name which takes advantage of all optimizations and indexes making it a lot faster.

That said there is nothing forbidding someone to do both, DalmatinerDB, for example, uses a column-based format for metric values but a row-based format (PostgreSQL) for dimensions.