I can’t tell from the screenshot - if you don’t apply colors to your tasks, are the task items even shorter height or does it always preserve height for the color bars?
Yeah,cards get shorter if there are no labels. Heights are also added by due dates and other attributes in addition to card title length itself.