Is the encoder style arch better for representing classification tasks at a given compute budget than a causal LM?
Is this because the final represention in bert style models more globally focused, rather than being optimized for next token prediction?