"I either have to do some hacky position:absolute things…"

Why do you think this is "hacky"? This is precisely what absolute positioning is for.