I think it sounds odd because it's disconnected from its preposition. In fact, I'm not sure I've ever heard a sentence starting with "whom."