Indeed that’s a famous letter but his implication is that his constitutional authority was to the union and as slavery was the driver of the war, the result was clear. A clever formulation, for a lawyer, designed to maintain maximum legitimacy for his actions.
His opinions, however, were clear, and that is part of why he was the republican party’s first nominee.