I agree with your call to study logic. There is something systematically empirical about logic & epistemological philosophy[1].
Over the years,I have read a bunch of critical thinking books and have found myself wanting something at a lower level of abstraction. A recommendation based on recent reading would be "An Enquiry Concerning Human Understanding"[2]
[1]https://plato.stanford.edu/entries/epistemology/
[2]https://en.wikipedia.org/wiki/An_Enquiry_Concerning_Human_Un...