You are correct, there are literal truths. Some person factually used some set of words in a particular order. The part that people care about is how we interpret those words. It being fact does not make the implication incorrect.
In an extreme example, if someone says "There are extremists in the world, and I am not going to be one of them", I can quote that as "There are extremists in the world, and I am ... one of them", which is factually true, that is a portion of what was said. The interpretation is the polar opposite, but it is factually true.
Likewise, there are facts that are misleading, but true. "The Earth is at the center of our solar system". It's a perfectly valid statement; the rotation of planets can be modeled where the Earth is at a fixed point and the other planets moved around it. It's messy, and the movements look extremely erratic, but it's perfectly valid to be modeled that way.
There are also facts that are true, given a set of circumstances. "I weigh 7 pounds". It's not true on Earth, but in the correct gravitational field, I do weigh 7 pounds. Determining whether that statement is true depends on the implication; was I implying that it is true on Earth or not? That depends on the context of that statement, and on how you interpret the context.