The speed limit itself is a separate convention and regulation. In some places you can be cited for obstructing traffic by going the speed limit in the passing lane if you are matching the speed of cars to your right, effectively blocking the road.
It's not just customary in many (most?) states, it's the law. People who sit in the left lane are the problem.
The last couple laws like that I checked only talked about limiting flow below the speed limit.
> (b) An operator of a vehicle on a roadway moving more slowly than the normal speed of other vehicles at the time and place under the existing conditions shall drive in the right-hand lane available for vehicles, or as close as practicable to the right-hand curb or edge of the roadway, unless the operator is:
> (1) passing another vehicle; or
> (2) preparing for a left turn at an intersection or into a private road or driveway.
https://tcss.legis.texas.gov/resources/TN/pdf/TN.545.pdf#545
Note this law specifically mentions "normal speed of other vehicles at the time and place" and doesn't directly mention speed limits. So by the text of this law, if you're driving the speed limit and hanging out in the left lane while the normal speed at that time is like 10 over you're technically breaking this law.
We have specific signage for highways where this is supposed to be the law.
https://www.txdot.gov/manuals/trf/smk/regulatory_signs/left_...
A custom that (where I live) is becoming more honored in the breach than the observance. It makes driving very much more dangerous.
In Britain they have a sardonic nickname for people who do this: CLARAs. "Centre Lane Residency Association".
To your point, I think the middle lane of a three-lane road is, ideally, the correct travel lane. Cruising there at the prevailing speed leaves one lane more lightly traveled for entering and exiting, and the other for passing. Predominantly using that lane minimizes lane changes, which are the most dangerous driving moments. You're right, though, that the strategy breaks down as traffic gets heavier, and gets ruined entirely when (as under discussion in this thread) people gum up the supposed "fast lane".
I think this undersells it a little. It does not just impede faster traffic, when the lanes are pacing each other it makes navigating harder -- simply switching lanes is more difficult. The highway moves so much more efficiently with a small but steady difference in speed between each lane.
When traffic isn't balled up so tight, you can plan for a lane change in advance and accomplish it without having to slow down traffic. Everything flows better.
I haven't experienced people riding right on me for a tiny speed change.