Since you generalized a large group of people, I'm also going to generalize them, too. Most swimmers are very reasonable people just trying to work out. If you ask questions while they're between sets or after their workout, I'm sure they will be willing to talk to you. There are only like 4 rules when swimming.

1) Stay to the right side of the lane. 2) If you're on the wall, stay in the corner of the lane (so people can turn). 3) If someone touches your foot, stop for a second and let them pass. 4) If multiple lanes are open, try to stay with people of similar speed.

If they're getting mad at you for anything else, those people are being jerks. Rule 1,2, and 4 are all things you would do while running or cycling. They are follow the flow and try not to be in the way. They are common courtesy everywhere, not just a pool. Rule 3 is the only one that may not be intuitive, but you'll be kicking someone in the fingertips and face for at least a whole lap if you don't let them pass. Plus, it can ruin a set if they're trying to hit a certain pace. I'm not saying anyone should ever be a jerk to you, but it's similar to when people walk 4 wide down the sidewalk.