High school and middle school PE were fucking miserable for me. I wasn't fat, but I was slow and in a group with very fit (naturally or by practice) guys. I was also uncoordinated as I had a massive growth spurt and, being extremely near sighted, wasn't active in any typical sport so my coordination lagged behind for years.

College PE is different. It's not (usually) a general class. You typically pick which one you want. Weight lifting, running, cycling, golf, swimming, etc. This means you're working on a physical activity that you enjoy or have some moderate interest in, unless you truly despise moving and can't even find one thing in the list that interests you (and somehow never fits into your schedule over the 4 years of college).

And all those options were at a cheap, no-name state school.

EDIT: As well, you typically end up setting your own goals within the constraints of the course. Usually an A requires: Be fitter/healthier/faster/stronger than when you started.