"I often go out of my way to talk to the wall flowers and shy people that no one is talking to, but I've found by and large, they are also much harder to talk to!"
Conversational ability comes with practice, and those people probably just haven't gotten enough practice yet.
Also, some people are just naturally reserved, and it takes a while for them to warm up to you and start talking more freely.