I've had a similar experience to yours. I grew up with an alcoholic and domestically violent father in a poverty stricken neighborhood. I was encouraged to go to college by a girlfriend of all things. I graduated in the lower half of my class and scored only a 900 on my SAT but despite this I finished my BS with a 3.55 overall GPA and a 4.0 in computer science. I'm currently working on my Master's degree.
I also feel like my intelligence is average but what helped me succeed is how much I enjoyed all my computer science courses.