The rest of the student body was made up of poor, mostly black students who came from families where education wasn't a priority. Every single year I was in high school the state tried to revoke our accreditation and we just barely passed a state audit to keep it. I was a math tutor, and it was common for students to come in who couldn't do basic addition (in high school).
To make matters more confusing, the gap between the good students and the bad was almost entirely caused by economics (wealthier parents resulted in smarter kids, as is normally the case) but because the economic divide was almost identical to the racial divide, many of the educational issues were easily conflated with racial issues. This led to the (mostly black) teachers and administration demonstrating favoritism toward white students, and it also created tremendous social pressure on black students not to try to succeed academically (otherwise you're an oreo).
I wouldn't trade my experience there for anything. I received a pretty good education, and I experienced a part of America that more privileged kids should experience, but most of the students at my school were definitely being left behind.