The first step to becoming more diverse is acknowledging the lack of diversity.
To get diversity more effort needs to happen in reaching out, most people don't do this because it requires extra work. As you mentioned, many people feel these things are not for them. Or perhaps they lack the confidence/skills/bravery to take things or even to enquire about the investment.
I saw the founders of Code Untapped give a talk the other week. They run hackathon/meetups and they are always experimenting and changing the times of them to try to reach more people who otherwise might not be able to attend.
And if you want to reach a diverse audience have diversity in your team, easier said than done, I know. But if you do things like Venture Partners, you could make a point of only having Venture Partners who are minorities. But also be prepared for white males to kick up a stink if you do this <--- I see this a lot!