I believe all the problems you have mentioned can be easily solved. Permit adversity scores to weigh in favour of course grades and graduate honors. Legislation can be passed to ensure adversity scores count towards corporate and government employment to prevent any bumps further down the road.
Furthermore, adversity scores should also be counted towards recruitment in elite military units, selection for senior military leadership and all significant promotions. The adversity you faced in childhood should give one guaranteed opportunity throughout life. This progressive path ensures harmonious diversity and a perfect union of our states. Adversity is Excellence! Merit is Privilege!