If you have other significant demands on your time then school is going to be hard no matter what strategy you choose and might just take a lot of years to finish (please vote for UBI to help change that status quo). Assuming you can commit to school full-time though, it's less of an issue:
- You can finish a typical bachelor's degree in 5y by devoting 48h/week to school (12h for classes, 36h for assigned work). That's more time than most teachers expect, and except for a few unlucky points in time where multiple large projects coexist you'll rarely have to actually spend that long studying.
- It's hard to express just how much of a difference it makes to start with a little easier content to ensure you have the requisite background; don't let your pride hold you back. E.g., if the university places you in math class X, start at class X-1. You'll spend less total time learning throughout your education, be far less stressed, get better grades, and understand the material better.
- If you just want the degree, a lot of the first 1-2yrs of college courses can be replaced with CLEP/AP/... tests. Anyone can take them (not just in high school), and it'll take a lot fewer hours and dollars to get a passing score on one of those than it will to get a good grade in an equivalent college course (at most universities such tests won't affect your GPA, so even a score of 40% on some of them still gets you a passing grade).
- It's quite a bit more efficient to take more courses at once, especially if they're closely related (e.g., topology, abstract algebra, real analysis, ...). If you have closer to 70-100h/w to spend in class and studying then consider finishing your degree in 2-3y instead of 4-5y. You'll still have plenty of time to do fun things over the summer and winter holidays, and I know I personally found it more motivating to have an end date in the near future.