I'm like this, and yes, very much so.
The (US) academic system is not set up to accommodate anyone, really. It's designed to get someone just below average through their standardized testing and not much more. If you don't fit that mold you're shit outta luck. You either suffer through it like "normal" people, or you find it utterly intolerable and fail or drop out.
Personally, I learn best backwards from everyone else. Building up very slowly from fundamentals and basically starting over with basic algebra every semester is actual hell. I need to see the goal concept fully formed and functional, and then work backwards to derive the fundamentals I'm missing.
Generally speaking, once I understand a concept I have it forever. I usually only need the briefest of refresher on mechanics and formulae as I use them. Spending the first month of calculus class going over 9th grade algebra is an unbelievable waste of my time.
My final attempt at college was a CS degree. I made it through one semester and did not even get to a single CS concept. It was at least a year and a half of bullshit prerequisites that I had to pay for. I dropped out when I had to write a presentation to the board of my hypothetical company on the benefits of upgrading their printers. I'm not kidding. I paid real money for this.
I've totally given up on the educational system. I don't fit into the cookie cutter ideal of the average idiot grinding out a degree. I just can't do it.
I think probably the ideal way for me to learn is to spend a lot of time one on one with a domain expert that can show me the final concept and work backwards with me to fill in the gaps in my knowledge. I don't want or need, nor can I tolerate spending time going over things I already know for the 30th time. I need to learn the things I don't know. School just doesn't work that way.