That's totally fine and all it means is you may need to start off in a lower course to shore up your missing foundations [1].
There have been so many people in this situation that we actually designed a Mathematical Foundations course sequence specifically for adults who want to get up to speed or relearn math skills they have forgotten (from fractions through calculus) as preparation for upper-level university math courses. More info here: https://www.mathacademy.com/adult-students.
Please do let me know if you have any follow-up questions about that or if anything is unclear. I'm always happy to chat with people who are serious about learning math.
---
Footnotes
[1] Note that we do check for missing foundations during diagnostics, and any that we find we'll add to your learning plan -- but currently there's a limit to how far we look back. For Linear Algebra, we only look back to the beginning of Algebra 2. So if you're rusty on any Algebra 1 stuff -- factoring, quadratic equations, systems of linear equations, etc. -- or arithmetic stuff like working with fractions/exponents, then you'll need to drop back to a lower course to shore up those foundations.