You need a great knowledge of math to internalize the complexity of neural nets. One particular area of mathematics that is not in the set of highschool-level ideas is topology.