Video games are safe e.g you can always try again, you can do incrementally well.
Video games are scary in an exhilarating way, you blood will pump and your pupils will dilate but at the end you can put it down, turn the lights on and watch funny cat videos to recover.
Math (at school) is hard because you get one shot. Finite lives. After enough retries society says "nope, you're done". It's hard because you can't test yourself when ever you want and your study is not directly related to your tests.
Math is scary like anxious scary. Like stay up at night dreading your exam or don't sleep for a weekend to finish your report scary. You can't watch cat videos and hope it goes away because it won't. You can't buy a new math or try it again in a few years.
You get one shot. You get little to no feed back.
That's hard and scary.