One thing that wasn't mentioned: if you're doing any kind of graphics programming, you use a huge amount of basic linear algebra every day. Vectors, matrices, dot/cross products, affine transforms, spaces and subspaces, that kind of thing. If you're doing more advanced stuff, a working knowledge of how to solve large linear systems (i.e. things like Gauss-Seidel/Jacobi methods) and the physics of light transmission is also useful. If you're doing physics simulations, obviously you need a working familiarity with calculus.
Again, though, all of these things are massively deep fields that people dedicate their lives to understanding, but as a programmer you just need to skim the surface, generally speaking.