Project Euler is like sex or diabetes (both of which I have or have had), what it means to you and how it fits into your life is very much an individual thing.
Yes, a lot of the problems can be done by hand. You could treat the problems as "a math problem to solve", or as "an exercise in solving a problem with software."
When you've solved a problem, you have either "obtained the correct answer," or you have "figured out how to obtain the correct answer with software." Or both.
For me, even solving the early trivial ones, that could be done by hand, are more an exercise in software than in math.
Most of the time, I try to write my solutions in a general way, so that I can change the parameters of the problem, run it through software I've already written, and get the right results. The challenge then is defining the boundaries and parameters, and imagining the possible and reasonable limits of those boundaries and parameters. Just like we do every day with the rest of our software challenges.
The fact that these are math problems make it more fun for me, because I am not at all mathematically inclined. So I have to learn a little more math, and then decide if I've learned it sufficiently that my solution is reasonable.
Reading other people's posted solutions is almost as fun as solving the puzzle. I've learned a lot, or at least been impressed, from the various languages and hand solutions.
As for learning a new language, I think Euler is a reasonable tool in the total box. I find that I don't really know a new language until I've struggled with something "real" that matters, to me or my employer. And even then, if I come back to code a year or two later, I sometimes cringe, since I've likely learned more about using the language's strengths in that time, beyond just writing C in a new language.