Note that this is with the benefit of a past where the Germans didn’t get the bomb, and we know they probably wouldn’t before the end of the war. That was no guarantee to these men.
In a past where the Germans (or even Soviets, possibly) get the bomb first, you could well have seen these scientists thinking “if only I had worked on that program”.