Sometimes things are so hard and so overwhelming that you become a slave to your overwhelming misery and do things that you would never do with a sound mind. That does not mean that those actions are exempt from criticism. Most of the people on death row were likely of unsound mind when they did whatever landed them in that pointless punitive boondoggle.
If you can stomach abusing a child physically with intent then you and your spouse will likely continue to abuse him in one way or another when things get a little too stressful. As a former abused child I can assure you that he will forget. Then, when he is able to see children from an adult’s perspective, he will remember. He may bury his contempt and convince himself that you were right and that he should hit his own kids. Or he may lose respect for you. The latter would be the healthier response.