Nope, it doesn't wreck your body, not even close.
This is because of homeostasis. You normally have close to, iirc, 2000Kcal floating in your blood, intracellular fluid, & cells. If you get too much more, then the pancreas will produce insulin to stimulate your body to store the excess as fat.
But when you are temporarily under-fed as with Intermittent Fasting or Time Restricted diet, the insulin simply doesn't arrive to force it out of the bloodstream so it can maintain the levels; levels can also be restored by converting fat stores to maintain the appropriate blood sugar level.
So, you've always got something like that ~2000Kcal store ready to go, and you don't need to worry about 'bonking' on any normal workout with any normal IF or TR diet.
I personally did once manage to do something like you worry about, during off-season training one year in my career in international-level alpine ski racing. I did a 3-day fast, with a solid weight workout (including big muscle groups) on Day#1. No problem. Then, at the end of fasting Day#3, I felt really energetic, so broke the plan and did another really heavy leg weight workout, which went great. However, about an hour+ afterwards, I felt really wobbly/shaky from low blood sugar. This was fixed with a single big orange juice smoothie. It was really cool to feel it take effect -- within minutes. And, there was no break in the training routine after these events, just continued on as usual.
Admittedly it's a single test, but it shows that it can take a lot more than one workout on a TR or IF day to even get to an out-of-the-ordinary state, and it's easily fixed when you do. Not really anything to worry about.
[Edits - clarity, commas]