For someone as excellent at sharing knowledge it seems like Feynman just didn't want to do the interview any longer. He sounds and acts a little pissed off with the question.
The issue with the 'why' questions is at the bottom most place in the the 'why' stack you need to have an axiom/postulation or at least a plain assumption, without which you can't explain the subsequent 'why' or 'how' questions.
Now this a classic trick in trying to prove science as useless and propel pseudo science concepts among crackpots.
A perfectly acceptable answer from Feynman would have been to just explain magnetic force in a basic (high school) level.
He could have made his argument about any of the questions the interviewer asked as pretty much every scientific question will get down to fundamental physics which will be beyond most people.