I think that this passage and various other parts of the book are a particularly English form of understated humour, where Babbage is playing up his public perception as a frustrated genius in order to add a little levity to what might otherwise be fairly dry writing.
Plenty of other sections of the book hint at this. As an example, search for the word "philosopher" throughout the book. We get section headings musing about whether a Philosopher would deny being descended from Cain, quotes attributed to a recently-scalloped oyster who was also a philosopher writing in its shell, a sequence of passages titled "The Philosopher writes a Ballet —Its rehearsal —Its high moral tone —Its rejection on the ground of the probable combustion of the Opera-house", and so on and so on. Does Babbage consider himself a philosopher? Maybe (probably, even -- he clearly had a good opinion of himself in any case), but he is also well aware of how the public perceives such people.
In other words, Babbage is doing a "bit" and playing with the public perception of him and of scientists of the time (and even now) -- i.e. as being so removed from the real world that they not only can't explain something, but can't even understand why they can't explain it. Which is an understandable thing to do, particularly if you're a mathematician and engineer writing an autobiography for a lay audience.
This doesn't mean that he was actually good at explaining anything, of course, or that he wasn't in fact a frustrated genius. It just means that he was aware of how he was perceived and was capable of a little humour at his own expense.