They're not just cartoons. They're caricatures associated with racist tropes, which have been used in conjunction with discrimination and even violence. The African pictures in particular look like minstrel shows, which were deliberately intended to demean black people in America.
It's not that the publishers were afraid of getting in trouble. It's that they were ashamed of their own book -- with good reason.