For a competing anecdote, most of my friends who've read the books say they ended up skimming most of the Bombadil section and found it quite boring.
I suspect Bombadil is disproportionately popular among the people who like to critique what they read compared to the general public.