Years ago I read a story of someone who took an Asian Art class as a fine art graduation requirement for college. They basically looked at photographs of Asian Art all semester.
At the end, he didn't think he got much out of it.
Then he went to a museum with Western Art. And found it a bit odd. He had grown used to the Asian styles and motifs.
Another story I have comes from a friend of mine. His (British) parents had a very odd cartoon book. It contained cartoons of people on fox hunts. They were actually political cartoons, and to understand them you had to know how mid-20th century British MPs were caricatured. He didn't, so it make no sense to him.
Duchamp's Fountain is something which is hard to understood without knowledge of art history - if only because that transgressiveness is now much more common because of Duchamp.
Someone deeply into classical Western music might have no interest in classical Indian ragas. Indeed, the music theory you learned doesn't describe ragas.
Innumerable art museum visitors have looked at something piece of art and exclaimed "my 4 year old can do better than that!"
And so on. For all styles of art.
That said, to get to your questions:
1) "How do you decide for yourself whether it's a quality problem or a problem with the consumer of the art?"
It's always a problem with the consumer. I may grow to admire the art of 4 year olds. I may find a pile tossed coffee grounds to be the next masterpiece.
But is it my problem that I don't care to put in that effort? No. Unless I want it to be.
2) "...something you'd previously thought was crap?"
I'm struggling to think of things which fit that description. Most things I file under "I don't know how to appreciate it."
Perhaps a good example is the false and snobbish belief I had that I was too mature to read, watch, and enjoy children's literature and shows. I remember dismissing Fraggle Rock because "that's for kids". I then watched it when I was 30-something and kicked myself for my earlier viewpoints.
In college I looked a bit down on a friend who enjoyed reading children's literature. Now that I have kids, I can say there's some really great stories out there - stories that don't require a kid in order to enjoy them.