You understand me correctly. I thought the film would select for either no one at all or only people actually interested in the movie.
As you said, a "weird" movie doesn't keep their attention, so they just distracted themselves with their phones to kill the time until the next movie.
As an aside, the funniest instance of this phenomenon I ever encountered was when I was at an evening screening of an incredibly embarrassing idol anime movie at an AMC and two extremely out of place people walked in partway through, lasted about 5 minutes and then had to bail when one of the singing numbers started lol.