Actually I think that maths and jazz have something in common in the general public peception that you have to be smart to "get it".

Nobody will try to perform a deep intellectual analysis of Lady Gaga's or Ed Sheeran's work the way they analyse Coltrane or Miles Davis (or Mozart, or Stravinsky). Those musicians are intellectuals of the sort Einstein is, unlike Lady Gaga or Ed Sheeran (in the collective perception). Jazz is intellectual music.

And when they analyse something, "smart" people use maths.

I am putting scare quotes around "smart" here to insist that this is largely a social perception and expected behaviour. However, maths can sensibly be used to analyse art, just like it's used elsewhere. This is not patronising, it is more that maths provides a useful language to talk about patterns.