The problems in the classical theory are easily understood. The
charge and current densities of point particles are not smooth
functions but distributions (think of the Dirac δ-“function”). If
they act as the sources of the EM field the EM field itself becomes
singular. Now if you try to solve the full Maxwell equations
including the backreaction of matter & radiation fields you would have
to multiply distributions which is ill defined.
There are similar problems in the quantum theory but the divergences
are less severe and can be dealt with in a systematic way. Most
physicist believe they will totally disappear in some more fundamental
underlying theory. From a mathematicians point of view there is the
hope that at least some QFTs are finite and the divergences are just
an artifact of the construction & pertubation theory.