I won’t dissuade you from reading (listening) to Gibbon as he is something of a literary classic, but you must always be aware that Gibbon was writing two centuries ago and had his own ideological ax to grind. Historians today, drawing on a wider range of sources than Gibbon had access to, ascribe many developments in Roman history to very different factors than Gibbon claimed.
In fact, it is advisable to read several more recent works on Roman history first, so that you are then reading Gibbon purely as a literary classic and not as any authoritative source.