The core problem of the Roman Empire was immigration and the speculators that it has attracted. For example, when gold was discovered in Dacia (Provincia Dacia, nowadays Romania), they rushed to the Eastern Europe building colonies near the Danube, something similar to what happened in the time of the Gold Rush in USA. Since immigration in Rome was embraced, many nobels stopped paying good wages to peasants and they started to bring slaves and cheap labour from all over the empire. These peasants, left without work emigrated to other successful Roman colonies like Provincia Britannia and Provincia Dacia which were very powerful back then. Even some nobels themselves took everything and left.
Also, the invasion of the barbarians on the Rhine and Danube fronts (mostly Goths - Visigoths and Ostrogohths -) didn't help. So many people were coming south of Rhine and Danube that the empire could not self-sustain itself anymore. Many families leaving for Constantinople after 400 AD. The population of Rome decreased from something around 1 million down to a few thousands. Also, Mediterranean Sea stopped playing an important role and the next big thing was the commerce between Europe and Asia. Constantinople being at the cross.
Edit: As Neagu Djuvara said in his book, don't imagine that all barbarians were the same and their only aim was plundering, there are many sources which show that they actually got asimilated into Roman life and some of them offered their services as Merc Corps to defend the borders of the empire, keeping their own culture and traditions. Historians enamoured with Roman Empire history depict barbarians as being savages which was in fact not true. For example in the Romanian language, Gothic toponyms have been preserved to this day. Also the word "viteaz" which means courageous in Romanian is stemming from the word viking. Which clearly shows that these cultures intersected.