You really don't need to make excuses for Justinian's incompetence, but if you want to, have at it.
Southern Italy and Africa turned out to be important in the fight with the Persian and the Arabs. Thinks like 'planted the seeds for collapse' isn't a strong argument for me.
Even with money issues its not clear that the empire was worse off when he left the scene.
Like many emperors he spend a lot of money and went on offensive wars to reunite the Roman Empire. He wasn't as successful as some, but more then many.
And the empire after him was strong enough to fight many wars against strong enemies, in the Lombards, the Avars and the Perians. So the empire clearly wasn't weak for the next century. The loss to the Arabs had more to do with Persian Wars that went on for decades.