The histories I've read of the early middle ages corroborate much of your intuition. Trade collapsed; many luxury goods became impossible to obtain at any price; illiteracy rose; population declined and many citizens died from famine and plague.
But there's often a trade-off between freedom and convenience, then and now. The Roman empire was built upon large-scale slavery and wealth inequality. If you were a member of the elite, then yes, the collapse was terrible; you lost access to all the privileges of the empire. If you were a slave, you probably ended up somewhat better off, becoming either a peasant or a bandit. Not being able to obtain pottery or steel doesn't matter much if you couldn't afford it to begin with, and dying at 40 from starvation is an improvement on dying at 25 in the arena.