No, password managers can take care of that.
I have seen folks use password managers to store their poor non-autogenerated passwords.
For users that do use the PW manager properly, having the PW manager store the TOTP secrets is indeed "putting all of your eggs in one basket".
I used to think that but I changed my mind.
First, you can set TOTP (or other second factors) authentication on your password manager account, which I think is good philosophically at least, because you gotta have access to your second factor to get access to you website TOTP.
Secondly, using a password manager with strong unique passwords that you don't know brings already a lot of benefits that pushes websites and administrator to push using a second factor (it's very often a way to avoid attacks using reused or bad passwords).
You do lose a bit of security (there is now a risk that your TOTP seed get stollen), but the extra convenience (especially when you lose your TOTP device) means you can enable it on more websites without too much annoyances.
On the other hand it doesn’t prove that someone has stolen your phone/laptop, defeated all of its own security, and then defeated the security of the password manager.
For my personal risk propensity, the former is worth having, the latter is too unlikely to worry about