I've done it on and off. Lower paid jobs are often highly correlated with bad jobs though. The people are more toxic, lazier, less intelligent. It can end up more stressful, because you're used to doing things faster and more efficiently, and then just end up with people problems.
FAANG is probably less likely to face this, but if it's a step down from where you are, there's probably reasons why it's that way.
You're also going to deal with shittier code. It doesn't matter how good you are; shitty code hurts. A problem with smarter companies is they have more educated people who don't know what they're doing. Optimizing underengineered code is easy. It's harder to merge overengineered code - often new programming languages, compilers, code being on the wrong abstraction layer. Often people argue that overengineered code is just merging, but it's usually merging things that are built in such a tightly coupled way that it can't be merged. Long build times are a frequent side effect too.
Also don't stick around too long. When I go in, there's some admiration and even a little skepticism why I chose to join. After a few months, you get treated as just another person on that rank, even if you were CTO-tier. I ended up staying longer than I should have because the engineering was fun and there was some desire to prove that I wasn't there for just a vacation.
I did manage to sit around and learn a lot more, but it's slower than learning from a better group. I spent so much time putting out fires that there was rarely ever any time to really learn something. It didn't exactly hurt my career, though, just gave me more drive not to do it again.