I work at AWS, and worked at private businesses who used AWS prior to joining AWS. I also have a handful of AWS (including DevOps Pro) and non-AWS certs.
An AWS cert will teach you how to do DevOps using AWS tools. It'll teach you how to build a CI/CD pipeline, deploy containers, manage Kubernetes, etc. all in AWS. But to really understand it all, you need to know the underlying tech.
I would suggest learning the underlying tech first. And then if you decide you want to deploy on AWS, get the AWS cert. But if you don't know the difference between a Packer file and a Dockerfile; or if you don't understand IaaC, the AWS cert (assuming you pass the exam) will be of little use.
If you want to do AWS, I may suggest getting a Developer Associate/DevOps Pro training course and going through it. But any time you run into a concept you don't know, stop the course and go backfill your general DevOps knowledge. It'll help you retain the AWS knowledge, and you'll also be in a better position to actually put that cert to use.