I couldn't find a primary source but all sources say the same as this one: https://www.imidaily.com/datacenter/portugal-data/
The program lets in on the order of a few thousand people per year.
Other visa programs there vastly outstrip this is number.
The golden visa is just a great marketing term for political ire. It implies wealthy privilege so it's easily targeted.
Of course, the reality of things is that Portugal has recently rightly earned a reputation for being a great place to live and people (not many of them super rich) decided to live there. Housing is being built to compensate.
The airbnb thing is more interesting and nuanced but harder to get a handle on. It might be that airbnb is causing some unhealthy things in the real estate market.