Yes, this can, and will, be abused for tracking users across domains that they don't expect to be related.
But there are also legitimate use cases for this.
For example, consider the stackexchange family of sites. They are clearly related, have a unified branding, etc. but are on separate domains. On Firefox, which blocks third party cookies, I have to log in to each of those domains separately. I can't log in to stackoverflow.com, then go to superuser.com and already be logged in. That is a problem that First party sets would solve.
You can argue that it would be better for those sites to be subdomains of a single unified domain, but when the sites were created there wasn't any compelling reason to need to do that, because third party cookies were still very much alive and kicking. And I can say from experience that migrating an app to a different domain without breaking things for users is a royal pain, and can be very expensive.
I'm not saying that First Party Sets should be accepted as is, but it is attempting to solve real problems. And I think a solution that simultaneously protects users' privacy and maintains a good experience for sites that are legitimately related will be difficult to find, or maybe impossible.