I guess that's what I meant; XSS would give them the user's session implicitly on my domain, regardless of CORS. CORS prevents them from using that from another domain, which is valuable, but moot if you already have a breach.
The important point though: You don't need any pre-existing vulnerability on your site in order for "Access-Control-Allow-Origin: *" to create a vulnerability.
This stuff is sure is confusing to talk/think about though.