I think I disagree. When you get past AGI to where it can do things that humans can't, a lot of current safeguards haven't been designed with it in mind. So things might be vulnerable.

The kinds of things I'm thinking about are the various countries nuclear arsenals might not be safe from actors with very advanced AI. This I think is the potential source of existential risk, in my opinion. So it could hurt a hell of a lot.

So I'm of the responsible disclosure point of view. You ask "Would releasing AI advancement X mess up someones security/economy". If so, you help them patch it before releasing it to the general public.

The majority of advancements aren't like that and they won't be for a while.