The concern is that the most informed names, and those spearheading the publicity around these letters, are the most conflicted.
Also, you can't scan bio lines for the affiliations that impact this kind of statement. I'm not disputing that there are honest reasons for concern, but besides job titles there are sponsorships, friendships, self publicity, and a hundred other reasons for smart, "politically sophisticated" people to look the other way on the fact that this statement will be used as a lobbying tool.
Almost everyone, certainly including myself, can agree that there should be active dialog about AI dangers. The dialog is happening! But by failing to make specifics or suggestions (in order to widen the tentpole and avoid the embarrassment of the last letter), they have produced an artifact of generalized fear, which can and will be used by opportunists of all stripes.
Signatories should consider that they are empowering SOMEBODY, but most will have little say in who that is.