Mostly they serve at the pleasure of the president, some have to be approved by Congress.
> Is the claim being made that these people have a constitutional right to run the FBI, USAID, etc forever, and the public is not allowed to elect someone to fire them?
No, the claim is that only Congress can allocates funds to create these agencies, and only Congress can de-allocate funds for them, so the executive branch (and whoever the executive branch empowers, whether or not they are a government employee) does not have a right to shut them down.
> From what I can tell, it does look a lot like a coup, of an unelected bureaucracy trying to prevent the elected president from fulfilling his campaign promises that the public elected him for.
The president is not a king. He is the head of the executive branch and has a large degree of control over how the bureaucracy is run, but not total control. Maybe you aren't American or maybe you don't remember elementary school civics, but the US operates on a system of checks-and-balances to prevent any one branch of governement from acquiring too much power. The coup-in-progress is the president apparently subverting those checks and balances.