Edward Rutledge, Arthur Middleton and Thomas Heyward Jr. were all imprisoned at the PoW camp at Fort St. Mark (aka Castillo de San Marcos), St. Augustine. Not a prison ship.
The Wikipedia article at https://en.wikipedia.org/wiki/Castillo_de_San_Marcos links to https://books.google.com/books?id=eCc8BQAAQBAJ&pg=PA126 which describes their captivity and that of others as:
> They lived in the unfinished statehouse and in private houses in town. Since many were paroled, most of them were given considerable freedom of movement. The prisoners' diet was salt fish, occasionally salt beef, vegetables, and potatoes, and one-quarter pint of rum daily. They were permitted to write home for money to purchase items from local vendors. At the statehouse they slept on mattresses on the floor, and stayed the better part of a year.
That is not brutal neglect and certainly not torture.
George Walton, the 5th captured signer, was captured during the Battle of Savannah in 1778. From https://archive.org/details/biographyofsign03sand/page/124/m... :
> Walton received a wound through the thigh, fell from his horse, and was taken prisoner. He was paroled until he recovered from his wound, and then transferred to Sunbury, as a prisoner of war. The high station of colonel Walton as a member of congress, and his signature to the declaration of independence, induced the British government to demand a brigadier general in exchange for him, but the term for which he was elected having expired, he was ultimately exchanged, as a lieutenant colonel, for a captain of the navy, in September, 1779.
Again, not brutal neglect, and not torture.