The crew (other than the captain who probably had his own cabin) probably stayed at the lower deck because the gun deck had better ventilation for the passengers. It took a long time after a ship first became visible for a gun battle to occur so the crew had enough time to get ready.