are you sure it's Signal? cause i have this problem with my husband too, but it happens with Hangouts as well as with Signal. my best guess is that his android phone, a cheaper LG model, has limited resources and puts some services to sleep...
Most of the time it would work fine, but quite frequently messages would be sent (have left my device) but not appear on the receiving device for ages, so i can only assume that sometimes the infra couldn't keep up.