I'll try to answer to the best of my knowledge (I'm not associated with project, I'm just a happy customer).
Does your ISP know that you are communicating with Signal servers? Yes, IP addresses.
Does it know to whom you are sending messages? No.
Does Google know you are using Signal? Yes.
Does it know whom of your contacts use Signal? Yes, because they have a full list of your contacts and they know if someone has installed Signal.
Does Google know you've sent a message? No.
Does Google know that you are receiving a message? Sometimes, because Signal servers ping your device via GCM with "wake up".
Does Google knows who from your contact list send this message? No, unless you have only one contact who uses Signal.
Can Google infer from pings who is communicating with whom? Yes, although pings are needed only if app has disconnected from server, and this severely limits usefulness of this technique.
Where else may any metadata coming from usage of Signal be? Nowhere.
As for Google having your contact list... Take a look into Flock.