Wireshark, Deep Packet Inspection and similar might be terms you can look after to find out more. RFCs are what you need to understand the protocols (if they are public standards)