These refer to 2 different protocols at the same layer: UDP, and TCP. Below these protocols there are layers that route little messages around local networks (like Ethernet) and that route little messages across the Internet (like IP), and UDP and TCP are another layer of abstraction on top of that.
TCP adds information about the ordering of messages, it has a mechanism for acknowledging receipt of a message, it has logic for resending a message, etc. UDP does not include these concepts, so if a message gets lost somewhere in the network, an application using UDP might not even notice.
The "connection" in this case, really refers to the state about these messages that is kept. It's a virtual / logical connection, not a physical connection.