In the context of seq2seq models, attention is a technique to compute a weighted average of hidden states of the encoder. When I first realized this simple fact, everything finally clicked.
In constrast to a vanilla seq2seq which takes only the encoder's hidden state at the last timestep as the context vector, the context vector of a seq2seq with attention is a weighted average of all hidden states of every timestep. The weight of a hidden state of the encoder is a similarity score between the hidden state and the decoder's previous output (the decoder's current state). The similarity function can be as simple as a dot product, but there are various ways to do it.
Attention can improve a seq2seq model because now the encoder's last hidden state doesn't need to well represent the whole input sequence, which is hard if the length is long — now the decoder takes at every timestep all the encoder's hidden states and computes an average of them with the weights uniquely different from the other timesteps. The weights at a timestep represent which input words are more important and thus to focus on when the decoder is to output a word at that timestep.
More generally, attention takes a set of values vectors and a query vector and computes a weighted average (or more generally, a weighted sum, if the weights don't sum up to 1) of the values based on the query. In the context of seq2seq models, the values are the encoder's hidden states and query is the decoder's previous output.