No, inputs to each layer are concatenated before being passed through H.