Can someone explain for the layman the following doubt?

One of the (many) things I still don't understand about RL even after having tried to read about it is how the loss function of Q Networks is computed if you don't have the target value. I understand that you are trying to predict the "value" of a pair (state,action), and I know that there is a formula for updating the weights based on the difference between the expected value of the next state and the maximum predicted value available plus the reward when you actually get to the state (probably I'm messing up here), but then doesn't that mean that the initialization is extremely relevant? As in, how will the rewards ever be superior to the random weights already present in the network?