The US declared war. They were pissed off about many things, but one of the more notable ones was that they were frustrated that the UK was supporting the sovereignty of Native American nations, which hindered the westward expansion that many people in the US viewed as their birthright.

The UK was actively making concessions to try and avoid war. The United States declared war anyway.

The UK then taking the fight to DC was absolutely fair game. People are allowed to fight back when you attack them.