You are correct. England has an established church, as does Scotland (but not the same one as England, interestingly). Wales used to have an established church, but it was "disestablished" in 1920. Other European countries have established churches as well.
Often this means that not only are they the official church, but that tax money is actually used to support them -- even tax money from people who don't belong to that church.
The Founders did not like paying taxes to churches they didn't belong to, and who can blame them for that?
Also interestingly, at the time the Constitution was enacted, some of the states did have established churches. The Constitution didn't do anything about that, as it only affected the Federal government.
The First Amendment has since been "incorporated" (by means of the Fourteenth Amendment) in such a way that states are also prohibited from establishing churches nowadays.