So I periodically think this is true, and wonder "Why do I get paid so fantastically well for just drizzling some AJAX goo on top of pre-existing APIs?", and then I actually talk to Real People (TM) and remember that I inhabit another planet from them.
On their planet, being able to calculate the area of a rectangle reliably, the first time, makes you anomalously good with math. The notion of programs as long strings of instructions, followed sequentially, with predictable results is a fantasy: computers are observed to operate mostly randomly, all of the time. Some days I go to my Googles and it's the Googles with the box in the middle,a some days I go to my Googles and it's the Googles with the box at the top. Why does Microsoft keep changing the Googles? THIS IS SO FRUSTRATING.
You and I might think there is a distinction between programs and data, and we could describe how these exist in different places in a hierarchical file system on a computer, which is an abstraction over sectors on a hard disk. Every technical word in that sentence is scary juju to most of the population. There are many otherwise intelligent people who cannot open MS Word without a document emailed to them first. If you took away the document, they wouldn't know how to open Word and wouldn't know how to correct that.
In addition, many folks who are unemployed or underemployed in the United States either cannot read this post or cannot understand enough of it to answer simple questions like "Does Patrick think that programming is hard?"
You can certainly become a passable programmer in 3 years starting from being well-educated, intelligent, and reasonably skilled at knowledge work in the information economy. If you're 3 for 3 on those, you don't have a problem right now.