Learning to program is actually pretty easy. You can get up and running in a few minutes with your first Hello World program and that's how most programming languages are designed.

For me personally, the hardest part was having the patience to actually manage to build something useful other than the sample toy programs that came with the textbook.

You need to have the patience to read tons of documentation, google for example code, ask questions on Stackoverflow when stuck, so on and so forth. Once you do this for a single project and get something useful built, it gets much easier :)