Even just using crypto primitives is dangerous unless you've got experience. There's an old post from matasano about it https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...
What you say as messing up with edge cases, etc. is a programming problem. The errors talked about in the link he posted is about using faulty crypto. Today it is common knowledge to never use ECB.
What cipher mode you use is a problem you think about before you even start writing a single line of code. But you are talking about programming errors, which one can make in all areas, not only ones involving crypto. Those errors can break security similar to messing up crypto implementation.