When I was learning C++ and Win32 API, I would implement small applications that would draw some graphics, because I loved graphics programming.
When I was learning java, I would go to freelancing sites like elance (and the now dead rentacoder), read project descriptions, design solutions and implement them. They'd be all kinds of projects - from CRUD web apps to custom compilers for some exotic languages. I'd never bid on any of those projects. For me, they were just real world problems that someone had and I figured that if I could solve their problems, I'd become proficient.
When I was learning python, I had also developed an interest in scientific computing. So I'd implement algorithms related to graphs, trees, linear algebra, machine learning, etc.
Initially, I found that if I couldn't come up with a clear roadmap of how to implement something, I'd dismiss it quickly as too tough and search for an easier project whose implementation roadmap lay inside the circle of my existing knowledge.
But then I realized that that's the wrong way to learn new things. The way to learn is to persist with tougher problems and expand that circle of knowledge. It requires some self discipline to stay the course because it can be frustrating, but in the end I feel that's what improved me as a developer.