Yes, but if you're going to replace it, then you really,
really want to take the time to understand why it was the way it was.
Sometimes, you really do run across things that could never make any sense to anyone who knew better, like the famous Daily WTF example where someone created a website for a restaurant by printing out the page design for the paper menu, laying it on a wooden table, taking a picture of it, and uploading the picture. But other times -- much more often, in my experience -- you run across hidden technical challenges and limitations that led to what look like really stupid decisions until you understand the real motivation. And even when there were some really stupid decisions, it's often better to leave the system largely intact while you incrementally fix the stupid parts.
And, as I said before, sometimes it really does make sense to go in and start over. It's just that the line for when it's warranted may not be where you think it is.
As far as learning things go, providing your own implementation to solve a problem can be a great learning tool. Likewise, writing your own compiler, database, or web server can be an absolutely fantastic means for understanding these tools better. But it will rarely be the best means of solving a problem that requires a database-backed web application.
Reuse production-proven existing code and tools wherever you can when you're on the clock for a job or building a company, and teach yourself how things work on the side. If you build something so good that it's worth swapping in for the original and going through all the steps to bulletproof it for production, then awesome. If not, then you've still learned what you cared about learning.