I call myself an interface designer. But I don't consider what I do purely visual at all. I find that it's hard to describe. How do you describe to someone the feeling you get when you put your tongue on a lemon? Or how it feels to click the "next page" button on newspond.com?
Design is not how it looks, how it feels, or how it works. It's all of those things, together-- it's the experience. On the web, it's the way a layout makes you feel on a subconscious, visceral level and how the buttons and interactive elements literally feel when you click them. On a more tangible level, it's the way the volume knob on a stereo feels stiff when you change the volume, or how a nice doorknob makes a thump-click when you close it.
There's no way to "design" an amazing product if you don't consider everything at the very beginning. On the web, there are three things to consider: visual design, interface architecture, and underlying technology (an analogy would be the tires and color of the exterior, the safety features and interior design, and the engine of a car). If you skimp on any of these three things, your product will seriously suffer. Do you own a Chevy Nova or a Dodge Neon? Of course not. Both of those cars were built by sacrificing visual design and safety features for cheapness.
There's a philosophy of rapidly developing, rapidly releasing, and then rapidly iterating. I think a lot of companies take this too literally and rapidly build everything half-assed, rapidly release half-assed products, and then rapidly iterate half-assed features without ever going back and fixing their half-assed first revision. I think a better process is to design an easy to use, awesome version 1 with the simplest, most necessary features first, then iterate based on that level of quality. At that point, you've set a bar of quality that you simply have to achieve with every release.
To summarize, don't skimp on designers or hackers. Skimp on features. Features are free, and can always be added later.