so long as you can still make money, you need to be careful with this... it is easy to spend an hour saving $10, but if you bill out at $80/hr, that doesn't make sense, unless you can't find any work at all.
Personally, looking over my personal expenses, I see I can save a lot of money eating in more. However, I think the returns diminish quite a lot when choosing cheaper raw materials over more expensive raw materials. Either way, groceries are still so cheap that it's not worth the time to optimize. (but then, I'm not really an organic/premium guy.. I guess for some people looking at that might make sense)
same thing in the business... optimizing the co-lo costs makes a lot of sense... but saving a hundred bucks per server by not using hot-swap drives is a false economy. It only takes one bad drive (and the resulting downtime) to pay for a whole bunch of hot-swap bays.
I think usually the easiest way to save money is to look for 'premium' services. If you are buying $15/Lb 'premium organic' beef, eh, you might want to think about switching to the normal steak at $5/lb. If you are going with rackspace.com, you might want to look at a less expensive hosting provider. (generally speaking, if you have to ask how much it costs, you are getting overcharged. If you have to talk to a sales guy to buy the product, remember that his salary is being added to the cost of your product.)
Another thing to point out here is the premium for renting servers. At what I pay (that is, small business prices) I can usually build servers for what it costs to rent them for two to three months. Sure, ec2 is great if the servers are only on a few hours a day, but you can save a whole lot of money by moving your core servers to dedicated boxes you own and co-locate.
"Never buy from a rich salesman. Always use a rich lawyer"
Remember that the confidence and good looks of the salesguy have zero correlation to the reliability of the service, and usually have a inverse correlation to the value for the dollar of a service. Good salesguys are expensive. If you like that sort of thing, during good times, it's fine to spend the extra money and get the guy to take you out to lunch. But during the hard times, that's a premium you maybe should look at giving up.
Let employees work from home, if you can give up the office. If you are good at communicating over the computer, think of hiring people to work from home in the midwest or the south.
Drop your recruiting firm or body shop. Hire directly through contacts, or if worst comes to worst, craigslist. Usually headhunters add little value, and they are quite expensive. (actually, my experience has been that body shops give you the resumes of mediocre people. They do cull the worst, but they also cull the best, has been my experience.) Going through resumes isn't that hard, and really it's something you can probably do better than most headhunters. The layoffs are cutting pretty deep, even people who are moderately good are getting the axe now. And fresh graduates? good god. Times are tough for them... more and more people with degrees (and even a little experience) are moving into my price range.