I want to love Hipmunk and I do a search on it every time I travel but Kayak has consistently returned flights that are $50 - $100 cheaper. I love the Hipmunk interface but not $100 worth.
EDIT: Okay, I tried Hipmunk for an upcoming flight.
Pros: It offers a 30 day window for a flexible date search as opposed just 7 days on Kayak. Pretty interface.
Cons: Far fewer search options. My fare was $150 cheaper on Kayak! (Chicago to Hawaii over New Years).
London to Toronto (cheaper with CanadianAffair.com)
Halifax, Canada to JFK (cheaper on US Airways site)
Edmonton, Canada to LGA (cheaper on US Airways site)
They're all at least partly non-US, but I was able to find similar price discrepancies with flights from NYC to SF and Seattle. For example, I am quoted $398 ($441 tax-in) for a flight from LGA to SEA round-trip from Aug 25 to Sept 8. Hipmunk gives me tickets from $398 each way, so I'd be paying twice as much minimum.
In Europe, I've found SkyScanner.net to be generally cheaper because they have more discount airlines (namely Jet2, easyJet, and GermanWings; I avoid RyanAir)
This tripped me up the first time as well. That price is $398 round trip on Hipmonk. They show the best round trip price, not the best one way price. They say "from $398" to indicate that the best you can do is $398 if you pick the correct return flight.
For what it's worth the other flights were one-way so my misunderstanding doesn't affect the price difference.
Hipmunk: $1092. Kayak: $917.
(Both reservations have one layover in each direction and include tax. Kayak sends me to United.com directly while Hipmunk takes me to Orbitz.)
The Kayak number doesn't include fees and taxes, whereas the Hipmunk number does.
Admittedly Kayak has one flight for $870 and a few for $896, but they involved two stops.
The UI is clean and great, but its search is just not comprehensive as local competitors.
On the flip side, competitors also don't have all the data (company's site usually has data no-one else has).