I agree! I read "Consider Phlebas" first and was turned off the series until I was recommended "The Player of Games" - much more compelling in my opinion.
I was actually surprised when I finished UoW regarding how different it was from PoG, while maintaining the quality (If not better, but PoG IS THAT GOOD anyway).
For the record, the series was recommended to be by Claude of all things, and it told me that Phlebas was not up to quality but I gave it a shot just for the sake of completeness of the series. Nop, not worth it.