In A/B tests we have compared against e-commerce stores existing recommendations which are either manual or made by other recommender systems. We will consider running experiments like the one you describe it is a good idea!