Sure, you could write an app that does A/B testing, but the whole idea is that people will use Sandstorm to run _personal_ servers/apps. And A/B testing your own reaction to an app doesn't make much sense.
Runtime permissions of these apps will be easy to control, so the Sandstorm platform will prevent apps from phoning home without your permission.