If you expect any outside civilizations might be hostile, wouldn't a slower frame of reference magnify the risk, both in terms of giving slower civ a disadvantage in adaptation/response times if any conflict did occur, and also in giving the faster frames more time to spawn hostile civilizations?