Actually, this has been done: https://arxiv.org/abs/1903.11391