I spent some (too much) time trying to get pretty much the same thing running using GOW [1]. Was quite a bit harder than I thought, requiring a hdmi dummy plug to get the xserver config right etc.

1: https://github.com/games-on-whales/gow