transformer-xl is a totally different net. the thing to keep in mind is that gpt will play along with anything unless they explicitly prevent it