If you are interested in this also check out EarthPT, which is also a time series decoding transformer (and has the code and weights released under the MIT licence): https://arxiv.org/abs/2309.07207