I don't think it was videos. Almost certainly it was replay files with a bunch of work to transform them into something that could be compared to the model's outputs. (Alphastar never 'sees' the game's interface, only a transformed version of information available via an API)
This was my understanding as well, as the replay files are all available anyway.
The YouTube documentary is actually very detailed about how they implemented everything.
Do you know if it was actual videos or some simpler inputs like game state and user inputs? I’d be impressed if it was the former at that time.
starcraft provides replay files that start with the initial game state and then every action in the game. Not user inputs, but the actions bound to them.