Are there any cons of serving audio-less videos as video instead of GIF?

A GIF is a lot more lightweight than having a real video.

But of course it's possible to have a video with lower quality and framerate as usual, to have an alternative to a GIF. And a lot of people use it e.g. for background videos on websites. As browsers now support video more and more in HTML5, and h.264 is established as codec across all important browsers, it gets feasible. However, you would have to disable the controls of the HTML5 video elements.