No simple (native) solution yet, as far as I know. While iframe has loading="lazy" support, it probably will be detrimental to accessibility. Could be a fun experiment, though. Maybe try setting preload to "none" on video? But this might be overridden by autoplay attribute.

Solution that would require more work involves removing autoplay attribute from videos and using IntersectionObserver to check when video is inside viewport and start playing it.