Creative Fansubbing Techniques: Part 2

We built our own KLT point tracker to make "Onscreen Text Substitution" stick to moving cars and/or during camera pans.

We reverse engineered Winny (a Windows-only filesharing software) to build a Linux server with WebUI so that you could conveniently remote-control a server in Japan.

We built a C-like language with AOT compiler that actually creates machine code to make particle effects render faster.

We used spectrogram analysis to assist with separating the audio track for along karaoke animations.

And of course there was multiple competing xdcc servers.

And then there was the truly crazy stuff like a TSR bootloader for your PS2 which would hook drawing calls in memory so that you can fan-translate video games, too.

We truly had an insane amount of cutting-edge tech back then, if you consider that this was just some people's hobby to enjoy Japanese content.

First of all, it's nice to see them receive attention outside the fansubbing scene! Anime has long made use of on-screen text in ways that most other mediums don't, and with the limited animation the medium is famous for, it tends to be extremely amenable to localization that aims to replicate that on-screen text presentation as closely as possible. With fansubbing, this is largely due to the power of the ASS (Advanced SubStation Alpha) subtitle format, and the Aegisub editor's capabilities for making use of that power (none of which is a coincidence - ASS was effectively developed for subtitling anime, and the same goes for Aegisub). Of special note is Aegisub's Automation features, which allow users to write Lua scripts to extend its capabilities, even building various extra GUI bits and bobs to make them usable for any non-coders.

To comment on one of the techniques in particular: it's always kind of fun to see people be impressed by masking, because technically speaking it's one of the more simple tricks that people do with ASS. The format includes basic vector drawing capabilities, which can be used to also make clipping masks for subtitles. How masking works is that usually the typesetter will be simply drawing vector clipping masks to clip the subtitles properly, frame-by-frame. Due to the limited animation, this usually doesn't take all that long. If the covering object remains static in shape, you could also just draw the mask once and then use motion tracking and an automation script to move it for the duration of the masked line. So all in all, not very complex, just a bunch of manual effort :)

And speaking of which: one of the biggest innovations of anime fansubbing happened around 2011-2012, when fansub groups started making use of motion tracking en masse. The popular program for doing the actual motion tracking was Mocha[1], and then various tools were used to apply its After Effects -compatible motion tracking data to typesetting lines in Aegisub. This development eventually culminated in Aegisub-Motion[2], which has been the de-facto motion tracking script for Aegisub for quite a while.

Motion tracking is also the main thing still lacking when it comes to typesetting in official anime releases today. This mostly comes down to the fact that while fansubs can have fancy motion tracked typesetting completely softsubbed since they expect their releases to be watched on reasonably powerful PCs, the same doesn't really apply to the much more limited and less powerful subtitle render used by official services. But honestly speaking? That's mostly just an excuse for the official services to not even try. You could have the fanciest on-screen text presentation in the world if you just burned your on-screen text translations into the video (hardsubbing, as opposed to softsubbing). Yes, this approach would work just fine even for multiple languages - modern video streaming is already based on short segmented chunks, so all you'd need to do is develop a system that will make multiple hardsubbed variants of only the segments where on-screen text is actually present. With this, for the vast majority of video, you would only need one variant. I developed the bones of a system like this myself for some official anime streaming I worked on some years back (unfortunately no longer available), and it very much was extremely feasible. This kind of segmented hardsubbing is even possible on Blu-ray, so the technique is disc-feasible too.

In conclusion, I completely, 100%, concur that I'd love to see a lot of these techniques be used with official anime releases. Especially since it would be very much possible to apply just about all of them as long as a publisher was simply motivated enough to make it happen.

[1] https://borisfx.com/products/mocha-pro/

[2] https://github.com/TypesettingTools/Aegisub-Motion

I don't remember anything about the anime except that the subtitles spun end over end and flew off the screen in perfect sync with the girl. It was beautiful and probably the best thing about the series if all I can remember from it is that one sub.