Yeah it's custom. Perhaps the old version used to be based on the open-source bit of Reddit's codebase. But I think the new one is totally home-grown. I think its source code is available though.
... but there is nothing there about the commenting features ...
A number of people have asked us whether we could somehow make it a package and support the same commenting systems on other sites, but it would be quite a bit of work and really doesn't factor out very nicely.
Glad to hear you find them aesthetic! We put a lot of work into them over the last few years.