For specific examples, I've looked idly at doing chord diagrams and trees in ggplot. Chord diagrams don't seem to exist, you can do them with another R package but I don't think it interacts with ggplot. There's a ggtree library that does interact with ggplot, but in kind of a weird way that I wouldn't describe as "you can now draw trees with ggplot".
(I haven't explored too closely. This was in the context of "I want to write a ggplot for python, and I'm curious whether supporting chord diagrams and trees at some point in the future is plausible".)
ggplot2 is much nicer than matplotlib and d3 as long as you stay on the beaten path; which is wide enough to accommodate the majority of use cases. But the second you step off that beaten path, it's hell.
It just looks like there's a fundamental tradeoff between visualization expressiveness and API complexity.
Btw, Seaborn is a framework on top of matplotlib that replicates the semantics of ggplot2 if that's what you're after.
There's ggpy (formerly ggplot), but that also doesn't have a grammar going on under the hood, and on top of that it pretty directly translates ggplot's API to python. ggplot's API might be fine for R, but it's wildly unpythonic. ggpy doesn't understand where it's coming from or where it's going to. Plus it's buggy and incomplete and seems abandoned.
The one I'm currently looking at is plotnine, which keeps the API but at least it also keeps the grammar.
I was working on my own library until I found plotnine, with IMO a better API (and a rudimentary CLI). Now I'm looking at building it on top of plotnine, and when I have a POC I plan to get in touch with the author of that and see if he's interested in adopting it. (I think this is a long shot, but worth trying.)
Building abstractions is just a matter of writing very simple imperative functions. Modifying existing abstractions is a matter of copying and pasting out of the source code for existing ones.
It can involve lots of tedious trial-and-error, and the documentation is a little terse, but it's about as close to drawing by hand as you can get.
Ggplot2 is great for prototyping, but when you want to really own your graphics, go for Base.
There's also a middle path, called Lattice. It's built on the same library as Ggplot2 (called Grid), but lets you dig down into the guts a little more easily, at the cost of your graphs looking "older", since it's based on Trellis graphs from SAS.
I will agree on reproducibility and portability across graphics devices (not to mention actoss installations). Grid takes care of so much annoyance in that regard.
https://cran.r-project.org/web/packages/gridGraphics/
I've said it before, but the attempts to replicate ggplot2 usually fail to implement the full stack. As a result we have somewhat incomplete implementations of surface features of ggplot2, without the depth that is afforded by the grid / ggplot2 combo.