GPGME is still just a wrapper around the gpg exeutable and the source code is 961 kB (!). It's not a real "library".
GPGME is a wrapper on top of the GPG executable. It also has a horrendous API. I'm still currently fighting with it just to simply encrypt/decrypt a tarball going to S3. It's really badly designed.
I know exactly what you are going though.
Don't look at the source code of GPG, it will give you nightmares.