Both ffmpeg and handbrake are front-ends for libavcodec.
The FFmpeg project has 8 libraries. libavcodec only deals with decoding/encoding and bitstream parsing & filtering. Demuxing/muxing, frame filtering, scaling.. are all handled by other separate libraries.
Handbrakes own docs say "HandBrake uses FFmpeg under the hood".