I see your point, but the comparison to sftp seems really off. Overlay2 [1] indeed works on whole file differences, just as sftp would do.
Additionally, I don't think making actual byte-level deltas would yield you much improvement on container size. It would also increase access time unless you keep a cache that doubles storage requirements per image. Dockerfiles primarily add files rather than change them, and actual changed files are often small.
[1] https://docs.docker.com/storage/storagedriver/overlayfs-driv...
If you think delta transfers would help, you're saying you have a bunch of data that doesn't change between image builds, but you have to re-upload all that data every time.
This actually sounds like a perfect use-case for image layers. If you can rearrange your Dockerfile to put the steps where files change more rarely before steps that change frequently, docker will automatically cache the earlier steps and not reupload if that layer already exists.
Image layers are at their heart already an implementation of a cache + delta transfer system. (Compared to some other delta systems it trades storage efficiency for computational efficiency.) You can get small delta transfers, but you have to work with docker via your Dockerfile to make it happen.
For example, that could mean: moving steps up or down the Dockerfile so more frequently changed layers come later; splitting a step into two steps to exploit the previous; making a step deterministic so you don't have to force rebuild; use multi-stage builds to make all of these easier to implement.