What you said is applicable to the loose object store where a full copy of the file is stored as plain text. Those could be deduplicated quite nicely, and git does just that, loose object store is a CAS.
However the packed object store is trickier. It stores duplicated object in a delta compressed format plus gzipped. So deduplicating packfiles on a file system level is almost never worth it.
Git is moving toward using packed object store more and more. With some of the latest patches, you can effectively use git with very little loose object storage ultilization (zero if you are on a server hosting git repositories).