If the repository is personal one managed only by you, you can squash the commits so the large files disappear from the history.

You can do that on shared repos too but that would cause a bad headache to other maintainers.