You are right about large binary files and gitfs currently has an option limiting the maximum file size.