You absolutely can check factories into version control. They are exported as 'blueprints' and many blueprint management sites exist. One such example: https://factorioprints.com/
Most tools operate on the compressed, binary header including, base64 encoded blueprint strings, not the raw blueprint JSON payload.
It would be comically trivial to setup a .gitattributes filter[1] to do the conversion for you on checkout (to the base64 form) and commit all JSON, allowing for beautiful diffs.
It's a bash one-liner[2] to turn JSON into their base-64 form and into your clipboard anyways.
[0] https://wiki.factorio.com/Blueprint_string_format [1] https://git-scm.com/docs/gitattributes#_filter [2] left as an exercise for the reader