These foundational models have already been created. Government backing has already been used to create a fully open source model. The effort has been multi-national. Government licenses aren't currently required to run one.
"An award of computer time was provided by the INCITE program. This research also used resources of the Oak Ridge Leadership Computing Facility (OLCF), which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725."
Models:
https://www.together.xyz/blog/redpajama-models-v1
Training Data: