Isn't that what deploy keys are for? See: http://docs.gitlab.com/ce/ssh/README.html#deploy-keys

Deploy keys give read-only access to clone a repository, and they are associated directly with the repository.