Maybe a restrictive license may help: if your company takes the code and modify it, make them under the obligation of releasing the source code too.
What is the current license of your project?
Ultimately, nothing prevents you from deleting your public Git repo. You can decide that your project is not available anymore, it is your right.