I did something similar. It puts my server to knees and takes something like a hour to complete, requiring all services to stop before I can run it. This is extremely unergonomic for me.
Building on separate computer will work, of course, but it makes everything more complex.
Have it run on a schedule to keep it updated, if necessary. Alternatively, GH Actions allows you to set a manual trigger so you can rebuild whenever.