I ask this because Fly has immutable Docker containers which wouldn't store any data unless you use Fly Volumes. So it could be that Fly is downloading the 100MB model each time it cold-boots.
If that's the case, a multi-stage Dockerfile could help in bundling the model in, and perhaps reducing cold-boot time even further.
I tested it initially because it’s the naivest implementation. The right implementation would bundle it in.
But I ended up primarily reporting timings that stop counting up as soon as control is handed over to user generated code - since that’s the number you care about the most.
Perhaps a good future idea would be to benchmark between bundling it in the Docker image, vs. using Fly Volumes as Simon suggested in a sibling comment.
So the problem of size is exactly the same, and RUN with curl will have identical size as COPY layer from COPY --from=stage
Am I missing something?
I can see only benefit for build cache reuse, so download is independed from building code so you won't redownload 14G when you change code, is that what you had in mind?
I don't know Docker that well. I literally figured it out as I went along to deploy on Fly...
I didn't wanted to sound snarky, I thought that I wasn't aware about some cool docker optimization hack :)