Yes, you're right if you don't have remote storage enabled, otherwise it should fetch the missing assets back again.
The main use case for this is ephemeral build hosts to share the cache, but I want to work all these issues out too, so thank you for the feedback