How big is the gap? I did some testing & found that so long as it's similar domain (eg don't pre-train for COCO on X-Rays) it still helps. https://blog.roboflow.com/transfer-learning-similarity/

The intuition for this is that lines, colors, textures, shapes are all general concepts that can be learned from a different domain & used in the earlier layers of the model to build up to more complex features.