There is a large overlap, for instance, the popular VIME exploration algorithm [1] uses part of the free energy objective function.

However, free energy isn't a theory of curiosity per se, its posed as description of self-organisation. It just so happens that you can express the free energy functional in terms of epistemic (curious) and instrumental (reward) components.

[1] https://arxiv.org/abs/1605.09674