Heard an interesting idea about the overlap of information theory and physics in one of Sean Carroll's recent podcast discussions. They blew past it but I think it bears its own subject.

The idea goes like this. If you want to talk about simplicity, beauty, or elegance of a physical law--maybe because you think that's more likely to be correct than a complicated law--then we've already got tools like Kolmogorov complexity to talk about such laws.