Ue rem when your reference is the base page. Use m when the refrerence. is surrounding text.

As examples, I size page elements in rem, but text elements (usally), in em.

My main, content, article, header, footer, and aside should all generally reference page, and are in rem.

Main body text width, headings (h1 ... h6), super- and sub-scripts, code and pre blocks (often jarringly sized by default) reference the containing unit and should be in em units. Also, generally, figure and table captions, table text, etc.

I prefer to give figure or sub-element (tables, callouts, note boxes, etc.) margins and paddings in em as well.