You're right about idiomatic React + JSX; I was incredibly wary of React at first because of the apparent requirement of a JSX preprocessor, to the extent that I went out of my way to use non-JSX React. It was very easy—the ease of opting to do that gave me a bit of extra appreciation of the modularity of React (JSX is 100% a separate thing).

That said, Typescript got me (maybe a little too) comfortable with having a preprocessor: I still use JSX-less React but in Typescript I've actually started using React-less JSX...

CSS-in-JS is silly. I put that idea into the same bucket as a lot of other nonsense in the React ecosystem. Don't.