The big difference is that Handlebars always produces strings that your browser has to parse as HTML. Handlebars also doesn't know how to mutate between states, meaning if you add a class 10 levels deep in the template the best you can do is re-render the entire thing.

JSX + React produce functions that return React's representation of the DOM, their "virtual DOM", and React knows how to make small mutations based on state. If a change in state only needs to add a class 10 levels deep in the hierarchy, that is the only change that happens in the real DOM; it doesn't have to re-render the entire template.