This is the true legacy of React, and it's awful. It's not just a case of "you're doing it wrong", because even cutting edge codebases (developed in the last 1 or so) are struggling with this.

"useMemo is the solution" many will tell you. Compared to the way things are handled in Vue, useCallback and useMemo are near enough code smells for something rotting in the center of the React codebase.