> My guess is that, he wants to

Don't do this. Don't assume what might be happening and create stories around that and plan your career on top a big pile of assumptions. If you have concerns, speak to your manager. It doesn't need to be defensive, you don't need to make a big deal out of that conversation, just check in to see how they are doing and let them know how you are. I find it surprising that you don't already have a scheduled regular one-to-one meeting.