It seems much more likely that the direct controller is sex hormone levels, not chromosomes. Though chromosomes may affect what levels of sex hormones your body produces naturally, hormones are what actually control dimorphism in the immune system[0]