How does it gets transferred from the swirling gas to whatever degenerate state of matter is inside the neutron star? What kind of interaction is that?
It's a consequence of the laws of physics being the same for all directions.
Noether's theorem can be paraphrased as:
If a system has a continuous symmetry property, then there are corresponding quantities whose values are conserved in time.
If the laws of physics are the same even if your coordinate system is rotated that means there is a continous symmetry for that. When The conserved property is written out is is angular momentum.
If that mass comes down at an angle to the star (and it's swirling, right?), the kinetic energy gives a sideways push where it hits, like someone brushing the side of a football - it adds some rotational energy.
But that seems a trivial answer so I guess you're asking something deeper?
Maybe a better one would be a spinning ice skater: going from wide arms to closed arms increases angular velocity.
Indeed consider the ingress mass and the neutron star as a single system, ingress mass gets closer (as the ice skater arms do) and angular velocity increases. Collapse is what makes it spin faster.
(but again I may be wrong so please correct me)
Generally, matter that's near a large mass is circling in a disc; if it isn't, then the particles will collide with each other until they've averaged out their motion. So, relative to that large mass, there will be a whole lot of angular momentum in that matter. The angular momentum doesn't go away as it approaches the mass; if it manages to hit the mass then the angular momentum will be added to the mass.
Fun fact: it's hard for us to send an object to collide with our Sun, because you have to cancel out all the angular momentum that everything on Earth carries.