In order to reach LIGO's frequency band, the objects must be very compact. If they are too large, they touch before they can orbit each other at sufficiently-high (audio!) frequencies.
The Schwarzschild radius of the black holes in question are less than 100km in size, while the size of a star as small as our Sun is ~700,000 km in size.
Moreover, the compact objects (neutron stars, black holes) that LIGO can observe are exceptionally dense ( > 3x10^17 kg/m^3), while stars have roughly the density of water (1000 kg/m^3) on average, and much less at their extremities. Compact objects can disrupt nearby stars long before they could touch, spreading the "merger" across many, many years (many, many millenia, I suspect), rather than the fraction-of-a-second typical of a LIGO black-hole observation.
For a visual image of the interaction of a large/fluffy star with a compact object, have a gander at a few of these images: https://www.google.com/search?q=roche+lobe+accretion
https://www.frontiersin.org/articles/10.3389/fspas.2020.0002...