They're a scrappy startup, and don't have much legal firepower. They're having hard time find people, so I agreed to help them. But after reading the comments here, I think it's better to back out instead of having potential future legal troubles.
And if your project becomes big money, they may sue because they are still struggling to get by and you aren't.
So I think your impetus to turn it down is the correct one.