Maybe he's not pulling the weight, but like a movie the other day, a Washing Machine works hard, so what.
Think through if he's benefitting the company in his total contribution. If you want more from him, ask. If he won't do it, explain it's unacceptable and go to the board.
Like badclient states, it's going to come down to perception based on asserted facts. Those facts will be interpreted through the lens of relationships.
I have been through this before, I can help if you want someone to bounce things off of real time.