Sure you are, in two ways:

1. You are using and supporting the same protocol that they are also using. As the application grows and improves because more people are using and supporting it, the criminals are also being helped.

2. You are creating cover. The more people that use Tor and Matrix, the more secure it becomes for all users within the network.

If you and a criminal are both using the same Matrix server, neither you nor the host would know. Your plain text messages are going into the protocol, getting mixed and encrypted, and then spit out the other end.