Optimal usage of conversations received in a chatbot

We have created a bot in house without using any platforms like wit.ai or api.ai. We treat is as a text classification problem. We are using word2vec to vectorize the input documents and a linear svc to classify it into one of our pre defined classes.

Also we have a feedback dashboard where the admin can tag and validate incoming user queries with correct class names (similar to what existing platforms provide). This is fine until a certain level but becomes impossible later.

So now we have a few (5k) tagged queries and a lot (350k) untagged user queries. Is there any way we can use these to enhance our system ?