Launch HN: Thematic (YC S17) Customer Feedback Analysis via NLP | Hacker News Reader

Launch HN: Thematic (YC S17) Customer Feedback Analysis via NLP

72 pointszelandiya31 comments

Hi! I’m the CEO of Thematic, http://www.getthematic.com. We analyse customer feedback to tell companies how to increase customer satisfaction and reduce churn.

We are one of the handful of companies that got into YC through the Startup School, and (I have to say) the only company that signed YC itself as a customer!

I have a PhD in NLP and ML and was consulting when two large media companies came to me with a problem: They collect tons of customer feedback in free text as part of their NPS surveys, but don’t have the time to sift through the responses.

This turned out to be common. Most companies collect feedback but, especially in large companies, nobody reads this data, and definitely not people who are in charge of strategy. Customers are screaming what’s wrong and what they want, but nobody is listening.

I tried a few open-source packages but found that none worked well. Developed on canonical text like news article or Wikipedia, they either failed to understand the variety of expressions, or were too hard to explain. I wrote a new approach capitalising on my PhD and new Deep Learning approaches. It's completely unsupervised: just needs raw data but, unlike topic modelling, produces clear and specific themes. My husband Nathan joined as a co-founder and for the next year we learned how to solve this problem in a way customer insights professionals find valuable.

Those media companies became customers and we quickly bootstrapped into a profitable startup. This is when Nathan signed up for YC’s Startup School. We grew 20% in those 10 weeks, loved the accountability and the focus. Our mentor suggested we apply for YC, which seemed like a crazy idea, but we gave it a go.

Fast-forward another 2 months, and we are just before Demo Day! Thematic grew 3x in that time, and we are working with brands like Vodafone, Air New Zealand, Stripe, Ableton, and Manpower Group.

Hope you found our story interesting, and happy to answer any questions.

Suggestions to your business growth:

Surveys are the bread and butter for many market research companies. Most of the corporates / enterprises typically engage with smaller to larger (and many a times multiple) MR agencies. These MR companies can benefit from your service. To explore these companies, you can check out the MR members directory list from ESOMAR ,the voice of the MR (www.esomar.org), perhaps a membership / participating in their events may help you. Other site is agencyspotter.com

Explore publishing an article at the Greenbook blog run by Leonard Murphy which is very relevant to this case and he also runs the IIEX events globally ( http://iiex-na.insightinnovation.org/ ) where tools /services such as yours are very much the hot thing..

Check out Unilever Foundry https://foundry.unilever.com/ . You can sign up and explore if you can help solve some of their problems with your solution. They select and fund Pilot projects

Twitter hashtags to get your tool noted in the MR industry #mrx #newmr

Ads / promotions: check out http://newmr.org/ , http://www.greenbookblog.org/

Explore if you have complementary business synergies to present with https://www.zappistore.com/ (e.g your product could be a part of the Zappistore platform as an App.)

Best Wishes, N.Sankar https://www.linkedin.com/in/nsk007

Wow, this is super helpful, thanks! I will check out these resources.

Congratulations and best wishes!

I for one really liked the demo and the blog - specifically, (a) I have great exemplars for what you mean by "theme", and (b) this post[1] shows great insights into your thinking about the problem faced by your customers

> Developed on canonical text like news article or Wikipedia, they either failed to understand the variety of expressions, or were too hard to explain.

It appears to me that the current methods and resulting tools are heavily dependent on the problem formulation (or domain in general). Moreover, no matter how fancy your technique is (or "how deep is your net"), the resulting model won't work unless you take specific steps to train it on data from the domain.

Yes, what I just said sounds borderline truism. However, I am more interested in discussing why it is so? Here's my initial thinking:

Let us look at (one of) the definition of Machine Learning, from Prof Tom Mitchell's textbook, "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E."

Here, experience E can be loosely considered as the amount of data you have for training - obviously, more data (i.e. training) should improve learning. However, the abstraction of T and P hides an important underlying problem of specification - or in other words, formulation of T (and E).

Thoughts?

> I wrote a new approach [capitalizing] on my PhD and new Deep Learning approaches.

I hope we get to see some of your insights in a paper or article (or blog post :)

[1] https://www.getthematic.com/post/visualizing-customer-feedba...

Thanks for your thoughts! Definitely sounds like an interesting thought to explore.

I like to think about ML in terms of how children learn language: through observation in their environment. The training data is a simulation of that environment.

Glad you liked the website and the blog post!

I really enjoyed reading about your company on Idealog (https://idealog.co.nz/technologymonth/2017/08/meet-thematic-...). It was especially great to see how you bootstrapped and found product-market fit rather than just raising a bunch of cash for an idea that may or may not be applicable in the real world. Congratulations on the launch and your success with YC. Go Kiwis!

Thanks so much! Appreciate the kind words. :)

Congratulations on your launch! This is very interesting and it's something that has great applicability. Being in financial services, we collect a ton of feedback and audio but not enough manpower to process it all. I love the themed approach but am curious about the subthemes. Would you be able to shed any light on that?

I'm also happy to make introductions if you're ever thinking about expanding up north to Canada.

Subthemes are basically more specific versions of the themes. Check out an example on our Twitch demo: https://demo-twitch.getthematic.com/#!/demo/main/dashboard/2... You will see a base theme "use chat", with subthemes like "chat in landscape mode", "chat on a tablet", "type in chat" etc. Does this help?

Thank you so much for the offer of introductions. Definitely interested, as we sell internationally, not just in the US.

Thanks. Would subthemes almost/always fall under the context of the overarching theme?

Great tool, this is an area I've been looking forward to more automation in.

I might have missed it on the website, how does pricing work?

Also, do you have any integrations with other tools like Intercom or Zendesk to ease data-sharing? A monthly insights report generated directly off of my main customer support tool can replace hours of manual work.

Thank you! Our pricing is custom, it's a subscription model. A monthly report is quite a common use case. We have a Zendesk but not yet an Intercom integration. Happy to talk further, please feel free to request a demo via our website getthematic.com

Love this! When I was at one of the big tech companies years ago, we tried to do something like this for reviews for all the products, and see how they stacked up in the marketplace. Was a really challenging problem to do in an automated way at scale -- definitely wish we had a solution like this!

Do you guys see yourselves sticking to a model that spits out analysis, and let customers decide what insights to gain from the data? Or could there be a path where eventually it lets users take specific actions based on the data?

This is a really interesting point. I think what you mean is operationalising the analysis. We let customers decide what those actions are and then provide integration to help this happen. For example, one customer has a call center that receives an email from Thematic each morning stating which customers are likely to churn based on themes they mention. They call them up first.

Ah, that's awesome -- the point on integration especially is a big one.

The more ways clients can hook the insights into their CRMs/workflow/user base, the more they can "operationalize it", as you eloquently put it, and make it a part of their workflows, the way the client you described does with their call center. I love it!

How do you go about evaluating the accuracy of the themes and action items? Do you create a test set with obvious themes and actions and check the results for example?

Good question! Accuracy of themes is subjective. We run several studies where we used opinions of several people on what the themes should be and then compared the overlap between people with the overlap Thematic has with each of them. This is also referred to as "indexing consistency" in Information Science.

You can check specific examples in our white paper: http://www.getthematic.com/net-promoter-score-verbatims/ (Don't need to download it, the bulk of the results is published on the page.)

In practice, some customers have read samples of responses and expect certain themes. Others have manually tagged them in the past. They compare their results with their fundings and can see straight away if our results are accurate and reliable.

would you say that your solution provides a sentiment analysis, similar to Quid, or do you focus on action items and things that product managers can actually address?

We do provide sentiment but the main benefit is definitely the themes which form the action items that can be addressed.

We show what areas to improve, not just how people feel (although that is important in deciding)

Awesome, thanks!

Great product and congratz on your success so far!

Would you mind please elaborating a little more on how you're thinking about theme-specific sentiment (for the NLP students here :) - is it along the lines of Socher's aspect-specific sentiment or something else?

I saw there were samples but is this similar to how BloomBerry comes up with "topics" for popular questions for a keyword? They use some sort of NLP extraction of noun/verb phrases.

Example: Most popular topics/themes related to Vodafone: https://app.bloomberry.com/questions;q=vodafone

Thanks for pointing out! Bloomberry has some useful themes, yes, it does look similar. Good to know they exist.

This is super cool, I also appreciate the demo's you have on your website (airlines, MBA schools etc.). Makes the end result super clear.

I don't know much about NLP but are you only using unsupervised learning on the raw data? I would think you would need an NLP layer as well that sorts out basic synonymical issues, phrasing differences etc.?

Thank you! No training data is required to use Thematic and customers don't need to tell us what they want to find in the data. Hence we say "unsupervised". And yes, we extract synonyms and paraphrases from raw data.

However, we do have tools to review and adjust the results of Thematic by hand. For example, we have an internal drag and drop interface. Some customers really like to change the themes based on their view of the data. But it also helps to remove any inaccuracies, e.g. an incorrectly merged theme.

Are you planning to provide your service as an API to other companies that want to wrap a product around your work?

No, not currently, but maybe in the future.

Congrats and good luck!

I work on the same, just for my own company to automate customer interaction (well, at least 99% of it).

Thank you! Yes, there are many applications beyond customer insights. We are dabbling into these also. What's your company?

Congrats on the launch! I think I saw you talk recently in NZ, sounds like you have an exciting path forward

Cheers!

You guys hiring? I'm not a PhD or anything, just an undergrad with an interest in NLP.

Reply on news.ycombinator.com