How one family built Qualtrics to an $8B exit

"analyzing every aspect of the customer experience to drive loyalty and referrals"

I guess as companies spend vast amounts to get visibility online (e.g. where Google and Facebook revenue comes from) they need to be able to measure what they are getting for all that money.

It may be helpful to look at one of the academic fields underpinning their work. Take a look at Industrial/Organizational Psychology: https://en.wikipedia.org/wiki/Industrial_and_organizational_...

Qualtrics basically brings the concepts from that field down into SaaS products and makes them more approachable.

It does depend a what level you are working at for example my main client is a major beverage supplier.

They have 200+ brands sold over about the same number of markets.

Its useful to provide high level dashboard summarising key metric.

I'm not so keen on the marketing fluff either, but there are definitely needs being met. The cases I'm most familiar with are in psychological research, which consists of having people fill in a lot of questionnaires (asking about mood, stress-levels, or testing cognitive skills). Qualtrics provides a service to easily build these questionnaires, send them out to respondents, and collect their response (and export the data for later analysis in packages like SPSS). With a little bit of extra work, you can even do highly interactive tasks like Stroop through Qualtrics. For the average psychologist this would simply be undoable without this type of software.

Similarly, class-action lawsuits often require sign-in through filling in some sort of questionnaire. Previously, these questionnaires would be handled through live phone interview, which is obviously very intensive for both the interviewee and interviewer. The questionnaires are often quite interactive, with different questions being asked dependent on answers to previous questions, or aggregates of previous questions. It would take skilled interviewers a while to learn to interview for any one questionnaire. Qualtrics (and similar survey software) makes this process much, much simpler.

You then also have your regular marketing research, the value of which can be debated... but it is obvious people are willing to pay for the creation and distribution of such questionnaires.

Beyond this, I'm sure there are a bunch of other applications of the software.

There's a few things involved, and they're not all BS!

You want to build a couple new features. You have 60 person-hours at your disposal. Which one do your users want?

Build two quick prototypes and see which one people actually use more.

You need to convince people to give you money (aka "get customers"). You're not sure what the best way to convince them is. Try 3 different approaches (Landing page with tech details? Youtube ads with customer stories? etc) and see which one works before you spend all of your budget on something nobody uses.

Out of a thousand people walking down the street, only one _might_ be interested in what you do. How do you make sure you focus your efforts on that one person?

Or "we have a million f2p video game players. 99% of them will never spend a dime. 0.1% of them will buy a $100 package once. 0.01% of them will buy a $9.99 package three thousand times. When should we present this package to them? Will we sell more $100 packages if we ALSO have a $200 package? Will we sell more $100 packages to Jane Doe who takes the bus to college at 8:45 AM and plays for 34 minutes 5 days a week?" (obviously that one gets a bit more troublesome)

The above comes with some severe implications re: privacy, filter bubbles, etc. but it's not there for no reason. Though there's plenty of people who just want pretty charts for their own sake; having worked in analytics the toughest question is "we have these charts.. umm... so what now?"

How does your organization decide what the chip you’re designing does?

Often for chips it’s a B2B transaction so you can probably mobilize some of your go-to salespeople or sales engineers and ask them what customers are saying about next gen.

Now what if your customer base were 100x-1000x larger? You might need automated tools to ask your customers what they’re worried about, what issues they have, whether or not they like certain features, what what the relative make-up is of the customer base.

We all know customers don’t always tell reality so this becomes one data point in many in determining product direction.

"data driven decision making"

Here's a real-world example from a client who uses Qualtrics heavily. At regular intervals following vehicle purchase, an automotive manufacturer sends surveys to vehicle owners asking about their perception of the quality of the vehicle, issues that may have come up, features that were not explained properly or don't work as expected, etc. We're talking hundreds of thousands of surveys a year, each with 20+ branching questions. The surveys themselves are built in Qualtrics (something that it is quite good at), managed by Qualtrics (yes, survey management is a big deal), and the top-level data analysis of the survey results is done in Qualtrics as well to determine if there are problematic trends that need to be addressed, more issues than in previous years, segments of customers who are for some reason perceiving quality differently than others, opportunities to redesign or better market things in the future, etc. This data is directly used to update products, training, marketing, etc. at the company.

This is, in my opinion, the biggest use case for Qualtrics - customer experience research based on surveys or data gathering methods managed from within the platform. You can think of it as Survey Monkey on steroids with some SPSS-lite data analysis functionality built in.

You have the advantage that you are building for a very small number of people who are a) very like yourselves, b) highly trained, and c) strongly motivated to figure out and help you understand what they want. Even so, I'm sure figuring out what your customers want isn't trivial. I imagine you put a lot of work into it. But that work can involve long discussion with customers. What if none of those things were true?

A more typical business is building for a wide variety of people from heterogeneous backgrounds and selling during relatively low-touch encounters. There if you want to know what is going on with customers you need ways to find out and ways to sift the data into useful forms. It's not an easy problem.

I consulted for one company in a consumer-focused service business. They were very energetic users of Net Promoter Score [1] surveys as their default measure. New product? Everybody gets the 1-question survey option after using it. They'd see how that compared to their other products and to competitors. If scores were too low, they'd dig in with other research tools to find out why; new product revisions would be made and measured similarly. It worked very well for them, in that it kept people focused on actual customer experience and not on the HiPPOs [2].

It was such a core part of their culture that they went for related tools internally. When they were changing their software development process, they used NPS scores and other internal surveys to see how it was working for their thousands of developers. To my surprise, I saw the executives taking the numbers very seriously, well aware of the limitations and risks with surveys. I expected to be kinda BS-ish, but I didn't see that at all.

As to your terms:

Value proposition is basically, "What's in this transaction for the customer?"

Customer segmentation is where you notice your customers respond differently to stimuli, so you divide them up into groups. For example, just yesterday we were talking about fancy LED Christmas lights. [3] One customer segment is general-audience consumers who want fancy light displays at the press of a remote. But our customer segment is people who like hacking things, the computer-skilled DIYer. So the value prop for the first segment is something like "make your Christmas tree interesting and different", while the value prop for the second is "this year, hack your Christmas tree!" There are other segments and other key value propositions, which is why there are more kinds of Chrismas lights on the market.

Targeting is just taking particular actions in terms of a particular customer segment. For example, if were making hackable Christmas lights, I wouldn't buy a superbowl ad. I'd use very targeted advertising to reach my computer-skilled DIY market. As well as content marketing, where I'd have somebody write good blog posts about exactly how to hack the lights, so that the posts would end up on HN, Slashdot, Reddit, etc.

Is that helpful?

[1] https://en.wikipedia.org/wiki/Net_Promoter

[2] https://www.bbc.com/news/business-39633499

[3] https://news.ycombinator.com/item?id=18428566

The Experience Economy https://stratechery.com/2018/the-experience-economy/ post on stratechery provides just that explanation.

Imagine if the true performance of your chips, instead of being something you can validate through simulation or measure in a lab, could only be measured in the brains of tens or hundreds of thousands of people.

For every chip you make, you have some expectation of how it will perform, which is its "value proposition."

You want to see if your chips are performing as you predict (fulfilling the value proposition) so you need to interact with a large number of the people whose brains are measuring the true performance of your chips. Since you're dealing with thousands of measurements, you summarize them using math. That's where "stats" come into play. A "metric" is any quantity that is supposed to reflect a measurement, but it usually refers to one specifically designed to be useful. For example, for your chips, the 99th percentile power draw experienced by your users might be a metric you use to guide your design process.

Imagine further that your chips perform differently for different groups of people. Figuring out the best way to divide people into groups to understand how your chips perform is "market segmentation." You probably care about some groups more than others, and depending on how well your chips perform for different groups, you may need to strategically change those priorities over time. That's "targeting."

Meanwhile, different versions of your chips are continually becoming available to people, and you want to detect as soon as possible if a new chip has an unexpected performance regression (or an unexpected improvement!) So you make the latest metrics available in a way that's quickly visually digestible, usually at the expense of all subtlety, but hey, your brain's executive function finds it gratifying. That's a "dashboard."

We (a market research/consulting house) use it pretty heavily to survey engineers to ask what types of technology they're using. I'm sure academics use it similarly - put in a list of emails, program and send out surveys, collect data, analyze said data, create reports.

The alternative was some god awful platform without a WYSIWYG editor where you have to code surveys by hand, an enormous waste of time and very inflexible compared to Qualtrics.

It helps in delivering a superior experience to customers which ultimately is one of the only things that really matters in business.

Where I went to graduate school, it seemed nearly every Psychology graduate student used Qualtrics to administer their surveys. When you are sending surveys to 100s-1000s of people, having a tool that manages the survey and helps with running the statistics and analysis is really helpful.

Qualtrics is HUGE in academia and enterprise. I hadn't heard of it either until I went to grad school and all the professors used it.

Qualtrics is pretty popular among their initial target audience mentioned in the article: academics. Among academic survey shops this product is the definite go-to.

I imagine, based on my own experience, the average tech worker thinks of surveys as small sets of questions used to easily and quickly collect a few data points. SurveyMonkey is fine for that. But for the types of real surveys that provide the data for serious research, Qualtrics is what you need. I've used it for government surveys, sociological, health/medical, etc., web and in-person, with lots of flow control, randomization, and all the other features needed to produce robust and accurate data.

Lots of people think this way. These are not tech companies; they are enterprise sales organizations.

Qualtrics looms large as a tech employer in Utah Valley, where they're based out of. I went to school in the area and almost ended up doing my first internship there.

Every scientific survey I’ve seen was on Qualitrics. SurveyMonkey seems to be used in informally polls like K12 student projects and group activities.

I am very happy to see this kind of success stories for multiple reasons.

1. It gives a great morale boost so many other not so sexy US places outside of silicon valley. YES big success possible outside of Silicon Valley.

2. Now with this big SAP investment, UTAH will get considerable 'tech employment' boost longer term instead of same old crowded Silicon valley & New York (sky rocking housing prices)

3. The way the company grew organically for long time instead of hitting VC money from day one (another big company that did was Atlassian)

4. a very rare father-son founder company and a family company (brother and many of family members in big roles)

> both his parents held doctorates and his father lectured about market research at the University of Oregon. The Smiths moved to Utah around the time Ryan’s father opted to work at Brigham Young University, and in 2002, the pair started Qualtrics, originally targeting academics that needed to conduct field research.

Please remember that Skype was bought by MS for 8.5 Billion (yes, crappy Skype). Of course it was bought for it's prevalence and big existing audience(i'm guessing). Certainly not for it's great UI/UX and Communication capabilities (I hate Skype).

https://www.wired.com/2011/05/microsoft-buys-skype-2/

So anything marginally better could certainly be worth much, much more (as in this powerful Qualtrics products suite).

Haha, I did the same thing when I was in grad school! It's so interesting to see how while we've gone on to do other things, they are still keeping their eyes on their (well-deserved) prize.

May I know what was your needs that required you to code your own survey?

The simple answer is that they (mostly) serve different markets

I heard that what you have to pay for a company is something like what the company will be able to make in 20-30 years.

Not sure if you couldn't build it your own for that kind of money.

From the article:

> SAP CEO Bill McDermott preempted the IPO with an all-cash offer that was more than 75 percent higher than the company’s projected valuation. McDermott said in a conference call that SAP had to pay up because Qualtrics roadshow was going well.

It's explained in the article

>Qualtrics had filed for an initial public offering in the U.S. and was planning to raise about $500 million. SAP CEO Bill McDermott preempted the IPO with an all-cash offer that was more than 75 percent higher than the company’s projected valuation. McDermott said in a conference call that SAP had to pay up because Qualtrics roadshow was going well

So bear in mind:

1) those are 2017 numbers, so if we assume they grew by the same amount, they’re over 400m this year, which makes it only 20x

2) the article mentions that SAP is paying a 75% premium over the IPO valuation, because the road show was going well, they felt like they had to do it.

3) a 10-12x revenue valuation is not unreasonable for publicly traded companies

So, they’re growing like crazy, IPO plans going great, Sap comes calling and qualtrics basically says “why should we sell? We did all the annoying IPO work, what are you going to offer us to make this worth our while?”

It’s basically the perfect negotiating position.

It's not normal for stable companies to be bought at 20x their stable revenue.

However, if the revenue is $190m followed by $290m followed by $400m, then that shows that the expected future revenue (which matters for the valuation) likely is much larger than $400m, so it takes a much smaller, more reasonable multiplier to reach $8B.

MongoDB has a similar 20x ratio.

I won’t argue if this is warranted, but companies like this have:

- Very high gross margins once you take out fixed costs. (80+%) This implies massive profitability once you exceed fixed costs.

- Negative net churn. (Their existing accounts grow more than they lose) This implies if they got rid of their sales force, they could still grow.

- Their payback on sales investments is quick, usually less than a year. This allows them to heavily invest in topline growth.

When you do all this, the math works at 10X in a low interest rate environment, or if you project a very long growth period. Perhaps 20X is Fear Of Missing Out?

The stock market’s reaction to SAP (Knocking off 5 non in value) suggests that 8-10X is more reasonable.

I'm not very familiar with IPOs or buy-outs, but for going concerns it's usual to look at the price-earnings ratio rather than the price-revenue ratio.

Ford, for example, has revenue of $37.67B last quarter but net income of only $991M. With a market cap of $37.31B, that means a price to earnings ratio of 9.5.

Microsoft had less revenue, at $30.08B last quarter, but net income of $8.87B. It has a market cap many times that of Ford, at $841.08B both because it has so much higher earnings and because it has a PE ratio of 23.9, which implies that the market things those earnings will grow at a good clip.

Given $8bn valuation, I think Qualtrics is spending heavily on marketing which keeps their net profit low. I think their operating margin is near 50% (being a software company it is possible), which means about $200mm in operating income. If their growth rate is expected to remain at the same level (40%) for the next two years, $8bn valuation can be justified. It's hard to know whether this is the case, but seems likely.

M&A guy here and disclaimer, former SAP employee.

It's not normal, but you have to define what normal is. A few things worth noting (which are all speculative btw):

- SAP doesn't have a solid CX platform, but has tried to push this to the market for years because the demand is there

- SAP has arguable the strongest foothold in the top 2000 businesses in the world.

- SAP once organically grew their ERP products but decided somewhere around the late 2000's that it couldn't innovate any longer but know cloud/SaaS is the future, so their growth has been largely due to acquisition of cloud/SaaS providers. They want to continue to tell the street they're growing subscriptions and this is one way to do it.

- SAP sales is top tier (whether you like it or not). If their revenue is $400m they'll easily be able to sell 2x that in short order to their existing client base. This is what M&A guys often call "synergies". If you assume 30% eventual gross margins, then a potential ROI is foreseeable.

That all being said, if the market for this type of service isn't there at enterprise scale ($1B/yr revenue), then it's probably overpriced.

> Whenever all those "where are the other silicon valleys?" articles come up, they never seem to bring up the Wasatch Front

Huh, interesting. This is exactly the opposite of my experience. I would have said those articles always bloviate about Silicon Slopes.

Well HN is just that, hacker news. The focus isn't on DIY and bootstrapping yourself. This is more a niche that IndieHackers and the like fill in.

If it helps with your inspiration, here's my story: I launched my company (Wingify) on HN when I was 22 and bootstrapped for last 8 years. Today we're not Qualtrics scale, but we're doing ~$20mn and have never made a loss in our history.

My guess is that businesses like mine are more common than VC funded ones but we only get to see VC funded ones because that's what the news covers.

If you haven’t, you should checkout IndieHackers.com. It’s a community for makers who are building revenue-based businesses without taking massive funding rounds. Most are just getting going with some initial success. Their stories will probably feel more relatable and inspiring.

Qualtrics is an exception to the typical unprofitable unicorn. they've been cashflow positive for over a decade.

(I was a growth engineer and an admin for the Stripe instance, Their stripe account had lots of money flowing through and that was just for sub-5k deals.)

I think what you're referring to is a reverse merger instead of an IPO -- This is entirely different, just a standard merger transaction. Reverse mergers are when a private company that is trying to go public (often with shaky financials) "merges" with a company that is already public, usually on some OTC board somewhere. This process avoids the scrutiny of the S1 / roadshow / etc and allows a cheap and easy way to 'go public'. Typical mergers involve a lot of due diligence from the acquiring company, they can still go wrong but it's a much different proposition.

https://en.wikipedia.org/wiki/Reverse_takeover

Many of the Chinese 'fraud-cap' companies were taken public in this way: https://www.nytimes.com/2011/07/24/business/global/reverse-m...

I've been to Ryan's house. he has a basketball court in his basement. I'm sure he paid himself a lot more than $100k/yr as some point. (not saying he doesn't have every right to)

As a software engineer in the "Silicon Slopes," I can say that we get multiple offers almost constantly for companies throughout the area. There is enough tech here to keep people employed where they want to be making the money they want to make.

Private tutoring is illegal?

It's not the "rolling out surveys" part that's considered valuable, it's the "data" part. These companies don't just drop a survey creation UI down and get $8B. They collect data, lots of it, and provide services around that data. The survey is just one mechanism they use to do that.