1 word in beta invite email subject reduces conversion by 14% | Hacker News Reader

1 word in beta invite email subject reduces conversion by 14%

tonywright.com

15 pointswebwright12 comments

Hey All-- thought any startup folks out there might find this interesting. We're rolling out more and more RescueTime invites and I saw conversion on the invite email plummet on the most recent invite (with the only change being removing 1 word from the subject).

Open articleView on HN

I should add that this word cost us 89 users directly (14% less people opened the email, 91% of people who open the email convert to an account).

I don't yet have the volume of data to understand word of mouth virality, but we see a big spike of signups every time we do a batch of invites, so losing 1 user costs us more than 1 user...

I think it's hard to form a conclusion from a sample set of 1.

If you were to test this though, I do think your theory would hold mostly true, at least to some extent.

SEO and other modern marketing "sciences" have shown us that minor changes in text can lead to major changes in how people read/consume/value/react.

Well, we did two rounds of fairly big invites (500+ each time) with the original language, and both hit that 65% number. The most recent (just a few weeks after the previous invite round) was where we made the change (700 invitees). But yaw, I'm suspicious of any conclusions drawn from stats like these. At the end of the day, all I could offer was my hypothesis. :-)

Were they both sent at the same time of day / day of week?

Knowing the significance of good copy and subtle changes is invaluable. Almost any site that hasn't put thought into their registration flow and copy could be tweaked in a day and increase signups by 20%+. The same kind of thing absolutely applies to email. Realizing this and making it a part of everything you do is a so valuable that it pains me when friends don't get it (but brings me joy when competitors don't).

FYI: Tracking "opens" is inherently inaccurate. An invisible image "beacon" won't automatically load on a great deal of mail readers.

Yaw, I shoulda mentioned the way CampaignMonitor tracks that... I amended the post via the comments.

If you were to post it on reddit, how about this ;)

Finally, the single most amazing tool that Ron Paul could use to be more productive [pic]

Lets see how it goes :) http://reddit.com/info/2z6fy/comments

LOL. I intended it as a joke, you know. In reality, though, a title like that would probably get many readers to click, but once they realize that the title is misleading, they'll down-vote it.

Interesting, out of 28 voters, 9 voted up :-)

It would have been very helpful to send half the emails with 'finally!' and half without, at the same time.

Next beta, I guess...

Good idea! It hadn't occurred to take this as an opportunity to experiment until the most recent batch yielded such interesting results. I'll definitely A/B test next round and post the results. We've got several thousand more people to invite (in blocks of 500-700) and 5-60 more signing up per day, so.... Plenty of opportunity!

Reply on news.ycombinator.com