Ask HN: Oprah's effect and scaling

Are you familiar with how middle-aged women actually use the Internet?

Step #1: Make sure you dominate rankings on "the Googles" for every conceivable way someone with half an ear on the television could mangle your name or category. She isn't going to product.com, she is googling for [product], [product.com], [products], [praduct], [pro ducks], [what was that thing on Oprah this morning?], etc.

Step #2: Expect less traffic than you think, because while Oprah's audience is ginormous they are in TV watching mode rather than browsing the Internet mode, and large portions are going to fall out of the funnel in a) trying to go to your website and b) doing so successfully. Oprah's peak audience is somewhere around 9 million people. If you're expecting 500k visitors to your website, I think you're being overly optimistic (absent Oprah literally doing a call to action on your behalf). So it isn't all that much worse than a Slashdotting, and you can expect the people to come in over a wider interval than "right the second the URL appears on the screen". Any reasonable website can chew through 500,000 visits spread over a 48 hour interval if it survives the initial bump.

Step #3: Scale the obvious bottlenecks and get ready to do some firefighting, though you probably won't need to. If the home page makes 10 SQL queries right now, a) fix and b) replace with static HTML for the day (or get a process in place where you can do that quickly if the server starts to chug). I'd put a big Welcome Oprah Viewers banner at the top of it with a sales pitch leveraging that social proof for all you are worth and giving them "special" advice for using the site, by the way.

Step #4: Have a backup plan. If it seems highly unlikely that you're going to survive, switch all or most users to a static site which explains that "due to overwhelming popularity" the service isn't available at the moment but you'll email them the second it comes back up if they just drop their email address into this MailChimp signup form. Do so. Give them an option from hearing from you in a less transient fashion, too. (I would strongly consider "Check if they're cookied for your site, if so let them in, if not give them the static pseudo-site" but that doesn't scale to the moon like "hello new client, here is index.html, bye" does.)

I'd put a big Welcome Oprah Viewers banner at the top of it with a sales pitch leveraging that social proof for all you are worth and giving them "special" advice for using the site, by the way.

^^ This. You are in a unique position of being able to PLAN for a huge spike in traffic. While planning for the technology impacts are important, do NOT forget the business aspects. You already have a damn good idea of the demographics of more than 100k visitors that might be visiting that day, so get started now.

Do you want to offer a special "discount" for Oprah visitors that day? Does the signup process need to be simplified for less tech-savvy visitors? I definitely agree with Patrick that you want to have a targeted set of instructions or otherwise customized copy for the visitors that will come that day. Above all else, PLAY UP THE OPRAH CONNECTION! A banner, special icon, her picture, etc. Visitors should be smacked in the face early with some indication that they have found the site mentioned on Oprah.

You might also want to find any Oprah-related message boards and create accounts now. Then on the day the site is mentioned in Oprah, go to those message boards and post a clickable link. Not only will this help the visitors get to the correct site, but they can use it to share the correct link with OTHERS.

We've had all kinds of press for our site, radio, tv, newspaper, etc. We get bigger spikes in traffic being featured on blogs nobody's ever heard of with a link to our site than being featured in national outlets like NPR and LA Times where viewers have to remember to look us up online later. That said, the people who hear about you offline will find you weeks and months later and sometimes make a purchase over a year later. I agree, put a "Welcome Oprah Viewers" banner on your site. Also, you can run an adwords campaign for a few weeks to capture Oprah's viewers. Set up your keywords keeping in mind they're going to be googling phrases like "website selling chocolate chip cookies (or whatever it is you do) as seen on oprah tv."

Thanks patio11

1. Good thing is that I don't think there'll be any problems for the mangling the name as it's a very simple name :-)...

2. It makes sense that the traffic is going to be a wider interval, hadn't thought about that

3. Yeah, I've already done that, all non logged in pages up to the signup form are static HTML. Hadn't thought about putting a Welcome Oprah Viewers banner but it definitely makes sense, will talk to my customers about it. Thanks :-)

Regarding #1, in my experience this isn't a phenomenon limited to middle-aged women. That's pretty much how "normals" use the Internet (as much as I dislike that term). This is what regular people, and quite a few techies, do.

Oddly enough, I worked on a project that had a whole show dedicated to a site that I built. There were banner ads on Oprah's website and a bumper in the show with the URL on it.

My boss and I sat watching and waiting and we saw virtually no effect from the show whatsoever. (If you were just watching top on the servers without knowing Oprah was talking about it, you would have assumed it was any other day.)

We spent time theorizing why this was. We came up with the following:

- The site that we built has a relatively long URL, so maybe it was hard to remember.

- The bumper that the URL appear at was towards the end of the show, at which point we figured most stay at home moms/dads/people that watch Oprah were probably starting to get dinner ready.

- I'm not sure that people that watch Oprah are the same people that eagerly sit in front of the TV with an iPad/laptop/etc.

Of course, your mileage may vary. If you're using Rackspace, I'd say you're in good hands. It may be worth mentioning to them when it's going to happen and seeing what they can do in case of sudden load, but for us it certainly wasn't the huge gush we were expecting/hoping for.

I had the same experience. We had a customer whose cookies were mentioned on Oprah and then Rachel Ray at a later date. We saw a bump in traffic, but the customer didn't warn us until a few days before air, so our recommendation was to leave the site alone rather than risk introducing changes that could produce bugs. We put a little time in to tweaking the site's error pages to be more friendly, but when the air dates came, nothing spectacular happened. We actually saw a little bit more traffic from Rachel Ray than Oprah, probably because cookies are a more relevant topic for her show.

Oh well. Still, your company can leverage "As seen on Oprah" for a conversion boost.

In a previous life, I was half of the engineering team at a call center. On more than one occasion, one or another of our customers (always a different one) would call us and say that their product/service/organization/whatever was going to be "mentioned" on Oprah "next week", and could we please set up some extra phone-answering capacity for them in order to handle the inevitable spike in demand.

At first, we would try and accommodate their requests- bring in an extra operator or two, do a special answering script, etc. Besides being good customer service, we figured that increased usage on the part of our customer translated into more money for us, so it was worth spending some time on our end helping them get ready for a spike.

After a little while, however, we eventually stopped taking "Oprah-induced capacity panics" so seriously. Why was this? Because we never- not once- saw any serious spike in calls from any of our customers' Oprah "mentions".

I'd suggest planning for some extra capacity, just in case, but don't go doing too much special-purpose engineering work for this one customer. Unless, of course, their "mention" on Oprah is really more of a "entire episode dedicated to their product" kind of a deal- that might be a different kettle of fish entirely.

It's more toward the side of the entire episode dedicated to the product than to the casual mention...

Hey, well, in that case, your mileage might definitely vary from what we usually saw. I suppose it's always a good idea to have plans for this sort of spike, Oprah or no...

First off, congrats! I was in charge of the website when Oprah featured The Secret a few times on her show. Here's a few pointers from my experience.

1. Remember the show airs in Chicago in the morning, then more than once across time zones, so this will be an all day treat. We had a lot of buzz (Oprah's own forums went down under load for our show), and our URL was displayed on the screen. Regardless, you can expect a lot of traffic, mostly during the show and a few days afterward.

2. Your db will likely be your bottleneck. Make sure you don't have any unoptimized queries, or better yet, cache as much as you can so you're only hitting the db on writes for all the orders you'll be getting.

3. If possible, offload static content to a cdn. Akamai saved us here, because the site was not optimized size-wise.

4. What ultimately caused us issues was slow clients. I would definitely try to have a caching proxy up front. The snowball effect would cause us to tap out our 100Mb connection at times. Since you're using cloud hosting hopefully that's not a hard limitation for you.

5. It's really fun watching your bandwidth. I used a console tool called bmon. You'll be able to tell when commercials happen during the show.

6. +1 for a backup plan. Have a static page ready that you can switch over to. I'd highly suggest having an alternate payment processor, even if you have to go with Paypal. Since you're SAAS, you can't rely on retail or Amazon, so you may not get visitors to come back if your site fails. Make sure you can capture those customers the first time.

7. If you don't plan on having a free account, offer something so you can build your customer list and capture contact info. A whitepaper, newsletter, etc.

Have fun!

Thanks a lot for the answers!

1. Can you tell me approximately how many unique visitors you had during the shows? I'm kind of curious...

3. We're using cloudfront... So you'd advise on using Akamai instead? or would cloudfront be ok?

6. For payments, we're using Authorize.net (with the CIM feature to capture credit card information)... Hadn't thought that they could be a bottle neck but I'll definitely at least check up with them...

7. Makes sense, I'll suggest that...

It'll be a fun few months preparing... I've been meaning to play around with Varnish, now's my chance :-)

1. Can't give you any real numbers, but dude, it's Oprah.

3. Any cdn should be ok. You just want to offload those files to somebody else. Remember that browsers will only download a few resources from a single domain concurrently, so even if you have a subdomain for statics you'll get a faster page load. Hint: use yslow for more frontend optimization tips.

6. Authorize.net can handle it--we used them. But if your shopping cart can't, it's always nice to have a Paypal 'buy it now' button ready to go. Not real elegant, but it's a backup plan.

Thanks!

3. I'm a big fan of yslow too... It's really great for that :-)

I have the lucky position of having to do scaling for not only Oprah, but american idol, the superbowl, x factor, inaugurations and more.

we have had artists on Oprah many times, probably a dozen in the last five years. It has nearly no effect. The largest bump we see is during the central timezone broadcast. The east and west code feeds have negligible effect unless you ask for it by doing an "expensive" call to action on the site. I define that to mean asking every user to do a db or server intensive task, like signing up for a site, logging in, commenting, etc. We generally avoid db writes during load events.

I would recommend a good cdn though and offloading as much as possible to them.

Incidentally the biggest impacts for us have been xfactor, american idol, glee, and the biggest was Tom petty on the superbowl. The only time I have ever watched it willingly.

I think part of the tricky part in this is that, most likely, it has to scale up, but not at all gradually, and then scale back down too, in terms of costs, as the traffic dries up again. Good luck:-)

Amazon EC2 might be a good option.

A good approach could be to put all the static stuff (the sales pages, pricing info, everything that doesn't require backend) on 1 domain, and put that on S3 and Cloudfront. That definitely won't go down, and it will be fast and take the brunt of the load.

Then put the interaction stuff (the forms etc.) on a subdomain, and put that on heroku or whatever with plenty of servers.

While I've never been on Oprah, I have had my startup featured (~8 min. segment) on a show on the CBC (Canada's national broadcaster.)

Hardly a blip in load.

We had a few more emails than usual offering "advice," though.

That said, it sounds like a good excuse to take a good look at your caching, landing pages, and conversion funnel. And, if you are worried about needing to scale fast, consider something like chef for automating infrastructure setup, so that even if you don't need it for Oprah, it's work that you'll use when you do need to scale.

heroku?

Heroku is great. However, if you use some other system besides rails iself (like custom processes, things other than DJ for job scheduling or calling process that require root access from you app) you may find it will be hard to migrate. Otherwise it is a piece of cake.

I hadn't thought about using them, I've had bad experiences before with engineyard's appcloud (mostly during their beta and because I needed to do something a bit more complicated than what they normally support out of the box, forcing me to play with custom chef recipes that started failing when they changed their base system)

But, yeah I think I should definitely check them out

Definitely this. Drag the dyno slider up before the show, then drag it back down once traffic returns to normal. If you need any sort of custom background process, you can start up some EC2 instances and access them over the fast, private Amazon AWS network.

Inadequate phone support is possible failure point.

maybe jhh will chime in here as he went through this exact situation @ the secret