The Impact of the Megaupload Shutdown on Movie Sales
papers.ssrn.com
papers.ssrn.com
The CMU lab is called Initiative for Digital Entertainment Analytics (IDEA), and according to CMU's press release "The creation of IDEA was made possible through a gift from the Motion Picture Association of America (MPAA)"
http://www.cmu.edu/homepage/society/2012/fall/entertainment-...
"The MPAA was interested in CMU in part because of our unique strengths in the performing arts, computer science and technology, and management," noted Smith.
Looks like MPAA also had something else in mind, when they funded this lab.
In my profession (law), proving loss of revenue is so difficult that it is in many cases considered impossible, even in civil cases. You have to take into consideration many factors and you can't just automatically ascribe declining or increasing revenues to one fact, even if it seems plausible. You can't just make up a model full of assumptions and attach fancy names and mathematical symbols to it. No decent judge would fall for that. To start with you would have to make a year-on-year comparison spanning 5-10 years. Then you would have to make a convincing argument why the revenues didn't just decline or increase due to one of the many other possible factors such as popularity of the movies, internet penetration, broadband access, weather, macro economic trends, changes in taste, normal seasonal changes, supply of alternatives (cable tv shows, sport games etc.), changes in payment options for the legal movie outlets, other changes in business model or marketing that could explain why the legal movie outlets sold more after the shutdown.
If the product had been more generic that might have been possible but with movies the popularity of the supply varies so much that changes could often be ascribed to differences in popularity.
I understand providing information beyond that would be likely very difficult, but having never been in a similar research situation, could someone outline what this influence is like? If there an underlying pretense during the research that they have to reach a specific outcome? Is it more subtle/obvious? Any additional insight would be appreciated.
for each additional 1% pre-shutdown Megaupload penetration, the post-shutdown sales unit change was 2.5% to 3.8% higher, suggesting that these increases are a causal effect of the shutdown.
This does not suggest causality. You need more rigorous inductive reasoning, and randomized trials, if you want to conclude causality.
Edit: Just found out that SSRN is not even a peer reviewed journal! ( http://www.ssrn.com/update/general/ssrn_faq.html#no_paper_su... )
It's not really a journal at all. It is a repository for papers in the social sciences, kind of like arXiv is for some other fields.
Most of their target audience is well aware of this, but it might seem confusing if you're coming from outside academia. They're not trying to pretend they're something they're not. Putting pre-prints of papers on SSRN is considered good practice.
SSRN is actually a pretty important source for open access research.
Nobody claimed it was. It's a platform for sharing preprints/working papers, like arxiv. Obviously you don't like this study or its conclusions much, but throwing the kitchen sink at it doesn't strengthen your argument either.
But are these the claims really that outrageous? ie. The biggest piracy site in the world has a slight effect on sales?
Using the same principle, I could posit (probably quite defensibly), that the amount of alleged copyright infringment of Megaupload Limited's limited access and centralized services would be dwarfed by that from Bittorrent and other P2P methods.
Well, there has been some evidence pointing towards there being a positive effect on sales. So not outrageous but controversial still.
This is usually impossible in social science research studying real-world phenomena that can't be reproduced in the laboratory. It's a cargo cult attitude to think all science must look like an FDA drug application. Part of science is making do with the information you can collect.
There is some evidence that people stay away from research that is considered controversial by the public [1]. In my own experience (biologist), a lot of biologists shun away from public debates on more controversial stuff like GMs or creatonism out of fear of being associated with the fringe-elements participating, or are scared of "loosing" their reputation.
There was a Nature survey in which 15% of 3247 scientists confessed of having changed their study-design, methodology or findings because of pressure from a funding agency [2].
More indirectly, this study checked out studies on a fat substitute from Procter & Gamble and found that "supportive authors were significantly more likely than critical or neutral authors to have financial relationships with P&G"... [3]
These are just glimpses, from my own experience I would say that the problem is larger than it's currently being described.
Edit: There is little to no research on what's happening directly with the scientists in a situation in which funding influences findings. Does the researcher self-censor out of fear of loosing funding? Is there a slightly threatening phone-call from the funding agency? No-one really knows!
[1] http://www.examiner.com/article/government-funded-research-o...
[2] http://www.nature.com/nature/journal/v435/n7043/fig_tab/4357...
[3] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1447808/?tool=pu...
Furthermore, and more importantly, avoiding public debate is very different from changing the results of your research. If you are implying that some people have results that support creationism and keep it to themselves, I would like to hear more about this. I cannot even imagine how you would conduct scientific research on creationism.
Another point: you are using the word "confess" which has negative connotations. It is perfectly normal for a funding agency to voice their concerns on, e.g., the design of an experiment, if they can think of an improvement. There is nothing wrong with this, there is nothing to "confess". In some cases the agency might be altering the design to game the results. That is wrong, of course; but your phrasing is too general.
If noone really knows, than maybe nothing substantial is happening. "slightly threatening phone-call from the funding agency" sounds near paranoid. And again, your wording is way too general. For example, it would include NASA (as the funding agency) ask a laboratory that is designing a detector to make modifications to allow higher resolution in higher energies.
It's fairly easy for people to rationalize not publishing a null result not favorable to their funding source when null results are often not published anyway. [1] However, in aggregate, this means that studies that show unfavorable results to a funder are less likely to see publication.
Imagine you're testing the efficacy of a new drug, which has a close-to-neutral or neutral effect (to keep things simple), and there are n labs testing it. You might expect there to be a normal distribution of effect around a mean of zero (whatever quantity we're measuring). However, the groups who didn't find any effect decided to not publish their null results for any minor technicality or due to some pressure like (e.g. are you sure you don't want to repeat this experiment?)
Left as an exercise: if you observe only at what got published, what do you expect to see?
That being said, there's obviously more than just your garden variety publication bias and conflict of interest at play here.
If you expect or are looking for specific results you are likely to unconsciously manipulate the experiment in order to find them. And when big corporations are involved uncounscious manipulation is often the smaller problem..
I expect to see industry-funded research disputing this soon.
The data is hard enough to analyse, we don't need misleading statistics on top of that. (To clarify my point: "statistical significance" is indicative of frequentist statistics. While the changes in the frequency properties of sales prior to and after the shutdown are relevant information, they are probably not the best way to use available information.)
One of my big pet peeves in data analysis is making major inferences and broad, sweeping statements based on a very small sample size. This paper appears to be a prime offender.
Looking at this case, there's a pretty good argument that Netflix and the LoveFilm app launching in the UK, QuickFlix Streaming launching in New Zealand, The Artist (with a French director and star actor) getting a lot of attention and being released to rental, a particularly chilly winter in western Europe, and god knows what else all contributed in whole or part to the observed phenomenon. I'd love to have whatever data the authors of the paper had, to facet the data by movie studio and country - did different countries peak on different weeks, and the like. Alas, the data consists of the sales and rentals of two anonymous major motion picture studios.
That none of these potentially seriously confounding factors were mentioned or explored before reaching for a conclusion is somewhere on the scale between negligence and dishonesty, and the lack of data transparency renders the the paper useful for nothing more than kindling.
Any fellow Tartans here on HN (I know there are a bunch of us) want to sign on to a letter requesting an investigation into the integrity of the paper and IDEA?
It's probably also worth writing a letter stating exactly that to your school, your department, and the department involved, as well as a letter to your alum magazine.
Side note: I would like to see a study on the quality of movies in 2011 vs 2008 based on economies 3 years previous. So when movies were getting funding in 2005 for release in 2008, we got some real gems. In 2008, when the economy took a down-turn, we see some bad ones in 2011.
http://boxofficemojo.com/yearly/chart/?yr=2011 http://en.wikipedia.org/wiki/84th_Academy_Awards
http://boxofficemojo.com/yearly/chart/?yr=2012&p=.htm http://en.wikipedia.org/wiki/85th_Academy_Awards
The only thing "gone in 60 seconds", is the credibility of the authors of this study.
Quite interesting looking at the comments here furiously looking for flaws in the study.
If the results had found the opposite, I doubt the study's legitimacy would have been so vigorously questioned.
I'd strongly question the results of a study funded by The Pirate Bay that said the ready availability of downloadable films increased movie sales.
There are a few points from this publication that are suspicious:
1. There's no mention of funding sources for this study. 2. It's a small sample size of "two major studios". 3. Most importantly, this study isn't peer reviewed.
There, opposite results. Feel free to criticize them.
Maybe because they would have made more sense ?
The presence of amazing startups finally punching through the incredible awkwardness and extreme hassle that is consuming content legally is certainly the driving force behind increased sales.
I've never understood why people think piracy is some form of theft. It makes more sense to think about it as a form of advertising/marketing. You see, media companies NEVER have sales. The closest you'll get is watching 20 minutes of ads per hour on cable TV while footing a pretty intense bill in addition.
For over 100 years, it's been sound business sense to give things away to increase market penetration and drive increased sales. Sales bring buyers who also tend to make other purchases, which leads them to advertising to their friends. Giving things away to people who broadcast the products to a large audience is advertising 101. Loss-leading is a modern strategy that is working very well for expensive electronics. It works by essentially giving people technology so that they can consume more easily, expecting them to compensate you for the difference over time.
This is all identical to how I have actually seen piracy affect the media industry. It's literally just marketing. People who pirate stuff typically aren't people who would buy it anyway. I really do not see a negative impact whatsoever. There's so much evidence to the contrary that I find someone making a claim that "piracy made more difficult, content sales up" is anything other than a context-free juxtaposition of facts absolutely inane. Piracy is easier today than it ever has been. For < $200 in hardware you can have your own personal Netflix but with all the latest movies and TV shows. Yet sales continue to rise, and the failed and hassle-filled pre-internet business models are falling away, leading to a minimal-cost sampled environment where the need for something like piracy is growing smaller all the time.
Then there are some industries that are devolving and paying for it severely. Remember demo disks when we were all kids? I could, every week, get a demo disk with all the latest games on it. Maybe I got to play 10% of the game, but I didn't have to go pay $60 to see if I liked it. Now that digital purchases are non-resellable, fewer and fewer companies offer demos of their games to people so they can try before they buy. Piracy is useful here, and still tends to be an advertisement rather than a hindrance. And these companies want us to buy into always-on, in-your-face, and anti-consumer DRM? They're making it harder to consume legitimately! That's the exactly wrong thing to do! I haven't pirated a game in a long, long while. But I haven't bought any either (except a few indie games and the humble bundles) because the industry seems to be incapable of making any good business decisions.
So yes. It really is so far-fetched that closing down a "major" (what, <5% of all piracy?) piracy site would increase legitimate downloads by even 1 sale.
Well, that sounds okay.
But I'd like a statistician to look at the numbers. (especially their model and their scatter plots.)
How much did it affect the promotion of movies though if people weren't talking and looking for it more? I actually think it ends up benefitting in the end or close to even. Exhibit A: The success of Windows Exhibit B: The success of Photoshop. Both gained a foothold via piracy initially. Piracy gets them hooked then later they buy or they buy it with budgets at work.
Is it worth recovering 1-2% for losing 98% of a user base? Does that eventually lead to more sales? It is a tough problem.
If we're going to criticize this study for being less than rigorous, I think citations like the above need to be backed up as well.
These are also people that have the time to pirate and were never going to buy right now. Later when they have no time and more money they will buy products, they weren't buying if they were taking the time to pirate it. So is it better for an artist or piece of content to be seen/heard or not? $10-$20 for a movie is cheap compared to the time it takes to get it at a certain point and is more convenient. Compared to software prices, music and movies have it locked up in the convenience factor if prices are low and content is actually available. Users will choose convenience and have on those types.
Piracy will be won via convenience, not against the current. Yes it can never be legal or the whole system breaks but it will never go away, use it. Piracy is a 10x+ modifier on user base (freemium is a response to this).
That was me with games. When I was a teenager with no money, it didn't bother me to copy games because there was no lost sale - I couldn't afford to buy no matter how badly I wanted the game. Now that I'm older I have a lot of money (relative to the cost of a game, anyway) and I don't have the time or inclination to pirate anything.
Definitely not suspicious!
I would assume they'd try to be fairly objective with this study, because they'd be very happy to discover that piracy is a good thing for them.
You assume their goal is to stop piracy rather than to promote "anti-piracy" legislation which just so happens to harm their competitors in markets related to content production (like content distribution and consumer devices) or otherwise attempts to make things more like the bad old days when reaching a large audience required the backing of a major studio.
I mean why do you think they keep defending the DMCA prohibition on circumventing DRM? It's obviously nothing to do with piracy because the pirates all just comprehensively ignore it. But what it does is give them a veto over consumer technology. If you have to interact with their DRM then they get to veto your device if they don't like it. Which they don't hesitate to do.[1]
[1] https://www.eff.org/deeplinks/2009/08/judge-rules-against-re...
I'm trying to say this with as little judgement as possible. I just know that deciding to look only at certain countries to support a hypothesis has been done before (c.f. Ancel Keys's Seven Countries Study).
I've not read the paper so I've no idea if this is an issue or not, but only using the data of two studios seems to be a low sample size.
Imagine if the week after the shutdown "Lord of the Rings" was released, or "Twilight", or any other blockbuster. Find two studios with big releases in the period after the shutdown and it looks like some significant causation.
Quote from page 8 of the paper (First paragraph of section 4 if you care to look it up in context).
My concerns are that (1) the study was funded by the same individuals who provide the data showing immediate conflict of interest and (2) the data does not span a 12 month spread. So even though they claim to make consideration for seasonal attributes like Christmas sales we don't have a full data set to know if the rise/fall pattern they display is typical for this timeframe. The release of "summer blockbusters" or other studio phenomenon cannot be compared to the data unless we have data spanning multiple years to know when peak seasons of sales are.
Heck, a third issue is their lack of physical sale data. So although they have source showing digital piracy to digital sale correlation is likely it doesn't account for digital adoption of newcomers to the market.
1. There's no mention of funding sources for this study.
2. It's a small sample size of "two major studios".
3. Most importantly, this study isn't peer reviewed. [1]
>This research was conducted as part of Carnegie Mellon University’s Initiative for Digital Entertainment Analytics (IDEA).
The IDEA is apparently a think-tank to help "groups like MPAA and policymakers [to] navigate the changing digital media landscape."[1]
Interestingly, funding for the IDEA has been made possible by "an unrestricted gift from the Motion Picture Association of America." [2]
Edit: From one of the author's blog it looks like the study has been submitted for peer review: "The study is currently under peer review at a good econ journal." [3] It seems that the SSRN where this paper is currently hosted is to social sciences and economics as to what arxiv is for natural sciences: a repository for yet unreviewed drafts of papers.
[1] http://www.bizjournals.com/pittsburgh/blog/innovation/2012/1...
[2] http://www.techtaffy.com/carnegie-mellon-launches-digital-en...
[3] http://brettdanaher.wordpress.com/2013/03/12/megaupload-shut...
Michael D. Smith: http://scholar.google.com/citations?user=rzPlMfQAAAAJ&hl...
From the paper:
Our [Megaupload] Internet penetration data is based ... Google AdWords estimates of the total number of unique visitors ... [and] ... the total number of Internet users in each country in our sample...
As such, the shutdown of Megaupload influenced the policies of several other cyberlockers focused on piracy, and our results necessarily measure the “net impact” of the Megaupload shutdown across the cyberlocker industry, as opposed to just measuring the impact of Megaupload.
How can you compare with something that is nonexistent? If they had a control, a timeline with piracy, then this comparison would be valid right? I mean, the effects of 6-10% higher could be just the hottest titles that came out in the summer such as The Avengers, etc...
How was the megaupload estimate calculated?
Various parts of the article use per week data to measure sales / rental trends, however the megaupload estimate for each country was held constant, given there's a likely correlation (with a potential time lag - i.e people buy stuff on the advice of people that have already seen it and recommended), how would this variation effect the modelling?
What sort of measurement error does the google adwords estimate have? How does this effect the model?
Are there any factors that might impact the adwords estimate on a per country basis (such as prevelance of use of particular browsers, adblocking plugins, etc.)? How does this effect the model?
Do you have year on year data to compare this data with?
Do you have any data to show correlation between sales and MPR over time?
Does your data differentiate on movies that were available on megaupload before it closed vs those that were released afterwards? How does this effect your model?
I wonder what dinner theatre ticket sales look like over this term, maybe digital piracy is killing the stage!?
To get multiple data points you need multiple events.
Call it 200,000 digital sales per week total, 350,000 digital rentals total. That's for 2 studios, so let's multiply by...I dunno, 4 sounds good to account for the other distributors (this might be way off).
So 800,000 sales, 350,000 rentals. Say the sales net an average of 8 bucks per sale and the rentals 2 bucks each. I made these numbers up, but they match my ancient retail experience.
So that's 6.4 million in sales and 700,000 in rentals per week.
Times 52 and that's 333 million in sales, 36 million in rentals. So, call it 370 million total for a year.
So 5% is 18.5 million. Be generous since we're ballparking and call it $20 million.
Hmm.
Digital alone would probably be a close call. But factor in dvd/bluray and theatrical and it would certainly be worth doing.
EDIT- I didn't do too bad! http://latimesblogs.latimes.com/entertainmentnewsbuzz/2011/0... says in 2010 the digital market for movies was $385 million. But then tv was an additional $366 million, so double that $20 million estimate to $40 million. And it's going to grow fast.
Perhaps a revenue model based solely on product placement could work, but most likely entirely new business models that haven't yet been dreamt up will resurrect entertainment for profit. Perhaps digital sales will serve purely to advertise performances or other tangible art experiences. Regardless, I imagine the entertainment industry will look vastly different in 10-15 years.
...or just exit the market and do something else. Musicians can play live; that doesn't work very well for film directors or actors. Sure, there's the theater, but but the size of the audience you can play to is limited to the distance within which actors can be easily seen and heard. You can't really stage a play in a stadium the way you can stage a music event.
Perhaps a revenue model based solely on product placement could work
Producers and screenwriters already make the most of their opportunities there. It's possible, up a point; if I'm writing a script based in the 19th century I can look for some premium brands with a long legacy for placement or cross-promotion opportunities (you'd be surprised how many there are). But that won't work for earlier periods, science-fiction, or contexts that brands typically prefer not to be associated with, like war movies or stories with gloomy themes.
Entirely new business models are pretty thin on the ground. 'Make something and charge admission to people who want to see it' has been around for a while. The latest attempt is the current crop of 3d movies, since that doesn't really work on the sofa, but it's a big increase in production costs for relatively little box-office gain.
Why, if there is no criminal offence, are governments getting involved? Why aren't the rights holders (who claim to be the ones losing money) suing for lost earnings?