Markets are efficient if and only if P = NP
arxiv.org
arxiv.org
Anyway, I wonder if solutions to NP-complete problems can be approximated using a market.
EDIT: According to the paper, they can.
Do we have a model of what markets are like under this type of model? It is not clear to me that prices are even close to what efficient ones might be under this situation (though they could be).
Also of course most of the missing markets are the contingent and future ones, which suggests the ways in which things might be biased (forward planning, the firm, the business cycle etc) which are quite important inefficiencies.
"Do we have a model of what markets are like under this
type of model? It is not clear to me that prices are even
close to what efficient ones might be under this situation
(though they could be)."
Adaptive Markets and the New World Order
"Under the AMH, markets are not always efficient, but they are highly competitive and adaptive, and can vary in their degree of efficiency as the economic environment and investor population change over time."
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1977721&#...;
Efficient Markets II (by Fama who was the one who originally came up with the efficient market hypothesis describing the three forms of market efficiency)
"A weaker and economically more sensible version of the efficiency hypothesis says that prices reflect information to the point where the marginal benefits of acting on information (the profits to be made) do not exceed the marginal costs (Jensen (1978))."
http://efinance.org.cn/cn/fm/Efficient%20markets%20II.pdf
"Also of course most of the missing markets are the
contingent and future ones, which suggests the ways in
which things might be biased (forward planning, the firm,
the business cycle etc) which are quite important
inefficiencies."
My thinking: It could very well be that markets are only efficient only as far as everything else allows it to be; If investors are not able to evaluate information over more than one business cycle, (due to the cost of doing so, economical or psychological or otherwise), then so be it. The market will move its efficiency to match the environment and its participants.
Try 'Debunking Economics' by Steve Keen for a good breakdown of real problems with many economic theories including EMH. Mostly by showing the logical fallacies in assumptions but also the naive maths and divergence from empirical reality.
http://www.amazon.com/Debunking-Economics-Expanded-Dethroned...
His blog is http://www.debtdeflation.com/blogs/
That is, in wishful thinking.
No other field has a worse prediction track record that economics.
Actually they make a killing on the fact that their are interests and money to be made based on their BS advice and predictions.
I think it was the RBA (or maybe the Australian Treasury) which admitted that their DSGE model was no more accurate than trend modelling in terms of it's predictive power, so they could only use it to give policy advice.
Oh, they do it all the time, to give "policy advice".
Plus even more assumptions: that participants cannot game the system (by influencing laws, abusing power or monopoly, bribing others participants, controlling public opinion by friendly or owned mass media, forming cartels, etc).
Oh, and "rational" doesn't just mean "clearly thinking and clever", but also perfect calculating actors in a game-theoretic way.
Oh, and there are no factors outside of direct market control, from natural disasters and resources, to foreign country policies and such.
So all those hold even remotely true only in some magic unicorn land.
I know people hate to hear that, and it's en vogue to rip on the field of economics, but it's true.
If geeks weren't trading MBS and selling NInJA loans as AAA rated assets to the insurance companies, would the blowup have happened?
If the Fed hadn't pumped easy credit, first during the dotcom bubble, then into the housing bubble, would the blowup have happened?
This is kind of like saying "Look how smart I am, I saved my leg from needing amputation after I shot myself in the foot!"
I think you're conflating "people who work on Wall Street" with economists. Most of the "dirtiest" of the loan originators were people with no financial education whatsoever; salesmen, and nothing more.
>"If the Fed hadn't pumped easy credit, first during the dotcom bubble, then into the housing bubble, would the blowup have happened?"
Yes. Blowups happen. It's the business cycle. A credit driven economy inflates until marginal borrows default, which cascades to a deleveraging. The Fed "pumped" money in the economy after the dotcom bubble because we saw some of the greatest destruction of wealth, ever, during that period. And much of the housing portion was political mandate.
Admittedly, economics and the economy is a finicky patient. But contrary to popular belief, these guys are smart and know what they're doing.
And I think you're conflating economists with people that didn't influence policy at the top level to enable those "salesmen" to do what they did, people that didn't hum along while the salesmen were doing it, people that didn't praise this thing happening, and people that didn't ensure the public that everything was perfectly OK before the crash.
Because economists, and top level ones at that, with Ivy League PhDs and all, did all of the above.
Case in point:
"""The former Federal Reserve chairman, Alan Greenspan, has conceded that the global financial crisis has exposed a "mistake" in the free market ideology which guided his 18-year stewardship of US monetary policy. A long-time cheerleader for deregulation, Greenspan admitted to a congressional committee yesterday that he had been "partially wrong" in his hands-off approach towards the banking industry and that the credit crunch had left him in a state of shocked disbelief. "I have found a flaw," said Greenspan, referring to his economic philosophy."""
But all other top dog economists policy influences were doing the same things, and praising the same "throw caution to the wind" attitude towards the "free market" and unsupervised banking...
Pray tell, what did they do? What options does the Fed have that can run opposite popular politics that wouldn't instantly cause them to lose independence?
>"people that didn't ensure the public that everything was perfectly OK before the crash"
This is simply not true. People, including academics, were writing about the housing market pressures as early as 2004. The "economists" can't force themselves onto MSNBC or CNN to tell people to stop buying houses, especially when that runs counter to what politicians want: a hot economy.
>"The former Federal Reserve chairman, Alan Greenspan, has conceded that the global financial crisis has exposed a "mistake" in the free market ideology which guided his 18-year stewardship of US monetary policy."
An old Randian, trying to ensure his legacy isn't completely tarnished. Hindsight is pretty easy.
How do you know before a tough, life-threatening operation whether the tumour you are getting removed will kill you? You don't.
1. We have statistical evidence from large numbers of SIMILAR people with similar tumors. Economic events tend to be highly dissimilar in terms of context (Great Depression? 1988 Savings and Loans Crisis? 1997 Asian Financial Crisis? 2008 Global Financial Crisis?), and only to be similar in terms of before-and-after on some narrow set of parameters. This makes prediction and counter-factual history for economic stuff really hard.
2. We also understand some of the mechanisms of how tumors work and affect physiology. These are founded upon a large body of knowledge of causation and empirics in medicine, chemistry, and biology. Economics lack a comparably reliable and large body of knowledge because of 1.
Exactly my point. We really have no idea what would have happened had the fed not intervened and allowed a normal bankruptcy to occur.
I'm not saying economists are lazy and stupid, I'm just saying they lack data, and that tatsuke's counterfactual (GDII) is not a very good one.
This is the fallacy: You really have no idea. That's not an insult. People who do this for a living, like, say, FRB economists, do know the consequences of letting bank runs occur and not providing liquidity during a crisis. It's well studied.
I'm not sure this is true, either. Financial crises are surprisingly similar [0].
[0]http://www.amazon.com/This-Time-Different-Centuries-Financia...
>"These are founded upon a large body of knowledge of causation and empirics in medicine, chemistry, and biology"
And what about new discoveries? What do they do then?
Some things are easy. Some things are new, you have little data, and have to make a best guess. Same as economics.
Contrary to popular belief, these guys aren't winging it. As I've said, go check out the research they're doing at the Fed. That's the cutting edge of economics.
"These guys" are clearly winging it, when we remember the definition of "These guys":
Ideologues touting Efficient Markets and the Washington Consensus, often in the pay of corporations, and using clearly falsified theories to do so.
(You can get an idea of how bought they are by watching "Inside Job".)
No one is saying that no economists anywhere ever got anything right. The problem is that the economists who do get things right are the dissenters from the orthodoxy of "These guys".
So why don't you go check out the research referenced at
http://www.paulcraigroberts.org/2012/07/29/escape-from-econo...
As for the Fed, Volcker's Fed yes, Greenspan and Bernanke's Fed no.
I read plenty of economic research, thank you. Every day. So, while I appreciate a link to a what is essentially the site of a pundit (whom I have read, by the way), this is not the bleeding edge of economics. When is the last time this guy actually wrote a paper?
>"As for the Fed, Volcker's Fed yes, Greenspan and Bernanke's Fed no."
Right. So, Volcker, dealing with intense inflationary pressure in the 1980s, and Bernanke, dealing with massive deflationary pressures in 2008, should use the same policy tools? I'm sorry, but you clearly need to brush up on the basics before you go attacking someone for their lack of economic knowledge. This is pointless.
Oh, and where did I say Volcker and Bernanke should use the same policy tools?
The big difference between Volcker and Greenspan is that Volcker stood up to congress, while Greenspan caved to the politicians and gave them the bubble they wanted. And politicized the Fed.
Or in your own words, "And much of the housing portion was political mandate."
Did you not know this?
And where did I attack you for lack of economic knowledge?
I attacked you for using strawman arguments and putting words in peoples' mouths, and I stand by that.
Really? So where were these "hundreds of trained economists" when the Great Recession came about? Out there, causing it.
As for "being saved from becoming Great Depression II", well, let's wait and see.
Causing it? That's preposterous, being as how the borrowers (ie, Joe Public) are 50% to blame for over extending themselves. Again, tell me what tools the FRB has at its disposal that could have prevented the housing meltdown.
>"As for "being saved from becoming Great Depression II", well, let's wait and see."
Indeed we shall. Care to put your money where your mouth is? I have, and continue to do so, participating in one of the biggest bull-market run-ups in history the past two years. Meanwhile, most people continue to spout doom and gloom. And I'm just some guy watching this happen from the sidelines, trying to piece it all together.
All of the topics you mention are popular topics of study. For example, gaming the system by influencing laws was studied by (picking a famous name here) Hayek. Monopolies/monopsonies are well studied and generally believed to be inefficient.
Information asymmetries and actors with bounded rationality are extremely popular topics to study today.
The paper we are discussing here is a perfect example describing the microfoundations of bounded rationality, for example.
I'm talking about the assumptions about market efficiency and the supposed superiority of the mythical "free market" here.
Hayek, since you mentioned it, was a western "free market" lackey, imposing his dogmas on the Chilean people --and lots of others-- (and through a dictator at that), with dire consequences. This kind of "policy advice" is 80% percent catering to interests and 20% ideology. No more scientific than Stalinist economics.
A science doesn't need a dictator (or an elected official) to enforce that "earth is round" or "water will boil at 100 degrees under the right conditions".
Heck, even hard science fails when there are economic interests (e.g big pharma, releasing BS half-baked drugs, or physicists making big BS claims to get funding).
Economy is all, and solely, about economic interests, so all public (non academic) use and discourse of it is inextricably tied to those.
As for the "popular topics to study today", those, while interesting from a math/game theory standpoint, are turned to shit as soon as they enter the political / economic policy field.
On my 25 years of following the stuff, I've haven't seen anything but BS, special interests, spin, greed, failed predictions and bad advice on all fronts. Which is always touted as "scientific" and "based on state of the art models" by the policy advisors.
If you take out the "cater to special interests" bias factor, the rest of economists performance can be had with any random walk methodology.
Market efficiency and "superiority" (I have several incompatible ideas of what you mean here, but if you could clarify...) of an idealized free market are not assumptions. They are conclusions derived from much simpler assumptions.
Different conclusions can also be derived under different assumptions. For example, markets are "inferior" when distributing signalling goods (e.g. suits or educational certification), since this leads to wasteful arms races and overconsumption.
As I said, you should learn some economics before attempting to critique it.
A science doesn't need a dictator (or an elected official) to enforce that "earth is round" or "water will boil at 100 degrees under the right conditions".
I'm confused - apart from the fact that you (and Hayek, incidentally) don't like dictators, what are you trying to say here?
As for the "popular topics to study today", those, while interesting from a math/game theory standpoint, are turned to shit as soon as they enter the political / economic policy field.
This is the nature of politics.
Well, highly debatable conclusions.
(As for what I meant, it's that I'm attacking both the idea of the EMH for financial markets, and the general belief in the superiority of the "free market capitalism", going back to the idea of the "invisible hand").
>I'm confused - apart from the fact that you (and Hayek, incidentally) don't like dictators, what are you trying to say here?
That a "science" that studies something _it itself creates_ through "policy advisory" and "legislation" is hardly a science.
Not to mention that it interprets it however fits those it is in ties with, or that its outcomes are more often than not directly inverse of the predicted ones, at which point they blame it on "not enough enforcement" etc, in a classic anti-falsifiability move.
As for Hayek not liking dictators, LOL.
>In 1978 he wrote to the London Times that he had “not been able to find a single person even in much maligned Chile who did not agree that personal freedom was much greater under Pinochet than it had been under Allende
(...)
For instance, Hayek—writing to The Times in 1978 and explicitly invoking Pinochet by name—noted that under certain “historical circumstances,” an authoritarian government may prove especially conducive to the long-run preservation of liberty: There are “many instances of authoritarian governments under which personal liberty was safer than under many democracies.”
http://coreyrobin.com/2012/07/08/hayek-von-pinochet/
Of course, his support to the regime is even worse and far more involved, utilizing it as way to fulfil his economic policy wet dreams. Yes, a great freedom loving guy, this Hayek.
>This is the nature of politics.*
And this is the role of economics and economists.
As for the academic part of economic research, when not done with specific political interests in mind and to support some specific monetary and financial policies, is as relevant as post-modernist gender-studies critical theory. That is, totally irrelevant.
Therefore, either you can solve such a problem efficiently (and P=NP) or the idea that all you need is rationality and information is false.
> Now, one may argue that a weaker form of market efficiency as per Fama (1991) and Jensen (1978) is that patterns are exploited until the marginal revenue of further discovery equals the marginal cost of further search. A counterargument to that would be graduate students and other hobbyists and day traders searching for an edge: to them, the marginal cost is zero and at times negative because the value of the search itself, regardless of the outcome, is a positive learning experience for them...
A hobbyist has to spend more computation time for every additional strategy they search, which costs real money. The hobbyist's cost is near-zero for small searches -- those that can be done by one person using the computers they already own. Once the size of the search increases beyond what they can manage with their immediately available resources, they have to hire assistants and/or buy computation time like anybody else.
Then again, I'm not an economist (coming from a mathematics backround), so maybe this is the expected form of a paper here?
http://dpennock.com/papers/chen-ec-2007-betting-on-permutati...
Here is a review of the literature from a while back, it cites plenty of actual math papers:
http://dpennock.com/papers/pennock-ijcai-workshop-2001-np-ma...
Subset-sum can be approximated within 1+epsilon for any requested epsilon>0 [a PTAS]
MAX-3SAT can be approximated within 8/7 and not better. Vertex cover can be aproximated within 2 and its not known if that's optimal.
Set cover can be approximated within O(log n) and not better.
Clique and TSP have no sensible approximation.
I would argue that potentially a loss of factor 2 is not an order of magnitude away from optimal. I don't know what is the situation for markets.
Also worth noting is that the expensive nature of being perfectly informed is one of the main drivers of the large sizes of firms. We can measure how well markets are doing with respect to the amount of excess, beyond normal profit that producers can capture in the long term. Current polarization of wealth and excess of rent suggest no, we are very far from efficient markets.
There is an argument one can give that the abstraction of interacting humans leads to a computer more powerful than a regular turing machine. I don't believe this but everyone who thinks AI is not possible on a turing machine does.
It's like those horoscopes that state - "you will meet someone of great personal importance soon!"
I found it rather enlightening - but the paper is still highly readable and I encourage HNers to peruse it and perhaps check out the video as well.
Previous discussions:
http://news.ycombinator.com/item?id=2895474
http://news.ycombinator.com/item?id=1144548
More from the professor: http://philipmaymin.com/cv.pdf?q=phil/cv.pdf
He begins so:
The efficient market hypothesis claims that all information relevant to future prices is immediately reflected in the current prices of assets. In other words, you cannot consistently make money using publicly available information.
Then, he encodes a 3-SAT problem into market orders and claims:
So what should the market do? If it is truly efficient, and there exists some way to execute all of those separate OCO orders in such a way that an overall profit is guaranteed ... then the market, by its assumed efficiency, ought to be able to find a way to do so.
So he defines market efficiency to be all-knowing, and then gives at an NP-hard problem to solve. It seems like he could have given it the halting problem as well, only it might be harder to encode as market orders. But the point is that an efficient market is "all knowing" only that which is "knowable". The market isn't an oracle that can answer any question about the past, but a true machine that can answer all questions about the past that are answerable using the machine.
What he's doing seems similar to saying "if God exists then P=NP". After all, if God is omniscient and omnipotent, He could solve all NP problems in polynomial time. :)
Before there were computers in an efficient market (supposing one exists), the current prices would reflect everything that is known and can be computed/deduced without computers by the present time.
Anyway, the paper misunderstands "known" to mean "anything which can theoretically be known", which obviously includes solutions NP-hard problems. It's just that nobody claims that's what efficient markets are.
The hypothesis is predicated on perfect rationality. http://en.wikipedia.org/wiki/Perfect_rationality
vs
Copyright and network effects are the main reasons monopolies exist in digital goods.
Great quote! I will take it.
Similarly, it is possible (theoretically at least) that the market has access to some means of computation not available to a turing machine. Given that most of the market is run by traditional, turing, computers, that does not seem to be the case.
He first understands efficient to mean godlike, and then shows that for God to work, He would need to solve NP-hard problems quickly. Well, duh.
It seems like the definition of effiecent market you are using would mean the market makes the best decisions given the information and computational resources it has.
I would make an even weaker claim, saying the market makes the best possible decision, given the information, computational resources, and known algorithms to utilize the information and computational resources, and given players whose goal is not achieving an optimal outcome from the market.
Economists have essentially no idea what is going on and neither do their financial compatriots and the EMH is by far the clearest example of this.
I'm trying to make it a personal rule that anything outside of the hard sciences (math/physics/chemistry/statistics etc) is bullshit to me until further notice.
I mean really - I do advanced math and statistics in moderately complex systems - AI/robotics - and I can barely wrap my head around them. I shake my head in disgust at those who apply platonic and unrealistic theories to the extremely complex system that is the world. Indeed, the surety they display in their theories amuses me, because to even think that one can reason about the entire world all at once is, in fact, hilarious.
After all, all science amounts to an approximate model, even your vaunted "hard sciences".
I mean if your entire field is dedicated to predicting the future of finance and economics and you don't predict them - it kind of shows that you really don't know what you are doing.
All I'm doing is pointing out that the social sciences have no clothes on.
You're reading too much into the disproportionate effect mistakes in economics have had on society. Science get things wrong all the time...failure is part of the learning process.
The social sciences don't have to because they aren't based on the scientific method - mainly the design and independence of repeatable experiments with controlled variables.
Hard science equations have no room for bullshit whereas the social ones do - hence EMH is still taught.
Also...I'm not an economist so I don't really know what's going on in this area...but maybe EMH is still being taught because it's the best model we have at the moment?
Are you sure it's being taught everywhere? Maybe some places have already dropped it? Change takes time.
EMH is taught globally and it will continue to be taught until such a point as people stop believing.
The same is true of many physical sciences - geophysics, oceanography, climate science, astronomy, etc. Your criticism applies to basically any scientific theory which has poorly understood microfoundations (i.e., a lot of them).
The EMH is taught because it's a useful approximation to reality, even if it's imperfect. Or, as the article puts it: Whether markets are efficient or not, and whether P = NP or not, there is no doubt that there will be markets that can allocate resources very close to efficiently and there will be algorithms that can solve problems very close to efficiently.
Incidentally, the EMH claims that financial crises are unpredictable. So the lack of useful predictions of the financial crisis is evidence in favor of the EMH.
[edit: Note, in response to Dn_Ab that 3SAT, the problem considered by the paper, is NP complete.]
Or for markets, aspects of it may invovle solving NP-Hard problems with efficient approximations that are themselves NP-Hard (you are better placed to opine on whether such a possibility is likely).
As for physics. The idea of not changing course is exactly what sets apart good theories from bad. If you have a new theory and it does not include the old at a limit then it is a good sign that you are doing something wrong. Newton was not wrong, just inaccurate. Einstein did not invalidate Newton, only corrected the extreme cases.
"I mean if your entire field is dedicated to predicting the
future of finance and economics and you don't predict them
- it kind of shows that you really don't know what you are
doing."
Read Kahneman's "Thinking,Fast and Slow" to get a better idea as to why.
Our biological heritage has left us - oh, OK, the vast majority of us - fundamentally incapable of understanding complex systems and making predictions that have any value beyond the immediate short term. Our intuition - our System 1 heuristics (read the book) - is completely ill-suited to this, and our reason - System 2 (ditto) - doesn't fare much better without advanced training.
Kahneman and others also cite many cases where trained experts make the same errors of judgement and analysis as the rest of us when they stray even fractionally from their areas of expertise.
The GP is right: We get a lot more of this "soft stuff" wrong than we get right because we generally don't bother to apply hard science methods to the problem.
Personal anecdote: I stopped listening to the news because I used to work in crypto. Well, that's the tl;dr reason.
I worked for a company that did PKI and noticed that tech journalists, who might be expected to at least have some greater technical background than just folk made egregious errors in their reporting, in reports that were researched and drafted and edited and considered and fact-checked prior to publication.
While PKI is complicated, it is not complex. Not like the economy. So if these "specialized" journalists with a technical leg up could get so much of something so simple so wrong, what were folks with degrees in English and/or journalism doing to something as advanced and out of our grasps as the economy?
Truth be told, I haven't stopped reading the news - I read it differently. Should an interesting headline catch my eye, I look for a few more. The more interesting the story, the more sources I check/read before accepting to learn anything from the reporting. This is a necessary side effect of how complex the world and its news are, and how simplistic reporting is.
Economists had no idea why their theories of rational actors were so flawed (but flawed they knew them to be) until Kahneman and Tversky came along. Hard science can only cast light on the dismal and dark, and the dismal and dark will remain so, will remain largely speculative, until they adopt our tools.
Let me help try help you out :). Here are some more of your quotes:
'Everyone is ruled by incentives'
'If it's a derivative of a psychological bias <...>'
Rather strange to be making claims involving incentives and psychology, for someone that rules everything outside the 'hard sciences' as bullshit.
(FWIW I studied math, cs, applied math and physics at varsity but learned a great deal of knowledge that corresponds well with reality, outside of those fields)
Hence - until further notice everything I have ever stated is probably wrong and biased.
Just like everyone else.
The article is quite probably BS as you point out, and I have no idea.
However keeping an open mind seems like a worthy goal, irrespective of the BS that goes on in any field and even if the social sciences do tend to be less empirically rigorous.
And while it's exceedingly complex, there are indeed some people (albeit a precious few), that understand how the economic machine works. Ray Dalio is one such example.
Complete and utter nonsense. "You" don't know what's going on, and assume others don't. The public is unbelievably ignorant of this subject. These men are brighter than you think.
>"EMH is by far the clearest example of this"
EMH is a 50 year-old theory. Like practically any other subject, economics that is taught in school at low-levels is based on old science. There is real, modern research happening on real-world things, just go poke around the Fed's site to see what they're doing.
I only have one question. If they know what they are doing why do we still have financial crises?
edit: physicists ugh too early in the morning
And their called physicists.
Quoting Wikipedia:
The Latin adverb sic ("thus"; in full:
sic erat scriptum, "thus was it written")
...
The usual purpose is to inform the reader
that any errors or apparent errors in the
transcribed material do not arise from
transcription errors, ...
Then you're ahead of me education-wise; we lend money until the default rate gets above a threshold, which causes a "crisis" of deleveraging. Then we lever up again. All of this happens around a long-term, upward growth trend. It's the business cycle. It's hard to time, but most people understand how it works.
How can you prevent crises when their foundations can come from anywhere, including natural disasters? It's impossible. But the powers-that-be do their best to soften the blow. And they have been very successful in this last recession. I know it's hard to convince people that it could have been worse, but it could have been worse.
Already they're working on the next problem: how do we not become Japan? Unlike some of the hard-sciences, economists get one kick at the can. If they miss, they have a new problem to solve.
Oh wait we're economists - free markets are sacrosanct and they are efficient - if there is an arbitrage opportunity to short surely the rational investors will price it correctly. Not.
That's easier said than done. Most of it is politics. But they do.
The Fed began tightening monetary policy back in 2004, when rates went from just below 2% to 5.5%. They did so to start to "cool" the economy, and it essentially popped the housing bubble.
For pedagogical purposes, I think it's a really good simplified and ideal illustrative model, until you learn the more axiomatic derivations of no-arbitrage (the arrow-debreau stuff). From your other posts, I gathered that you're not beyond undergrad studies in economics - you'll likely learn about the more complex models when you reach a more advanced and rigorous stage, if you pursue the theoretical path.
If I were you, I wouldn't be too concerned about others arguing about the EMH - without the proper foundations, you won't understand them beyond a superficial level.
Not to mention that it's popular for the less educated to bitch about the EMH, probably because it's an English acronym with commonly used words. I've almost never heard laymen talk about how unrealistic the nash equilibrium is (:
But you do assume that I don't know these other types of knowledge when in fact I do.
Your open mind argument can be applied to spiritual and alternative medicine just as well - all knowledge is not equal.
So if you claim that something is equivalent to it, you better be prepared to define what exactly "something" is, and give a rigorous argument as to why it's equivalent. Some hand-waving with vaguely defined and ambiguous terms just doesn't cut it.
The definition of an efficient market actually means that that the price of a security at time T represents all that market players can deduce about its history by time T, using information travel and processing speeds that market players possess. It really just means that everything that is historically known about the security can and will be used to determine its price as fast as possible. If it isn't possible - it won't determine the price.
"Economists have essentially no idea what is going on and
neither do their financial compatriots and the EMH is by
far the clearest example of this."
"I shake my head in disgust at those who apply platonic and
unrealistic theories to the extremely complex system that
is the world. "
From the author of the EMH in 1991, 21 years after proposing the EMH.
"A weaker and economically more sensible version of the efficiency hypothesis says that prices reflect information to the point where the marginal benefits of acting on information (the profits to be made) do not exceed the marginal costs (Jensen (1978))." http://efinance.org.cn/cn/fm/Efficient%20markets%20II.pdf
My reading of Fama and Mandelbrot seems to indicate that markets are essentially random and unpredictable. Being largely uninformed on the matter, though, I would appreciate any explanation of Mandelbrot's theories and their relevance to the topic at hand, as they seem to have been largely ignored by economists -- I could not speculate as to why.
To me the tools of chaos theory seem apt to describe markets, insofar as they may be described. If they happen to yield no useful results, I would believe that such may not be obtained. I would appreciate correction on this matter.
Most professional money managers make their gains by actively interviewing companies in a form a stylized intentional financial journalism if you will.
In the very short term, EMH is false for computational reasons. Efficient markets require someone (i.e. an arbitrageur) to keep them efficient. The profits of arbitrage are the incentive for people to keep markets as efficient as possible, and they tend to have this effect. This is an extremely competitive business, and in this day in age, it often comes down to microseconds, but it can be done and billions of dollars are made every day by people who are doing it (and, contrary to popular depictions, arbitrage is actually good for the markets and economy).
In the very-long term, EHM is probably also false. By very-long, I'm talking about 10+ years. The reason for this is rooted in the scarcity of money: there are a lot of projects and companies that will produce and capture economic value in the future, but people don't have enough money to fund them all. Hence, stocks are "cheaper" than they should be, and risky growth stocks (much less illiquid private equity) especially so. ("Equity premium.") There are some people who (demonstrably) "get" value investing and are better at predicting long-term corporate futures than others. The issue here is the long feedback cycle. If your frequency of investments is that low, you have no way of knowing if you're actually good, or just getting lucky, especially in the context of the non-normal (i.e. fat-tailed) distribution of equity returns driven by "black swan" events.
EMH is true enough that if you don't have the technical machinery to trade at microsecond latency, nor the reputation that will allow you to invest for the very long term despite market caprice-- i.e. even if the market tanks, people will trust Warren Buffett's judgment-- you probably can't reliably make a better profit on the stock market than you'd get if you invested in an index fund.
What does EMH rely upon? Ultimately, it says that if there is expectancy to be made selling or buying a security at a price other than P, it will be sold or bought until the price reaches P. This assumes an infinite amount of capital ("smart money") that people are willing to deploy in order to exploit pricing inefficiencies or inconsistencies. This is an obviously false assumption, but for liquid securities of known expected value, it's close enough. Leverage (borrowing) generates a lot of "additional" smart money, so that even a 20bp (0.2%) discrepancy can be levered up into a 10% gain. (If you have $100, borrow $4900, and turn that $5,000 into $5,010, your equity position has gone from $100 to $110.) Microprofit opportunities will be exploited so long as there's sufficient leverage to make them worthwhile, but the willingness of lenders is not infinite.
EMH is usually used to make mathematical analyses work. It's a guideline, but no one who understands financial markets believes it to be literally and universally true. No one can actually predict the future or human behavior, but the (false) assumption that there is no arbitrage produces closed-form numbers that are often very close to the real values.
Also necessary is the distinction between smart and dumb money. The latter isn't a pejorative; "dumb money" means that there are incentives other than informed speculation. For example, when you buy a house because you want to live somewhere, that's dumb money. Or when an index fund buys stocks because of its chartered requirement to do so, that's "dumb money", not because the buyer is an idiot, but because his purchase doesn't convey information about the stock's real value in the way that smart money would. Markets are efficient when there's enough smart money to keep the dumb flow from pushing the price around. This is going to be true of highly liquid stocks, currency rates, and commodities, but not true of assets like real estate. Financial engineers tend to discount "dumb" activity as harmless Brownian motion, but the 2008 subprime mortgage meltdown established that not to be always wise.
Smart money can be dumb as well - because it pays to do so. I'm quite sure that the number of rational investors/capital are greatly outnumbered by the irrational investors/capital such that many correct trades become essentially insolvent before the market becomes rational again. See value shorters for the last two booms.
Keynes was quoted as saying "Markets can remain irrational longer than you can remain solvent."
_________________
[1] NB: information assymetries. By introducing and/or leveraging them, zero sum games can be more profitable than "fair games".
[2] http://en.wikipedia.org/wiki/Heteroscedasticity Heteroscedasticity does not cause...estimates to be biased, although it can cause...estimates of the variance (and, thus, standard errors) of the coefficients to be biased, possibly above or below the true or population variance. Thus, regression analysis using heteroscedastic data will still provide an <unbiased estimate> for the relationship between the predictor variable and the outcome, but standard errors and therefore inferences obtained from data analysis are suspect.
The maintenance of <unbiased estimate> preserves EMH.
[3] The arguments here are not trivial. But they are beyond the scope of many (if not most) professional "economists".
-- Behavioural assumptions of market participants
-- Preference Neutality with respect Volatility levels
-- Non-existence of externatlities from market microstrucure
These are avenues to be exploited, and the exploitations work better the more EMH is disseminated without "hypothesis" being spelled out. These are orthogonal directions of attack, but more dangerous and more powerful than many understand.