The Inefficient Market Hypothesis
The efficient-market hypothesis (EMH) is the idea that there are no hundred-dollar bills lying on the sidewalk because someone smarter than you would have picked them up by now. The EMH is a good tool for most people.
If you’re smart enough then you should reverse this advice into “There are hundred-dollar bills lying on the sidewalk”. If you are a genius then you should reverse it to the extreme. “There are hundred-dollar bills[1] lying around all over the place.”
Hundred-dollar bills lying on the sidewalk are called “alpha”. Alpha is a tautologically self-keeping secret. You can’t read about it in books. You can’t find it on blogs[2]. You will never be taught about it in school. “You can only find out about [alpha] if you go looking for it and you’ll only go looking for it if you already know it exists.”
Where should you look?
Abstractions
A system is only as secure as its weakest link. Cracking a system tends to happen on an overlooked layer of abstraction[3].
It’s easier to install a keylogger than to break a good cryptographic protocol.
It’s easier to disassemble a computer and read the hard drive directly[4] than to crack someone’s password.
The best attacks (those requiring the least work) happen on an separate dimension of orthogonality entirely.
The easiest way to talk to someone powerful is just to call zir company and ask by first name[5].
Won’t this technique stop working now that Tim Ferris has published it in a bestselling book? Not necessarily. Quantum mechanics has been public knowledge for decades yet most people can’t do it. The hard part of pickpocketing isn’t finding pockets to pick.
Perhaps you don’t need to talk to anyone rich and powerful. That is a good problem to have.
I think you should find a problem that’s easy for you to solve. Optimizing in solution-space is familiar and straightforward, but you can make enormous gains playing around in problem-space.
― What Startups Are Really Like by Paul Graham
Problem-space tends to have higher dimensionality than solution space.
Case study
According to Joel Spolsky, the best programmers have the ability “to think in abstractions, and, most importantly, to view a problem at several levels of abstraction simultaneously.” Also according to Joel Spolsky, a business is an “abstraction [that] exists solely to create the illusion that the daily activities of a programmer (design and writing code, checking in code, debugging, etc.) are all that it takes to create software products and bring them to market.”
The ideal programmer employee is someone who can see all the way down to the level of bits, yet can’t raise zir head high enough to manipulate the financial machinery of venture capital.
Homework assignment: How can you harvest alpha from this local equilibrium?
How to tell when you get it right
Alpha often feels like a magic trick. You know the phrase “A magician never reveals his secrets”? Magic secrets are not secret. The Magician’s Oath is not real. David Copperfield patents his inventions[6]. You can look them up in a public government registry. You don’t because magical secrets are boring. Disappointingly so.
Magicians cheat. The purest alpha should feel like cheating too. The greatest compliment you can receive about your alpha source isn’t “You’re a genius.” It’s “That shouldn’t be possible. I’m disillusioned to live in a world is so inefficient.”
Of course, you should never hear either response because you should never flaunt these discoveries in the first place.
- ↩︎
A month ago I offered to put Westerners in touch with an N95 mask exporter in China. Only two readers took the effort to message me about it. One of them couldn’t be bothered to use WeChat.
- ↩︎
Actually, I did find alpha on a blog post once. The tutorial has since been taken down.
- ↩︎
For practice, check out What is the fastest you can sort a list of ints in Java?
- ↩︎
Most computers are not encrypted. Professional software engineers are consistenly surprised by my ability to recover files from a broken laptop without their login information.
- ↩︎
- ↩︎
Edit: I can only find one patent invented by David Copperfield, patent number 9017177⁄9358477. Most of his patentable illusions seem to be invented by other people.
- [Letter] Advice for High School #1 by (20 Apr 2021 4:09 UTC; 93 points)
- Orthogonality by (21 May 2020 3:17 UTC; 21 points)
- Re: Attempted Gears Analysis of AGI Intervention Discussion With Eliezer by (15 Nov 2021 10:02 UTC; 20 points)
- Non Polemic: How do you personally deal with “irrational” people? by (2 Nov 2020 13:44 UTC; 12 points)
- Non Polemic: How do you personally deal with “irrational” people? by (2 Nov 2020 13:44 UTC; 12 points)
- 's comment on Six Specializations Makes You World-Class by (22 Dec 2021 9:53 UTC; 9 points)
- Roundabout Strategy by (28 Jan 2021 0:44 UTC; 7 points)
This is sort of true, but you need to factor in that you’re not competing against the average person, you’re competing against the smartest person who could pick up the bill and wasn’t too busy picking up other bills.
The notion of “alpha” seems to be relevant only to zero-sum games, where revealing a method is bad because you don’t want the other players to succeed. Personally, I am more interested in the sort of “bills” picking up which is a win in altruistic (“utilitarianism”) terms rather than selfish terms. (Of course, sometimes to score an altruistic win you need to outmaneuver someone opposed to it, whether out of foolishness or out of malice.)
I’m using “alpha” as the term is used in quantitative finance. This includes non-zero-sum games. For example, if you were the first person to diversity across continents then you could lower your own risk profile (and therefore increase risk-adjusted returns) without increasing risk for anyone else.
Distributing the value extracted from a gold mine is zero-sum. Prospecting for gold is non-zero sum.
But, if the game is not zero-sum, why is it a “self-keeping secret”? Why wouldn’t someone who finds it tell to everyone? Why “you should never flaunt these discoveries in the first place”?
This is a good question and I had to think about it for six months before I could come up with a good answer.
Whether or not alpha is “non-zero sum” is a red herring. I was wrong concerning that point. There is a more important reason not to flaunt discoveries of alpha.
Alpha is a bet against your society’s beliefs. If you proselytize a powerful alpha source then other people will (in the short term) think you are nuts. This is no use to anyone.
If you want people to trust you in the long term then you can do one of two things.
Publicly bet reputation by flaunting your discoveries.
Privately bet capital.
If you are confident in your alpha source then you should bet capital regardless of whether you also bet reputation. If you are already betting capital then betting reputation is redundant. You gain no additional upside. But there is a long-term cost to your reputation if you turn out to be wrong. There is an unavoidable short-term cost to your reputation regardless of whether you turn out to be wrong.
Flaunting discoveries of alpha creates downside without upside.
I have three problems with this argument.
First, it’s not always possible to bet capital. For example, suppose you figured out quantum gravity. How would you bet capital on that?
Second, secrecy is costly and it’s not always worth it to pay the price. For example, it’s much easier to find collaborators if you go public with your idea instead of keeping it secret.
Third, sometimes there is no short-term cost to reputation. If your idea goes against established beliefs, but you have really good arguments for it, other people won’t necessarily think you’re nuts, or at least the people who think you’re nuts might be compensated by the people who think you’re a genius.
In this case:
we generally prefer that other people make such discoveries rather than not make them; each discovery has positive externalities for everyone else
… but we want other people to not discover the SAME gold mines we do.
Everybody working the same goldmine is in a zero-sum game, but I still benefit from somebody else working some other gold mine.
In general, it’s often the case that a game is only zero-sum along some pareto frontier—i.e. negotiation games—so we get some competitive behavior, but can still have nonzero-sum gains. People can fight in a zero-sum manner over how to divide the gains from trade, even though the gains themselves are nonzero-sum.
If we’re both working for the common good and don’t want to discover the same mines, then the rational strategy is not keeping secrets. It is exactly the opposite: coordinate with each other what to work on.
If we’re both working for the common good, then it doesn’t matter whether the game is zero-sum or nonzero-sum in the first place. If everybody’s working for the common good, then there isn’t really much point to game theory at all.
The point is that nonzero-sum does not imply that the rational strategy is to cooperate all the time.
I know that nonzero-sum does not imply that the rational strategy is to cooperate all the time. I would agree with the OP if ey said that it’s sometimes rational to keep secrets. But eir actual recommendation was entirely unconditional. Incidentally, OP also seems to act against the spirit of eir own advice by giving this advice in the first place.
Sharing knowledge and building a model based on that is still going to be cognitively consolidative even if later counteracted. For example in a jury system the jury has only access to evidence that has passed admissability challenges. Telling them something and later telling them to disregard any conclusions or impressions based on that is rather laboursome. The decision to disregard can be waeker than the reflex to integrate sensed data.
There might also be such effects that a person assigned a mine known to be of lesser yield might work on it less enthusiastically. If you think what you are doing is the best method known to man you might throw your whole mind behind it way more wholeheartdly even if your assement of civilizational peak is in error and due to ignorance.
Then there is the case when the combination of individual efforts would be more efficient / smarter than banding together to be smart. Establising a strategy that everyone things is sound and everyone can get on board can get a very small common denominator. Limiting the variety of psychological profiles working on a subfield can make it more internally coherent. You tell someone about berries and they try to “help” and end up poisoning themselfs or the whole tribe. A way of limiting that knowledge can’t be abused is to limit acccess to it. Berry picking adaptations might make you dangerous fisher so it might make sense that you keep the details of your methods within your own operations.
I think the idea is like:
1. Consider “metals”. The fact that there is an industry in which deposits are discovered and mines are created to extract these resources, is not a secret.
2. But if you find a gold deposit, you should buy the land in order to create a mine there, or sell it at a higher price, rather than telling the current owner of the land.
3. People in a business which uses metals, miners, and shipping companies, may have something to gain by there being more mines. They may have advice on how to a) find deposits, or b) setup mines, or even c) tell you where some deposits are. (People who know their land has a deposit may advertise this fact, and seek a buyer that will pay the higher price in order to build a mine there.)
4. But while people who make and run mines might tell you how to a or b, they are unlikely to tell you c. (Unless there are a lot of such deposits, perhaps distant from their mines, or there are a few mines, and mine owners wanting to trade investments in one mine for investments in more mines (which don’t exist yet) in order to reduce their risk.)
So the article says:
1. There are hundred dollar bills lying on the street.
2. Pick them up.
3. ???
4. Don’t tell other people—see 2. (Unless you are provably rich from this method, and have moved on to serve the role of 3 - being paid to help people figure out where to find, and how to pick up, the hundred dollar bills lying on the street.)
Should you? That really depends on your goals and code of ethics.
“Should” in the sense that you can* “gain” something by doing so. (Perhaps the OP should have been more clear about this, or “should” shouldn’t be used.)
*It was intended as a metaphor, which implicitly assumed that there are ‘dollar bills lying on the ground that you can pickup’.
In an MMO I think I went through an experience of feeling of inefficient hypothesis. But it didn’t feel to involve secrets. It involved dangerous areas paying out more and the common wisdom being that the reward was not worth the risk. It tended to involve stuff like new faces not previously seen in the dangerous areas were piled up on to discourage them from upping the population (with possibly the important structure that “familiar” faces were attacked based on material efficency consideration and not scare considerations). But if you lived there a little while it wasn’t unreasonbale to come to the conclusion that it is mostly free money.
You could see both efficient and inefficient market hypothesis in effect at once in a kid of tug-of-war. In one sense it might have been that players preferring a anxiety and risk free play experience extorted real measurable economic effects. So in that sense there still might not be free lunches, what you have free on one axis you migth suffer on another axis.
Do you have a link?
Upon looking into this it appears he may only have patented one illusion himself, patent number 9358477. Others, like his famous flying illusion 5354238 appear to have been invented by other people. You can search US patents here.
This is good. The illusion is probably resilient enough that you could even point your discovery out and most people wouldn’t acknowledge it unless forced to.
I think there are two ways to read this article. “Markets are inefficient and you can often beat them” or “most systems are fairly broken and you can usually outperform the average person a lot by just using your brain”.
I think the second hypothesis is true. I think the first hypothesis is probably not. Most people can’t generate market beating returns and are better off investing in index funds.