Markets are Anti-Inductive

[Eliezer Yudkowsky]

I suspect there’s a Pons Asinorum of probability between the bettor who thinks that you make money on horse races by betting on the horse you think will win, and the bettor who realizes that you can only make money on horse races if you find horses whose odds seem poorly calibrated relative to superior probabilistic guesses.

There is, I think, a second Pons Asinorum associated with more advanced finance, and it is the concept that markets are an anti-inductive environment.

Let’s say you see me flipping a coin. It is not necessarily a fair coin. It’s a biased coin, and you don’t know the bias. I flip the coin nine times, and the coin comes up “heads” each time. I flip the coin a tenth time. What is the probability that it comes up heads?

If you answered “ten-elevenths, by Laplace’s Rule of Succession”, you are a fine scientist in ordinary environments, but you will lose money in finance.

In finance the correct reply is, “Well… if everyone else also saw the coin coming up heads… then by now the odds are probably back to fifty-fifty.”

Recently on Hacker News I saw a commenter insisting that stock prices had nowhere to go but down, because the economy was in such awful shape. If stock prices have nowhere to go but down, and everyone knows it, then trades won’t clear—remember, for every seller there must be a buyer—until prices have gone down far enough that there is once again a possibility of prices going up.

So you can see the bizarreness of someone saying, “Real estate prices have gone up by 10% a year for the last N years, and we’ve never seen a drop.” This treats the market like it was the mass of an electron or something. Markets are anti-inductive. If, historically, real estate prices have always gone up, they will keep rising until they can go down.

To get an excess return—a return that pays premium interest over the going rate for that level of riskiness—you need to know something that other market participants don’t, or they will rush in and bid up whatever you’re buying (or bid down whatever you’re selling) until the returns match prevailing rates.

If the economy is awful and everyone knows it, no one’s going to buy at a price that doesn’t take into account that knowledge.

If there’s an obvious possibility of prices dropping further, then the market must also believe there’s a probability of prices rising to make up for it, or the trades won’t clear.

This elementary point has all sorts of caveats I’m not bothering to include here, like the fact that “up” and “down” is relative to the risk-free interest rate and so on. Nobody believes the market is really “efficient”, and recent events suggest it is less efficient than previously believed, and I have a certain friend who says it’s even less efficient than that… but still, the market does not leave hundred-dollar-bills on the table if everyone believes in them.

There was a time when the Dow systematically tended to drop on Friday and rise on Monday, and once this was noticed and published, the effect went away.

Past history, e.g. “real estate prices have always gone up”, is not private info.

And the same also goes for more complicated regularities. Let’s say two stock prices are historically anticorrelated—the variance in their returns moves in opposite directions. As soon as everyone believes this, hedge-fund managers will leverage up and buy both stocks. Everyone will do this, meaning that both stocks will rise. As the stocks rise, their returns get more expensive. The hedge-fund managers book profits, though, because their stocks are rising. Eventually the stock prices rise to the point they can go down. Once they do, hedge-fund managers who got in late will have to liquidate some of their assets to cover margin calls. This means that both stock prices will go down—at the same time, even though they were originally anticorrelated. Other hedge funds may lose money on the same two stocks and also sell or liquidate, driving the price down further, etcetera. The correlative structure behaves anti-inductively, because other people can observe it too.

If mortage defaults are historically uncorrelated, so that you can get an excess return on risk by buying lots of mortages and pooling them together, then people will rush in and buy lots of mortgages until (a) rates on mortgages are bid down (b) individual mortgage failure rates rise (c) mortgage failure rates become more correlated, possibly looking uncorrelated in the short-term but having more future scenarios where they all fail at once.

Whatever is believed in, stops being real. The market is literally anti-inductive rather than anti-regular—it’s the regularity that enough participants induce, which therefore goes away.

This, as I understand it, is the standard theory of “efficient markets”, which should perhaps have been called “inexploitable markets” or “markets that are not easy to exploit because others are already trying to exploit them”. Should I have made a mistake thereof, let me be corrected.

Now it’s not surprising, on the one hand, to see this screwed up in random internet discussions where a gold bug argues from well-known observations about the past history of gold. (This is the equivalent of trying to make money at horse-racing by betting on the horse that you think will win—failing to cross the Pons Asinorum.)

But it is surprising is to hear histories of the financial crisis in which prestigious actors argued in crowded auditoriums that, previously, real-estate prices had always gone up, or that previously mortage defaults had been uncorrelated. This is naive inductive reasoning of the sort that only works on falling apples and rising suns and human behavior and everything else in the universe except markets. Shouldn’t everyone have frowned and said, “But isn’t the marketplace an anti-inductive environment?”

Not that this is standard terminology—but perhaps “efficient market” doesn’t convey quite the same warning as “anti-inductive”. We would appear to need stronger warnings.

PS: To clarify, the coin example is a humorous exaggeration of what the world would be like if most physical systems behaved the same way as market price movements, illustrating the point, “An exploitable pricing regularity that is easily inducted degrades into inexploitable noise.” Here the coin coming up “heads” is analogous to getting an above-market return on a publicly traded asset.

[Psy-Kosh]

I think the pons is shaking my asinorum off and into the river (efficiently).

I’m a bit confused. I can understand some of the anti-inductive situations, but not the general principle (other than “stuff bottoms out because there really is a lower bound (zero, obviously (baring items that can become so nasty/​wasteful that you’d be willing to pay someone to take them off your hands, but that’s not really an issue here...)) And upper bounds because even if someone thinks something is a good investment, sooner or later they simply won’t be able to afford it… which means sellers can’t sell it, which means it will get cheaper”

Is this what you were going for? because, if not, I find myself a bit confused.

The knowledge about increasing prices would seem to me to itself push prices to increase faster (ie, everyone thinks something is a good investment, so many many buyers, so high demand, etc… up to the above mentioned limit) while if something is sinking and everyone believes it’s sinking, the demand will get lower, which means to get it sold will take faster price drops, which will lead to more people believing it’s sinking, etc..

I’m probably being stupid here, so can anyone clarify this for me? (Or is it just the bounds that I mentioned making things have to start going the other way eventually?)

[gwern]

The way I think of it: imagine you’re in the middle of Tulipomania and you’re thinking about sinking your entire fortune into a tulip bulb. You reason that the bulb is largely worthless since its function is to look nice and the value only comes from your expectation that you will sell it to someone else for more, who will reason the same way. This whole spiral feeds on something: credit, suckers, etc. The rational thing is to jump off just before the spiral runs out (and being an exponential process, the spiral won’t starve slowly but will crash hard—it won’t be that the banks are demanding somewhat higher interest rates for tulip-backed loans, but that there won’t be any at all). Of course, the other people are reasoning the same way, so they will be looking for the jump-off too; since one of the things the spiral is feeding on is you, your jump will contribute to the crash.

But what sparks you to think that the crash is about to occur (and cause a self-fulfilling prophecy)? Could be some exogenous event like a war or terrorist attack, but there must be one sooner or later: there are only so many tulips to buy & sell and investors are only willing to hold tulips for so long, they want to cash out at some point, but for there to be a seller, there must be a buyer...

If you can calculate this limit, then you know when you must jump off before, and so does everyone else who isn’t a fool, and you become a pack of penguins inching nervously to the edge, wondering who the first one shoved over will be to test the water for killer whales. At that point, a crash is only logical. The upper limit for bubbles is where not enough people can afford to buy the good or to hold the good; the lower limit is where the good is unfairly priced in the sense that it can be diverted to some other use (eg. we can imagine the US dollar depreciating only so far, because if a dollar is cheap enough, we could start buying it in bulk, shredding it, and turning it into fertilizer or linen or something).

I don’t know if this helps you, but it’s the way I see it.

[MoritzG]

In reality there are smart penguins and dumb penguins and penguin news papers. The professional penguins will tell other penguins how great it has been going so they can get out before the ledge breaks of and they all fall into the water.

To realize those booked earnings you have to sell without causing the crash, so you have to setup potential buyers first. That is why I consider articles about investing into something in major papers the last warning before the crash. When I read that the only smart thing to do, is to invest into … I know not too.

[Silas]

Well, now you f*in’ tell me.

[John_Maxwell_Old]

Seconding Psy-Kosh in being confused & unconvinced.

[Jay]

Psy-Kosh:

You are not confused. You have clearly articulated the structure of a market bubble and its resulting crash.

[Z._M._Davis]

Psy and John, I think the idea is this: if you want to buy a hundred shares of OB at ten dollars each, because you think it’s going to go way up, you have to buy them from someone else who’s willing to sell at that price. But clearly that person does not likewise think that the price of OB is going to go way up, because if she did, why would she sell it to you now, at the current price? So in an efficient market, situations where everyone agrees on the future movement of prices simply don’t occur. If everyone thought the price of OB was going to go to thirty dollars a share, then said shares would already be trading at thirty (modulo expectations about interest rates, inflation, &c.).

[Vilhelm_S]

If someone notices that morgage defaults are uncorrelated, and new instruments that exploit that lack of correlation are invented, investors will be more likely to buy securitized morgages, so the rates are bid down and they become cheaper for homeowners. But that in itself should not cause defaults to be more correlated.

In the event, the models for correlation were off, and defaults were more correlated from the start. But that’s not a problem with public/​private knowledge or anything; they were just bad models. If the prizing had been done correctly to start with, I see nothing paradoxical in assuming that introducing CDOs would lead to persistently lower morgage rates—the lack of correlation doesn’t stop being real just because people believe in it.

[nazgulnarsil3]

the general public being unaware of the fact that stock prices are an equilibrium of beliefs about whether the stock will rise or fall is not a major cause for concern.

[Cyan2]

Vilhelm S., companies and people who lose money on CDOs have mortgages to pay and employ people who have mortgages to pay… Once the system gets coupled like that, one unlucky bet can start the cascade. I’m not saying this actually happened, but it’s a mechanism which could falsify the assertion “the lack of correlation doesn’t stop being real just because people believe in it”.

[Russ_Andersson]

Elizer, I accept your general point, but in your coin flipping example, it was unclear to me what your trade would be. Would you bet on heads or tails and in what circumstances? In a real world scenario I suspect its more likely to be the opposite of what I think you suggested but that is not entirely clear to me.

It all depends on where the market price is. There is a theory formed by a relatively respected speculator called “reflexivity,” basically that markets tend to perpetuate trends and overshoot fair values http://​​www.geocities.com/​​ecocorner/​​intelarea/​​gs1.html). And there is some other evidence of this in books such as Bob Schiller’s Irrational Exuberance about information cascades etc. So given there have been 9 heads in a row, maybe your average bear would think its more likely to come up heads than it’s genuine expected value, so I would argue that the market would probably overvalue the true likelihood (which according to you is 10/​​11ths), and so they would bet on heads and you would want to be short (bet on tails) if their expectation/​​price is greater than 10/​​11ths. The difference between their price and your price is called the edge, and you would capture that edge by betting on tails. So it would be a relatively modest profit opportunity. Not the other way round as I think you described, where you seem to be saying that the market would be pricing it at 50/​​50 and fair value is 10/​​11ths and you should bet on heads. It all depends on where the market would trade, above fair value or below fair value, I say above, in which scnenario one should bet on tails, and there is a minor profit opportunity, you seem to be saying below, bet on heads, and there is a major profit opportunity. A related question is how much should you actually bet?

Anyway, maybe just a case of my misunderstanding things. Another, perhaps, simpler way to think of markets are as beauty contests, where you are trying to pick the winner of the contest, which is not the girl you think is most beautiful, but the girl that everybody else thinks is the most beautiful: http://​​en.wikipedia.org/​​wiki/​​Keynesian_beauty_contest

[bigjeff5]

So given there have been 9 heads in a row, maybe your average bear would think its more likely to come up heads than it’s genuine expected value, so I would argue that the market would probably overvalue the true likelihood (which according to you is 10/​11ths), and so they would bet on heads and you would want to be short (bet on tails) if their expectation/​price is greater than 10/​11ths.

That was the point of the coin flip example. It was to point out that the market is not random even if it appears to be about as random as a coin flip. Information from the previous flip factors into the next flip, reducing the likelihood that any given trend will continue.

What I think you are missing is the fact that everybody knows that the way to take advantage of an inflated price is to sell short—it is not a unique insight on your part. Since it’s common knowledge, obviously everybody knows that everybody else knows that the way to exploit this situation is to sell short. Therefore there is a very high probability that a significant portion of the market will bet on tails to capture the edge. This destroys your likelihood of successfully beating the edge, because too many people are going to attempt the same thing you are. Those who recognize this (and there are many) also know that the next level of exploitation is to bet on heads.

The end result is a wash—there is a ⁵⁰⁄₅₀ chance that the 11′th flip will be heads, not a ¹⁰⁄₁₁ chance like traditional probability suggests, because everybody is trying to out-exploit everybody else. The market is anti-inductive.

Think of poker. Someone who knows the probabilities of poker hands can win in far more situations than someone who does not. They will know when to bet and when to fold, maximizing their success. However, when playing with players who know the probabilities of poker hands this strategy becomes much less successful. Because everyone is only playing cards they can win with, it is only the really lucky players who get more good cards than bad who come out ahead.

However, by exploiting the likelihood of given cards, they can pretend they have cards they do not have and convince the rest of the players to give up. This makes their probability of winning skyrocket, and the net result is that the best hand winds far less often than probability suggests. This is the result of everybody knowing how to exploit the probabilities—everybody knows to go for the edge.

In high level poker, however, the best hand wins 99% of the time. Each particular hand wins at almost exactly the rate its likelihood of appearing suggests. Why? Because everybody at the table knows the probabilities for a given hand, and everybody is going to attempt to exploit those probabilities. Furthermore, everybody knows that everybody knows this, so it is only on extremely rare occasions that someone is actually able to exploit the probabilities and win with a weaker hand. In this scenario it is extremely difficult to inflate the value of the cards you are holding by bluffing, because the person you are bluffing knows the likelihood that you have the hand you are pretending to have and can compare that to their own hand to get their chances of winning. It still works on occasion though, and can be pretty spectacular.

This is the efficiency of the market. It is anti-inductive because information flows freely. As soon as an exploit is discovered it is nullified by the fact that it cannot be hidden, and everybody will therefore take advantage of it.

[jefftk]

In high level poker, however, the best hand wins 99% of the time.

Really? This seems surprisingly high.

[skeptical_lurker]

Mostly because its not actually true. If bluffing only worked 1% of the time, then no-one would bluff, so people would just fold mediocre hands against bets, so bluffing works again. If you solved some simplfied version of poker, so everyone is playing according to the exact Nash equibrium, there would still be plenty of bluffing.

[bigjeff5]

That was four years ago, but I’m pretty sure I was using hyperbole. Pros don’t bluff often, and when they do they are only expecting to break even, but I doubt it’s as low as 2% (the bluff will fail half the time).

I’d also put in a caveat that the best hand wins among hands that make it all the way to the river. There are plenty of times where a horrible hand like a 6 2, which is an instant fold if you respect the skills of your fellow players, ends up hitting a straight by the river and being the best hand but obviously didn’t win. Certainly more often than 1%, and there are plenty of better hands that you still almost always fold pre-flop that are going to hit more often.

So, at best it was poorly stated (i.e. hyperbole without saying so), at worst it’s just wrong.

[skeptical_lurker]

the bluff will fail half the time

I know you’re using hyperbole, but I’m going to do the calculations anyway :) If you bet a fraction x of the pot, with prob p of winning, and no outs, then your EV is p-(1-p)x. Clearly, EV>0 for optimal play, and a half-pot sized bet is common, so p-(1-p)/​2>0 ⇒ p>1/​3.

So the bluff should succeed at least ¹⁄₃ of the time.

Now suppose I have made some large bets, and you think I have at least JJ with 95% prob, and am bluffing with junk with 5% prob. I think you can beat JJ with 30% probability. I might chose to bet half the pot with all my possible hands (I’m now playing a probability distribution, not a hand), in which case you have to fold with 70% of your hands because 0.05 (1+0.5)<0.95 1. So in this case, my bluff succeeds 70% of the time, with EV 0.7-(1-0.7)/​2=0.55.

Of course this is a massively simplified example.

Apparently, according to a book I read, if two pros playing head up no-limit are dealt 9 4, the author estimated that the person who has position (plays second) has around ²⁄₃ chance of winning by bluffing his opponent off the hand, and of course the person who plays first might win by bluffing as well. So this seems to indicate that there is a reasonable chance to win by bluffing.

Overall, I think pros don’t make so many dramatic all-in bluffs, and in fact tend to semi-bluff, by betting with hands that have outs anyway.

[Doug_S.]

But clearly that person does not likewise think that the price of OB is going to go way up, because if she did, why would she sell it to you now, at the current price?

Maybe she needs to increase liquidity for some reason? For example, the IRS wants payment in cash, not stock. Additionally, she might want to use the resources for immediate consumption rather than for investments; she might want to use the cash to take a vacation, or pay medical bills. There are all sorts of reasons why someone might want to sell a stock, even if they think it will go up.

(Myself, I sort of suspect that, if a stock doesn’t pay dividends, it’s mostly worthless. To quote some guy with a blog:

If you put your Mickey Mantle rookie card on your desk, and a share of your favorite non-dividend paying stock next to it, and let it sit there for 20 years. After 20 years you would still just have two pieces of paper sitting on your desk.)

[behemoth]

Eliezer, your main point is correct and interesting, but the coin flip example is definitely wrong. The market’s beliefs don’t affect the bias of the coin! The map doesn’t affect the territory.

The relevant FINANCE question is ‘how much would you pay for a contract that pays $1 if the coin comes up heads?’. This is then the classic prediction market type contract.

The price should indeed be ten elevenths. Of course, you don’t expect to make money buying this contract, which was exactly your point.

What WILL be true is that the expected change in the price of the contract from one period to the next will be zero. This need not mean that it goes up 50% of the time, but the expected value next period (in this case) is the current price.

The first proof that I know of this was done by Paul Samuelson in 1965, in his paper ‘Proof that Properly Anticipated Prices Fluctuate Randomly’.

[kebko]

I think this is why there will always be opportunities for high returns. Quantifiable correlations will always be bid away. But, there will always be unquantifiable uncertainty. Much of investing is earning rents on uncertainty, so that if you’re even slightly skilled at picking the right uncertainties, you can make huge returns. It seems to me that finance seems to either quantify uncertainties into risk models or pretends they aren’t important. But, for an individual investor they can be the most important thing to consider, and due to the very nature of uncertainty, it’s not likely to be anti-inductive. That’s why great investors like Buffett & Munger are as skilled at recognizing biases people have in the face of uncertainty as they are at doing finance. If you asked Charlie Munger what it took to be a good investor, I bet he’d be more likely to give you a rundown of biases than to offer anything quantitative.

[Z._M._Davis]

Doug, I meant ceteris paribus.

[Russ_Andersson]

Behemouth, i’m not sure the coin flip example is completely wrong, perhaps the fair thing to say would be that its open to interpretation, but Elizer can clarify his thinking for us.

Elizer seems to be saying that the market price will be around 50%, fair value is 91%, and you should bet heads on that basis, and should expect to capture an edge of 41%.

Another alternative is that the market price could be above 91%, say 95%, and in that instance you should bet on tails, and expect to capture only a 4% edge.

We could informally test this … we could ask/​email 20 people if given they know a coin is baised and given that it just landed heads 9 times in a row, how many heads in 100 tosses will the coin produce. Their responses could become the market price, if that number averages out to say 90 heads, then the market price is 90% for example, and then we could see if a bet on heads or tails would be more appropriate. I am not sure of the final answer, but I think its unlikely that the market price will be near ⁵⁰⁄₅₀.

[Eliezer Yudkowsky]

See added PS. The coin example is intended as a humorous exaggeration of the way the world would be if most physical systems behaved like market prices, with the coin coming up “heads” being analogous to prices rising.

[Doug_S.]

Once upon a time, I thought I saw some important news that should have affected a company’s stock price, as it would have led to significantly decreased demand for one of the company’s flagship products.

I looked at the stock price, and after the trade show in which the announcement was made, the value of the stock had, to my great surprise, actually increased.

That company was Sony. The announcement was that Final Fantasy XIII would no longer be a Playstation 3 exclusive; its English version would also be released for the Xbox 360. As a game player, I knew that Final Fantasy was a game series that was popular enough to drive sales of whatever console it was available for, regardless of its other merits. However, this announcement meant that many people who might have bought a Playstation 3 would no longer do so. As the original Playstation and the Playstation 2 were one of Sony’s largest revenue sources, further bad news regarding the the Playstation 3′s future should have negatively affected its stock price. Why didn’t it?

I came up with three guesses:

1) Average stock traders don’t know as much as I do about the video game market. This is possible, but “Hey, Sony just lost its exclusive Killer App!” should be something that anyone actually paying attention should notice—stock traders aren’t that stupid, are they? 2) Sony is a huge conglomerate. It sells so many other products that bad news in one area either just didn’t matter (nobody expected the Playstation 3 to sell as many units as its earlier versions, for the simple reason that it was much more expensive) or it was outweighed by good news about other markets, such as digital cameras. 3) There is some other explanation, which I have not yet thought of.

[Nick_Tarleton]

but still, the market does not leave hundred-dollar-bills on the table if everyone believes in them.

People fail to act on stated beliefs all the time. Why wouldn’t this sometimes happen in the market?

Doug: Sony loses money on every PS3, so it would be bad news if people bought them for FF and bought few other games; I don’t know how plausible that is.

[behemoth]

Russ, I think that if you take the example literally, the price would be 91%, not 50%, and you wouldn’t expect to make money.

Eliezer, the PS definitely clarifies matters.

Although I also think the example is actually instructive if taken literally too. In particular, if you see nine heads in a row, each additional head means you expect a higher chance of heads next flip. But you do not expect an increase in the price of the contract that pays $1 if heads comes up. THAT still has an expected price change of zero, even thought we expect more heads going forward.

In other words, future EVENTS can be predictable, but future PRICES cannot.

[John_Maxwell_Old]

Eliezer does a good job of explaining a mechanism by which two investments with negatively correlated returns can switch to having positively correlated returns. But he doesn’t do a good job of convincing me that a stock’s price has a tendency to go down when it has just gone up, and vice versa.

I can think of an argument against this position. It seems plausible that stock traders see the past movement of a stock as an indicator of it’s future movement. If a majority of traders share this belief, this will compel them to buy the stock from those who don’t, inflating it’s value and reinforcing the cycle. This would indicate that markets are inductive, which is the opposite of what the title suggests.

[MTGandP]

At some point, all traders with this belief will have already bought the stock and the price will stop going up at that point, thus making the price movement anti-inductive.

[Doug_S.]

Doug: Sony loses money on every PS3, so it would be bad news if people bought them for FF and bought few other games; I don’t know how plausible that is.

Yeah, that would probably be bad, too; a PS3 sold that doesn’t generate much additional revenue is just a straight loss. On the other hand, once you’ve already decided to get the system because of the blockbuster title that you Need To Have, non-blockbuster games for the system become more attractive than they were before, so the pattern of “very few games” over the entire lifetime of the system isn’t really all that likely. In the short run, though, it could definitely be an issue. Personally, I currently have only two PS3 games: Disgaea 3, and Metal Gear Solid 4. I expect to get more eventually, but not for a while.

[Mike_Blume]

John: I don’t think Eliezer’s saying that a stock that has recently risen is now more likely to fall. Quite the opposite in fact. Any given stock should be about as likely to fall as to rise, at least if we weight by the amount of the rise. That is, if I hold a share of XYZ, which costs $100, and I anticipate a 99% chance that the stock will rise to $101 tomorrow, then I should also expect a 1% chance that the stock will drop to $1 tomorrow. Were that not true, the share would be worth nearly $101 right now, not tomorrow.

See also: Conservation of Expected Evidence

[Arthur_B.]

Correlation is not “arbitraged away” because there’s no inherent arbitrage in correlation. I think in your first example, you had in mind pairs trading where there is lagged correlation, or negative autocorrelation of the spread. In the second example, mortgages, what you’re saying’s just wrong.

An important aspect that you also omit to mention is that efficient market means the risk free expectation is the risk free rate. That does not mean that up and down are equally likely. A junk bond is much more likely to go up than down, only it’ll go up a little and down a lot. Returns can be skewed.

Last point, arbitrages are costly and often risky (the information available is not certain). You may not have more information than other people, but you may be less risk averse or you may be able to arbitrage at a lower cost.

[waveman]

Last point, arbitrages are costly and often risky (the information available is not certain). You may not have more information than other people, but you may be less risk averse or you may be able to arbitrage at a lower cost.

There is a googleable paper “the limits of arbitrage” by Shleifer which explains this concept in detail. Well worth the read

[From the original article] To get an excess return—a return that pays premium interest over the going rate for that level of riskiness—you need to know something that other market participants don’t, or they will rush in and bid up whatever you’re buying (or bid down whatever you’re selling) until the returns match prevailing rates.

There are other ways to get an excess return. They are all hard to do. Here is a partial list.

• Exploit (or serve) someone else’s need for liquidity. People buying in Japan now may be doing this.

• Taxation arbitrage. Provide trades that allow someone else to improve their after-tax position eg splitting a bond into coupon payments (tax-free to a non-profit) and capital gains (taxable at a concessional rate to other people).

• Exploit agency problems. Eg If you can take the other side of a trade from someone who is trying to massage short term earnings regardless of longer term impact you can win. One example of this is trend following, where you mostly lose, but occasionally win big. The other side of the trade is the counter-trend trader, who wins month after month (and gets big bonuses for doing so), then their employer has a big loss. But hey, the trader already got his bonuses.

• Exploit large numbers of stupid people such as dot.com investors. Sometimes there are so many of them they overwhelm the smart money. By getting your timing right you can exploit them. George Soros does this a lot.

• Exploit impatience. See Shleifer’s article referenced above. For a fund manager, to win in the long term is useless, because his investors pulled their money long ago and his fund was closed down.

[Jeremy2]

(Myself, I sort of suspect that, if a stock doesn’t pay dividends, it’s mostly worthless. To quote some guy with a blog:

If you put your Mickey Mantle rookie card on your desk, and a share of your favorite non-dividend paying stock next to it, and let it sit there for 20 years. After 20 years you would still just have two pieces of paper sitting on your desk.) Maybe it would help to think of a sole proprieter running a small retail business who starts with perhaps $500k in personal equity. He pays himself a relatively meager salary and invests all other net income back into the business to increase inventory, advertise, move to a better location or open another branch etc. After running this business for forty years its expanded successfully and worth $8MM dollars, even though it hasn’t paid a single dividend. Did he waste the last forty years of his life? Do you think he’ll be able to extract that earned value?

From there you should be able to see that a privately held corporation (only owned by family) could work exactly the same, and from that its not a leap at all to see how a publically held company can increase in value without ever actually extracting the value from it.

3) There is some other explanation, which I have not yet thought of.

Maybe the sales of the game on the other platform will be more profitable than the PS3 sales they miss out on? The margin for games is much higher than for consoles and you definitely can broaden your market by including 360 owners who do not wish to shell out $400 just to play that game. Is it conceivably possible that Sony knows as much or more about the gaming industry as you do?

[Mike_Blume]

Jeremy: we’re drifting from the topic, but I don’t believe the Final Fantasy games are produced, distributed, or sold by Sony. Thus the decision to release FF for multiple platforms was not a decision made by Sony, simply one which affected Sony.

[Jay]

Doug and Jeremy:

Generally, the fundamental value of a stock is determined by one of two things. It can be defined by the net present value of the future dividends of the stock. Alternately, it can be determined by the per-share liquidation value of the company’s assets (after creditors are paid). Which one applies or what mix depends on whether and how the company will be liquidated.

The rest is a mix of market psychology and adjustments for risk/​uncertainty.

A big wrinkle is that the many investors look for companies with major earnings growth potential. As a result, most traded companies try to give the appearance of major earnings growth potential. One way to signal growth potential is to allocate little or no cash to dividends, because using the cash for expansion (or for activities that superficially resemble expansion) signals that the company has strong growth opportunities. If investors believe this signal, then the stock price rises from its fair value as an existing business to its expected fair value as a much bigger company in the future. The management rewards everyone with bonuses and all is good, until the future arrives disappoints everyone.

I was going for a peacock analogy, but perhaps lemmings would be more appropriate.

[[deleted]]

“”Technical analysis bases decisions on past results. EMH, however, believes past results cannot be used to outperform the market. As a result, EMH negates the use of technical analysis as a means to generate investment returns.

With respect to fundamental analysis, the EMH also states that all publicly available information is reflected in security prices and as such, abnormal returns are not achievable through the use of this information. This negates the use of fundamental analysis as a means to generate investment returns.”″

[Doug_S.]

It can be defined by the net present value of the future dividends of the stock. Alternately, it can be determined by the per-share liquidation value of the company’s assets (after creditors are paid).

So if the stock does not pay dividends, and never will, and the corporation’s assets equal its liabilities, and always will, then the appropriate value of the stock is, in fact, zero? (Well, there are voting rights, but still...)

[Trader_M]

It all depends on where the market price is. There is a theory formed by a relatively respected speculator called “reflexivity,” basically that markets tend to perpetuate trends and overshoot fair values

Yes, this is how I make day trading profits. Wait until there is a big crowd on the “wrong side” of the trade, watch for the signs of them starting to panic, and make $$$ when they all go stampeding through the doorway driving the price in your direction. This is a good example.

[Jeremy2]

It can be defined by the net present value of the future dividends of the stock. Alternately, it can be determined by the per-share liquidation value of the company’s assets (after creditors are paid).

So if the stock does not pay dividends, and never will, and the corporation’s assets equal its liabilities, and always will, then the appropriate value of the stock is, in fact, zero? (Well, there are voting rights, but still...)

No, the value also includes the NPV of future net earnings, even if they are not currently being paid as dividends. When you value a business for the purpose of selling it, revenue, assets and intangibles like brand awareness are all things you look at.

Also if a company’s assets = its liabilities then it has not been profitable or its losing money (or its a very new business). Obviously failing businesses are not worth anything and stock prices reflect that; but a business that is earning a net profit has a value regardless of if dividends are paid. If dividends are not paid they are re-invested in the business, which isn’t really different from you getting the dividend and investing it in that or some other stock except that you are giving the company’s management credit for making wise investments (bonuses are an expense and are sub-tracted from net income and so are not relevant here).

[Will_Pearson]

All competitive situations against ideal learning agents are anti inductive in this sense. Because they can note regularities in their actions and avoid them in the future as well as you can note regularities in their actions and exploit them. The usefulness of induction is based on the relative speeds of the induction of the learning agents.

As such anti induction appears in situations like bacterial resistance to antibiotics. We spot a chink in the bacterias armour, and we can predict that that chink will become less prevalent and our strategy less useful.

So I wouldn’t mark markets as special, just the most extreme example.

[Michael_E_Sullivan2]

kebko: ” Much of investing is earning rents on uncertainty, so that if you’re even slightly skilled at picking the right uncertainties, you can make huge returns.”

The last year has demonstrated that it is not always trivial to differentiate between earning rent on uncertainty and running a variation on the martingale.

IMO, a fair amount of the profit in the highest flying risk arbitrage schemes is based on leveraging with the bankruptcy put.

If I am allowed to run martingales with an exposure of 5-10 times my actual capital, I will make me and everyone who invests with me some very high and consistent returns most of the time. But when the returns go bad, they go devastatingly bad. This is pretty much what happened in large segments of the financial markets over the last 20 years. 20% across the board drops in housing prices was the casino cutting us off before we could double down again.

[waveman]

IMO, a fair amount of the profit in the highest flying risk arbitrage schemes is based on leveraging with the bankruptcy put.

Yes! Selling insurance, but with no capital to back it up.

If these people were smart they would take money off the table as they go. But the trouble is that they starting drinking their own kool-aid. Eg Long Term Capital Management.

[Jay]

Jeremy and Doug,

There’s less disagreement than you think here. Future net earnings enter the equation as future dividends (appropriately discounted) or share buybacks (driving up farther-future dividends and per-share liquidation value). I didn’t specify that assets on liquidation had to be tangible; trademarks and the like are valuable (both because they improve the business as it operates and because they can be sold on liquidation).

If earnings are reinvested in the company, that’s fine. The investments will be profitable or not; profitable investments will result in even more earnings. The cycle can continue until either (1) management decides that it has no further opportunity for reinvestment, and returns the now-compounded earnings to shareholders as dividends, or (2) investments are made that are unprofitable, and the money is wasted. Option 2 is a surprisingly popular choice.

In any case, a stock that has no dividends, no liquidation value, and no prospect of either of those facts ever changing is worthless. You may be able to convice a greater fool to buy it, but there’s no actual wealth attached to the stock.

[homunq_homunq]

What about Moore’s Law?

I understand it is a totally different situation. There is no opponent, and plenty of positive feedback. However, I still think that “up so far, so up forever” is just as much of a fallacy with chips as with markets.

[Eric6]

Doesn’t this mean that once everyone realizes that markets are anti-inductive, markets will become inductive?

[Liron]

No, “markets are anti-inductive” means “markets act to lower the Bayes-score of low-entropy probability distributions over future asset prices”.

If people realize that markets are anti-inductive, that doesn’t cause their belief state about future asset prices to have less entropy. On the contrary, it makes them think twice before entering a low-entropy belief state.

[SilasBarta]

Well, of course! After all, it never worked before...

[Chris_H]

As well as a towel, an interstellar hitchhiker should pack a can opener.

[Alexei_Turchin]

This resoning works good if I assume that I have average IQ—and so my predictions are averege and so I should not try to predict the market. But most people think that they are smarter then most people. So do the traders.

Smartest traders can first notice trends in psevdorandom enviroment. But most traders mistakenly think that they are smartest.

[waveman]

There are some pretty smart traders. But there is also a lot of dumb money.

The classic example of this was Peter Lynch, a legendary stock trader. His fund ( Magellan) outperformed the market greatly, especially in the early years. But if you weight his returns by the capital in his fund, it actually underperformed.

What this means is that the investors in his fund managed to more than completely negate his stock-picking skill by adversely timing their entries and exits from his fund. Amazing!

[[deleted]]

People who say “we can predict the market” are factually wrong, because the thing to do with a prediction is exploit it, and the moment you exploit it you invalidate your own prediction.

[wedrifid]

People who say “we can predict the market” are factually wrong, because the thing to do with a prediction is exploit it, and the moment you exploit it you invalidate your own prediction.

Exploiting and predicting is not inherently contradictory. You can predict your own exploitation. In fact, that is exactly what you do when making the expected utility calculation.

[[deleted]]

My farther has a degree in organization theory, and frequently encounters people who say they can predict the market, but often even fail to take into account their own interactions.

[wedrifid]

My farther has a degree in organization theory, and frequently encounters people who say they can predict the market, but often even fail to take into account their own interactions.

My father has a degree in Industrial Chemistry, my cousin has a degree in Visual Arts and those specific people that your father talks about are wrong. Most other people who claim to be able to predict the market also happen to be wrong—people are often silly and overconfident and markets are hard to predict even when only moderately efficient.

Nevertheless, the claim:

People who say “we can predict the market” are factually wrong, because the thing to do with a prediction is exploit it, and the moment you exploit it you invalidate your own prediction.

… is muddled thinking.

[[deleted]]

Or poor wording. Retracted.

And another tick in the “Don’t comment on LW at 2 AM” box.

[A1987dM]

Not that this is standard terminology—but perhaps “efficient market” doesn’t convey quite the same warning as “anti-inductive”. We would appear to need stronger warnings.

“Reverse Tinkerbell effect”.

[zanglebert]

What a beautiful article. Too bad I find it only now, most likely too late to enter a discussion about it. I will leave a not-too-long comment on it anyway, even at the risk of total tumbleweed.

First, much respect for a precise and concise description of the effect, and coming up with a great label for it at the same time. Don’t know if the term anti inductive in relation to markets originated here, but it seems everything on the Internet at least refers back to this post.

Now, I entirely agree on the description of the anti inductive property of markets. I would also add: this anti inductive property is precisely the reason why markets tend towards efficiency—assuming enough participants with enough talent and/​or training are participating. Note: “tend towards” does not necessarily imply “reach it”, hence few if any markets are “efficient, period”.

That said, here is the point at which I respectfully disagree: I don’t think markets can be fully described as anti inductive.

I would add: the anti inductive property is only one of two key properties of markets. The other one being—you guessed it—the inductive property of markets.

This inductive element is at play before a combined and weighted (in terms of capital) observation effect destroys the observed pattern, the one that everyone naively believes in and thinks will go on forever. This inductive element is perhaps the main reason why a trend accelerates for some time—until it breaks under its own weight.

I would then form—somewhat simplistic, for the sake of brevity—the following hypothesis:

The inductive element of markets is related to, and perhaps equivalent with, what traders or technical analysts traditionally call momentum.

It is the tendency of market participants to look for patterns, find and observe them at their beginning, then to act upon it, and thereby, strengthening it.

The anti inductive element is what traders and analysts refer to as return to mean.

Any trend must end at some point, usually after it becomes overextended. Which often happens rather violently, thus forming a new trend in “the opposite direction”.

This takes place if a critical mass of market capital comes to the conclusion that the pattern is done and finished, and then acts according to that insight. In addition, their combined insight has to “right” of course. Either, in correctly identifying that the pattern was becoming overextended, or alternatively, by effectively make the pattern falter through their sheer capital’s actions’ force.

An aside: these two possibilites (that the anti inductive market forces were right, or that their actions made them right) are, in my view, essentially indistinguishable on markets. That’s more or less the corollary of the claim “the market can be wrong longer than you can be solvent”.

[Benquo]

Might want to correct “mortage” to “mortgage”.

[dankane]

This seems like too general a principle. I agree that in many circumstances, public knowledge of a pattern in pricing will lead to effects causing that pattern to disappear. However, it is not clear to me that this is always to case, or that the size of the effect will be sufficient to complete cancel out the original observation.

For example, I observe that two different units of Google stock have prices that are highly correlated with each other. I doubt that this observation will cause separate markets to spring up giving wildly divergent prices to different shares of the same stock. I also note that stock prices are always non-negative. I also doubt that this will cease to be the case any time soon.

Although these are somewhat tautological, one can imagine non-tautological observations that will not disappear. If stocks A and B are known to be highly correlated, this may well lead to a larger gap as hedge funds who predict a small difference in expected returns will buy one and short the other. However, if they are correlated for structural reasons part of this might be that it is hard to detect effects that will cause their prices to diverge significantly, so the observation of the effect will likely not be enough to actually remove all of the correlation.

One can also imagine general observations about the market itself, like the approximate frequency of crashes, or log normality of price changes that might not disappear simply because they are known. In order for an effect to disappear there needs to be a way to make a profit off of it.

[a gently pricked vein]

Is it unfair to say that prediction markets will deal with all of these cases?

I understand that’s like responding to “This is a complicated problem that may remain unsolved, it is not clear that we will be able to invent the appropriate math to deal with this.” with “But Church-Turing thesis!”.

But all I’m saying is that it does apply generally, given the right apparatus.

[dankane]

Unless you can explain to me how prediction markets are going to break the pattern that two different shares of the same stock have correlated prices.

I’m actually not sure how prediction markets are supposed to have an effect on this issue. My issue is not that people have too much difficulty recognizing patterns. My issue is that some patterns once recognized do not provide incentives to make that pattern disappear. Unless you can tell me how prediction markets might fix this problem, your response seems like a bit of a non-sequitur.

[UchuuStranger]

Your point about any existing stock market exploits quickly degrading into noise has some truth to it, information such as news and events and financial documents is indeed the most important fundamental aspect of trade. But I’d like to point out that EMH proponents are known for comparing stock trading with casino gambling and claiming that technical analysis is useless. I’m not sure if you endorse these statements or not, and I don’t want to knock down a strawman in case you don’t, but I think that EMH overestimates the rationality of market participants and treats them as homogeneous automatons that always arrive to the same conclusions.

Even if information the participants have is the same (which is still not strictly true), there are different ways to interpret that information. For example, people who looked into how Bitcoin works invested in it and got rewarded for their insight while mainstream economists remain skeptical.

There’s no fundamental difference between investing in a stock market and investing in a business. Humans do handpick the stocks better than a blindfolded monkey throwing darts randomly, and it is empirically evident that technical analysis does give a bit of the edge in trade. These and some other arguments can be read here: https://​​mises-media.s3.amazonaws.com/​​rae10_2_2_5.pdf