I thought of this recently upon reading an article claiming that the multiverse interpretation of quantum mechanics is appealing because it is so simple. Really?? The multiverse explanation proposes the creation of an infinitude of new universes at every instant. To me, that makes it an egregiously complex hypothesis. But if someone decides that it is simple, I have no basis for refutation, since the notion of what it means for a theory to be simple has never been specified.
I’ve read part way through the article. The first paragraph seemed to be carrying on a continuing conversation (John Searle comes to mind). Then it seemed to change direction abruptly, addressing a problem in mechanism design, namely how to assign payoffs so as to incentivise an agent in a certain game to be honest about his predictions.
These are interesting topics, but I struggle to see the relevance.
EY’s article is also very long. I haven’t read it to the end. Can you point out where to look or, better, summarise the point you were making?
I or Habryka might be able to summarize the key points sometime later, but one of the important bits here is that LessWrong is generally a site where people are expected to have read through the sequences (not necessarily meaning that you have to right away, they are indeed super long. But if you’re going to pose questions that are answered in the sequences, longterm users will probably ask that you read them before putting in time clarifying misunderstandings)
(I realize this is a huge ask, but the site is sort of built around the notion that we build knowledge over time, rather than rehashing the same arguments over and over. This does mean accumulating more and more background reading, alas. We do have projects underway to distill the background reading into smaller chunks but it’s an ongoing process)
Raemon, I understand your remark. But I’ve detected another problem. I’ve dropped the ball by posting my reply to the wrong remark. So, I’m going to have to do some cutting and pasting. Please bear with me.
The EY article really is super long (but interesting) and seems to go all over the place. I’d like to do habryka the courtesy of an answer reasonably promptly. I hope I’m not out of order by asking habryka for guidance about what is on his mind.
I’m gratified to see my humble contribution receive attention, including from you. I’m learning. So thanks.
This is my first independent posting (I’ve commented before) and I didn’t notice it appearing in the front page “latest posts”. I understand you are a LW organisor. Can you help me understand the trigger criteria for an article to appear under “latest posts”? Thanks a lot! JH
Latest Posts follows a hackernews algorithm, where things appear in Latest Posts based on how much karma they have, and how recent they are. Your post has relatively low karma (most likely because the topic wasn’t that novel for LW readers), so it probably appeared for a few hours in the Latest Posts column and then eventually moved off the bottom of the page.
(By now, a 4 days later, it most likely is appearing in Latest Posts but you have to click ‘load more’ many times before it’ll show up)
It wasn’t my purpose to open a discussion of interpretation of quantum mechanics. I only took this as an example.
My point is something else entirely: scientists have been leaning very heavily on William of Occam for a long while now. But try to pin down what they mean by a the relative complexity of an explanation, and they shrug their shoulders.
It’s not even the case that scientists disagree on which metric to apply. (That would just be normal business!) But, as far as I know, no one has made a serious effort to define a metric. Maybe because they can’t?
Yes, and the sequence (as well as the post I linked below) tries to define a complexity measure based on Solomonoff Induction, which is a formalization of Occam’s Razor.
I have the impression that Solomonoff Induction provides a precise procedure to a very narrow set of problems with little practical applicability elsewhere.
How would you use Solomonoff Induction to choose between the two alternative theories mentioned in the article: one based on Newton’s Force Laws, the other based on the principle of least action. (Both theories have the same range of validity and produce the identical results).
But it isn’t very successful, because if you cast SI on terms of a linear string of bits, as is standard, you are building in a kind of single universe assumption.
if you cast SI on terms of a linear string of bits, as is standard, you are building in a kind of single universe assumption.
First, I assume you mean a sequential string of bits. “Linear” has a well defined meaning in math that doesn’t make sense in the context you used it.
Second, can you explain what you mean by that? It doesn’t sound correct. I mean, an agent can only make predictions about its observable universe, but that’s true of humans too. We can speculate about multiverses and how they may shape our observations (e.g. the many worlds interpretation of QFT), but so could an SI agent.
I think you’re example of interpreting quantum mechanics gets pretty close to the heart of the matter. It’s one thing to point at solomonoff induction and say, “there’s your formalization”. It’s quite another to understand how Occam’s Razor is used in practice.
Nobody actually tries to convert the Standard Model to the shortest possible computer program, count the bits, and compare it to the shortest possible computer program for string theory or whatever.
What you’ll find, however; is that some theories amount to other theories but with an extra postulate or two (e.g. many worlds vs. Copenhagen). So they are strictly more complex. If it doesn’t explain more than the simpler theory the extra complexity isn’t justified.
A lot of the progression of science over the last few centuries has been toward unifying diverse theories under less complex, general frameworks. Special relativity helped unify theories about the electric and magnetic forces, which were then unified with the weak nuclear force and eventually the strong nuclear force. A lot of that work has helped explain the composition of the periodic table and the underlying mechanisms to chemistry. In other words, where there used to be many separate theories, there are now only two theories that explain almost every phenomenon in the observable universe. Those two theories are based on surprisingly few and surprisingly simple postulates.
Over the 20th century, the trend was towards reducing postulates and explaining more, so it was pretty clear that Occam’s razor was being followed. Since then, we’ve run into a bit of an impasse with GR and QFT not nicely unifying and discoveries like dark energy and dark matter.
This specific topic has also been addressed quite extensively in the sequences. See the complete quantum mechanics sequence: https://lesswrong.com/s/Kqs6GR7F5xziuSyGZ
Help me out here, habryka.
I’ve read part way through the article. The first paragraph seemed to be carrying on a continuing conversation (John Searle comes to mind). Then it seemed to change direction abruptly, addressing a problem in mechanism design, namely how to assign payoffs so as to incentivise an agent in a certain game to be honest about his predictions.
These are interesting topics, but I struggle to see the relevance.
EY’s article is also very long. I haven’t read it to the end. Can you point out where to look or, better, summarise the point you were making?
Thanks a lot!
I or Habryka might be able to summarize the key points sometime later, but one of the important bits here is that LessWrong is generally a site where people are expected to have read through the sequences (not necessarily meaning that you have to right away, they are indeed super long. But if you’re going to pose questions that are answered in the sequences, longterm users will probably ask that you read them before putting in time clarifying misunderstandings)
(I realize this is a huge ask, but the site is sort of built around the notion that we build knowledge over time, rather than rehashing the same arguments over and over. This does mean accumulating more and more background reading, alas. We do have projects underway to distill the background reading into smaller chunks but it’s an ongoing process)
Raemon, I understand your remark. But I’ve detected another problem. I’ve dropped the ball by posting my reply to the wrong remark. So, I’m going to have to do some cutting and pasting. Please bear with me.
The EY article really is super long (but interesting) and seems to go all over the place. I’d like to do habryka the courtesy of an answer reasonably promptly. I hope I’m not out of order by asking habryka for guidance about what is on his mind.
HI Raemon,
I’m gratified to see my humble contribution receive attention, including from you. I’m learning. So thanks.
This is my first independent posting (I’ve commented before) and I didn’t notice it appearing in the front page “latest posts”. I understand you are a LW organisor. Can you help me understand the trigger criteria for an article to appear under “latest posts”? Thanks a lot! JH
Latest Posts follows a hackernews algorithm, where things appear in Latest Posts based on how much karma they have, and how recent they are. Your post has relatively low karma (most likely because the topic wasn’t that novel for LW readers), so it probably appeared for a few hours in the Latest Posts column and then eventually moved off the bottom of the page.
(By now, a 4 days later, it most likely is appearing in Latest Posts but you have to click ‘load more’ many times before it’ll show up)
hi habryka,
It wasn’t my purpose to open a discussion of interpretation of quantum mechanics. I only took this as an example.
My point is something else entirely: scientists have been leaning very heavily on William of Occam for a long while now. But try to pin down what they mean by a the relative complexity of an explanation, and they shrug their shoulders.
It’s not even the case that scientists disagree on which metric to apply. (That would just be normal business!) But, as far as I know, no one has made a serious effort to define a metric. Maybe because they can’t?
A very unscientific behaviour indeed!
Yes, and the sequence (as well as the post I linked below) tries to define a complexity measure based on Solomonoff Induction, which is a formalization of Occam’s Razor.
I have the impression that Solomonoff Induction provides a precise procedure to a very narrow set of problems with little practical applicability elsewhere.
How would you use Solomonoff Induction to choose between the two alternative theories mentioned in the article: one based on Newton’s Force Laws, the other based on the principle of least action. (Both theories have the same range of validity and produce the identical results).
But it isn’t very successful, because if you cast SI on terms of a linear string of bits, as is standard, you are building in a kind of single universe assumption.
First, I assume you mean a sequential string of bits. “Linear” has a well defined meaning in math that doesn’t make sense in the context you used it.
Second, can you explain what you mean by that? It doesn’t sound correct. I mean, an agent can only make predictions about its observable universe, but that’s true of humans too. We can speculate about multiverses and how they may shape our observations (e.g. the many worlds interpretation of QFT), but so could an SI agent.
I think you’re example of interpreting quantum mechanics gets pretty close to the heart of the matter. It’s one thing to point at solomonoff induction and say, “there’s your formalization”. It’s quite another to understand how Occam’s Razor is used in practice.
Nobody actually tries to convert the Standard Model to the shortest possible computer program, count the bits, and compare it to the shortest possible computer program for string theory or whatever.
What you’ll find, however; is that some theories amount to other theories but with an extra postulate or two (e.g. many worlds vs. Copenhagen). So they are strictly more complex. If it doesn’t explain more than the simpler theory the extra complexity isn’t justified.
A lot of the progression of science over the last few centuries has been toward unifying diverse theories under less complex, general frameworks. Special relativity helped unify theories about the electric and magnetic forces, which were then unified with the weak nuclear force and eventually the strong nuclear force. A lot of that work has helped explain the composition of the periodic table and the underlying mechanisms to chemistry. In other words, where there used to be many separate theories, there are now only two theories that explain almost every phenomenon in the observable universe. Those two theories are based on surprisingly few and surprisingly simple postulates.
Over the 20th century, the trend was towards reducing postulates and explaining more, so it was pretty clear that Occam’s razor was being followed. Since then, we’ve run into a bit of an impasse with GR and QFT not nicely unifying and discoveries like dark energy and dark matter.