You should note that every problem you list is a special case. Obviously, there are ways of cheating at Newcomb’s problem if you’re aware of salient details beforehand. You could simply allow a piece of plutonium to decay, and do whatever the resulting Geiger counter noise tells you to. That does not, however, support your thesis that Newcomb’s problem is a totally artificial problem with no logical intrusions into reality.
As a real-world example, imagine an off-the-shelf stock market optimizing AI. Not sapient, to make things simpler, but smart. When any given copy begins running, there are already hundreds or thousands of near-identical copies running elsewhere in the market. If it fails to predict their actions from its own, it will do objectively worse than it might otherwise do.
i don’t see how your example is apt or salient. My thesis is that Newcomb-like problems are the wrong place to be testing decision theories because they do not represent realistic or relevant problems. We should focus on formalizing and implementing decision theories and throw real-world problems at them rather than testing them on arcane logic puzzles.
Well… no, actually. A good decision theory ought to be universal. It ought to be correct, and it ought to work. Newcomb’s problem is important, not because it’s ever likely to happen, but because it shows a case in which the normal, commonly accepted approach to decision theory (CDT) failed miserably. This ‘arcane logic puzzle’ is illustrative of a deeper underlying flaw in the model, which needs to be addressed. It’s also a flaw that’d be much harder to pick out by throwing ‘real world’ problems at it over and over again.
Seems unlikely to work out to me. Humans evolved intelligence without Newcomb-like problems. As the only example of intelligence that we know of, it’s clearly possible to develop intelligence without Newcomb-like problems. Furthermore, the general theory seems to be that AIs will start dumber than humans and iteratively improve until they’re smarter. Given that, why are we so interested in problems like these (which humans don’t universally agree about the answers to)?
I’d rather AIs be able to help us with problems like “what should we do about the economy?” or even “what should I have for dinner?” instead of worrying about what we should do in the face of something godlike.
Additionally, human minds aren’t universal (assuming that universal means that they give the “right” solutions to all problems), so why should we expect AIs to be? We certainly shouldn’t expect this if we plan on iteratively improving our AIs.
It might be nice to be able to see the voting history (not the voters’ names, but the number of up and down votes) on a comment. I can’t tell if my comments are controversial or just down-voted by two people. Perhaps even just the number of votes would be sufficient (e.g. −2/100 vs. −2/2).
If it helps: it’s a fairly common belief in this community that a general-purpose optimization tool is both far superior to, and more interesting to talk about, than a variety of special-purpose tools.
Of course, that doesn’t mean you have to be interested in general-purpose optimization tools; if you’re more interested in decision theory for dinner-menu or economic planners, by all means post about that if you have something to say.
But I suspect there are relatively few communities in which “why are you all so interested in such a stupid and uninteresting topic?” will get you much community approval, and this isn’t one of them.
I’m interested in general purpose optimizers, but I bet that they will be evolved from AIs that were more special purpose to begin with. E.g., IBM Watson moving from Jeopardy!-playing machine to medical diagnostic assistant with a lot of the upfront work being on rapid NLP for the J! “questions”.
Also, there’s no reason that I’ve seen here to believe that Newcomb-like problems give insights into how to develop to decision theories that allow us to solve real-world problems. It seems like arguing about corner cases. Can anyone establish a practical problem that TDT fails to solve because it fails to solve these other problems?
Beyond this, my belief is that without formalization and programming of these decision frameworks, we learn very little. Asking what does xDT do in some abstract situation, so far, seems very handy-wavy. Furthermore, it seems to me that the community is drawn to these problems because they are deceptively easy to state and talk about online, but minds are inherently complex, opaque, and hard to reason about.
I’m having a hard time understanding how correctly solving Newcomb-like problems is expected to advance the field of general optimizers. It seems out of proportion to the problems at hand to expect a decision theory to solve problems of this level of sophistication when the current theories don’t seem to obviously “solve” questions like “what should we have for lunch?”. I get the feeling that supporters of research on these theories assume that, of course, xDT can solve the easy problems so let’s do the hard ones. And, I think evidence for this assumption is very lacking.
Again, if you are interested in more discussion about automated optimization on the level of “what should we have for lunch?” I encourage you to post about it; I suspect a lot of other people are interested as well.
You should note that every problem you list is a special case. Obviously, there are ways of cheating at Newcomb’s problem if you’re aware of salient details beforehand. You could simply allow a piece of plutonium to decay, and do whatever the resulting Geiger counter noise tells you to. That does not, however, support your thesis that Newcomb’s problem is a totally artificial problem with no logical intrusions into reality.
As a real-world example, imagine an off-the-shelf stock market optimizing AI. Not sapient, to make things simpler, but smart. When any given copy begins running, there are already hundreds or thousands of near-identical copies running elsewhere in the market. If it fails to predict their actions from its own, it will do objectively worse than it might otherwise do.
i don’t see how your example is apt or salient. My thesis is that Newcomb-like problems are the wrong place to be testing decision theories because they do not represent realistic or relevant problems. We should focus on formalizing and implementing decision theories and throw real-world problems at them rather than testing them on arcane logic puzzles.
Well… no, actually. A good decision theory ought to be universal. It ought to be correct, and it ought to work. Newcomb’s problem is important, not because it’s ever likely to happen, but because it shows a case in which the normal, commonly accepted approach to decision theory (CDT) failed miserably. This ‘arcane logic puzzle’ is illustrative of a deeper underlying flaw in the model, which needs to be addressed. It’s also a flaw that’d be much harder to pick out by throwing ‘real world’ problems at it over and over again.
Seems unlikely to work out to me. Humans evolved intelligence without Newcomb-like problems. As the only example of intelligence that we know of, it’s clearly possible to develop intelligence without Newcomb-like problems. Furthermore, the general theory seems to be that AIs will start dumber than humans and iteratively improve until they’re smarter. Given that, why are we so interested in problems like these (which humans don’t universally agree about the answers to)?
I’d rather AIs be able to help us with problems like “what should we do about the economy?” or even “what should I have for dinner?” instead of worrying about what we should do in the face of something godlike.
Additionally, human minds aren’t universal (assuming that universal means that they give the “right” solutions to all problems), so why should we expect AIs to be? We certainly shouldn’t expect this if we plan on iteratively improving our AIs.
Harsh crowd.
It might be nice to be able to see the voting history (not the voters’ names, but the number of up and down votes) on a comment. I can’t tell if my comments are controversial or just down-voted by two people. Perhaps even just the number of votes would be sufficient (e.g. −2/100 vs. −2/2).
If it helps: it’s a fairly common belief in this community that a general-purpose optimization tool is both far superior to, and more interesting to talk about, than a variety of special-purpose tools.
Of course, that doesn’t mean you have to be interested in general-purpose optimization tools; if you’re more interested in decision theory for dinner-menu or economic planners, by all means post about that if you have something to say.
But I suspect there are relatively few communities in which “why are you all so interested in such a stupid and uninteresting topic?” will get you much community approval, and this isn’t one of them.
I’m interested in general purpose optimizers, but I bet that they will be evolved from AIs that were more special purpose to begin with. E.g., IBM Watson moving from Jeopardy!-playing machine to medical diagnostic assistant with a lot of the upfront work being on rapid NLP for the J! “questions”.
Also, there’s no reason that I’ve seen here to believe that Newcomb-like problems give insights into how to develop to decision theories that allow us to solve real-world problems. It seems like arguing about corner cases. Can anyone establish a practical problem that TDT fails to solve because it fails to solve these other problems?
Beyond this, my belief is that without formalization and programming of these decision frameworks, we learn very little. Asking what does xDT do in some abstract situation, so far, seems very handy-wavy. Furthermore, it seems to me that the community is drawn to these problems because they are deceptively easy to state and talk about online, but minds are inherently complex, opaque, and hard to reason about.
I’m having a hard time understanding how correctly solving Newcomb-like problems is expected to advance the field of general optimizers. It seems out of proportion to the problems at hand to expect a decision theory to solve problems of this level of sophistication when the current theories don’t seem to obviously “solve” questions like “what should we have for lunch?”. I get the feeling that supporters of research on these theories assume that, of course, xDT can solve the easy problems so let’s do the hard ones. And, I think evidence for this assumption is very lacking.
That’s fair.
Again, if you are interested in more discussion about automated optimization on the level of “what should we have for lunch?” I encourage you to post about it; I suspect a lot of other people are interested as well.
Yeah, I might, but here I was just surprised by the down-voting for contrary opinion. It seems like the thing we ought to foster not hide.
As I tried to express in the first place, I suspect what elicited the disapproval was not the contrary opinion, but the rudeness.
Sorry. It didn’t seem rude to me. I’m just frustrated with where I see folks spending their time.
My apologies to anyone who was offended.