I don’t see how I can be described as trying to halt the conversation.
Allow me to disclaim that you usually don’t. But that particular referenced post did, and taw tried it even more blatantly—to label further conversation as suspect, ill-advised, and evidence of morally nonvirtuous (giving in to pride and the temptation to show off) “inside viewing”. I was frustrated with this at the time, but had too many higher-priority things to say before I could get around to describing exactly what frustrated me about it.
It’s also not clear to me how you think someone should be allowed to proceed from the point where you say “My abstractions are closer to the surface than yours, so my reference class is better”, or if you think you just win outright at that point. I tend to think that it’s still a pretty good idea to list out the underlying events being used as alleged evidence, stripped of labels and presented as naked facts, and see how much they seem to tell us about the future event at hand, once the covering labels are gone. I think that under these circumstances the force of implication from agriculture to self-improving AI tends to sound pretty weak.
predicting that your evidence will probably be of low quality if it takes a certain form.
Robin seems to think that some of your evidence is a causal analysis of mechanisms based on poorly-grounded abstractions. Given that it’s not logically rude for him to think that your abstractions are poorly grounded, it’s not logically rude for him to predict that they will probably offer poor evidence, and so to predict that they will probably not change his beliefs significantly.
I’m not commenting here on whose predictions are higher-quality. I just don’t think that Robin was being logically rude. If anything, he was helpfully reporting which arguments are mostly likely to sway him. Furthermore, he seems to welcome your trying to persuade him to give other arguments more weight. He probably expects that you won’t succeed, but, so long as he welcomes the attempt, I don’t think that he can be accused of trying to halt the conversation.
Thanks to the OB/LW split, it’s pretty awkward to try to find all the posts in sequence. I thinkTotal Nano Domination is the first one*, and Total Tech Wars was Robin’s reply. They went back and forth after that for a few days (you can follow along in the archives), and then restored the congenial atmosphere by jointly advocating cryonics. In fall 2009 they got into it again in a comment thread on OB.
Don’t neglect the surrounding context. The underlying disagreements have been echoing about all over the place in the form of “Contrarians boo vs Correct Contrarians yay!” and “here is a stupid view that can be classed as an inside view therefore inside view sucks!” vs “high status makes you stupid” and “let’s play reference class tennis”.
There are many sorts of arguments that tend to be weak, and weak-tending arguments deserved to be treated warily, especially if their weakness tends not to be noticed. But pointing that out is not the same as trying to end a conversation.
It seems to me the way to proceed is to talk frankly about various possible abstractions, including their reliability, ambiguity, and track records of use. You favor the abstractions “intelligence” and “self-improving”—be clear about what sort of detail those summaries neglect, why that neglect seems to you reasonable in this case, and look at the track record of others trying to use those abstractions. Consider other abstractions one might use instead.
I’ve got no problem with it phrased that way. To be clear, the part that struck me as unfair was this:
Excess inside viewing usually continues even after folks are warned that outside viewing works better; after all, inside viewing better show offs inside knowledge and abilities. People usually justify this via reasons why the current case is exceptional. (Remember how all the old rules didn’t apply to the new dotcom economy?) So expect to hear excuses why the next singularity is also an exception where outside view estimates are misleading. Let’s keep an open mind, but a wary open mind.
Another example that made me die a little inside when I read it was this:
As Brin notes, many would-be broadcasters come from an academic area where for decades the standard assumption has been that aliens are peaceful zero-population-growth no-nuke greens, since we all know that any other sort quickly destroy themselves. This seems to me an instructive example of how badly a supposed “deep theory” inside-view of the future can fail, relative to closest-related-track-record outside-view.
The inside-view tells me that is an idiotic assumption to make.
Agreed that this is the conflict of two inside views, not an inside view versus an outside view. You could as easily argue that most stars don’t seem to have been eaten, therefore, the outside view suggests that any aliens within radio range are environmentalists. And certainly Robin is judging one view right and the other wrong using an inside view, not an outside view.
I simply don’t see the justification for claiming the power and glory of the Outside View at all in cases like this, let alone claiming that there exists a unique obvious reference class and you have it.
It seems to me the obvious outside view of future contact is previous examples of contact. Yes uneaten stars is also an outside stat, which does (weakly) suggest aliens don’t eat stars. I certainly don’t mean to imply there is always a unique inside view.
Why isn’t the obvious outside view to draw a line showing the increased peacefulness of contacts with the increasing technological development of the parties involved, and extrapolate to super-peaceful aliens? Isn’t this more or less exactly why you argue that AIs will inevitably trade with us? Why extrapolate for AIs but not for aliens?
To be clear on this, I don’t simply distrust the advice of an obvious outside view, I think that in cases like these, people perform a selective search for a reference class that supports a foregone conclusion (and then cry “Outside View!”). This foregone conclusion is based on inside viewing in the best case; in the worst case it is based entirely on motivated cognition or wishful thinking. Thus, to cry “Outside View!” is just to conceal the potentially very flawed thinking that went into the choice of reference class.
Yes uneaten stars is also an outside stat, which does (weakly) suggest aliens don’t eat stars.
Weakly? What are your conditional probabilities that we would observe stars being eaten, given that there exist star-eating aliens (within range of our attempts at communication), and given that such aliens do not exist? Or, if you prefer, what is your likelyhood ratio?
This is an excellently put objection—putting it this way makes it clear just how strong the objection is. The likelihood ratio to me sounds like it should be more or less T/F, where for the sake of conservatism T might equal .99 and F might equal .01. If we knew for a fact that there were aliens in our radio range, wouldn’t this item of evidence wash out any priors we had about them eating stars? We don’t see the stars being eaten!
Forbidding “reliance” on pain of loss of prestige isn’t all that much better than forbidding “exploration” on pain of loss of prestige. People are allowed to talk about my arguments but of course not take them seriously? Whereas it’s perfectly okay to rely on your “outside view” estimate? I don’t think the quoted paragraph is one I can let stand no matter how you reframe it...
I simply don’t see the justification for claiming the power and glory of the Outside View at all in cases like this, let alone claiming that there exists a unique obvious reference class and you have it.
It could even be somewhat worse. Forbidden things seem to be higher status ‘bad’ than things that can be casually dismissed.
Forbidding “reliance” on pain of loss of prestige isn’t all that much better than forbidding “exploration” on pain of loss of prestige.
The “on pain of loss of prestige” was implicit, if it was there at all. All that was explicit was that Robin considered your evidence to be of lower quality than you thought. Insofar as there was an implicit threat to lower status, such a threat would be implicit in any assertion that your evidence is low-quality. You seem to be saying that it is logically rude for Robin to say that he has considered your evidence and to explain why he found it wanting.
Allow me to disclaim that you usually don’t. But that particular referenced post did, and taw tried it even more blatantly—to label further conversation as suspect, ill-advised, and evidence of morally nonvirtuous (giving in to pride and the temptation to show off) “inside viewing”. I was frustrated with this at the time, but had too many higher-priority things to say before I could get around to describing exactly what frustrated me about it.
It’s also not clear to me how you think someone should be allowed to proceed from the point where you say “My abstractions are closer to the surface than yours, so my reference class is better”, or if you think you just win outright at that point. I tend to think that it’s still a pretty good idea to list out the underlying events being used as alleged evidence, stripped of labels and presented as naked facts, and see how much they seem to tell us about the future event at hand, once the covering labels are gone. I think that under these circumstances the force of implication from agriculture to self-improving AI tends to sound pretty weak.
I think that we should distinguish
trying to halt the conversation, from
predicting that your evidence will probably be of low quality if it takes a certain form.
Robin seems to think that some of your evidence is a causal analysis of mechanisms based on poorly-grounded abstractions. Given that it’s not logically rude for him to think that your abstractions are poorly grounded, it’s not logically rude for him to predict that they will probably offer poor evidence, and so to predict that they will probably not change his beliefs significantly.
I’m not commenting here on whose predictions are higher-quality. I just don’t think that Robin was being logically rude. If anything, he was helpfully reporting which arguments are mostly likely to sway him. Furthermore, he seems to welcome your trying to persuade him to give other arguments more weight. He probably expects that you won’t succeed, but, so long as he welcomes the attempt, I don’t think that he can be accused of trying to halt the conversation.
Can someone please link to the posts in question for the latecomers?
http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_Debate
Thanks to the OB/LW split, it’s pretty awkward to try to find all the posts in sequence. I think Total Nano Domination is the first one*, and Total Tech Wars was Robin’s reply. They went back and forth after that for a few days (you can follow along in the archives), and then restored the congenial atmosphere by jointly advocating cryonics. In fall 2009 they got into it again in a comment thread on OB.
* maybe it was prompted by Abstract/Distant Future Bias.
Don’t neglect the surrounding context. The underlying disagreements have been echoing about all over the place in the form of “Contrarians boo vs Correct Contrarians yay!” and “here is a stupid view that can be classed as an inside view therefore inside view sucks!” vs “high status makes you stupid” and “let’s play reference class tennis”.
Good point. Hard to track down the links though.
There are many sorts of arguments that tend to be weak, and weak-tending arguments deserved to be treated warily, especially if their weakness tends not to be noticed. But pointing that out is not the same as trying to end a conversation.
It seems to me the way to proceed is to talk frankly about various possible abstractions, including their reliability, ambiguity, and track records of use. You favor the abstractions “intelligence” and “self-improving”—be clear about what sort of detail those summaries neglect, why that neglect seems to you reasonable in this case, and look at the track record of others trying to use those abstractions. Consider other abstractions one might use instead.
I’ve got no problem with it phrased that way. To be clear, the part that struck me as unfair was this:
Another example that made me die a little inside when I read it was this:
The inside-view tells me that is an idiotic assumption to make.
Agreed that this is the conflict of two inside views, not an inside view versus an outside view. You could as easily argue that most stars don’t seem to have been eaten, therefore, the outside view suggests that any aliens within radio range are environmentalists. And certainly Robin is judging one view right and the other wrong using an inside view, not an outside view.
I simply don’t see the justification for claiming the power and glory of the Outside View at all in cases like this, let alone claiming that there exists a unique obvious reference class and you have it.
It seems to me the obvious outside view of future contact is previous examples of contact. Yes uneaten stars is also an outside stat, which does (weakly) suggest aliens don’t eat stars. I certainly don’t mean to imply there is always a unique inside view.
Why isn’t the obvious outside view to draw a line showing the increased peacefulness of contacts with the increasing technological development of the parties involved, and extrapolate to super-peaceful aliens? Isn’t this more or less exactly why you argue that AIs will inevitably trade with us? Why extrapolate for AIs but not for aliens?
To be clear on this, I don’t simply distrust the advice of an obvious outside view, I think that in cases like these, people perform a selective search for a reference class that supports a foregone conclusion (and then cry “Outside View!”). This foregone conclusion is based on inside viewing in the best case; in the worst case it is based entirely on motivated cognition or wishful thinking. Thus, to cry “Outside View!” is just to conceal the potentially very flawed thinking that went into the choice of reference class.
Weakly? What are your conditional probabilities that we would observe stars being eaten, given that there exist star-eating aliens (within range of our attempts at communication), and given that such aliens do not exist? Or, if you prefer, what is your likelyhood ratio?
This is an excellently put objection—putting it this way makes it clear just how strong the objection is. The likelihood ratio to me sounds like it should be more or less T/F, where for the sake of conservatism T might equal .99 and F might equal .01. If we knew for a fact that there were aliens in our radio range, wouldn’t this item of evidence wash out any priors we had about them eating stars? We don’t see the stars being eaten!
You think that it is idiotic to believe that many (not all) “broadcasters” expect that any aliens advanced enough to harm us will be peaceful?
No. The idiotic assumption is that which is described as the ‘standard’ assumption. No indirection.
To clarify, I meant an excess of reliance on the view, not of exploration of the view.
Forbidding “reliance” on pain of loss of prestige isn’t all that much better than forbidding “exploration” on pain of loss of prestige. People are allowed to talk about my arguments but of course not take them seriously? Whereas it’s perfectly okay to rely on your “outside view” estimate? I don’t think the quoted paragraph is one I can let stand no matter how you reframe it...
It could even be somewhat worse. Forbidden things seem to be higher status ‘bad’ than things that can be casually dismissed.
The “on pain of loss of prestige” was implicit, if it was there at all. All that was explicit was that Robin considered your evidence to be of lower quality than you thought. Insofar as there was an implicit threat to lower status, such a threat would be implicit in any assertion that your evidence is low-quality. You seem to be saying that it is logically rude for Robin to say that he has considered your evidence and to explain why he found it wanting.