You have my apologies if you thought I was attacking or pigeonholing your argument. While I lack the technical expertise to critique the technical portion of your argument, I think it could benefit from a more explicit avoidance of the fallacy mentioned above. I thought the article was very interesting and I will certainly come back to it if I ever get to the point where I can understand your distinctions between swarm intelligence and CFAI. I understand you have been facing attacks for your position in this article, but that is not my intention. Your meticulous arguments are certainly impressive, but you do them a disservice by dismissing well intentioned critique, especially as it applies to the structure of your argument and not the substance.
Einstein made predictions about what the universe would look like if there were a maximum speed. Your prediction seems to be that well built ai will not misunderstand its goals (please assume that I read your article thoroughly and that any misunderstandings are benign). What does the universe look like if this is false?
I probably fall under category a in your disjunction. Is it truly pointless to help me overcome my misunderstanding? From the large volume of comments, it seems likely that this misunderstanding is partially caused by a gap between what you are trying to say, and what was said. Please help me bridge this gap instead of denying its existence or calling such an exercise pointless.
Let me see if I can deal with the “no true scotsman” line of attack.
The way that that fallacy might apply to what I wrote would be, I think, something like this:
MIRI says that a superintelligence might unpack a goal statement like “maximize human happiness” by perpetrating a Maverick Nanny attack on humankind, but Loosemore says that no TRUE superintelligence would do such a thing, because it would be superintelligent enough to realize that this was a ‘mistake’ (in some sense).
This would be a No True Scotsman fallacy, because the term “superintelligence” has been, in effect, redefined by me to mean “something smart enough not to do that”.
Now, my take on the NTS idea is that it cannot be used if there are substantive grounds for saying that there are two categories involved, rather than a real category and a fake category that is (for some unexplained reason) exceptional.
Example: Person A claims that a sea-slug caused the swimmer’s leg to be bitten off, but Person B argues that no “true” sea-slug would have done this. In this example, Person B is not using a No True Scotsman argument, because there are darned good reasons for supposing that sea-slugs cannot bite off the legs of swimmers.
So it all comes down to whether someone accused of NTS is inventing a ficticious category distinction (“true” versus “non-true” Scotsman) solely for the purposes of supporting their argument.
In my case, what I have argued is right up there with the sea-slug argument. What I have said, in effect, is that if we sit down and carefully think about the type of “superintelligence” that MIRI et al. put into their scenarios, and if we explore all the implications of what that hypothetical AI would have to be like, we quickly discover some glaring inconsistencies in their scenarios. The sea-slug, in effect, is supposed to have bitten through bone with a mouth made of mucous. And the sea-slug is so small it could not wrap itself around the swimmer’s leg. Thinking through the whole sea-slug scenario leads us into a mass of evidence indicating that the proposed scenario is nuts. Similarly, thinking through the implications of an AI that is so completely unable to handle context, that it can live with Grade A contradictions at the heart of its reasoning, leads us to a mass of unbelievable inconsistencies in the ‘intelligence’ of this supposed superintelligence.
So, where the discussion needs to be, in respect of the paper, is in the exact details of why the proposed SI might not be a meaningful hypothetical. It all comes down to a meticulous dissection of the mechanisms involved.
To conclude: sorry if I seemed to come down a little heavy on you in my first response. I wasn’t upset, it was just that the NTS critique had occurred before. In some of those previous cases the NTS attack was accompanied by language that strongly implied that I had not just committed an NTS fallacy, but that I was such an idiot that my idiocy was grounds for recommending to all not to even read the paper. ;-)
″… thinking through the implications of an AI that is so completely unable to handle context, that it can live with Grade A contradictions at the heart of its reasoning, leads us to a mass of unbelievable inconsistencies in the ‘intelligence’ of this supposed superintelligence.”
This is all at once concise, understandable, and reassuring. Thank you. I still wonder if we are accurately broadening the scope of defined “intelligence” out too far, but my wonder comes from gaps in my specific knowledge and not from gaps in your argument.
Einstein made predictions about what the universe would look like if there were a maximum speed. Your prediction seems to be that well built ai will not misunderstand its goals
Or a (likely to be built) AI won’t even have the ability to compartmentalise it’s goals from its knowledge base.
It’s not No True Scotsnan to say that no competent researcher would do it that way.
Thank you for responding and attempting to help me clear up my misunderstanding. I will need to do another deep reading, but a quick skim of the article from this point of view “clicks” a lot better for me.
You have my apologies if you thought I was attacking or pigeonholing your argument. While I lack the technical expertise to critique the technical portion of your argument, I think it could benefit from a more explicit avoidance of the fallacy mentioned above. I thought the article was very interesting and I will certainly come back to it if I ever get to the point where I can understand your distinctions between swarm intelligence and CFAI. I understand you have been facing attacks for your position in this article, but that is not my intention. Your meticulous arguments are certainly impressive, but you do them a disservice by dismissing well intentioned critique, especially as it applies to the structure of your argument and not the substance.
Einstein made predictions about what the universe would look like if there were a maximum speed. Your prediction seems to be that well built ai will not misunderstand its goals (please assume that I read your article thoroughly and that any misunderstandings are benign). What does the universe look like if this is false?
I probably fall under category a in your disjunction. Is it truly pointless to help me overcome my misunderstanding? From the large volume of comments, it seems likely that this misunderstanding is partially caused by a gap between what you are trying to say, and what was said. Please help me bridge this gap instead of denying its existence or calling such an exercise pointless.
Hey, no problem. I was really just raising an issue with certain types of critique, which involve supposed fallacies that actually don’t apply.
I am actually pressed for time right now, so I have to break off and come back to this when I can. Just wanted to clarify if I could.
Feel free to disengage; TheAncientGeek helped me shift my paradigm correctly.
Let me see if I can deal with the “no true scotsman” line of attack.
The way that that fallacy might apply to what I wrote would be, I think, something like this:
MIRI says that a superintelligence might unpack a goal statement like “maximize human happiness” by perpetrating a Maverick Nanny attack on humankind, but Loosemore says that no TRUE superintelligence would do such a thing, because it would be superintelligent enough to realize that this was a ‘mistake’ (in some sense).
This would be a No True Scotsman fallacy, because the term “superintelligence” has been, in effect, redefined by me to mean “something smart enough not to do that”.
Now, my take on the NTS idea is that it cannot be used if there are substantive grounds for saying that there are two categories involved, rather than a real category and a fake category that is (for some unexplained reason) exceptional.
Example: Person A claims that a sea-slug caused the swimmer’s leg to be bitten off, but Person B argues that no “true” sea-slug would have done this. In this example, Person B is not using a No True Scotsman argument, because there are darned good reasons for supposing that sea-slugs cannot bite off the legs of swimmers.
So it all comes down to whether someone accused of NTS is inventing a ficticious category distinction (“true” versus “non-true” Scotsman) solely for the purposes of supporting their argument.
In my case, what I have argued is right up there with the sea-slug argument. What I have said, in effect, is that if we sit down and carefully think about the type of “superintelligence” that MIRI et al. put into their scenarios, and if we explore all the implications of what that hypothetical AI would have to be like, we quickly discover some glaring inconsistencies in their scenarios. The sea-slug, in effect, is supposed to have bitten through bone with a mouth made of mucous. And the sea-slug is so small it could not wrap itself around the swimmer’s leg. Thinking through the whole sea-slug scenario leads us into a mass of evidence indicating that the proposed scenario is nuts. Similarly, thinking through the implications of an AI that is so completely unable to handle context, that it can live with Grade A contradictions at the heart of its reasoning, leads us to a mass of unbelievable inconsistencies in the ‘intelligence’ of this supposed superintelligence.
So, where the discussion needs to be, in respect of the paper, is in the exact details of why the proposed SI might not be a meaningful hypothetical. It all comes down to a meticulous dissection of the mechanisms involved.
To conclude: sorry if I seemed to come down a little heavy on you in my first response. I wasn’t upset, it was just that the NTS critique had occurred before. In some of those previous cases the NTS attack was accompanied by language that strongly implied that I had not just committed an NTS fallacy, but that I was such an idiot that my idiocy was grounds for recommending to all not to even read the paper. ;-)
″… thinking through the implications of an AI that is so completely unable to handle context, that it can live with Grade A contradictions at the heart of its reasoning, leads us to a mass of unbelievable inconsistencies in the ‘intelligence’ of this supposed superintelligence.”
This is all at once concise, understandable, and reassuring. Thank you. I still wonder if we are accurately broadening the scope of defined “intelligence” out too far, but my wonder comes from gaps in my specific knowledge and not from gaps in your argument.
Or a (likely to be built) AI won’t even have the ability to compartmentalise it’s goals from its knowledge base.
It’s not No True Scotsnan to say that no competent researcher would do it that way.
Thank you for responding and attempting to help me clear up my misunderstanding. I will need to do another deep reading, but a quick skim of the article from this point of view “clicks” a lot better for me.