Why do you think I used the word “non-trivial”? Are you not aware that in technical fields “non-trivial” means “difficult”?
I didn’t state disagreement with you. I stated my disdain for most of LW community which just glosses it out as a detail not worth discussing. edit: or worse yet as inherent part of any ‘AI’.
It’s dangerous because it’s more powerful than other types of AI?
“Powerful” is a bad concept. I wouldn’t expect it to be a better problem solver for things like ‘how to make a better microchip’, but perhaps it could be a better problem solver for ‘how to hack internet’ because it is unethical but can come up with the idea and be ‘motivated’ to do it, while others aren’t. (I do not think that UDT is relevant to the difficult issues there—fortunately)
and why do you think other AI researchers won’t eventually invent the same ideas
The ideas in question (to the extent to which they are developed by SI so far) are trivial. They are also entirely useless for solving problems like how to make a better microchip, or how to drive a car. I do not expect non-SI funded research into automated problem solving to try to work out this kind of stuff, due to it’s uselessness. (note: the implementation of such ideas would be highly non trivial for anything like ‘real world paperclips with the intelligence module not solving the problem of breaking the paperclip counter’).
I didn’t state disagreement with you. I stated my disdain for most of LW community which just glosses it out as a detail not worth discussing. edit: or worse yet as inherent part of any ‘AI’.
“Powerful” is a bad concept. I wouldn’t expect it to be a better problem solver for things like ‘how to make a better microchip’, but perhaps it could be a better problem solver for ‘how to hack internet’ because it is unethical but can come up with the idea and be ‘motivated’ to do it, while others aren’t. (I do not think that UDT is relevant to the difficult issues there—fortunately)
The ideas in question (to the extent to which they are developed by SI so far) are trivial. They are also entirely useless for solving problems like how to make a better microchip, or how to drive a car. I do not expect non-SI funded research into automated problem solving to try to work out this kind of stuff, due to it’s uselessness. (note: the implementation of such ideas would be highly non trivial for anything like ‘real world paperclips with the intelligence module not solving the problem of breaking the paperclip counter’).