How? The person I’m responding to gets the math of probability wrong and uses it to make a confusing claim that “there’s nothing wrong” as though we have no more agency over the development of AI than we do over the chaotic motion of a dice.
It’s foolish to liken the development of AI to a roll of the dice. Given the stakes, we must try to study, prepare for, and guide the development of AI as best we can.
This isn’t hypothetical. We’ve already built a machine that’s more intelligent than any man alive and which brutally optimizes toward a goal that’s incompatible with the good of man kind. We call it, “Global Capitalism”. There isn’t a man alive who knows how to stock the shelves of stores all over the world with #2 pencils that cost only 2 cents each, yet it happens every day because *the system* knows how. The problem is: that system operates with a sociopathic disregard for life (human or otherwise) and has exceeded all limits of sustainability without so much as slowing down. It’s a short-sighted, cruel leviathan and there’s no human at the reigns.
At this point, it’s not about waiting for the dice to settle, it’s about figuring out how to wrangle such a beast and prevent the creation of more.
uses it to make a confusing claim that “there’s nothing wrong” as though we have no more agency over the development of AI than we do over the chaotic motion of a dice.
It’s foolish to liken the development of AI to a roll of the dice. Given the stakes, we must try to study, prepare for, and guide the development of AI as best we can.
I think you’re misinterpreting the original comment. Scott was talking about there being “nothing wrong” with this conception of epistemic uncertainty before the 1 arrives, where each new roll doesn’t tell you anything about when the 1 will come. He isn’t advocating pacifism about AI risk, though. Ironically enough, in his capacity as lead of the Agent Foundations team at MIRI, Scott is arguably one of the least AI-risk-passive people on the planet.
The person I’m responding to gets the math of probability wrong.
No, they are correctly describing a Poisson distribution, which is correct for this situation and compatible with what you say too. AFAICT they’re not saying anything about AI or morality.
How? The person I’m responding to gets the math of probability wrong and uses it to make a confusing claim that “there’s nothing wrong” as though we have no more agency over the development of AI than we do over the chaotic motion of a dice.
It’s foolish to liken the development of AI to a roll of the dice. Given the stakes, we must try to study, prepare for, and guide the development of AI as best we can.
This isn’t hypothetical. We’ve already built a machine that’s more intelligent than any man alive and which brutally optimizes toward a goal that’s incompatible with the good of man kind. We call it, “Global Capitalism”. There isn’t a man alive who knows how to stock the shelves of stores all over the world with #2 pencils that cost only 2 cents each, yet it happens every day because *the system* knows how. The problem is: that system operates with a sociopathic disregard for life (human or otherwise) and has exceeded all limits of sustainability without so much as slowing down. It’s a short-sighted, cruel leviathan and there’s no human at the reigns.
At this point, it’s not about waiting for the dice to settle, it’s about figuring out how to wrangle such a beast and prevent the creation of more.
I think you’re misinterpreting the original comment. Scott was talking about there being “nothing wrong” with this conception of epistemic uncertainty before the 1 arrives, where each new roll doesn’t tell you anything about when the 1 will come. He isn’t advocating pacifism about AI risk, though. Ironically enough, in his capacity as lead of the Agent Foundations team at MIRI, Scott is arguably one of the least AI-risk-passive people on the planet.
No, they are correctly describing a Poisson distribution, which is correct for this situation and compatible with what you say too. AFAICT they’re not saying anything about AI or morality.