Look, we already have superhuman intelligences. We call them corporations and while they put out a lot of good stuff, we’re not wild about the effects they have on the world. We tell corporations ‘hey do what human shareholders want’ and the monkey’s paw curls and this is what we get.
Anyway yeah that but a thousand times faster, that’s what I’m nervous about.
(b) Look, we already have superhuman intelligences. We call them governments and while they put out a lot of good stuff, we’re not wild about the effects they have on the world. We tell governments ‘hey do what human voters want’ and the monkey’s paw curls and this is what we get.
Anyway yeah that but a thousand times faster, that’s what I’m nervous about.
I think this would benefit from being turned into a longer-form argument. Here’s a quote you could use in the preface:
“Sure, cried the tenant men,but it’s our land…We were born on it, and we got killed on it, died on it. Even if it’s no good, it’s still ours….That’s what makes ownership, not a paper with numbers on it.”
“We’re sorry. It’s not us. It’s the monster. The bank isn’t like a man.”
”Yes, but the bank is only made of men.”
″No, you’re wrong there—quite wrong there. The bank is something else than men. It happens that every man in a bank hates what the bank does, and yet the bank does it. The bank is something more than men, I tell you. It’s the monster. Men made it, but they can’t control it.” ― John Steinbeck, The Grapes of Wrath
I had no idea that this angle existed or was feasible. I think these are best for ML researchers, since policymakers and techxecutives tend to think of institutions as flawed due to the vicious self-interest of the people who inhabit them (the problem is particularly acute in management). In which they might respond by saying that AI should not split into subroutines that compete with eachother, or something like that. One way or another, they’ll see it as a human problem and not a machine problem.
“We only have two cases of generally intelligent systems: individual humans and organizations made of humans. When a very large and competent organization is sent to solve a task, such as a corporation, it will often do so by cutting corners in undetectable ways, even when total synergy is achieved and each individual agrees that it would be best not to cut corners. So not only do we know that individual humans feel inclined to cheat and cut corners, but we also know that large optimal groups will automatically cheat and cut corners. Undetectable cheating and misrepresentation is fundamental to learning processes in general, not just a base human instinct”
I’m not an ML researcher and haven’t been acquainted with very many, so I don’t know if this will work.
(a)
Look, we already have superhuman intelligences. We call them corporations and while they put out a lot of good stuff, we’re not wild about the effects they have on the world. We tell corporations ‘hey do what human shareholders want’ and the monkey’s paw curls and this is what we get.
Anyway yeah that but a thousand times faster, that’s what I’m nervous about.
(b)
Look, we already have superhuman intelligences. We call them governments and while they put out a lot of good stuff, we’re not wild about the effects they have on the world. We tell governments ‘hey do what human voters want’ and the monkey’s paw curls and this is what we get.
Anyway yeah that but a thousand times faster, that’s what I’m nervous about.
I think this would benefit from being turned into a longer-form argument. Here’s a quote you could use in the preface:
I had no idea that this angle existed or was feasible. I think these are best for ML researchers, since policymakers and techxecutives tend to think of institutions as flawed due to the vicious self-interest of the people who inhabit them (the problem is particularly acute in management). In which they might respond by saying that AI should not split into subroutines that compete with eachother, or something like that. One way or another, they’ll see it as a human problem and not a machine problem.
“We only have two cases of generally intelligent systems: individual humans and organizations made of humans. When a very large and competent organization is sent to solve a task, such as a corporation, it will often do so by cutting corners in undetectable ways, even when total synergy is achieved and each individual agrees that it would be best not to cut corners. So not only do we know that individual humans feel inclined to cheat and cut corners, but we also know that large optimal groups will automatically cheat and cut corners. Undetectable cheating and misrepresentation is fundamental to learning processes in general, not just a base human instinct”
I’m not an ML researcher and haven’t been acquainted with very many, so I don’t know if this will work.
“Undetectable cheating, and misrepresentation, is fundamental to learning processes in general; it’s not just a base human instinct”