Gain-of-lambda-function research: yes, this is among the worser things you could be researching, up there with the Codex code evolution & Adept Transformer agents. There are… uh, not many realistic, beneficial applications for this work. No one really needs a Diplomacy AI, and applications to things like ad auctions are tenuous. (Note the amusing wriggling of FB PR when they talk about “a strategy game which requires building trust, negotiating and cooperating with multiple players”—you left out some relevant verbs there...) And as we’ve seen with biological research, no matter how many times bugs escape research laboratories and literally kill people, the déformation professionnelle will cover it up and justify it. Researchers who scoff at the idea that a website should be able to set a cookie without a bunch of laws regulating it suddenly turn into don’t-tread-on-me anarchocapitalists as soon as it comes to any suggestion that their research maybe shouldn’t be done.
But this is far from the most blatantly harmful research (hey, they haven’t killed anyone yet, so the bar is high), so we shouldn’t be too hard on them or personalize it. Let’s just treat this as a good example for those who think that researchers collectively have any fire alarm and will self-regulate. (No, they won’t, and someone out there is already calculating how many megawatts the Torment Nexus will generate and going “Sweet!” and putting in a proposal for a prototype ‘Suffering Swirlie’ research programme culminating in creating a Torment Nexus by 2029.)
The really dangerous part of the research is that this is the type of game which incentivizes deceptive alignment by default, which is extremely dangerous to do. It ranks amongst one of the worst failure modes in AI Alignment, and this is at the top 5 risky directions to go to.
The really dangerous part of the research is that this is the type of game which incentivizes deceptive alignment by default, which is extremely dangerous to do. It ranks amongst one of the worst failure modes in AI Alignment, and this is at the top 5 risky directions to go to.