Thanks for being open to suggestions :) Here’s one: you could award half the prize pool to compelling arguments against AI safety. That addresses one of John’s points.
For example, stuff like “We need to focus on problems AI is already causing right now, like algorithmic fairness” would not win a prize, but “There’s some chance we’ll be better able to think about these issues much better in the future once we have more capable models that can aid our thinking, making effort right now less valuable” might.
That idea seems reasonable at first glance, but upon reflection, I think it’s a really bad idea. It’s one thing to run a red-teaming competition, it’s another to spend money building rhetorically optimised tools for the other side. If we do that, then maybe there was no point running the competition in the first place as it might all cancel out.
This makes sense if you assume things are symmetric. Hopefully there’s enough interest in truth and valid reasoning that if the “AI is dangerous” conclusion is correct, it’ll have better arguments on its side.
Thanks for the idea, Jacob. Not speaking on behalf of the group here—but my first thought is that enforcing symmetry on discussion probably isn’t a condition for good epistemics, especially since the distribution of this community’s opinions is skewed. I think I’d be more worried if particular arguments that were misleading went unchallenged, but we’ll be vetting submissions as they come in, and I’d also encourage anyone who has concerns with a given submission to talk with the author and/or us. My second thought is that we’re planning a number of practical outreach projects that will make use of the arguments generated here—we’re not trying to host an intra-community debate about the legitimacy of AI risk—so we’d ideally have the prize structure reflect the outreach value for which arguments are responsible.
I’m potentially up to opening the contest to arguments for or against AI risk, and allowing the distribution of responses to reflect the distribution of the opinions of the community. Will discuss with the rest of the group.
It seems better to award some fraction of the prize pool to refutations of the posted arguments. IMO the point isn’t to be “fair to both sides”, it’s to produce truth.
Wait, the goal here, at least, isn’t to produce truth, it is to disseminate it. Counter-arguments are great, but this isn’t about debating the question, it’s about communicating a conclusion well.
Thanks for being open to suggestions :) Here’s one: you could award half the prize pool to compelling arguments against AI safety. That addresses one of John’s points.
For example, stuff like “We need to focus on problems AI is already causing right now, like algorithmic fairness” would not win a prize, but “There’s some chance we’ll be better able to think about these issues much better in the future once we have more capable models that can aid our thinking, making effort right now less valuable” might.
That idea seems reasonable at first glance, but upon reflection, I think it’s a really bad idea. It’s one thing to run a red-teaming competition, it’s another to spend money building rhetorically optimised tools for the other side. If we do that, then maybe there was no point running the competition in the first place as it might all cancel out.
This makes sense if you assume things are symmetric. Hopefully there’s enough interest in truth and valid reasoning that if the “AI is dangerous” conclusion is correct, it’ll have better arguments on its side.
Thanks for the idea, Jacob. Not speaking on behalf of the group here—but my first thought is that enforcing symmetry on discussion probably isn’t a condition for good epistemics, especially since the distribution of this community’s opinions is skewed. I think I’d be more worried if particular arguments that were misleading went unchallenged, but we’ll be vetting submissions as they come in, and I’d also encourage anyone who has concerns with a given submission to talk with the author and/or us. My second thought is that we’re planning a number of practical outreach projects that will make use of the arguments generated here—we’re not trying to host an intra-community debate about the legitimacy of AI risk—so we’d ideally have the prize structure reflect the outreach value for which arguments are responsible.
I’m potentially up to opening the contest to arguments for or against AI risk, and allowing the distribution of responses to reflect the distribution of the opinions of the community. Will discuss with the rest of the group.
It seems better to award some fraction of the prize pool to refutations of the posted arguments. IMO the point isn’t to be “fair to both sides”, it’s to produce truth.
Wait, the goal here, at least, isn’t to produce truth, it is to disseminate it. Counter-arguments are great, but this isn’t about debating the question, it’s about communicating a conclusion well.
This is PR, not internal epistemics, if I’m understanding the situation correctly.