Fair enough. I’ll share a few examples of progress, though these won’t be surprising to people who are on every mailing list, read every LW post, or are in the Bay Area and have regular conversations with us.
much progress on the strategic landscape, e.g. differential technological development analyses, which you’ll see in the forthcoming Anna/Luke chapter and in Nick’s forthcoming monograph, and which you’ve already seen in several papers and talks over the past couple years (most of them involving Carl).
progress on decision theory, largely via the decision theory workshop mailing list, in particular on UDT
progress in outlining the sub-problems of singularity research, which I’ve started to write up here.
progress on the value-loading problem, explained here and in a forthcoming paper by Dewey.
progress on the reflectivity problem in the sense of identifying lots of potential solutions that probably won’t work. :)
progress on the preference extraction problem via incorporating the latest from decision neuroscience
Still, I’d say more of our work has been focused on movement-building than on cutting-edge research, because we think the most immediate concern is not on cutting-edge research but on building a larger community of support, funding, and researchers to work on these problems. Three researchers can have more of an impact if they create a platform by which 20 researchers can work on the problem than if they merely do research by themselves.
Is the value-loading or value-learning problem the same thing as the problem of moral uncertainty? If no, what am I missing; if yes, why are the official solution candidates different?
Thanks, this is quite informative, especially your closing paragraph:
Still, I’d say more of our work has been focused on movement-building than on cutting-edge research, because we think the most immediate concern is not on cutting-edge research but on building a larger community of support, funding, and researchers to work on these problems.
This makes sense to me; have you considered incorporating this paragraph into your core mission statement ? Also, what are your thresholds for deciding when to transition from (primarily) community-building to (primarily) doing research ?
Also, you mentioned (in your main post) that the SIAI has quite a few papers in the works, awaiting publication; and apparently there are even a few books waiting for publishers. Would it not be more efficient to post the articles and books in question on Less Wrong, or upload them to Pirate Bay, or something to that extent—at least, while you wait for the meat-space publishers to get their act together ? Sorry if this is a naive question; I know very little about the publishing world.
what are your thresholds for deciding when to transition from (primarily) community-building to (primarily) doing research ?
We’re not precisely sure. It’s also a matter of funding. Researchers who can publish “platform research” for academic outreach, problem space clarification, and community building are less expensive than researchers who can solve decision theory, safe AI architectures, etc.
Would it not be more efficient to post the articles and books in question on Less Wrong, or upload them to Pirate Bay, or something to that extent—at least, while you wait for the meat-space publishers to get their act together ?
Like many academics, we generally do publish early drafts of forthcoming articles long before the final version is written and published. Examples: 1, 2, 3, 4.
An example: The subject matter of the second chapter of this book (the three competing systems of motivation) looks to have some implications for the value extraction problem. This is the kind of information about how our preferences work that I imagine we’ll use to extrapolate our preferences — or that an AI would use to do the extrapolation for us.
Fair enough. I’ll share a few examples of progress, though these won’t be surprising to people who are on every mailing list, read every LW post, or are in the Bay Area and have regular conversations with us.
much progress on the strategic landscape, e.g. differential technological development analyses, which you’ll see in the forthcoming Anna/Luke chapter and in Nick’s forthcoming monograph, and which you’ve already seen in several papers and talks over the past couple years (most of them involving Carl).
progress on decision theory, largely via the decision theory workshop mailing list, in particular on UDT
progress in outlining the sub-problems of singularity research, which I’ve started to write up here.
progress on the value-loading problem, explained here and in a forthcoming paper by Dewey.
progress on the reflectivity problem in the sense of identifying lots of potential solutions that probably won’t work. :)
progress on the preference extraction problem via incorporating the latest from decision neuroscience
Still, I’d say more of our work has been focused on movement-building than on cutting-edge research, because we think the most immediate concern is not on cutting-edge research but on building a larger community of support, funding, and researchers to work on these problems. Three researchers can have more of an impact if they create a platform by which 20 researchers can work on the problem than if they merely do research by themselves.
Thank you, this is exactly the kind of answer I was hoping for.
Is the value-loading or value-learning problem the same thing as the problem of moral uncertainty? If no, what am I missing; if yes, why are the official solution candidates different?
Thanks, this is quite informative, especially your closing paragraph:
This makes sense to me; have you considered incorporating this paragraph into your core mission statement ? Also, what are your thresholds for deciding when to transition from (primarily) community-building to (primarily) doing research ?
Also, you mentioned (in your main post) that the SIAI has quite a few papers in the works, awaiting publication; and apparently there are even a few books waiting for publishers. Would it not be more efficient to post the articles and books in question on Less Wrong, or upload them to Pirate Bay, or something to that extent—at least, while you wait for the meat-space publishers to get their act together ? Sorry if this is a naive question; I know very little about the publishing world.
We’re not precisely sure. It’s also a matter of funding. Researchers who can publish “platform research” for academic outreach, problem space clarification, and community building are less expensive than researchers who can solve decision theory, safe AI architectures, etc.
Like many academics, we generally do publish early drafts of forthcoming articles long before the final version is written and published. Examples: 1, 2, 3, 4.
I’d love to hear more about what areas you’re looking into within decision neuroscience.
For those who are also interested and somehow missed these:
Crash Course in Neuroscience of Motivation
and these two neuroeconomics book reviews.
An example: The subject matter of the second chapter of this book (the three competing systems of motivation) looks to have some implications for the value extraction problem. This is the kind of information about how our preferences work that I imagine we’ll use to extrapolate our preferences — or that an AI would use to do the extrapolation for us.