Even if we succeeded at (1), it would be hard to know that we’d succeeded without progress on (4). If we’re using one or more proxies, we don’t have a way to talk about how accurate they are without (4) - we can’t evaluate how closely the proxies match the thing they’re supposed to proxy, without grounding that thing.
For (2), if we want to talk about “low-impact” or anything like it, then we need a grounding of what kind of impact we care about—and that question falls under (4). If we forget about some kind of impact that humans actually do care about, then we’re in trouble.
Thanks for clarifying! I haven’t brought this up on your research agenda because I prefer to have the discussion during an upcoming sequence of mine, and it felt unfair to comment on your agenda, “I disagree but I won’t elaborate right now”.
It’s not obvious to me why this ought to be the case. Could you elaborate?
Even if we succeeded at (1), it would be hard to know that we’d succeeded without progress on (4). If we’re using one or more proxies, we don’t have a way to talk about how accurate they are without (4) - we can’t evaluate how closely the proxies match the thing they’re supposed to proxy, without grounding that thing.
For (2), if we want to talk about “low-impact” or anything like it, then we need a grounding of what kind of impact we care about—and that question falls under (4). If we forget about some kind of impact that humans actually do care about, then we’re in trouble.
Yep ^_^ I make those points in the research agenda (section 3).
Exactly. You explained it better than I could :)
I also am curious why this should be so.
I also continue to disagree with Stuart on low impact in particular being intractable without learning human values.
To be precise: I argue low impact is intractable without learning a subset of human values; the full set is not needed.
Thanks for clarifying! I haven’t brought this up on your research agenda because I prefer to have the discussion during an upcoming sequence of mine, and it felt unfair to comment on your agenda, “I disagree but I won’t elaborate right now”.