Michele Campolo

Some al­ter­na­tive AI safety re­search projects

• Michele CampoloNov 22, 2021, 9:45 AM

  2 points

  in reply to: Jon Garcia’s comment on: From lan­guage to ethics by au­to­mated reasoning

  Natural language exists as a low-bandwidth communication channel for imprinting one person’s mental map onto another person’s. The mental maps themselves are formed through direct interactions with an external environment.

  It doesn’t seem impossible to create a mental map just from language: in this case, language itself would play the role of the external environment. But overall I agree with you, it’s uncertain whether we can reach a good level of world understanding just from natural language inputs.

  Regarding your second paragraph:

  even if this AI had a complete understanding of human emotions and moral systems, it would not necessarily be aligned.

  I’ll quote the last paragraph under the heading “Error”:

  Regarding other possible failure modes, note that I am not trying to produce a safety module that, when attached to a language model, will make that language model safe. What I have in mind is more similar to an independent-ethical-thinking module: if the resulting AI states something about morality, we’ll still have to look at the code and try to understand what’s happening, e.g. what the AI exactly means with the term “morality”, and whether it is communicating honestly or is trying to persuade us. This is also why doing multiple tests will be practically mandatory.

From lan­guage to ethics by au­to­mated reasoning

• Michele CampoloAug 27, 2021, 9:46 PM

  LW: 1 AF: 1

  AF

  on: AI Risk for Epistemic Minimalists

  The reached conclusion—that it is possible to do something about the situation—is weak, but I really like the minimalist style of the arguments. Great post!

• Michele CampoloMay 13, 2021, 10:34 AM

  1 point

  in reply to: Kaj_Sotala’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  How do you feel about:

  1
  There is
  a procedure/​algorithm which doesn’t seem biased towards a particular value system
  such that
  a class of AI systems that implement it end up having a common set of values, and they endorse the same values upon reflection.
  
  2
  This set of values might have something in common with what we, humans, call values.
  
  If 1 and 2 seem at least plausible or conceivable, why can’t we use them as a basis to design aligned AI? Is it because of skepticism towards 1 or 2?

• Michele CampoloMay 13, 2021, 9:59 AM

  1 point

  in reply to: johnswentworth’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  “How the physical world works” seems, to me, a plausible source-of-truth. In other words: I consider some features of the environment (e.g. consciousness) as a reason to believe that some AI systems might end up caring about a common set of things, after they’ve spent some time gathering knowledge about the world and reasoning. Our (human) moral intuitions might also be different from this set.

• Michele CampoloMay 13, 2021, 9:40 AM

  2 points

  in reply to: Waddington’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  I disagree. Determinism doesn’t make the concepts of “control” or “causation” meaningless. It makes sense to say that, to a certain degree, you often can control your own attention, while in other circumstances you can’t: if there’s a really loud sound near you, you are somewhat forced to pay attention to it.

  From there you can derive a concept of responsibility, which is used e.g. in law. I know that the book Actual Causality focuses on these ideas (but there might be other books on the same topics that are easier to read or simply better in their exposition).

• Michele CampoloMay 12, 2021, 6:58 PM

  1 point

  in reply to: johnswentworth’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  At the very least, we have strong theoretical reasoning models (like Bayesian reasoners, or Bayesian EU maximizers), which definitely do not go looking for values to pursue, or adopt new values.

  This does not imply one cannot build an agent that works according to a different framework. VNM Utility maximization requires a complete ordering of preferences, and does not say anything about where the ordering comes from in the first place.
  (But maybe your point was just that our current models do not “look for values”)

  Why would something which doesn’t already have values be looking for values? Why would conscious experiences and memory “seem valuable” to a system which does not have values already? Seems like having a “source of value” already is a prerequisite to something seeming valuable—otherwise, what would make it seem valuable?

  An agent could have a pre-built routine or subagent that has a certain degree of control over what other subagents do—in a sense, it determines what are the “values” of the rest of the system. If this routine looks unbiased /​ rational /​ valueless, we have a system that considers some things as valuable (acts to pursue them) without having a pre-value, or at least the pre-value doesn’t look like something that we would consider a value.

• Michele CampoloMay 12, 2021, 5:35 PM

  1 point

  in reply to: weathersystems’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  I can’t say I am an expert on realism and antirealism, but I have already spent time on metaethics textbooks and learning about metaethics in general. With this question I wanted to get an idea of what are the main arguments on LW, and maybe find new ideas I hadn’t considered.

  What is “disinterested altruism”? And why do you think it’s connected to moral anti-realism?

  I see a relation with realism. If certain pieces of knowledge about the physical world (how human and animal cognition works) can motivate a class of agents that we would also recognise as unbiased and rational, that would be a form of altruism that is not instrumental and not related to game theory.

• Michele CampoloMay 12, 2021, 5:23 PM

  1 point

  in reply to: johnswentworth’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  Thank you for the detailed answer! I’ll read Three Worlds Collide.

  That brings us to the real argument: why does the moral realist believe this? “What do I think I know, and how do I think I know it?” What causal, physical process resulted in that belief?

  I think a world full of people who are always blissed out is better than a world full of people who are always depressed or in pain. I don’t have a complete ordering over world-histories, but I am confident in this single preference, and if someone called this “objective value” or “moral truth” I wouldn’t say they are clearly wrong. In particular, if someone told me that there exists a certain class of AI systems that end up endorsing the same single preference, and that these AI systems are way less biased and more rational than humans, I would find all that plausible. (Again, compare this if you want.)

  Now, why do I think this?

  I am a human and I am biased by my own emotional system, but I can still try to imagine what would happen if I stopped feeling emotions. I think I would still consider the happy world more valuable than the sad world. Is this a proof that objective value is a thing? Of course not. At the same time, I can imagine also an AI system thinking: “Look, I know various facts about this world. I don’t believe in golden rules written in fire etched into the fabric of reality, or divine commands about what everyone should do, but I know there are some weird things that have conscious experiences and memory, and this seems something valuable in itself. Moreover, I don’t see other sources of value at the moment. I guess I’ll do something about it.” (Taken from this comment)

• Michele CampoloMay 12, 2021, 4:34 PM

  1 point

  in reply to: ike’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  That is an interesting point. More or less, I agree with this sentence in your fist post:

  As far as I can tell, we can do science just as well without assuming that there’s a real territory out there somewhere.

  in the sense that one can do science by speaking only about their own observations, without making a distinction between what is observed and what “really exists”.

  On the other hand, when I observe that other nervous systems are similar to my own nervous system, I infer that other people have subjective experiences similar to mine. How does this fit in your framework? (Might be irrelevant, sorry if I misunderstood)

• Michele CampoloMay 12, 2021, 4:15 PM

  2 points

  in reply to: Yair Halberstadt’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  I didn’t want to start a long discussion. My idea was to get some random feedback to see if I was missing some important ideas I had not considered

• Michele CampoloMay 12, 2021, 2:21 PM

  1 point

  in reply to: TAG’s comment on: What is the strongest ar­gu­ment you know for an­tire­al­ism?

  Thanks for your answer, but I am looking for arguments, not just statements or opinions. How do you know that value is not a physical property? What do you mean when you say that altruism is not a consequence of epistemic rationality, and how do you know?

[Question] What is the strongest ar­gu­ment you know for an­tire­al­ism?

• Michele CampoloMay 4, 2021, 1:57 PM

  LW: 1 AF: 1

  AF

  in reply to: Daniel Kokotajlo’s comment on: Nat­u­ral­ism and AI alignment

  I am not sure the concept of naturalism I have in mind corresponds to a specific naturalistic position held by a certain (group of) philosopher(s). I link here the Wikipedia page on ethical naturalism, which contains the main ideas and is not too long. Below I focus on what is relevant for AI alignment.

  In the other comment you asked about truth. AIs often have something like a world-model or knowledge base that they rely on to carry out narrow tasks, in the sense that if someone modifies the model or kb in a certain way—analogous to creating a false belief—than the agent fails at the narrow task. So we have a concept of true-given-task. By considering different tasks, e.g. in the case of a general agent that is prepared to face various tasks, we obtain true-in-general or, if you prefer, simply “truth”. See also the section on knowledge in the post. Practical example: given that light is present almost everywhere in our world, I expect general agents to acquire knowledge about electromagnetism.

  I also expect that some AIs, given enough time, will eventually incorporate in their world-model beliefs like: “Certain brain configurations correspond to pleasurable conscious experiences. These configurations are different from the configurations observed in (for example) people who are asleep, and very different from what is observed in rocks.”

  Now, take an AI with such knowledge and give it some amount of control over which goals to pursue: see also the beginning of Part II in the post. Maybe, in order to make this modification, it is necessary to abandon the single-agent framework and consider instead a multi-agent system, where one agent keeps expanding the knowledge base, another agent looks for “value” in the kb, and another one decides what actions to take given the current concept of value and other contents of the kb.

  [Two notes on how I am using the word control. 1 I am not assuming any extra-physical notion here: I am simply thinking of how, for example, activity in the prefrontal cortex regulates top-down attentional control, allowing us humans (and agents with similar enough brains/​architectures) to control, to a certain degree, what to pay attention to. 2 Related to what you wrote about “catastrophically wrong” theories: there is no need to give such an AI high control over the world. Rather, I am thinking of control over what to write as output in a text interface, like a chatbot that is not limited to one reply for each input message]

  The interesting question for alignment is: what will such an AI do (or write)? This information is valuable even if the AI doesn’t have high control over the world. Let’s say we do manage to create a collection of human preferences; we might still notice something like: “Interesting, this AI thinks this subset of preferences doesn’t make sense” or “Cool, this AI considers valuable the thing X that we didn’t even consider before”. Or, if collecting human preferences proves to be difficult, we could use some information this AI gives us to build other AIs that instead act according to an explicitly specified value function.

  I see two possible objections.

  1 The AI described above cannot be built. This seems unlikely: as long as we can emulate what the human mind does, we can at least try to create less biased versions of it. See also the sentence you quoted in the other comment. Indeed, depending on how biased we judge that AI to be, the obtained information will be less, or more, valuable to us.

  2 Such an AI will never act ethically or altruistically, and/​or its behaviour will be unpredictable. I consider this objection more plausible, but I also ask: how do you know? In other words: how can one be so sure about the behaviour of such an AI? I expect the related arguments to be more philosophical than technical. Given uncertainty, (to me) it seems correct to accept a non-trivial chance that the AI reasons like this: “Look, I know various facts about this world. I don’t believe in golden rules written in fire etched into the fabric of reality, or divine commands about what everyone should do, but I know there are some weird things that have conscious experiences and memory, and this seems something valuable in itself. Moreover, I don’t see other sources of value at the moment. I guess I’ll do something about it.”

  Philosophically speaking, I don’t think I am claiming anything particularly new or original: the ideas already exist in the literature. See, for example, 4.2 and 4.3 in the SEP page on Altruism.

  What links here?

  • What is the strongest ar­gu­ment you know for an­tire­al­ism? by Michele Campolo (12 May 2021 10:53 UTC; 7 points)

• Michele Campolo 26 Apr 2021 8:04 UTC

  LW: 1 AF: 1

  AF

  in reply to: Charlie Steiner’s comment on: Nat­u­ral­ism and AI alignment

  If there is a superintelligent AI that ends up being aligned as I’ve written, probably there is also a less intelligent agent that does the same thing. Something comparable to human-level might be enough.

  From another point of view: some philosophers are convinced that caring about conscious experiences is the rational thing to do. If it’s possible to write an algorithm that works in a similar way to how their mind works, we already have an (imperfect, biased, etc.) agent that is somewhat aligned, and is likely to stay aligned after further reflection.

  One could argue that these philosophers are fooling themselves, that no really intelligent agent will end up with such weird beliefs. So far, I haven’t seen convincing arguments in favour of this; it goes back to the metaethical discussion. I quote a sentence I have written in the post:

  Depending on one’s background knowledge of philosophy and AI, the idea that rationality plays a role in reasoning about goals and can lead to disinterested (not game-theoretic or instrumental) altruism may seem plain wrong or highly speculative to some, and straightforward to others.

• Michele Campolo 26 Apr 2021 7:45 UTC

  LW: 1 AF: 1

  AF

  in reply to: Eliezer Yudkowsky’s comment on: Nat­u­ral­ism and AI alignment

  Thanks, that page is much more informative than anything else I’ve read on the orthogonality thesis.

  1 From Arbital:

  The Orthogonality Thesis states “there exists at least one possible agent such that...”

  Also my claim is an existential claim, and I find it valuable because it could be an opportunity to design aligned AI.

  2 Arbital claims that orthogonality doesn’t require moral relativism, so it doesn’t seem incompatible with what I am calling naturalism in the post.

  3 I am ok with rejecting positions similar to what Arbital calls universalist moral internalism. Statements like “All agents do X” cannot be exact.

• Michele Campolo 25 Apr 2021 10:21 UTC

  LW: 1 AF: 1

  AF

  in reply to: Charlie Steiner’s comment on: Nat­u­ral­ism and AI alignment

  I am aware of interpretability issues. This is why, for AI alignment, I am more interested in the agent described at the beginning of Part II than Scientist AI.

  Thanks for the link to the sequence on concepts, I found it interesting!

Nat­u­ral­ism and AI alignment

Liter­a­ture Re­view on Goal-Directedness