Guillaume Corlouer

An in­for­ma­tion-the­o­retic study of ly­ing in LLMs

De­gen­era­cies are sticky for SGD

• Guillaume Corlouer 16 May 2023 23:02 UTC

  2 points

  0

  in reply to: Alexander Gietelink Oldenziel’s comment on: Towards Mea­sures of Optimisation

  Right, I got confused because I thought your problem was about trying to define a measure of optimisation power—for ex analogous to the Yudkowsky measure—that was also referring to a utility function, while being invariant from scaling and translation but this is different from asking

  “what fraction of the default expected utility comes from outcomes at least as good as this one?’”

• Guillaume Corlouer 16 May 2023 18:52 UTC

  LW: 1 AF: 1

  0

  AF

  on: Towards Mea­sures of Optimisation

  What about optimisation power of $x^{\prime }$ as a measure of outcome that have utility greater than the utility of $x^{\prime }$?

  Let $U_{x^{\prime }}$ be the set of outcome with utility greater than $x^{\prime }$ according to utility function $u$:

  $$U_{x^{\prime }}:=\{x\in X|u(x)\ge u(x^{\prime }\left)\right\}$$

  The set $U_{x^{\prime }}$ is invariant under translation and non-zero rescaling of the utility function $u$ and we define the optimisation power of the outcome $x$′ according to utility function $u$ as:

  $$O{P}_u\left(x^{\prime }\right):=-\log \left({\int }_{U_{x^{\prime }}}p\right)=-\log \left({\int }_{x\in U_{x^{\prime }}}p\right(x\left)dx\right)$$

  This does not suffer from comparing w.r.t a worst case and seem to satisfies the same intuition as the original OP definition while referring to some utility function.

  This is in fact the same measure as the original optimisation power measure with an order given by the utility function

Un­der­stand­ing mesa-op­ti­miza­tion us­ing toy models

• Guillaume Corlouer 21 Jun 2022 20:21 UTC

  6 points

  on: “Brain en­thu­si­asts” in AI Safety

  Nice recommendations! In addition to brain enthusiasts being useful for empirical work, there also are theoretical tools from system neuroscience that could be useful for AI safety. One area in particular would be for interpretability: if we want to model a network at various levels of “emergence”, recent development in information decomposition and multivariate information theory to move beyond pairwise interaction in a neural network might be very useful. Also see recent publications to model synergestic information and dynamical independance to perhaps automate macro variables discovery which could also be well worth exploring to study higher levels of large ML models. This would actually require both empirical and theoretical work as once the various measures of information decomposition are clearer one would need to empirically estimate test them and use them in actual ML systems for interpretability if they turn out to be meaningful.

• Guillaume Corlouer 31 Aug 2019 13:57 UTC

  LW: 1 AF: 1

  AF

  in reply to: Wei Dai’s comment on: Me­tal­ign­ment: De­con­fus­ing metaethics for AI al­ign­ment.

  
  

  Thanks for all the useful links! I’m also always happy to receive more feedback.

  I agree that the sense in which I use metaethics in this post is different from what academic philosophers usually call metaethics. I have the impression that metaethics, in academic sense, and metaphilosophy are somehow related. Studying what morality itself is, how to select ethical theories and what is the process behind ethical reasoning seems not independent. For example if moral nihilism is more plausible then it seems to be less likely that there is some meaningful feedback loop to select ethical theories or that there is such a meaningful thing as a ‘good’ ethical theory (at least in an observer independent way) . If moral emotivism is more plausible then maybe reflecting on ethics is more like emotions rationalisation, e.g. typically expressing in a sophisticated way something that just fundamentally means ‘boo suffering’. In that case having better understanding of metaethics in the academic sense seems to bring some light to a process that generates ethical theories, at least in humans.

• Guillaume Corlouer 31 Aug 2019 13:33 UTC

  1 point

  in reply to: Gordon Seidoh Worley’s comment on: Me­tal­ign­ment: De­con­fus­ing metaethics for AI al­ign­ment.

  Sure, I’m happy to read/​discuss your ideas about this topic.

• Guillaume Corlouer 23 Aug 2019 18:36 UTC

  1 point

  in reply to: Teerth Aloke’s comment on: Me­tal­ign­ment: De­con­fus­ing metaethics for AI al­ign­ment.

  I am not sure about what computer aided analysis mean but one possibility could be to have formal ethical theories and prove some theorem inside their formal framework. But this raises questions about the sort of formal framework that one could use to ‘prove theorems’ under ethics in a meaningful way.

Me­tal­ign­ment: De­con­fus­ing metaethics for AI al­ign­ment.