The current literature on scheming appears to have been inspired by Paul Christiano’s speculations about malign intelligences in Solomonoff induction
This doesn’t seem right. The linked post by Paul here is about the (extremely speculative) case where consequentialist life emerges organically inside of full blown simulations (e.g. evolving from scratch) while arguments about ML models never go here.
Regardless, concerns and arguments about scheming are much older than Paul’s posts on this topic.
(That said, I do think that people have made scheming style arguments based on intuitions from thinking about AIXI and the space of turing machines at various points. Though this was never very key and I don’t believe these arguments are ever in reference to cases where a literal simulation evolves life.)
This doesn’t seem right. The linked post by Paul here is about the (extremely speculative) case where consequentialist life emerges organically inside of full blown simulations (e.g. evolving from scratch) while arguments about ML models never go here.
Regardless, concerns and arguments about scheming are much older than Paul’s posts on this topic.
(That said, I do think that people have made scheming style arguments based on intuitions from thinking about AIXI and the space of turing machines at various points. Though this was never very key and I don’t believe these arguments are ever in reference to cases where a literal simulation evolves life.)