So much this. It’s a common lesswrong mantra but it’s hard to really internalize the implications of you are running on hostile hardware. These sorts of experiments are akin to gathering intelligence on a clever enemy.
I like to describe policy-breakage-rationalization as “Your brain will work very hard to lie to you.”
Where’s the original source of the hostile hardware quote, by the way? It’s attributed to Justin Corwin. I wanted to look it up in context, but all the google hits lead back here.
So much this. It’s a common lesswrong mantra but it’s hard to really internalize the implications of you are running on hostile hardware. These sorts of experiments are akin to gathering intelligence on a clever enemy.
See also: Alien parasite technical guy.
I like to describe policy-breakage-rationalization as “Your brain will work very hard to lie to you.”
Where’s the original source of the hostile hardware quote, by the way? It’s attributed to Justin Corwin. I wanted to look it up in context, but all the google hits lead back here.
Weird, I could have sworn I’ve seen it a bunch of times. I guess it just made an impression on me and I use it a lot.