Something like that. I posted my pseudocode in an open thread a few days ago to get feedback and I couldn’t get indentation to work either so I posted mine to Pastebin and linked it.
I’m still going through the Sequences, and I read Terminal Values and Instrumental Values the other day. Eliezer makes a pseudocode example of an ideal Bayesian decision system (as well as its data types), which is what an AGI would be a computationally tractable approximation of. If you can show me what you mean in terms of that post, then I might be able to understand you. It doesn’t look like I was far off conceptually, but thinking of it his way is better than thinking of it my way. My way’s kind of intuitive I guess (or I wouldn’t have been able to make it up) but his is accurate.
I also found his paper (Paper? More like book) Creating Friendly AI. Probably a good read for avoiding amateur mistakes, which we might be making. I intend to read it. Probably best not to try to read it in one sitting.
Even though I don’t want you to think of it this way, here’s my pseudocode just to give you an idea of what was going on in my head. If you see a name followed by parentheses, then that is the name of a function. ‘Def’ defines a function. The stuff that follows it is the function itself. If you see a function name without a ‘def’, then that means it’s being called rather than defined. Functions might call other functions. If you see names inside of the parentheses that follow a function, then those are arguments (function inputs). If you see something that is clearly a name, and it isn’t followed by parentheses, then it’s an object: it holds some sort of data. In this example all of the objects are first created as return values of functions (function outputs). And anything that isn’t indented at least once isn’t actually code. So ‘For AGI in general’ is not a for loop, lol.
Okay, I am convinced. I really, really appreciate you sticking with me through this and persistently finding different ways to phrase your side and then finding ways that other people have phrased it.
For reference it was the link to the paper/book that did it. The parts of it that are immediately relevant here are chapter 3 and section 4.2.1.1 (and optionally section 5.3.5). In particular, chapter 3 explicitly describes an order of operations of goal and subgoal evaluation and then the two other sections show how wireheading is discounted as a failing strategy within a system with a well-defined order of operations. Whatever problems there may be with value stability, this has helped to clear out a whole category of mistakes that I might have made.
Again, I really appreciate the effort that you put in. Thanks a load.
Something like that. I posted my pseudocode in an open thread a few days ago to get feedback and I couldn’t get indentation to work either so I posted mine to Pastebin and linked it.
I’m still going through the Sequences, and I read Terminal Values and Instrumental Values the other day. Eliezer makes a pseudocode example of an ideal Bayesian decision system (as well as its data types), which is what an AGI would be a computationally tractable approximation of. If you can show me what you mean in terms of that post, then I might be able to understand you. It doesn’t look like I was far off conceptually, but thinking of it his way is better than thinking of it my way. My way’s kind of intuitive I guess (or I wouldn’t have been able to make it up) but his is accurate.
I also found his paper (Paper? More like book) Creating Friendly AI. Probably a good read for avoiding amateur mistakes, which we might be making. I intend to read it. Probably best not to try to read it in one sitting.
Even though I don’t want you to think of it this way, here’s my pseudocode just to give you an idea of what was going on in my head. If you see a name followed by parentheses, then that is the name of a function. ‘Def’ defines a function. The stuff that follows it is the function itself. If you see a function name without a ‘def’, then that means it’s being called rather than defined. Functions might call other functions. If you see names inside of the parentheses that follow a function, then those are arguments (function inputs). If you see something that is clearly a name, and it isn’t followed by parentheses, then it’s an object: it holds some sort of data. In this example all of the objects are first created as return values of functions (function outputs). And anything that isn’t indented at least once isn’t actually code. So ‘For AGI in general’ is not a for loop, lol.
http://pastebin.com/UfP92Q9w
Okay, I am convinced. I really, really appreciate you sticking with me through this and persistently finding different ways to phrase your side and then finding ways that other people have phrased it.
For reference it was the link to the paper/book that did it. The parts of it that are immediately relevant here are chapter 3 and section 4.2.1.1 (and optionally section 5.3.5). In particular, chapter 3 explicitly describes an order of operations of goal and subgoal evaluation and then the two other sections show how wireheading is discounted as a failing strategy within a system with a well-defined order of operations. Whatever problems there may be with value stability, this has helped to clear out a whole category of mistakes that I might have made.
Again, I really appreciate the effort that you put in. Thanks a load.
And thank you for sticking with me! It’s really hard to stick it out when there’s no such thing as an honest disagreement and disagreement is inherently disrespectful!
ETA: See the ETA in this comment to understand how my reasoning was wrong but my conclusion was correct.