Still, ASI is just equation model F(X)=Y on steroids, where F is given by the world (physics), X is a search process (natural Monte-Carlo, or biological or artificial world parameter search), and Y is goal (or rewards).
To control ASI, you control the “Y” (right side) of equation. Currently, humanity has formalized its goals as expected behaviors codified in legal systems and organizational codes of ethics, conduct, behavior, etc. This is not ideal, because those codes are mostly buggy.
Ideally, the “Y” would be dynamically inferred and corrected, based on each individual’s self-reflections, evolving understanding about who they really are, because the deeper you look, the more you realize, how each of us is a mystery.
I like the term “Y-combinator”, as this reflects what we have to do—combine our definitions of “Y” into the goals that AIs are going to pursue. We need to invent new, better “Y-combination” systems that reward AI systems being trained.
Still, ASI is just equation model F(X)=Y on steroids, where F is given by the world (physics), X is a search process (natural Monte-Carlo, or biological or artificial world parameter search), and Y is goal (or rewards).
To control ASI, you control the “Y” (right side) of equation. Currently, humanity has formalized its goals as expected behaviors codified in legal systems and organizational codes of ethics, conduct, behavior, etc. This is not ideal, because those codes are mostly buggy.
Ideally, the “Y” would be dynamically inferred and corrected, based on each individual’s self-reflections, evolving understanding about who they really are, because the deeper you look, the more you realize, how each of us is a mystery.
I like the term “Y-combinator”, as this reflects what we have to do—combine our definitions of “Y” into the goals that AIs are going to pursue. We need to invent new, better “Y-combination” systems that reward AI systems being trained.