Firstly, it could refer to some sort of ‘self awareness’ whatever that means.
Secondly it could refer to possessing some sort of system for reasoning about abstract goals.
Thirdly, it could refer to having any goals whatsoever.
In any case, it does us good to realise that “make me into an agent” is what a tool AI would consider the best possible plan for many goals. So without a hint of agency, it’s motivated to make us make it into a agent.
Regardless of which definition of agent I am using, this makes no sense to me. If its capable of creating a plan for modifying into an agent, then it already is an agent by definition.
If its capable of creating a plan for modifying into an agent, then it already is an agent by definition.
It could simply be listing plans by quality, as a tool might. It turns out the top plan is “use this piece of software as am agent”. That piece of software is the tool, but it achieved that effect simply my listing and ranking plans.
Use this piece of software .as an agent.....for what? An agent is only good for fulfilling open ended goals, like “make as much money as possible”. So it would seem we can avoid tools rewriting themselves as agents by not giving them open ended goals.
If you can write a non opened ended goal in the sense you’re implying, you’ve solved the “reduced impact AI” problem, and most of the friendliness problem as well.
I believe I’ve done that every time I’ve used Google maps.
“How do I get from location A to location B” is more open ended than “How do I get from location A to location B in an automobile” which is even still much more open ended than “How do I get from a location near A to a location near B obeying all traffic laws in a reasonably minimal time while operating a widely available automobile (that can’t fly, jump over traffic jams, ford rivers, rappel, etc.)”
Google is drastically narrowing the search space for achieving your goal, and presumably doing it manually and not with an AGI they told to host a web page with maps of the world that tells people how to quickly get from one location to another. Google is not alone in sending drivers off cliffs, into water, the wrong way down one way streets, or across airport tarmacs.
Safely narrowing the search space is the hard problem.
Safely narrowing the search space is the hard problem.
...If you are dealing with an entity that can’t add context (or ask for clarifications) the way a human would.
However, an entity that is posited as have a human level intelligence, and the ability to understand natural language would have the ability to contextualise. It wouldn’t be able to pass a turing test without it.
Less intelligent and more specialised systems have an inherently narrow search space.
What does that leave...the dreaded AIXI? Theoretically it doesn’t have actual language, and theoretically , it does have wide search space.… but practically,it does nothing.
...If you are dealing with an entity that can’t add context (or ask for clarifications) the way a human would.
Can we note you’ve moved from “the problem is not open ended” to “the AGI is programmed in such a way that the problem is not open ended”, which is the whole of the problem.
In a sense. Non openness is a non problem for fairly limited AIs, because their limitations prevent them having a wide search space that would need to be narrowed down. Non openness is also something that is part of, or an implication of, an ability that is standardly assumed in a certain class of AGIs, namely those with human level linguistic ability. To understand a sentence correctly is to narrow down its space of possible meanings.
Only AIXIs have an own oneness that would need additional measures to narrow them down.
They are no threat at the moment, and the easy answer to AI safety might be to not use them....like we don’t build hydrogen filled airships.
I have three ideas about what agent could mean.
Firstly, it could refer to some sort of ‘self awareness’ whatever that means. Secondly it could refer to possessing some sort of system for reasoning about abstract goals. Thirdly, it could refer to having any goals whatsoever.
Regardless of which definition of agent I am using, this makes no sense to me. If its capable of creating a plan for modifying into an agent, then it already is an agent by definition.
It could simply be listing plans by quality, as a tool might. It turns out the top plan is “use this piece of software as am agent”. That piece of software is the tool, but it achieved that effect simply my listing and ranking plans.
Use this piece of software .as an agent.....for what? An agent is only good for fulfilling open ended goals, like “make as much money as possible”. So it would seem we can avoid tools rewriting themselves as agents by not giving them open ended goals.
If you can write a non opened ended goal in the sense you’re implying, you’ve solved the “reduced impact AI” problem, and most of the friendliness problem as well.
I believe I’ve done that every time I’ve used Google maps.
“How do I get from location A to location B” is more open ended than “How do I get from location A to location B in an automobile” which is even still much more open ended than “How do I get from a location near A to a location near B obeying all traffic laws in a reasonably minimal time while operating a widely available automobile (that can’t fly, jump over traffic jams, ford rivers, rappel, etc.)”
Google is drastically narrowing the search space for achieving your goal, and presumably doing it manually and not with an AGI they told to host a web page with maps of the world that tells people how to quickly get from one location to another. Google is not alone in sending drivers off cliffs, into water, the wrong way down one way streets, or across airport tarmacs.
Safely narrowing the search space is the hard problem.
...If you are dealing with an entity that can’t add context (or ask for clarifications) the way a human would.
However, an entity that is posited as have a human level intelligence, and the ability to understand natural language would have the ability to contextualise. It wouldn’t be able to pass a turing test without it.
Less intelligent and more specialised systems have an inherently narrow search space.
What does that leave...the dreaded AIXI? Theoretically it doesn’t have actual language, and theoretically , it does have wide search space.… but practically,it does nothing.
Can we note you’ve moved from “the problem is not open ended” to “the AGI is programmed in such a way that the problem is not open ended”, which is the whole of the problem.
In a sense. Non openness is a non problem for fairly limited AIs, because their limitations prevent them having a wide search space that would need to be narrowed down. Non openness is also something that is part of, or an implication of, an ability that is standardly assumed in a certain class of AGIs, namely those with human level linguistic ability. To understand a sentence correctly is to narrow down its space of possible meanings.
Only AIXIs have an own oneness that would need additional measures to narrow them down.
They are no threat at the moment, and the easy answer to AI safety might be to not use them....like we don’t build hydrogen filled airships.