I’ll try and explain a bit more why I am not very interested in probabilities and DTs. I am interested in how decisions are made, but I am far more interested in how an agent gets to have a certain model in the first place (before it is converted into an action). With a finite agent there are questions such as why have model X rather than Y. Which I think impinges on the question on what topics we should discuss. I’d view most people not having a low probability that DTs are important, but simply not storing a probability for that preposition at all. They have never explored it so have no evidence either way.
The model of the world you have can dominate the DT, in determining the action taken. And in the end that is what we care about, the action taken in response to the input and history.
I also think that DT with its fixed model ignores the possibility of communication between the bit running through the model and picking an action and the bit that creates the model. For example if I see a very good contest/offer I might think it too good to be true, and look for more information to alter my model and find the catch before taking the offer up.
For example if I see a very good contest/offer I might think it too good to be true, and look for more information to alter my model and find the catch before taking the offer up.
How is this case different from any other decision? You compute the current probabilities for this is a fraud and this is an unusually good deal. You compute the cost of collecting more data in a specific fashion, and the probability distribution over possible futures containing a future version of you with better knowledge about this problem. You do the same for various alternative actions you could take instead of collecting more data right now, calculate expected long-run utility for each of the considered possible futures, and choose an action based on that information—either to prod the universe to give you more data about this, or doing something else.
I am glossing over all the interesting hard parts, of course. But still, is there anything fundamentally different about manipulating the expected state of knowledge of your future-self from manipulating any other part of reality?
Interesting question. Not quite what I was getting at. I hope you don’t mind if I use a situation where extra processing can get you more information.
A normal decision theory can be represented as simple function from model to action. It should halt.
decisiontheory :: Model → Action
Lets say you have a model that you can keep on expanding the consequences of and get a more accurate picture of what is going to happen, like playing chess with a variable amount of look ahead. What the system is looking for is a program that will recursively self improve and be Friendly (where making an action is considered making an AI).
It has a function that can either carry on expanding the model or return an action.
modelOrAct :: Model → Either Action Model
You can implement decisiontheory with this code
decisiontheory :: Model → Action
decisiontheory m = either (decisionModel) (id) (modelOrAct m)
However this has the potential to infinite loop due to its recursive definition. This would happen if the expected utility of increasing the accuracy of the model is greater than performing an action and there is no program it can prove safe. You would want some way of interrupting it to update the model with information from the real world as well as the extrapolation.
So I suppose the difference in this case is that due to making a choice on which mental actions to perform you can get stuck not getting information from the world about real world actions.
The model of the world you have can dominate the DT, in determining the action taken. And in the end that is what we care about, the action taken in response to the input and history.
No, the model of the world you have can not dominate the DT or, for that matter, do anything at all. There must be a decision theory either explicit or implicit in some action generating algorithm that you are running. Then it is just a matter of how much much effort you wish to spend developing each.
I also think that DT with its fixed model ignores the possibility of communication between the bit running through the model and picking an action and the bit that creates the model. For example if I see a very good contest/offer I might think it too good to be true, and look for more information to alter my model and find the catch before taking the offer up.
A Decision Theory doesn’t make you naive or impractical. Deciding to look for more information is just a good decision.
No, the model of the world you have can not dominate the DT or, for that matter, do
anything at all. There must be a decision theory either explicit or implicit in some action
generating algorithm that you are running. Then it is just a matter of how much much
effort you wish to spend developing each.
I spoke imprecisely. I meant that the part of the program that generates the model of the world dominates the DT in terms of what action is taken. That is; with a fixed DT you can make it perform any action dependent upon what model you give it. The converse is not true as the model constrains the possible actions.
A Decision Theory doesn’t make you naive or impractical. Deciding to look for more
information is just a good decision.
I think in terms of code and Types. Most discussions of DTs don’t have discussions of feeding back the utilities to the model making section, so I’m assuming a simple type. It might be wrong, but at least I can be precise about what I am talking about. See my reply to Sebastian.
Thanks.
I’ll try and explain a bit more why I am not very interested in probabilities and DTs. I am interested in how decisions are made, but I am far more interested in how an agent gets to have a certain model in the first place (before it is converted into an action). With a finite agent there are questions such as why have model X rather than Y. Which I think impinges on the question on what topics we should discuss. I’d view most people not having a low probability that DTs are important, but simply not storing a probability for that preposition at all. They have never explored it so have no evidence either way.
The model of the world you have can dominate the DT, in determining the action taken. And in the end that is what we care about, the action taken in response to the input and history.
I also think that DT with its fixed model ignores the possibility of communication between the bit running through the model and picking an action and the bit that creates the model. For example if I see a very good contest/offer I might think it too good to be true, and look for more information to alter my model and find the catch before taking the offer up.
How is this case different from any other decision? You compute the current probabilities for this is a fraud and this is an unusually good deal. You compute the cost of collecting more data in a specific fashion, and the probability distribution over possible futures containing a future version of you with better knowledge about this problem. You do the same for various alternative actions you could take instead of collecting more data right now, calculate expected long-run utility for each of the considered possible futures, and choose an action based on that information—either to prod the universe to give you more data about this, or doing something else.
I am glossing over all the interesting hard parts, of course. But still, is there anything fundamentally different about manipulating the expected state of knowledge of your future-self from manipulating any other part of reality?
Interesting question. Not quite what I was getting at. I hope you don’t mind if I use a situation where extra processing can get you more information.
A normal decision theory can be represented as simple function from model to action. It should halt.
decisiontheory :: Model → Action
Lets say you have a model that you can keep on expanding the consequences of and get a more accurate picture of what is going to happen, like playing chess with a variable amount of look ahead. What the system is looking for is a program that will recursively self improve and be Friendly (where making an action is considered making an AI).
It has a function that can either carry on expanding the model or return an action.
modelOrAct :: Model → Either Action Model
You can implement decisiontheory with this code
decisiontheory :: Model → Action
decisiontheory m = either (decisionModel) (id) (modelOrAct m)
However this has the potential to infinite loop due to its recursive definition. This would happen if the expected utility of increasing the accuracy of the model is greater than performing an action and there is no program it can prove safe. You would want some way of interrupting it to update the model with information from the real world as well as the extrapolation.
So I suppose the difference in this case is that due to making a choice on which mental actions to perform you can get stuck not getting information from the world about real world actions.
No, the model of the world you have can not dominate the DT or, for that matter, do anything at all. There must be a decision theory either explicit or implicit in some action generating algorithm that you are running. Then it is just a matter of how much much effort you wish to spend developing each.
A Decision Theory doesn’t make you naive or impractical. Deciding to look for more information is just a good decision.
I spoke imprecisely. I meant that the part of the program that generates the model of the world dominates the DT in terms of what action is taken. That is; with a fixed DT you can make it perform any action dependent upon what model you give it. The converse is not true as the model constrains the possible actions.
I think in terms of code and Types. Most discussions of DTs don’t have discussions of feeding back the utilities to the model making section, so I’m assuming a simple type. It might be wrong, but at least I can be precise about what I am talking about. See my reply to Sebastian.