I’m glad that you guys are interested in working on IBP/PreDCA. Here are a few points that might help you:
The scope of this project seems extremely ambitious. I think that the road from here to empirical demonstrations (assuming that you mean in the real-world rather than some artificial toy setting) is a programme for many people working over many years. Therefore, I think you will benefit from zooming in and deciding on the particular first step you want to take along that road.
Technicality: the definition of g in the IBP post is for a fixed loss function, because that was sufficient for purposes of that post, but the definition of the cartesian version is loss-function-agnostic. Ofc it’s completely straightforward to write a loss-function-specific cartesian version and a loss-function-agnostic physicalist version.
Regarding the agentometer/utiliscope, I think it’s probably best to start from studying the cartesian versions, because that’s likely to be simpler and the physicalist theory will build on that.
Specifically, we want to get theorems along the lines of: (i) For an agent with g≫0, the asymptotic behavior of the utility function can be inferred nearly unambiguously. (ii) Inferring the utility function of agent-1 and then optimizing it via agent-2 that e.g. has a richer observation channel leads to results that are better for agent-1 than what agent-1 can do on its own, in the long time horizon limit.
The epistemic status of the ulitiscope formula from my presentation is: I’m pretty optimistic that there is some correct formula along those lines, but the specific formula I wrote there is just my best guess after thinking for a short time and I am far from confident it is correct. My confidence would become much better if we demonstrated some non-trivial theorems that show it satisfies some intuitive desiderata.
The epistemic status of the definition of g is: I’m pretty optimistic it is close to correct, but there is definitely room for quibbling over the details.
While studying the computational complexity of the relevant mathematical objects is reasonable, I advise to steer clear of practical implementations of IBRL/IBP (assuming that “practical” means “competitive+ with ANNs”) because of the associated risks, until we are much further along on the theoretical side.
Also, I am completely open to discussing the details of your project in private, if you’re serious about it.
Thank you so much for your thoughtful reply. To respond to a few of your points:
We only mean to test this in an artificial toy setting. We agree that empirical demonstrations seem very difficult.
Thanks for pointing out the cartesian versions -- I hadn’t read this before, and this is a nice clarification on how to measure g in a loss-function agnostic way.
It’s good to know about the epistemic status of this part of the theory, we might take a stab at proving some of these bounds.
We will definitely make sure to avoid competitive implementations because of the associated risks.
We would very much appreciate discussing details in private, we are serious about it. I’ll follow up with a DM on LessWrong soon.
I’m glad that you guys are interested in working on IBP/PreDCA. Here are a few points that might help you:
The scope of this project seems extremely ambitious. I think that the road from here to empirical demonstrations (assuming that you mean in the real-world rather than some artificial toy setting) is a programme for many people working over many years. Therefore, I think you will benefit from zooming in and deciding on the particular first step you want to take along that road.
Technicality: the definition of g in the IBP post is for a fixed loss function, because that was sufficient for purposes of that post, but the definition of the cartesian version is loss-function-agnostic. Ofc it’s completely straightforward to write a loss-function-specific cartesian version and a loss-function-agnostic physicalist version.
Regarding the agentometer/utiliscope, I think it’s probably best to start from studying the cartesian versions, because that’s likely to be simpler and the physicalist theory will build on that.
Specifically, we want to get theorems along the lines of: (i) For an agent with g≫0, the asymptotic behavior of the utility function can be inferred nearly unambiguously. (ii) Inferring the utility function of agent-1 and then optimizing it via agent-2 that e.g. has a richer observation channel leads to results that are better for agent-1 than what agent-1 can do on its own, in the long time horizon limit.
The epistemic status of the ulitiscope formula from my presentation is: I’m pretty optimistic that there is some correct formula along those lines, but the specific formula I wrote there is just my best guess after thinking for a short time and I am far from confident it is correct. My confidence would become much better if we demonstrated some non-trivial theorems that show it satisfies some intuitive desiderata.
The epistemic status of the definition of g is: I’m pretty optimistic it is close to correct, but there is definitely room for quibbling over the details.
While studying the computational complexity of the relevant mathematical objects is reasonable, I advise to steer clear of practical implementations of IBRL/IBP (assuming that “practical” means “competitive+ with ANNs”) because of the associated risks, until we are much further along on the theoretical side.
Also, I am completely open to discussing the details of your project in private, if you’re serious about it.
Hi Vanessa!
Thank you so much for your thoughtful reply. To respond to a few of your points:
We only mean to test this in an artificial toy setting. We agree that empirical demonstrations seem very difficult.
Thanks for pointing out the cartesian versions -- I hadn’t read this before, and this is a nice clarification on how to measure g in a loss-function agnostic way.
It’s good to know about the epistemic status of this part of the theory, we might take a stab at proving some of these bounds.
We will definitely make sure to avoid competitive implementations because of the associated risks.
We would very much appreciate discussing details in private, we are serious about it. I’ll follow up with a DM on LessWrong soon.