I don’t understand your counterexample in the appendix Details for penalizing inconsistencies across different inputs. You present a cheating strategy that requires the reporter to run and interpret the predictor a bunch of times, which seems plausibly slower than doing honest translation. And then you say you fix this issue with:
But this dependence could be avoided if there was an intermediate model between the predictor’s Bayes net (which we are assuming is very large) and the human’s Bayes net. Errors identified by the intermediate model are likely to be “real” errors in the human’s model. But we can identify those errors with an amount of computation that depends only on the intermediate model, rather than needing to actually run the full predictor. For example, if the human’s Bayes net has size N, the intermediate model has size 2N, and the predictor’s Bayes net has size a 1e12 x N, then the cost of using the intermediate model many times can still be small relative to the cost of direct translation.
Roughly speaking, I don’t see how the reporter with an intermediate model systematically wins out over the translator given that the intermediate model will miss many things that the predictor understands.
Taking it more slowly, using the example from the report:
Let’s say we have X1=‘do rowhammer’ (which the intermediate model does understand) and X2=‘do eldritch hacks that the intermediate model isn’t good enough to understand’.
Let’s say we have reporters:
s, which does naive human simulation,
i, which has an intermediate model which it always uses to look for an fix strange correlations.
t, a direct translator.
Let’s say the reporter is asked “Is the diamond safe?” on an input that includes X1, and the human runs consistency checks across many actions containing X1 (and none containing X2). Here, it seems like i and t both pass the consistency checks, and i is faster than t, so i wins.
But now let’s say that the reporter is asked “Is the diamond safe?” on an input that includes X2, and the human runs consistency checks across many actions containing X2. In this case, the reporter i will fail the consistency checks (since the intermediate model’s predictions won’t match the predictor’s predictions, and so the reporter will fail to adjust for the revealing correlations), so t will come out ahead.
So if these reporters are the only competitors, it seems like we should be able to tune the regularization to make t win.
I agree that i does slightly worse than t on consistency checks, but i also does better on other regularizers you’re (maybe implicitly) using like speed/simplicity, so as long as i doesn’t do too much worse it’ll still beat out the direct translator.
One possible thing you might try is some sort of lexicographic ordering of regularization losses. I think this rapidly runs into other issues with consistency checks, like the fact that the human is going to be systematically wrong about some correlations, so i potentially is more consistent than t.
I agree that i does slightly worse than t on consistency-checks, but i also does better on other regularizers you’re (maybe implicitly) using like speed/simplicity, so as long as i doesn’t do too much worse it’ll still beat out the direct translator.
Any articulable reason for why i just does slightly worse than t? Why would a 2N-node model fix a large majority of disrepancys between an N-node model and a 1e12*N-node model? I’d expect it to just fix a small fraction of them.
I think this rapidly runs into other issues with consistency checks, like the fact that the human is going to be systematically wrong about some correlations, so i potentially is more consistent than t.
Yeah, if you can get better-looking consistency than the direct translator in some cases, I agree that a sufficiently high consistency penalty will just push towards exploiting that (even if the intermediate model needs to be almost as large as the full predictor to exploit it properly).
I’m curious whether you think this is the main obstacle. If we had a version of the correlation-consistency approach that always gave the direct translator minimal expected consistency loss, do we as-of-yet lack a counterexample for it?
The high-level reason is that the 1e12N model is not that much better at prediction than the 2N model. You can correct for most of the correlation even with only a vague guess at how different the AI and human probabilities are, and most AI and human probabilities aren’t going to be that different in a way that produces a correlation the human finds suspicious. I think that the largest correlations are going to be produced by the places the AI and the human have the biggest differences in probabilities, which are likely also going to be the places where the 2N model has the biggest differences in probabilities, so they should be not that hard to correct.
I’m curious whether you think this is the main obstacle. If we had a version of the correlation-consistency approach that always gave the direct translator minimal expected consistency loss, do we as-of-yet lack a counterexample for it?
I think it wouldn’t be clear that extending the counterexample would be possible, although I suspect it would be. It might require exhibiting more concrete details about how the consistency check would be defeated, which would be interesting. In some sense, maintaining consistency across many inputs is something that you expect to be pretty hard for the human simulator to do because it doesn’t know what set of inputs it’s being checked for. I would be excited about a consistency check that gave the direct translator minimal expected consistency loss. Note that I would also be interested in basically any concrete proposal for a consistency check that seemed like it was actually workable.
Hypothesis: Maybe you’re actually not considering a reporter i that always use an intermediate model; but instead a reporter i’ that does translations on hard questions, and just uses the intermediate model on questions where it’s confident that the intermediate model understands everything relevant. I see three different possible issues with that idea:
1. To do this, i’ needs an efficient way (ie one that doesn’t scale with the size of the predictor) to (on at least some inputs) be highly confident that the intermediate model understands everything relevant about the situation. I think this is a reasonable “worst-case” assumption, but I’m not sure. If you’re using it, I’d be curious to know.
2. Even when the reporter gets inputs that the intermediate model fully understands, it seems like the reporter will run into issues if its sampling-process (for generating distributions) runs into some inputs that the intermediate model doesn’t fully understand. (I.e., if i’ gets an input for which X1 is true, and then simulates many more random inputs for which X1 is true, and by chance X2 is true for one of them, then the reporter will have to do translation on that sample). Which makes it seem unlikely that i’ wouldn’t have to do translation at least once per input.
3. If the model is confident that the intermediate model understands everything relevant about the situation, it seems more efficient to return the intermediate model’s answer to the question at hand than to loop over it many times, trying to fix correlations. So really we should get a reporter i″ that does translation on the predictor on hard questions and returns an intermediate model’s latent knowledge on easy questions. That seems like an ok reporter to get.
I don’t understand your counterexample in the appendix Details for penalizing inconsistencies across different inputs. You present a cheating strategy that requires the reporter to run and interpret the predictor a bunch of times, which seems plausibly slower than doing honest translation. And then you say you fix this issue with:
Roughly speaking, I don’t see how the reporter with an intermediate model systematically wins out over the translator given that the intermediate model will miss many things that the predictor understands.
Taking it more slowly, using the example from the report:
Let’s say we have X1=‘do rowhammer’ (which the intermediate model does understand) and X2=‘do eldritch hacks that the intermediate model isn’t good enough to understand’.
Let’s say we have reporters:
s, which does naive human simulation,
i, which has an intermediate model which it always uses to look for an fix strange correlations.
t, a direct translator.
Let’s say the reporter is asked “Is the diamond safe?” on an input that includes X1, and the human runs consistency checks across many actions containing X1 (and none containing X2). Here, it seems like i and t both pass the consistency checks, and i is faster than t, so i wins.
But now let’s say that the reporter is asked “Is the diamond safe?” on an input that includes X2, and the human runs consistency checks across many actions containing X2. In this case, the reporter i will fail the consistency checks (since the intermediate model’s predictions won’t match the predictor’s predictions, and so the reporter will fail to adjust for the revealing correlations), so t will come out ahead.
So if these reporters are the only competitors, it seems like we should be able to tune the regularization to make t win.
I agree that i does slightly worse than t on consistency checks, but i also does better on other regularizers you’re (maybe implicitly) using like speed/simplicity, so as long as i doesn’t do too much worse it’ll still beat out the direct translator.
One possible thing you might try is some sort of lexicographic ordering of regularization losses. I think this rapidly runs into other issues with consistency checks, like the fact that the human is going to be systematically wrong about some correlations, so i potentially is more consistent than t.
Any articulable reason for why i just does slightly worse than t? Why would a 2N-node model fix a large majority of disrepancys between an N-node model and a 1e12*N-node model? I’d expect it to just fix a small fraction of them.
Yeah, if you can get better-looking consistency than the direct translator in some cases, I agree that a sufficiently high consistency penalty will just push towards exploiting that (even if the intermediate model needs to be almost as large as the full predictor to exploit it properly).
I’m curious whether you think this is the main obstacle. If we had a version of the correlation-consistency approach that always gave the direct translator minimal expected consistency loss, do we as-of-yet lack a counterexample for it?
The high-level reason is that the 1e12N model is not that much better at prediction than the 2N model. You can correct for most of the correlation even with only a vague guess at how different the AI and human probabilities are, and most AI and human probabilities aren’t going to be that different in a way that produces a correlation the human finds suspicious. I think that the largest correlations are going to be produced by the places the AI and the human have the biggest differences in probabilities, which are likely also going to be the places where the 2N model has the biggest differences in probabilities, so they should be not that hard to correct.
I think it wouldn’t be clear that extending the counterexample would be possible, although I suspect it would be. It might require exhibiting more concrete details about how the consistency check would be defeated, which would be interesting. In some sense, maintaining consistency across many inputs is something that you expect to be pretty hard for the human simulator to do because it doesn’t know what set of inputs it’s being checked for. I would be excited about a consistency check that gave the direct translator minimal expected consistency loss. Note that I would also be interested in basically any concrete proposal for a consistency check that seemed like it was actually workable.
Hypothesis: Maybe you’re actually not considering a reporter i that always use an intermediate model; but instead a reporter i’ that does translations on hard questions, and just uses the intermediate model on questions where it’s confident that the intermediate model understands everything relevant. I see three different possible issues with that idea:
1. To do this, i’ needs an efficient way (ie one that doesn’t scale with the size of the predictor) to (on at least some inputs) be highly confident that the intermediate model understands everything relevant about the situation. I think this is a reasonable “worst-case” assumption, but I’m not sure. If you’re using it, I’d be curious to know.
2. Even when the reporter gets inputs that the intermediate model fully understands, it seems like the reporter will run into issues if its sampling-process (for generating distributions) runs into some inputs that the intermediate model doesn’t fully understand. (I.e., if i’ gets an input for which X1 is true, and then simulates many more random inputs for which X1 is true, and by chance X2 is true for one of them, then the reporter will have to do translation on that sample). Which makes it seem unlikely that i’ wouldn’t have to do translation at least once per input.
3. If the model is confident that the intermediate model understands everything relevant about the situation, it seems more efficient to return the intermediate model’s answer to the question at hand than to loop over it many times, trying to fix correlations. So really we should get a reporter i″ that does translation on the predictor on hard questions and returns an intermediate model’s latent knowledge on easy questions. That seems like an ok reporter to get.