> At first blush this is vulnerable to the same counterexample described in the last section [complexity]… But the situation is a little bit more complex… the direct translator may be able to effectively “re-use” that inference rather than starting from scratch
It seems to me that this “counter-counterexample” also applies for complexity – if the translator is able to reuse computation from the predictor, wouldn’t that both reduce the complexity and the time?
(You don’t explicitly state that this “reuse” is only helpful for time, so maybe you agree it is also helpful for complexity – just trying to be sure I understand the argument.)
It seems to me that this “counter-counterexample” also applies for complexity – if the translator is able to reuse computation from the predictor, wouldn’t that both reduce the complexity and the time?
The difference is that the complexity savings are bounded by a constant that doesn’t depend on the size of the predictor’s model (namely the complexity of the human’s model). But in the case of computation time, the savings can be as large as “compute used to do inference in the human’s model.” And that scales up with the size of our computers.
In “Strategy: penalize computation time” you say:
> At first blush this is vulnerable to the same counterexample described in the last section [complexity]… But the situation is a little bit more complex… the direct translator may be able to effectively “re-use” that inference rather than starting from scratch
It seems to me that this “counter-counterexample” also applies for complexity – if the translator is able to reuse computation from the predictor, wouldn’t that both reduce the complexity and the time?
(You don’t explicitly state that this “reuse” is only helpful for time, so maybe you agree it is also helpful for complexity – just trying to be sure I understand the argument.)
The difference is that the complexity savings are bounded by a constant that doesn’t depend on the size of the predictor’s model (namely the complexity of the human’s model). But in the case of computation time, the savings can be as large as “compute used to do inference in the human’s model.” And that scales up with the size of our computers.