Gurkenglas comments on Conditioning Predictive Models: Outer alignment via careful conditioning

[Gurkenglas]

don’t generally make real predictions about the future, and instead mostly simulate counterfactual presents instead

I don’t think this slices GPT at the joints. Of course it assumes the future text was peppered into its training data, and therefore secretly written by an author today who may well use “today” to give an aside to the reader.

[Johannes Treutlein]

I am not sure I understand. Are you saying that GPT thinks the text is genuinely from the future (i.e., the distribution that it is modeling contains text from the future), or that it doesn’t think so? The sentence you quote is intended to mean that it does not think the text is genuinely from the future.

[Gurkenglas]

I agree that it doesn’t think the text is from the future. I am nitpicking a technical detail because the text I was line-commenting upon seemed confused. (How vexing to find this thread below the post, in the place for louder discussions!)

Instead of conjecturing that it doesn’t think the text is from the future, you should conjecture that it thinks the text is from the training data, because:

1. The latter implies the former.

2. We have technical reasons to believe the latter.