ChatGPT is a token-predictor, but it is often able to generate text that contains novel, valid causal and counterfactual reasoning. What it isn’t able to do, at least not yet, is enforce an interaction with the user that guarantees that it will proceed through a desired chain of causal or counterfactual reasoning.
Many humans are inferior to ChatGPT at explicit causal and counterfactual reasoning. But not all of ChatGPT’s failures to perform a desired reasoning task are due to inability—many are due to the fact that at baseline, its goal is to successfully predict the next token. Iff that goal is aligned with accurate reasoning, it will reason accurately.
Judea Pearl frames associative, causal and counterfactual reasoning as “rungs,” and at least in April 2023, thought ChatGPT was still on a purely associative “rung” based on it responding to his question “Is it possible that smoking causes grade increase on the average and, simultaneously, smoking causes grade decrease in every age group?” with a discussion of Simpson’s paradox. The correct interpretation is that ChatGPT assumed Pearl was himself confused and was using causal terminology imprecisely. Simply by appending the phrase “Focus on causal reasoning, not associative reasoning” to Pearl’s original question, GPT-4 not only performs perfect causal reasoning, but also addresses any potential confusion on the prompter’s part about the difference between associative and causal reasoning.
From now on, I will be interpreting all claims about ChatGPT being unable to perform causal or counterfactual reasoning primarily as evidence that the claimant is both bad and lazy at prompt engineering.
My dad, on the other hand, has it figured out. When I showed him at dinner tonight that ChatGPT is able to invent new versions of stories he used to tell me as a kid, he just looked at me, smiled, and said “we’re fucked.”
ChatGPT is a token-predictor, but it is often able to generate text that contains novel, valid causal and counterfactual reasoning. What it isn’t able to do, at least not yet, is enforce an interaction with the user that guarantees that it will proceed through a desired chain of causal or counterfactual reasoning.
Many humans are inferior to ChatGPT at explicit causal and counterfactual reasoning. But not all of ChatGPT’s failures to perform a desired reasoning task are due to inability—many are due to the fact that at baseline, its goal is to successfully predict the next token. Iff that goal is aligned with accurate reasoning, it will reason accurately.
Judea Pearl frames associative, causal and counterfactual reasoning as “rungs,” and at least in April 2023, thought ChatGPT was still on a purely associative “rung” based on it responding to his question “Is it possible that smoking causes grade increase on the average and, simultaneously, smoking causes grade decrease in every age group?” with a discussion of Simpson’s paradox. The correct interpretation is that ChatGPT assumed Pearl was himself confused and was using causal terminology imprecisely. Simply by appending the phrase “Focus on causal reasoning, not associative reasoning” to Pearl’s original question, GPT-4 not only performs perfect causal reasoning, but also addresses any potential confusion on the prompter’s part about the difference between associative and causal reasoning.
From now on, I will be interpreting all claims about ChatGPT being unable to perform causal or counterfactual reasoning primarily as evidence that the claimant is both bad and lazy at prompt engineering.
My dad, on the other hand, has it figured out. When I showed him at dinner tonight that ChatGPT is able to invent new versions of stories he used to tell me as a kid, he just looked at me, smiled, and said “we’re fucked.”