That doesn’t actually solve the problem. The system could just encode the desired information in the semantics of some unrelated sentences—e.g. talk about pasta to indicate X = 0, or talk about rain to indicate X = 1.
I expected you to bring up the Natural Abstraction Hypothesis here. Wouldn’t the communication between the parties naturally use the same concepts?
Same concepts yes, but that does not necessarily imply that they’re encoded in the same way as humans typically use language.
That doesn’t actually solve the problem. The system could just encode the desired information in the semantics of some unrelated sentences—e.g. talk about pasta to indicate X = 0, or talk about rain to indicate X = 1.
I expected you to bring up the Natural Abstraction Hypothesis here. Wouldn’t the communication between the parties naturally use the same concepts?
Same concepts yes, but that does not necessarily imply that they’re encoded in the same way as humans typically use language.