I think a brain architecture/algorithm that would debate about free will would have been adapted for large amounts of social interaction in its daily life. This interaction would use markedly different skills (eg language) from those of more mundane activities. More importantly it would require a different level of modeling to achieve any kind of good results. One brain would have to contain models for complicated human social, kin and friendly relationships, as well as models for individuals’ personalities.
At the center of the mesh of social interactions would be the tightest wad connections. That would be the brain/person, interacting with and modeling all the other members of their tribe/society. However, their brain cannot model itself modeling others with perfect fidelity, and so many simplifications are made even there. These simplifications pile on top of the perceptual differences that a human sees between (itself, other humans) and (everything else). A whole different mental vocabulary arises between descriptions/models of fellow humans and descriptions/models of everything else. Only in humans does it make predictive sense to talk about intent, capability, and inclination, and the wide gap between these kinds of perceived “properties” of fellow socially interacting humans, and the generally much simpler properties seen in inanimate objects and animals, leads the brain to allocate them to widely separated groups of buckets. It is this perceived separation in mental thing-space that leads to the the a free-will boundary being drawn around the cluster of socially interacting humans. When this boundary is objected to, people go their natural arguing ways.
This is just a first attempt, so I think I may have fallen for some of the traps specifically proscribed against in the post. I hope others will attempt to put up their own explanations and maybe even poke some holes in mine :)
I’ll definitely pay attention to further comments on this homework assignment.
Eliezer: “you will not see the known laws of physics violated in any particular.” ← shouldn’t there be a “way” at the end of that?