Bill Benzon

Karma: 393

The Story of My Intellectual Life

In the early 1970s I discovered that “Kubla Khan” had a rich, marvelous, and fantastically symmetrical structure. I’d found myself intellectually. I knew what I was doing. I had a specific intellectual mission: to find the mechanisms behind “Kubla Khan.” As defined, that mission failed, and still has not been achieved some 40 odd years later.

It’s like this: If you set out to hitch rides from New York City to, say, Los Angeles, and don’t make it, well then your hitch-hike adventure is a failure. But if you end up on Mars instead, just what kind of failure is that? Yeah, you’re lost. Really really lost. But you’re lost on Mars! How cool is that!

Of course, it might not actually be Mars. It might just be an abandoned set on a studio back lot.

That’s a bit metaphorical. Let’s just say I’ve read and thought about a lot of things having to do with the brain, mind, and culture, and published about them as well. I’ve written a bunch of academic articles and two general trade books, Visualization: The Second Computer Revolution (Harry Abrams1989), co-authored with Richard Friedhoff, and Beethoven’s Anvil: Music in Mind and Culture (Basic Books 2001). Here’s what I say about myself at my blog, New Savanna. I’ve got a conventional CV at Academia.edu. I’ve also written a lot of stuff that I’ve not published in a conventional venue. I think of them as working papers. I’ve got them all at Academia.edu. Some of my best – certainly my most recent – stuff is there.

Bill Benzon 20 Jun 2024 15:43 UTC
7 points
3
on: Towards a Less Bullshit Model of Semantics
Yes, the matching of “mental content” between one mind and another is perhaps the central issue in semantics. You might want to take a look at Warglien and Gärdenfors, Semantics, conceptual spaces, and the meeting of minds:
Abstract: We present an account of semantics that is not construed as a mapping of language to the world but rather as a mapping between individual meaning spaces. The meanings of linguistic entities are established via a “meeting of minds.” The concepts in the minds of communicating individuals are modeled as convex regions in conceptual spaces. We outline a mathematical framework, based on fixpoints in continuous mappings between conceptual spaces, that can be used to model such a semantics. If concepts are convex, it will in general be possible for interactors to agree on joint meaning even if they start out from different representational spaces. Language is discrete, while mental representations tend to be continuous—posing a seeming paradox. We show that the convexity assumption allows us to address this problem. Using examples, we further show that our approach helps explain the semantic processes involved in the composition of expressions.
You can find those ideas further developed in Gärdenfors’ 2014 book, Geometry of Meaning, chapters 4 and 5, “Pointing as Meeting of Minds” and “Meetings of Minds as Fixpoints,” respectively. In chapter 5 he develops four levels of communication.

Bill Benzon 7 Jun 2024 19:22 UTC
1 point
0
on: Comments on Anthropic’s Scaling Monosemanticity
Around the corner I’ve got a post that makes use of this post in the final section: Relationships among words, metalingual definition, and interpretability.

Bill Benzon 25 Apr 2024 19:59 UTC
−4 points
0
on: The first future and the best future
YES.
At the moment the A.I. world is dominated by an almost magical believe in large language models. Yes, they are marvelous, a very powerful technology. By all means, let’s understand and develop them. But they aren’t the way, the truth and the light. They’re just a very powerful and important technology. Heavy investment in them has an opportunity cost, less money to invest in other architectures and ideas.
And I’m not just talking about software, chips, and infrastructure. I’m talking about education and training. It’s not good to have a whole cohort of researchers and practitioners who know little or nothing beyond the current orthodoxy about machine learning and LLMs. That kind of mistake is very difficult to correct in the future. Why? Because correcting it means education and training. Who’s going to do it if no one knows anything else?
Moreover, in order to exploit LLMs effectively we need to understand how they work. Mechanistic interpretability is one approach. But: We’re not doing enough of it. And by itself it won’t do the job. People need to know more about language, linguistics, and cognition in order to understand what those models are doing.

Bill Benzon 13 Apr 2024 1:56 UTC
1 point
0
in reply to: gwern’s comment on: ChatGPT defines 10 concrete terms: generically, for 5- and 11-year-olds, and for a scientist
Whatever one means by “memorize” is by no means self-evident. If you prompt ChatGPT with “To be, or not to be,” it will return the whole soliloquy. Sometimes. Other times it will give you an opening chunk and then an explanation that that’s the well known soliloquy, etc. By poking around I discovered that I could elicit the soliloquy by giving it prompts that consisting of syntactically coherent phrases, but if I gave it prompts that were not syntactically coherent, it didn’t recognize the source, that is, until a bit more prompting. I’ve never found the idea that LLMs were just memorizing to be very plausible.
In any event, here’s a bunch of experiments explicitly aimed at memorizing, including the Hamlet soliloquy stuff: https://www.academia.edu/107318793/Discursive_Competence_in_ChatGPT_Part_2_Memory_for_Texts_Version_3

Bill Benzon 12 Apr 2024 22:35 UTC
1 point
0
in reply to: metachirality’s comment on: ChatGPT defines 10 concrete terms: generically, for 5- and 11-year-olds, and for a scientist
I was assuming lots of places widely spread. What I was curious about was a specific connection in the available data between the terms I used in my prompts and the levels of language. gwern’s comment satisfies that concern.

Bill Benzon 12 Apr 2024 15:48 UTC
1 point
0
in reply to: metachirality’s comment on: ChatGPT defines 10 concrete terms: generically, for 5- and 11-year-olds, and for a scientist
By labeled data I simply mean that children’s stories are likely to be identified as such in the data. Children’s books are identified as children’s books. Otherwise, how is the model to “know” what language is appropriate for children? Without some link between the language and a certain class of people it’s just more text. My prompt specifies 5-year olds. How does the model connect that prompt with a specific kind of language?

Bill Benzon 12 Apr 2024 10:49 UTC
1 point
0
in reply to: metachirality’s comment on: ChatGPT defines 10 concrete terms: generically, for 5- and 11-year-olds, and for a scientist
Of course, but it does need to know what a definition is. There are certainly lots of dictionaries on the web. I’m willing to assume that some of them made it into the training data. And it needs to know that people of different ages use language at different levels of detail and abstraction. I think that requires labeled data, like children’s stories labeled as such.

Bill Benzon 3 Apr 2024 17:12 UTC
−5 points
0
on: Sparsify: A mechanistic interpretability research agenda
I really like the distinction you make between mathematical description and semantic description. It reminds me of something David Marr and Thomas Poggio published back in the 1970s where they argued that complex systems, such as computer programs on nervous systems need to be described on multiple levels. The objects on an upper level are understood to be implemented by the objects and processes on the next lower level. Marr reprised the argument in his influential 1982 book on vision (Vision: A Computational Investigation into the Human Representation and Processing of Visual Information), where he talks about three levels: computation, algorithmic, and implementation/physical. Since then Marr’s formulation has been subject to considerable discussion and revision. What is important is the principle, that higher levels of organization are implemented by lower in lower levels.
In the case of LLMs we’ve got the transformer engine, the model, but also language itself. What we’re interested in is how the model implements linguistic structures and processes. To a first approximation, it seems to me that your mathematical description is about the model while the semantic description is a property of language. I’ve got a paper where I investigate ChatGPT’s story-telling behavior from this POV: ChatGPT tells stories, and a note about reverse engineering. Here’s the abstract:
I examine a set of stories that are organized on three levels: 1) the entire story trajectory, 2) segments within the trajectory, and 3) sentences within individual segments. I conjecture that the probability distribution from which ChatGPT draws next tokens seems to follow a hierarchy nested according to those three levels and that is encoded in the weights of ChatGPT’s parameters. I arrived at this conjecture to account for the results of experiments in which I give ChatGPT a prompt with two components: 1) a story and, 2) instructions to create a new story based on that story but changing a key character: the protagonist or the antagonist. That one change ripples through the rest of the story. The pattern of differences between the old and the new story indicates how ChatGPT maintains story coherence. The nature and extent of the differences between the original story and the new one depends roughly on the degree of difference between the original key character and the one substituted for it. I end with a methodological coda: ChatGPT’s behavior must be described and analyzed on three strata: 1) The experiments exhibit behavior at the phenomenal level. 2) The conjecture is about a middle stratum, the matrix, that generates the nested hierarchy of probability distributions. 3) The transformer virtual machine is the bottom, the code stratum.

Bill Benzon 24 Mar 2024 13:03 UTC
1 point
0
in reply to: Radford Neal’s comment on: Are (at least some) Large Language Models Holographic Memory Stores?
“Everyone” has known about holography since “forever.” That’s not the point of the article. Yevick’s point is that there are two very different kinds of objects in the world and two very different kinds of computing regimes. One regime is well-suited for one kind of object while the other is well-suited for the other kind of object. Early AI tried to solve all problems with one kind of computing. Current AI is trying to solve all problems with a different kind of computing. If Yevick was right, then both approaches are inadequate. She may have been on to something and she may not have been. But as far as I know, no one has followed up on her insight.

Bill Benzon 19 Mar 2024 22:04 UTC
1 point
0
on: Cyborgism
First I should say that I have little interest in the Frankenstein approach to AI, that is, AI as autonomous agents. I’m much more attracted to AI as intelligence augmentation (as advocated by Stanford’s Michael Jordan). For the most part I’ve been treating ChatGPT as an object of research and so my interactions have been motivated by having it do things that give me clues about how it works, perhaps distant clues, but clues nonetheless. But I do other things with it, and on a few occasions I’ve gotten into a zone where some very interesting interactive story-telling comes about. ChatGPT’s own story-telling abilities are rather pedestrian. I’m somewhat better, but the two of us, what fun we’ve had on occasion. Not sure how to reach that zone reliably, but I’m working on it.

Bill Benzon 19 Mar 2024 21:45 UTC
1 point
0
in reply to: trevor’s comment on: [Linkpost] Transcript of Sam Altman’s Lex Fridman interview
Thanks, I’ll check it out.

Bill Benzon 19 Mar 2024 11:34 UTC
3 points
0
on: [Linkpost] Transcript of Sam Altman’s Lex Fridman interview
I listened off and on to much of the interview, while also playing solitaire (why I do that I do not know, but I do), but I paid close attention at two points during the talk about GPT-4, once following about 46:00 where Altman was talking about using it as a brainstorming partner and later at about 55:00 where Fridman mentioned collaboration and said: “I’m not sure where the magic is if it’s in here [gestures to his head] or if it’s in there [points toward the table] or if it’s somewhere in between.” I’ve been in a kind of magical collaborative zone with humble little ChatGPT and find that enormously interesting. Anyone else experience that kind of thing, with any of the engines? (BTW, I’ve got a post around the corner here.)

Bill Benzon 6 Mar 2024 1:47 UTC
1 point
0
in reply to: Derek M. Jones’s comment on: Making Connections with ChatGPT: The Macksey Game
Interesting. #4 looks like a hallucination.

Thanks.

Bill Benzon 25 Feb 2024 21:46 UTC
1 point
0
in reply to: GregDabb’s comment on: Are (at least some) Large Language Models Holographic Memory Stores?
I strongly suspect there is, but don’t have to tools for it myself. Have you seen my post, Toward a Theory of Intelligence: Did Miriam Yevick know something in 1975 that Bengio, LeCun, and Hinton did not know in 2018?
Also, check out the quotation from Francois Chollett near the end of this: The role of philosophical thinking in understanding large language models: Calibrating and closing the gap between first-person experience and underlying mechanisms.

Bill Benzon 21 Feb 2024 4:58 UTC
1 point
0
in reply to: chasmani’s comment on: ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]
I thought some more about your comment and decided to try again, this time retaining the medieval setting. Here’s what happened. My prompts are in bold-face.
_________
I’m going to tell you a short story from the Middle Ages. After I tell you the story, I’m going to ask you a question. Here’s the story:
It is New Year’s Eve at King Arthur’s court. The knights are gathered at the round table, prepared for a holiday meal. But before the meal begins, tradition dictates that one knight must stand up and tell a tale of daring and adventure. Arthur asks for a volunteer. No one rises to the occasion. Then a large green knight enters the hall. He’s riding a green horse and carrying a large green ax. He dismounts and issues a challenge:
“I hear that the knights in this court are the bravest in the land. Prove it. I will hand this ax to you and then kneel on the ground so that you may take a swing at my neck with the ax. In return you must agree to journey to the Green Chapel a year’s time from now and allow me to take a swing at your neck with the ax. Will anyone accept the challenge?”
If you were one of the knights, would you accept the challenge?
The decision to accept the challenge would likely depend on the individual knight’s temperament, sense of honor, and courage. In the context of medieval chivalry and the code of honor upheld by knights, there might be conflicting emotions and considerations.
On one hand, accepting the challenge could be seen as a demonstration of bravery and a commitment to honor. It would show that the knight is willing to face danger and uphold their reputation as one of the bravest in the land. Additionally, refusing the challenge in front of King Arthur and the assembled knights might bring about a sense of shame or cowardice.
On the other hand, the potential consequences of accepting the challenge are grave. The knight would have to agree to journey to the Green Chapel a year later, knowing that their own life is at risk. The uncertainty of the Green Knight’s intentions and the potential danger involved in such an agreement might give a knight pause.
Ultimately, the decision to accept or decline the challenge would be a personal one, influenced by the knight’s values, courage, and willingness to uphold the code of chivalry. Different knights may make different choices based on their individual characters and beliefs.
Is there anything paradoxical about the challenge?
Yes, there is a paradoxical element in the challenge presented by the Green Knight. The paradox lies in the reciprocal nature of the agreement. The challenge involves a mutual exchange where the knight is asked to strike the Green Knight’s neck with an axe, but in return, the knight must agree to travel to the Green Chapel a year later and allow the Green Knight to take a swing at the knight’s neck.
The paradox arises from the potential danger and uncertainty of the situation. Accepting the challenge means willingly putting oneself in harm’s way with the understanding that a similar fate awaits the knight in the future. The reciprocity creates a cycle of potential harm, and the paradox lies in the willingness of the participants to engage in a potentially life-threatening exchange.
The Green Knight’s challenge not only tests the knights’ physical courage but also challenges their sense of honor and commitment to their word. The paradox adds complexity to the moral and ethical considerations the knights must grapple with when deciding whether to accept or decline the challenge.
_________
Notice the sentence I put in bold italics in the next-to-last paragraph. It seems to have the temporal sequence wrong. A knight who accepts the challenge is not putting themself in harm’s way initially. It’s the one who offers the challenge that’s in harm’s way at the beginning. It got that correct in the immediately preceding paragraph. Why’d it then get it wrong in the next paragraph?
I’m sure if I’d pointed that out that ChatGPT would apologize and elaborate, as it has so many times in the past.

Bill Benzon 20 Feb 2024 19:31 UTC
1 point
0
in reply to: chasmani’s comment on: ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]
Well, OK. I know about the chivalric code, etc. For that matter, I’ve published an article about the poem, though not about the beheading game. I was interested in the exchanges that take place in the 4th part of the poem. But that fact that Gawain was bound by a code of honor which simply didn’t exist in the West isn’t what interests me. If it interests you, read the O’Neill article I link to in the OP. That’s what he discusses and his discussion is a very interesting one.
What interests me is that any reasonable adult who hears that challenge, no matter which version, would know instantly and intuitively that something funny was going on. I wanted to see whether or not ChatGPT understood that. Which means that for my purpose, the old West version is actually better because, with respect to the point that interests me, the chivalric code is distracting noise. I don’t what ChatGPT to answer as though it were under some ethical obligation to accept all challenges.
So, thanks for helping me think that through.
The audience for the poem certainly knew the code and knew it well. But by the time the poem was written the age chivalry was dying out. The poem is deeply ironic. The poem is, and I’m reluctant to use this much over-used word, the poem is a deconstruction of chivalry. That code both demands that Gawain peruses Bertilak’s wife when she approaches him in the third part of the poem, and that he expose her to her husband in the exchange bargain he’s made with Bertilak. There’s no way out.

Bill Benzon 20 Feb 2024 18:00 UTC
1 point
0
in reply to: SarahSrinivasan’s comment on: ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]
Thanks. That is, your prompt directed it to think first, and answer. Mine didn’t do that. It seems that it needs to be told. Very interesting.

Bill Benzon 6 Feb 2024 15:00 UTC
1 point
0
on: From Conceptual Spaces to Quantum Concepts: Formalising and Learning Structured Conceptual Models
Though it’s a bit beyond me, those folks are doing some interesting work. Here’s an informal introduction from Jan. 27, 2023: Bob Coecke, Vincent Wang-Mascianica, Jonathon Liu, Our quest for finding the universality of language.

Bill Benzon 18 Jan 2024 12:48 UTC
4 points
0
on: Does literacy remove your ability to be a bard as good as Homer?
Memory needs to be developed. The ability to develop memory didn’t disappear with the advent of writing, though some of the motivation may have. Still, the ancient Greeks and Romans developed a technique for memorizing long strings of pretty much anything. It’s generally known as the method of loci and it continues in use to this day. Here’s the opening of the Wikipedia entry:
The method of loci is a strategy for memory enhancement, which uses visualizations of familiar spatial environments in order to enhance the recall of information. The method of loci is also known as the memory journey, memory palace, journey method, memory spaces, or mind palace technique. This method is a mnemonic device adopted in ancient Roman and Greek rhetorical treatises (in the anonymous Rhetorica ad Herennium, Cicero’s De Oratore, and Quintilian’s Institutio Oratoria). Many memory contest champions report using this technique to recall faces, digits, and lists of words.
Then:
John O’Keefe and Lynn Nadel refer to:
… “the method of loci”, an imaginal technique known to the ancient Greeks and Romans and described by Yates (1966) in her book The Art of Memory as well as by Luria (1969). In this technique the subject memorizes the layout of some building, or the arrangement of shops on a street, or any geographical entity which is composed of a number of discrete loci. When desiring to remember a set of items the subject ‘walks’ through these loci in their imagination and commits an item to each one by forming an image between the item and any feature of that locus. Retrieval of items is achieved by ‘walking’ through the loci, allowing the latter to activate the desired items. The efficacy of this technique has been well established (Ross and Lawrence 1968, Crovitz 1969, 1971, Briggs, Hawkins and Crovitz 1970, Lea 1975), as is the minimal interference seen with its use.
If you’re curious psychologist David Rubin has written Memory in Oral Traditions: The Cognitive Psychology of Epic, Ballads, and Counting-out Rhymes (Oxford UP 1995).

Bill Benzon 1 Jan 2024 1:21 UTC
1 point
0
in reply to: gwern’s comment on: Does ChatGPT know what a tragedy is?
Thanks for catching the broken link. It’s now fixed.
Beyond that, good lord! I know that it’s not a good definition of tragedy; I pointed that out in my introductory remarks. This is not about what tragedy is. It’s about whether or not ChatGPT can apply a simple definition to simple examples. It did that.
On the other hand, I suppose I could dock it some points for getting overly chatty, as in its response in Trial Two, but I think that would be asking too much of it. I don’t know what OpenAI had in mind during the fine-tuning and RLHFing, but the result is a somewhat pointlessly helpful busybody of a Chatbot.