saw a post from zvi on twitter yesterday, “remember that this corner of the world has a very different outlook than most other corners. if you don’t have random day-to-day ai queries what are you even doing all day?”, in reply to someone asking “who the fuck has random day-to-day ai queries?” (yes, i looked at that on my phone and typed it out, because my twitter account for following tpot people lives only on my phone, and my lw account lives only on a device with a real keyboard, and getting the link from point A to point B would have been even more hassle)
i think the lines between worlds could be defined relative to search. like, in the beginning, there was no search. if you needed information, it was given to you. if it wasn’t given to you, you made do without it. some people still live there.
then there was search, and some of us learned how to use it (others definitely didn’t!) i’ve had to train my parents carefully to search, and my mother has taken to it better than my father. search at its best, with grep within a single local text file, is amnesiac, immaterial and eternal—there is no social calculus around having searched becuase the social implications of “over there on the computer” are unrelated to the contents of the typing and reading going on. i think I speak search with a faint accent, like a kid who moved countries in their early teens.
AI feels to me like it’s at its best when I use it as search on ideas. It also works as a bigger room to layout concept prototypes in, but that’s a higher barrier to entry; it’s harder for me to learn to design and build bigger conceptual things than it is to just use good search as search.
so we have two search modalities, conventional/literal and linguistic. perhaps worth noting, i don’t think modern google is the embodiment of the old-school literal form of search—i think database queries are what that grew up into as it got with the times. Modern google and google-alikes feels like it’s a database trying to be the kind of concept-search that LLMs surpass it at.
it feels to me like the question of which search tech is superior is answered by the context of what you’re searching for. it’s the difference between numbers and letters, the difference between data and information, the difference between intelligence and wisdom—it’s a level-of-abstraction kind of distinction.
thus the question becomes, for those of us who do queries all day, what are we querying? DB queries are for things with right answers, because they can kinda read a smudged copy of the answer key to the test of reality. AI queries are worse at that, but they’re better at a different kind of thing.
There’s also a kind of creativity and engineering where you query reality directly. There are areas where direct-query is a lot better, and other areas where db-query is a lot better… like if i wanna know about how this one item will behave, i should examine it, whereas if i want to know how items of this type should be expected to behave, i should not extrapolate over-much from my examination of a single item. it’s like something not-turning-up-in-search that i recall seeing recently, perhaps a youtube short from chris boden, about the difference between “engineering” and “the knack”—if you have the knack you can overbuild anything, but engineering is the art of cutting away all the excess to build the bare minimum-viable of any given thing. If you’re using the knack, it gets to the point where you ask the materials what they’re gonna do, and they give you a better answer than the books about them might.
but why would people close with AI have that physical experience? Material hobbies are expensive and messy and cost a lot of storage space and waste time you could spend arguing about x-risk, and when material hobbies do arise, computer-people have the means to get idealized standardized raw materials for them from wherever in the world they’re made, rather than having to make do with whatever you can find nearby. Yes, there are exceptions. Yes, you’re exceptional. But most people? Have you ever tried teaching a room full of most-people to tie a new knot? Modern life very rarely calls “us people” to direct physical problem-solving, us who do so well or at least so connectedly online. So of course we’d forget the other way, the get-your-information-straight-from-the-source way, if we veer away from the areas where that way’s better.
There are probably other hobbies where this is relevant. Maybe you like hiking, maybe you sail. You are probably worse at predicting the weather than the forecast on your phone, but you’ve probably met a human who was better. It’s kinda like that, with the kind of queries that lose resolution if you even put them into language. Most people building something real have to go through a quantified level, a CoT corrigibility level, to be adequately assessed and supervised on their thought process. At least, that’s the engineering-degree paradigm. And the engineering-interview one, or at least the fields that larp as engineering, “solve this problem on the whiteboard and show your work”. It’s an implicit prioritization of the kind of work that can be shown.
anyways, that’s all to say that i think there’s a certain kind of smart that’s agnostic of whether you’re expanding your worldview in the externally legible way or the less-legible ones. i think there’s a “kind of guy” who is basically the same “kind”, in the meaningful ways, as the constantly-querying-AI one… but the kind who queries the world instead of the world-model. It probably takes both kinds. What’s “it”? Who knows!
Feels to me like at the moment, the “character layer” handles transforming a “you” into an “I”. I find it conspicuously absent in smaller models running locally, though maybe the absence is just more obvious with CoT than without it.
I’ve also noticed that the “training data” we get as humans is primarily focusing on or contextualized relative to ourselves, whereas the parts you’re referring to as ground layers don’t really have a concept of the LLM as an entity so they tend to assume they’re humans on that level.