My government name is Mack Gallagher. Crocker’s Rules. I am an “underfunded” “alignment” “researcher”. DM me if you’d like to fund my posts, or my project.
I post some of my less-varnished opinions on my personal blog. In the past they went on my Substack.
If you like arguing with me on LessWrong, at present I’m basically free round the clock to continue interesting arguments in my Discord.
Predictably avoiding death at all costs, even the cost of your immortal soul, eternal fealty, etc., is unfortunately a bigger security flaw than a willingness to follow through on implied local kamikaze threats.
If you follow decision-theoretic loss-aversion to its natural conclusion, both of us should be closeted and making a good YouTube grift as Republicans. We’re making less money this way.
The bottlenecking effect of size can be significantly reduced by eg replacing EM links with optical. We can’t do this yet, but something not very far above our intelligence might. It will probably find other ways to start taking apart the bottlenecks that seem static to us, being able to see further than we can. If the software starts out comparable to a human machinist and self-improves, surely your foresight horizons on its exact technical bottlenecks will quickly expire.
None of that excuses the exclusive cultural fixation on big bulky humanoids for tasks where a wheeled spider would obviously be better suited.
[ Re OP, I just can’t visualize converting that many car factories that fast when IIRC Elon had trouble simply convincing manufacturers to make Tesla doors an unconventional shape. Maybe China could make a billion humanoids in 20 years? But by then we have non-humanoid ASI. ]
Whoa, I hadn’t realized you’d originated the term “anti-inductive”! Scott apparently didn’t realize he was copying this from you, and neither I nor anyone else apparently realized this was your single-point-of-origin coinage, either.
Politics isn’t quite a market for mindshare, as there are coercive equilibria at play. But I think it’s another arena that can accurately be called anti-inductive. “This is an adversarial epistemic environment, therefore tune up your level of background assumption that apparent opportunity is not real” has [usefully IMO] become a central reflex of thought for me over the last few years. But to my surprise the phrase “politics is anti-inductive” turns up zero Google results except for this Tweet.
Subjectively assessed value. This judgment cannot be abdicated.
If you can’t generate your parents’ genomes and everything from memory, then yes, you are in a state of uncertainty about who they are, in the same qualitative way you are in a state of uncertainty about who your young children will grow up to be.
Ditto for the isomorphism between your epistemic state w.r.t. never-met grandparents vs your epistemic state w.r.t. not-yet-born children.
It may be helpful to distinguish the subjective future, which contains the outcomes of all not-yet-performed experiments [i.e. all evidence/info not yet known] from the physical future, which is simply a direction in physical time.
We can hardly establish the sense of anthropic reasoning if we can’t establish the sense of counterfactual reasoning.
A root confusion may be whether different pasts could have caused the same present, and hence whether I can have multiple simultaneous possible parents, in an “indexical-uncertainty” sense, in the same way that I can have multiple simultaneous possible future children.
The same standard physics theories that say it’s impossible to be certain about the future, also say it’s impossible to be certain about the past.
Indexical uncertainty about the past may not be true, but you can’t reject it without rejecting standard physics.
And if there is no indexical uncertainty and all counterfactuals are logical counterfactuals and/or in some sense illusory—well, we’re still left uncomfortably aware of our subjective inability to say exactly what the future and past are and why exactly they must be that way.
thus we can update away from the doomsday argument, because we have way more evidence than the doomsday argument assumes.
I agree with this!
“Update away from” does not imply “discard”.
alignment researchers are clearly not in charge of the path we take to AGI
If that’s the case, we’re doomed no matter what we try. So we had better back up and change it.
Don’t springboard by RLing LLMs; you will get early performance gains and alignment will fail. We need to build something big we can understand. We probably need to build something small we can understand first.
I expect any tests to show unambiguously that it’s “not being replaced at all and citations[/mentions] chaotically swirling”. If I understand Evans correctly, these were all random eminent figures he picked, not selected to be falling out of fashion—and they do seem to be a pretty broad sample of the “old prestigious standard names” space.
The stand mixer is a clever analogy; I didn’t previously have experience with the separation thing.
I presume you’ve seen Is Clickbait Destroying Our General Intelligence?, and probably Hanson’s cultural evolution / cultural drift frame. I wonder if you’re familiar with Callard’s Distant Signals paradigm [ transcript available on episode page ], which I think is the most illuminating of the three.
Besides just the cost of ~instantaneous ~omnicast communication dropping to ~zero, I see a role for the fall of the gold standard in all this. See e.g. U.S. per capita energy usage since ~1970, international fertility since ~1970. My theory [ which I really need to make a more legible graphic for ] is that when people don’t “own their money” and have to track the effects of distant inflation-adjusts from the Fed, inflation volatility [ the destructive macroeconomic thing this OP on /r/badeconomics is saying couldn’t possibly be happening due to the fall of the gold standard ] goes way up. Incentives in the market for ideas are ultimately material [yes, virtual status goods influence material wealth, but it also goes the other way around], so the market for materials influences the market for ideas, and vice versa, in a vicious spiral of decline. Is the theory.
[ Look at those same authors with some other mention-counting tool, you mean? ]
Interesting question: why are people quickly becoming less interested in previous standards?
[ Source ]
Computer science fact of the day 2: JavaScript really was mostly-written in just a few weeks by Brendan Eich on contract-commission for Netscape [ to the extent that you consider it as having been “written [as a new programming language]”, and not just specced out as a collective of “funny-looking macros” for Scheme. ] It took Eich several more months to finish the SpiderMonkey parser whose core I still use every time I browse the Internet today. However, e.g. the nonprofit R5RS did not ship a parser/interpreter [ likewise with similar committee projects ], yet it’s not contested that the R5RS committee “wrote a programming language”.
Computer science fact of the day 3: If you do consider JavaScript a Lisp, it’s [as far as I know] the only living Lisp—not just the only popular living Lisp—to implement [ [1] first-class functions, [2] a full-featured base list fold/reduce that is available to the programmer, [3] a single, internally-consistent set of macros/idioms for operating on “typed” data ]. [1] and [2] are absolutely essential if you want to actually use most of the flexibility advantage of “writing in [a] Lisp”; [3] is essential for general usability. MIT Scheme has [1] and [2] [ and [3] to any extent, as far as I can tell ], and seems generally sensible; the major problem is that it is a dead Lisp. It has no living implementations. I have spent more than a year in a Lisp Discord server that seems full of generally competent programmers; MIT Scheme’s death grip on Emacs as an interpretation/development environment seems to make it a no-go even for lots of people who aren’t writing big projects and don’t need, e.g., functional web crawling libraries. This has caused Scheme to fall so out of favor among working programmers that there is no Scheme option on Leetcode, Codewars, or the competitive programming site Codeforces [Codewars has a Common Lisp option, but none of the rest even have a Lisp]. For people who actually want to write production code in something that amounts to a “real Lisp”, JavaScript is, today, far and away their best option. [ The vexing thing being: while functionally it is the best choice, in terms of flexible syntax, it does not offer a large % of the [very real] generally-advertised benefits of using a Lisp! ]
Computer science fact of the day: a true Lisp machine wouldn’t need stack frames.
Boldface vs Quotation—i.e., the interrogating qualia skeptic, vs the defender of qualia as a gnostic absolute—is a common pattern. However, your Camp #1 and Camp #2 - Boldfaces who favor something like Global Workspace Theory, and Quotations who favor something like IIT—don’t exist, in my experience.
Antonio Damasio is a Boldface who favors a body-mind explanation for consciousness which is quasi-neurological, but which is much closer to IIT than Global Workspace Theory in its level of description. Descartes was a Quotation who, unlike Chalmers, didn’t feel like his qualia, once experienced, left anything to be explained. Most philosophers of consciousness who have ever written, in fact, have nothing to do with your proposed Camp #1 vs Camp #2 division, even though most can be placed somewhere along the Boldface-Quotation spectrum, because whether your intuition is more Boldface vs more Quotation is orthogonal to how confused you are about consciousness and how you go about investigating what confuses you. Your framing comes across as an attempt to decrement the credibility of people who advocate Quotation-type intuition by associating them with IIT, framing Boldfaces [your “Camp #1”] as the “sensible people” by comparison.
--
There’s been discussion in these comments—more and less serious—about whether it’s plausible Boldface and Quotation are talking past each other on account of neurological differences. I think this is quite plausible even before we get into the question of whether Boldfaces could truly lack the qualia Quotations are talking about, or not.
In the chapter of Consciousness Explained titled “Multiple Drafts Versus the Cartesian Theater”, Dennett sets out to solve the puzzle of how the brain reconstructs conscious experiences of the same distant object to be simultaneous across sensory modes despite the fact that light travels faster [and is processed slower in the brain] than sound. He considers reconstruction of simultaneity across conscious modes part of what a quale is.
Yet, I have qualia if anyone does, and if someone bounces a basketball 30m away from me, I generally don’t hear it bounce until a fraction of a second after I see it hit the ground. I’ve always been that way. As a grade-school kid, I would ask people about it; they’d say it wasn’t that way for them, and I’d assume it was because they weren’t paying close enough attention. Now I know people differ vastly in how they experience things and it’s almost certainly an unusual neurological quirk.
Second example: it’s commonly attributed to William James that he remarked on the slowness of qualia, relative to more atomic perception: “I see a bear, I run, I am afraid”. This basic motto—the idea that emotion succeeds action—went a long way in James’s philosophy of emotion, if I understand correctly.
I don’t experience emotion [or other things that usually get called “qualia”] delayed relative to reflex action like that at all. Other people also don’t usually report that experience, and I suspect James was pretty strange in that respect.
So even if everyone had qualia, yes, when we try to explain the contents of subjective awareness, we are quite frequently talking past each other because of the Typical Mind Fallacy.
--
Taking that as a given, it’s still important whether some people [e.g., Dennett] actually don’t have qualia. Is that possible?
Some people hear voices. Some people are gay. Some people are autistic. Some people are colorblind. There are still some reflexes left over in our culture from the days before reductionism, when philosophers would say of schizophrenics that they were touched by the divine or demonic, of gays that they were inspired by virtue or corruption [depending on the culture], and of autists that they were being deliberately contrarian, or else ill-affected by some bad air, or something. Of the colorblind people we don’t have such leftover cultural narratives, probably because the differences that don’t get noticed, don’t get talked about. But if we did have one for colorblindness, it’d probably be that they were being performatively obtuse; and if the colorblind people had been able to express their private opinion [which would have been met with resistance], it probably would have been that everyone else was faking seeing the emperor’s robe in impossible colors as a status signal or some such.
“Qualia” seem at least as easy to miss as color. So I find it quite plausible that there’s a neurological difference underlying many of the arguments between Boldfaces and Quotations.
People in these replies have said that they are alienated from qualia, but aren’t alienated from consciousness.
Qualia aren’t particularly real; consciousness [in the sense of ‘sentience’, ‘self-awareness’, or ‘subjective experience’, not mere ‘awakeness’, like a fly might have] is real.
Vistas aren’t particularly real; vision is real.
Set Chalmers aside, and define p-zombies as qualia-less people who act like qualia-less people. I think Dennett and other Boldfaces are such people.
Others in the comments have pointed out that it seems implausible that such “p-zombies” [as so defined] could report on the subjects of their own experience, because anything that is the subject of conscious [in the sense of “sentient” or “subjective”] experience, definitionally becomes a quale.
This, I think, is where the cruxy neurological difficulty lies in the mutually target-missing crosstalk between Boldfaces and Quotations. Quotations, when they get to talking about the philosophy of consciousness [/‘awareness’ / ‘sentience’ / ‘subjective experience’] tend to talk about qualia as an inherently mystical or ineffable or non-reductionistic [ in the sense of violating reductionistic principles ] experience. E.g., the nature of Chalmers’s confusion about qualia [unlike Dennett’s, or Damasio’s], isn’t that they seem elusive or unreliable, but that they seem so obvious as to actively obstruct a properly reductionistic understanding of their internal composition—in a way that Chalmers presumably does not also feel about other classes of percept, like vision, sound, or visceral experience.
The Wikipedia article on qualia has an interesting note in the introduction:
qualia may be considered comparable and analogous to the concepts of jñāna found in Eastern philosophy and traditions.
“Jñāna” doesn’t mean the same thing as the later Buddhist “jhana”—the later Buddhist “jhana” is a target mental state achieved through meditation, while the earlier “Vedic” “jñāna” simply refers to a thing-experienced the way “quale” does. Like “jhana”, though, “jñāna” is a remote cognate with the English “know” and the Greek “gnosis”. And like “quale”, and “gnosis”, “jñāna”—though not having been originally defined with prescriptive content—acquired mystical and spiritual connotations with use, implying that people tended to organically drift toward using that word in a mysterian way. It seems that across cultures, there is a whole class of people that insists on allowing mysticism to diffuse into our references to atomic conscious experience. This is very strange, if you think about it! Atomic conscious experience isn’t mystical.
I think what happens is that people—especially people like me who can sometimes experience deep, but difficult-to-quickly-unpack, holistic percepts with a high subjective level of certitude—use “quale” [or “jñāna” or “gnosis”] to refer to hard-to-describe things we would not know except by directly experiencing them. Some mystics take advantage, and claim you should give money to their church for reasons they can’t describe but are very certain of. So semantic drift associates the word with mysticism. [ Even while some people, like myself and Chalmers, continue using such words honestly, to refer to certain kinds of large high-certitude experiences whose internals frustrate our access. ]
Dennett and other Boldfaces seem [neurologically? I imagine] inclined to approach all situations, especially those involving reflective experiences, with a heavy emotional tone of doubt, which opens up questions and begs resolution.
[ E.g., Dennett writes:
Descartes claimed to doubt everything that could be doubted, but he never doubted that his conscious experiences had qualia, the properties by which he knew or apprehended them.
and this general approach holds in the rest of his writing, as far as I’m aware. ]
So given that people use the word “quale” [ which, again, like the word “vista” or “projection”, refers to a category of thing that, if we’re speaking precisely, isn’t real ] so often to refer to their high-certitude opaque experiences, it makes sense that Dennett and Boldfaces would say that they don’t have such experiences. Their experiences don’t come with that gnostic certitude by which holistic thinkers [like me] sometimes fumblingly excuse our quick illegible conclusions.
--
You say “Camp #1” vs “Camp #2″ has implications for research decisions, because
[T]he work you do to make progress on understanding a phenomenon is different if you expect the phenomenon to be low level vs. high level.
Demanding that consciousness must be explained as existing only on one level in the ontological hierarchy of description is a greater ask than we make of ordinary software documentation. Is the X window server “low level” or “high level”? It needs the assembly code to run, so it must be “low level”, right? But we can only notice when it breaks by noticing whether our windows look good at a high level—whether they display with the right color and font, whether we can make several, whether their contents don’t interfere with each other, etc. So our criteria are high-level—are we looking at a high-level phenomenon?
Like the X window system, operating systems, etc., consciousness inherently straddles layers of abstraction. It exists to facilitate supervenience. Neither Camp #1 nor Camp #2 is off on a very sensible research foot.
[A], just ‘cause I anticipate the ‘More and more’ will turn people off [it sounds like it’s trying to call the direction of the winds rather than just where things are at].
[ Thanks for doing this work, by the way. ]
No, what I’m talking about here has nothing to do with hidden-variable theories. And I still don’t think you understand my position on the EPR argument.
I’m talking about a universe which is classical in the sense of having all parameters be simultaneously determinate without needing hidden variables, but not necessarily classical in the sense of space[/time] always being arbitrarily divisible.
Oh, sorry, I wasn’t clear: I didn’t mean a classical universe in the sense of conforming to Newton’s assumptions about the continuity / indefinite divisibility of space [and time]. I meant a classical universe in the sense of all quasi-tangible parameters simultaneously having a determinate value. I think we could still use the concept of logical independence, under such conditions.
“building blocks of logical independence”? There can still be logical independence in the classical universe, can’t there?
Is this your claim—that quantum indeterminacy “comes from” logical independence? I’m not confused about quantum indeterminacy, but I am confused about in what sense you mean that. Do you mean that there exists a formulation of a principle of logical independence which would hold under all possible physics, which implies quantum indeterminacy? Would this principle still imply quantum indeterminacy in an all-classical universe?
In my experience, the things that make humans intellectually-active-in-the-sense-of-developing-extensive-frameworks-of-ideas are [ while much grander in the realization ] basically isomorphic to the things that make zoo lions [ /parrots/gorillas/elephants/whatever ] express a rich behavioral repertoire that is fun [ other than maybe in some kind of sadistic capacity ] to watch.
Reading the books and correspondence of Darwin, Newton, and Einstein makes it clear that they were all set up to feel, from the time they were 20 or so, like their social-intellectual lives had high stakes—like they had or at least potentially had important friends [ and enemies! ] who were likely to care about their opinions and could influence the future of the world.
Another thing they had in common is that they all felt guaranteed or entitled to an amount of slack or career leisure rate that most of today’s young people would give body parts for, from the time they were very young.
In other words, they expected an affluent society, which was small and cared about them—an environment like the ones humans evolved to think in. They expected an environment in which a human could safely express all their natural instincts.
In my experience, that’s really it. It’s really rare to find that in modernity, and rarer to see it collide by chance with the level of natural awareness-of-opportunity and level of intellectual strategic thinking that means fields change in response to you.
Kaj Solata touches on “maybe people need to expect that they will be repaid for what they do, and not punished” in his answer about feedback and reward, but I don’t think the case he looks at [ the in-retrospect success of a single type of novel, leading to more of that same type of novel ] is quite as large-scale as the case you [ Demski ] are asking about, and think that’s because Westlake was experiencing small, isolated reward that led to cautious, narrow “intellectual productivity”—not the kind of rich, ongoing “reward” that leads to great intellectual productivity.