Mitchell_Porter

Karma: 9,068

Mitchell_Porter May 17, 2025, 1:14 AM
3 points
0
on: Regarding South Africa
This has never happened at OpenAI.
“This prompt (sometimes) makes ChatGPT think about terrorist organisations”

Mitchell_Porter May 16, 2025, 12:51 PM
4 points
0
on: reflecting on criticism
One is not philosophically obliged to regard the nature of reality as ineffable or inescapably uncertain.
Quarks are a good place to explore this point. The human race once had no concept of quarks. Now it does. You say that inevitably, one day, we’ll have some other concept. Maybe we will. But why is that inevitable? Why can’t quarks just turn out to be part of how reality actually is?
You cite Nagarjuna and talk about emptiness, so that gives me some idea of where you are coming from. This is a philosophy which emphasizes the role of concepts in constituting experience, and the role of the mind in constituting concepts, and typically concludes that reality has no essence, no nature that can be affirmed, because all such affirmations involve concepts that are introduced by the mind, rather than being inherent to anything.
This conclusion I think is overreaching. I actually consider direct experience to be the ultimate proof that reality is not just formlessness carved by mind. Consciousness is not just raw being, it is filled with form. My words and concepts may not capture it properly, I may not even notice everything that is implied by what I see or what I am. But I do see that complexity and multiplicity are there in reality—at the very least, they are there in my own consciousness.
Non-attachment to theories and concepts is a good thing if you’re interested in truth, and know that you don’t know the truth. It also has some pragmatic value if reality changes around you and you need to adapt. But in fundamental matters, one does not have to regard every concept and hypothesis that we have, as necessarily temporary. In some of them we may have latched onto the actual objective truth.
P.S. Having criticized the philosophy of emptiness, let me add ironically that just a few hours ago, I investigated a proposal for AI alignment that someone had posted here a few months ago, and found it to be very good—and its model of the mind, a nondual viewpoint is the highest form. So your philosophy may actually put you in a good position to appreciate the nuances of this potentially important work.

Mitchell_Porter May 16, 2025, 12:14 PM
3 points
0
on: It Is Untenable That Near-Future AI Scenario Models Like “AI 2027” Don’t Include Open Source AI
You could say there are two conflicting scenarios here: superintelligent AI taking over the world, and open-source AI taking over daily life. In the works that you mention, superintelligence comes so quickly that AI mostly remains a service offered by a few big companies, and open-source AI is just somewhere in the background. In an extreme opposite scenario, superintelligence might take so long to arrive, that the human race gets completely replaced by human-level AI before superintelligent AI ever exists.
It would be healthy to have all kinds of combinations of these scenarios being explored. For example, you focus a bit on open-source AI as a bioterror risk. I don’t think a supervirus is going to wipe out the human race or even end civilization, because (as Covid experience shows), we are capable of extreme measures in order to contain truly deadly disease. But a supervirus could certainly bring the world to a halt again, and if it was known to have been designed with open-source AI, that would surely have a huge impact on AI’s trajectory. (I suspect that in such a scenario, AI for civilian purposes would suffer, but deep states worldwide would insist on pressing forward, and that there would also be a lobby arguing for AI as a defense against superviruses. Also, it’s very plausible that a supervirus might be designed by AI, but that there would be no proof of it, in which case there wouldn’t even be a backlash.)
Another area where futurology about open-source AI might be good, is in the area of gradual disempowerment and replacement of humanity. We have societies with a division of roles, humans presently fill those roles but AI and robots will be capable of filling more and more of them; eventually every role in the economic, cultural, and political structure could be filled by AIs rather than by humans. The story of how that could happen, certainly deserves to be explored.
Still another area when open-source AI scenarios deserve to be studied, is in the highly concrete realm of near-future economics and culture. What does an AI economy look like if o4-level models are just freely available? This really is an urgent question for anyone concerned with concrete questions like, who will lead the AI industry and how will it be structured, because there seem to be factions in both China and America who are thinking in this direction. One should want to understand what they envision, and what kind of competitive landscape they are likely to create in the short term.
My own belief is that this would be such an upheaval, that it would inevitably end up invalidating many conventional political and economic premises. The current world order of billionaires and venture capitalists, stock markets and human democracies, I just don’t see it surviving such a transition, even without superintelligence appearing. There are just too many explosive possibilities, too many new symbioses of AI with human mind, for the map of the world and the solar system to not be redrawn.
However, in the end I believe in short timelines to superintelligence, and that makes all the above something of a secondary concern, because something is going to emerge that will overshadow humans and human-level AI equally. It’s a little monotonous to keep referring back to Iain Banks’s Culture universe, but it really is the outstanding depiction of a humanly tolerable world in which superintelligence has emerged. His starfaring society is really run by the “Minds”, which are superintelligent AIs characteristically inhabiting giant spaceships or whole artificial worlds, and the societies over which they invisibly preside, include both biological intelligences (such as humans) and human-level AIs (e.g. the drones). The Culture is a highly permissive anarchy which mostly regulates itself via culture, i.e. shared values among human-level intelligences, but it has its own deep state, in the form of special agencies and the Minds behind them, who step in when there’s a crisis that has escaped the Minds’ preemptive strategic foresight.
This is one model of what relations between superintelligence and lesser intelligences might be like. There are others. You could have an outcome in which there are no human-level intelligences at all, just one or more superintelligences. You could have superintelligences that have a far more utilitarian attitude to lower intelligences, creating them for temporary purposes and then retiring them when they are no longer needed. I’m sure there are other possibilities.
The point is that from the perspective of a governing superintelligence, open-source AIs are just another form of lower intelligence, that may be useful or destabilizing depending on circumstance, and I would expect a superintelligence to decide how things should be on this front, and then to make it so, just as it would with every other aspect of the world that it cared about. The period in which open-source AI was governed only by corporate decisions, user communities, and human law would only be transitory.
So if you’re focused on superintelligence, the real question is whether open-source AI matters in the development of superintelligence. I think potentially it does—for example, open source is both a world of resources that Big Tech can tap into, as well as a source of destabilizing advances that Big Tech has to keep up with. But in the end, superintelligence—not just reasoning models, but models that reason and solve problems with strongly superhuman effectiveness—looks like something that is going to emerge in a context that is well-resourced and very focused on algorithmic progress. And by definition, it’s not something that emerges incrementally and gets passed back and forth and perfected by the work of many independent hands. At best, that would describe a precursor of superintelligence.
Superintelligence is necessarily based on some kind of incredibly powerful algorithm or architecture, that gets maximum leverage out of minimum information, and bootstraps its way to overwhelming advantage in all domains at high speed. To me, that doesn’t sound like something invented by hobbyists or tinkerers or user communities. It’s something that is created by highly focused teams of genius, using the most advanced tools, who are also a bit lucky in their initial assumptions and strategies. That is something you’re going to find in an AI think tank, or a startup like Ilya Sutskever’s, or a rich Big Tech company that has set aside serious resources for the creation of superintelligence.
I recently posted that superintelligence is likely to emerge from the work of an “AI hive mind” or “research swarm” of reasoning models. Those could be open-source models, or they could be proprietary. What matters is that the human administrators of the research swarm (and ultimately, the AIs in the swarm itself) have access to their source code and their own specs and weights, so that they can engaged in informed self-modification. From a perspective that cares most about superintelligence, this is the main application of open source that matters.

Mitchell_Porter May 16, 2025, 10:25 AM
3 points
0
on: PRISM: Perspective Reasoning for Integrated Synthesis and Mediation (Interactive Demo)
Somehow this has escaped comment, so I’ll have a go. I write from the perspective of whether it’s suitable as the value system of a superintelligence. If PRISM became the ethical operating system of a posthuman civilization born on Earth, for as long as that civilization managed to survive in the cosmos—would that be a satisfactory outcome?
My immediate thoughts are: It has a robustness, due to its multi-perspective design, that gives it some plausibility. At the same time, it’s not clear to me where the seven basis worldviews come from. Why those seven, and no others? Is there some argument that these seven form a necessary and sufficient basis for ethical behavior by human-like beings and their descendants?
If I dig a little deeper into the paper, the justification is actually in part 2. Specifically, on page 12, six brain regions and their functions are singled out, as contributing to human decision-making at increasingly abstract levels (for the hierarchy, see page 15). The seven basis worldviews correspond to increasing levels of mastery of this hierarchy.
I have to say I’m impressed. I figured that the choice of worldviews would just be a product of the author’s intuition, but they are actually grounded in a theory of the brain. One of the old dreams associated with CEV, was that the decision procedure for a human-friendly AI would be extrapolated in a principled way from biological facts about human cognition, rather than just from a philosophical system, hallowed tradition, or set of community principles. June Ku’s MetaEthical AI, for example, is an attempt to define an algorithm for doing this. Well, this is a paper written by a human being, but the principles in part 2 are sufficiently specific, that one could actually imagine an automated process following them, and producing a form of PRISM as its candidate for CEV! I’d like @Steven Byrnes to have a look at this.

Mitchell_Porter May 15, 2025, 2:13 PM
8 points
0
in reply to: dirk’s comment on: dirk’s Shortform
We had Golden Gate Claude, now we have White Genocide Grok…

Mitchell_Porter May 15, 2025, 9:52 AM
2 points
0
on: Absolute Zero: Alpha Zero for LLM
This seems like a Chinese model for superintelligence! (All the authors are Chinese, though a few are working in the West.) Not in the AIXI sense of something which is optimal from the beginning, but rather something that could bootstrap its way to superintelligence. One could compare it to Schmidhuber’s Godel machine concept, but more concrete, and native to the deep learning era.
(If anyone has an argument as to why this isn’t a model that can become arbitrarily intelligent, I’m interested.)

Mitchell_Porter May 15, 2025, 5:15 AM
12 points
3
on: Why OpenAI projects only $174B of revenue by 2030?
If I have understood correctly, you’re saying that OpenAI should be forecasting greater revenue than this, if they truly think they will have AIs capable of replacing entire industries. But maybe they’re just being cautious in their forecasts?
Suppose I have a 3d printing / nanotechnology company, and I think that a year from now I’ll have an unlimited supply of infinity boxes capable of making any material artefact. World manufacturing is worth over US$10 trillion. If I thought I could put it all out of business, by charging just 10% of what current manufacturers charge, I could claim expected revenue of $1 trillion.
Such a prediction would certainly be attention-grabbing, but maybe it would be reckless to make it? Maybe my technology won’t be ready. Maybe my products will be blocked from most markets. Maybe someone will reverse-engineer and open-source the infinity boxes, and prices will crash to $0. Maybe I don’t want the competition or the government to grasp just how big my plans are. Maybe the investors I want wouldn’t believe such a scenario. There are a lot of reasons why a company that thinks it might be able to take over the economy or even the world, would nonetheless not put that in its prospectus.

Mitchell_Porter May 15, 2025, 4:56 AM
8 points
2
on: Mitchell_Porter’s Shortform
OpenAI gets a lot of critical attention here, because it’s been the leader in many ways. But what about Google AI? Many people think it’s the leader now, yet I don’t see anywhere near as much critical scrutiny of its decision-making process.

Mitchell_Porter May 14, 2025, 1:00 AM
4 points
0
on: October The First Is Too Late
That gives “Eternal September” a new meaning…

Mitchell_Porter May 13, 2025, 10:59 AM
11 points
0
in reply to: Baram Sosis’s comment on: Book Review: “Encounters with Einstein” by Heisenberg
The view that Heisenberg advocates—reductionism had reached a limit, and a new paradigm was needed—was a highly influential school of thought in the 1960s. In particle physics, there is a mathematical object called the S-matrix (scattering matrix), which tabulates scattering amplitudes (the quasiprobability that if these N particles enter a collision, these other M particles will be what comes out). Quantum electrodynamics (a theory with electrons and photons, let’s say) is a prototypical quantum field theory in which the S-matrix can be calculated from the stipulation that electrons and photons are fundamental. For the weak interactions (later unified with electromagnetism), this reductionist method also works.
But for the strong interactions, field theory looked intractable, and a new philosophy was advanced that the S-matrix itself should be the central mathematical object in the theory. Remember that quarks were never seen by themselves, only protons, neutrons, and a hundred other types of “hadron”. The idea of nuclear democracy was that the S-matrix for these hundred seemingly equi-fundamental particle species, would be derived from postulates about the properties of the S-matrix, rather than from an underlying field theory. This was called the bootstrap program, it is how the basic formulae of string theory were discovered (before they had even been identified as arising from strings), and it’s still used to study the S-matrix of computationally intractable theories.
These days, the philosophy that the S-matrix is primary, still has some credibility in quantum gravity. Here the problem is not that we can’t identify ultimate constituents, but rather that the very idea of points of space-time seems problematic, because of quantum fluctuations in the metric. The counterpart of the 1960s skepticism about quarks, would be that the holographic boundary of space-time is fundamental. For example, in the AdS/CFT correspondence, scattering events in Anti de Sitter space (in which particles approach each other “from the boundary”, interact, and then head back to the boundary) can be calculated entirely within the boundary CFT, without any reference to AdS space at all, which is regarded as emergent from the boundary space. The research program of celestial holography is an attempt to develop the same perspective within the physically relevant case of flat space-time. The whole universe that we see, would be a hologram built nonlocally from entanglement within a lower-dimensional space…
The eventual validation of quarks as particles might seem like a sign that this radical version of the holographic philosophy will also be wrong in the end, and perhaps it will be. But it really shows the extent to which the late thoughts of Heisenberg are still relevant. Holographic boundaries are the new S-matrix, they are a construct which has made quantum gravity uniquely tractable, and it’s reasonable to ask if they should be treated as fundamental, just as it was indeed entirely reasonable for Heisenberg and the other S-matrix theorists to ask whether the S-matrix itself is the final word.

Mitchell_Porter May 13, 2025, 8:23 AM
2 points
0
in reply to: RogerDearnaley’s comment on: What if we just…didn’t build AGI? An Argument Against Inevitability
Ugh, I was using LW’s custom reaction emoticons to annotate this comment, and through a fumble, have ended up expressing a confidence of 75% in the scenario that AI will “otherwise enforce on us a pause”, and I don’t see how to remove that annotation.
I will say that, alignment aside, the idea that an advanced AI will try to halt humanity’s AI research so it doesn’t produce a rival, makes a lot of sense to me.

Mitchell_Porter May 13, 2025, 8:17 AM
3 points
0
on: A Live Look at the Senate AI Hearing
Ted Cruz mentioned how his daughter was using ChatGPT when texting him. I wonder how many of these senators and CEOs, and their staffers and advisors, are already doing the same, when they try to decide AI policy. I guess that would be an example of weak-to-strong superalignment :-)

Mitchell_Porter May 11, 2025, 1:37 PM
26 points
0
on: Book Review: “Encounters with Einstein” by Heisenberg
Heisenberg versus quarks is one of the best lifelong physics thinkers, encountering one of the most subtle topics in physics. When he says
The questions about the statistics of quarks, about the forces that hold them together, about the particles corresponding to these forces, about the reasons why quarks never appear as free particles, about the pair-creation of quarks in the interior of the elementary particle—all these questions are more or less left in obscurity.
… he is raising all the right questions, and their resolution required a field theory with completely unprecedented behaviors. The statistics of quarks required a new threefold quantum property, “color”; the forces that hold them together were described by a new kind of quantum field, a strongly coupled Yang-Mills field; the particles corresponding to those forces are the gluons, and like quarks, they never appear as free particles, because of a new behavior, “confinement”.
The deeper story here is a struggle of paradigms in the 1960s, between the old search for the most elementary particles, and the idea of a “nuclear democracy” of transformations among a large number of equally fundamental particle species. We now see those many particle types as due to different combinations of quarks, but at the time no-one understood how quarks could literally be particles (for the reasons touched on by Heisenberg), and they were instead treated as bookkeeping devices, akin to conserved quantities.
The quark theorists won, once the gauge theory of gluons was figured out; but nuclear democracy (in the guise of “S-matrix theory”, pioneered by Heisenberg) left its mark too, because it gave rise to string theory. They aren’t even completely distinct paradigms; there are very close relationships between gauge theory and string theory, though not close enough that we know exactly what string theory corresponds to the quarks and gluons of the standard model.

Mitchell_Porter May 10, 2025, 2:10 AM
2 points
0
in reply to: Steven Byrnes’s comment on: steve2152′s Shortform
Incidentally, is there any meaningful sense in which we can say how many “person-years of thought” LLMs have already done?
We know they can do things in seconds that would take a human minutes. Does that mean those real-time seconds count as “human-minutes” of thought? Etc.

Mitchell_Porter May 9, 2025, 9:17 AM
4 points
0
on: Jim Babcock’s Mainline Doom Scenario: Human-Level AI Can’t Control Its Successor
Liron: … Turns out the answer to the symbol grounding problem is like you have a couple high dimensional vectors and their cosine similarity or whatever is the nature of meaning.
Could someone state this more clearly?
Jim: … a paper that looked at the values in one of the LLMs as inferred from prompts setting up things like trolley problems, and found first of all, that they did look like a utility function, second of all, that they got closer to following the VNM axioms as the network got bigger. And third of all, that the utility function that they seemed to represent was absolutely bonkers
What paper was this?

Mitchell_Porter May 6, 2025, 12:20 AM
3 points
0
on: The First Law of Conscious Agency: Linguistic Relativity and the Birth of “I”
scientifically falsifiable
How is it falsifiable?

Mitchell_Porter May 4, 2025, 10:08 AM
9 points
−5
on: GTFO of the Social Internet Before you Can’t: The Miro & Yindi Story
I think it’s very good to have people around who are saying “cut back on social media”, “get off social media”, as a counter to its addictive power.
And yet… If I take the title literally, I am being told that I should quit social media entirely, as soon as possible, because in the near future, it will be so addictive that I will be literally unable to quit.
When you first raised this idea, I asked what will happen to people who don’t get out in time? In this post, we now have a concrete scenario. The protagonist doesn’t die. They don’t go mad. They don’t become anyone’s minion. They just… spend a lot of time irritated, spend all day watching videos, and lose touch with some real people.
Well, that’s nobody’s ideal, but it’s not actually worse than the human condition has been, for large numbers of people throughout history. “Lives of quiet desperation” I think have been pretty common in the agricultural and industrial eras. In terms of historical experience, it is actually normal for people to end up limping through life with some affliction that they never quite get over, whether it’s injury, trauma, some familial or national destiny that they just can’t escape… To learn that, in the information age, some people become unwholesome computer or media addicts, is just to write the next chapter of that.
Let me be clear, I’m not quite urging apathy about social media addiction. It’s just that I was expecting something more apocalyptic as the payoff, that humanity would be utter captives of the content farms, perhaps later to be herded into Matrix pods or assembled into armies of meme-controlled zombies. Instead, what you’re describing is more like a chronic misery specific to the information age.
It’s like someone warning that if you abandon hunting and gathering, you’ll end up standing around all day watching farm animals, or if you leave the farm for the big industrial city, you’ll end up stooped and maimed by factory work. All that actually happened, but there were also huge upsides to the new order in each case.
After all, there’s actually a lot of good stuff that comes through social media. With a keyword search, I can find up-to-the-second information and perspectives, on something that is happening, including censored perspectives. I can follow the news about topics that interest only a very niche audience. I can eavesdrop on, and even participate in, all kinds of discussions that would otherwise be out of my reach. I can track down lost friends, find work, simply indulge my curiosity.
Of course there are formidable downsides too. You can overindulge (I have a weakness for reaction videos), you can burn out certain faculties, you can forget your own life amidst a million distractions, and just as in real life, there are far worse things lying in wait: scammers, grifters, toxic personalities and communities, political botnets; and perhaps there are even architects of addiction who deserve death as much as any fentanyl dealer does.
It’s just that you haven’t really made the case, that the social Internet will become nothing but a prison of blighted lives. Life in that space is much more like living in a city. There are risks and evils specific to urban life, and city dwellers must learn to avoid them, and lots of people fall prey to them. But there are also good things that can only happen in cities, and there are new good things that only happen online.
I haven’t mentioned the AI factor so far, but it is central to your scenario. My response again is that there are positives and negatives, and in some cases it may not even be clear which is which, The combination of AI and social media may lead to new symbioses that look horrifying to outsiders, but which have a merit and integrity of their own. As AI becomes more and more capable, the question of AI on social media just blends into the broader question of humanity’s destiny in a world with AI, and ultimately, a world with artificial superintelligence.

Mitchell_Porter May 2, 2025, 4:19 AM
2 points
1
in reply to: Wei Dai’s comment on: Our Reality: A Simulation Run by a Paperclip Maximizer
How often will a civilization with the capability to perform such a simulation, have anything to learn from it?

Mitchell_Porter Apr 30, 2025, 9:54 PM
10 points
4
in reply to: Thomas Larsen’s comment on: Early Chinese Language Media Coverage of the AI 2027 Report: A Qualitative Analysis
Yes, thanks. And someone should do the same analysis, regarding coverage of AI 2027 in American/Western media. (edit: A quick survey by o3)

Mitchell_Porter Apr 30, 2025, 8:20 AM
4 points
2
in reply to: Wei Dai’s comment on: Wei Dai’s Shortform
spreading the idea of “heroic responsibility” seems, well, irresponsible
Is this analogous to saying “capabilities research is dangerous and should not be pursued”, but for the human psyche rather than for AI?