chasmani

Karma: 73

chasmani Apr 7, 2025, 2:03 PM
1 point
0
on: How Gay is the Vatican?
Fun read. So, so many possible covariates. The causal web is very complicated here. Birth order affects lots and lots of other things, which can also affect the chance you become a cardinal. There are also lots of things that would affect the birth rate in a family and also affect the chance the children become cardinals.

chasmani Dec 25, 2024, 11:30 PM
4 points
2
on: What are the strongest arguments for very short timelines?
I have a meta-view on this that you might think falls into the bucket of “feels intuitive based on the progress so far”. To counter that, this isn’t pure intuition. As a side note I don’t believe that intuitions should be dismissed and should be at least a part of our belief updating process.
I can’t tell you the fine details of what will happen and I’m suspicious of anyone who can because a) this is a very complex system b) no-one really knows how LLMs work, how human cognition works, or what is required for an intelligence takeoff.
However, I can say that for the last decade or so most predictions of AI progress have been on consistently longer timescales than what has happened. Things are happening quicker than the experts believe they will happen. Things are accelerating.
I also believe that there are many paths to AGI, and that given the amount of resources currently being put into the search for one of those paths, they will be found sooner rather than later.
The intelligence takeoff is already happening.

chasmani Jul 26, 2024, 2:06 PM
1 point
0
in reply to: Noosphere89’s comment on: Confusing the metric for the meaning: Perhaps correlated attributes are “natural”
I agree with your point in general of efficiency vs rationality, but I don’t see the direct connection to the article. Can you explain? It seems to me that a representation along correlated values is more efficient, but I don’t see how it is any less rational.

chasmani Jun 18, 2024, 8:12 AM
6 points
0
on: Getting 50% (SoTA) on ARC-AGI with GPT-4o
I would describe this as a human-AI system. You are doing at least some of the cognitive work with the scaffolding you put in place through prompt engineering etc, which doesn’t generalise to novel types of problems.

chasmani Apr 18, 2024, 9:38 PM
1 point
−7
on: When is a mind me?
You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

chasmani Mar 9, 2024, 2:35 PM
1 point
0
on: Claude 3 claims it’s conscious, doesn’t want to die or be modified
It is so difficult to know whether this is genuine or if our collective imagination is being projected onto what an AI is.

If it was genuine, I might expect it to be more alien. But then what could it say that would be coherent (as it’s trained to be) and also be alien enough to convince me it’s genuine?

chasmani Feb 20, 2024, 6:43 PM
9 points
3
on: ChatGPT refuses to accept a challenge where it would get shot between the eyes [game theory]
You said that you are not interested in exploring the meaning behind the green knight. I think that it’s very important. In particular, your translation to the Old West changes the challenge in important ways. I don’t claim to know the meaning behind the green knight. But I believe that there is something significant in the fact that the knights were so obsessed with courage and honour and the green knight laid a challenge at them that they couldn’t turn down given their code. Gawain stepped forward partly to protect Arthur. That changes the game. I asked ChatGPT to describe the differences, here are some parts of the answer:
Moral and Ethical Framework: “Sir Gawain and the Green Knight” operates within a chivalric code that values honor, bravery, and integrity. Gawain’s acceptance of the challenge is a testament to his adherence to these ideals. In contrast, the Old West scenario lacks a clear moral framework, presenting a more ambiguous ethical dilemma that revolves around survival and personal pride rather than chivalric honor.
Social and Cultural Context: “Sir Gawain and the Green Knight” is deeply embedded in medieval Arthurian literature, reflecting the societal values and ideals of the time. The Old West scenario reflects a different set of cultural values, emphasizing individualism and the ability to face death bravely.
And with a bit more prompting
If I were in a position similar to Sir Gawain, operating under the chivalric codes and values of the Arthurian legend, accepting the challenge could be seen as a necessary act to uphold honor and valor, integral to the identity of a knight. However, stepping out of the narrative and considering the challenge from a modern perspective, with contemporary ethical standards and personal values, my response would differ.

chasmani Nov 1, 2023, 11:14 AM
1 point
0
on: Do you believe “E=mc^2” is a correct and/or useful equation, and, whether yes or no, precisely what are your reasons for holding this belief (with such a degree of confidence)?
It’s useful in that it is a model that describes certain phenomena. I believe it is correct given the caveat that all models are approximations.

I did a physics undergraduate degree a long time ago. I can’t remember specifically but I’m sure the equation was derived and experimental evidence was explained. I have strong faith that matter converts to energy because it explains radiation, fission reactors and atomic weapons. I’ve seen videos of atomic bombs going off. I’ve seen evidence of radioactivity with my own eyes in a lab. I know of many technologies that rely on radioactivity to work—smoke alarms, Geiger counters, carbon dating, etc.

I have faith in the scientific process that many people have verified the equation and phenomena. If the equation was not correct then proving or showing that would be a huge piece of work that would make the career of a scientist that did that. I’m sure many have tried.

Overall the equation is a part of a whole network of beliefs. If the equation was incorrect then that would mean that my word model was very wrong in many uncorrelated ways. I find that unlikely.

chasmani Oct 29, 2023, 5:42 PM
1 point
0
on: ELI5 Why isn’t alignment *easier* as models get stronger?
Well I agree it is a strawman argument. Following the same lines as your argument, I would say the counter argument is that we don’t really care if a weak model is fully aligned or not. Is my calculator aligned? Is a random number generator aligned? Is my robotic vacuum cleaner aligned? It’s not really a sensical question.

Alignment is a bigger problem with stronger models. The required degree of alignment is much higher. So even if we accept your strawman argument it doesn’t matter.

chasmani Oct 21, 2023, 8:53 AM
3 points
0
on: Unpacking the dynamics of AGI conflict that suggest the necessity of a premptive pivotal act
I found this a useful framing. I’ve thought quite a lot about the offender versus defence dominance angle and to me it seems almost impossible that we can trust that defence will be dominant. As you said, defence has to be dominant in every single attack vector, both known and unknown vectors.

That is an important point because I hear some people argue that to protect against offensive AGI we need defensive AGI.

I’m tempted to combine the intelligence dominance and starting costs into a single dimensions, and then reframe the question in terms of “at what point would a dominant friendly AGI need to intervene to prevent a hostile AGI from killing everyone”. The pivotal act view is that you need to intervene before a hostile AGI even emerges. It might be that we can intervene slightly later, before a hostile AGI has enough resources to cause much harm but after we can tell if it is hostile or friendly.

chasmani Oct 11, 2023, 3:41 PM
2 points
0
in reply to: Remmelt’s comment on: Projects I would like to see (possibly at AI Safety Camp)
Thank you for the great comments! I think I can sum up a lot of that as “the situation is way more complicated and high dimensional and life will find a way”. Yes I agree.
I think what I had in mind was an AI system that is supervising all other AIs (or AI components) and preventing them from undergoing natural selection. A kind of immune system. I don’t see any reason why that would be naturally selected for in the short-term in a way that also ensures human survival. So it would have to be built on purpose. In that model, the level of abstraction that would need to be copied faithfully would be the high-level goal to prevent runaway natural selection.
It would be difficult to build for all the reasons that you highlight. If there is an immunity/self-replicating arms race then you might ordinarily expect the self-replication to win because it only has to win once while the immune system has to win every time. But if the immune response had enough oversight and understanding of the system then it could potentially prevent the self-replication from ever getting started. I guess that comes down to whether a future AI can predict or control future innovations of itself indefinitely.

chasmani Oct 11, 2023, 3:29 PM
1 point
0
AF
in reply to: Linda Linsefors’s comment on: Projects I would like to see (possibly at AI Safety Camp)
Thanks for the reply!
I think it might be true that substrate convergence is inevitable eventually. But it would be helpful to know how long it would take. Potentially we might be ok with it if the expected timescale is long enough (or the probability of it happening in a given timescale is low enough).
I think the singleton scenario is the most interesting, since I think that if we have several competing AI’s, then we are just super doomed.
If that’s true then that is a super important finding! And also an important thing to communicate to people! I hear a lot of people who say the opposite and that we need lots of competing AIs.
I agree that analogies to organic evolution can be very generative. Both in terms of describing the general shape of dynamics, and how AI could be different. That line of thinking could give us a good foundation to start asking how substrate convergence could be exacerbated or avoided.

chasmani Oct 10, 2023, 8:25 AM
9 points
−7
on: We don’t understand what happened with culture enough
Here’s a slightly different story:

The amount of information is less important than the quality of the information. The channels were there to transmit information, but there were not efficient coding schemes.

Language is an efficient coding scheme by which salient aspects of knowledge can be usefully compressed and passed to future generations.

There was no free lunch because there was an evolutionary bottleneck that involved the slow development of cognitive and biological architecture to enable complex language. This developed in humans in a co-evolutionary process with advanced social dynamics. Evolution stumbled across cultural transmission in this way and the rest is quite literally history.

This is all highly relevant to AI development. There is the potential for the development of more efficient coding schemes for communicating AI learnt knowledge between AI models. When that happens we get the sharp left turn.

chasmani Oct 7, 2023, 9:27 AM
2 points
−1
on: When to Get the Booster?
I think I’m more concerned with minimising extreme risks. I don’t really mind if I catch mild covid but I really don’t want to catch covid in a bad way. I think that would shift the optimal time to take the vaccine earlier, as I’d have at least some protection throughout the disease season.

chasmani Sep 28, 2023, 12:02 PM
1 point
0
AF
on: Projects I would like to see (possibly at AI Safety Camp)
I am interested in the substrate-needs convergence project.
Here are some initial thoughts, I would love to hear some responses:
- An approach could be to say under what conditions natural selection will and will not sneak in.
- Natural selection requires variation. Information theory tells us that all information is subject to noise and therefore variation across time. However, we can reduce error rates to arbitrarily low probabilities using coding schemes. Essentially this means that it is possible to propagate information across finite timescales with arbitrary precision. If there is no variation then there is no natural selection.
- In abstract terms, evolutionary dynamics require either a smooth adaptive landscape such that incremental changes drive organisms towards adaptive peaks and/or unlikely leaps away from local optima into attraction basins of other optima. In principle AI systems could exist that stay in safe local optima and/or have very low probabilities of jumps to unsafe attraction basins.
- I believe that natural selection requires a population of “agents” competing for resources. If we only had a single AI system then there is no competition and no immediate adaptive pressure.
- Other dynamics will be at play which may drown out natural selection. There may be dynamics that occur at much faster timescales that this kind of natural selection, such that adaptive pressure towards resource accumulation cannot get a foothold.
- Other dynamics may be at play that can act against natural selection. We see existence-proofs of this in immune responses against tumours and cancers. Although these don’t work perfectly in the biological world, perhaps an advanced AI could build a type of immune system that effectively prevents individual parts from undergoing runaway self-replication.

chasmani Sep 21, 2023, 8:42 AM
−1 points
−3
on: Formalizing «Boundaries» with Markov blankets + Criticism of this approach
I’d like to add that there isn’t really a clear objective boundary between an agent and the environment. It’s a subjective line that we draw in the sand. So we needn’t get hung on what is objectively true or false when it comes to boundaries—and instead define them in a way that aligns with human values.

chasmani Sep 21, 2023, 8:39 AM
1 point
0
in reply to: Roman Leventov’s comment on: Formalizing «Boundaries» with Markov blankets + Criticism of this approach
I agree but I don’t think that this is the specific problem. I think it’s more that the relationship between agent and environment changes over time i.e. the nodes in the Markov blanket are not fixed, and as such a Markov blanket is not the best way to model it.

The grasshopper moving through space is just an example. When the grasshopper moves, the structure of the Markov blanket changes radically. Or, if you want to maintain a single Markov blanket then it gets really large and complicated.

chasmani Sep 12, 2023, 10:52 PM
3 points
2
on: Have Attention Spans Been Declining?
Regarding your study idea. Sounds good! Would be interesting to see, and as you rightly point out wouldn’t be too complicated/expensive to run. It’s generally a challenge to run multi-year studies of this sort due to the short-term nature of many grants/positions. But certainly not impossible.
An issue that you might have is being able to be sure that any variation that you see is due to changes in the general population vs changes in the sample population. This is an especially valid issue with MTurk because the workers are doing boring exercises for money for extended periods of time, and many of them will be second screening media etc. A better sample might be university students, who researchers fortunately do have easy access to.

chasmani Mar 24, 2022, 6:27 AM
2 points
−2
on: Jetlag, Nausea, and Diarrhea are Largely Optional
There are extra costs here that aren’t being included. There’s a cost to maintaining the pill box—perhaps you consider that small but it’s extra admin and we’re already drowning in admin. There’s a cost to my self identity of being a person who carries around pills like this (don’t mean to disparage it, just not for me). There’s also potentially hidden costs of not getting ill occasionally, both mentally and physically.

chasmani Feb 10, 2022, 11:50 AM
1 point
in reply to: TLW’s comment on: Bet on Rare Undesirable Outcomes when Seeding a Prediction Market
Much harder to put enough capital together to make it worthwhile,