Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)
Gunnar_Zarncke
Karma: 9,558
I have never heard of the rock/wave communication strategy and can’t seem to google it.
these are pretty standard communications tactics in the modern era.
Is this just unusual naming? Anybody have links?
It is not a biological argument. Well, maybe it was, but you can also compare what aging does to corporations. Companies also age, even though their substrate doesn’t. They get slower and more conservative. They have optimized harder in the past and have more to lose due to change. I bet these effects also happen to people and not for biological reasons (though these may make it worse).
Yeah, but how do we set this up as a stable environment? We can maybe create the initial population, but later, all Johns have to maintain it, otherwise it will fall apart.
We have to make resources scarce enough and difficult to extract, such that John has to collaborate to get them. Then, resources have to be somewhat unpredictable, such that John has to explore and share information. With competition, more collaborative groups will win.
Nice.
How do you get norm-following with limited data? That seems like quite a hard problem.
Just make it in John’s self-interest.
When discussing the GPT-4o model, my son (20) said that it leads to a higher bandwidth of communication with LLMs and he said: “a symbiosis.” We discussed that there are further stages than this, like Neuralink. I think there is a small chance that this (a close interaction between a human and a model) can be extended in such a way that it gets aligned in a way a human is internally aligned, as follows:
This assumes some background about Thought Generator, Thought Assessor, and Steering System from brain-like AGI.
The model is already the Though Generator. The human already has a Steering System, albeit it is not accessible, but plausibly, it can be reverse-engineered. What is missing is the Thought Assessor, something that learns to predict how well the model satisfies the Steering System.
Staying closer to the human may be better than finding global solutions. Or it may allow smaller-scale optimization and iteration.
Now, I don’t think this is automatically safe. The human Steering System is running already outside its specs and a powerful model can find the breaking points (same as global commerce can find the appetite breaking points). But these are problems we already have and it provides a “scale model” or working on them.
The Big Rule Adjustment
This section is really the timeless part of the post and comes much too late or maybe rather deserves its own post. I’d like to see a link to where Zvi has “said this before.”
I recommend making this into a full link-post. I agree about the relevance for AI alignment.
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
For what it’s worth, I think we will soon see “robots” or LLMs or some such systems that have meta-consciousness or self-consciousness. There are reports of LLMs passing the mirror test and if they can do that and argue the case—and I have seen pretty advanced arguments about reflection too—then you have meta-consciousness also.
Sentience: This feels like a continuous thing that gets less and less sophisticated as we go up the information history. In each generation, the code gets a little better at using the laws of physics and chemistry to preserve itself.
I think that getting better at using the laws of physics to reproduce, is some stage before sentience. Sentience as defined by Singer is about responses to pleasure and pain stimuli—which is a specific adaptation that requires specific neural pathways that are not present, e.g., in bacteria. I’m fine with adding another layer before sentience, let’s call it reproduction, and maybe that one is continuous as you suggest, but it stretches what people call conscious. Sure, you can define consciousness to include that layer, and maybe that is what people call panpsychism, but to me, that seems more like expanding a definition by applying an affect heuristic.
But the last two still feel too strong. I will think more about it.
I’m not sure what “the last two”. :confused:
I like the idea! If you have it, the question is when does it start? Let’s look at it for different aspects of consciousness:
-
Sentience (Bentham, Singer): Behavioral responses to pleasure or pain stimuli and physiological measures. This is observable across animal species, from mammals to some invertebrates and it should be known when responses to such stimuli start in the embryo.
-
Wakefulness: Measureable in virtually all animals with a central nervous system by physiological indicators such as EEG, REM, and muscle tone. The fetus is known to have a sleep wake rhythm, but I don’t know when it starts.
-
Dennet’s Intentionality: Treating living beings as if they have beliefs and desires makes good predictions for many animal species, esp. social, like primates, cetaceans, and birds. Infants show goal directed behaviors right after birth. I remember ultrasound photos that show babies suckling their thumb. I think we can identify when the nervous system is first capable of goal direction.
-
Dehaene’s Phenomenal Consciousness: A perception or thought is conscious if you can report on it. As this requires language or measuring neural patterns that are similar to humans during comparable reports, I think this starts when communicable representstions of perceptions first form, for toddlers around age one at the earliest with baby sign language.
-
Gallup’s Self-Consciousness: Recognition of oneself e.g. in a mirror. Requires sufficient sensual resolution and intelligence for a self-model. Dito.
-
Rosenthal’s Meta-Consciousness: This is investigated through introspective reports on self-awareness of cognitive processes or self-reflective behaviors. Requires more abstraction. Maybe at age five?
-
Hm. You could make quizzes yourself, but that was some effort. It seems the paiq quizzes are standardized and easy to make. Nice. Many Okcupid tests were more like MBTI tests. Here is where people are discussing one of the bigger ones.
People try new dating platforms all the time. It’s what Y Combinator calls a tarpit. The problem sounds solvable, but the solution is elusive.
As I have said elsewhere: Dating apps are broken because the incentives of the usual core approach don’t work.
On the supplier side: Misaligned incentives (keep users on the platform) and opaque algorithms lead to bad matches.
On the demand side: Misaligned incentives (first impressions, low cost to exit) and no plausible deniability lead to predators being favored.
People start dating portals all the time. If you start with a targetted group that takes high value from it, you could plausibly do it in terms of network effect. Otherwise, you couldn’t start any network app or the biggest one would automatically win. So I think your argument proves too much.
The quizzes sounds is something Okcupid also used to have. Also everything that reduces the need for first impressions. I hope they keep it.
Interest groups without an organizer.
This is a product idea that solves a large coordination problem. With billion people, there could be a huge number of groups of people sharing multiple interests. But currently, the number of valuable groups of people is limited by a) the number of organizers and b) the number of people you meet via a random walk. Some progress has been made on (b) with better search, but it is difficult to make (a) go up because of human tendencies—most people are lurkers—and the incentive to focus on one area to stand out. So what is the idea? Cluster people by interests and then suggest the group to all members. If people know that the others know that there is interest, the chance of the group coming together gets much higher.
I said die, not kill. Let the predators continue to use the dating platforms if they want. It will keep them away from other more wholesome places.
As I have said elsewhere:
Dating apps are broken. Maybe it’s better dating apps die soon.
On the supplier side: Misaligned incentives (keep users on the platform) and opaque algorithms lead to bad matches.
On the demand side: Misaligned incentives (first impressions, low cost to exit) and no plausible deniability lead to predators being favored.
Real dating happens when you can observe many potential mates and there is a path to getting closer. Traditionally that was schools, clubs, church, work. Now, not so much. Let’s build something that fosters what was lost, now double down on a failed principle − 1-to-1 matching.
100 times more parameter efficient (102 vs 104 parameters) [this must be a typo, this would only be 1.01 times more parameter efficient].
clearly, they mean 10^2 vs 10^4. Same with the “10−7 vs 10−5 MSE”. Must be some copy-paste/formatting issue.
I like it. It’s quite evocative.