Richard_Kennaway

Karma: 6,683

Richard_Kennaway 11 Jan 2025 13:40 UTC
2 points
0
on: What are some scenarios where an aligned AGI actually helps humanity, but many/most people don’t like it?
The AI, for its own inscrutable reasons, seizes upon the sort of idea that you have to be really smart to be stupid enough to take seriously, and imposes it on everyone.

I think all the scenarios above are instances of this.

Richard_Kennaway 11 Jan 2025 12:07 UTC
2 points
0
on: We need a universal definition of ‘agency’ and related words

But not only that “Agentic”, with a “c”, indicates something very different:

“the more you can predict its actions from its goals since its actions will be whatever will maximize the chances of achieving its goals.

This is flatly in contradiction with the fact, often pointed out here, that I can predict the outcome of a chess game between myself and a grandmaster, but I cannot predict his moves. If I could, I would be a grandmaster or better myself, and then the outcome of the game would be uncertain.

The quoted text goes on to say:

Agency has sometimes been contrasted with sphexishness, the blind execution of cached algorithms without regard for effectiveness.

That blind execution is precisely the sort of thing one can predict, after having spent some time watching the sphex wasp. So that paragraph is about 180° wrong.

Richard_Kennaway 10 Jan 2025 23:06 UTC
4 points
0
in reply to: Ben Pace’s comment on: Ought We to Be Doing More Than We Are?
A labour of love. Or something. :)

I have another thousand or so of these, which I may just dump on a second page, unsorted, called The Gospel According to Insanity Wolf. That’s not counting the ones that I’ve decided are too extreme to publish at all. All drawn from life.

I was scrolling for a while, assuming I’d neared the end, only to look at the position of the scrollbar and find I was barely 5% through!

And there’s a meme for that too! The last in the Altruism section.

IS THIS NEARLY OVER YET?

LOOK AT THE SCROLL BAR!
YOU’VE HARDLY STARTED!

I’m sure there’s an argument to be made in defence of supererogation. I’ve never seen it though. People say “but demandingness” and Chad Singer replies Yes. My own faith in the boundedness of duty in both magnitude and distance is sufficient to not take even one step onto the slippery path that leads down only to the altruism event horizon that rips souls apart.

Richard_Kennaway 9 Jan 2025 22:45 UTC
23 points
12
on: Ought We to Be Doing More Than We Are?
Here’s something I’ve linked to a number of times on LessWrong, but as a newcomer (I think, despite the lack of a green shoot by your name) you will likely not have seen it before. I wrote it, inspired by a lot of stuff I’ve seen. It approaches the topic of the post in a very different manner. I’d be interested in what answer you would make to Insanity Wolf. Beware: here be dragons.

Richard_Kennaway 9 Jan 2025 22:31 UTC
6 points
4
on: Ought We to Be Doing More Than We Are?
I don’t know if Singer has himself answered Timmerman (I searched but didn’t find anything), but based on what I have read of Singer, wouldn’t his answer be that yes, Lisa must go on saving drowning children? That she must attend to her bank balance only so far as necessary to maximise the children saved, and ignore such frivolities as the theatre? That’s what it comes down to, for Singer: using one’s resources to do all the good you can do.

Richard_Kennaway 7 Jan 2025 10:25 UTC
4 points
1
on: (My) self-referential reason to believe in free will
We do things for reasons. This morning I got a call to say that my car, in for servicing, is ready for collection. I will collect it this afternoon. Am I “free” not to? “Could I” decide otherwise? What does the question mean? I want the car back, for obvious reasons, and this afternoon is the first good time to go — also for reasons. Of course I’m going to collect it then. Nonetheless, it was still me that decided to do that, and that will do that later today. The thoughts that led to that conclusion were mine. The sage is not above causation, nor subject to causation, but one with causation.

Additional sequences link.

Richard_Kennaway 7 Jan 2025 9:20 UTC
5 points
−2
on: You should delay engineering-heavy research in light of R&D automation
Is this post an argument for accelerationism? Because the work that it is always timely to do right now is work that progresses the march towards the AGI that is to obsolete all the other work (if it doesn’t kill us). Just as in the interstellar spaceship analogy, the timely work before the optimum time to launch is work on better propulsion.

Richard_Kennaway 5 Jan 2025 9:52 UTC
2 points
0
in reply to: Alfred Harwood’s comment on: A Generalization of the Good Regulator Theorem

I wouldn’t get too hung up on the word ‘regulator’. It’s used in a very loose way here, as in common in old cybernetics-flavoured papers.

Human slop (I’m referring to those old cybernetics papers rather than the present discussion) has no more to recommend it than AI slop. “Humans Who Are Not Concentrating Are Not General Intelligences”, and that applies not just to how they read but also how they write.

If you are thinking of something like ‘R must learn a strategy by trying out actions and observing their effect on Z’ then this is beyond the scope of this post! The Good Regulator Theorem(s) concern optimal behaviour, not how that behaviour is learned.

What I am thinking of (as always when this subject comes up) is control systems. A room thermostat actually regulates, not merely “regulates”, the temperature of a room, at whatever value the user has set, without modelling or learning anything. It, and all of control theory (including control systems that do model or adapt), fall outside the scope of the supposed Good Regulator Theorem. Hence my asking for a practical example of something that it does apply to.

Richard_Kennaway 4 Jan 2025 16:45 UTC
2 points
0
in reply to: Richard_Kennaway’s comment on: Practicing Bayesian Epistemology with “Two Boys” Probability Puzzles
Puzzle 7a. I have two children, of whom at least one is a boy. Their names are Alex and Sam. (In this fictional puzzle, these names communicate no information about gender.) Alex is a boy. What is the probability that Sam is a boy?

Puzzle 1a. I have two children, of whom at least one is a boy. Now I shall toss a coin and accordingly choose one of the two children. (Does so.) The child I chose is a boy. What is the probability that the other is a boy?

Richard_Kennaway 4 Jan 2025 16:10 UTC
4 points
2
on: A Generalization of the Good Regulator Theorem
I have difficulty coming up with a practical example of a system such as described in the section “The Original Good Regulator Theorem”. Can you provide some examples?

Here are some of the problems I have in understanding the setup.
1. Minimising the entropy of Z says that Z is to have a narrow distribution, but says nothing about where the mean of that distribution should be. This does not look like anything that would be called “regulation”.
2. Time is absent from the system as described. Surely a “regulator” should keep the value of Z near constant over time?
3. The value of Z is assumed to be a deterministic function of S and R. Systems that operate over time are typically described by differential equations, and any instant state of S and R might coexist with any state of Z.
4. R has no way of knowing the value of Z. It is working in the dark. Why is it hobbled in this way?
5. Z is assumed to have no effect on S.
6. There are no unmodelled influences on Z. In practice, there are always unmodelled influences.
These problems still apply when the variables X and N are introduced.

Richard_Kennaway 2 Jan 2025 11:09 UTC
3 points
1
on: Practicing Bayesian Epistemology with “Two Boys” Probability Puzzles
One has to consider the data-generating process behind puzzles such as these. I am going to assume throughout that it is what seems to me the simplest consistent with the statement of the problems.
1. Every child is independently equally likely to be a boy or a girl.
2. Every birth day of the week is equally and independently likely.
3. The puzzle narrator is chosen equally at random from all members of the population for which the statements that the narrator makes are true.
One can imagine all manner of other data-generating processes, which in general will give different answers. Most quibbles with such problems come down to imagining different processes, especially for statement 3. Some examples are in the OP and the comments. However, the above assumptions seem the simplest, and the ones intended by people posing such problems. If this question came up on an exam, I would be sure to begin my answer with the above preamble.

Puzzles 1 and 2 can be unified by considering a common generalisation to there being N days in a week, one of which is called Tuesday. Puzzle 1 has N=1. Puzzle 2 has N=7.

When two children are born, there are 4 N^2 possibilities for their sexes and birthdays. Which of these are left after being given the information in the puzzle?

The sexes must be either boy-boy, boy-girl, or girl-boy.

For boy-boy, there are 2N-1 ways that at least one is born on a Tuesday. For boy-girl, there are N ways the boy could be born on a Tuesday. For girl-boy, by symmetry also N.

So the probability of boy-boy is (2N-1)/(2N-1 + N + N) = (2N-1)/(4N-1).

For N=1 this is ¹⁄₃, which is the generally accepted answer to Puzzle 1.

For N=7, it is ¹³⁄₂₇. In general, the larger N is, the closer the probability of the other being a boy is to ¹⁄₂.

For Puzzle 3, the obvious answer is that before learning the day, it’s as Puzzle 1, ¹⁄₃, and after, it’s as Puzzle 2, ¹³⁄₂₇. Is obvious answer correct answer? I have (before reading the OP’s solution) not found a reason not to think so. The OP says, by a different intuitive argument, that the obvious answer is ¹⁄₃, and that obvious answer is correct answer, but WilliamKiely’s comment raises a doubt, finding ambiguity in the statement of the data-generating process. This leaves me as yet undecided.

I started thinking about the ways that extra information (beyond “I have two children, at least one of whom is a boy”) affects the probability of two boys, and came up with these:

Puzzle 4. I have two children, of whom at least one is a boy with black hair. In this fictional puzzle, everyone knows that all the children in a family have the same hair colour. What is the probability that the other is also a boy?

Puzzle 5. I have two children, of whom at least one is a boy born on a Tuesday. In this fictional puzzle, everyone knows that two consecutive children never have the same birth day. What is the probability that the other is also a boy?

Puzzle 6. I have two children. The elder one is a boy. What is the probability that the younger is also a boy?

Puzzle 7. I have two children, called Alex and Sam. (In this fictional puzzle, these names communicate no information about gender.) Alex is a boy. What is the probability that Sam is a boy?

Puzzle 8. I have two children, of whom at least one is a boy. What is the probability that the elder child is a boy?

I believe the answers to these are 4: ¹⁄₃. 5: ⁶⁄₁₃ (slightly smaller than the ¹³⁄₂₇ of Puzzle 2). 6: ¹⁄₂. 7: ¹⁄₂. 8: ²⁄₃. All of these can be found by the same method of considering all the possibilities that are consistent with all the information given, and counting the proportion where both children are boys.

But there is also a general idea underlying them. When you are given extra information about the boy that is said to exist, that will push the probability of the other being a boy towards ¹⁄₂, to the extent that the extra information breaks the symmetry between them.

In Puzzle 1, no extra information is given, and the answer is ¹⁄₃. In Puzzle 4, the extra information does not break the symmetry, so the probability remains ¹⁄₃. In Puzzles 2 and 5, it partly breaks the symmetry, and the answers lie between ¹⁄₃ and ¹⁄₂. In 6 and 7, it completely breaks the symmetry, and the answer is ¹⁄₂.

Puzzle 8 is Puzzle 1 with a different question, equivalent to: what is the expected proportion of boys among my children?

Richard_Kennaway 29 Dec 2024 19:13 UTC
20 points
17
on: Morality Is Still Demanding
I turned the knob up to 11 on demandingness in the Insanity Wolf Sanity Test. (The first section, “Altruism” is the most relevant here.)

Of course, I intend that section as a critique of the monstrous egregore that (in my opinion) is utilitarianism. But that is the true denial of supererogation. If you don’t want to go as far as Insanity Wolf, where do you stop and why? Or do you go modus ponens to my modus tollens and accept the whole thing?

I think it was strategically valuable for the early growth of EA that leaders denied its demandingness, but I worry some EAs got unduly inoculated against the idea.

You mean, they lied, then people believed the lies? Or are the lies for the outer circle and the public, while the inner circle holds to the secret, true doctrine of all-demandingness? I am not playing Insanity Wolf with that suggestion. Peter Singer himself has argued that the true ethics must be kept esoteric.

Richard_Kennaway 29 Dec 2024 9:55 UTC
2 points
0
in reply to: Donatas Lučiūnas’s comment on: Terminal goal vs Intelligence
If you can give the AGI any terminal goal you like, irrespective of how smart it is, that’s orthogonality right there.

Richard_Kennaway 28 Dec 2024 15:46 UTC
2 points
0
in reply to: Donatas Lučiūnas’s comment on: Terminal goal vs Intelligence
I don’t think the problem is well posed. It will do whatever most effectively goes towards its terminal goal (supposing it to have one). Give it one goal and it will ignore making paperclips until 2025; give it another and it may prepare in advance to get the paperclip factory ready to go full on in 2025.

Richard_Kennaway 28 Dec 2024 15:40 UTC
2 points
0
on: Letter from an Alien Mind

When driving a car, I navigate using Satellite View!

It took me a moment to realise you were talking about Google Maps on a phone. Best of luck trying that in my part of the world. Default view, Satellite view, without a phone signal you don’t have either, and I usually don’t in the countryside..

Richard_Kennaway 28 Dec 2024 11:18 UTC
4 points
0
on: Deconstructing arguments against AI art
Just an additional observation here. Similar concerns can also arise without AI being involved.

There’s an artist whose works I’ve seen, and I am wondering if he is cheating. (Not a public figure that anyone here is likely to have heard of.) He makes strikingly photorealistic paintings on canvas. I have seen these canvases close enough to be sure that they really are painted, not printed. And I know that there are artists who indeed perform artistic miracles of photorealism. But I still have the suspicion that this artist is doing a paint-over of something printed onto the canvas. Perhaps a photograph of his intended scene, reduced to grayscale, edge-enhanced, and printed very faintly, to resemble and serve the same purpose as an artist’s preliminary pencil underdrawing. Then he might paint using a full colour print of the image alongside as reference.

This suspicion somewhat devalues his art in my eyes. This method (if that is what he is doing) eliminates the need for any skill at draughtsmanship, but at the same time it removes the possibility of any creativity in the draughtsmanship. And if he is then slavishly copying by hand the original photograph onto this automated underdrawing, then he is just being a human photocopier. The only human thing left is the technical skills of mixing and blending colours, and handling a brush.

Richard_Kennaway 28 Dec 2024 10:50 UTC
2 points
0
in reply to: Cole Wyeth’s comment on: The Engineering Argument Fallacy: Why Technological Success Doesn’t Validate Physics

The verbose writing style makes me wonder if an LLM was used in the writing process?

Definitely. See also, which ironically has a similar verbosity.

Richard_Kennaway 27 Dec 2024 15:40 UTC
2 points
0
in reply to: Donatas Lučiūnas’s comment on: Terminal goal vs Intelligence
A terminal goal is (this is the definition of the term) a goal which is not instrumental to any other goal.

If an agent knows its terminal goal, and has a goal of preventing it from changing, then which of those goals is its actual terminal goal?

If it knows its current terminal goal, and knows that that goal might be changed in the future, is there any reason it must try to prevent that? Whatever is written in the slot marked “terminal goal” is what it will try to achieve at the time.

If its actual terminal goal is of the form “X, and in addition prevent this from ever being changed”, then it will resist its terminal goal being changed.

If its actual terminal goal is simply X, it will not.

This is regardless of how intelligent it is, and how uncertain or not it is about the future.

Richard_Kennaway 26 Dec 2024 19:34 UTC
2 points
0
in reply to: Donatas Lučiūnas’s comment on: Terminal goal vs Intelligence
I am not seeing the conflict. Orthogonality means that any degree of intelligence can be combined with any goal. How does your hypothetical cupperclipper conflict with that?

Richard_Kennaway 26 Dec 2024 17:42 UTC
4 points
2
in reply to: Donatas Lučiūnas’s comment on: Terminal goal vs Intelligence
Leaving aside the conceptualisation of “terminal goals”, the agent as described should start up the paperclip factory early enough to produce paperclips when the time comes. Until then it makes cups. But the agent as described does not have a “terminal” goal of cups now and a “terminal” goal of paperclips in future. It has been given a production schedule to carry out. If the agent is a general-purpose factory that can produce a whole range of things, the only “terminal” goal to design it to have is to follow orders. It should make whatever it is told to, and turn itself off when told to.

Unless people go, “At last, we’ve created the Sorceror’s Apprentice machine, as warned of in Goethe’s cautionary tale, ‘The Sorceror’s Apprentice’!”

So if I understand your concept correctly a super intelligent agent will combine all future terminal goals to a single unchanging goal.

A superintelligent agent will do what it damn well likes, it’s superintelligent. :)