Alfred Harwood

Karma: 96

Alfred Harwood 5 Jan 2025 14:40 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: A Generalization of the Good Regulator Theorem
Regarding your request for a practical example.
Short Answer: It’s a toy model. I don’t think I can come up with a practical example which would address all of your issues.
Long Answer, which I think gets at what we disagree about:
I think we are approaching this from different angles. I am interested in the GRT from an agent foundations point of view, not because I want to make better thermostats. I’m sure that GRT is pretty useless for most practical applications of control theory! I read John Wentworth’s post where he suggested that the entropy-reduction problem may lead to embedded-agency problems. Turns out it doesn’t but it would have been cool if it did! I wanted to tie up that loose end from John’s post.
Why do I care about entropy reduction at all?
- I’m interested in ‘optimization’, as it pertains to the agent-like structure problem, and optimization is closely related to entropy reduction, so this seemed like an interesting avenue to explore.
- Reducing entropy can be thought of as one ‘component’ of utility maximization, so it’s interesting from that point of view.
- Reducing entropy is often a necessary (but not sufficient) condition for achieving goals. A thermostat can achieve an average temperature of 25C by ensuring that the room temperature comes from a uniform distribution over all temperatures between 75C and −25C. But a better thermostat will ensure that the temperature is distributed over a narrower (lower entropy) distribution around 25C .

Alfred Harwood 5 Jan 2025 14:11 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: A Generalization of the Good Regulator Theorem
I think we probably agree that the Good Regulator Theorem could have a better name (the ‘Good Entropy-Reducer Theorem’?). But unfortunately, the result is most commonly known using the name ‘Good Regulator Theorem’. It seems to me that 55 years after the original paper was published, it is too late to try to re-brand.
I decided to use that name (along with the word ‘regulator’) so that readers would know which theorem this post is about. To avoid confusion, I made sure to be clear (right in the first few paragraphs) about the specific way that I was using the word ‘regulator’. This seems like a fine compromise to me.

Alfred Harwood 4 Jan 2025 20:36 UTC
5 points
0
in reply to: Richard_Kennaway’s comment on: A Generalization of the Good Regulator Theorem
Minimising the entropy of Z says that Z is to have a narrow distribution, but says nothing about where the mean of that distribution should be. This does not look like anything that would be called “regulation”.
I wouldn’t get too hung up on the word ‘regulator’. It’s used in a very loose way here, as in common in old cybernetics-flavoured papers. The regulator is regulating, in the the sense that it is controlling and restricting the range of outcomes.
Time is absent from the system as described. Surely a “regulator” should keep the value of Z near constant over time?
You could imagine that this is a memoryless system where the process repeats every timestep (and S/X has no memory of what Z was previously). Then, a regulator which chose a policy which minimized entropy of Z (and used the same policy each timestep) would keep the value of Z as near constant as possible. (Maybe this is closer to what you were thinking of as a ‘regulator’ the previous point?)
The value of Z is assumed to be a deterministic function of S and R. Systems that operate over time are typically described by differential equations, and any instant state of S and R might coexist with any state of Z.
This post deals with discrete systems where S and R happen ‘before’ Z. If you want something similar but for continuous systems over time, you might want to take a look at the Internal Model Principle, which is a similar kind of result, which can apply to continuous time systems modelled with differential equations.
R has no way of knowing the value of Z. It is working in the dark. Why is it hobbled in this way?
In this setup, S is just defined as the input to R. If you wanted, you could think of as S as the ‘initial state’ and Z as the ‘final state’ of the system, provided they had the same range. But this setup also allows S to have a range different to S. R can know the value of S/X, which is what it needs to know in order to affect Z. If it knows the dynamics of the setup then it can predict the outcome and doesn’t need to ‘see’ Z.
If you are thinking of something like ‘R must learn a strategy by trying out actions and observing their effect on Z’ then this is beyond the scope of this post! The Good Regulator Theorem(s) concern optimal behaviour, not how that behaviour is learned.
There are no unmodelled influences on Z. In practice, there are always unmodelled influences.
If by ‘unmodelled’ influences, you mean ‘variables that the regulator cannot see, but still affect Z’, then this problem is solved in the framework with X (and implicitly N).

Alfred Harwood 26 Nov 2024 22:04 UTC
2 points
0
on: [bounty $100] Why are there no interesting (1D, 2-state) quantum cellular automata?
When you are considering finite tape length, how do you deal with $x^{(k - 1)}$ when $k = 0$ or $x^{(k + 1)}$ when $k = N$ ?

Alfred Harwood 20 Nov 2024 21:53 UTC
3 points
2
on: Why I Think All The Species Of Significantly Debated Consciousness Are Conscious And Suffer Intensely
Should there be a ‘d’ on the end of ‘Debate’ in the title or am I parsing it wrong?

Alfred Harwood 19 Nov 2024 15:34 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: A Straightforward Explanation of the Good Regulator Theorem
It is meant to read 350°F. The point is that the temperature is too high to be a useful domestic thermostat. I have changed the sentence to make this clear (and added a ° symbol ). The passage now reads:
Scholten gives the evocative example of a thermostat which steers the temperature of a room to 350°F with a probability close to certainty. The entropy of the final distribution over room temperatures would be very low, so in this sense the regulator is still ‘good’, even though the temperature it achieves is too high for it to be useful as a domestic thermostat.
(Edit: I’ve just realised that 35°F would also be inappropriate for a domestic thermostat by virtue of being too cold so either works for the purpose of the example. Scholten does use 350, so I’ve stuck with that. Sorry, I’m unfamiliar with Fahrenheit!)

Alfred Harwood 6 Nov 2024 22:24 UTC
1 point
0
in reply to: Jonathan Claybrough’s comment on: Abstractions are not Natural
Your reaction seems fair, thanks for your thoughts! Its a good a suggestion to add an epistemic status—I’ll be sure to add one next time I write something like this.

Alfred Harwood 6 Nov 2024 22:12 UTC
3 points
0
in reply to: johnswentworth’s comment on: Abstractions are not Natural
Got it, that makes sense. I think I was trying to get at something like this when I was talking about constraints/selection pressure (a system has less need to use abstractions if its compute is unconstrained or there is no selection pressure in the ‘produce short/quick programs’ direction) but your explanation makes this clearer. Thanks again for clearing this up!

Alfred Harwood 5 Nov 2024 14:23 UTC
6 points
0
in reply to: johnswentworth’s comment on: Abstractions are not Natural
Thanks for taking the time to explain this. This is a clears a lot of things up.
Let me see if I understand. So one reason that an agent might develop an abstraction is that it has a utility function that deals with that abstraction (if my utility function is ‘maximize the number of trees’, its helpful to have an abstraction for ‘trees’). But the NAH goes further than this and says that, even if an agent had a very ‘unnatural’ utility function which didn’t deal with abstractions (eg. it was something very fine-grained like ‘I value this atom being in this exact position and this atom being in a different position etc…’) it would still, for instrumental reasons, end up using the ‘natural’ set of abstractions because the natural abstractions are in some sense the only ‘proper’ set of abstractions for interacting with the world. Similarly, while there might be perceptual systems/brains/etc which favour using certain unnatural abstractions, once agents become capable enough to start pursuing complex goals (or rather goals requiring a high level of generality), the universe will force them to use the natural abstractions (or else fail to achieve their goals). Does this sound right?
Presumably its possible to define some ‘unnatural’ abstractions. Would the argument be that unnatural abstractions are just in practice not useful, or is it that the universe is such that its ~impossible to model the world using unnatural abstractions?

Alfred Harwood 4 Nov 2024 23:24 UTC
3 points
0
in reply to: deepthoughtlife’s comment on: Abstractions are not Natural
Its late where I am now so I’m going to read carefully and respond to comments tomorrow, but before I go to bed I want to quickly respond to your claim that you found the post hostile because I don’t want to leave it hanging.
I wanted to express my disagreements/misunderstandings/whatever as clearly as I could but had no intention to express hostility. I bear no hostility towards anyone reading this, especially people who have worked hard thinking about important issues like AI alignment. Apologies to you and anyone else who found the post hostile.

Alfred Harwood 4 Nov 2024 16:53 UTC
2 points
0
in reply to: Jonas Hallgren’s comment on: Abstractions are not Natural
Thanks for taking the time to explain this to me! I would like to read your links before responding to the meat of your comment, but I wanted to note something before going forward because there is a pattern I’ve noticed in both my verbal conversations on this subject and the comments so far.
I say something like ‘lots of systems don’t seem to converge on the same abstractions’ and then someone else says ‘yeah, I agree obviously’ and then starts talking about another feature of the NAH while not taking this as evidence against the NAH.
But most posts on the NAH explicitly mention something like the claim that many systems will converge on similar abstractions ^[1]. I find this really confusing!
Going forward it might be useful to taboo the phrase ‘the Natural Abstraction Hypothesis’ (?) and just discuss what we think is true about the world.
Your comment that its a claim about ‘proving things about the distribution of environments’ is helpful. To help me understand what people mean by the NAH could you tell me what would (in your view) constitute strong evidence against the NAH? (If the fact that we can point to systems which haven’t converged on using the same abstractions doesn’t count)
1. ^
  Natural Abstractions: Key Claims, Theorems and Critiques: ‘many cognitive systems learn similar abstractions’,
  Testing the Natural Abstraction Hypothesis: Project Intro ‘a wide variety of cognitive architectures will learn to use approximately the same high-level abstract objects/concepts to reason about the world’
  The Natural Abstraction Hypothesis: Implications and Evidence ’there exist abstractions (relatively low-dimensional summaries which capture information relevant for prediction) which are “natural” in the sense that we should expect a wide variety of cognitive systems to converge on using them.
  ′

Alfred Harwood 12 Sep 2024 12:45 UTC
4 points
0
on: Open Thread Summer 2024
Hello! My name is Alfred. I recently took part in AI Safety Camp 2024 and have been thinking about the Agent-like structure problem. Hopefully I will have some posts to share on the subject soon.