Primarily interested in agent foundations and AI macrostrategy.
I endorse and operate by Crocker’s rules.
I have not signed any agreements whose existence I cannot mention.
Mateusz Bagiński
Karma: 966
terminal values in the first place, as opposed to active blind spots masquerading as terminal values.
Can’t one’s terminal values be exactly (mechanistically implemented as) active blind spots?
I predict that you would say something like “The difference is that active blind spots can be removed/healed/refactored ‘just’ by (some kind of) learning, so they’re not unchanging as one’s terminal values would be assumed to be.”?
cover ups
Why do you think there are cover-ups?
More specifically, do you mean that people-in-the-know are not willing to report it or that there is some active silencing or [discouragement of those who would like to bring attention to it] going on?
There was one community alert about Zizians 2y ago here. Before that, there was a discussion of Jessica Taylor’s situation being downstream from Vassar’s influence but as far as I remember Scott Alexander eventually retracted his claims about this.
In any case, I think this kind of stuff deserves a top-level alert post, like the one about Ziz.
Also: anybody have any recommendations for pundits/analysis sources to follow on the Taiwan situation? (there’s Sentinel but I’d like something more in-depth and specifically Taiwan-related)
I think Mesa is saying something like “The missing pieces are too alien for us to expect to discover them by thinking/theorizing but we’ll brute-force the AI into finding/growing those missing pieces by dumping more compute into it anyway.” and Tsvi’s koan post is meant to illustrate how difficult it would be to think oneself into those missing pieces.
Estonia. (Alternatively, Poland, in which case: PLN, not EUR.)
I’m considering donating. Any chance of setting up some tax deduction for Euros?
I think you meant to hide these two sentences in spoiler tags but you didn’t
guilt-by-association
Not necessarily guilt-by-association, but maybe rather pointing out that the two arguments/conspiracy theories share a similar flawed structure, so if you discredit one, you should discredit the other.
Still, I’m also unsure how much structure they share, and even if they did, I don’t think this would be discursively effective because I don’t think most people care that much about (that kind of) consistency (happy to be updated in the direction of most people caring about it).
Reminds me of how a few years ago I realized that I don’t feel some forms of stress but can infer I’m stressed by noticing reduction in my nonverbal communication.
FYI if you want to use o1-like reasoning, you need to check off “Deep Think”.
It’s predictably censored on CCP-sensitive topics.
(In a different chat.) After the second question, it typed two lines (something like “There have been several attempts to compare Winnie the Pooh to a public individual...”) and then overwrote it with “Sorry...”.
glitch tokens are my favorite example
I directionally agree with the core argument of this post.
The elephant(s) in the room according to me:
What is an algorithm? (inb4 a physical process that can be interpreted/modeled as implementing computation)
How do you distinguish (hopefully, in a principled way) between (a) an algorithm changing; (b) you being confused about what algorithm the thing is actually running and in reality being more nuanced so that what “naively” seems like a change of the algorithm is “actually” a reparametrization of the algorithm?
I haven’t read the examples in this post super carefully, so perhaps you discuss this somewhere in the examples (though I don’t think so because the examples don’t seem to me like the place to include such discussion).
Thanks for the post! I expected some mumbo jumbo but it turned out to be an interesting intuition pump.
Based on my attending Oliver’s talk, this may be relevant/useful:
I too have reservations about points 1 and 3 but not providing sufficient references or justifications doesn’t imply they’re not on SL1.
Compositional agency?