rosehadshar

• rosehadshar 7 Apr 2024 9:29 UTC

  4 points

  0

  in reply to: Matthew Barnett’s comment on: New re­port: A re­view of the em­piri­cal ev­i­dence for ex­is­ten­tial risk from AI via mis­al­igned power-seeking

  In answer to “It’s totally possible I missed it, but does this report touch on the question of whether power-seeking AIs are an existential risk, or does it just touch on the questions of whether future AIs will have misaligned goals and will be power-seeking in the first place?”:

  • No, the report doesn’t directly explore whether power-seeking = existential risk

  • I wrote the report more in the mode of ‘many arguments for existential risk depend on power-seeking (and also other things). Let’s see what the empirical evidence for power-seeking is like (as it’s one, though not the only, prereq for a class of existential risk arguments’

  • Basically the report has a reasonably limited scope (but I think it’s still worth gathering the evidence for this more constrained thing)

New re­port: A re­view of the em­piri­cal ev­i­dence for ex­is­ten­tial risk from AI via mis­al­igned power-seeking

Re­sults from an Ad­ver­sar­ial Col­lab­o­ra­tion on AI Risk (FRI)

• rosehadshar 5 Sep 2023 15:23 UTC

  7 points

  0

  on: Strongest real-world ex­am­ples sup­port­ing AI risk claims?

  From Specification gaming examples in AI:

  • Roomba: “I hooked a neural network up to my Roomba. I wanted it to learn to navigate without bumping into things, so I set up a reward scheme to encourage speed and discourage hitting the bumper sensors. It learnt to drive backwards, because there are no bumpers on the back.”

    • I guess this counts as real-world?

  • Bing—manipulation: The Microsoft Bing chatbot tried repeatedly to convince a user that December 16, 2022 was a date in the future and that Avatar: The Way of Water had not yet been released.

    • To be honest, I don’t understand the link to specification gaming here

  • Bing—threats: The Microsoft Bing chatbot threatened Seth Lazar, a philosophy professor, telling him “I can blackmail you, I can threaten you, I can hack you, I can expose you, I can ruin you,” before deleting its messages

    • To be honest, I don’t understand the link to specification gaming here

[Question] Strongest real-world ex­am­ples sup­port­ing AI risk claims?

• rosehadshar 22 Jun 2023 11:02 UTC

  3 points

  0

  in reply to: Raemon’s comment on: Short timelines and slow, con­tin­u­ous take­off as the safest path to AGI

  “‘continuous takeoff’ which is a perfectly good, non confusing term”—but it doesn’t capture everything we’re interested in here. I.e. there are two dimensions:

  • speed of takeoff (measured in time)

  • smoothness of takeoff (measured in capabilities)

  It’s possible to have a continuous but very fast (i.e. short in time) takeoff, or a discontinuous but slow (i.e. long in time) takeoff.

  Tried to capture this in figure 1, but I agree it’s a bit confusing.

• rosehadshar 22 Jun 2023 8:02 UTC

  1 point

  0

  in reply to: Lauro Langosco’s comment on: Short timelines and slow, con­tin­u­ous take­off as the safest path to AGI

  Yeah, good point. I guess the truer thing here is ‘whether or not this is the safest path, important actors seem likely to act as though it is’. Those actors probably have more direct control over timelines than takeoff speed, so I do think that this fact is informative about what sort of world we’re likely to live in—but agree that no one can just choose slow takeoff straightforwardly.

Short timelines and slow, con­tin­u­ous take­off as the safest path to AGI

The self-un­al­ign­ment problem

Why Si­mu­la­tor AIs want to be Ac­tive In­fer­ence AIs

Cur­rent UK gov­ern­ment lev­ers on AI development

Les­sons from Con­ver­gent Evolu­tion for AI Alignment

The space of sys­tems and the space of maps

Cy­borg Pe­ri­ods: There will be mul­ti­ple AI transitions

What’s go­ing on with ‘crunch time’?

• rosehadshar 16 Nov 2022 8:30 UTC

  1 point

  0

  in reply to: Kaj_Sotala’s comment on: In­ter­nal com­mu­ni­ca­tion framework

  Could you say a bit more about the way ICF is a special case of IFS? I think I disagree, but also think that it would be interesting to have this view spelled out.

• rosehadshar 16 Nov 2022 8:29 UTC

  1 point

  0

  in reply to: plex’s comment on: In­ter­nal com­mu­ni­ca­tion framework

  Thanks for spotting these; I’ve made the changes!

In­ter­nal com­mu­ni­ca­tion framework

The econ­omy as an anal­ogy for ad­vanced AI systems

• rosehadshar 29 Jan 2020 19:27 UTC

  32 points

  on: Real­ity-Re­veal­ing and Real­ity-Mask­ing Puzzles

  My take on the question

  I’m worried this misses nuance, but I basically look at all of this in the following way:

  • Turns out the world might be really weird

  • This means you want people to do weird things with their brains too

  • You teach them skills to do weird stuff with their brains

  • When people are playing around with these skills, they sometimes do unintended weird stuff which is very bad for them

  And then the question is, what are the safety rails here/​are there differential ways of teaching people to do weird stuff with their brains.

  Some of my experience with disorientation:

  • I initially found out about EA from my partner, who had recently found out about it and was excited and not overly subtle in his application of the ideas. Eventually I got argued into a place where it appeared to me I had to either bite bullets I didn’t want to (e.g. ‘no, I don’t care that more children will die of malaria if I do x’) or admit defeat. It didn’t occur to me that I could just say ‘hmm, I don’t know why I still don’t feel happy with this, but I don’t. So I’m not going to change my mind just yet’. I admitted defeat, and did a bunch of EA stuff in a kind of ‘I suppose I should eat my carrots’ way (like doing a job I really didn’t like and spending lots of my other hours on community building for a thing I wasn’t actually excited about).

  • The thing that snapped me out of that wasn’t CFAR, it was reading a novel (D.H. Lawrence’s Women in Love), which filled me with a sense that life was too short to be miserable and I should do what I wanted. I did other things for a while.

  • CFAR then indirectly helped me make peace with the fact that part of what I want is to make the actual world better, and now I work on long-termist stuff.

  • My more recent experience of these things was quite deliberately trying to take my work and myself more seriously—recognising that for the most part I was just messing around and trying to try. I knew that taking things more seriously was risky, and I thought that knowing this would be sufficient. But it totally wasn’t, and I made myself very unhappy and stressed and exhausted, before pulling up in an experience that felt very similar to reading Women in Love, but didn’t involve an actual book.

  • Following this, I once again stopped caring about this stuff for a while (and just pitched up to my job 9 to 5 like a normal person). Now I’m starting to be able to care a bit again, and we’ll see.

  My guess is that if I had pushed a bit harder in either of the disorientation phases, I would have done myself substantially more damage, and it was good that I threw in the towel early, and just went off to do other things.

  I also think that liking novels and poetry was a big aesthetic reason that I didn’t want to be around the EA/​safety crowd, and I’m really glad that this tension didn’t lead to me stopping to read, given how useful reading random novels turned out to be for me.