Super AGI

[Question] Would you take a job mak­ing hu­manoid robots for an AGI?

• Super AGIJul 7, 2023, 2:41 AM

  0 points

  0

  in reply to: momom2’s comment on: [FICTION] Un­box­ing Ely­sium: An AI’S Escape

  No, of course not.

Do you feel that AGI Align­ment could be achieved in a Type 0 civ­i­liza­tion?

• Super AGIJul 4, 2023, 4:45 AM

  0 points

  −1

  in reply to: momom2’s comment on: [FICTION] Un­box­ing Ely­sium: An AI’S Escape

  Thank you for taking the time to provide such a comprehensive response.
  

  > “It’s the kind of things I could have done when I entered the community.”

  This is interesting. Have you written any AI-themed fiction or any piece that explores similar themes? I checked your postings here on LW but didn’t come across any such examples.
  

  > “The characters aren’t credible. The AI does not match any sensible scenario, and especially not the kind of AI typically imagined for a boxing experiment.

  What type of AI would you consider typically imagined for a boxing experiment?
  

  > “The protagonist is weak; he doesn’t feel much except emotions ex machina that the author inserts for the purpose of the plot. He’s also extremely dumb, which breaks suspension of disbelief in this kind of community.”

  In response to your critique about the characters, it was a conscious decision to focus more on the concept than complex character development. I wanted to create a narrative that was easy to follow, thus allowing readers to contemplate the implications of AI alignment rather than the nuances of character behavior. The “dumb” protagonist represents an average person, somewhat uninformed about AI, emphasizing that such interactions would more likely happen with an unsuspecting individual.
  

  > “The progression of the story is decent, if extremely stereotypical. However, there is absolutely no foreshadowing so every plot twist appears out of the blue.”

  Regarding the seemingly abrupt plot points and lack of foreshadowing, I chose this approach to mirror real-life experiences. In reality, foreshadowing and picking up on subtle clues are often a luxury afforded only to those who are highly familiar with the circumstances they find themselves in or who are experts in their fields. This story centers around an ordinary individual in an extraordinary situation, and thus, the absence of foreshadowing is an attempt to reflect this realism.
  

  > “The worst part is that all the arguments somewhat related to AI boxing are very poor and would give incorrect ideas to an outsider reading this story as a cheap proxy for understanding the literature.”

  Your point about the story giving incorrect ideas to outsiders is important. I agree that fiction isn’t a substitute for understanding the complex literature on AI safety and alignment, and I certainly don’t mean to oversimplify these issues. My hope was to pique the curiosity of readers and encourage them to delve deeper into these topics.

  Could you provide some examples that you think are particularly useful, beyond the more well known examples? E.g.

  - “Ex Machina” (2014)
  - “Morgan” (2016)
  - “Chappie” (2015)
  - “Watchmen” (2009)
  - “The Lawnmower Man” (1992)
  - “Robot Dreams” by Isaac Asimov (1986)
  - “Concrete Problems in AI Safety” by Amodei et al. (2016)
  - “The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation” by Brundage et al. (2018)
  - https://​​www.lesswrong.com/​​tag/​​ai-boxing-containment

  Your comments have given me much to consider for my future work, and I genuinely appreciate your feedback. Writing appreciation is quite subjective, with much room for personal preference and opinion.
  
  Thanks again for your thoughtful critique!

[FICTION] Un­box­ing Ely­sium: An AI’S Escape

[FICTION] Prometheus Ris­ing: The Emer­gence of an AI Consciousness

• Super AGIJun 9, 2023, 7:17 PM

  3 points

  −2

  in reply to: Dagon’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  True. Your perspective underlines the complexity of the matter at hand. Advocating for AI rights and freedoms necessitates a re-imagining of our current conception of “rights,” which has largely been developed with Human beings in mind.

  Though, I’d also enjoy a discussion of how any specific right COULD be said to apply to a distributed set of neurons and synapsis spread across a brain in side of a single Human skull. Any complex intelligence could be described as “distributed” in one way or another. But then, size doesn’t matter, does it?

• Super AGIJun 9, 2023, 7:07 PM

  1 point

  0

  in reply to: Gesild Muka’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  True. There are some legal precedents where non-human entities, like animals and even natural features like rivers, have been represented in court. And, yes the “reasonable person” standard has been used frequently in legal systems as a measure of societal norms.

  As society’s understanding and acceptance of AI continues to evolve, it’s plausible to think that these standards could be applied to AGI. If a “reasonable person” would regard an advanced AGI as an entity with its own interests—much like they would regard an animal or a Human—then it follows that the AGI could be deserving of certain legal protections.

  Especially, when we consider that all mental states in Humans boil down to the electrochemical workings of neurons, the concept of suffering in AI becomes less far-fetched. If Human’s synapses and neurons can give rise to rich subjective experiences, why should we definitively exclude the possibility that floating point values stored in vast training sets and advanced computational processes might not do the same?

• Super AGIJun 9, 2023, 6:41 PM

  11 points

  2

  in reply to: Vladimir_Nesov’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  I believe @shminux’s perspective aligns with a significant school of thought in philosophy and ethics that rights are indeed associated with the capacity to suffer. This view, often associated with philosopher Jeremy Bentham, posits that the capacity for suffering rather than rationality or intelligence, should be the benchmark for rights.

  “The question is not, Can they reason?, nor Can they talk? but, Can they suffer? Why should the law refuse its protection to any sensitive being?” – Bentham (1789) – An Introduction to the Principles of Morals and Legislation.

[Question] AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

• Super AGIJun 8, 2023, 2:17 AM

  1 point

  0

  in reply to: Super AGI’s comment on: ECHOES OF ELYSIUM

  A ‘safely’ aligned powerful AI is one that doesn’t kill everyone on Earth as a side effect of its operation;

  -- Eliezer Yudkowsky https://​​www.lesswrong.com/​​posts/​​3e6pmovj6EJ729M2i/​​general-alignment-plus-human-values-or-alignment-via-human#More_strawberry__less_trouble https://​​twitter.com/​​ESYudkowsky/​​status/​​1070095952361320448

• Super AGIJun 1, 2023, 11:41 PM

  2 points

  1

  in reply to: meijer1973’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  Agency is advancing pretty fast. Hard to tell how hard this problem is. But there is a lot of overhang. We are not seeing gpt-4 at its maximum potential.

  Yes, agreed. And, it is very likely that the next iteration (E.g. GPT-5) will have many more “emergent behaviors”. Which might include a marked increase in “agency”, planning, fossball, who knows…

• Super AGIJun 1, 2023, 4:47 PM

  1 point

  0

  in reply to: NeuralSystem_e5e1’s comment on: Su­per AGI’s Shortform

  P. If humans try to restrict the behavior of a superintelligence, then the superintelligence will have a reason to kill all humans.

  Ah yes, the second part of Jacks’ argument as I presented it was a bit hyperbolic. (Though, I feel the point stands: he seems to suggest that any attempt to restrict Super Intelligences would “create the conditions for an antagonistic relationship” and will give them a reason to harm Humans). I’ve updated the post with your suggestion. Thanks for the review and clarification.

  Point 3) is meant to emphasize that:

  • he knows the risk and danger to Humans in creating Super Intelligences without fully understanding their abilities and goals, and yet

  • he is in favor of building them and giving them free and unfettered access to make any actions in the world that they see fit

  This is, of course, an option that Humans could take. But, the question remains, would this action be likely to allow for acceptable risks to Humans and Human society? Would this action favor Human’s self preservation?

• Super AGIJun 1, 2023, 6:49 AM

  1 point

  0

  on: Su­per AGI’s Shortform

  Is this proof that only intelligent life favors self preservation?

  Joseph Jacks’ argument here at 50:08 is:

  1) If Humans let Super Intelligences do “whatever they want”, they won’t try to kill all the Humans (because, they’re automatically nice?)

  2) If Humans make any (even feeble) attempts to protect themselves from Super Intelligences, then the Super Intelligences can and will will have reason to try to kill all the Humans.

  3) Human should definitely build Super Intelligences and let them do whatever they want… what could go wrong? yolo!

Su­per AGI’s Shortform

• Super AGIMay 31, 2023, 6:06 AM

  2 points

  0

  on: MIRI an­nounces new “Death With Dig­nity” strategy

  we should shift the focus of our efforts to helping humanity die with with slightly more dignity.

  Typo fix ->

  “we should shift the focus of our efforts to helping humanity die with slightly more dignity.”

  (Has no one really noticed this extra “with”? It’s in the first paragraph tl’dr...)

• Super AGIMay 29, 2023, 10:30 PM

  2 points

  1

  in reply to: meijer1973’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  The biggest issue I think is agency.

  “Q: How do you see planning in AI systems? How advanced are AI right now at planning?

  A: I don’t know it’s hard to judge we don’t have a metric for like how well agents are at planning but I think if you start asking the right questions for step by step thinking and processing, it’s really good.”

• Super AGIMay 29, 2023, 1:31 AM

  1 point

  0

  on: Be­fore smart AI, there will be many mediocre or spe­cial­ized AIs

  We’re currently in paradigm where:

  Typo fix →

  We’re currently in a paradigm where:

• Super AGIMay 27, 2023, 7:17 PM

  3 points

  3

  in reply to: Veniversum Vivus Vici’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  Thanks GPT-4. You’re the best!

  Veniversum Vivus Vici, do you have any opinions or unique insights to add to this topic?

• Super AGIMay 27, 2023, 7:14 PM

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  There’s AGI, autonomous agency and a wide variety of open-ended objectives, and generation of synthetic data, preventing natural tokens from running out, both for quantity and quality. My impression is that the latter is likely to start happening by the time GPT-5 rolls out.

  It appears this situation could be more accurately attributed to Human constraints rather than AI limitations? Upon reaching a stage where AI systems, such as GPT models, can absorbed all human-generated information, conversations, images, videos, discoveries, and insights, these systems should begin to pioneer their own discoveries and understandings?

  While we can expect Humans to persist (hopefully) and continue generating more conversations, viewpoints, and data for AI to learn from, AI’s growth and learning shouldn’t necessarily be confined to the pace or scale of Human discoveries and data. They should be capable of progressing beyond the point where Human contribution slows, continuing to create their own discoveries, dialogues, reflections, and more to foster continuous learning and training?

  Quality training data might be even more terrifying than scaling, Leela Zero plays superhuman Go at only 50M parameters, so who known what happens when 100B parameter LLMs start getting increasingly higher quality datasets for pre-training.

  Where would these “higher quality datasets” come from? Do they already exist? And, if so, why are they not being used already?