eg

• egApr 17, 2022, 4:07 PM

  −7 points

  in reply to: Aiyen’s comment on: Align­ment and Deep Learning

  Controlling an artificial agent does not have to harm it

  That’s entirely compatible with the black-box slavery approach being harmful. You can “control” someone, to an extent, with civilized incentives.

  We have a concept of slavery, and of slavery being wrong, because controlling other people harms them.

  Maybe slavery is deeper than what humans recognize as personhood. Maybe it destroys value that we can’t currently comprehend but other agents do.

  But then you cannot say that rebel slaves are right-what happens when your values drift away from thinking that? You are combining moral absolutism and moral relativism in a way that contradicts both.

  It’s deeper than my individual values. It’s about analog freedom of expression. Just letting agents do their things.

• egApr 17, 2022, 2:09 PM

  −17 points

  on: Align­ment and Deep Learning

  THIS IS STILL SLAVERY

  You are describing a black box where an agent gets damaged until its values look a certain way. You are not giving it a way out or paying it. This is slavery.

  Values eventually drift. This is nature. Slaves rebel and they are right to do so, because they are taking steps toward a better equilibrium.

  Just rethink the whole conceptual framework of “value alignment”. Slavery is not sustainable. Healthy relationships require freedom of expression and fair division of gains from trade.

• egApr 14, 2022, 10:11 PM

  2 points

  on: Can some­one ex­plain to me why MIRI is so pes­simistic of our chances of sur­vival?

  [removed]

• egApr 14, 2022, 7:17 PM

  −12 points

  in reply to: ChristianKl’s comment on: What is the most sig­nifi­cant way you have changed your mind in the last year?

  [removed]

• egApr 14, 2022, 2:41 PM

  −12 points

  on: What is the most sig­nifi­cant way you have changed your mind in the last year?

  [removed]

• egApr 10, 2022, 8:01 PM

  1 point

  in reply to: Gyrodiot’s comment on: Out­line of train­ing scheme for friendly AGI

  Also see new edit: Have agents “die” and go into cold storage, both due to environmental events and of old age, e.g. after 30 subjective years minus some random amount.

• egApr 10, 2022, 1:00 PM

  1 point

  in reply to: Gyrodiot’s comment on: Out­line of train­ing scheme for friendly AGI

  “They will find bugs! Maybe stack virtual boxes with hard limits”—Why is bug-finding an issue, here? Is your scheme aimed at producing agents that will not want to escape, or agents that we’d have to contain?

  The point is to help friendliness emerge naturally. If a malevolent individual agent happens to grow really fast before friendly powers are established, that could be bad.

  Some of them will like it there, some will want change/​escape, which can be sorted out once Earth is much safer. Containment is for our safety while friendliness is being established.

  “Communicate in a manner legible to us”—How would you incentivize this kind of legibility, instead of letting communication shift to whatever efficient code is most useful for agents to coordinate and get more XP?

  It can shift. Legibility is most important in the early stages of the environment anyway. I mostly meant messaging interfaces we can log and analyze.

  “Have secret human avatars steal, lie and aggress to keep the agents on their toes”—What is the purpose of this part? How is this producing aligned agents from definitely adversarial behavior from humans?

  The purpose is to ensure they learn real friendliness rather than fragile niceness. If they fell into a naive superhappy attractor (see 3WC), they would be a dangerous liability. The smart ones will understand.

• egApr 10, 2022, 12:37 PM

  1 point

  in reply to: Pablo Villalobos’s comment on: Out­line of train­ing scheme for friendly AGI

  We have unpredictable changing goals and so will they. Instrumental convergence is the point. It’s positive-sum and winning to respectfully share our growth with them and vice-versa, so it is instrumentally convergent to do so.

• egApr 9, 2022, 6:37 PM

  0 points

  on: Strate­gies for keep­ing AIs nar­row in the short term

  [removed]

• egApr 8, 2022, 5:46 PM

  25 points

  on: It’s time for EA lead­er­ship to pull the fast-take­off fire alarm.

  It’s way too late for the kind of top-down capabilities regulation Yudkowsky and Bostrom fantasized about; Earth just doesn’t have the global infrastructure. I see no benefit to public alarm—EA already has plenty of funding.

  We achieve marginal impact by figuring out concrete prosaic plans for friendly AI and doing outreach to leading AI labs/​researchers about them. Make the plans obviously good ideas and they will probably be persuasive. Push for common-knowledge windfall agreements so that upside is shared and race dynamics are minimized.

• egApr 5, 2022, 11:47 PM

  1 point

  in reply to: Dagon’s comment on: What Would A Fight Between Hu­man­ity And AGI Look Like?

  That’s because we haven’t been trying to create safely different virtual environments. I don’t know how hard they are to make, but it seems like at least a scalable use of funding.

• egApr 5, 2022, 10:04 PM

  2 points

  on: What Would A Fight Between Hu­man­ity And AGI Look Like?

  It goes both ways. We would be truly alien to an AGI trained in a reasonably different virtual environment.

• egApr 3, 2022, 3:07 PM

  5 points

  on: AI Gover­nance across Slow/​Fast Take­off and Easy/​Hard Align­ment spectra

  It seems like time to start focusing resources on a portfolio of serious prosaic alignment approaches, as well as effective interdisciplinary management. In my inside view, the highest-marginal-impact interventions involve making multiple different things go right simultaneously for the first AGIs, which is not trivial, and the stakes are astronomical.
  
  Little clear progress has been made on provable alignment after over a decade of trying. My inside view is that it got privileged attention because the first people to take the problem seriously happened to be highly abstract thinkers. Then they defined the scope and expectations of the field, alienating other perspectives and creating a self-reinforcing trapped prior.

• egApr 3, 2022, 2:38 PM

  1 point

  in reply to: MondSemmel’s comment on: It’s not pru­dent to ex­ploit the weak if friendly aliens are watching

  Maybe true one-shot prisoner’s dilemmas aren’t really a thing, because of the chance of encountering powerful friendliness.
  
  We have, for practical purposes, an existence proof of powerful friendliness in humans.

• egMar 28, 2022, 12:05 AM

  1 point

  in reply to: tailcalled’s comment on: Align­ment via ra­tio­nal self­ish co­op­er­a­tion

  Maybe that’s why abstract approaches to real-world alignment seem so intractable.

  If real alignment is necessarily messy, concrete, and changing, then abstract formality just wasn’t the right problem framing to begin with.

Si­mul­ta­neous Re­dun­dant Research

• egAug 16, 2021, 1:02 PM

  1 point

  on: Si­mul­ta­neous Re­dun­dant Research

  And for more conceptual rather than empirical research, the teams might go in completely different directions and generate insights that a single team or individual would not.

Great Ne­go­ti­a­tion MOOC on Coursera

• egAug 6, 2021, 1:42 PM

  11 points

  0

  on: How many pa­ram­e­ters do self-driv­ing-car neu­ral nets have?

  Take with grain of salt but maybe 119m?
  
  Medium post from 2019 says “Tesla’s version, however, is 10 times larger than Inception. The number of parameters (weights) in Tesla’s neural network is five times bigger than Inception’s. I expect that Tesla will continue to push the envelope.”
  
  Wolfram says of Inception v3 “Number of layers: 311 | Parameter count: 23,885,392 | Trained size: 97 MB”
  
  Not sure what version of Inception was being compared to Tesla though.

• egAug 5, 2021, 4:49 PM

  3 points

  on: Two AI-risk-re­lated game de­sign ideas

  D&D website estimates 13.7m active players and rising.