the-citizen

Karma: 110

the-citizen 8 Jun 2015 5:45 UTC
0 points
in reply to: DanielLC’s comment on: Open Thread, May 18 - May 24, 2015
True! I was actually trying to be funny in (4), tho apparently I need more work.

the-citizen 24 May 2015 7:20 UTC
3 points
on: Leaving LessWrong for a more rational life

mistakes that are too often made by those with a philosophical background rather than the empirical sciences: the reasoning by analogy instead of the building and analyzing of predictive models

While there are quite a few exceptions, most actual philosophy is not done through metaphors and analogies. Some people may attempt to explain philosophy that way, while others with a casual interest in philosophy might not known the difference, but few actual philosophers I’ve met are silly enough not to know an analogy is an analogy. Philosophy and empirical science aren’t conflicting approaches or competing priorities. They interact and refine eachother in useful ways. For example philosophy may help improve reasoning where we have only limited evidence, or it may help us understand the appropriate way for evidence to be used, classified or interpreted. It’s only problematic when its used for social purposes or motivated reasoning rather than challenging our own assumptions.

I think there are certain specific instances where LW’s dominant philosophical interpretations are debatable, and I’d like to hear more of what your objections of those kind are.

I now regard the sequences as a memetic hazard, one which may at the end of the day be doing more harm than good

I think just being wrong or misleading (assuming you think the main thrust of the sequences is problematic), isn’t enough to be a memetic hazard. Otherwise we’d be banning all sorts of stuff floating around in books and the internet. I suggest memetic hazard ought to be things that are uniquely dangerous in leading to immediate harm to mental health (suicide, extreme depression or extreme aggression).

the-citizen 19 May 2015 7:42 UTC
0 points
on: Open Thread, May 18 - May 24, 2015
Suffering and AIs

Disclaimer—For the sake of argument this post will treat utilitarianism as true, although I do not neccesarily think that

One future moral issue is that AIs may be created for the purpose of doing things that are unpleasant for humans to do. Let’s say an AI is designed with the ability to have pain, fear, hope and pleasure of some kind. It might be reasonable to expect in such cases the unpleasant tasks might result in some form of suffering. Added to this problem is the fact that a finite lifespan and an approaching termination/shutdown might cause fear, another form of suffering. Taking steps to shut down an AI would also then become morally unacceptable, even though they perform an activity that might be useless or harmful. Because of this, we might face a situation where we cannot shutdown AIs even when there is good reason to.

Basically, if suffering AIs were some day extremely common, we would be introducing a massive amount of suffering into the world, which under utilitarianism is unacceptable. Even assuming some pleasure is created, we might search for ways to create that pleasure without creating the pain.

If so, would it make sense to adopt a principle of AI design that says AIs should be designed so it (1) does not suffer or feel pain (2) should not fear death/shutdown (eg. views own finite life as acceptable). This would minimise suffering (potentially you could also attempt to maximise happiness).

Potential issues with this: (1) Suffering might be in some way relative, so that a neutral lack of pleasure/happiness might become “suffering”. (2) Pain/suffering might be useful to create a robot with high utility, and thus some people may reject this principle. (3) I am troubled by this utilitarian approach I have used here as it seems to justify tiliing the universe with machines whose only purpose and activity is to be permanently happy for no reason. (4) Also… killer robots with no pain or fear of death :-P

the-citizen 19 May 2015 7:40 UTC
0 points
in reply to: hairyfigment’s comment on: Open Thread, May 11 - May 17, 2015
That seems like an interesting article, though I think it is focused on the issue of free-will and morality which is not my focus.

the-citizen 17 May 2015 6:52 UTC
0 points
on: Open Thread, May 11 - May 17, 2015
Suffering and AIs

Disclaimer—Under utilitarianism suffering is an intrinsically bad thing. While I am not a utilitarian, many people are and I will treat it as true for this post because it is the easiest approach for this issue. Also, apologies if others have already discussed this idea, which seems quite possible

One future moral issue is that AIs may be created for the purpose of doing things that are unpleasant for humans to do. Let’s say an AI is designed with the ability to have pain, fear, hope and pleasure of some kind. It might be reasonable to expect in such cases the unpleasant tasks might result in some form of suffering. Added to this problem is the fact that a finite lifespan and an approaching termination/shutdown might cause fear, another form of suffering. Taking steps to shut down an AI would also then become morally unacceptable, even though they perform an activity that might be useless or harmful. Because of this, we might face a situation where we cannot shutdown AIs even when there is good reason to.

Basically, if suffering AIs were some day extremely common, we would be introducing a massive amount of suffering into the world, which under utilitarianism is unacceptable. Even assuming some pleasure is created, we might search for ways to create that pleasure without creating the pain.

If so, would it make sense to adopt a principle of AI design that says AIs should be designed so it (1) does not suffer or feel pain (2) should not fear death/shutdown (eg. views own finite life as acceptable). This would minimise suffering (potentially you could also attempt to maximise happiness).

Potential issues with this: (1) Suffering might be in some way relative, so that a neutral lack of pleasure/happiness might become “suffering”. (2) Pain/suffering might be useful to create a robot with high utility, and thus some people may reject this principle. (3) I am troubled by this utilitarian approach I have used here as it seems to justify tiliing the universe with machines whose only purpose and activity is to be permanently happy for no reason. (4) Also… killer robots with no pain or fear of death :-P

the-citizen 2 Jan 2015 8:22 UTC
0 points
in reply to: buybuydandavis’s comment on: SciAm article about rationality corresponding only weakly with IQ
Yeah I think you’re right on that one. Still, I like and share his moral assumption that my-side-ism is harmful because it distorts and is often opposed to the truth in communication.

I retracted an earlier incorrect assertion and then edited to make this one instead. Not sure how that works exactly...

the-citizen 29 Dec 2014 14:30 UTC
0 points
in reply to: buybuydandavis’s comment on: SciAm article about rationality corresponding only weakly with IQ

His gratuitous imposition of his own moral assumptions are worse.

I don’t see the problem with moral assumptions, as long as they are clear and relevant. I think generally the myside effect is a force that stands against truth-seeking—I guess its a question of definition whether you consider that to be irrational or not. People that bend the truth to suit themselves distort the information that rational people use for decision making, so I think its relevant.

*The lack of an “unretract” feature is annoying.

the-citizen 17 Dec 2014 12:38 UTC
0 points
in reply to: TheAncientGeek’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
Indeed I do! :-)

the-citizen 15 Dec 2014 6:02 UTC
2 points
in reply to: Rob Bensinger’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
Cheers thanks for the informative reply.

the-citizen 15 Dec 2014 5:44 UTC
0 points
in reply to: ChristianKl’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
Yes some humans seem to have adopted this view where intelligence moves from being a tool and having instrumental value to being instrinsically/terminally valuable. I find often the justifcation for this to be pretty flimsy, though quite a few people seem to have this view. Let’s hope a AGI doesn’t lol.

the-citizen 15 Dec 2014 5:39 UTC
0 points
in reply to: TheAncientGeek’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
Thanks for reply. That makes more sense to me now. I agree with a fair amount of what you say. I think you’d have a sense from our previous discussions why I favour physicalist approaches to the morals of a FAI, rather than idealist or dualist, regardless of whether physicalism is true or false. So I won’t go there. I pretty much agree with the rest.

EDIT> Oh just on the deep ecology point, I believe that might be solvable by prioritising species based on genetic similarity to humans. So basically weighting humans highest and other species less so based on relatedness. I certainly wouldn’t like to see a FAI adopting the view that people have of “humans are a disease” and other such views, so hopefully we can find a way to avoid that sort of thing.

the-citizen 14 Dec 2014 7:39 UTC
0 points
in reply to: TheAncientGeek’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
Thanks that’s informative. Not entirely sure your own position is from your post, but I agree with what I take your implication to be—that a rationally discoverable set of ethics might not be as sensible notion as it sounds. But on the other hand human preference satisfaction seems a really bad goal—many human preferences in the world are awful—take a desire for power over others for example. Otherwise human society wouldn’t have wars, torture, abuse etc etc. I haven’t read up on CEV in detail, but from what I’ve seen it suffers from a confusion that somehow decent preferences are gained simply by obtaining enough knowledge? I’m not fully up to speed here so I’m willing to be corrected.

EDIT> Oh… CEV is the main accepted approach at MIRI :-( I assumed it was one of many

the-citizen 14 Dec 2014 7:29 UTC
0 points
in reply to: Rob Bensinger’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
The stability under self-modification is a core problem of AGI generally, isn’t it? So isn’t that an effort to solve AGI, not safety/friendliness (which would be fairly depressing given its stated goals)? Does MIRI have a way to define safety/friendliness that isn’t derivative of moral philosophy?

Additionally, many human preferences are almost certainly not moral… surely a key part of the project would be to find some way to separate the two. Preference satisfaction seems like a potentially very unfriendly goal...

the-citizen 14 Dec 2014 7:27 UTC
0 points
in reply to: Rob Bensinger’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
For the record, my current position is that if MIRI doesn’t think it’s central, then it’s probably doing it wrong.

the-citizen 13 Dec 2014 13:56 UTC
0 points
in reply to: Rob Bensinger’s comment on: A forum for researchers to publicly discuss safety issues in advanced AI
But perhaps moral philosophy is important for a FAI? Like for knowing right and wrong so we can teach/build it into the FAI? Understanding right and wrong in some form seems really central to FAI?

the-citizen 13 Dec 2014 8:12 UTC
0 points
on: A forum for researchers to publicly discuss safety issues in advanced AI
What do you feel is bad about moral philosophy? It looks like you dislike it because place it next to anthropormorphic thinking and technophobia.

the-citizen 10 Dec 2014 12:37 UTC
0 points
in reply to: KatjaGrace’s comment on: Superintelligence 12: Malignant failure modes
I’ll leave these two half-baked ideas here in case they’re somehow useful:

DO UNTIL - Construct an AI to perform its utility function until an undesirable failsafe condition is met. (Somehow) make the utility function not take the failsafe into account when calculating utility (can it be made blind to the failsafe somehow? Force the utility function to exclude their existence? Make lack of knowledge about failsafes part of the utility function?) Failsafes could be every undesirable outcome we can think of, such as human death rate exceeds X, biomass reduction, quantified human thoughts declines by X, mammalian species extictions, quantified human suffering exceeds X, or whatever. One problem is how to objectively attribute these triggers causally to the AI (what if another event occurs and shuts down the AI which we now rely on).

Energy limit - Limit the AIs activities (through its own utility function?) through an unambiguous quantifiable resource—matter moved around or energy expended. The energy expended would (somehow) include all activity under its control. Alternatively this could be a rate rather than a limit, but I think this would be more likely to go wrong. The idea would be to let the AGI go foom, but not let it have energy for other stuff like a paperclip universe. I am not sure about this idea achieving all that much safety, but here it is.

I don’t know if an intelligence explosion will truely be possible, but plenty of people smarter than I seem to think so… good luck in this field of work!

the-citizen 10 Dec 2014 12:09 UTC
2 points
in reply to: SteveG’s comment on: Superintelligence 12: Malignant failure modes
Wouldn’t most AGI goals disregard property rights unless it was explicitly built in? And if it was built in, wouldn’t an AGI just create a situation (eg. progressive blackmail or deception or something) where we wanted to sell it the universe for a dollar?

the-citizen 10 Dec 2014 11:55 UTC
0 points
in reply to: KatjaGrace’s comment on: Superintelligence 12: Malignant failure modes
The risk of course is the AI predicting that it’s nested in this sort of environment and finding a way to signal to observers. Even if it’s blind to the other layers it might try it just in case. What you want is to develop a way for the simulated world environment to detect a harmful intellgience explosion and send a single bit binary communication “out of the box” to indicate that it has occurred. Then you can shut it down and keep trying multiple instances until you get a success at this level of safety. I guess you can then slowly expand the amount of information that can come “out of the box” for safety. I have no idea how the detection process could be implemented, though… perhaps the difficultly of its implementation would defeat the usefulness of this idea?

EDIT> Interaction between the AGI and the detection mechanism could be problematic… it could predict its existence and find a way to deceive it?

the-citizen 2 Dec 2014 8:05 UTC
0 points
in reply to: [deleted]’s comment on: [Link] If we knew about all the ways an Intelligence Explosion could go wrong, would we be able to avoid them?
Thanks for comment. I will reply as follows:
- Knowing how things could go wrong gives useful knowledge about scenarios/pathways to avoid
- Our knowledge of how to make things go right is not zero
My intention with the article is to draw attention to some broader non-technical difficulties in implementing FAI. One worrying theme in the reponses I’ve gotten is a conflation between knowledge of AGI risk and building a FAI. I think they are separate projects, and that success of the second relies on comprehensive prior knowledge of the first. Apparently MIRI’s approach doesn’t really acknowledge the two as separate.