Phil_Goetz
- Elaborate, please?
“I think that Eliezer is dangerous, because he thinks he’s smart enough to make a safe AI.”
As far as I can tell, he’s not going to go and actually make that AI until he has a formal proof that the AI will be safe. Now, because of the verification problem, that’s no surefire guarantee that it will be safe, but it makes me pretty comfortable.
Good grief.Considering the nature of the problem, and the nature of Eliezer, it seems more likely to me that he will convince himself that he has proven that his AI will be safe, than that he will prove that his AI will be safe. Furthermore, he has already demonstrated (in my opinion) that he has higher confidence than he should that his notion of “safe” (eg., CEV) is a good one.
Many years ago, I made a mental list of who, among the futurists I knew, I could imagine “trusting” with godlike power. At the top of the list were Anders Sandberg and Sasha Chislenko. This was not just because of their raw brainpower—although they are/were in my aforementioned top ten list—but because they have/had a kind of modesty, or perhaps I should say a sense of humor about life, that would probably prevent them from taking giant risks with the lives of, and making decisions for, the rest of humanity, based on their equations.
Eliezer strikes me more as the kind of person who would take risks and make decisions for the rest of humanity based on his equations.
To phrase this in Bayesian terms, what is the expected utility of Eliezer creating AI over many universes? Even supposing he has a higher probability of creating beneficial friendly AI than anyone else, that doesn’t mean he has a higher expected utility. My estimation is that he excels on the upside—which is what humans focus on—having a good chance of making good decisions. But my estimation is also that, in the possible worlds in which he comes to a wrong conclusion, he has higher chances than most other “candidates” do of being confident and forging ahead anyway, and of not listening to others who point out his errors. It doesn’t take (proportionally) many such possible worlds to cancel out the gains on the upside.
This post highlights an important disagreement I have with Eliezer.
Eliezer thinks that a group of AI scientists may be dangerous, because they aren’t smart enough to make a safe AI.
I think that Eliezer is dangerous, because he thinks he’s smart enough to make a safe AI.
Asking how a “rational” agent reasons about the actions of another “rational” agent is analogous to asking whether a formal logic can prove statements about that logic. I suggest you look into the extensive literature on completeness, incompleteness, and hierarchies of logics. It may be that there are situations such that it is impossible for a “rational” agent to prove what another, equally-rational agent will conclude in that situation.
I always find it strange that, every year, the US Congress passes a budget that assumes that nothing will go wrong over the next year. Every long-range budget plan also assumes that nothing will go wrong. (On the flip side, they also assume that nothing will go right: Planning for health care assumes that investment in health research will have no effect.)
The estimate you would like to have for a project is the investment needed to complete it in the average case. But humans don’t think in terms of averages; they think in terms of typicality. They are drawn to the mode of a distribution rather than to its mean.
When distributions are symmetric, this isn’t a problem. But in planning, the distribution of time or cost to completion is bounded below by zero, and hence not symmetric. The average value will be much larger than the modal value.
Angel:In fact, as someone who benefits from privilege, the kindest thing you can probably do is open a forum for listening, instead of making post after post wherein white men hold forth about gender and race.
This is that forum. Unless you mean that we should open a forum where women, but not men, have the right to talk.
This is part of why I don’t believe you when you say that you define feminism as believing men and women have equal rights. I suspect that you would call anyone who believed a sexist.
BTW, I found an astonishing definition of morality in the President’s Council on Bioethics 2005 “Alternative sources of human pluripotent stem cells: A white paper”, in the section on altered nuclear transfer. They argued that ANT may be immoral, because it is immoral to allow a woman to undergo a dangerous procedure (egg extraction) for someone else’s benefit. In other words, it is immoral to allow someone else to be moral.
This means that the moral thing to do, is to altruistically use your time+money getting laws passed to forbid other people to be moral. The moral thing for them to do, of course, is to prevent you from wasting your time doing this.
It’s hard for me to figure out what the question means.
I feel sad when I think that the universe is bound to wind down into nothingness, forever. (Tho, as someone pointed out, this future infinity of nothingness is no worse than the past infinity of nothingness, which for some reason doesn’t bother me as much.) Is this morality?
When I watch a movie, I hope that the good guys win. Is that morality? Would I be unable to enjoy anything other than “My Dinner with Andre” after incorporating the proof that there was no morality? Does having empathic responses to the adventures of distant or imaginary people require morality?
(There are movies and videogames I can’t enjoy, that other people do, where the “good guys” are bad guys. I can’t enjoy slasher flicks. I can’t laugh when an old person falls down the stairs. Maybe people who do have no morals.)
If I do something that doesn’t benefit me personally, but might benefit my genes or memes, or a reasonable heuristic would estimate might benefit them, or my genes might have programmed me to do because it gave them an advantage, is it not a moral action?
I worry that, when AIs take over, they might not have an appreciation for art. Is that morality?
I think that Beethoven wrote much better music than John Cage; and anyone who disagrees doesn’t have a different perspective, they’re just stupid. Is that morality?
I think little kids are cute. Sometimes that causes me to be nice to them. Is that morality?
These examples illustrate at least 3 problems:
1. Disinguishing moral behavior from evolved behavior would require distinguishing free-willed behavior from deterministic behavior.
2. It’s hard to distinguish morality from empathy.
3. It’s hard to distinguish morality from aesthetics.
I think there are people who have no sense of aesthetics and no sense of empathy, so the concept has some meaning. But their lack of morality is a function of them, not of the world.
You are posing a question that might only make sense to someone who believes that “morality” is a set of behaviors defined by God.
Nick:
I don’t need to justify that I enjoy pie or dislike country music any more than I need to justify disliking murder and enjoying sex.
If you enjoyed murder, you would need to justify that more than disliking country music. These things are very different.
Roland wrote:
.I cannot imagine myself without morality because that wouldn’t be me, but another brain.
Does your laptop care if the battery is running out? Yes, it will start beeping, because it is hardwired to do so. If you removed this hardwired beeping you have removed the laptop’s morality.
Morality is not a ghost in the machine, but it is defined by the machine itself.
Well put.
I’d stop being a vegetarian. Wait; I’m not a vegetarian. (Are there no vegetarians on OvBias?) But I’d stop feeling guilty about it.
I’d stop doing volunteer work and donating money to charities. Wait; I stopped doing that a few years ago. But I’d stop having to rationalize it.
I’d stop writing open-source software. Wait; I already stopped doing that.
Maybe I’m not a very good person anymore.
People do some things that are a lot of work, with little profit, mostly for the benefit of others, that have no moral dimension. For instance, running a website for fans of Harry Potter. Writing open-source software. Organizing non-professional conventions.
(Other people.)
The thought of I—and yes, since there are no originals or copies, the very I writing this—having a guaranteed certainty of ending up doing that causes me so much anguish that I can’t help but thinking that if true, humanity should be destroyed in order to minimize the amount of branches where people end up in such situations. I find little comfort in the prospect of the “betrayal branches” being vanishingly few in frequency—in absolute numbers, their amount is still unimaginably large, and more are born every moment.
To paraphrase:Statistically, it is inevitable that someone, somewhere, will suffer. Therefore, we should destroy the world.
Eli’s posts, when discussing rationality and communication, tend to focus on failures to communicate information. I find that disagreements that I have with “normal people” are sometimes because they have some underlying bizarre value function, such as Kaj’s valuation (a common one in Western culture since about 1970) that Utility(good things happening in 99.9999% of worlds—bad things happening in 0.0001% of worlds) < 0. I don’t know how to resolve such differences rationally.
Think of how odd all this would sound without the Einstein sequence. Then think of how odd the Einstein sequence would have sounded without the many-worlds sequence...
An Einstein sequence is a unique identifier given to an observation from the Einstein observatory. If you mean something else, please explain.
If you want to appreciate the inferential distances here, think of how odd all this would sound without the Einstein sequence. Then think of how odd the Einstein sequence would have sounded without the many-worlds sequence...
The Einstein sequence is a unique identifying number attached to an astronomical observation from the Einstein observatory.
If you mean something different, you should explain.
If you take a population of organisms, and you divide it arbitrarily into 2 groups, and you show the 2 groups to God and ask, “Which one of these groups is, on average, more fit?”, and God tells you, then you have been given 1 bit of information.
But if you take a population of organisms, and ask God to divide it into 2 groups, one consisting of organisms of above-average fitness, and one consisting of organisms of below-average fitness, that gives you a lot more than 1 bit. It takes n lg(n) bits to sort the population; then you subtract out the information needed to sort each half, so you gain n lg(n) − 2(n/2)lg(n/2) = n[lg(n) - lg(n/2)]
= nlg(2) = n bits.If you do tournament selection, you have n/2 tournaments, each of which gives you 1 bit, so you get n/2 bits per generation.
EO Wilson has a section in his autobiography, /Naturalist/, on what Gould and Lewontin did after the publication of Wilson’s /Sociobiology/. They formed a study group, which met every week to criticize Sociobiology, then after a few months, published their results.
The kicker is that they held their meetings about a 30-second walk from Wilson’s office in Harvard—but never told him about them.
This proves to me that science and truth never were their primary concern.