Tim_Tyler

Karma: 217

Tim_Tyler Jan 30, 2009, 10:35 PM
−1 points
on: Value is Fragile
But there is no principled way to derive an utility function from something that is not an expected utility maximizer!

You can model any agent as in expected utility maximizer—with a few caveats about things such as uncomputability and infinitely complex functions.

You really can reverse-engineer their utility functions too—by considering them as Input-Transform-Output black boxes—and asking what expected utility maximizer would produce the observed transformation.

A utility function is like a program in a Turing-complete language. If the behaviour can be computed at all, it can be computed by a utility function.

Tim_Tyler Jan 30, 2009, 10:22 PM
−1 points
on: Value is Fragile
Another way of saying this is that human beings are not expected utility maximizers, not as individuals and certainly not as societies.

They are not perfect expected utility maximizers. However, no expected utility maximizer is perfect. Humans approach the ideal at least as well as other organisms. Fitness maximization is the central explanatory principle in biology—and the underlying idea is the same. The economic framework associated with utilitarianism is general, of broad applicability, and deserves considerable respect.

Tim_Tyler Jan 30, 2009, 6:50 PM
−1 points
0
on: Value is Fragile
I agree with Eliezer that an imprecisely chosen value function, if relentlessly optimized, is likely to yield a dull universe.

So: you think a “paperclip maximiser” would be “dull”?

How is that remotely defensible? Do you think a “paperclip maximiser” will master molecular nanotechnology, artificial intelligence, space travel, fusion, the art of dismantling planets and stellar farming?

If so, how could that possibly be “dull”? If not, what reason do you have for thinking that those technologies would not help with the making of paper clips?

Apparently-simple processes can easily produce great complexity. That’s one of the lessons of Conway’s game.

Tim_Tyler Jan 29, 2009, 10:55 PM
−2 points
on: Value is Fragile
Thanks for the probability assessments. What is missing are supporting arguments. What you think is relatively clear—but why you think it is not.

...and what’s the deal with mentioning a “sense of humour”? What has that to do with whether a civilization is complex and interesting? Whether our distant descendants value a sense of humour or not seems like an irrelevance to me. I am more concerned with whether they “make it” or not—factors affecting whether our descendants outlast the exploding sun—or whether the seed of human civilisation is obliterated forever.

Tim_Tyler Jan 29, 2009, 6:40 PM
−2 points
0
on: Value is Fragile
This post seems almost totally wrong to me. For one thing, its central claim—that without human values the future would, with high probability be dull is not even properly defined.

To be a little clearer, one would need to say something like: if you consider a specified enumeration over the space of possibile utility functions, a random small sample from that space would be “dull” (it might help to say a bit more about what dullness means too, but that is a side issue for now).

That claim might well be true for typical “shortest-first” enumerations in sensible languages—but it is not a very interesting claim—since the dull utility functions would be those which led to an attainable goal—such as “count up to 10 and then stop”.

The “open-ended” utilility functions—the ones that resulted in systems that would spread out—would almost inevitably lead to rich complexity. You can’t turn the galaxy into paper-clips (or whatever) without extensively mastering science, technology, intergalactic flight, nanotechnology—and so on. So, you need scientists and engineers—and other complicated and interesting things. This conclusion seems so obvious as to hardly be worth discussing to me.

I’ve explained all this to Eleizer before. After reading this post I still have very little idea about what it is that he isn’t getting. He seems to think that making paper clips are boring. However, they are not any more boring than making DNA sequences, and that’s the current aim of most living systems.

A prime-seeking civilisation has a competitive disadvantage over one that doesn’t have silly, arbitrary bits tacked on to its utility function. It is more likely to be wiped out in a battle with an alien race—and it’s more likely to suffer from a mutiny from within. However, that is about all. They are unlikely to lack science, technology, or other interesting stuff.

Tim_Tyler Jan 20, 2009, 9:37 PM
3 points
on: Sympathetic Minds
The core of most of my disagreements with this article find their most concentrated expression in:

“Happiness” is an idiom of policy reinforcement learning, not expected utility maximization.

Under Omohundro’s model of intelligent systems, these two approaches converge. As they do so, the reward signal of reinforcement learning and the concept of expected utility also converge. In other words, it is rather inappropriate to emphasize the differences between these two systems as though it was a fundamental one.

There are differences—but they are rather superficial. For example, there is often a happiness “set point”, for example—whereas that concept is typically more elusive for an expected utility maximizer. However, the analogies between the concepts are deep and fundamental: an agent maximising its happiness is doing something deeply and fundamentally similar to an agent maximising its expected utility. That becomes obvious if you substitute “happiness” for “expected utility”.

In the case of real organisms, that substitution is doubly appropriate—because of evolution. The “happiness” function is not an arbitrarily chosen one—it is created in such a way that it converges closely on a function that favours behaviour resulting in increased expected ancestral representation. So, happiness gets an “expectation” of future events built into it automatically by the evolutionary process.

Tim_Tyler Jan 18, 2009, 11:43 AM
1 point
on: BHTV: de Grey and Yudkowsky
Here is Aubrey on the topic of cryonics.

Tim_Tyler Jan 12, 2009, 8:11 PM
1 point
on: Eutopia is Scary
Dictionaries disagree with Ferris—e.g.:

“Happiness [...] antonym: sadness”—Encarta

“Boredom” makes a terrible opposite of “happiness”. What is the opposite of boredom? Something interesting, to be sure, but many more things than just happiness fit that description.

Tim_Tyler Jan 10, 2009, 11:06 AM
4 points
on: Serious Stories
David Pearce has written extensively on the topic of the elimination of suffering—e.g. see: THE ABOLITIONIST PROJECT and Paradise Engineering.

Tim_Tyler Jan 8, 2009, 9:07 PM
1 point
on: Living By Your Own Strength
Eliezer, I think you have somehow gotten very confused about the topic of my now-deleted post.

That post was entirely about cultural inheritance—contained absolutely nothing about sexual selection.

Please don’t delete my posts—unless you have a good reason for doing so.

Tim_Tyler Jan 8, 2009, 8:12 PM
2 points
on: Living By Your Own Strength
[Deleted. Tim, you’ve been requested to stop talking about your views on sexual selection here. --EY]

Tim_Tyler Jan 7, 2009, 8:13 PM
0 points
on: Emotional Involvement
As I recall, Arnold’s character faced pretty-much this dilemma in Total Recall.

There’s a broadly-similar episode of Buffy the Vampire Slayer.

Both characters wind up going on with their mission of saving the planet.

Tim_Tyler Jan 7, 2009, 4:59 PM
2 points
on: Emotional Involvement
General Optimizer, you seem like a prospect for responding to this question: “in the interests of transparency, would anyone else like to share what they think their utility function is?”

Tim_Tyler Jan 5, 2009, 6:52 PM
2 points
on: Changing Emotions
Intelligent machines will not really be built “from scratch” because augmentation of human intelligence by machines makes use of all the same technology as is present is straight machine intelligence projects, plus a human brain. Those projects have the advantage of being competitive with humans out of the box—and they interact synergetically with traditional machine intelligence projects. For details see my intelligence augmentation video/essay.

The thing that doesn’t make much sense is building directly on the human brain’s wetware with more of the same. Such projects are typically banned at the moment—and face all kinds of technical problems anyway.

Tim_Tyler Jan 4, 2009, 7:25 PM
0 points
on: Growing Up is Hard
It seems like a general property of an intelligent system that it can’t know everything about how with would react to everything. That falls out of the halting theorem (and for that matter Godel’s first incompleteness theorem) fairly directly.

Er, no, it doesn’t.

Tim_Tyler Jan 4, 2009, 2:27 PM
1 point
on: Growing Up is Hard
Robin, it sounds as though you are thinking about the changes that could be made after brain digitalisation.

That seems like a pretty different topic to me. Once you have things in a digital medium, it is indeed much easier to make changes—even though you are still dealing with a nightmarish mess of hacked-together spaghetti code.

Tim_Tyler Jan 4, 2009, 11:39 AM
1 point
on: Growing Up is Hard
This is one reason to be wary of, say, cholinergic memory enhancers: if they have no downsides, why doesn’t the brain produce more acetylcholine already?

There’s considerable scope for the answer to this question being: “because of resource costs”. Resource costs for nutrients today are radically different from those in the environment of our ancestors.

We are not designed for our parts to be upgraded. Our parts are adapted to work exactly as they are, in their current context, every part tested in a regime of the other parts being the way they are.

That’s true—but things are not quite as bad as that makes it sound. Evolution is concerned with things like modularity and evolvability. Those contribute to the modularity of our internal organs—and that helps explain why things like kidney transplants work. Evolution didn’t plan for organ transplant operations—but it did arrange things in a modular fashion. Modularity has other benefits—and ease of upgrading and replacement is a side effect.

People probably broke in the ancestral environment too. Organisms are simply fragile, and most fail to survive and reproduce.

Another good popular book on the evolution of intelligence is “The Runaway Brain”. I liked it, anyway. I also have time for Sue Blackmore’s exposition on the topic, in “The Meme Machine”.

“Hm… to get from a chimpanzee to a human… you enlarge the frontal cortex… so if we enlarge it even further...” The road to +Human is not that simple.

Well, we could do that. Cesarian sections, nutrients, drugs, brain growth factor gene therapy, synthetic skulls, brains-in-vats—and so on.

It would probably only add a year or so onto the human expiration date, but it might be worth doing anyway—since the longer humans remain competitive for, the better the chances of a smooth transition. The main problem I see is the “yuck” factor—people don’t like looking closely at that path.

Tim_Tyler Jan 3, 2009, 4:55 PM
0 points
on: The Uses of Fun (Theory)
My guess is that it’s a representation of my position on sexual selection and cultural evolution. I may still be banned from discussing this subject—and anyway, it seems off-topic on this thread, so I won’t go into details.

If this hypothesis about the comment is correct, the main link that I can see would be: things that Eliezer and Tim disagree about.

Tim_Tyler Jan 2, 2009, 11:46 PM
0 points
on: The Uses of Fun (Theory)
Well, that is so vague as to hardly be worth the trouble of responding to—but I will say that I do hope you were not thinking of referring me here.

However, I should perhaps add that I overspoke. I did not literally mean “any sufficiently-powerful optimisation process”. Only that such things are natural tendencies—that tend to be produced unless you actively wire things into the utility function to prevent their manifestation.

Tim_Tyler Jan 2, 2009, 9:02 PM
−7 points
on: The Uses of Fun (Theory)
Going into the details of Fun Theory helps you see that eudaimonia is actually complicated—that there are a lot of properties necessary for a mind to lead a worthwhile existence. Which helps you appreciate just how worthless a galaxy would end up looking (with extremely high probability) if it was optimized by something with a utility function rolled up at random.

Something with a utility function “rolled at random” typically does not “optimise the universe”. Rather it dies out. Of those agents with utility functions that do actually spread themselves throughout the universe, it is not remotely obvious that most of them are “worthless” or “uninteresting”—unless you choose to define the term “worth” so that this is true, for some reason.

Indeed, rather the opposite—since such agents would construct galactic-scale civilisations, they would probably be highly interesting and valuable instances of living systems in the universal community.

Complex challenges? Novelty? Individualism? Self-awareness? Experienced happiness? A paperclip maximizer cares not about these things.

Sure it would: as proximate goals. Animals are expected gene-fitness maximisers. Expected gene-fitness is not somehow intrinsically more humane than expected paperclip number. Both have about the same chance of leading to the things you mentioned being proximate goals.

Novelty-seeking and self-awareness are things you get out of any sufficiently-powerful optimisation process—just as they all develop fusion, space travel, nanotechnology—and so on.