I’m sad the original FB posts were deleted. Now I can never show my kids the occasion where Eliezer endorsed a comment of mine =(
Gust
Karma: 170
Brain dump of a quick idea:
A sufficiently complex bridge law might say that the agent is actually a rock which, through some bizarre arbitrary encoding, encodes a computation[1]. Meanwhile the actual agent is somewhere else. Hopefully the agent has some adequate Occamian prior and he never assigns this hypothesis any relevance because of the high complexity of the encoding code.
In idea-space, though, there is a computation which is encoded by a rock using a complex arbitrary encoding, which, by virtue of having a weird prior, concludes that it actually is that rock, and to whom them breaking of the rock would mean death. We usually regard this computation as irrelevant for moral purposes—only computations corresponding to “natural” interpretations of physical phenomena count. But this distinction between natural and arbitrary criterions of interpretation seems, well, arbitrary.
We regard a person’s body as “executing” the computation that is a person that thinks she is in that body. But we do not regard the rock as actually “executing” the computation that is a weird agent that thinks it’s the computation encoded by the rock (through some arbitrary encoding).
Why?
The pragmatic reason is obvious: you can’t do anything to help the rock-computation in anyway, and whatever you do, you’d be lowering utility for some other ideal computation.
But maybe the kind of reasoning the rock-computation has to make to keep seeing itself as the rock is relevant.
A rock-as-computation hypothesis (I am this rock in this universe = my phenomenological experiences correspond to the states os atoms in several points of this rock as translated by this [insert very long bridge law, full of specifics] bridge law) is doomed to fail at the few next steps in the computation. Because the bridge law is so ad hoc, it won’t correctly predict the next phenomena perceived by the computation (in the ideal or real world where it actually executes). So if the rock-computation does induction at all, it will have to change the bridge law, and give up on being that rock.
In other words, if we built a robot with a prior that privileges the bridge law hypothesis that it’s a computation encoded in a rcok though some bizarre encoding, it would have to abandon that hypothesis very very soon. And, as phenomelogical data came in, it would approach the correct hypothesis that it’s the robot. Unless it’s doing some kind of privileged-hypothesis anti-induction where it keeps adopting increasingly complex bridge laws to keep believing it is that one rock.
So, proposal: a substrate is regarded to embody a computation-agent if that computation-agent is one that, by doing induction in the same sense we do to find out about ourselves, will eventually arrive at the correct bridge law that it is being executed in said substrate.
--
[1] Rock example is from DRESCHER, Gary, Good and Real, 2006, p. 55.
The ULH suggests that most everything that defines the human mind is cognitive software rather than hardware: the adult mind (in terms of algorithmic information) is 99.999% a cultural/memetic construct.
I think a distinction worth tracing here is the diferrence between “learning” in the neural-net-sense and “learning” in the human pedagogical/psychological sense.
The “learning” done by a piece of cortex becoming a visual cortex after receiving neural impulses from the eye isn’t something you can override by teaching a person (in the usual sense o the word “teaching”) - you’d need to rewire their brain. I don’t think you can call it cultural/memetic because this neural learning does not (seem to) occur through the mechanism(s) that deals with concepts, ideas and feelings, which is involved in learning a language or a social custom or a scientific theory.
In the same way, maybe the availability heuristic isn’t genetically coded, but is learned through the type of data certain parts of the brain have to work with. That would mean you could fix it through some input rewiring during gestation, but doesn’t mean you can change it through a new human education system—it may be too low level, like a generic part of the cortex becoming the visual cortex. If that’s the case, I wouldn’t say it’s a cultural/memetic construct (although it is an environmental construct).
Well, you’d have to hardcode at least a learning algorithm for values if you expect to have any real chance that the AI behaves like a useful agent, and that falls within the category of important functionalities. But then I guess you’ll agree with that.
You have to hardcode something, don’t you?
You’re a Brazilian studying Law who’s been around LW since 2013 and I’d never heard of you? Wow. Please show up!
Meetup : São Paulo, Brazil—Meetup at Base Sociedade Colaborativa
If you keep the project open source, I might be able help with the programming (although I don’t know much about Rails, I could help with the client side). The math is a mystery to me, too, but can’t you charge ahead with a simple geometric mean for the combination of estimates while you figure it out?
We’re translating to Brazilian Protuguese only, since that’s our native language.
Hi, and thanks for the awesome job! Will you keep a public record of changes you make to the book? I’m coordinating a translation effort, and that would be important to keep it in sync if you change the actual text, not just fix spelling and hyperlinking errors.
Edit: Our translation effort is for Portuguese only, and can be found at http://racionalidade.com.br/wiki .
Interesting idea. Brazilian law explicitly admits lottery as a form of settling, but I’m not sure if that example with a penalty for not winning a lawsuit would be admissible.
I guess I misunderstood what you meant by “There are many ways to tackle this question, but I mean this in a homo economicus, not biased perspective.” then. See my reply to ShardPhoenix.
He specifically said he’s talking about “homo economicus”-”rational”-like decision. An agent like that should have no need to punish itself—by having a negative emotion—since the potential loss of utility itself is a compelling reason to take action beforehand. So self-punishing is out. How do you think sadness would serve as a signalling device, in this case?
Not sure what you mean by “you SHOULD be sad when you miss an opportunity1”? What’s the advantage of being sad instead of just shrugging and replanning?
I’ve read Kolak’s Cognitive Science, which you recomended in that textbook list post. I’ve enjoyed it a lot and it didn’t feel like I needed some previous introductory reading. Any reason why you left it out now?
Awesome project. I really liked the facebook discussion, and this post explains clearly and concretely a part that some people found confusing. Very well written. Congratulation, Robb.
This just feels really promising, although I can’t say I’ve really followed it all (you’ve lost me a couple posts ago on the math, but that’s my fault). I’m waiting eagerly for the re-post.
All the content in the post just fell in place after I read Giles summary. Still a great post, though.
Necessary entities, Moses ben Maimonides Anselm’s ontological, Summa Theologica I think these are switched.
The way I see it, having intuitions and trusting them is not necessarily harmful. But you should actually recognize them by what they are: snap judgements made by subconscious heuristics that have little to do with actual arguments you come up with. That way, you can take it as a kind of evidence/argument, instead of a Bottom Line—like an opinion from a supposed expert which tells you the “X is Y”, but doesn’t have the time to explain. You can then ask: “is this guy really an expert?” and “do other arguments/evidence outweight the expert’s opinion?”