CoffeeStain

Karma: 343

CoffeeStain Sep 12, 2014, 11:38 PM
2 points
in reply to: Shmi’s comment on: Rationality Quotes September 2014
Living in the same house and coordinating lives isn’t a method for ensuring that people stay in love; being able to is proof that they are already in love. An added social construct is a perfectly reasonable option to make it harder to change your mind.

CoffeeStain Jul 28, 2014, 9:54 PM
4 points
in reply to: Richard_Kennaway’s comment on: Confused as to usefulness of ‘consciousness’ as a concept

It sometimes seems to me that those of us who actually have consciousness are in a minority, and everyone else is a p-zombie.

When I myself run across apparent p-zombies, they usually look at my arguments as if I am being dense over my descriptions of consciousness. And I can see why, because without the experience of consciousness itself, these arguments must sound like they make consciousness out to be an extraneous hypothesis to help explain my behavior. Yet, even after reflecting on this objection, it still seems there is something to explain besides my behavior, which wouldn’t bother me if I were only trying to explain my behavior, including the words in this post.

It makes sense to me that from outside a brain, everything in the brain is causal, and the brain’s statements about truths are dependent on outside formalizations, and that everything observable about a brain is reducible to symbolic events. And so an observation of a zombie-Chalmers introspecting his consciousness would yield no shocking insights on the origins of his English arguments. And I know that when I reflect on this argument, an observer of my own brain would also find no surprising neural behaviors.

But I don’t know how to reconcile this with my overriding intuition/need/thought that I seek not to explain my behavior but the sense experience itself when I talk about it. Fully aware of outside view functionalism, the sensation of red still feels like an item in need of explanation, regardless of which words I use to describe it. I also feel no particular need to feel that this represents a confusion, because the sense experience seems to demand that it place itself in another category than something you would explain functionally from the outside. All this I say even while I’m aware that to humans without this feeling, these claims seem nothing like insane, and they will gladly inspect my brain for a (correct) functional explanation of my words.

The whole ordeal still greatly confuses me, to an extent that surprises me given how many other questions have been dissolved on reflection such as, well, intelligence.

CoffeeStain Jul 26, 2014, 6:44 PM
1 point
on: Knightian Uncertainty and Ambiguity Aversion: Motivation

Perhaps abiguity aversion is merely a good heuristic.

Well of course. Finite ideal rational agents don’t exist. If you were designing decision-theory-optimal AI, that optimality is a property of its environment, not any ideal abstract computing space. I can think of at least one reason why ambiguity aversion could be the optimal algorithm in environments with limited computing resources:

Consider a self-modification algorithm that adapts to new problem domains. Restructuring (learning) is considered the hardest of tasks, and so the AI modifies scarcely. Thus, as it encounters new decision-theoretic problems, it often does not choose self-modification, instead clodging together old circuitry and/or answers to conserve compute cycles. And so when choosing answers to your 3 problems, it would fear solutions which, when repeating the answer multiple times, maximizes expected value in its environment, which includes its own source code.

Ambiguity aversion then would be commitment-risk aversion, where future compounded failures change the value of dollars per ulility. Upon each iteration of the problem, the value of a dollar can change, and if you don’t maximize minimum expected value, you may end up with betting all of your $100, which is worth infinite value to you, versus gaining $100, which is worth far less, even if you started with $1000.

We see this in ourselves all the time. If you make a decision, expect to be more likely to make the decision in the future, and if you change your lifestyle, expect it to be hard to change back, even if you later know that changing back is the deletion of a bias.

And if so, do we need a different framework that can capture a broader class of “rational” agents, including maximizers of minimum expected utility?

Rational agents have source code whose optimality is a function of their environments. There is no finite cross-domain Bayesian in compute-space; only in the design-space that includes environments.

CoffeeStain Apr 7, 2014, 3:17 AM
−1 points
on: Items to Have In Case of Emergency...Maybe.
Shouldn’t this post be marked [Human] so that uploads and AIs don’t need to spend cycles reading it?

...I’d like to think that this joke bears the more subtle point that a possible explanation for the preparedness gap in your rationalist friends is that they’re trying to think like ideal rational agents, who wouldn’t need to take such human considerations.

CoffeeStain Mar 22, 2014, 11:54 PM
5 points
on: Open thread, 18-24 March 2014
I have a friend with Crohn’s Disease, who often struggles with the motivation to even figure out how to improve his diet in order to prevent relapse. I suggested he should find a consistent way to not have to worry about diet, such as prepared meals, a snack plan, meal replacements (Soylent is out soon!), or dietary supplement.

As usual, I’m pinging the rationalists to see if there happens to be a medically inclined recommendation lurking about. Soylent seems promising, and doesn’t seem the sort of thing that he and his doctor would have even discussed. My appraisal of his doctor consulations seem to be something along the lines of “You should track your diet according to these guidelines, and try to see what causes relapse” rather than “Here’s a cure all solution not entirely endorsed by the FDA that will solve all of your motivational and health problems in one fell swoop.” For my friend, drilling into sweeping diet changes and tracking seems like an insurmountable challenge, especially with the depression caused by simply having the disease.

I’d like to be able to purchase something for him that would let him go about his life without having to worry about it so much. Any ideas on whether Soylent could be the solution, in particular as to its potential for Crohn’s?

CoffeeStain Feb 1, 2014, 10:58 AM
7 points
in reply to: djm’s comment on: Humans can drive cars

There has been mathematically proven software and the space shuttle came close though that was not proven as such.

Well… If you know what you wish to prove then it’s possible that there exists a logical string that begins with a computer program and ends with it as a necessity. But that’s not really exciting. If you could code in the language of proof-theory, you already have the program. The mathematical proof of a real program is just a translation of the proof into machine code and then showing it goes both ways.

You can potentially prove a space shuttle program will never crash, but you can’t prove the space shuttle won’t crash. Source code is just source code, and bugs aren’t always known to be such without human reflection and real world testing. The translation from intent to code is what was broken in the first place, you actually have to keep applying more intent in order to fix it.

The problem with AGI is that the smartest people in the world write reams trying to say what we even wish to prove, and we’re still sort of unsure. Most utopias are dystopias, and it’s hard to prove a eutopia, because eutopias are scary.

CoffeeStain Jan 17, 2014, 2:33 AM
6 points
in reply to: CronoDAS’s comment on: Things I Wish They’d Taught Me When I Was Younger: Why Money Is Awesome
Depends if you count future income. Highest paying careers are often so because only those willing to put in extra effort at their previous jobs get promoted. This is at least true in my field, software engineering.

CoffeeStain Jan 8, 2014, 7:28 AM
4 points
on: A big Singularity-themed Hollywood movie out in April offers many opportunities to talk about AI risk
The film’s trailer strikes me as being aware of the transhumanist community in a surprising way, as it includes two themes that are otherwise not connected in the public consciousness: uploads and superintelligence. I wouldn’t be surprised if a screenwriter found inspiration from the characters of Sandberg, Bostrom, or of course Kurzweil. Members of the Less Wrong community itself have long struck me as ripe for fictionalization… Imagine if a Hollywood writer actually visited.

CoffeeStain Dec 8, 2013, 7:40 AM
2 points
in reply to: Risto_Saarelma’s comment on: December Monthly Bragging Thread

They can help with depression.

I’ve personally tried this and can report truth, but will caveat that the expectation that I will force myself into a morning cold shower often causes oversleeping, which rather exacerbates depression.

CoffeeStain Nov 25, 2013, 1:25 AM
3 points
in reply to: David_Chapman’s comment on: Probability and radical uncertainty

Often in Knightian problems you are just screwed and there’s nothing rational you can do.

As you know, this attitude isn’t particularly common ’round these parts, and while I fall mostly in the “Decision theory can account for everything” camp, there may still be a point there. “Rational” isn’t really a category so much as a degree. Formally, it’s a function on actions that somehow measures how much that action corresponds to the perfect decision-theoretic action. My impression is that somewhere there’s Godelian consideration lurking, which is where the “Omega fines you exorbitantly for using TDT” thought experiment comes into play.

That thought experiment never bothered me much, as it just is what it is: a problem where you are just screwed, and there’s nothing rational you can do to improve your situation. You’ve already rightly programmed yourself to use TDT, and even your decision to stop using TDT would be made using TDT, and unless Omega is making exceptions for that particular choice (in which case you should self-modify to non-TDT), Omega is just a jerk that goes around fining rational people.

In such situations, the words “rational” and “irrational” are less useful descriptors than just observing source code being executed. If you’re formal about it using metric R, then you would be more R, but its correlation to “rational” wouldn’t really be at point.

But in this case, again, I think there’s a straightforward, simple, sensible approach (which so far no one has suggested...)

So, I don’t think the black box is really one of the situations I’ve described. It seems to me a decision theorist training herself to be more generally rational is in fact improving her odds at winning the black box game. All the approaches outlined so far do seem to also improve her odds. I don’t think a better solution exists, and she will often lose if she lacks time to reflect. But the more rational she is, the more often she will win.

CoffeeStain Nov 24, 2013, 11:24 PM
2 points
in reply to: David_Chapman’s comment on: Probability and radical uncertainty

Part of the motivation for the black box experiment is to show that the metaprobability approach breaks down in some cases.

Ah! I didn’t quite pick up on that. I’ll note that infinite regress problems aren’t necessarily defeaters of an approach. Good minds that could fall into that trap implement a “Screw it, I’m going to bed” trigger to keep from wasting cycles even when using an otherwise helpful heuristic.

Maybe the thought experiment ought to have specified a time limit. Personally, I don’t think enumerating things the boxes could possibly do would be helpful at all. Isn’t there an easier approach?

Maybe, but I can’t guarantee you won’t get blown up by a black box with a bomb inside! As a friend, I would be furiously lending you my reasoning to help you make the best decision, worrying very little what minds better and faster than both of ours would be able to do.

It is, at the end of the day, just the General AI problem: Don’t think too hard on brute-force but perfect methods or else you might skip a heuristic that could have gotten you an answer within the time limit! But when do you know whether the time limit is at that threshold? You could spend cycles on that too, but time is wasting! Time limit games presume that the participant has already underwent a lot of unintentional design (by evolution, history, past reflections, etc.). This is the “already in-motion” part which, frustratingly, cannot ever be optimal unless somebody on the outside designed you for it. It’s a formal problem what source code performs best under what game. Being a source code involves taking the discussion we’re having now and applying it the best you can, because that’s what your source code does.

CoffeeStain Nov 24, 2013, 11:08 PM
3 points
in reply to: Gunnar_Zarncke’s comment on: Probability and radical uncertainty

But the point about meta probability is that we do not have the nodes. Each meta level corresponds to one nesting of networks in nodes.

Think of Bayesian graphs as implicitly complete, with the set of nodes being every thing to which you have a referent. If you can even say “this proposition” meaningfully, a perfect Bayesian implemented as a brute-force Bayesian network could assign it a node connected to all other nodes, just with trivial conditional probabilities that give the same results as an unconnected node.

A big part of this discussion has been whether some referents (like black boxes) actually do have such trivial conditional probabilities which end up returning an inference of 50%. It certainly feels like some referents should have no precedent, and yet it also feels like we still don’t say 50%. This is because they actually do have precedent (and conditional probabilities), it’s just that our internal reasonings are not always consciously available.

CoffeeStain Nov 24, 2013, 10:53 PM
5 points
in reply to: Vaniver’s comment on: Probability and radical uncertainty
It is helpful, and was one of the ways that helped me to understand One-boxing on a gut level.

And yet, when the problem space seems harder, when “optimal” becomes uncomputable and wrapped up in the fact that I can’t fully introspect, playing certain games doesn’t feel like designing a mind. Although, this is probably just due to the fact that games have time limits, while mind-design is unconstrained. If I had an eternity to play any given game, I would spend a lot of time introspecting, changing my mind into the sort that could play iterations of the game in smaller time chunks. Although there would still always be a part of my brain (that part created in motion) that I can’t change. And I would still use that part to play the black box game.

In regards to metaprobabilities, I’m starting to see the point. I don’t think it alters any theory about how probablity “works,” but its intuitive value could be evidence that optimal AIs might be able to more efficiently emulate perfect decision theory with CalcMetaProbability implemented. And it’s certainly useful to many here.

CoffeeStain Nov 24, 2013, 5:55 AM
4 points
in reply to: David_Chapman’s comment on: Probability and radical uncertainty
“How often do listing sorts of problems with some reasonable considerations result in an answer of ‘None of the above’ for me?”

If “reasonable considerations” are not available, then we can still:

“How often did listing sorts of problems with no other information available result in an answer of ‘None of the above’ for me?”

Even if we suppose that maybe this problem bears no resemblance to any previously encountered problem, we can still (because the fact that it bears no resemblance is itself a signifier):

“How often did problems I’d encountered for the first time have an answer I never thought of?”

CoffeeStain Nov 24, 2013, 5:49 AM
5 points
in reply to: David_Chapman’s comment on: Probability and radical uncertainty
My LessWrongian answer is that I would ask my mind that was created already in motion what the probability is, then refine it with as many further reflections as I can come up with. Embody an AI long enough in this world, and it too will have priors about black boxes, except that reporting that probability in the form of a number is inherent to its source code rather than strange and otherworldly like it is for us.

The point that was made in that article (and in the Metaethics sequence as a whole) is that the only mind you have to solve a problem is the one that you have, and you will inevitably use it to solve problems unoptimally, where “unoptimal” if taken strictly means everything anybody has ever done.

The reflection part of this is important, as it’s the only thing we have control over, and I suppose could involve discussions about metaprobabilities. It doesn’t really do it for me though, although I’m only just a single point in the mind design space. To me, metaprobability seems isomorphic to a collection of reducible considerations, and so doesn’t seem like a useful shortcut or abstraction. My particular strategy for reflection would be something like that in dspeyer’s comment, things such as reasoning about the source of the box, possibilities for what could be in the box that I might reasonably expect to be there. Depending on how much time I have, I’d be very systematic about it, listing out possibilities, solving infinite series on expected value, etc.

CoffeeStain Nov 24, 2013, 3:29 AM
5 points
on: Probability and radical uncertainty
The idea of metaprobability still isn’t particularly satisfying to me as a game-level strategy choice. It might be useful as a description of something my brain already does, and thus give me more information about how my brain relates to or emulates an AI capable of perfect Bayesian inference. But in terms of picking optimal strategies, perfect Bayesian inference has no subroutine called CalcMetaProbability.

My first thought was that your approach elevates your brain’s state above states of the world as symbols in the decision graph, and calls the difference “Meta.” By Luke’s analogy, information about the black box is unstable, but all that means is that the (yes, single) probability value we get when we query the Bayesian network is conditionally dependent on nodes with a high degree of expected future change (including many nodes referring to your brain). If you maintain discipline and keep yourself (and your future selves) as a part of the system, you can as perfectly calculate your current self’s expected probability without “metaprobability.” If you’re looking to (losslessly or otherwise) optimize your brain to calculate probabilities, then “metaprobability” is a useful concept. But then we’re no longer playing the game, we’re designing minds.

CoffeeStain Nov 23, 2013, 11:22 PM
2 points
in reply to: Scott Alexander’s comment on: 2013 Less Wrong Census/Survey
Right down the middle: 25-75

CoffeeStain Nov 23, 2013, 12:30 AM
0 points
in reply to: Nornagest’s comment on: 2013 Less Wrong Census/Survey
Hmm, come to think of it, deciding the size of the cash prize (for it being interesting) is probably worth more to me as well. I’ll just have to settle for boring old cash.

CoffeeStain Nov 23, 2013, 12:04 AM
15 points
on: 2013 Less Wrong Census/Survey
I defected, because I’m indifferent to whether the prize-giver or prize-winner has 60 * X dollars, unless the prize-winner is me.

CoffeeStain Oct 31, 2013, 8:33 AM
0 points
in reply to: timujin’s comment on: The best 15 words

Am I walking the wrong path?

Eh, probably not. Heuristically, I shy away from modes of thought that involve intentional self-deception, but that’s because I haven’t been mindful of myself long enough to know ways I can do this systematically without breaking down. I would also caution against letting small-scale pride translate into larger domains where there is less available evidence for how good you really are. “I am successful” has a much higher chance of becoming a cached self than “I am good at math.” The latter is testable with fewer bits of evidence, and the former might cause you to think you don’t need to keep trying.

As for other-manipulation, it seems the confidence terminology can apply to social dominance as well. I don’t think desiring superior charisma necessitates an actual belief in your awesomeness compared to others, just the belief that you are awesome. The latter to me is more what it feels like to be good at being social, and has the benefit of not entrenching a distance from others or the cached belief that others are useful manipulation targets rather than useful collaborators.

People vary on how they can use internal representations to produce results. It’s really hard to use probabilistic distributions on outcomes as sole motivator for behavior, so we do need to cache beliefs in the language of conventional social advice sometimes. The good news is that good people who are non-rationalists are a treasure trove for this sort of insight.