I guess I just don’t see it as a weak point in the doom argument that goal-orientedness is a convergent attractor in the space of self-modifying intelligences?
It feels similar to pondering the familiar claim of evolution, that systems that copy themselves and seize resources are an attractor state. Sure it’s not 100% proven but it seems pretty solid.
Context is a huge factor in all these communications tips. The scenario I’m optimizing for is when you’re texting someone who has a lot of options, and you think it’s high expected value to get them to invest in a date with you, but the most likely way that won’t happen is if they hesitate to reply to you and tap away to something else. That’s not always the actual scenario though.
Imagine you’re the recipient, and the person who’s texting you met your minimum standard to match with, but is still a-priori probably not worth your time and effort going on a date with, because their expected attractiveness+compatibility score is too low, though you haven’t investigated enough to be confident yet. (This is a common epistemic state of e.g. a woman with attractive pics on a dating app that has more male users.)
Maybe the first match who asks you “how’s your week going” feels like a nice opportunity to ramble how you feel, and a nice sign that someone out there cares. But if that happens enough on an app, and the average date-worthiness of the people that it happens with is low, then the next person who sends it doesn’t make you want to ramble anymore. Because you know from experience that rambling into a momentumless conversation will just lead it to stagnate in its next momentumless point.
It’s nice when people care about you, but it quickly gets not so nice when a bunch of people with questionable date-appeal are trying to trade a cheap care signal for your scarce attention and dating resources.
If the person sending you the message has already distinguished themselves to you as “dateworthy”, e.g. by having one of the best pics and/or profile in your judgment, then “How’s your week going” will be a perfectly adequate message from them; in some cases maybe even an optimal message. You can just build rapport and check for basic red flags, then set up a date.
But if you’re not sold on the other person being dateworthy, and they start out from a lower-leverage position in the sense that they initially consider you more dateworthy than you consider them, then they better send a message that somehow adds value to you, to help them climb the dateworthiness gap.
But again, context is always the biggest factor, and context has a lot of detail. E.g. if you don’t consider someone dateworthy, but you’re in a scenario where someone just making conversation with you is adding value to you (e.g. not a ton of matches demanding your attention using the same unoriginal rapport-building gambit), then “How’s it going” can work great.
This is actually the default context if you’re brave enough to approach strangers you want to date in meatspace. The stranger can be much more physically attractive or higher initially-perceived dating market value than you. Yet just implicitly signaling your social confidence through boldness, body language, and friendly/fun way of speaking and acting, raises your dateworthiness significantly, and the real-world-interaction modality doesn’t have much competition these days, so the content of the conversation that leads up to a date can be super normal smalltalk like “How’s it going”.
Yeah nice. A statement like “I’m looking for something new to watch” lowers the stakes by making the interaction more like what friends talk about rather than about an interview for a life partner, increasing the probability that they’ll respond rather than pausing for a second and ending up tapping away.
You can do even more than just lowering the stakes if you inject a sense that you’re subconsciously using the next couple conversation moves to draw out evidence about the conversation partner, because you’re naturally perceptive and have various standards and ideas about people you like to date, and you like to get a sense of who the other person is.
If done well, this builds a curious sense that the question is a bit more than just making formulaic conversation, but somehow has momentum to it. The best motivation for someone to keep talking to you on a dating app is if they feel they’re being seen by a savvy evaluator who will reflect back a valuable perspective about them. The person talking to you can then be subconsciously thinking about how attractive/interesting/unique/etc they are (an engaging experience). Also, everyone wants to feel like they’re maximizing their potential by finding someone to date who’s in the upper range of their “league”, and there are ways to engage in conversation that are more consistent with that ideal.
IMO the best type of conversation to have after a few opening back&forths, is to get them talking about something they find engaging, which is generally also something that reflects them in a good light, which makes it fun and engaging for them while also putting you in a position to give a type of casual “feedback”, ultimately leading up to a statement of interest which shows them why you’re not just another random match but rather someone they have more reason to meet and not flake on. Your movie question could be a good start toward discovering something like that, but probably not an example of that unless they’re a big movie person.
I’d try to look at their profile to clues of something they do in their life where they make an effort that someone ought to notice and appreciate, and get em talking about that.
Those are just some thoughts I have about how to distinguish yourself in the middle part of the conversation between opening interest and asking them on a date.
So you simply ask them: “What do you want to do”? And maybe you add “I’m completely fine with anything!” to ensure you’re really introducing no constraints whatsoever and you two can do exactly what your friend desires.
This error reminds me of people on a dating app who kill the conversation by texting something like “How’s your week going?”
When texting on a dating app, if you want to keep the conversation flowing nicely instead of getting awkward/strained responses or nothing, I believe the key is to anticipate that a couple seconds of low-effort processing on the recipient’s part will allow them to start typing their response to your message.
“How’s your week going?” is highly cognitively straining. Responding to it requires remembering and selecting info about one’s week (or one’s feelings about one’s week), and then filtering or modifying the selection so as to make one sound like an interesting conversationalist rather than an undifferentiated bore, while also worrying that one’s selection about how to answer doesn’t implicitly reveal them as being too eager to brag, or complain, or obsess about a particular topic.
You can be “conversationally generous” by intentionally pre-computing some of their cognitive work, i.e. narrowing the search space. For instance:
“I’m gonna try cooking myself 3 eggs/day for lunch so I don’t go crazy on DoorDash. How would you cook them if you were me?”
With a text like this (ideally adjusted to your actual life context), they don’t have to start by narrowing down a huge space of possible responses. They can immediately just ask themselves how they’d go about cooking an egg. And they also have some context of “where the conversation is going”: it’s about your own lifestyle. So it’s not just two people interviewing each other, it has this natural motion/momentum.
Using this computational kindness technique is admittedly kind of contrived on your end, but on their end, it just feels effortless and serendipitous. For naturally contrived nerds like myself looking for a way to convert IQ points into social skills, it’s a good trade.
The computational kindness principle in these conversations works much like the rule of improv that says you’re supposed to introduce specific elements to the scene (“My little brown poodle is digging for his bone”) rather than prompting your scene partners to do the cognitive work (“What’s that over there?”).
Oh and all this is not just a random piece of advice, it’s yet another Specificity Power.
Your baseline scenario (0 value) thus assumes away the possibility that civilization permanently collapses (in some sense) in the absence of some path to greater intelligence (whether via AI or whatever else), which would also wipe out any future value. This is a non-negligible possibility.
Yes, my mainline no-superintelligence-by-2100 scenario is that the trend toward a better world continues to 2100.
You’re welcome to set the baseline number to a negative, or tweak the numbers however you want to reflect any probability of a non-ASI existential disaster happening before 2100. I doubt it’ll affect the conclusion.
To be honest the only thing preventing me from granting paperclippers as much or more value than humans is uncertainty/conservatism about my metaethics
Ah ok, the crux of our disagreement is how much you value the paperclipper type scenario that I’d consider a very bad outcome. If you think that outcome is good then yeah, that licenses you in this formula to conclude that rushing toward AI is good.
Founder here :) I’m biased now, but FWIW I was also saying the same thing before I started this company in 2017: a good dating/relationship coach is super helpful. At this point we’ve coached over 100,000 clients and racked up many good reviews.
I’ve personally used a dating coach and a couples counselor. IMO it helps twofold:
Relevant insights and advice that the coach has that most people don’t, e.g. in the domain of communication skills, common tactics that best improve a situation, pitfalls to avoid.
A neutral party who’s good at letting you (and potentially a partner) objectively review and analyze the situation.
Relationship Hero hires, measures and curates the best coaches, and streamlines matching you to the best coach based on your scenario. Here’s a discount link for LW users to get $50 off.
Personally I just have the habit of reaching for specifics to begin my communication to help make things clear. This post may help.
Unlike the other animals, humans can represent any goal in a large domain like the physical universe, and then in a large fraction of cases, they can think of useful things to steer the universe toward that goal to an appreciable degree.
Some goals are more difficult than others / require giving the human control over more resources than others, and measurements of optimization power are hard to define, but this definition is taking a step toward formalizing the claim that humans are more of a “general intelligence” than animals. Presumably you agree with this claim?
It seems the crux of our disagreement factors down to a disagreement about whether this Optimization Power post by Eliezer is pointing at a sufficiently coherent concept.
I don’t get what point you’re trying to make about the takeaway of my analogy by bringing up the halting problem. There might not even be something analogous to the halting problem in my analogy of goal-completeness, but so what?
I also don’t get why you’re bringing up the detail that “single correct output” is not 100% the same thing as “single goal-specification with variable degrees of success measured on a utility function”. It’s in the nature of analogies that details are different yet we’re still able to infer an analogous conclusion on some dimension.
Humans are goal-complete, or equivalently “humans are general intelligences”, in the sense that many of us in the smartest quartile can output plans with the expectation of a much better than random score on a very broad range of utility functions over arbitrary domains.
These 4 beefs are different and less serious than the original accusations, or at least feel that way to me. Retconning a motte after the bailey is lost? That said, they’re reasonable beefs for someone to have.
I’m not saying “mapping a big category to a single example is what it’s all about”. I’m saying that it’s a sanity check. Like why wouldn’t you be able to do that? Yet sometimes you can’t, and it’s cause for alarm.
Meaningful claims don’t have to be specific; they just have to be able to be substantiated by a nonzero number of specific examples. Here’s how I imagine this conversation:
Chris: Love your neighbor!
Liron: Can you give me an example of a time in your life where that exhortation was relevant?
Chris: Sure. People in my apartment complex like to smoke cigarettes in the courtyard and the smoke wafts up to my window. It’s actually a nonsmoking complex, so I could complain to management and get them to stop, but I understand the relaxing feeling of a good smoke, so I let them be.
Liron: Ah I see, that was pretty accommodating of you.
Chris: Yeah, and I felt love in my heart for my fellow man when I did that.
Liron: Cool beans. Thanks for helping me understand what kind of scenarios you mean for your exhortation to “love your neighbor” to map to.
Sweet thanks
I agree that if a goal-complete AI steers the future very slowly, or very weakly—as by just trying every possible action one at a time—then at some point it becomes a degenerate case of the concept.
(Applying the same level of pedantry to Turing-completeness, you could similarly ask if the simple Turing machine that enumerates all possible output-tape configurations one-by-one is a UTM.)
The reason “goal-complete” (or “AGI”) is a useful coinage, is that there’s a large cluster in plausible-reality-space of goal-complete agents with a reasonable amount of goal-complete optimization power (e.g. humans, natural selection, and probably AI starting in a few years), and another large distinguishable cluster of non-goal-complete agents (e.g. the other animals, narrow AI).
Yeah, no doubt there are cases where people save money by having a narrower AI, just like the scenario you describe, or using ASICs for Bitcoin mining. The goal-complete AI itself would be expected to often solve problems by creating optimized problem-specific hardware.
Hmm it seems to me that you’re just being pedantic about goal-completeness in a way that you aren’t symmetrically being for Turing-completeness.
You could point out that “most” Turing machines output tapes full of 10^100 1s and 0s in a near-random configuration, and every computing device on earth is equally hopeless at doing that.
That’s getting into details of the scenario that are hard to predict. Like I said, I think most scenarios where goal-complete AI exists are just ones where humans get disempowered and then a single AI fooms (or a small number make a deal to split up the universe and foom together).
As to whether humans will prevent goal-complete AI: some of us are yelling “Pause AI!”
Thanks I’ll look into that. Maybe try the transcript generated by YouTube?