TW123

• TW123Jan 11, 2023, 7:11 PM

  6 points

  3

  on: What’s the deal with AI con­scious­ness?

  It looks like I got one or possibly two strong downvotes, but it doesn’t seem like from either of the commenters. If you downvoted this (or think you understand why it was downvoted), please let me know in the comments so I can improve!

• TW123Jan 11, 2023, 7:09 PM

  2 points

  1

  in reply to: Gerald Monroe’s comment on: What’s the deal with AI con­scious­ness?

  Consciousness therefore only happens if it improves performance at the task we have assigned. And some tasks like interacting directly with humans it might improve performance.

  I don’t think this is necessarily true. Consciousness could be a side effect of other processes that do improve performance.

  The way I’ve heard this put: a polar bear has thick hair so that it doesn’t get too cold, and this is good for its evolutionary fitness. The fact that the hair is extremely heavy is simply a side effect of this. Consciousness could possibly me similar.

What’s the deal with AI con­scious­ness?

Im­pli­ca­tions of simulators

“AI” is an indexical

A Year of AI In­creas­ing AI Progress

• TW123Dec 25, 2022, 6:38 AM

  6 points

  2

  in reply to: Caridorc Tergilti’s comment on: Ap­plied Lin­ear Alge­bra Lec­ture Series

  I don’t think these are necessarily bad suggestions if there were a future series. But my sense is that John did this for the people in the audience, somebody asked him to record it so he did, and now he’s putting them online in case they’re useful to anyone. It’s very hard to make good production quality lectures, and it would have required more effort. But it sounds like John knew this and decided he would rather spend his time elsewhere, which is completely his choice to make. As written, these suggestions feel a bit pushy to me.

• TW123Dec 24, 2022, 6:15 AM

  3 points

  0

  in reply to: Remmelt’s comment on: Let’s think about slow­ing down AI

  Sorry if I missed it earlier in the thread, but who is this “polymath”?

• TW123Dec 23, 2022, 4:15 AM

  1 point

  0

  in reply to: Ben Pace’s comment on: Let’s think about slow­ing down AI

  Yeah, I agree with all this.

• TW123Dec 23, 2022, 3:39 AM

  1 point

  0

  in reply to: Ben Pace’s comment on: Let’s think about slow­ing down AI

  I just do not think that the post is written for people who think “slowing down AI capabilities is robustly good.” If people thought that, then why do they need this post? Surely they don’t need somebody to tell them to think about it?

  So it seems to me like the best audience for this post would be those (including those at some AI companies, or those involved in policy, which includes people reading this post) who currently think something else, for example that the robustly good thing is for their chosen group to be ahead so that they can execute whatever strategy they think they alone can do correctly.

  The people I’ve met who don’t want to think about slowing down AI capabilities just don’t seem to think that slowing down AI progress would be robustly good, because that just wouldn’t be a consistent view! They often seem to have some view that nothing is robustly good, or maybe some other thing (“get more power”) is robustly good. Such people just won’t really be swayed by the robust priors thing, or maybe they’d be swayed in the other direction.

• TW123Dec 23, 2022, 2:11 AM

  2 points

  1

  in reply to: Ben Pace’s comment on: Let’s think about slow­ing down AI

  The claim being made is something like the following:

  1) AGI is a dangerous technology.

  2) It is robustly good to slow down dangerous technologies.

  3) Some people might say that you should not actually do this because of [complicated unintelligible reason].

  4) But you should just do the thing that is more robustly good.

  I argue that many people (yes, you’re right, in ways that conflict with one another) believe the following:

  1) X is a dangerous country.

  2) It is robustly good to always be ahead of X in all technologies, including dangerous ones.

  3) Some people might say that you should not actually do this because of [complicated unintelligible reason]. This doesn’t make very much sense.

  4) But you should just do the thing that is more robustly good.

  My point is that which argument is the obvious, robust one, and which one is the weird inside view one depends on your perspective. Therefore, it doesn’t seem like (4) is a very good generalized argument. For example, if I were one of these powerful people, I think it would be wrong for me to be convinced to “focus on the robustly good measures, not the weird inside view measures” because it would lead me to do bad things like trying to advance AI capabilities. As a result, the argument seems suspect to me. It feels like it only works for this community, or people who are already very concerned by AI x-risk.

  In comparison, there are specific arguments like “AGI is dangerous” and “slowing down dangerous technologies is actually robustly good” (some of these were presented in this post) that I think are, ironically, must more robustly good, because they don’t seem to have negative effects as reliably when presented to people who hold beliefs I think are wrong.

  Edit: I no longer endorse this comment. It claims too much, specifically that any reasoning procedure is suspect if it leads to people who believe false premises taking bad actions.

  I think what I was really trying to get at in my original comment was that that particular argument seems aimed at people who already think that it would be robustly good to slow down dangerous technologies. But the people who would most benefit from this post are those who do not already think this; for them it doesn’t help much and might actively hurt.

• TW123Dec 23, 2022, 1:27 AM

  15 points

  1

  on: Let’s think about slow­ing down AI

  There are things that are robustly good in the world, and things that are good on highly specific inside-view models and terrible if those models are wrong. Slowing dangerous tech development seems like the former, whereas forwarding arms races for dangerous tech between world superpowers seems more like the latter.

  It may seem the opposite to some people. For instance, my impression is that for many adjacent to the US government, “being ahead of China in every technology” would be widely considered robustly good, and nobody would question you at all if you said that was robustly good. Under this perspective the idea that AI could pose an existential risk is a “highly specific inside-view model” and it would be terrible if we acted on the model and it is wrong.

  I don’t think your readers will mostly think this, but I actually think a lot of people would, which for me makes this particular argument seem entirely subjective and thus suspect.

• TW123Dec 7, 2022, 5:33 PM

  3 points

  1

  in reply to: scasper’s comment on: Where to be an AI Safety Pro­fes­sor

  I think this is all true, but also since Yale CS is ranked poorly the graduate students are not very strong for the most part. You certainly have less competition for them if you are a professor, but my impression is few top graduate students want to go to Yale. In fact, my general impression is often the undergraduates are stronger researchers than the graduate students (and then they go on to PhDs at higher ranked places than Yale).

  Yale is working on strengthening its CS department and it certainly has a lot of money to do that. But there are a lot of reasons that I am not that optimistic. There is essentially no tech scene in New Haven, New Haven is not that great in general, the Yale CS building is extremely dingy (I think this has an actual effect on people), and it’s really hard to affect the status quo. However, I’m more optimistic that Yale will successfully forge a niche of interdisciplinary research, which is really a strength of the university.

• TW123Dec 7, 2022, 3:46 PM

  6 points

  2

  on: Where to be an AI Safety Pro­fes­sor

  I’ll just comment on my experience as an undergrad at Yale in case it’s useful.

  At Yale, the CS department, particularly when it comes to state of the art ML, is not very strong. There are a few professors who do good work, but Yale is much stronger in social robotics and there is also some ML theory. There are a couple AI ethics people at Yale, and there soon will be a “digital ethics” person, but there aren’t any AI safety people.

  That said, there is a lot of latent support for AI safety at Yale. One of the global affairs professors involved in the Schmidt Program for Artificial Intelligence, Emerging Technology, and National Power is quite interested in AI safety. He invited Brian Christian and Stuart Russell to speak and guest teach his classes, for example. The semi-famous philosopher L.A. Paul is interested in AI safety, and one of the theory ML professors had a debate about AI safety in one of his classes. One of the professors most involved in hiring new professors specifically wants to hire AI safety people (though I’m not sure he really knows what AI safety is).

  I wouldn’t really recommend Yale to people who are interested in doing very standard ML research and want an army of highly competent ML researchers to help them. But for people whose work interacts with sociotechnical considerations like policy, or is more philosophical in nature, I think Yale would be a fantastic place to be, and in fact possibly one of the best places one could be.

• TW123Dec 2, 2022, 9:59 PM

  3 points

  0

  in reply to: Robert Kennedy’s comment on: Did ChatGPT just gaslight me?

  I appreciate this. I don’t even consider myself part of the rationality community, though I’m adjacent. My reasons for not drinking have nothing to do with the community and existed before I knew what it was. I actually get the sense this is the case for a number of people in the community (more of a correlation or common cause rather than caused by the community itself). But of course I can’t speak for all.

  I will be trying it on Sunday. We will see how it is.

• TW123Dec 2, 2022, 7:26 PM

  5 points

  2

  in reply to: Douglas_Knight’s comment on: Did ChatGPT just gaslight me?

  I’ve thought about this comment, because it certainly is interesting. I think I was clearly confused in my questions to ChatGPT (though I will note: My tequila-drinking friends did not and still don’t think tequila tastes at all sweet, including “in the flavor profile” or anything like that. But it seems many would say they’re wrong!) ChatGPT was clearly confused in its response to me as well.

  I think this part of my post was incorrect:

  It was perfectly clear: ChatGPT was telling me that tequila adds a sweetness to the drink. So it was telling me that tequila is a sweet drink (at least, as sweet as orange juice).

  I have learned today that a drink does not have to be sweet in order for many to consider it to add “sweetness.” To be honest, I don’t understand this at all, and at the time considered it a logical contradiction. It seems a lot less clear cut to me now.

  However, the following (and the quote above it) is what I focused on most in the post. I quoted the latter part of it three different times. I believe it is entirely unaffected by whether or not tequila is canonically considered to be sweet:

  “I was not referring to the sweetness that comes from sugar.” But previously, ChatGPT had said “tequila has a relatively low alcohol content and a relatively high sugar content.” Did ChatGPT really forget what it had said, or is it just pretending?

  Is ChatGPT gaslighting me?

  Thomas: You said tequila has a “relatively high sugar content”?

  ChatGPT: I apologize if my previous response was unclear. When I said that tequila has a “relatively high sugar content,” I was not suggesting that tequila contains sugar.

• TW123Dec 2, 2022, 7:06 PM

  3 points

  0

  in reply to: ChristianKl’s comment on: Did ChatGPT just gaslight me?

  It should! I mentioned that probable future outcome in my original post.

• TW123 2 Dec 2022 5:40 UTC

  9 points

  7

  in reply to: Robert Kennedy’s comment on: Did ChatGPT just gaslight me?

  I’m going to address your last paragraph first, because I think it’s important for me to respond to, not just for you and me but for others who may be reading this.

  When I originally wrote this post, it was because I had asked ChatGPT a genuine question about a drink I wanted to make. I don’t drink alcohol, and I never have. I’ve found that even mentioning this fact sometimes produces responses like yours, and it’s not uncommon for people to think I am mentioning it as some kind of performative virtue signal. People choose not to drink for all sorts of reasons, and maybe some are being performative about it, but that’s a hurtful assumption to make about anyone who makes that choice and dares to admit it in a public forum. This is exactly why I am often hesitant to mention this fact about myself, but in the case of this post, there really was no other choice (aside from just not posting this at all, which I would really disprefer). I’ve generally found the LW community and younger generations to be especially good at interpreting a choice not to drink for what it usually is: a personal choice, not a judgment or a signal or some kind of performative act. However, your comment initially angered and then saddened me, because it greets my choice through a lens of suspicion. That’s generally a fine lens through which to look at the world, but I think in this context, it’s a harmful one. I hope you will consider thinking a little more compassionately in the future with respect to this issue.

  ---

  To answer your object-level critiques:

  The problem is that it clearly contradicts itself several times, rather than admitting a contradiction it doesn’t know how to reconcile. There is no sugar in tequila. Tequila may be described as sweet (nobody I talked to described it as such, but some people on the internet do) for non-sugar reasons. In fact, I’m sure ChatGPT knows way more about tequila than I do!

  It is not that it “may not know” how to reconcile those facts. It is that it doesn’t know, makes something up, and pretends it makes sense.

  A situation where somebody interacting with the chatbot doesn’t know much about the subject area is exactly the kind of situation we need to be worried about with these models. I’m entirely unconvinced that the fact that some people describe tequila as sweet says much at all about this post. That’s because the point of the post was rather that ChatGPT claimed tequila has high sugar content, then claimed that actually the sweetness is due to something else, and it never really meant that tequila has any sugar. That is the problem, and I don’t think my description of it is overblown.

• TW123 2 Dec 2022 1:57 UTC

  6 points

  0

  in reply to: Unnamed’s comment on: Did ChatGPT just gaslight me?

  Interesting! I hadn’t come across that. Maybe ChatGPT is right that there is sweetness (perhaps to somebody with trained taste) that doesn’t come from sugar. However, the blatant contradictions remain (ChatGPT certainly wasn’t saying that at the beginning of the transcript).

• TW123 1 Dec 2022 13:18 UTC

  5 points

  0

  in reply to: Charlie Steiner’s comment on: Did ChatGPT just gaslight me?

  OpenAI has in the past not been that transparent about these questions, but in this case, the blog post (linked in my post) makes it very clear it’s trained with reinforcement learning from human feedback.

  However, of course it was initially pretrained in an unsupervised fashion (it’s based on GPT-3), so it seems hard to know whether this specific behavior was “due to the RL” or “a likely continuation”.