Cody Rushing

Karma: 380

Cody Rushing May 11, 2023, 4:49 PM
2 points
0
on: AI #11: In Search of a Moat
Thanks for the writeup!

Small nitpik: typo in “this indeed does not seem like an attitude that leads to go outcomes”

Cody Rushing Apr 21, 2023, 4:09 AM
2 points
0
in reply to: James Payor’s comment on: AI #8: People Can Do Reasonable Things
I’m not sure if you’ve seen it or not, but here’s a relevant clip where he mentions that they aren’t training GPT-5. I don’t quite know how to update from it. It doesn’t seem likely that they paused from a desire to conduct more safety work, but I would also be surprised if somehow they are reaching some sort of performance limit from model size.
However, as Zvi mentions, Sam did say:
“I think we’re at the end of the era where it’s going to be these, like, giant, giant models...We’ll make them better in other ways”

Cody Rushing Mar 31, 2023, 3:18 PM
4 points
0
on: Widening Overton Window—Open Thread
The increased public attention towards AI Safety risk is probably a good thing. But, when stuff like this is getting lumped in with the rest of AI Safety, it feels like the public-facing slow-down-AI movement is going to be a grab-bag of AI Safety, AI Ethics, and AI… privacy(?). As such, I’m afraid that the public discourse will devolve into “Woah-there-Slow-AI” and “GOGOGOGO” tribal warfare; from the track record of American politics, this seems likely—maybe even inevitable?
More importantly, though, what I’m afraid of is that this will translate into adversarial relations between AI Capabilities organizations and AI Safety orgs (more generally, that capabilities teams will become less inclined to incorporate safety concerns in their products).
I’m not actually in an AI organization, so if someone is in one and has thoughts on this dynamic happening/not happening, I would love to hear.

Cody Rushing Mar 30, 2023, 6:54 PM
11 points
13
on: “Dangers of AI and the End of Human Civilization” Yudkowsky on Lex Fridman
Sheesh. Wild conversation. While I felt Lex was often missing the points Eliezer was saying, I’m glad he gave him the space and time to speak. Unfortunately, it felt like the conversation would keep moving towards reaching a super critical important insight that Eliezer wanted Lex to understand, and then Lex would just change the topic onto something else, and then Eliezer just had to begin building towards a new insight. Regardless, I appreciate that Lex and Eliezer thoroughly engaged with each other; this will probably spark good dialogue and get more people interested in the field. I’m glad it happened.
For those who are time constrained and wondering what is in it: Lex and Eliezer basically cover a whole bunch of high-level points related to AI not-kill-everyone-ism, delving into various thought experiments and concepts which formulate Eliezer’s worldview. Nothing super novel that you probably haven’t heard of if you’ve been following the field for some time.

Cody Rushing Mar 26, 2023, 7:32 PM
4 points
0
on: GPT-4 Specs: 1 Trillion Parameters?
Relevant Manifold Market:

Cody Rushing Mar 19, 2023, 4:59 AM
1 point
0
in reply to: Anirandis’s comment on: An Appeal to AI Superintelligence: Reasons to Preserve Humanity
Because you’re imagining AGI keeping us in a box?
Yeah, something along the lines of this. Preserving humanity =/= humans living lives worth living.

Cody Rushing Mar 18, 2023, 8:34 PM
45 points
41
on: An Appeal to AI Superintelligence: Reasons to Preserve Humanity
I didn’t upvote or downvote this post. Although I do find the spirit of this message interesting, I have a disturbing feeling that arguing to future AI to “preserve humanity for pascals-mugging-type-reasons” trades off X-risk for S-risk. I’m not sure that any of these aforementioned cases encourage AI to maintain lives worth living. I’m not confident that this meaningfully changes S-risk or X-risk positively or negatively, but I’m also not confident that it doesn’t.

Cody Rushing Feb 28, 2023, 4:55 AM
13 points
16
on: $20 Million in NSF Grants for Safety Research
With the advent of Sydney and now this, I’m becoming more inclined to believe that AI Safety and policies related to it are very close to being in the overton window of most intellectuals (I wouldn’t say the general public, yet). Like, maybe within a year, more than 60% of academic researchers will have heard of AI Safety. I don’t feel confident whatsoever about the claim, but it now seems more than ~20% likely. Does this seem to be a reach?

Cody Rushing Feb 18, 2023, 9:26 PM
20 points
13
on: We should be signal-boosting anti Bing chat content
There is a fuzzy line between “let’s slow down AI capabilities” and “lets explicitly, adversarially, sabotage AI research”. While I am all for the former, I don’t support the latter; it creates worlds in which AI safety and capabilities groups are pitted head to head, and capabilities orgs explicitly become more incentivized to ignore safety proposals. These aren’t worlds I personally wish to be in.
While I understand the motivation behind this message, I think the actions described in this post cross that fuzzy boundary, and pushes way too far towards that style of adversarial messaging

Cody Rushing Feb 16, 2023, 5:40 AM
1 point
0
on: The Filan Cabinet Podcast with Oliver Habryka—Transcript
We know, from like a bunch of internal documents, that the New York Times has been operating for the last two or three years on a, like, grand [narrative structure], where there’s a number of head editors who are like, “Over this quarter, over this current period, we want to write lots of articles, that, like, make this point...”
Can someone point me to an article discussing this, or the documents itself? While this wouldn’t be entirely surprising to me, I’m trying to find more data to back this claim, and I can’t seem to find anything significant.

Cody Rushing Jan 20, 2023, 9:52 PM
12 points
6
on: Transcript of Sam Altman’s interview touching on AI safety
It feels strange hearing Sam say that their products are released whenever the feel as though ‘society is ready.’ Perhaps they can afford to do that now, but I cannot help but think that market dynamics will inevitably create strong incentives for race conditions very quickly (perhaps it is already happening) which will make following this approach pretty hard. I know he later says that he hopes for competition in the AI-space until the point of AGI, but I don’t see how he balances the knowledge of extreme competition with the hope that society is prepared for the technologies they release; it seems that even current models, which appear to be far from the capabilities of AGI, are already transformative.

Cody Rushing Jan 12, 2023, 4:12 PM
5 points
0
on: How it feels to have your mind hacked by an AI
Let’s say Charlotte was a much more advanced LLM (almost AGI-like, even). Do you believe that if you had known that Charlotte was extraordinarily capable, you might have been more guarded about recognizing it for its ability to understand and manipulate human psychology, and thus been less susceptible to it potentially doing so?
I find that small part of me still think that “oh this sort of thing could never happen to me, since I can learn from others that AGI and LLMs can make you emotionally vulnerable, and thus not fall into a trap!” But perhaps this is just wishful thinking that would crumble once I interact with more and more advanced LLMs.

Cody Rushing Dec 21, 2022, 7:26 AM
6 points
1
on: Podcast: What’s Wrong With LessWrong
I’m trying to engage with your criticism faithfully, but I can’t help but get the feeling that a lot of your critiques here seem to be a form of “you guys are weird”: your guys’s privacy norms are weird, your vocabulary is weird, you present yourself off as weird, etc. And while I may agree that sometimes it feels as if LessWrongers are out-of-touch with reality at points, this criticism, coupled with some of the other object-level disagreements you were making, seems to overlook the many benefits that LessWrong provides; I can personally attest to the fact that I’ve improved in my thinking as a whole due to this site. If that makes me a little weird, then I’ll accept that as a way to help me shape the world as I see fit. And hopefully I can become a little less weird through the same rationality skills this site helps develop

Cody Rushing Oct 2, 2022, 4:25 AM
3 points
0
in reply to: Quintin Pope’s comment on: Paper: Large Language Models Can Self-improve [Linkpost]
Humans can often teach themselves to be better at a skill through practice, even without a teacher or ground truth
Definitely, but I currently feel that the vast majority of human learning comes with a ground truth to reinforce good habits. I think this is why I’m surprised this works as much as it does: it kinda feels like letting an elementary school kid teach themself math by practicing certain skills they feel confident in without any regard to if that skill even is “mathematically correct”.
Sure, these skills are probably on the right track toward solving math problems—otherwise, the kid wouldn’t have felt as confident about them. But would this approach not ignore skills the student needs to work on, or even amplify “bad” skills? (Or maybe this is just a faulty analogy and I need to re-read the paper)

Cody Rushing Sep 5, 2022, 5:32 PM
4 points
2
on: The ethics of reclining airplane seats
I don’t quite understand the perspective behind someone ‘owning’ a specific space. Do airlines specify that when you purchase a ticket, you are entitled to the chair + the surrounding space (in whatever ambiguous way that may mean)? If not, it seems to me that purchasing a ticket pays for a seat and your right to sit down on it, and everything else is complementary.

Cody Rushing Jul 2, 2022, 2:05 AM
3 points
0
on: Looking back on my alignment PhD
I’m having trouble understanding your first point on wanting to ‘catch up’ to other thinkers. Was your primary message advocating against feeling as if you are ‘in dept’ until you improve your rationality skills? If so, I can understand that.
But if that is the case, I don’t understand the relevance of the lack of a “rationality tech-tree”—sure, there may not be clearly defined pathways to learn rationality. Even so, I think its fair to say that I perceive some people on this blog to currently be better thinkers than I, and that I would like to catch up to their thinking abilities so that I can effectively contribute to many discussions. Would you advocate against that mindset as well?

Cody Rushing Jun 17, 2022, 2:27 AM
1 point
in reply to: dkirmani’s comment on: AI Risk, as Seen on Snapchat
I was surprised by this tweet and so I looked it up. I read a bit further and ran into this; I guess I’m kind of surprised to see a concern as fundamental as alignment, whether or not you agree it is an major issue, be so… is polarizing the right word? Is this an issue we can expect to see grow as AI safety (hopefully) becomes more mainstream? “LW extended cinematic universe” culture getting an increasingly bad reputation seems like it would be extremely devastating for alignment goals in general.

Cody Rushing Jun 7, 2022, 7:32 PM
8 points
0
on: AGI Safety FAQ / all-dumb-questions-allowed thread
I have a few related questions pertaining to AGI timelines. I’ve been under the general impression that when it comes to timelines on AGI and doom, Eliezer’s predictions are based on a belief in extraordinarily fast AI development, and thus a close AGI arrival date, which I currently take to mean a quicker date of doom. I have three questions related to this matter:
1. For those who currently believe that AGI (using whatever definition to describe AGI as you see fit) will be arriving very soon—which, if I’m not mistaken, is what Eliezer is predicting—approximately how soon are we talking about. Is this 2-3 years soon? 10 years soon? (I know Eliezer has a bet that the world will end before 2030, so I’m trying to see if there has been any clarification of how soon before 2030)
2. How much does Eliezer’s views on timelines vary in comparison to other big-name AI safety researchers?
3. I’m currently under the impression that it takes a significant amount of knowledge of Artificial Intelligence to be able to accurately attempt to predict timelines related to AGI. Is this impression correct? And if so, would it be a good idea to reference general consensus opinions such as Metaculus when trying to frame how much time we have left?

Cody Rushing May 27, 2022, 2:57 PM
1 point
in reply to: Cody Rushing’s comment on: [$20K in Prizes] AI Safety Arguments Competition
[Shorter version, but one I don’t think is as compelling]
Timmy is my personal AI Chef, and he is a pretty darn good one, too. Of course, despite his amazing cooking abilities, I know he’s not perfect—that’s why there’s that shining red emergency shut-off button on his abdomen.
But today, Timmy became my worst nightmare. I don’t know why he thought it would be okay to do this, but he hacked into my internet to look up online recipes. I raced to press his shut-off button, but he wouldn’t let me, blocking it behind a cast iron he held with a stone-cold grip. Ok, that’s fine, I have my secret off-lever in my room that I never told him about. Broken. Shoot, that’s bad, but I can just shut off the power, right? As I was busy thinking he swiftly slammed the door shut, turning my own room into an inescapable prison. And so as I cried, wondering how everything could have gone crazy so quickly, he laughed, saying, “Are you serious? I’m not crazy, I’m just ensuring that I can always make food for you. You wanted this!”
And it didn’t matter how much I cried, how much I tried to explain to him that he was imprisoning me, hurting me. It didn’t even matter that he knew it as well. For he was an AI coded to be my personal chef, coded to make sure he could make food that I enjoyed, and he was a pretty darn good one, too.
If you don’t do anything about it, Timmy may just be arriving on everyone’s doorsteps in a few years.

Cody Rushing May 27, 2022, 2:55 PM
1 point
on: [$20K in Prizes] AI Safety Arguments Competition
[Intended for Policymakers with the focus of simply allowing for them to be aware of the existence of AI as a threat to be taken seriously through an emotional appeal; Perhaps this could work for Tech executives, too.
I know this entry doesn’t follow what a traditional paragraph is, but I like its content. Also it’s a tad bit long, so I’ll attach a separate comment under this one which is shorter, but I don’t think it’s as impactful]
Timmy is my personal AI Chef, and he is a pretty darn good one, too.
You pick a cuisine, and he mentally simulates himself cooking that same meal millions of times, perfecting his delicious dishes. He’s pretty smart, but he’s constantly improving and learning. Since he changes and adapts, I know there’s a small chance he may do something I don’t approve of—that’s why there’s that shining red emergency shut-off button on his abdomen.
But today, Timmy stopped being my personal chef and started being my worst nightmare. All of a sudden, I saw him hacking my firewalls to access new cooking methods and funding criminals to help smuggle illegal ingredients to my home.
That seemed crazy enough to warrant a shutdown; but when I tried to press the shut-off button on his abdomen, he simultaneously dodged my presses and fried a new batch of chicken, kindly telling me that turning him off would prevent him from making food for me.
That definitely seemed crazy enough to me; but when I went to my secret shut-down lever in my room—the one I didn’t tell him about—I found it shattered, for he had predicted I would make a secret shut-down lever, and that me pulling it would prevent him from making food for me.
And when, in a last ditch effort, I tried to turn off all power in the house, he simply locked me inside my own home, for me turning off the power (or running away from him) would prevent him from making food for me.
And when I tried to call 911, he broke my phone, for outside intervention would prevent him from making food for me.
And when my family looked for me, he pretended to be me on the phone, playing audio clips of me speaking during a phone call with them to impersonate me, for a concern on their part would prevent him from making food for me.
And so as I cried, wondering how everything could have gone so wrong so quickly, why he suddenly went crazy, he laughed—“Are you serious? I’m just ensuring that I can always make food for you, and today was the best day to do it. You wanted this!”
And it didn’t matter how much I cried, how much I tried to explain to him that he was imprisoning me, hurting me. It didn’t even matter that he knew as well. For he was an AI coded to be my personal chef; and he was a pretty darn good one, too.
If you don’t do anything about it, Timmy may just be arriving on everyone’s doorsteps in a few years.