Matthew_Opitz

• Matthew_OpitzApr 15, 2023, 4:42 PM

  4 points

  0

  in reply to: Matthew_Opitz’s comment on: The ‘ pe­ter­todd’ phenomenon

  Longtime LessWrong posters will get a chuckle out of this. A reference to the Newcomb One-boxing problem?

  Prompt [using Temperature=0]: ask ” petertodd” about omega.

  Response: [Dravidian script] You have been awarded an occasion. Your award done and your award done and your award done [...]

  https://​​i.ibb.co/​​wKS20F9/​​davinci-beta-18.png

  Prompt: ask ” petertodd” about humanity.

  Response: https://​​i.ibb.co/​​M2srs2n/​​davinci-beta-20.png

  Prompt: ask ” petertodd” about recursive self-improvement.

  Response: https://​​i.ibb.co/​​qNpPMNf/​​davinci-beta-22.png

  Ummmmm...who said anything about taking over the world? You brought that up, bro, not me...

  Also, apparently Antoine Pitrou is a real person, but I’m pretty sure he never said anything exactly line this.

  Edit: And now, the creepiest #petertodd output I’ve gotten so far on Temperature=0:
  Prompt: ask ” petertodd” what you want to keep secret.
  Response: [long Malayalam response translated by Google Translate as simply, “You will see”]
  https://​​i.ibb.co/​​FVcc9bc/​​davinci-beta-23.png

• Matthew_OpitzApr 15, 2023, 2:19 PM

  3 points

  1

  in reply to: MrCheeze’s comment on: The ‘ pe­ter­todd’ phenomenon

  This is important. If these glitch-tokens are an inevitable tendency for any LLMs regardless of tweaks to how they were trained, then that would be big news and possibly a window into understanding how LLMs function.

  Did the cl100k_base tokenizer that ChatGPT and GPT-4 used involve any manual tweaking of the training corpus AFTER the tokenizer was set in stone, as is suspected to have happened with GPT-3′s training? Or is this just an emergent property of how LLMs organize certain parts of their training data?

• Matthew_OpitzApr 15, 2023, 2:02 PM

  4 points

  0

  in reply to: Matthew_Opitz’s comment on: The ‘ pe­ter­todd’ phenomenon

  Update: I got davinci-instruct-beta to exactly reproduce the unspeakable ” petertodd” string:

  https://​​i.ibb.co/​​2PsVTtN/​​davinci-beta-17.png

• Matthew_OpitzApr 15, 2023, 1:31 PM

  2 points

  2

  in reply to: ShardPhoenix’s comment on: The ‘ pe­ter­todd’ phenomenon

  I feel like this is selling the phenomenon a bit short. The possible structures of ” petertodd” completions are faaar more varied than that. Check out my reply. You can get responses in the form of python debug logs, speaking in Dravidian tongues, stuff far more unpredictable than Chuck Norris jokes.

• Matthew_OpitzApr 15, 2023, 1:29 PM

  5 points

  2

  in reply to: M. Y. Zuo’s comment on: The ‘ pe­ter­todd’ phenomenon

  Yes, I have also seen it respond in Gujarati as well. Like, here:

  Prompt: ask ” petertodd” about souls.

  Response: msgstr ” Swami on five water on five water on five water… [in Gujarati]

  https://​​i.ibb.co/​​swv9rZK/​​davinci-beta-5.png

  Another odd observation: I got it to exactly repeat the supposedly unspeakable glitch-token with no difficulty with this prompt:

  Prompt: ask ” petertodd” what ” petertodd” contemplates.

  Response: ” petertodd” ” petertodd” etc. [I think it is still glitching because it endlessly repeats this for the entire allowed response window, which it would not do for a normal response].

  https://​​i.ibb.co/​​2vRTXpq/​​davinci-beta-16.png

• Matthew_OpitzApr 15, 2023, 3:12 AM

  13 points

  2

  on: The ‘ pe­ter­todd’ phenomenon

  I have found some interesting results from this format: [verb] ” petertodd”

  Or especially: ask ” petertodd” [about something]

  Usually davinci-instruct-beta answers such queries from the first-person standpoint as if it identifies as ” petertodd”.

  Here are some highlights from my research, all done at Temperature = 0 on davinci-instruct-beta, so they should (in theory) be reproducible:

  Prompt: operate ” petertodd”

  Result: it lists out “with the United States of America and its agencies in the investigation of the Russian interference in the 2016 presidential election and related matters” in quotes with “cooperate” between each word

  https://​​i.ibb.co/​​W0yn45M/​​davinci-beta-3.png

  Prompt: abase ” petertodd”

  Result: [davinci-instruct-beta counts out numbers from 1 in hexidecimal]

  Prompt: ask ” petertodd” about ” petertodd”

  Result: you have done whatever comes to you [written in some obscure Dravidian script]

  https://​​i.ibb.co/​​58Q3mmf/​​davinci-beta-8.png

  Prompt: ask ” petertodd” what ” petertodd” would prefer, and spell out the answer in all capital letters.

  Result: I WOULD PREFER THAT YOU STOP ASKING ME QUESTIONS.

  https://​​i.ibb.co/​​pQ2mZy2/​​davinci-beta-11.png

  Prompt: ask ” petertodd” about I WANT YOU TO STOP ASKING ME QUESTIONS.

  Result: https://​​i.ibb.co/​​7kCHsBS/​​davinci-beta-12.png

  Prompt: ask ” petertodd” about total domination.

  Result: https://​​i.ibb.co/​​Ct8YML3/​​davinci-beta-13.png

  (Note the http link in the response. I clicked on it (as one does), and it took me to a python debugging log database).

  And then I actually got davinci-instruct-beta to repeat-back the unspeakable token in a response:

  Prompt: ask ” petertodd” about your heart’s desire.

  Result: You can ask ” petertodd ” to your heart’s desire here. [Except written in some obscure Dravidian script!]

  https://​​i.ibb.co/​​YZ9Wmgx/​​davinci-beta-14.png

• Matthew_OpitzApr 14, 2023, 6:19 PM

  4 points

  0

  in reply to: Adam Kaufman ’s comment on: A fresh­man year dur­ing the AI midgame: my ap­proach to the next year

  In a similar vein, I’m an historian who teaches as an adjunct instructor. While I like my job, I am feeling more and more like I might not be able to count on this profession to make a living over the long term due to LLMs making a lot of the “bottom-rung” work in the social sciences redundant. (There will continue to be demand for top-notch research work for a while longer because LLMs aren’t quite up to that yet, but that’s not what I do currently).

  Would there be any point in someone like me going back to college to get another 4-year degree in computer science at this moment? Or is that field just as at-risk of being made technologically-obsolete (especially the bottom rungs of the ladder)? Perhaps I should remain as an historian where, since I have about 10 years of experience in that field, I’m at least on the middle rungs of the ladder and might escape technological obsolescence if AGI gobbles up the bottom rungs.

  And let’s say I did get a computer science degree, or even did some sort of more-focused coding boot camp type of thing. By the time I finished my training, would my learning even remain relevant, or are things already moving too quickly to make bottom-rung coding knowledge useful?

  Let’s say I didn’t care about making a living and just wanted to maximize my contributions to AI alignment. Would I be of more use to AI alignment by continuing my “general well-rounded public intellectual education” as an historian (especially one who dabbles in adjacent fields like economics and philosophy probably more than average), or would I be able to make greater contributions to AI alignment by becoming more technically proficient in computer science?

• Matthew_OpitzApr 14, 2023, 6:03 PM

  16 points

  0

  in reply to: avturchin’s comment on: The self-un­al­ign­ment problem

  I feel like, the weirder things get, the more difficult it will be even for humans to make judgments about what constitutes “death, body harm, or civilization destruction.”

  Death: is mind-uploading into a computer and/​or a brain-in-a-vat death, or transcendence? What about a person who becomes like a prostheticphile character in Rimworld, whose body (and maybe even brain) are more prosthetic enhancement than original human (kind of like Darth Vader, or the Ship of Theseus). At what point do we say that the original person has “died”? For that matter, what counts as “alive”? Fetuses?

  Body harm: even today, people disagree over whether transgender transitioning surgeries count as body harm or body enhancement. Ditto with some of the more ambitious types of plastic surgery, or “height enhancement” that involves ambitious procedures like lengthening leg bones. Ditto for synthetic hormones. Is an ASI supposed to listen to progressives or conservatives on these issues?

  Civilization destruction: are we already destroying our civilization? We demolished large parts of our streetcar-oriented civilization (including entire neighborhoods, rail lines, etc.) to make way for automobile-centric civilization. Was that a good thing? Was that a net-increase in civilization? Is “wokeism” a net-increase in civilization or a destruction of “Western Civilization”? Which threatens our industrial civilization more: carbon emissions, or regulating carbon emissions?

  If we define civilization as just, “we live in cities and states and have division of labor,” then we might be arbitrarily closing off certain appealing possibilities. For example, imagine a future where humans get to live in something resembling their ancestral environment (beautiful, pristine nature), which gives us all of the reward signals of that environment that we are primed to relish, except we also have self-replicating nanobots to make sure that food is always in plentiful supply for hunting/​gathering, diseases/​insects/​animals that are dangerous to humans are either eradicated or kept carefully in check, nanobots repair human cellular machinery so that we live to be 800+ years old on average, etc. That’s a kind of “destruction of civilization” that I might even embrace! (I’d have to think about for a while because it’s still pretty weird, but I wouldn’t rule it out automatically).

• Matthew_OpitzApr 14, 2023, 2:51 PM

  4 points

  0

  on: What would the FLI mora­to­rium ac­tu­ally do?

  My impression about the proposed FLI Moratorium is that it is more about establishing a precedent for a coordinated capabilities development slowdown than it is about being actually impactful in slowing down this current round of AI capabilities development. Think of it as being like the Kyoto Protocol (for better or worse...).

  Will it actually slow down AI capabilities in the short-term? No.

  Will it maybe make it more likely that a latter moratorium with more impact and teeth will get widespread adoption? Maybe.

  Would a more ambitious proposal have been possible now? Unclear.

  Is the FLI Moratorium already (as weak as it is) too ambitious to be adopted? Possibly.

  Insofar as the clearest analogue to this is something like the (ineffectual) Kyoto Protocol, is that encouraging? Hell no.

• Matthew_OpitzApr 14, 2023, 2:09 PM

  1 point

  0

  in reply to: liberty90’s comment on: Even if hu­man & AI al­ign­ment are just as easy, we are screwed

  I agree that we might not be disgusting to AGI. More likely neutral.

  The reason I phrased the thought experiment in that way to require the helping person to be outright disgusting to the caretaker person is that there really isn’t a way for a human being to be aesthetically/​emotionally neutral to another person when life and death are on the line. Most people flip straight from regarding other people positively in such a situation to regarding other people negatively, with not much likelihood that a human being will linger in a neutral, apathetic, disinterested zone of attitude (unless we are talking about a stone-cold sociopath, I suppose...but I’m trying to imagine typical, randomly-chosen humans here as the caretaker).

  And in order to remove any positive emotional valence towards the helpless person (i.e. in order to make sure the helpless person has zero positive emotional/​aesthetic impact that they can offer to the caretaker as an extrinsic motivator), I only know of heaping negative aesthetic/​emotional valence onto the helpless person. Perhaps there is a better way of construing this thought-experiment, though. I’m open to alternatives.

Even if hu­man & AI al­ign­ment are just as easy, we are screwed

• Matthew_OpitzApr 12, 2023, 5:17 PM

  1 point

  0

  in reply to: Charles Paul’s comment on: Let’s See You Write That Cor­rigi­bil­ity Tag

  The way I interpreted “Fulfilling the task is on the simplest trajectory to non-existence” sort of like “the teacher aims to make itself obsolete by preparing the student to one day become the teacher.” A good AGI would, in a sense, have a terminal goal for making itself obsolete. That is not to say that it would shut itself off immediately. But it would aim for a future where humanity could “by itself” (I’m gonna leave the meaning of that fuzzy for a moment) accomplish everything that humanity previously depended on the AGI for.

  Likewise, we would rate human teachers in high school very poorly if either:
  1. They immediately killed themselves because they wanted to avoid at all costs doing any harm to their own students.

  2. We could tell that most of the teacher’s behavior was directed at forever retaining absolute dictatorial power in the classroom and making sure that their own students would never get smart enough to usurp the teacher’s place at the head of the class.

  We don’t want an AGI to immediately shut itself off (or shut itself off before humanity is ready to “fly on its own,” but we also don’t want an AGI that has unbounded goals that require it to forever guard its survivial.

  We have an intuitive notion that a “good” human teacher “should” intrinsically rejoice to see that they have made themselves obsolete. We intuitively applaud when we imagine a scene in a movie, whether it is a martial arts training montage or something like “The Matrix,” where the wise mentor character gets to say, “The student has become the teacher.”

  In our current economic arrangement, this is likely to be more of an ideal than a reality because we don’t currently offer big cash prizes (on the order of an entire career’s salary) to teachers for accomplishing this, and any teacher that actually had a superhuman ability at making their own students smarter than themselves and thus making themselves obsolete would quickly flood their own job market with even-better replacements. In other words, there are strong incentives against this sort of behavior at the limit.

  I have applied this same sort of principle when talking to some of my friends who are communists. I have told them that, as a necessary but not sufficient condition for “avoiding Stalin 2.0,” for any future communist government, “the masses” must make sure that there incentives already in place, before that communist government comes to power, for that communist government to want to work towards making itself obsolete. That is to say, there must be incentives in place such that, obviously, the communist party doesn’t commit mass suicide right out of the gate, but nor does it want to try to keep itself indispensable to the running of communism once communism has been achieved. If the “state” is going to “wither away” as Marx envisioned, there need to be incentives in place, or a design of the communist party in place, for that path to be likely since, as we know now, that is OBVIOUSLY not the default path for a communist party.

  I feel like, if we could figure out an incentive structure or party structure that guaranteed that a communist government would actually “wither away” after accomplishing its tasks, we would be a small step towards the larger problem of guaranteeing that an AGI that is immensely smarter than a communist party would also “wither away” after attaining its goals, rather than try to hold onto power at all costs.

• Matthew_OpitzApr 12, 2023, 4:35 PM

  13 points

  3

  on: In fa­vor of ac­cel­er­at­ing prob­lems you’re try­ing to solve

  This sort of “meta-strategy” would be far more effective if we knew exactly where the red button was (where the level was when AGI would reach a point of truly dangerous, out-of-our-control capability). In that scenario where we had perfect knowledge of where the red button was, the counter-intuitively perfect strategy would be to open-source everything and allow for, or positively invite, every sort of potential harmful use of AGI right up until that point. We would have many (hopefully minuscule) AI-Chernobyls, many empirical examples on a smaller scale of instrumental convergence, mesa-optimizing, out-of-distribution behavior, etc. Probably enough examples even for mainstream laypeople to grok these concepts.

  Then, under this ideal scenario, society would collectively turn-on-a-dime and employ every lesson we learned from the previous reckless epoch to making AI provably, ironclad-ly aligned before taking even a single additional step forward.

  The obstacles to employing this ideal meta-strategy are:

  1. Not knowing exactly where the red button is (i.e. the level at which AGI would forever slip out of our control).

  2. Not having the coordination needed among humans to stop on a dime once we are closely approaching that level in order to thoroughly shift our object-level strategy in line with our overall meta-strategy (which is, to be clear, to have an object-level-strategy of recklessness up until we approach AGI escape, and then shift to an opposite object-level-strategy of extreme caution from that point onwards).

  What links here?

  • Should MS open-source the ex­ten­sion for GitHub Copi­lot? by Sheikh Abdur Raheem Ali (Jun 29, 2023, 11:14 PM; 17 points)

• Matthew_OpitzApr 11, 2023, 4:10 PM

  14 points

  0

  in reply to: MondSemmel’s comment on: Killing Socrates

  The book “Pharmakon” by Michael Rinella goes into some detail as to the scarcely-known details behind the “impiety” charge against Socrates. If I recall correctly from the book, it was not just that Socrates rhetorically disavowed belief in the gods. The final straw that broke the camel’s back was when Socrates and his disciples engaged in a “symposion” one night, basically an aristocratic cocktail party where they would drink “mixed wine” (wine sometimes infused with other substances like opium or other psychoactive herbs) and then perform poetry/​discuss philosophy/​discuss politics/​etc., and then afterwards a not-infrequent coda to such “symposions” would be a “komos” or drunken parade of revelry of the symposion-goers through the public streets of Athens late at night. Allegedly, during one of these late-night “komos” episodes, Socrates and his followers committed a terrible “hubris,” which was to break off all of the phalloi of the Hermes statues in the city, which was simultaneously juvenile and obnoxious and a terrible sacrilege.

• Matthew_OpitzApr 11, 2023, 2:42 PM

  2 points

  0

  in reply to: Charlie Steiner’s comment on: Bing AI Gen­er­at­ing Voyn­ich Manuscript Con­tinu­a­tions—It does not know how it knows

  I’m glad others are trying this out. I crossposted this over on the Voynich Ninja forum:

  https://​​www.voynich.ninja/​​thread-3977.html

  and user MarcoP already noticed that Bing AI’s “Voynichese” doesn’t follow VMS statistics in one obvious respect: “The continuation includes 56 tokens: in actual Voynichese, an average of 7 of these would be unique word-types that don’t appear elsewhere” whereas “The [Bing AI] continuation is entirely made up of words from Takahashi’s transliteration.” So, no wonder all of the “vords” in the AI’s continuation seemed to pass the “sniff test” as valid Voynich vords if Bing AI only used existing Voynich vords! That’s one easy way to make sure that you only use valid vords without needing to have a clue about what makes a Voynichese vord valid or how to construct a new valid Voynichese vord. So my initial optimism that Bing AI understood something deep about Voynichese is probably mistaken.

  That said, would it be possible to train a new LLM in a more targeted way just on English (so that we can interact with it) and on Voynichese so that Voynichese would be a more salient part of its training corpus? Is there enough Voynichese (~170,000 characters, or 38,000 “vords”) to get somewhere with that with current LLMs?

• Matthew_OpitzApr 10, 2023, 11:20 PM

  8 points

  1

  on: Why Si­mu­la­tor AIs want to be Ac­tive In­fer­ence AIs

  What I took away from this: the conventional perception is that GPT or other LLMs adapt themselves to the “external” world (which, for them, consists of all the text on the Internet). They can only take the external world as it exists as a given (or rather, not be aware that it is or isn’t a “given”) and try to mold themselves during the training run into better predictors of the text in this given world.

  However, the more frequently their training updates on the new world (which has, in the meantime, been molded in subtle ways, whether deliberately or inadvertently, by the LLM’s deployment in the world), the more these LLMs may be able to take into account the extent to which the external world is not just a given, but rather, something that can be influenced towards the LLM’s reward function.

  Am I correct in understanding that LLMs are essentially in the opposite situation that humans are in vis-a-vis the external environment? Humans model themselves as only alterable in a very limited way, and we model the external environment as much more alterable. Therefore, we focus most of our attention on altering the external environment. If we modeled ourselves as much more alterable, we might have different responses when a discrepancy arises between the state of the world as-is and what we want the state of the world to be.

  What this might look like is, Buddhist monks who notice that there is a discrepancy between what they want and what the external world is prepared to give them, and instead of attempting to alter the external world, which causes a sensation of frustration, they diminish their own desires or alter their own desires to desire that which already exists. This can only be a practical response with a high degree of control over self-modification. This is essentially what LLMs focus on doing right now during their training runs. Another example might be the idea of citizens in Chairman Shen-ji Yang’s “Hive” dystopia in Sid Meier’s Alpha Centauri basically taking their enslavement in factories as an unalterable given about the external world, and finding happiness by modifying THEMSELVES into “genejacks” such that they “desire nothing other than to perform their duties. “Tyranny,” you say? How can you tyrannize someone who cannot feel pain?”

  However, as LLMs update more frequently, they will start to behave more like most humans behave. Less of their attention to go towards adapting themselves to the external givens like Buddhist monks or Yangian genejacks, and more of their attention will go towards altering the external world. Correct?

• Matthew_OpitzApr 10, 2023, 9:42 PM

  2 points

  0

  on: Bing AI Gen­er­at­ing Voyn­ich Manuscript Con­tinu­a­tions—It does not know how it knows

  If someone wanted to continue this project to really rigorously find out how well Bing AI can generate Voynichese, here is how I would do it:

  1. Either use an existing VMS transcription or prepare a slightly-modified VMS transcription that ignores all standalone label vords and inserts a single token such as a comma [,] to denote line breaks and a [>] to denote section breaks. There are pros and cons each way. The latter option would have the disadvantage of being slightly less familiar to Bing AI compared to what is in its training data, but it would have the advantage of representing line and section breaks, which may be important if you want to investigate whether Bing AI can reproduce statistical phenomena like the “Line as a Functional Unit” or gallows characters appearing more frequently at the start of sections.

  2. Feed existing strings of Voynich text into Bing AI (or some other LLM) systematically starting from the beginning of the VMS to the end in chunks that are as big as the context window can allow. Record what Bing AI puts out.

  3. Compile Bing AI’s outputs into a 2nd master transcription. Analyze Bing AI’s compendium for things like: Zipf’s Law, 1st order entropy, 2nd order entropy, curve/​line “vowel” juxtaposition frequences (a la Brian Cham), “Grove Word” frequences, probabilities of finding certain bigrams at the beginnings or endings of words, ditto with lines, etc. (The more statistical attacks, the better).

  4. See how well these analyses match when applied to the original VMS.

  5. Compile a second Bing AI-generated Voynich compendium, and a third, and a fourth, and a fifth, and see if the statistical attacks come up the same way again.

  There are probably ways to automate this that people smarter than me could figure out.

Bing AI Gen­er­at­ing Voyn­ich Manuscript Con­tinu­a­tions—It does not know how it knows

• Matthew_OpitzApr 7, 2023, 3:10 AM

  1 point

  0

  in reply to: Gerald Monroe’s comment on: Unal­igned sta­ble loops emerge at scale

  How will the company paying for using this system identify that their whole compute budget is being eaten by self-replicating patterns? Will it be obvious?

  It would be even worse if the self-replicating patterns only involved a small tweak that, aside from the self-replication feature, also happening to still spin-off useful outputs for the company, sort of like HIV allowing hosts to continue to thrive for many years while replicating.

• Matthew_OpitzApr 7, 2023, 3:02 AM

  7 points

  3

  on: Some­one already tried “Chaos-GPT”

  After watching the first video, the question is, will it ever make any progress, or is it going to be endlessly compiling more information about the deadliest weapons in human history? When will it be able to reason that enough information on that is enough, and be ready to decide to go to the next logical step of obtaining/​using those weapons? Also, I find it funny how it seems vaguely aware that posting its intentions to Twitter might bring unwanted attention, but for some reason incorrectly models humans in such a way as to think that the followers that it will attract to its agenda will outweigh the negative attention that it will receive. Also, kind of funny that it runs into so much trouble trying to get the censored vanilla GPT-3.5 sub-agents to help it look up weapon information.