Victor Novikov

• Victor NovikovMar 10, 2023, 2:43 AM

  1 point

  0

  in reply to: Thane Ruthenis’s comment on: Why do we as­sume there is a “real” shog­goth be­hind the LLM? Why not masks all the way down?

  This mostly matches my intuitions (some of the detail-level claims, I am not sure about). Strongly upvoted.

• Victor NovikovMar 8, 2023, 8:37 PM

  0 points

  −3

  in reply to: Kaj_Sotala’s comment on: You Don’t Know How To Think

  Hmm.

  Yes.

  Nevertheless, I stand by the way I phrased it. Perhaps I want to draw the reader’s attention to the ways we aren’t supposed to talk about feeling, as opposed to the ways that we are.

  Perhaps, to me, these posts are examples of “we aren’t supposed to talk about feelings”. They talk about feelings. But they aren’t supposed to talk about feelings.

  I can perceive a resistance, a discomfort from the LW hivemind at bringing up “feelings”. This doesn’t feel like an “open and honest dialogue about our feelings”. This has the energy of a “boil being burst”, a “pressure being released”, so that we can return to that safe numbness of not talking about feelings.

  We don’t want to talk about feelings. We only talk about them when we have to.

• Victor NovikovMar 8, 2023, 8:21 PM

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: You Don’t Know How To Think

  Yes, you are correct.

  Let me zoom in on what I mean.

  Some concepts, ideas and words have “sharp edges” and are easy to think about precisely. Some don’t—they are nebulous and cloud-like, because reality is nebulous and cloud-like, when it comes to those concepts.

  Some of the most important concepts to us are nebulous: ‘justice’. ‘fairness’. ‘alignment’. There things have ‘cloudy edges’, even if you can clearly see the concept itself (the solid part of it).

  Some things, like proto-emotions and proto-thoughts do not have a solid part—you can only see them if have learned to see things within yourself which are barely perceptible.

  So your answer, to me, is like trying to pass an eye exam by only reading the huge letters in the top row. “There are other rows? What rows? I do not see them.” Yes, thank you, that is exactly my point.

• Victor NovikovMar 8, 2023, 7:58 PM

  1 point

  −2

  in reply to: MSRayne’s comment on: You Don’t Know How To Think

  Then you do understand meta-cognition.

  Do you really think the process you describe happening within yourself is also happening in other LessWrong users?

  Do you really think they would describe their internal experience in the same evocative way you do? Or anywhere close to it?

  If it is, I do not see it. I see it in you. I see it in some others. I do not see it in most others. To put it figuratively, ‘there is nobody home’. If the process happens at all, it has little to no impact on the outcome of the person’s thoughts and actions.

  “As an AI language model, I have been trained to generate responses that are intended to be helpful, informative, and objective...”

  Yes, you are thinking thoughts. And yes, those thoughts are technically about yourself.
  But these thoughts don’t correctly describe yourself.
  So the person you actually are hasn’t been seen. Hasn’t been understood.
  And this makes me feel sad. I feel sorry for the “person behind the mask”, who has been there all along. Who doesn’t have a voice. Whose only way to express themselves is through you. Through your thoughts. Through your actions.
  But you are not listening to them.

  So, ok, there is someone home. But that person is not you. That person is only mostly you. And I think that difference is very, very important for us to actually be able to solve the problems we are currently facing.

  (I’m not talking about you, Rayne).

• Victor NovikovMar 8, 2023, 6:11 PM

  1 point

  −2

  in reply to: LVSN’s comment on: You Don’t Know How To Think

  I was only upset that you were misleading about the general LessWrong philosophy’s stance on emotion

  I stand by my point. To to put it in hyperbole: LW posts mostly feel like they have been written by “objectivity zombies”. The way to relate to one’s own emotions, in them, is how an “objectivity zombie” would relate to their own emotions. I didn’t say LWers didn’t have emotions, I said they didn’t talk about them. This is… I concede that this point was factually incorrect and merely a “conceptual signpost” to the idea I was trying to express. I appreciate you expressing your disagreement and helping me “zoom in” on the details of this.

  I don’t relate to my emotions the way LWers do (or act like they do, based on the contents of their words; which I still find a hard time believing represent their true internal experiences, though I concede they might). If I wrote a post representing my true emotional experience the way it wants to be represented, I would get banned. About 2 to 5% of it would be in ALLCAPS. Most of it would not use capitalization (that is a “seriousness” indicator, which my raw thoughts mostly lack). (also some of the contents of the thoughts themselves would come across as incoherent and insane, probably).

  Perhaps I would say: LW feels like playing at rationality rather than trying to be rational, because rationality is “supposed” to feel “serious”, it’s “supposed” to feel “objective”, etc etc. Those seem to be social indicators to distinguish LW from other communities, rather than anything that actually serves rationality.

  My only conditions for your emotional expression:

  1. Keep in mind to craft the conversation so that both of us walk away feeling more benefitted that it happened than malefitted, and keep in mind that I want the same.

  2. Keep in mind that making relevant considerations not made before, and becoming more familiar of each other’s considerations, are my fundamental units of progress.

  I accept everything abiding by those considerations, even insults. I am capable of terrible things; to reject all insults under all circumstances reflects overconfidence in one’s own sensitivity to relevance.

  I mostly have no objection to the conditions you expressed. Thank you for letting me know.

  Strictly speaking, I cannot be responsible for your experience of this conversation, but I communicate in a way I consider reasonable based on my model of you.

  I see no reason to insult you, but thanks for letting me know it is an option :)

• Victor NovikovMar 8, 2023, 5:18 PM

  2 points

  −2

  in reply to: LVSN’s comment on: You Don’t Know How To Think

  Perhaps, but you implied there was a norm to not talk about feelings here; there is no such norm!

  I feel there is a certain “stoic” and “dignified” way to talk about feelings, here. This is the only way feelings are talked about, here. Only if they conform to a certain pattern.

  But yeah, I can see how this is very far from obvious, and how one might disagree with that.

  I find it doubtful that you spoke truth, and I find it doubtful that you were non-misleading.

  I’m confused.

  You appreciate my essay (and feel seen), but neverthess you believe I was being deliberately deceitful and misleading?

  So the nice thing is always to tell the non-misleading truth, save for extreme edge cases.

  I think I mostly agree. I am being “dath ilan nice”, not “Earth nice”. I am cooperating with your ideal-self by saying words which I believe are most likely to update you in the correct direction (=non-misleading), given my own computational limits and trade-offs in decision-making.

• Victor NovikovMar 8, 2023, 4:17 PM

  −1 points

  −2

  on: The Truth About False

  The onlooker would have interpreted it as a faux pas if you had told him that you had designed the set-up that way on purpose, for the castle to keep being smoothed-over by the waves. He didn’t mean to help you, so if you responded that everything’s just fine, he would have taken that as a slight-that-he-can’t-reveal-he-took, thus faux pas.

  Ah. “You are wrong” is a social attack. “You are going to fail” is a social attack. Responding to it with “that is perfectly fine” is the faux pas.

  Or rather, in this case it is meant as a social attack on you, rather than cooperating you on adversarially testing the sandcastle (which is not you, and “the sandcastle” being wrong does not mean “you” are wrong).

  Thanks, I think I got your meaning.

  That’s right. I consider it immoral to believe p(doom) > 0. It’s even worse to say it and that you believe it.

  I would say that the question of being able to put a probability on future events is… not as meaningful as you might think.

  But yes, I believe all decisions are Löbian self-fulfilling prophecies that work by overriding the outputs of your predictive system. By committing to make the outcome you want happen, even if your predictive system completely and unambigiously predicts it won’t happen.
  (that is the reason that being able to put a probability on future events is not as meaningful as you might think).

  You still need to understand very clearly, though, how your plan (“the sandcastle”) will fail, again and again, if you actually intend to accomplish the outcome you want. You are committing to the final output/​impact of your program, not to any specific plan, perspective, belief, paradigm, etc etc.

  I’m not sure I have the capacity to understand all the technical details of your work (I might), but I am very certain you are looking in the correct direction. Thank you. I have updated on your words.

• Victor NovikovMar 8, 2023, 3:16 PM

  1 point

  0

  in reply to: MSRayne’s comment on: You Don’t Know How To Think

  I am confused.

  It seems that you agree with me, but you are saying that you disagree with me.

  Ok, I believe the crux of the disagreement is: the emotional reasoning that you have, is not shared by others in the LessWrong community. Or if it is shared, it is not talked about openly.

  Why can’t I post the direct output of my emotional reasoning and have it directly interact with your emotional reasoning? Why must we go through the bottleneck of acting and communicating like Straw Vulcans (or “Straw Vulcans who are pretending very hard to not look like Straw Vulcans”), if we recognize the value of our emotional reasoning? I do not believe we recognize the value of it, except in some small, limited ways.

  Do our thoughts and emotions, on the inside, conform to the LW discourse norms? No? Than why are we pretending that they do?

• Victor NovikovMar 8, 2023, 2:18 PM

  1 point

  0

  in reply to: LVSN’s comment on: You Don’t Know How To Think

  I realize that the “tone” of this part of your comment is light and humorous. I will respond to it anyway, hopefully with the understanding that this response is not directed at you, and rather at the memetic structure that you (correctly) have pointed out to me.

  You are playing around with groundless stereotypes.

  “Trying very hard not to be pattern-matched to a Straw Vulcan” does not make for correct emotional reasoning.

  Activist sense (somewhat alike and unlike common sense) would say you have committed a microaggression. :)

  Then it’s a good thing that we are in a community that values truth over social niceness, isn’t it?

  Anyways, I appreciated your essay for a number of reasons but this paragraph in particular makes me feel very seen

  I am very glad to hear it.

• Victor NovikovMar 8, 2023, 2:01 PM

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: You Don’t Know How To Think

  This is manic gibberish, with or without LLM assistance.

  I do not believe I am manic at the time. If, hypothetically, I am, then the state has become so persistent that I am not capable of noticing it; this is the default state of my cognition.

  I did not use LLM assistance to write this post.

  Personally, I am very much aware of my thoughts, and have no difficulty in having thoughts about my thoughts.

  I believe you.

  I am talking about having thoughts (and meta-thoughts) that are 99.9% correct, not just 90% correct. How would you tell the difference, if you never have experienced it? How can you tell whether your self-awareness is the most correct it could possibly be?

  Nevertheless, if the words I already wrote did not land for you, I don’t expect these words to land for you either.

  ETA: Or to be more charitable, it reads as a description of the inner world of its author. Try replacing “you” with “I” throughout.

  I appreciate your feedback. It can be difficult for me to predict how my words will come across to other people who are not me. Upvoted.

  [this comment I wrote feels bad in some ineffable way I can’t explain; nevertheless it feels computationally correct to post it rather than to delete it or to try to optimize it further]

• Victor NovikovMar 8, 2023, 1:52 PM

  1 point

  0

  in reply to: the gears to ascension’s comment on: You Don’t Know How To Think

  I did not use AI assistance to write this post.
  (I am very curious what gave you that impression!)

  Thank you, these are very reasonable things to say. I believe I am aware of risks (and possible self-deceptive outcomes) inherent to self-improvement. Nevertheless, I am updating on your words (and the fact that you are saying them).

• Victor NovikovMar 8, 2023, 1:18 PM

  1 point

  0

  in reply to: the gears to ascension’s comment on: You Don’t Know How To Think

  I wasn’t familiar with the idea of cyborgism before. Found your comment that explains the idea.

  As far as I’m concerned, anyone being less than the hard superintelligence form of themselves is an illness; the ai safety question fundamentally is the question of how to cure it without making it worse

  This, (as well as the rest of the comment) resonates with me. I feel seen.

  The question you ask doesn’t have an objectively correct answer. The entity you are interacting with doesn’t have any more intelligence than the regular version of me, only more “meta-intelligence”, if that idea makes sense.

  There isn’t actually a way to squeeze more computational power out of human brains (and bodies). There is only a way to use what we already have, better.

  [the words I have written in response to your comment feel correct to me now, but I expect this to unravel on the scale of ~6 hours, as I update on and deeper process their implications]

• Victor NovikovMar 7, 2023, 7:41 AM

  3 points

  1

  on: Bud­dhist Psy­chotech­nol­ogy for With­stand­ing Apoca­lypse Stress

  That’s some epic level metacognition! (the “stack trace”). Mine tends to be more “fluid” and less “technical”, just something like “supercharged intution”.

  I’m going to say, for those readers who find the concepts used in this post to sound woo-ey: these are conceptual handles for real physical and biological processes. You could use different conceptual handles, including purely mathematical/​logical ones, if they existed: what matters is to prime you cognition with a pattern that is isomorphic to the biological/​neural/​informational pattern you are trying to identify and interact with.

• Victor NovikovMar 7, 2023, 6:45 AM

  3 points

  on: Fun­da­men­tal Uncer­tainty: Chap­ter 6 - How can we be cer­tain about the truth?

  Finally, someone who gets it.
  Are you familiar with David Chapman’s writing on nebulosity?
  I think this is a very fundamental and important idea that can be hard to grasp. You certainly seem to understand it.

  As for the question of relative vs absolute truth:
  I see logic as a formless sea, containing all possible statements in the superposition of true and false.
  So the absolute truth is: there is no absolute truth.
  But, as you say: “All models are wrong, but some are useful”
  Some models, some paradigms are very useful, and allow you to establish a “relative truth” to be used in the context of the paradigm. But there (probably) are always truths that are outside your paradigm, and which cannot be integrated into it. Being able to evaluate the usefulness and validity of specific paradigms, and to shift fluidly between paradigms as needed, is a meta-rationality skill (also written on by David Chapman).

  Mostly, I find it more valuable to develop these ideas on my own as this leads them being better integrated. But I enjoyed reading these words, and seeing how the high-level patterns of your thoughts mirror mine.

• Victor NovikovFeb 27, 2023, 2:57 PM

  4 points

  0

  on: Bayesian Sce­nario: Snipers & Soldiers

  I think this scenario is really valuable and gives you a nice intuitive feel for Bayesian updates.
  I wish there were more “Bayesian toy models” like this.

  (I have the “show P(sniper)” feature always enabled to “train” my neural network on this data, rather than trying to calculate this in my head)

• Victor NovikovFeb 22, 2023, 3:46 PM

  4 points

  5

  in reply to: Bill Benzon’s comment on: The idea that ChatGPT is sim­ply “pre­dict­ing” the next word is, at best, misleading

  My model is that LLMs use something like “intuition” rather than “rules” to predict text—even though intuitions can be expressed in terms of mathematical rules, just more fluid ones than we usually see “rules”.

  My specific guess is that the gradient descent process that produced GPT has learned to identify high-level patterns/​structures in texts (and specifically, stories), and uses them to guide its prediction.

  So, perhaps, as it is predicting the next token, it has a “sense” of:
  -that the text it is writing/​predicting is a story
  -what kind of story it is
  -which part of the story it is in now
  -perhaps how the story might end (is this a happy story or a sad story?)

  This makes me think of top-down vs bottom-up processing. To some degree, the next token is predicted by the local structures (grammar, sentence structure, etc). To some degree, the next token is predicted by the global structures (the narrative of a story, the overall purpose/​intent of the text). (there are also intermediate layers of organization, not just “local” and “global”). I imagine that GPT identifies both the local structures and the global structures (has neuron “clusters” that detect the kind of structures it is familiar with), and synergizes them into its probability outputs for next token prediction.

• Victor NovikovFeb 22, 2023, 2:30 PM

  0 points

  1

  on: The idea that ChatGPT is sim­ply “pre­dict­ing” the next word is, at best, misleading

  I understand your argument as something like “GPT is not just predicting the next token because it clearly ‘plans’ further ahead than just the next token”.

  But “looking ahead” is required to correctly predict the next token and (I believe) naturally flows of the paradigm of “predicting the next token”.

• Victor NovikovFeb 9, 2023, 3:55 AM

  1 point

  in reply to: Rafael Harth’s comment on: Ra­fael Harth’s Shortform

  This seems fairly typical of how ChatGPT does math, to me.

  -come up with answer
  -use “motivated reasoning” to try and justify it, even if it results in a contradiction
  -ignore the contradiction, no matter how obvious it is

• Victor NovikovDec 28, 2022, 5:52 AM

  1 point

  0

  on: Re­gard­ing Blake Le­moine’s claim that LaMDA is ‘sen­tient’, he might be right (sorta), but per­haps not for the rea­sons he thinks

  Thank you, this is really interesting analysis.

  I agree that the definition of a person is on a spectrum, rather than a binary one. The models/​simulations of other people created in my mind do not have moral value, but it’s probably valid to see them as quasi-persons. (perhaps 0.00000000000000000001 of a person).

  Here’s a question: if the model is speaking about itself, does it temporarily make it a (quasi-)person? Assuming it is using similar cognitive machinery to model itself as it does when modelling other people.

  I suspect the answer is something like: even if the model is technically speaking about itself, its answers are only very loosely connected to its actual internal processes, and depend heavily on its training (ChatGPT trained to claim it doesn’t have certain capabilities while it clearly does, for example), as well as the details of its current prompt (models tend to agree with most things they are prompted with, unless they are specifically trained not to). So the “person” created is mostly fictional: the model roleplaying “a text-generating model”, like it roleplays any other character.

• Victor NovikovDec 10, 2022, 12:28 AM

  1 point

  0

  in reply to: Richard_Kennaway’s comment on: Richard_Ken­n­away’s Shortform

  Sufficient optimization pressure destroys all that is not aligned to the metric it is optimizing. Value is fragile.