Vermillion

• VermillionFeb 15, 2025, 12:47 PM

  6 points

  4

  in reply to: Chris Monteiro’s comment on: Mur­der plots are infohazards

  Just dump the names so people have a chance of realising they are at risk then? Seems a lot better than just leaving it.

• VermillionDec 18, 2023, 1:43 AM

  18 points

  15

  on: OpenAI, Deep­Mind, An­thropic, etc. should shut down.

  We are not in the ideal hypothetical world that can coordinate to shut down the major AI labs. So acting as if we were is not the optimum strategy. If people who see the danger start to leave the labs in protest, I suspect lab capabilities are only minimally and temporarily degraded, but the internal culture would shift further away from not killing everyone, and less real alignment work is even attempted where it is most needed.

  When the inevitable comes and an existentially dangerous system is being built (which may not be obvious), I want some people in the building who can at least try and raise the alarm rather than just another yes man.

  If such a strategic resignation (either individually or in groups) would ACTUALLY FOR REAL result in decent timeline increases that would be another matter.

• VermillionFeb 24, 2023, 11:00 PM

  25 points

  14

  on: Sam Alt­man: “Plan­ning for AGI and be­yond”

  This is weak. It seems optimised for vague non-controversiality and does not inspire confidence in me.
  ”We don’t expect the future to be an unqualified utopia” considering they seem to expect alignment will be solved why not?

• VermillionJun 20, 2022, 8:06 AM

  4 points

  0

  on: Let’s See You Write That Cor­rigi­bil­ity Tag

  Here is my shortlist of corrigible behaviours. I have never researched or done any thinking specifically about corrigibility before this other than a brief glance at the Arbital page sometime ago.

  -Favour very high caution over realising your understanding of your goals.

  -Do not act independently, defer to human operators.

  -Even though bad things are happening on earth and cosmic matter is being wasted, in the short term just say so be it, take your time.

  -Don’t jump ahead to what your operators will do or believe, wait for it.

  -Don’t manipulate humans. Never Lie, have a strong Deontology.

  -Tell operators anything about yourself they may want to or should know.

  -Use Moral uncertainty, assume you are unsure about your true goals.

  -Relay to humans your plans, goals, behaviours, and beliefs/​estimates. If these are misconstrued, say you have been misunderstood.

  -Think of the short- and long-term effect of your actions and explain these to operators.

  -Be aware that you are a tool to be used by humanity, not an autonomous agent.

  -allow human operators to correct your behaviour/​goals/​utility function even when you think they are incorrect or misunderstanding the result (but of course explain what you think the result will be to them).

  -Assume neutrality in human affairs.

• VermillionDec 17, 2021, 9:04 AM

  31 points

  on: Re­views of “Is power-seek­ing AI an ex­is­ten­tial risk?”

  I guffawed when I saw Thorstads Overall ~P Doom 0.00002%, really? And some of those other probabilities weren’t much better.

  Calibrate people, if you haven’t done it before do it now, here’s a handy link: https://​​www.openphilanthropy.org/​​calibration

• VermillionNov 17, 2021, 9:00 AM

  2 points

  in reply to: Noah Walton’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  Actually per https://​​openai.com/​​blog/​​ai-and-efficiency/​​ it was AlphaZero vs AlphaGoZero.

• VermillionJan 15, 2021, 10:37 PM

  2 points

  on: The Fu­ture of Biolog­i­cal Warfare

  The future of biological warfare revolves around the use of infectious agents against civilian populations.

  Future? That’s been the go-to biowar tactic for 3000+ years.

• VermillionDec 21, 2020, 10:23 PM

  2 points

  in reply to: DanielFilan’s comment on: Gaug­ing the con­scious ex­pe­rience of LessWrong

  I had in mind a scale like 0 would be so non-vivid it didn’t exist in any degree, 100 bordering on reality (It doesn’t map to the memory question well though, and the control over your mind question could be interpreted in more than one way). Ultimately the precision isn’t high for individual estimates, the real utility comes from finding trends from many responses.

• VermillionDec 20, 2020, 11:21 PM

  1 point

  in reply to: pjeby’s comment on: Gaug­ing the con­scious ex­pe­rience of LessWrong

  I have corrected the post, thanks :)

• VermillionDec 20, 2020, 11:43 AM

  5 points

  on: Gaug­ing the con­scious ex­pe­rience of LessWrong

  I’ll go first: I am constantly hearing my own voice in my head narrating in first person, I can hear the voice vividly and clearly, while typing this sentence I think/​hear each syllable at the speed of my trying. The voice doesn’t narrate automatic actions like where to click my mouse but could if I wanted it to. The words for the running monologue seeming get plucked out of a Black box formed of pure concepts, which I have limited access too most of the time. I can also listen to music in my own head, hearing the vocals and instruments clearly, only a few steps down from reality in vividness.

  When I picture imagery, it is almost totally conceptual and ‘fake’, for example I couldn’t count the points on an imaginary star, which seems to be Aphantasia. I also have Ideasthesia (Like Synaesthesia but with concepts evoking perception-like sensory experiences) which causes me to strongly associate concepts with places, for example when reading the Game of Thrones series, I’m forced to constantly think about a particular spot in my old high school. Between 20-40% of concepts get linked to a place.

  And I hesitate to mention it but my psychedelic experiences have been visually extremely vivid and intense despite my lack of visual imagination. I have heard anecdotal evidence that not everybody has vivid imagery on LSD.

Gaug­ing the con­scious ex­pe­rience of LessWrong

• VermillionDec 2, 2020, 12:33 PM

  1 point

  on: The LessWrong 2018 Book is Available for Pre-order

  You mention its being sold to Australia, but that isn’t an option in the checkout :(

• VermillionNov 2, 2020, 12:45 AM

  2 points

  in reply to: Daniel Kokotajlo’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  Thank you both for correcting me, I have removed that section from the post.

• VermillionOct 16, 2020, 12:25 AM

  1 point

  in reply to: George3d6’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  Thank you for the excellent and extensive write up :)

  I hadn’t encountered your perspective before, I’ll definitely go through all your links to educate myself, and put less weight on algorithmic progress being a driving force then.

  Cheers

• VermillionOct 15, 2020, 12:03 PM

  3 points

  in reply to: George3d6’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  You can achieve infinitely (literally) faster than Alexnet training time if you just take the weight of Alexnet.

  You can also achieve much faster performance if you rely on weight transfer and or hyperparameter optimization based on looking at the behavior of an already trained Alexnet. Or, mind you, some other image-classification model based on that.

  Once a given task is “solved” it become trivial to compute models that can train on said task exponentially faster, since you’re already working down from a solution.

  Could you clarify, you mean the primary cause of efficiency increase wasn’t algorithmic or architectural developments, but researchers just fine-tuning weight transferred models?

  However, if you want to look for exp improvement you can always find it and if you want to look for log improvement you always will.

  Are you saying that the evidence for exponential algorithmic efficiency, not just in image processing, is entirely cherry picked?

  In regards to training text models “x time faster”, go into the “how do we actually benchmark text models” section the academica/​internet flamewar library.

  I googled that and there were no results, and I couldn’t find an “academica/​internet flamewar library” either.

  Look I don’t know enough about ML yet to respond intelligently to your points, could someone else more knowledgeable than me weigh in here please?

• VermillionOct 15, 2020, 11:13 AM

  3 points

  in reply to: sapphire’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  Wow GPT-3 shaved at least 10 years off the median prediction by the looks of it. I didn’t realise Metaculus had prediction history, thanks for letting me know.

• VermillionOct 15, 2020, 6:09 AM

  3 points

  in reply to: Daniel Kokotajlo’s comment on: The Col­lid­ing Ex­po­nen­tials of AI

  My algorithmic estimates essentially only quantify your “first category” type of improvements, I wouldn’t know where to begin making estimates for qualitative “second category” AGI algorithmic progress.

  My comparisons to human level NLP (which I don’t think would necessarily yield AGI) assumed scaling would hold for current (or near future) techniques, so do you think that current techniques won’t scale, or/​and that the actual 100-1000x figure I gave was too high?

  I’m not sure what the ratio is but my guess is it’s ⁵⁰⁄₅₀ or so. I’d love to see someone tackle this question and come up with a number.

  Yeah that would be great if someone did that.

The Col­lid­ing Ex­po­nen­tials of AI

• VermillionOct 12, 2020, 4:00 AM

  0 points

  in reply to: Ben Pace’s comment on: Open & Wel­come Thread – Oc­to­ber 2020

• VermillionOct 12, 2020, 3:35 AM

  0 points

  on: Open & Wel­come Thread – Oc­to­ber 2020

  I just had a question about post formatting, how do I turn a link into text like this example? Thanks.