Decaeneus

• DecaeneusFeb 9, 2024, 6:14 PM

  3 points

  0

  in reply to: porby’s comment on: porby’s Shortform

  For what it’s worth (perhaps nothing) in private experiments I’ve seen that in certain toy (transformer) models, task B performance gets wiped out almost immediately when you stop training on it, in situations where the two tasks are related in some way.

  I haven’t looked at how deep the erasure is, and whether it is far easier to revive than it was to train it in the first place.

• DecaeneusFeb 9, 2024, 6:08 PM

  3 points

  0

  on: De­caeneus’s Shortform

  Reflecting on the particular ways that perfectionism differs from the optimal policy (as someone who suffers from perfectionism) and looking to come up with simple definitions, I thought of this:

  • perfectionism looks to minimize the distance between an action and the ex-post optimal action but heavily dampening this penalty for the particular action “do nothing”

  • optimal policy says to pick the best ex-ante action out of the set of all possible actions, which set includes “do nothing”

  So, perfectionism will be maximally costly in an environment where you have lots of valuable options of new things you could do (breaking from status quo) but you’re unsure whether you can come close to the best one, like you might end up choosing something that’s half as good as the best you could have done. Optimal policy would say to just give it your best, and that you should be happy since this is an amazingly good problem to have, whereas perfectionism will whisper in your ear how painful it might be to only get half of this very large chunk of potential utility, and wouldn’t it be easier if you just waited.

• DecaeneusJan 28, 2024, 1:52 AM

  1 point

  0

  in reply to: Perhaps’s comment on: De­caeneus’s Shortform

  The parallel to athlete pre game rituals is an interesting one, but I guess I’d be interested in seeing the comparison between the following two groups:

  group A: is told to meditate the usual way for 30 minutes /​ day, and does

  group B: is told to just sit there for 30 minutes /​ day, and does

  So both of the groups considered are sitting quietly for 30 minutes, but one group is meditating while the other is just sitting there. In this comparison, we’d be explicitly ignoring the benefit from meditation which acts via the channel of just making it more likely you actually sit there quietly for 30 minutes.

• DecaeneusJan 27, 2024, 7:58 PM

  2 points

  0

  on: De­caeneus’s Shortform

  Is meditation provably more effective than “forcing yourself to do nothing”?

  Much like sleep is super important for good cognitive (and, of course, physical) functioning, it’s plausible that waking periods of not being stimulated (i.e. of boredom) are very useful for unlocking increased cognitive performance. Personally I’ve found that if I go a long time without allowing myself to be bored, e.g. by listening to podcasts or audiobooks whenever I’m in transition between activities, I’m less energetic, creative, sharp, etc.

  The problem is that as a prescription “do nothing for 30 minutes” would be rejected as unappealing by most. So instead of “do nothing” it’s couched as “do this other thing” with a focus on breathing and so on. Does any of that stuff actually matter or does the benefit just come from doing nothing?

• DecaeneusJan 26, 2024, 9:34 PM

  1 point

  0

  in reply to: tailcalled’s comment on: De­caeneus’s Shortform

  To be sure, I’m not an expert on the topic.

  Declines in male fertility I think are regarded as real, though I haven’t examined the primary sources.

  Regarding female fertility, this report from Norway outlines the trend that I vaguely thought was representative of most of the developed world over the last 100 years.

  Female fertility is trickier to measure, since female fertility and age are strongly correlated, and women have been having kids later, so it’s important (and likely tricky) to disentangle this confounder from the data.

• DecaeneusJan 26, 2024, 2:56 PM

  6 points

  0

  on: De­caeneus’s Shortform

  Infertility rates are rising and nobody seems to quite know why. Below is what feels like a possible (trivial) explanation that I haven’t seen mentioned anywhere.

  I’m not in this field personally so it’s possible this theory is out there, but asking GPT about it doesn’t yield the proposed explanation: https://​​chat.openai.com/​​share/​​ab4138f6-978c-445a-9228-674ffa5584ea

  Toy model:

  • a family is either fertile or infertile, and fertility is hereditary

  • the modal fertile family can have up to 10 kids, the modal infertile family can only have 2 kids

  • in the olden days families aimed to have as many kids as they could

  • now families aim to have 2 kids each

  Under this model, in the olden days we would find a high proportion of fertile people in the gene pool, but in the modern world we wouldn’t. Put differently, the old convention lead to a strong positive correlation between fertility and participation in the gene pool, and the new convention leads to 0 correlation. This removes the selective pressure on fertility, hence we should expect fertility to drop /​ infertility to rise.

  Empirical evidence for this would be something like an analysis of the time series of family size variance and infertility—is lower variance followed by increased infertility?

De­caeneus’s Shortform

• DecaeneusJan 12, 2024, 8:23 PM

  LW: 3 AF: 3

  0

  AF

  in reply to: Steven Byrnes’s comment on: My AGI safety re­search—2022 re­view, ’23 plans

  Thanks for the thoughtful reply. I read the fuller discussion you linked to and came away with one big question which I didn’t find addressed anywhere (though it’s possible I just missed it!)

  Looking at the human social instinct, we see that it indeed steers us towards not wanting to harm other humans, but it weakens when extended to other creatures, somewhat in proportion to their difference from humans. We (generally) have lots of empathy for other humans, less so for apes, less so for other mammals (who we factory farm by the billions without most people particularly minding it) probably less so for octopi (who are bright but quite different) and almost none to the zillion microorganisms, some of which we allegedly evolved from. I would guess that even canonical Good Person Paul Christiano probably doesn’t lose much sleep over his impact on microorganisms.

  This raises the question of whether the social instinct we have, even if fully reverse engineered, can be deployed separately from the identity of the entity to which it is attached. In other words, if the social instinct circuitry humans have is “be nice to others in proportion to how similar to yourself they are”, which seems to match the data, then we would need more than just the ability to place that circuitry in AGIs (which would presumably make the AGIs want to be nice to other similar AGIs). We would in fact need to be able to tease apart the object of empathy, and replace it with something that is very different than how humans operate, since no human is nice to microorganisms, so I see no evidence that the existing social instincts ever make any person be nice to something very different, and much weaker, than them, so I would expect it to work similarly in an AGI.

  This is speculative, but it seems reasonably likely to me to turn out to be an actual problem. Curious if you have thoughts on it.

• DecaeneusJan 11, 2024, 8:53 PM

  LW: 3 AF: 2

  0

  AF

  on: My AGI safety re­search—2022 re­view, ’23 plans

  This is drifting a bit far afield from the neurobio aspect of this research, but do you have an opinion about the likelihood that a randomly sampled human, if endowed with truly superhuman powers, would utilize those powers in a way that we’d be pleased to see from an AGI?

  It seems to me like we have many salient examples of power corrupting, and absolute power corrupting to a great degree. Understanding that there’s a distribution of outcomes, do you have an opinion about the likelihood of benevolent use of great power, among humans?

  This is not to say that this understanding can’t still be usefully employed, but somehow it seems like a relevant question. E.g. if it turns out that most of what keeps humans acting pro-socially is the fear that anti-social behavior will trigger their punishment by others, that’s likely not as juicy a mechanism since it may be hard to convince a comparatively omniscient and omnipotent being that it will somehow suffer if it does anti-social things.

• DecaeneusMay 22, 2023, 4:25 PM

  1 point

  0

  in reply to: Andy_McKenzie’s comment on: The ba­sic rea­sons I ex­pect AGI ruin

  Understood, and agreed, but I’m still left wondering about my question as it pertains to the first sigmoidal curve that shows STEM-capable AGI. Not trying to be nitpicky, just wondering how we should reason about the likelihood that the plateau of that first curve is not already far above the current limit of human capability.

  A reason to think so may be something to do with irreducible complexity making things very hard for us at around the same level that it would make them hard for a (first-gen) AGI. But a reason to think the opposite would be that we have line of sight to a bunch of amazing tech already, it’s just a question of allocating the resources to support sufficiently many smart people working out the details.

  Another reason to think the opposite is that having a system that’s (in some sense) directly optimized to be intelligent might just have a plateau drawn from a higher-meaned distribution than one that’s optimized for fitness, and develops intelligence as a useful tool in that direction, since the pressure-on-intelligence for that sort of caps out at whatever it takes to dominate your immediate environment.

• DecaeneusMay 11, 2023, 7:24 PM

  1 point

  0

  in reply to: Andy_McKenzie’s comment on: The ba­sic rea­sons I ex­pect AGI ruin

  As a result, rather than indefinite and immediate exponential growth, I expect real-world AI growth to follow a series of sigmoidal curves, each eventually plateauing before different types of growth curves take over to increase capabilities based on different input resources (with all of this overlapping).

  Hi Andy—how are you gauging the likely relative proportions of AI capability sigmoidal curves relative to the current ceiling of human capability? Unless I’m misreading your position, it seems like you are presuming that the sigmoidal curves will (at least initially) top out at a level that is on the same order as human capabilities. What informs this prior?
  
  Due to the very different nature of our structural limitations (i.e. a brain that’s not too big for a mother’s hips to safely carry and deliver, specific energetic constraints, the not-very-precisely-directed nature of the evolutionary process) vs an AGI’s system’s limitations (which are simply different) it’s totally unclear to me why we should expect the AGI’s plateaus to be found at close-to-human levels.

[Question] Daisy-chain­ing ep­silon-step verifiers

• DecaeneusApr 5, 2023, 3:59 PM

  1 point

  on: You Have About Five Words

  Might LLMs help with this? You could have a 4.3 million word conversation with an LLM (with longer context windows than what’s currently available) which could then, in parallel, have similarly long conversations with arbitrarily many members of the organization, adequately addressing specific confusions individually, and perhaps escalating novel confusions to you for clarification. In practice, until the LLMs become entertaining enough, members of the organization may not engage for long enough, but perhaps this lack of seductiveness is temporary.