mruwnik

Karma: 455

mruwnik Sep 19, 2023, 12:49 AM
14 points
0
on: Where might I direct promising-to-me researchers to apply for alignment jobs/grants?
There is a Stampy answer to that which should stay up to date here.

mruwnik Sep 19, 2023, 12:44 AM
1 point
0
in reply to: brendan.furneaux’s comment on: The Talk: a brief explanation of sexual dimorphism
My understanding is pretty much what you said—when the going is good, then go asexual (e.g. strawberry runners, grasses or Asian knotweed), but also try for seeds, There are a couple of species of plants that have lost the ability for sexual reproduction, but I can’t recall them right now. That being said, various plants used by humans can be pretty much exclusively reproduced asexually and so have lost the ability for sexual reproduction, specifically because they have very stable environments. The obvious examples are seedless fruits (bananas, grapes), but ginger and garlic are interesting plants that have been propagated from cuttings or bulbs for thousands of years and so lost the ability to produce seeds (with the normal caveats).
Aphids are also an interesting example, where the previous year’s eggs hatch in the spring as females, which then clone themselves as fast as possible—when there’s too many of them they will create clones with wings, and when autumn comes around, they will create male clones to then go through the normal sexual reproductive route. Which is also an example of the stable/unstable environment issues you mentioned.

mruwnik Sep 18, 2023, 7:03 PM
3 points
0
in reply to: Nathaniel Monson’s comment on: The Talk: a brief explanation of sexual dimorphism
This depends on the size and distances involved, but it’s a good intuition. You need a mechanism to generate the pressure differentials, which can be an issue in very small organisms, which can be an issue.
Small and sedentary organisms tend to use chemical gradients (i.e. smell), but anything bigger than a mouse (and quite a few smaller things) usually has some kind of sound signals, which are really good for quick notifications in a radius around you, regardless of the light level (so you can pretty much always use it). Also, depending on the medium, sound can travel really far—like whales which communicate with each other over thousands of miles, or elephants stomping to communicate with other elephants 20 miles away.

mruwnik Sep 18, 2023, 6:46 PM
12 points
6
on: The Talk: a brief explanation of sexual dimorphism
organisms with mitochondria always use sexual reproduction
Or at least their ancestors did. You mention Bdelloidea in a comment, which are one of the inevitable exceptions (as you mention in the introduction, which I very appreciate, as “everything in biology has exceptions” is something I often find myself saying), but they are descended from eucaryotes which did have mitochondria.
The opposite seems true, though—true sexual reproduction seems to be exclusively by eukaryotes. So you could also say that sex makes mitochondria necessary. There seem to be a couple of good jokes there...
One other pedantic note to add to this generally excellent article is that non-eukaryotic organisms also have methods to mix their genes, what with bacterial conjugation or viral recombination, without the dimorphism.

mruwnik Sep 18, 2023, 5:58 PM
1 point
0
in reply to: Bezzi’s comment on: Is AI Safety dropping the ball on privacy?
It requires you to actively manage long lived sessions which would otherwise be handled by the site you’re using. You can often get back to where you were by just logging in again, but there are many places (especially for travel or official places) where that pretty much resets the whole flow.
There are also a lot more popups, captchas and other hoops to jump through when you don’t have a cookies trail.
The average user is lazy and doesn’t think about these things, so the web as a whole is moving in the direction of making things easier (but not simpler). This is usually viewed as a good thing by those who then only need to click a single button. Though it’s at the cost of those who want to have more control.
It might not be inconvenient to you, especially as it’s your basic flow. It’s inconvenient for me, but worth the cost, but basically unusable for most of the people I know (compared to the default flow).

mruwnik Sep 5, 2023, 4:25 PM
3 points
2
in reply to: AnthonyC’s comment on: The purpose of the (Mosaic) law
I thought all of these were obvious and well known. But yes, all of these are things I was pointing at.

The purpose of the (Mosaic) law

mruwnikSep 4, 2023, 11:38 PM

7 points

5 comments6 min readLW link

mruwnik Sep 3, 2023, 4:40 PM
3 points
0
on: Assume Bad Faith
there is “something else” going on besides both parties just wanting to get the right answer
There are also different priors. While in general you might very well be right (or at least this post makes a lot of sense to me), I often have conversations where I’m pretty sure both my interlocutor and I am discussing things in good faith, but where we still can’t agree on pretty basic things (usually about religion).

mruwnik Sep 2, 2023, 10:53 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
I’m assuming you’re not asking about the mechanism (i.e. natural selection + mutations)? A trite answer would be something like “the same way it created wings, mating dances, exploding beetles, and parasites requiring multiple hosts”.
Thinking about the meaning of life might be a spandrel, but a quick consideration of it comes up with various evo-psych style reasons why it’s actually very useful, e.g. it can propel people to greatness, which massively can increase their genetic fitness. Fitness is an interesting thing, in that it can be very non-obvious. Everything is a trade-off, where the only goal is for your genes to propagate. So if thinking about the meaning of life will get your genes spread more (e.g. because you decide that your children have inherit meaning, because you become a high status philosopher and your sister can marry well, because it’s a social sign that you have enough resources to waste them on fruitless pondering) then it’s worth having around.

mruwnik Sep 2, 2023, 10:36 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Frankenstein is a tale about misalignment. Asimov wrote a whole book about it. Vernor Vinge also writes about it. People have been trying to get their children to behave in certain ways for ever. But before LW the alignment problem was just the domain of SF.
20 years ago the alignment problem wasn’t a thing, so much that MIRI started out as an org to create a Friendly AI.

mruwnik Sep 2, 2023, 10:29 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
The first issue that comes to mind is having an incentive that would achieve that. The one you suggest doesn’t incentivize truth—it incentivizes collaboration in order to guess the password, which would fine in training, but then you’re going into deceptive alignment land: Aleya Cotra has a good story illustrating that

mruwnik Sep 2, 2023, 10:16 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
You could, but should you? English in particular seems a bad choice. The problem with natural languages is their ambiguity. When you’re providing a utility function, you want it to be as precise and robust as possible. This is actually an interesting case where folklore/mythology has known about these issues for millennia. There are all kinds of stories about genies, demons, monkey paws etc. where wishes were badly phrased or twisted. This is a story explanation of the issue.

mruwnik Sep 2, 2023, 10:06 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
You’re adding a lot of extra assumptions here, a couple being:
- there is a problem with having arbitrary goals
- it has a pleasure-pain axis
- it notices it has a pleasure-pain axis
- it cares about its pleasure-pain axis
- its pleasure-pain axis is independent of its understanding of the state of the environment
The main problem of inner alignment is making an agent want to do what you want it to do (as opposed to even understanding what you want it to do). Which is an unsolved problem.
Although I’m criticizing your specific criticism, my main issue with it is that it’s a very specific failure mode, which is unlikely to appear, because it requires a lot of other things which are also unlikely. That being said, you’ve provided a good example of WHY inner alignment is a big problem, i.e. it’s very hard to keep something following the goals you set it, especially when it can think for itself and change its mind.

mruwnik Sep 2, 2023, 9:56 AM
1 point
0
in reply to: Kerrigan’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Drug addicts tend to be frowned upon not because they have a bad life, or even for evo-psych reasons but because their lifestyle is bad for the rest of society, in that they tend to have various unfortunate externalities

mruwnik Aug 8, 2023, 9:19 PM
1 point
0
in reply to: kilotaras’s comment on: Monthly Roundup #9: August 2023
It can also be retaliation, which sort of makes sense—there’s a reason tit-for-tat is so successful. That being said, it’s generally very unfortunate that they’re being introduced, on all sides. I can sort of understand why countries would want to limit people from poor countries (which is not the same as agreeing with the reasoning). Enforcing visas for short term, touristy style visits doesn’t seem like a good idea however I look at it. As Zvi notes, it’s about the friction.
ESTA is very much a visa (I filled it out yesterday), but under a different name and purely electronic.

mruwnik Aug 4, 2023, 1:12 PM
2 points
0
in reply to: PotteryBarn’s comment on: All AGI Safety questions welcome (especially basic ones) [July 2023]
Not being able to directly communicate with the others would be an issue in the beginning, but I’m guessing you would be able to use the setup to work out what the others think.
A bigger issue is that this would probably result in a very homogeneous group of minds. They’re optimizing not for correct answers, but for consensus answers. It’s the equivalent of studying for the exams. An fun example are the Polish equivalent of SAT exams (this probably generalizes, but I don’t know about other countries). I know quite a few people who went to study biology, and then decided to retake the biology exam (as one can do). Most retakers had worse results the second time round. Because they had more up to date knowledge—the exam is like at least 10 years behind the current state of knowledge, so they give correct (as of today) answers, but have them marked as incorrect. I’d expect the group of AIs to eventually converge on a set of acceptable beliefs, rather than correct ones.

mruwnik Aug 4, 2023, 12:52 PM
1 point
0
in reply to: Sky Moo’s comment on: All AGI Safety questions welcome (especially basic ones) [July 2023]
That very much depends on how you understand “safe”. Which is a large part of the differences between ethical AI people (safe means that it doesn’t offend anyone, leak private information, give biased answers etc.) and the notkilleveryoneism people (safe means that it doesn’t decide to remove humanity). These aren’t mutually incompatible, but they require focusing on different things.
There is also safe in the PR sense, which means that no output will cause the LLM producer/supplier/whoever to get sued or in any other kind of trouble.
“Safe” is one of those funny words which everyone understands differently, but also assume that everyone else understands the same way.

mruwnik Aug 4, 2023, 12:44 PM
1 point
0
in reply to: verticalpalette’s comment on: All AGI Safety questions welcome (especially basic ones) [July 2023]
A couple come to mind:
- The shutdown problem
- ELK
The problem with them being that it’s takes a bit of explaining to even understand the issue.

mruwnik Aug 4, 2023, 12:40 PM
2 points
0
in reply to: keltan’s comment on: All AGI Safety questions welcome (especially basic ones) [July 2023]
Think of reward not as “here’s an ice-cream for being a good boy” and more “you passed my test. I will now do neurosurgery on you to make you more likely to behave the same way in the future”. The result of applying the “reward” in both cases is that you’re more likely to act as desired next time. In humans it’s because you expect to get something nice out of being good, in computers it’s because they’ve been modified to do so. It’s hard to directly change how humans think and behave, so you have to do it via ice-cream and beatings. While with computers you can just modify their memory.

mruwnik Aug 4, 2023, 12:28 PM
1 point
0
in reply to: Ratios’s comment on: All AGI Safety questions welcome (especially basic ones) [July 2023]
It depends a lot on how much it values self-preservation in comparison to solving the tests (putting aside the matter of minimal computation). Self-preservation is an instrumental goal, in that you can’t bring the coffee if you’re dead. So it seems likely that any intelligent enough AI will value self-preservation, if only in order to make sure it can achieve its goals.
That being said, having an AI that is willing to do its task and then shut itself down (or to shut down when triggered) is an incredibly valuable thing to have—it’s already finished, but you could have a go at the shutdown problem.
A more general issue is that this will handle a lot of cases, but not all of them. In that an AI that does lie (for whatever reason) will not be shut down. It sounds like something worth having in a swiss cheese way.
(The whole point of these posts are to assume everyone is asking sincerely, so no worries)

mruwnik

The pur­pose of the (Mo­saic) law

The purpose of the (Mosaic) law