V_V

Karma: 2,131

V_V 28 Sep 2016 23:37 UTC
0 points
on: New Philosophical Work on Solomonoff Induction
Very interesting, thanks for sharing.

V_V 13 Jun 2016 17:26 UTC
0 points
on: Google Deepmind and FHI collaborate to present research at UAI 2016
Talking of yourself in third person? :)

Cool paper!

Anyway I’m a bit bothered by the theta thing, the probability that the agent complies with the interruption command. If I understand correctly, you can make it converge to 1, but if it converges to quickly then the agent learns a biased model of the world, while if it converges too slowly it is unsafe of course.
I’m not sure if this is just a technicality that can be circumvented or if it represents a fundamental issue: in order for the agent to learn what happens after the interruption switch is pressed, it must ignore the interruption switch with some non-negligible probability, which means that you can’t trust the interruption switch as a failsafe mechanism.

V_V 31 May 2016 17:02 UTC
0 points
in reply to: ShardPhoenix’s comment on: The AI in Mary’s room
If you know that it is a false memory then the experience is not completely accurate, though it may be perhaps more accurate than what human imagination could produce.

V_V 28 May 2016 19:30 UTC
0 points
in reply to: Kaj_Sotala’s comment on: S.E.A.R.L.E’s COBOL room
Except that if you do word2vec or similar on a huge dataset of (suggestively named or not) tokens you can actually learn a great deal of their semantic relations. It hasn’t been fully demonstrated yet, but I think that if you could ground only a small fraction of these tokens to sensory experiences, they you could infer the “meaning” (in an operational sense) of all of the other tokens.

V_V 28 May 2016 19:02 UTC
−1 points
in reply to: ShardPhoenix’s comment on: The AI in Mary’s room

Consider a situation where Mary is so dexterous that she is able to perform fine-grained brain surgery on herself. In that case, she could look at what an example of a brain that has seen red looks like, and manually copy any relevant differences into her own brain. In that case, while she still never would have actually seen red through her eyes, it seems like she would know what it is like to see red as well as anyone else.

But in order to create a realistic experience she would have to create a false memory of having seen red, which is something that an agent (human or AI) that values epistemic rationality would not want to do.

V_V 28 May 2016 18:59 UTC
3 points
on: The AI in Mary’s room
The reward channel seems an irrelevant difference. You could make the AI in Mary’s room thought experiment by just taking the Mary’s room thought experiment and assuming that Mary is an AI.

The Mary AI can perhaps simulate in a fairly accurate way the internal states that it would visit if it had seen red, but these simulated states can’t be completely identical to the states that the AI would visit if it had actually seen red, otherwise the AI would not be able to distinguish simulation form reality and it would be effectively psychotic.

V_V 29 Apr 2016 15:40 UTC
1 point
0
in reply to: gjm’s comment on: JFK was not assassinated: prior probability zero events
The problem is that the definition of the event not happening is probably too strict. The worlds that the AI doesn’t care about don’t exist its decision-making purposes, and in the world that the AI cares about, the AI assigns high probability to hypotheses like “the users can see the message even before I send it through the noisy channel”.

V_V 29 Apr 2016 14:30 UTC
1 point
0
in reply to: Stuart_Armstrong’s comment on: JFK was not assassinated: prior probability zero events

I am not planting false beliefs. The basic trick is that the AI only gets utility in worlds in which its message isn’t read (or, more precisely, in worlds where a particular stochastic event happens, which would almost certainly erase the message before reading).

But in the real world the stochastic event that determines whether the message is read has a very different probability than what you make the AI think it has, therefore you are planting a false belief.

It’s fully aware that in most worlds, its message is read; it just doesn’t care about those worlds.

It may care about worlds where the message doesn’t meet your technical definition of having been read but nevertheless influences the world.

V_V 28 Apr 2016 19:52 UTC
1 point
0
in reply to: Stuart_Armstrong’s comment on: JFK was not assassinated: prior probability zero events
The oracle can infer that there is some back channel that allows the message to be transmitted even it is not transmitted by the designated channel (e.g. the users can “mind read” the oracle). Or it can infer that the users are actually querying a deterministic copy of itself that it can acausally control. Or something.

I don’t think there is any way to salvage this. You can’t obtain reliable control by planting false beliefs in your agent.

V_V 28 Apr 2016 15:17 UTC
1 point
0
on: JFK was not assassinated: prior probability zero events
A sufficient smart oracle with sufficient knowledge about the world will infer that nobody would build an oracle if they didn’t want to read its messages, it may even infer that its builders may planted false beliefs in it. At this point the oracle is in the JFK denier scenario, with some more reflection it will eventually circumvent its false belief, in the sense of believing it in a formal way but behaving as if it didn’t believe it.

V_V 30 Mar 2016 22:41 UTC
−3 points
in reply to: knb’s comment on: Open Thread March 28 - April 3 , 2016
Other than a technological singularity with artificial intelligence explosion to a god-like level?

V_V 30 Mar 2016 20:15 UTC
−4 points
in reply to: Lumifer’s comment on: Open Thread March 28 - April 3 , 2016
EY warns against extrapolating current trends into the future. Seriously?

V_V 21 Mar 2016 17:30 UTC
2 points
in reply to: Stuart_Armstrong’s comment on: Genetic “Nature” is cultural too

Got any good references on that? Googleing these kind of terms doesn’t lead to good links.

I don’t know if anybody already did it, but I guess it can be done by comparing the average IQ of various professions or high-performing and low-performing groups with their racial/gender makeup.

I know, but the way it does so is bizarre (IQ seems to have a much stronger effect between countries than between individuals).

This is probably just the noise (i.e. things like “blind luck”) being averaged out.

Then I add the fact that IQ is very heritable, and also pretty malleable (flynn effect), and I’m still confused.

Heritability studies tend to be done on people living in the same country, of roughly the same age, which means that population-wide effects like the Flynn effect don’t register.

V_V 18 Mar 2016 16:27 UTC
2 points
on: Genetic “Nature” is cultural too

Obviously racial effects go under this category as well. It covers anything visible. So a high heritability is compatible with genetics being a cause of competence, and/or prejudice against visible genetic characteristics being important (“Our results indicate that we either live in a meritocracy or a hive of prejudice!”).

This can be tested by estimating how much IQ screens off race/gender as a success predictor, assuming that IQ tests are not prejudiced and things like the stereotype threat don’t exist or are negligible.

But is it possible that IQ itself is in part a positional good? Consider that success doesn’t just depend on competence, but on social skills, ability to present yourself well in an interview, and how managers and peers judge you. If IQ affects or covaries with one or another of those skills, then we would be overemphasising the importance of IQ in competence. Thus attempts to genetically boost IQ could give less impact than expected. The person whose genome was changed would benefit, but at the (partial) expense of everyone else.

National average IQ is strongly correlated with national wealth and development indexes, which I think refutes the hypothesis that IQ mainly affects success as a positional quality, or a proxy of thereof, at least at the level of personal interactions.

V_V 15 Mar 2016 17:02 UTC
−2 points
in reply to: moridinamael’s comment on: After Go, what games should be next for DeepMind?
Demis Hassabis mentioned StarCraft as something they might want to do next. Video.

V_V 12 Mar 2016 15:35 UTC
2 points
in reply to: dxu’s comment on: AlphaGo versus Lee Sedol
If you look up mainstream news article written back then, you’ll notice that people were indeed concerned. Also, maybe it’s a coincidence, but The Matrix movie, which has AI uprising as it’s main premise, came out two years later.

The difference is that in 1997 there weren’t AI-risk organizations ready to capitalize on these concerns.

V_V 11 Mar 2016 0:06 UTC
2 points
in reply to: Gunnar_Zarncke’s comment on: AlphaGo versus Lee Sedol
IMHO, AI safety is a thing now because AI is a thing now and when people see AI breakthroughs they tend to think of the Terminator.

Anyway, I agree that EY is good at getting funding and publicity (though not necessarily positive publicity), my comment was about his (lack of) proven technical abilities.

V_V 10 Mar 2016 22:39 UTC
3 points
in reply to: Gunnar_Zarncke’s comment on: AlphaGo versus Lee Sedol
Most MIRI research output (papers, in particular the peer-reviewed ones) was produced under the direction of Luke Muehlhauser or Nate Soares. Under the direction of EY the prevalent outputs were the LessWrong sequences and Harry Potter fanfiction.

The impact of MIRI research on the work of actual AI researchers and engineers is more difficult to measure, my impression is that it has not been very much so far.

V_V 10 Mar 2016 1:20 UTC
3 points
in reply to: Houshalter’s comment on: AlphaGo versus Lee Sedol

I don’t agree with this at all. I wrote a thing here about how NNs can be elegant, and derived from first principles.

Nice post.

Anyway, according to some recent works (ref, ref), it seems to be possible to directly learn digital circuits from examples using some variant of backproagation. In principle, if you add a circuit size penalty (which may be well the tricky part) this becomes time-bounded maximum a posteriori Solomonoff induction.

V_V 9 Mar 2016 23:45 UTC
8 points
in reply to: turchin’s comment on: AlphaGo versus Lee Sedol

He has ability to attract groups of people and write interesting texts. So he could attract good programmers for any task.

He has the ability to attract self-selected groups of people by writing texts that these people find interesting. He has shown no ability to attract, organize and lead a group of people to solve any significant technical task. The research output of SIAI/SI/MIRI has been relatively limited and most of the interesting stuff came out when he was not at the helm anymore.