Kyre

Karma: 535

Kyre Dec 5, 2015, 11:13 PM
0 points
in reply to: Algernoq’s comment on: Crazy Ideas Thread, December 2015
That’s the unbounded computation case.

Kyre Dec 5, 2015, 6:33 AM
2 points
in reply to: Tem42’s comment on: Crazy Ideas Thread, December 2015

It seems like there is a lot of room between “one simulation” and “unbounded computational resources”

Well the point is that if we are running on bounded resources, then the time until it runs out depends very sensitively on how many simulations we (and simulations like us) launch on average. Say that our simulation has a million years allocated to it, and we launch simulations starting a year back from the time when we launch a simulation.

If we don’t launch any, we get a million years.

If we launch one, but that one doesn’t launch any, we get half a million.

If we launch one, and that one launches one etc, then we get on the order of a thousand years.

If we launch two, and that one launches two etc, then we get on the order of 20 years.

Also, it is a bit odd to think that when computational resources start running low the correct thing to do is wipe everything clean.

True, ‘terminates’ is probably the wrong word. There’s no reason why the simulation would be wiped. It just couldn’t continue.

It also assumes a full-word simulation, and not just a preferred-actors simulation, which is a possibility, and maybe a probability, but not a given

I’m not sure. I think the trilemma applies to a simulation of a single actor, if that actor decides to launch simulations of their own life.

Kyre Dec 4, 2015, 4:55 AM
2 points
on: Crazy Ideas Thread, December 2015
Here is a second Simulation Trilemma.

If we are living in a simulation, at least one of the following is true:

1) we are running on a computer with unbounded computational resources, or

2) we will not launch more than one simulation similar to our world, or

3) the simulation we are in will terminate shortly after we launch our own simulations.

Here ‘short’ is on the order of the period between the era we start the simulation at and when the simulation reaches our stage.

Kyre Nov 10, 2015, 4:44 AM
11 points
in reply to: [deleted]’s comment on: Open thread, Nov. 09 - Nov. 15, 2015

I heard strawberry jam can be made with just strawberries, water and sugar on a frying pan on the radio.

I’d use a stove.

Kyre Nov 3, 2015, 5:12 AM
9 points
on: Newcomb, Bostrom, Calvin: Credence and the strange path to a finite afterlife

A short hook headline like “avoiding existential risk is key to afterlife” can get a conversation going. I can imagine Salon, etc. taking another swipe at it, and in doing so, creating publicity which would help in finding more similar minded folks to get involved in the work of MIRI, FHI, CEA etc. There are also some really interesting ideas about acausal trade …

Assuming you get good feedback and think that you have an interesting, solid arguments … please think carefully about whether such publicity helps the existential risk movement more than it harms. On the plus side, you might get people thinking about existential risk that otherwise would not have. On the minus side, most people aren’t going to understand what you write, and some of the the ones that half-understand it are going to loudly proclaim it as more evidence that MIRI etc are full of insane apocalyptic cultists.

Kyre Sep 17, 2015, 5:42 AM
11 points
in reply to: CarlShulman’s comment on: A toy model of the control problem
It’s a trade-off. The example is simple enough that the alignment problem is really easy to see, but it also means that it is easy to shrug it off and say “duh, just the use obvious correct utility function for B”.

Perhaps you could follow it up with an example with more complex mechanics (and or more complex goal for A) where the bad strategy for B is not so obvious. You then invite the reader to contemplate the difficulty of the alignment problem as the complexity approaches that of the real world.

Kyre Jul 9, 2015, 6:23 AM
5 points
in reply to: Viliam’s comment on: Crazy Ideas Thread
Nitpick: we have equations for (special) relativistic quantum physics. Dirac was one of the pioneers, and the Standard Model for instance is a relativistic quantum field theory. I presume you mean general relativity (gravity) and quantum mechanics that is the problem.

(Douglas_Knight) Moreover, the predictions that QFT makes about chemistry are too hard. I don’t think it is possible with current computers to compute the spectrum of helium, let alone lithium. A quantum computer could do this, though.

In the spirit of what Viliam suggested, maybe you could do computational searches for tractable approximations to QFT for chemistry i.e. automatically find things like density functional theory. A problem there might be that you do not get any insight from the result, and you might end up overfitting.

Kyre Jun 2, 2015, 4:54 AM
1 point
in reply to: ahbwramc’s comment on: Stupid Questions June 2015
Things that are unsexy but I can actually verify as having been useful more than once:

In wallet, folded up tissue. For sudden attack of sniffles (especially on public transport), small cuts, emergency toilet paper.

In bag I carry every day: small pack of tissues, multitool, tiny torch, ibuprofin, pad and pencil, USB charging cable for phone, plastic spork, wet wipe thing from KFC (why do they always shovel multiples of those things in with my order ?).

Kyre May 21, 2015, 5:55 AM
4 points
in reply to: Lumifer’s comment on: Visions and Mirages: The Sunk Cost Dilemma
Very rough toy example.

Say I’ve started a project which I can definitely see 5 days worth of work. I estimate there’ll be some unexpected work in there somewhere, maybe another day, so I estimate 6 days.

I complete day one but have found another day’s work. When should I estimate completion now ? Taking the outside view, finishing in 6 days (on day 7) is too optimistic.

Implicit in my original estimate was a “rate of finding new work” of about 0.2 days per day. But, now I have more data on that, so I should update the 0.2 figure. Let’s see, 0.2 is my prior, I should build a model for “rate of finding new work” and figure out what the correct Bayesian update is … screw it, let’s assume I won’t find any more work today and estimate the rate by Laplace’s rule of succession. My updated rate of finding new work is 0.5. Hmmm that’s pretty high, the new work I find is itself going to generate new work, better sum the geometric series … 5 known days work plus 5 more unknown, so I should finish in 10 days (ie day 11).

I complete day 2 and find another day’s work ! Crank the handle around, should finish in 15 days (ie day 17).

… etc …

If this state of affairs continues, my expected total amount of work grows really fast, and it won’t be very long before it becomes clear that it is not profitable.

Contrast this with: I can see 5 days of work, but experience tells me that the total work is about 15 days. The first couple of days I turn up additional work, but I don’t start to get worried until around day 3.

Kyre May 2, 2015, 2:53 PM
0 points
in reply to: ahbwramc’s comment on: Shawn Mikula on Brain Preservation Protocols and Extensions
Thank you, you saved me a lot of typing. No amount of straight copying of that GIF will generate a conscious experience; but if you print out the first frame and give it to a person with a set of rules for simulating neural behaviour and tell them to calculate the subsequent frames into a gigantic paper notebook, that might generate consciousness.

Kyre May 1, 2015, 9:21 AM
2 points
in reply to: brainmaps’s comment on: Shawn Mikula on Brain Preservation Protocols and Extensions
Thanks for replying ! Sorry if the bit I quoted was too short and over-simplified.

That does clarify things, although I’m having difficulty understanding what you mean by the phrase “causal structure”. I take it you do not mean the physical shape or substance, because you say that a different computer architecture could potentially have the right causal structure.

And I take it you don’t mean the cause and effect relationship between parts of the computer that are representing parts of the brain, because I think that can be put into one-to-one correspondence with the cause and effect relationship of the things being represented.

For example, If neuron N1 causes changes to neurons N2, N3 and N4; and I have a simulated S1 causing changes to simulated S2, S3 and S4, then that simulated cause and effect happens by honest-to-god physical cause and effect: voltage levels in the memory gates representing S1 propagate through the architecture to the gates representing S2, S3, S4 causing them to change.

Using a different computer architecture may avert this problem …

So consciousness would have to then be something that flesh brains and “correctly causally structured” computer hardware have in common, but which is not shared by a simulation of either of those things running on a conventional computer ?

Kyre Apr 29, 2015, 5:32 AM
5 points
on: Shawn Mikula on Brain Preservation Protocols and Extensions
That is very interesting; there does seem to be quite rapid progress in this area.

From the blog entry:

… the reason for this is because simulating the neural activity on a Von Neumann (or related computer) architecture does not reproduce the causal structure of neural interactions in wetware. Using a different computer architecture may avert this problem …

Can anyone explain what that means ? I can’t see how it can be correct.

Kyre Apr 21, 2015, 4:35 AM
1 point
on: Self-verification
Tattoo private key on inside of thigh.

Kyre Apr 15, 2015, 5:09 AM
6 points
in reply to: Houshalter’s comment on: Concept Safety: Producing similar AI-human concept spaces

What’s to stop the AI from instead learning that “good” and “bad” are just subjective mental states or words from the programmer, rather than some deep natural category of the universe? So instead of doing things it thinks the human programmer would call “good”, it just tortures the programmer and forces them to say “good” repeatedly.

The pictures and videos of torture in the training set that are labelled “bad”.

It is not perfect, but I think the idea is that with a large and diverse training set the hope is that it alternative models of “good/bad” become extremely contrived, and the human one you are aiming for becomes the simplest model.

I found the material in the post very interesting. It holds out hope that after training your world model, it might not be as opaque as people fear.

Kyre Mar 25, 2015, 5:52 AM
0 points
on: Request for Steelman: Non-correspondence concepts of truth
I’m not sure succeeding at number 4 helps you with with the unattractiveness and discomfort of number 3.

Say you do find some alternative steel-manned position on truth that is comfortable and intellectually satisfying. What are the odds that this position will be the same position as that held by “most humans”, or that understanding it will help you get along with them ?

Regardless of the concept of truth you arrive at, you’re still faced with the challenge of having to interact with people who have not-well-thought-out concepts of truth in a way that is polite, ethical, and (ideally) subtly helpful.

Kyre Jan 26, 2015, 8:32 AM
2 points
in reply to: [deleted]’s comment on: What are the resolution limits of medical imaging?
I thought CLARITY was an interesting development—a brain preservation technique that renders tissue transparent. I imagine in the near future there’s likely to be benefits going both was from preservation and imaging research.

Kyre Jan 9, 2015, 5:07 AM
7 points
in reply to: Vika’s comment on: The Importance of Sidekicks
Buffy / Xander, Motoko / Batu, Deunan / Briareos

(although I’m not sure “Sidekick” is exactly right here)

Kyre Dec 9, 2014, 5:13 AM
15 points
in reply to: Eniac’s comment on: Stupid Questions December 2014
I thought this sounded familiar

Kyre Nov 23, 2014, 11:13 PM
1 point
in reply to: Document’s comment on: xkcd on the AI box experiment
Ah, my mistake, thanks again.

Kyre Nov 22, 2014, 4:08 AM
1 point
in reply to: XiXiDu’s comment on: xkcd on the AI box experiment
Downvoted for bad selective quoting in that last quote. I read it and thought, wow, Yudkowsky actually wrote that. Then I thought, hmmm, I wonder if the text right after that says something like “BUT, this would be wrong because …” ? Then I read user:Document’s comment. Thank you for looking that up.