Signer

Karma: 547

Signer Oct 1, 2024, 12:41 PM
1 point
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform

Our observations are compatible with a world that is generated by a Turing machine with just a couple thousand bits.

Yes, but this is kinda incompatible with QM without mangled worlds.

Signer Sep 26, 2024, 5:59 PM
3 points
0
in reply to: Steven Byrnes’s comment on: [Intuitive self-models] 2. Conscious Awareness

Imagining two apples is a different thought from imagining one apple, right?

I mean, is it? Different states of the whole cortex are different. And the cortex can’t be in a state of imagining only one apple and, simultaneously, be in a state of imagining two apples, obviously. But it’s tautological. What are we gaining from thinking about it in such terms? You can say the same thing about the whole brain itself, that it can only have one brain-state in a moment.

I guess there is a sense in which other parts of the brain have more various thoughts relative to what cortex can handle, but, like you said, you can use half of cortex capacity, so why not define song and legal document as different thoughts?

As abstract elements of provisional framework cortex-level thoughts are fine, I just wonder what are you claiming about real constrains, aside from “there limits on thoughts”. because, for example, you need other limits anyway—you can’t think arbitrary complex thought even if it is intuitively cohesive. But yeah, enough gory details.

On the other hand, I can’t have two songs playing in my head simultaneously, nor can I be thinking about two unrelated legal documents simultaneously.

I can’t either, but I don’t see just from the architecture why it would be impossible in principle.

Again, I think autoassociative memory / attractor dynamics is a helpful analogy here. If I have a physical instantiation of a Hopfield network, I can’t query 100 of its stored patterns in parallel, right? I have to do it serially.

Yes, but you can theoretically encode many things in each pattern? Although if your parallel processes need different data, one of them will have to skip some responses… Would be better to have different networks, but I don’t see brain providing much isolation. Well, it seems to illustrate complications of parallel processing that may played a role in humans usually staying serial.

Signer Sep 26, 2024, 9:33 AM
4 points
0
on: [Intuitive self-models] 2. Conscious Awareness
I still don’t get this “only one thing in awareness” thing. There are multiple neurons in cortex and I can imagine two apples—in what sense there can only be one thing in awareness?

Or equivalently, it corresponds equally well to two different questions about the territory, with two different answers, and there’s just no fact of the matter about which is the real answer.

Obviously the real answer is the model which is more veridical^^. The latter hindsight model is right not about the state of the world at t=0.1, but about what you thought about the world at t=0.1 later.

Signer Sep 23, 2024, 2:58 PM
0 points
−2
on: The Sun is big, but superintelligences will not spare Earth a little sunlight

If that’s your hope—then you should already be alarmed at trends

Would be nice for someone to quantify the trends. Otherwise it may as well be that trends point to easygoing enough and aligned enough future systems.

For some humans, the answer will be yes—they really would do zero things!

Nah, it’s impossible for evolution to just randomly stumble upon such complicated and unnatural mind-design. Next you are going to say what, that some people are fine with being controlled?

Where an entity has never had the option to do a thing, we may not validly infer its lack of preference.

Aha, so if we do give the option to an entity and it doesn’t always kills all humans, then we have evidence it cares, right?

If there is a technical refutation it should simplify back into a nontechnical refutation.

Wait, why prohibiting successors would stop OpenAI from declaring easygoing system a failure? Ah, right—because there is no technical analysis, just elements of one.

Signer Sep 21, 2024, 5:48 PM
1 point
0
in reply to: nostalgebraist’s comment on: GPT-4o1
I genuinely think it’s a “more dakha” situation—the difficulty of communication is often underestimated, but it is possible to reach a mutual understanding.

Signer Sep 20, 2024, 6:34 AM
1 point
0
on: RLHF is the worst possible thing done when facing the alignment problem

RLHF does not solve the alignment problem because humans can’t provide good-enough feedback fast-enough.

Yeah, but the point is that the system learns values before an unrestricted AI vs AI conflict.

As mentioned in the beginning, I think the intuition goes that neural networks have a personality trait which we call “alignment”, caused by the correspondence between their values and our values. But “their values” only really makes sense after an unrestricted AI vs AI conflict, since without such conflicts, AIs are just gonna propagate energy to whichever constraints we point them at, so this whole worldview is wrong.

I mean, if your definition of values doesn’t make sense for real systems, then it’s the problem of your definition. As a hypothesis describing reality “alignment trait makes AI not splash harm on humans” is coherent enough. So the question is how do you know it is unlikely to happen?

This has not lead to the destruction of humanity yet because the biggest adversaries have kept their conflicts limited (because too much conflict is too costly) so no entity has pursued an end by any means necessary. But this only works because there’s a sufficiently small number of sufficiently big adversaries (USA, Russia, China, …), and because there’s sufficiently much opportunity cost.

First, “alignment is easy” is compatible with “we need to keep the set of big adversaries small”. But more generally, without numbers it seems like generalized anti-future-technology argument—what’s stopping human-regulation mechanisms from solving this adversarial problem, that didn’t stop them from solving previous adversarial problems?

It makes conflict more viable for small adversaries against large adversaries

Not necessary? It’s not unconceivable for future defense being more effective than offence (trivially true if “defense” is not giving AI to attackers). It kind of required for any future where humans have more power, than in present day?

Signer Sep 1, 2024, 3:41 AM
−1 points
−2
in reply to: habryka’s comment on: Jeremy Gillen’s Shortform

But also, if you predict a completion model where a very weak hash is followed by its pre-image, it will probably have learned to undo the hash, even though the source generation process never performed that (potentially much more complicated than the hashing function itself) operation, which means it’s not really a simulator.

I’m saying that this won’t work with current systems at least for strong hash, because it’s hard, and instead of learning to undo, the model will learn to simulate, because it’s easier. And then you can vary the strength of hash to measure the degree of predictorness/simulatorness and compare it with what you expect. Or do a similar thing with something other than hash, that also distinguishes the two frames.

The point is that without experiments like these, how have you come to believe in the predictor frame?

I don’t understand, how is “not predicting errors” either a thing we have observed, or something that has anything to do with simulation?

I guess it is less about simulation being the right frame and more about prediction being the wrong one. But I think we have definitely observed LLMs mispredicting things we wouldn’t want them to predict. Or is this actually a crux and you haven’t seen any evidence at all against the predictor frame?

Signer Sep 1, 2024, 2:14 AM
−1 points
−2
in reply to: habryka’s comment on: Jeremy Gillen’s Shortform

And I don’t think we’ve observed any evidence of that.

What about any time a system generalizes favourably, instead of predicting errors? You can say it’s just a failure of prediction, but it’s not like these failures are random.

That is the central safety property we currently rely on and pushes things to be a bit more simulator-like.

And the evidence for this property, instead of, for example, the inherent bias of NNs, being central is what? Why wouldn’t predictor exhibit more malign goal-directedness even for short term goals?

I can see that this whole story about modeling LLMs as predictors, and goal-directedness, and fundamental laws of cognition is logically coherent. But where is the connection to reality?

Signer Aug 31, 2024, 11:14 AM
1 point
0
in reply to: habryka’s comment on: Jeremy Gillen’s Shortform
Why wouldn’t myopic bias make it more likely to simulate than predict? And does’t empirical evidence about LLMs support the simulators frame? Like, what observations persuaded you, that we are not living in the world, where LLMs are simulators?

Signer Aug 28, 2024, 6:01 AM
1 point
0
on: Am I confused about the “malign universal prior” argument?

In order to be “UP-like” in a relevant way, this procedure will have to involve running TMs, and the set of TMs that might be run needs to include the same TM that implements our beings and their world.

Why? The procedure just need to do some reasoning, constrained by UP and outer TM. And then UP-beings can just simulate this fast reasoning without problems of self-simulation.

Yes, AI that practically uses UP may fail to predict whether UP-beings simulate it in the center of their universe or on the boundary. But the point is that the more correct AI is in its reasoning, the more control UP-beings have.

Or you can not create AI that thinks about UP. But that’s denying the assumption.

Signer Aug 21, 2024, 7:35 PM
1 point
0
in reply to: Milan W’s comment on: Milan W’s Shortform

Yet, you can find valence in your own experiences

But why must you care about valence? It’s not an epistemic error to not care. You don’t have direct experience of there being a law that you must care about valence.

Signer Aug 10, 2024, 7:09 AM
3 points
0
on: “Which Future Mind is Me?” Is a Question of Values

Everyone but Elon himself would say the above is a different scenario from reality. Each of us knows which body our first-person perspective resides in. And that is clearly not the physical human being referred as Elon Musk. But the actual and imaginary scenarios are not differentiated by any physical difference of the world, as the universe is objectively identical.

They are either differentiated by a physically different location of some part of your experience—like your memory being connected to Elon’s sensations, or your thought being executed in other location—or it would be wrong to say that this scenario is different from reality: what you imagine would just correctly correspond to Elon’s experiences also being real.

Signer Jul 20, 2024, 7:33 AM
−1 points
−5
on: What are the actual arguments in favor of computationalism as a theory of identity?
Computationalism is an ethical theory, so it is fine for it to be based on high-level abstractions—ethics is arbitrary.

Signer Jul 5, 2024, 2:39 PM
3 points
0
in reply to: JBlack’s comment on: The Potential Impossibility of Subjective Death

For (1) the multiverse needs to be immensely larger than our universe, by a factor of at least 10¹⁰6 or so “instances”. The exact double exponent depends upon how closely people have to match before it’s reasonable to consider them to be essentially the same person. Perhaps on the order of millions of data points is enough, maybe more are needed. Evidence for MWI is nowhere near strong enough to justify this level of granularity in the state space and it doesn’t generalize well to space-time quantization so this probably isn’t enough.

Why? Even without unphysically ordering arbitrary point-states, isn’t the whole splitting behavior creates at least all subjectively-distinguishable instances?

Signer Jul 5, 2024, 2:21 PM
3 points
1
in reply to: Richard_Kennaway’s comment on: The Potential Impossibility of Subjective Death
There is non-zero measure on a branch that starts with you terminally ill and gradually proceeds to you miraculously recovering. So if you consider normally recovered you to be you, nothing stops you from considering this low-measure you to also be you.

I have never heard of anyone going to sleep as one of a pair of twins and waking up as the other.

According to MWI everyone wakes up as multiple selves all the time.

Signer 5 Jul 2024 14:00 UTC
0 points
0
on: Doomsday Argument and the False Dilemma of Anthropic Reasoning
Still don’t get how souls would get you psychic powers. Otherwise randomness and causality don’t matter—you may as well simultaneously create people in numbered rooms and people in low-numbered rooms would have the same problems.

Signer 3 Jul 2024 15:15 UTC
−2 points
−3
on: Isomorphisms don’t preserve subjective experience… right?

conscious in the way that we are conscious

Whether it’s the same way is an ethical question, so you can decide however you want.

So there should be some sort of hardware-dependence to obtain subjective experience.

I certainly don’t believe in subjective experience without any hardware, but no, there is no much dependence except for your preferences for hardware.

As for generally accepted conclusions… I think it’s generally accepted that some preferences for hardware are useful in epistemic contexts, so you can be persuaded to say “rock is not conscious” for the same reason you say “rock is not calculator”.

Signer 21 Jun 2024 10:10 UTC
−1 points
−2
on: Enriched tab is now the default LW Frontpage experience for logged-in users
Not sure this qualifies, but I try to avoid instantiating complicated models for ethical reasons.

Signer 17 Jun 2024 13:45 UTC
1 point
−2
in reply to: Daniel Kokotajlo’s comment on: Matthew Barnett’s Shortform
What does “dumb” mean? Corrigibility basically is being selectively dumb. You can give power to a LLM and it would likely still follow instructions.

Signer 1 Jun 2024 8:04 UTC
8 points
0
in reply to: Matthew Barnett’s comment on: MIRI 2024 Communications Strategy

Given a low prior probability of doom as apparent from the empirical track record of technological progress, I think we should generally be skeptical of purely theoretical arguments for doom, especially if they are vague and make no novel, verifiable predictions prior to doom.

And why such use of the empirical track record is valid? Like, what’s the actual hypothesis here? What law of nature says “if technological progress hasn’t caused doom yet, it won’t cause it tomorrow”?

MIRI’s arguments for doom are often difficult to pin down, given the informal nature of their arguments, and in part due to their heavy reliance on analogies, metaphors, and vague supporting claims instead of concrete empirically verifiable models.

And arguments against are based on concrete empirically verifiable models of metaphors.

If your model of reality has the power to make these sweeping claims with high confidence, then you should almost certainly be able to use your model of reality to make novel predictions about the state of the world prior to AI doom that would help others determine if your model is correct.

Doesn’t MIRI’s model predict some degree of the whole Shoggoth/actress thing in current system? Seems verifiable.