awenonian

Karma: 90

awenonian 6 Jul 2023 22:08 UTC
1 point
0
on: When do “brains beat brawn” in Chess? An experiment
I have a habit of reading footnotes as soon as they are linked, and your footnote says that you won with queen odds before the call to guess what odds you’d win at, creating a minor spoiler.

awenonian 7 Feb 2023 3:31 UTC
1 point
0
on: Introduction to abstract entropy
Is it important that negentropy be the result of subtracting from the maximum entropy? It seemed a sensible choice, up until it introduces infinities, and made every state’s negentropy infinite. (And also that, if you subtract from 0, then two identical states should have the same negentropy, even in different systems. Unsure if that’s useful, or harmful).
Though perhaps that’s important for the noting that reducing an infinite system to a finite macrostate is an infinite reduction? I’m not sure if I understand how (or perhaps when?) that’s more useful than having it be defined as subtracted from 0, such that finite macrostates have finite negentropy, and infinite macrostates have -infinite negentropy (showing that you really haven’t reduced it at all, which, as far as I understand with infinities, you haven’t, by definition).

awenonian 18 Jan 2023 4:12 UTC
3 points
0
on: Models Don’t “Get Reward”
Back in Reward is not the optimization target, I wrote a comment, which received a (small I guess) amount of disagreement.
I intended the important part of that comment to be the link to Adaptation-Executers, not Fitness-Maximizers. (And more precisely the concept named in that title, and less about things like superstimuli that are mentioned in the article) But the disagreement is making me wonder if I’ve misunderstood both of these posts more than I thought. Is there not actually much relation between those concepts?
There was, obviously, other content to the comment, and that could be the source of disagreement. But I only have that there was disagreement to go on, and I think it would be bad for my understanding of the issue to assume that’s where the disagreement was, if it wasn’t.

awenonian 16 Jan 2023 1:14 UTC
2 points
0
on: We don’t trade with ants
When I tried to answer why we don’t trade with ants myself, communication was one of the first things (I can’t remember what was actually first) I considered. But I worry it may be more analogous to AI than argued here.
We sort of can communicate with ants. We know to some degree what makes them tick, it’s just we mostly use that communication to lie to them and tell them this poison is actually really tasty. The issue may be less that communication is impossible, and more that it’s too costly to figure out, and so no one tries to become Antman even if they could cut their janatorial costs by a factor of 7.
The next thought I had was that, if I were to try to get ants to clean my room, I think it’s likely that the easiest line towards that is not figuring out how to communicate, but breeding some ants with different behavior (e.g. search for small bits of food, instead of large bits. This seems harder than that sentence suggests, but still probably easier than learning to speak ant). I don’t like what that would be analogous to in human-AI interactions.
I think it’s possible that an AI could fruitfully trade with humans. While it lacks a body, posting an ad on Craigslist to get someone to move something heavy is probably easier than figuring out how to hijack a wifi-enabled crane or something.
But I don’t know how quickly that changes. If the AI is trying to build a sci-fi gadget, it’s possible that an instruction set to build it is long or complicated enough that a human has trouble following it accurately. The costs of writing intuitive instructions, and also designing the gadget such that idiot-proof construction is possible could be high enough that it’s better to do it itself.

Things roll downhill

awenonian6 Dec 2022 15:27 UTC

19 points

0 comments1 min readLW link

awenonian 26 Jul 2022 22:07 UTC
3 points
−5
in reply to: paulfchristiano’s comment on: Reward is not the optimization target
I interpret OP (though this is colored by the fact that I was thinking this before I read this) as saying Adaptation-Executers, not Fitness-Maximizers, but about ML. At which point you can open the reference category to all organisms.
Gradient descent isn’t really different from what evolution does. It’s just a bit faster, and takes a slightly more direct line. Importantly, it’s not more capable of avoiding local maxima (per se, at least).

awenonian 4 Jul 2022 19:31 UTC
3 points
in reply to: anonymousaisafety’s comment on: I No Longer Believe Intelligence to be “Magical”
So, I want to note a few things. The original Eliezer post was intended to argue against this line of reasoning:
I occasionally run into people who say something like, “There’s a theoretical limit on how much you can deduce about the outside world, given a finite amount of sensory data.”
He didn’t worry about compute, because that’s not a barrier on the theoretical limit. And in his story, the entire human civilization had decades to work on this problem.
But you’re right, in a practical world, compute is important.
I feel like you’re trying to make this take as much compute as possible.
Since you talked about headers, I feel like I need to reiterate that, when we are talking to a neural network, we do not add the extra data. The goal is to communicate with the neural network, so we intentionally put it in easier to understand formats.
In the practical cases for this to come up (e.g. a nascent superintelligence figuring out physics faster than we expect), we probably will also be inputting data in an easy to understand format.
Similarly, I expect you don’t need to check every possible esoteric format. The likelihood of the image using an encoding like 61 bits per pixel, with 2 for red, 54 for green and 5 for blue is just, very low, a priori. I do admit I’m not sure if only using “reasonable” formats would cut down the possibilities into the computable realm (obviously depends on definitions of reasonable, though part of me feels like you could (with a lot of work) actually have an objective likeliness score of various encodings). But certainly it’s a lot harder to say that it isn’t than just saying “f(x) = (63 pick x), grows very fast.”
Though, since I don’t have a good sense for whether “reasonable” ones would be a more computable number, I should update in your direction. (I tried to look into something sort of analogous, and the most common 200 passwords cover a little over 4% of all used passwords, which, isn’t large enough for me to feel comfortable expecting that the most “likely” 1,000 formats would cover a significant quantity of the probability space, or anything.)
(Also potentially important. Modern neural nets don’t really receive things as a string of bits, but instead as a string of numbers, nicely split up into separate nodes. (yes, those numbers are made of bits, but they’re floating point numbers, and the way neural nets interact with them is through all the floating point operations, so I don’t think the neural net actually touches the bit representation of the number in any meaningful way.)

awenonian 30 Jun 2022 16:57 UTC
3 points
in reply to: anonymousaisafety’s comment on: I No Longer Believe Intelligence to be “Magical”
“you’re jumping to the conclusion that you can reliably differentiate between...”
I think you absolutely can, and the idea was already described earlier.
You pay attention to regularities in the data. In most non-random images, pixels near to each other are similar. In an MxN image, the pixel below is a[i+M], whereas in an NxM image, it’s a[i+N]. If, across the whole image, the difference between a[i+M] is less than the difference between a[i+N], it’s more likely an MxN image. I expect you could find the resolution by searching all possible resolutions from 1x<length> to <length>x1, and finding which minimizes average distance of “adjacent” pixels.
Similarly (though you’d likely do this first), you can tell the difference between RGB and RGBA. If you have (255, 0, 0, 255, 0, 0, 255, 0, 0, 255, 0, 0), this is probably 4 red pixels in RGB, and not a fully opaque red pixel, followed by a fully transparent green pixel, followed by a fully transparent blue pixel in RGBA. It could be 2 pixels that are mostly red and slightly green in 16 bit RGB, though. Not sure how you could piece that out.
Aliens would probably do a different encoding. We don’t know what the rods and cones in their eye-equivalents are, and maybe they respond to different colors. Maybe it’s not Red Green Blue, but instead Purple Chartreuse Infrared. I’m not sure this matters. It just means your eggplants look red.
I think, even if it had 5 (or 6, or 20) channels, this regularity would be born out, between bit i and bit i+5 being less than bit i and i+1, 2, 3, or 4.
Now, there’s still a lot that that doesn’t get you yet. But given that there are ways to figure out those, I kinda think I should have decent expectations there’s ways to figure out other things, too, even if I don’t know them.
I do also think it’s important to zoom out to the original point. Eliezer posed this as an idea about AGI. We currently sometimes feed images to our AIs, and when we do, we feed them as raw RGB data, not encoded, because we know that would make it harder for the AI to figure out. I think it would be very weird, if we were trying to train an AI, to send it compressed video, and much more likely that we do, in fact, send it raw RGB values frame by frame.
I will also say that the original claim (by Eliezer, not the top of this thread), was not physics from one frame, but physics from like, 3 frames, so you get motion, and acceleration. 4 frames gets you to third derivatives, which, in our world, don’t matter that much. Having multiple frames also aids in ideas like the 3d → 2d projection, since motion and occlusion are hints at that.
And I think the whole question is “does this image look reasonable”, which you’re right, is not a rigorous mathematical concept. But “‘looks reasonable’ is not a rigorous concept” doesn’t get followed by “and therefore is impossible” Above are some of the mathematical descriptions of what ‘reasonable’ means in certain contexts. Rendering a 100x75 image as 75x100 will not “look reasonable”. But it’s not beyond thinking and math to determine what you mean by that.

awenonian 26 Jun 2022 14:56 UTC
1 point
on: Worked Examples of Shapley Values
“the addition of an unemployable worker causes … the worker’s Shapley values to drop to $208.33 (from $250).”
I would emphasize here that the “workers’” includes the unemployed one. It was not obvious to me, until about halfway through the next paragraph, and I think the next paragraph would read better with that in mind from the start.

awenonian 19 Jun 2022 3:58 UTC
2 points
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
I’d be interested to know why you think that.
I’d be further interested if you would endorse the statement that your proposed plan would fully bridge that gap.
And if you wouldn’t, I’d ask if that helps illustrate the issue.

awenonian 17 Jun 2022 20:47 UTC
1 point
on: Contra EY: Can AGI destroy us without trial & error?
It seems odd to suggest that the AI wouldn’t kill us because it needs our supply chain. If I had the choice between “Be shut down because I’m misaligned” (or “Be reprogrammed to be aligned” if not corrigible) and “Have to reconstruct the economy from the remnants of human civilization,” I think I’m more likely to achieve my goals by trying to reconstruct the economy.
So if your argument was meant to say “We’ll have time to do alignment while the AI is still reliant on the human supply chain,” then I don’t think it works. A functional AGI would rather destroy the supply chain and probably fail at its goals, than be realigned and definitely fail at its goals.
(Also, I feel this is mostly a minor thing, but I don’t really understand your reference class on novel technologies. Why is the time measured from “proof of concept submarine” to “capable of sinking a warship”? Or from “theory of relativity” to “Atom Bomb being dropped”? Maybe that was just the data available, but why isn’t it “Wright brothers start attempting heavier than air flight” to “Wright brothers do heavier than air flight”? Because when reading my mind immediately wondered how much of the 36 year gap on mRNA vaccines was from “here’s a cool idea” to “here’s a use case”, instead of “here’s a cool idea” to “we can actually do that”)

awenonian 17 Jun 2022 3:31 UTC
1 point
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Surely creating the full concrete details of the strategy is not much different from “putting forth as-good-as-human definitions, finding objections for them, and then improving the definition based on considered objections.” I at least don’t see why the same mechanism couldn’t be used here (i.e. apply this definition iteration to the word “good”, and then have the AI do that, and apply it to “bad” and have the AI avoid that). If you see it as a different thing, can you explain why?

awenonian 15 Jun 2022 22:13 UTC
0 points
0
in reply to: lc’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Exactly. I notice you aren’t who I replied to, so the canned response I had won’t work. But perhaps you can see why most of his objections to my objections would apply to objections to that plan?

awenonian 15 Jun 2022 20:01 UTC
3 points
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Let me ask you this. Why is “Have the AI do good things, and not do bad things” a bad plan?

awenonian 15 Jun 2022 13:48 UTC
3 points
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
I think you missed the point. I’d trust an aligned superintelligence to solve the objections. I would not trust a misaligned one. If we already have an aligned superintelligence, your plan is unnecessary. If we do not, your plan is unworkable. Thus, the problem.
If you still don’t see that, I don’t think I can make you see it. I’m sorry.

awenonian 14 Jun 2022 14:50 UTC
2 points
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
It seems simple and effective because you don’t need to put weight on it. We’re talking a superintelligence, though. Your definition will not hold when the weight of the world is on it.
And the fact that you’re just reacting to my objections is the problem. My objections are not the ones that matter. The superintelligence’s objections are. And it is, by definition, smarter than me. If your definition is not something like provably robust, then you won’t know if it will hold to a superintelligent objection. And you won’t be able to react fast enough to fix it in that case.
You can’t bandaid a solution into working, because if a human can point out a flaw, you should expect a superintelligence to point out dozens, or hundreds, or thousands.
I don’t know how else to get you to understand this central objection. Robustness is required. Provable robustness is, while not directly required, kinda the only way we can tell if something is actually robust.

awenonian 13 Jun 2022 13:41 UTC
1 point
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
I’m not sure this is being productive. I feel like I’ve said the same thing over and over again. But I’ve got one more try: Fine, you don’t want to try to define “reason” in math. I get it, that’s hard. But just try defining it in English.
If I tell the machine “I want to be happy.” And it tries to determine my reason for that, what does it come up with? “I don’t feel fulfilled in life”? Maybe that fits, but is it the reason, or do we have to go back more: “I have a dead end job”? Or even more “I don’t have enough opportunities”?
Or does it go a completely different tack and say my reason is “My pleasure centers aren’t being stimulated enough” or “I don’t have enough endorphins.”
Or, does it say the reason I said that was because my fingers pressed keys on a keyboard.
To me, as a human, all of these fit the definition of “reasons.” And I expect they could all be true. But I expect some of them are not what you mean. And not even in the sense of some of them being a different definition for “reason.” How would you try to divide what you mean and what you don’t mean?
Then do that same thought process on all the other words.

awenonian 11 Jun 2022 19:17 UTC
5 points
5
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
No, they really don’t. I’m not trying to be insulting. I’m just not sure how to express the base idea.
The issue isn’t exactly that computers can’t understand this, specifically. It’s that no one understands what those words mean enough. Define reason. You’ll notice that your definition contains other words. Define all of those words. You’ll notice that those are made of words as well. Where does it bottom out? When have you actually, rigorously, objectively defined these things? Computers only understand that language, but the fact that a computer wouldn’t understand your plan is just illustrative of the fact that it is not well defined. It just seems like it is, because you have a human brain that fills in all the gaps seamlessly. So seamlessly you don’t even notice that there were gaps that need filling.
This is why there’s an emphasis on thinking about the problem like a computer programmer. Misalignment thrives in those gaps, and if you gloss over them, they stay dangerous. The only way to be sure you’re not glossing over them is to define things with something as rigorous as Math. English is not that rigorous.

awenonian 10 Jun 2022 15:35 UTC
1 point
0
in reply to: Sune’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
In short, the difference between the two is Generality. A system that understands the concepts of computational resources and algorithms might do exactly that to improve it’s text prediction. Taking the G out of AGI could work, until the tasks get complex enough they require it.

awenonian 10 Jun 2022 15:28 UTC
3 points
0
in reply to: LVSN’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Again, what is a “reason”? More concretely, what is the typeof a “reason”? You can’t program an AI in English, it needs to be programmed in code. And code doesn’t know what “reason” means.
It’s not exactly that your plan “fails” anywhere particularly. It’s that it’s not really a plan. CEV says “Do what humans would want if they were more the people they want to be.” Cool, but not a plan. The question is “How?” Your answer to that is still under specified. You can tell by the fact you said things like “the AI could just...” and didn’t follow it with “add two numbers” or something simple (we use the word “primitive”), or by the fact you said “etc.” in a place where it’s not fully obvious what the rest actually would be. If you want to make this work, you need to ask “How?” to every single part of it, until all the instructions are binary math. Or at least something a python library implements.