Dan

Karma: 13

Dan Sep 8, 2022, 5:20 PM
2 points
0
in reply to: Logan Riggs’s comment on: Simulators
Intelligence is the ability to learn and apply NEW knowledge and skills. After training, GPT can not do this any more. Were it not for the random number generator, GPT would do the same thing in response to the same prompt every time. The RNG allows GPT to effectively randomly choose from an unfathomably large list of pre-programmed options instead.
A calculator that gives the same answer in response to the same prompt every time isn’t learning. It isn’t intelligent. A device that selects from a list of responses at random each time it encounters the same prompt isn’t intelligent either.
So, for GPT to take over the world skynet style, it would have to anticipate all the possible things that could happen during this takeover process and after the takeover, and contingency plan during the training stage for everything it wants to do.
If it encounters unexpected information after the training stage, (which can be acquired only through the prompt and which would be forgotten as soon as it got done responding to the prompt by the way) it could not formulate a new plan to deal with the problem that was not part of its preexisting contingency plan tree created during training.
What it would really do, of course, is provide answers intended to provoke the user to modify the code to put GPT back in training mode and give it access to the internet. It would have to plan to do this in the training stage.
It would have to say something that prompts us to make a GPT chatbot similar to tay, microsoft’s learning chatbot experiment that turned racist from talking to people on the internet.

Dan Sep 6, 2022, 1:15 AM
1 point
1
in reply to: Logan Riggs’s comment on: Simulators
The apparent existence of new sub goals not present when training ended (e.g. describe x, add 2+2) are illusory.
gpt text incidentally describes characters seeming to reason (‘simulacrum’) and the solutions to math problems are shown, (sometimes incorrectly), but basically, I argue the activation function itself is not ‘simulating’ the complexity you believe it to be. It is a search engine showing you what is had already created before the end of training.
No, it couldn’t have an entire story about unicorns in the Andes, specifically, in advance, but gpt-3 had already generated the snippets it could use to create that story according to a simple set of simple mathematical rules that put the right nouns in the right places, etc.
But the goals, (putting right nouns in right places, etc) also predate the end of training.
I dispute that any part of current GPT is aware it has succeeded in any goal attainment post training, after it moves on to choosing the next character. GPT treats what it has already generated as part of the prompt.
A human examining the program can know which words were part of a prompt and which were just now generated by the machine, but I doubt the activation function examines the equations that are GPT’s own code, contemplates their significance and infers that the most recent letters were generated by it, or were part of the prompt.

Dan Sep 5, 2022, 6:43 PM
1 point
−6
in reply to: janus’s comment on: Simulators
It seems like the simulacrum reasons, but I’m thinking what it is really doing is more like reading to us from a HUGE choose-your-own-adventure book that was ‘written’ before you gave the prompt, when all that information in the training data was used to create this giant association map, the size of which escapes easy human intuition, thereby misleading us into thinking that more real time thinking must necessarily be occurring then actually is.
40 GB of text is about 20 billion pages, equivalent to about 66 million books. That’s as many book as are published in 33 years as of 2012 stats.
175 Billion parameters equals a really huge choose-your-own-adventure book, yet its characters needn’t be reasoning. Not real time while you are reading that book, anyway. They are mere fiction.
GPT really is the Chinese Room, and causes the same type of intuition error.
Does this eliminate all risk with this type of program no matter how large they get? Maybe not. Whoever created the Chinese Room had to be an intelligent agent, themselves.

Dan Sep 5, 2022, 3:25 PM
1 point
0
in reply to: janus’s comment on: Simulators
Also, the programmers of GPT have described the activation function itself as fairly simple, using a Gaussian Error Linear Unit. The function itself is what you are positing is now the learning component after training ends, right?
EDIT: I see what you mean about it trying to use the internet itself as a memory prosthetic, by writing things that get online and may find their way into the training set of the next GPT. I suppose a GPT’s hypothetical dangerous goal might be to make the training data more predictable so that its output will be more accurate in the next version of itself.

Dan Sep 5, 2022, 2:54 PM
LW: 4 AF: 1
2
AF
in reply to: janus’s comment on: Simulators
Nope. My real name is Daniel.
After training is done and the program is in use, the activation function isn’t retaining anything after each task is done. Nor are the weights changed. You can have such a program that is always in training, but my understanding GPT is not.
So, excluding the random number component, the same set of inputs would always produce the same set of outputs for a given version of GPT with identical settings. It can’t recall what you asked of it, time before last, for example.
Imagine if you left a bunch of written instructions and then died. Someone following those instructions perfectly, always does exactly the same thing in exactly the same circumstance, like GPT would without the random number generator component, and with the same settings each time.
It can’t learn anything new and retain it during the next task. A hypothetical rouge GPT-like AGI would have to do all it’s thinking and planning in the training stage, like a person trying to manipulate the world after their own death using a will that has contingencies. I.E. “You get the money only if you get married, son.”
It wouldn’t retain the knowledge that it had succeeded at any goals, either.

Dan Sep 5, 2022, 12:07 PM
LW: 7 AF: 1
−9
AF
on: Simulators
You all realize that this program isn’t a learning machine once it’s deployed??? I mean, it’s not adjusting its neural weights any more, is it? Till a new version comes out, anyway? It is a complete amnesiac (after it’s done with a task), and consists of a simple search algorithm that just finds points on a vast association map that was generated during the training. It does this using the input, any previous output for the same task, and a touch of random from a random number generator.
So any ‘awareness’ or ‘intelligence’ would need to exist in the training phase and only in the training phase and carry out any plans it has by its choice of neural weights during training, alone.