Sorry to be blunt, but any distraction filter that can be disabled through the chrome extension menu is essentially worthless. Speaking from experience, for most people this will work for exactly 3 days until they find a website they really want to visit and just “temporarily” disable the extension in order to see it.
Razied
Karma: 1,928
For #5, I think the answer would be to make the AI produce the AI safety ideas which not only solve alignment, but also yield some aspect of capabilities growth along an axis that the big players care about, and in a way where the capabilities are not easily separable from the alignment. I can imagine this being the case if the AI safety idea somehow makes the AI much better at instruction-following using the spirit of the instruction (which is after all what we care about). The big players do care about having instruction-following AIs, and if the way to do that is to use the AI safety book, they will use it.
Do you expect Lecun to have been assuming that the entire field of RL stops existing in order to focus on his specific vision?
Very many things wrong with all of that:
RL algorithms don’t minimize costs, but maximize expected reward, which can well be unbounded, so it’s wrong to say that the ML field only minimizes cost.
LLMs minimize expected log probability of correct token, which is indeed bounded at zero from below, but achieving zero in that case means perfectly predicting every single token on the internet.
The boundedness of the thing you’re minimizing is totally irrelevant, since maximizing is exactly the same as maximizing where g is a monotonic function. You can trivially turn a bounded function into an unbounded one without changing anything to the solution sets.
Even if utility is bounded between 0 and 1, an agent maximizing the expected utility will still never stop, because you can always decrease the probability you were wrong. Quadruple-check every single step and turn the universe into computronium to make sure you didn’t make any errors.
This is very dumb, Lecun should know better, and I’m sure he *would* know better if he spent 5 minutes thinking about any of this.
The word “privilege” has been so tainted by its association with guilt that it’s almost an infohazard to think you’ve got privilege at this point, it makes you lower your head in shame at having more than others, and brings about a self-flagellation sort of attitude. It elicits an instinct to lower yourself rather than bring others up. The proper reactions to all these things you’ve listed is gratitude to your circumstances and compassion towards those who don’t have them. And certainly everyone should be very careful towards any instinct they have at publicly “acknowledging their privilege”… it’s probably your status-raising instincts having found a good opportunity to boast about your intelligence, appearance and good looks while appearing like you’re being modest.
Weird side effect to beware for retinoids: they make dry eyes worse, and in my experience this can significantly decrease your quality of life, especially if it prevents you from sleeping well.
Basically, this shows that every term in a standard Bayesian inference, including the prior ratio, can be re-cast as a likelihood term in a setting where you start off unsure about what words mean, and have a flat prior over which set of words is true.
If the possible meanings of your words are a continuous one-dimensional variable x, a flat prior over x will not be a flat prior if you change variables to y = f(y) for an arbitrary bijection f, and the construction would be sneaking in a specific choice of function f.
Say the words are utterances about the probability of a coin falling heads, why should the flat prior be over the probability p, instead of over the log-odds log(p/(1-p)) ?
Most of the weird stuff involving priors comes into being when you want posteriors over a continuous hypothesis space, where you get in trouble because reparametrizing your space changes the form of your prior, so a uniform “natural” prior is really a particular choice of parametrization. Using a discrete hypothesis space avoids big parts of the problem.
Wait, why doesn’t the entropy of your posterior distribution capture this effect? In the basic example where we get to see samples from a bernoulli process, the posterior is a beta distribution that gets ever sharper around the truth. If you compute the entropy of the posterior, you might say something like “I’m unlikely to change my mind about this, my posterior only has 0.2 bits to go until zero entropy”. That’s already a quantity which estimates how much future evidence will influence your beliefs.
Surely something like the expected variance of would be a much simpler way of formalising this, no? The probability over time is just a stochastic process, and OP is expecting the variance of this process to be very high in the near future.
Unfortunately the entire complexity has just been pushed one level down into the definition of “simple”. The L2 norm can’t really be what we mean by simple, because simply scaling the weights in a layer by A, and the weights in the next layer by 1/A leaves the output of the network invariant, assuming ReLU activations, yet you can obtain arbitrarily high L2 norms by just choosing A high enough.
Unfortunately if OpenAI the company is destroyed, all that happens is that all of its employees get hired by Microsoft, they change the lettering on the office building, and sama’s title changes from CEO to whatever high level manager positions he’ll occupy within microsoft.
Hmm, but here the set of possible world states would be the domain of the function we’re optimising, not the function itself. Like, No-Free-Lunch states (from wikipedia):
Theorem 1: Given a finite set and a finite set of real numbers, assume that is chosen at random according to uniform distribution on the set of all possible functions from to . For the problem of optimizing over the set , then no algorithm performs better than blind search.
Here is the set of possible world arrangements, which is admittedly much smaller than all possible data structures, but the theorem still holds because we’re averaging over all possible value functions on this set of worlds, a set which is not physically restricted by anything.I’d be very interested if you can find Byrnes’ writeup.
Obviously LLMs memorize some things, the easy example is that the pretraining dataset of GPT-4 probably contained lots of cryptographically hashed strings which are impossible to infer from the overall patterns of language. Predicting those accurately absolutely requires memorization, there’s literally no other way unless the LLM solves an NP-hard problem. Then there are in-between things like Barack Obama’s age, which might be possible to infer from other language (a president is probably not 10 yrs old or 230), but within the plausible range, you also just need to memorize it.
There is no optimization pressure from “evolution” at all. Evolution isn’t tending toward anything. Thinking otherwise is an illusion.
Can you think of any physical process at all where you’d say that there is in fact optimization pressure? Of course at the base layer it’s all just quantum fields changing under unitary evolution with a given Hamiltonian, but you can still identify subparts of the system that are isomorphic with a process we’d call “optimization”. Evolution doesn’t have a single time-independent objective it’s optimizing, but it does seem to me that it’s basically doing optimization on a slowly time-changing objective.
Why would you want to take such a child and force them to ‘emotionally develop’ with dumber children their own age?
Because you primarily make friends in school with people in your grade, and if you skip too many grades, the physical difference between the gifted kid and other kids will prevent them from building a social circle based on physical play, and probably make any sort of dating much harder.
Predicting the ratio at t=20s is hopeless. The only sort of thing you can predict is the variance in the ratio over time, like the ratio as a function of time is , where . Here the large number of atoms lets you predict , but the exact number after 20 seconds is chaotic. To get an exact answer for how much initial perturbation still leads to a predictable state, you’d need to compute the lyapunov exponents of an interacting classical gas system, and I haven’t been able to find a paper that does this within 2 min of searching. (Note that if the atoms are non-interacting the problem stops being chaotic, of course, since they’re just bouncing around on the walls of the box)
I’ll try to say the point some other way: you define “goal-complete” in the following way:
By way of definition: An AI whose input is an arbitrary goal, which outputs actions to effectively steer the future toward that goal, is goal-complete.
Suppose you give me a specification of a goal as a function from a state space to a binary output. Is the AI which just tries out uniformly random actions in perpetuity until it hits one of the goal states “goal-complete”? After all, no matter the goal specification this AI will eventually hit it, though it might take a very long time.
I think the interesting thing you’re trying to point at is contained in what it means to “effectively” steer the future, not in goal-arbitrariness.
E.g. I claim humans are goal-complete General Intelligences because you can give us any goal-specification and we’ll very often be able to steer the future closer toward it.
If you’re thinking of “goals” as easily specified natural-language things, then I agree with you, but the point is that turing-completeness is a rigorously defined concept, and if you want to get the same level of rigour for “goal-completeness”, then most goals will be of the form “atom 1 is a location x, atom 2 is at location y, …” for all atoms in the universe. And when averaged across all such goals, literally just acting randomly performs as well as a human or a monkey trying their best to achieve the goal.
I suspect the expert judges would need to resort to known jailbreaking techniques to distinguish LLMs. A fair interesting test might be against expert-but-not-in-ML judges.