I often find information about AI development on X (f.k.a.Twitter) and sometimes other websites. They usually don’t warrant their own post, so I’ll use this thread to share. I’ll be placing a fairly low filter on what I share.
There’s someone on X (f.k.a.Twitter) called Jimmy Apples (🍎/acc) and he has shared some information in the past that turned out to be true (apparently the GPT-4 release date and that OAI’s new model would be named “Gobi”). He recently tweeted, “AGI has been achieved internally.” Some people think that the Reddit comment below may be from the same guy (this is just a weak signal, I’m not implying you should consider it true or update on it):
Where is the evidence that he called OpenAI’s release date and the Gobi name? All I see is a tweet claiming the latter but it seems the original tweet isn’t even up?
Predicting the GPT-4 launch date can easily be disproven with the confidence game. It’s possible he just created a prediction for every day and deleted the ones that didn’t turn out right.
For the Gobi prediction it’s tricky. The only evidence is the Threadreader and a random screenshot from a guy who seems clearly related to jim. I am very suspicious of the Threadreader one. On one hand I don’t see a way it can be faked, but it’s very suspicious that the Gobi prediction is Jimmy’s only post that was saved there despite him making an even bigger bombshell “prediction”. It’s also possible, though unlikely, that the Information’s article somehow found his tweet and used it as a source for their article.
What kills Jimmy’s credibility for me is his prediction back in January (you can use the Wayback Machine to find it) that OAI had finished training GPT-5, no not a GPT-5 level system, the ACTUAL GPT-5 in October 2022 and that it was 125T parameters.
Also goes without saying, pruning his entire account is suspicious too.
I’ll try to find them, but this was what people were saying. They also said he deleted past tweets so that evidence may forever be gone.
I remember one tweet where Jimmy said something like, “Gobi? That’s old news, I said that months ago, you need to move on to the new thing.” And I think he linked the tweet though I’m very unsure atm. Need to look it up, but you can use the above for a search.
New tweet by Jimmy Apples. This time, he’s insinuating that OpenAI is funding a stealth startup working on BCI.
If this is true, then it makes sense they would prefer not to do it internally to avoid people knowing in advance based on their hires. A stealth startup would keep things more secret.
Not sure exactly what this means, but Jimmy Apples has now tweeted the following:
My gut is telling me that he apple-bossed too close to the sun (released info he shouldn’t have, and now that he’s concerned about his job or some insider’s job), and it’s time for him to stop sharing stuff (the apple being bitten symbolizing that he is done sharing info).
This is because the information in my shortform was widely shared on X and beyond.
He also deleted all of his tweets (except for the retweets).
Or that he was genuinely just making things up and tricking us for fun, and a cryptic exit is a perfect way to leave the scene. I really think people are looking way too deep into him and ignoring the more outlandish predictions he’s made (125T GPT-4 and 5 in October 2022), along with the fact there is never actual evidence of his accurate ones, only 2nd hand very specific and selective archives.
He did say some true things before. I think it’s possible all of the new stuff is untrue, but we’re getting more reasons to believe it’s not entirely false. The best liars sprinkle in truth.
I think, as a security measure, it’s also possible that even people within OpenAI know all the big details of what’s going on (this is apparently the case for Anthropic). This could mean, for OpenAI employees, that some details are known while others are not. Employees themselves could be forced to speculate on some things.
Either way, I’m not obsessing too much over this. Just sharing what I’m seeing.
More predictions/insights from Jimmy and crew. He’s implying that people (like I have also been saying) that some people are far too focused on scale over training data and architectural improvements. IMO, the bitter lesson is a thing, but I think we’ve over-updated on it.
“Most interesting part of @sama talk: GPT5 and GPT6 are “in the bag” but that’s likely NOT AGI (eg something that can solve quantum gravity) without some breakthroughs in reasoning. Strong agree.”
AGI is “something that can solve quantum gravity”?
That’s not just a criterion for general intelligence, that’s a criterion for genius-level intelligence. And since general intelligence in a computer has advantages of speed, copyability, little need for down time, that are not possessed by general intelligence, AI will be capable of contributing to its training, re-design, agentization, etc, long before “genius level” is reached.
This underlines something I’ve been saying for a while, which is that superintelligence, defined as AI that definitively surpasses human understanding and human control, could come into being at any time (from large models that are not publicly available but which are being developed privately by Big AI companies). Meanwhile, Eric Schmidt (former Google CEO) says about five years until AI is actively improving itself, and that seems generous.
So I’ll say: timeline to superintelligence is 0-5 years.
capable of contributing to its training, re-design, agentization, etc, long before “genius level” is reached
In some models of the world this is seen as unlikely to ever happen, these things are expected to coincide, which collapses the two definitions of AGI. I think the disparity between sample efficiency of in-context learning and that of pre-training is one illustration for how these capabilities might come apart, in the direction that’s opposite to what you point to: even genius in-context learning doesn’t necessarily enable the staying power of agency, if this transient understanding can’t be stockpiled and the achieved level of genius is insufficient to resolve the issue while remaining within its limitations (being unable to learn a lot of novel things in the course of a project).
My guess is that he is implying that they will be releasing open source mixture of experts models in a few months from now. They are currently running them on CPUs.
Occasionally reading what OSS AI gurus say, they definitely overhype their stuff constantly. The ones who make big claims and try to hype people up are often venture entrepreneur guys rather than actual ML engineers.
The open source folks I mostly keep an eye on are the ones who do actually code and train their own models. Some are entrepreneurs, but they know a decent amount. Not top engineers, but they seem to be able to curate datasets and train custom models.
There’s some wannabe script kiddies too, but once you lurk enough, you become aware of who are actually decent engineers (you’ll find some at Vector Institute and Jeremy Howard is pro- open source, for example). I wouldn’t totally discount them having an impact, even though some of them will overhype.
Jacques’ AI Tidbits from the Web
I often find information about AI development on X (f.k.a.Twitter) and sometimes other websites. They usually don’t warrant their own post, so I’ll use this thread to share. I’ll be placing a fairly low filter on what I share.
There’s someone on X (f.k.a.Twitter) called Jimmy Apples (🍎/acc) and he has shared some information in the past that turned out to be true (apparently the GPT-4 release date and that OAI’s new model would be named “Gobi”). He recently tweeted, “AGI has been achieved internally.” Some people think that the Reddit comment below may be from the same guy (this is just a weak signal, I’m not implying you should consider it true or update on it):
Where is the evidence that he called OpenAI’s release date and the Gobi name? All I see is a tweet claiming the latter but it seems the original tweet isn’t even up?
This is the tweet for Gobi: https://x.com/apples_jimmy/status/1703871137137176820?s=46&t=YyfxSdhuFYbTafD4D1cE9A
I asked someone if it’s fake. Apparently not, you can find it on google archive: https://threadreaderapp.com/thread/1651837957618409472.html
Predicting the GPT-4 launch date can easily be disproven with the confidence game. It’s possible he just created a prediction for every day and deleted the ones that didn’t turn out right.
For the Gobi prediction it’s tricky. The only evidence is the Threadreader and a random screenshot from a guy who seems clearly related to jim. I am very suspicious of the Threadreader one. On one hand I don’t see a way it can be faked, but it’s very suspicious that the Gobi prediction is Jimmy’s only post that was saved there despite him making an even bigger bombshell “prediction”. It’s also possible, though unlikely, that the Information’s article somehow found his tweet and used it as a source for their article.
What kills Jimmy’s credibility for me is his prediction back in January (you can use the Wayback Machine to find it) that OAI had finished training GPT-5, no not a GPT-5 level system, the ACTUAL GPT-5 in October 2022 and that it was 125T parameters.
Also goes without saying, pruning his entire account is suspicious too.
I’ll try to find them, but this was what people were saying. They also said he deleted past tweets so that evidence may forever be gone.
I remember one tweet where Jimmy said something like, “Gobi? That’s old news, I said that months ago, you need to move on to the new thing.” And I think he linked the tweet though I’m very unsure atm. Need to look it up, but you can use the above for a search.
New tweet by Jimmy Apples. This time, he’s insinuating that OpenAI is funding a stealth startup working on BCI.
If this is true, then it makes sense they would prefer not to do it internally to avoid people knowing in advance based on their hires. A stealth startup would keep things more secret.
Might be of interest, @lisathiergart and @Allison Duettmann.
Not sure exactly what this means, but Jimmy Apples has now tweeted the following:
My gut is telling me that he apple-bossed too close to the sun (released info he shouldn’t have, and now that he’s concerned about his job or some insider’s job), and it’s time for him to stop sharing stuff (the apple being bitten symbolizing that he is done sharing info).
This is because the information in my shortform was widely shared on X and beyond.
He also deleted all of his tweets (except for the retweets).
Or that he was genuinely just making things up and tricking us for fun, and a cryptic exit is a perfect way to leave the scene. I really think people are looking way too deep into him and ignoring the more outlandish predictions he’s made (125T GPT-4 and 5 in October 2022), along with the fact there is never actual evidence of his accurate ones, only 2nd hand very specific and selective archives.
He did say some true things before. I think it’s possible all of the new stuff is untrue, but we’re getting more reasons to believe it’s not entirely false. The best liars sprinkle in truth.
I think, as a security measure, it’s also possible that even people within OpenAI know all the big details of what’s going on (this is apparently the case for Anthropic). This could mean, for OpenAI employees, that some details are known while others are not. Employees themselves could be forced to speculate on some things.
Either way, I’m not obsessing too much over this. Just sharing what I’m seeing.
More predictions/insights from Jimmy and crew. He’s implying that people (like I have also been saying) that some people are far too focused on scale over training data and architectural improvements. IMO, the bitter lesson is a thing, but I think we’ve over-updated on it.
Relatedly, someone shared a new 13B model that apparently is better and comparable to GPT-4 in logical reasoning (based on benchmarks, which I don’t usually trust too much). Note that the model is a solver-augmented LM.
Here’s some context regarding the paper:
Sam Altman at a YC founder reunion: https://x.com/smahsramo/status/1706006820467396699?s=46&t=YyfxSdhuFYbTafD4D1cE9A
“Most interesting part of @sama talk: GPT5 and GPT6 are “in the bag” but that’s likely NOT AGI (eg something that can solve quantum gravity) without some breakthroughs in reasoning. Strong agree.”
AGI is “something that can solve quantum gravity”?
That’s not just a criterion for general intelligence, that’s a criterion for genius-level intelligence. And since general intelligence in a computer has advantages of speed, copyability, little need for down time, that are not possessed by general intelligence, AI will be capable of contributing to its training, re-design, agentization, etc, long before “genius level” is reached.
This underlines something I’ve been saying for a while, which is that superintelligence, defined as AI that definitively surpasses human understanding and human control, could come into being at any time (from large models that are not publicly available but which are being developed privately by Big AI companies). Meanwhile, Eric Schmidt (former Google CEO) says about five years until AI is actively improving itself, and that seems generous.
So I’ll say: timeline to superintelligence is 0-5 years.
In some models of the world this is seen as unlikely to ever happen, these things are expected to coincide, which collapses the two definitions of AGI. I think the disparity between sample efficiency of in-context learning and that of pre-training is one illustration for how these capabilities might come apart, in the direction that’s opposite to what you point to: even genius in-context learning doesn’t necessarily enable the staying power of agency, if this transient understanding can’t be stockpiled and the achieved level of genius is insufficient to resolve the issue while remaining within its limitations (being unable to learn a lot of novel things in the course of a project).
Someone in the open source community tweeted: “We’re about to change the AI game. I’m dead serious.”
My guess is that he is implying that they will be releasing open source mixture of experts models in a few months from now. They are currently running them on CPUs.
Lots of cryptic tweet from the open source LLM guys: https://x.com/abacaj/status/1705781881004847267?s=46&t=YyfxSdhuFYbTafD4D1cE9A
“If you thought current open source LLMs are impressive… just remember they haven’t peaked yet”
To be honest, my feeling is that they are overhyping how big of deal this will be. Their ego and self-importance tend to be on full display.
Occasionally reading what OSS AI gurus say, they definitely overhype their stuff constantly. The ones who make big claims and try to hype people up are often venture entrepreneur guys rather than actual ML engineers.
The open source folks I mostly keep an eye on are the ones who do actually code and train their own models. Some are entrepreneurs, but they know a decent amount. Not top engineers, but they seem to be able to curate datasets and train custom models.
There’s some wannabe script kiddies too, but once you lurk enough, you become aware of who are actually decent engineers (you’ll find some at Vector Institute and Jeremy Howard is pro- open source, for example). I wouldn’t totally discount them having an impact, even though some of them will overhype.