It sounds pretty implausible to me, intellectual productivity is usually at its peak from mid-20s to mid-30s(for high fluid-intelligence fields like math and physics)
interstice
Karma: 2,325
Confused as to why this is so heavily downvoted.
These emails and others can be found in document 32 here.
but it seems that even on LW people think winning on a noisy N=1 sample is proof of rationality
It’s not proof of a high degree of rationality but it is evidence against being an “idiot” as you said. Especially since the election isn’t merely a binary yes/no outcome, we can observe that there was a huge republican blowout exceeding most forecasts(and in fact freddi bet a lot on republican pop vote too at worse odds, as well as some random states, which gives a larger update) This should increase our credence that predicting a republican win was rational. There were also some smart observers with IMO good arguments that trump was favored pre-election, e.g. https://x.com/woke8yearold/status/1851673670713802881
“Guy with somewhat superior election modeling to Nate Silver, a lot of money, and high risk tolerance” is consistent with what we’ve seen. Not saying that we have strong evidence that Freddi is a genius but we also don’t have much reason to think he is an idiot IMO.
Looks likely that tonight is going to be a massive transfer of wealth from “sharps”(among other people) to him. Post hoc and all, but I think if somebody is raking in huge wins while making “stupid” decisions it’s worth considering whether they’re actually so stupid after all.
Good post, it’s underappreciated that a society of ideally rational people wouldn’t have unsubsidized, real-money prediction markets.
unless you’ve actually got other people being wrong even in light of the new actors’ information
Of course in real prediction markets this is exactly what we see. Maybe you could think of PMs as they exist not as something that would exist in an equilibrium of ideally rational agents, but as a method of moving our society closer to such an equilibrium, subsidized by the bets of systematically irrational people. It’s not a perfect such method, but does have the advantage of simplicity. How many of these issues could be solved by subsidizing markets?
Discord Message
What discord is this, sounds cool.
That’s probably the one I was thinking of.
I know of only two people who anticipated something like what we are seeing far ahead of time; Hans Moravec and Jan Leike
I didn’t know about Jan’s AI timelines. Shane Legg also had some decently early predictions of AI around 2030(~2007 was the earliest I knew about)
Some beliefs can be worse or better at predicting what we observe, this is not the same thing as popularity.
Far enough in the future ancient brain scans would be fascinating antique artifacts like rare archaeological finds today, I think people would be interested in reviving you on that basis alone(assuming there are people-like things with some power in the future)
I like the decluttering. I think the title should be smaller and have less white space above it. Also think that it would be better if the ToC was maybe just faded a lot until mouseover, the sudden appearance/disappearance feels too sudden.
No I don’t think so because people could just airgap the GPUs.
Weaker AI probably wouldn’t be sufficient to carry out an actually pivotal act. For example the GPU virus would probably be worked around soon after deployment, via airgapping GPUs, developing software countermeasures, or just resetting infected GPUs.
This discussion is a nice illustration of why x-riskers are definitely more power-seeking than the average activist group. Just like Eskimos proverbially have 50 words for snow, AI-risk-reducers need at least 50 terms for “taking over the world” to demarcate the range of possible scenarios. ;)
Nice overview, I agree but I think the 2016-2021 plan could still arguably be described as “obtain god-like AI and use it to take over the world”(admittedly with some rhetorical exaggeration, but like, not that much)
I would be happy to take bets here about what people would say.
Sure, I DM’d you.
I think making inferences from that to modern MIRI is about as confused as making inferences from people’s high-school essays about what they will do when they become president
Yeah, but it’s not just the old MIRI views, but those in combination with their statements about what one might do with powerful AI, the telegraphed omissions in those statements, and other public parts of their worldview e.g. regarding the competence of the rest of the world. I get the pretty strong impression that “a small group of people with overwhelming hard power” was the ideal goal, and that this would ideally be controlled by MIRI or by a small group of people handpicked by them.
I think they talked explicitly about planning to deploy the AI themselves back in the early days(2004-ish) then gradually transitioned to talking generally about what someone with a powerful AI could do.
But I strongly suspect that in the event that they were the first to obtain powerful AI, they would deploy it themselves or perhaps give it to handpicked successors. Given Eliezer’s worldview I don’t think it would make much sense for them to give the AI to the US government(considered incompetent) or AI labs(negligently reckless)
It wasn’t specified but I think they strongly implied it would be that or something equivalently coercive. The “melting GPUs” plan was explicitly not a pivotal act but rather something with the required level of difficulty, and it was implied that the actual pivotal act would be something further outside the political Overton window. When you consider the ways “melting GPUs” would be insufficient a plan like this is the natural conclusion.
doesn’t require replacing existing governments
I don’t think you would need to replace existing governments. Just block all AI projects and maintain your ability to continue doing so in the future via maintaining military supremacy. Get existing governments to help you, or at least not interfere, via some mix of coercion and trade. Sort of a feudal arrangement with a minimalist central power.
People asked for a citation so here’s one: https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://www.kellogg.northwestern.edu/faculty/jones-ben/htm/age%2520and%2520scientific%2520genius.pdf&ved=2ahUKEwiJjr7b8O-JAxUVOFkFHfrHBMEQFnoECD0QAQ&sqi=2&usg=AOvVaw0HF9-Ta_IR74M8df7Av6Qe
Although my belief was more based on anecdotal knowledge of the history of science. Looking up people at random: Einstein’s annus mirabilis was at 26; Cantor invented set theory at 29; Hamilton discovered Hamiltonian mechanics at 28; Newton invented calculus at 24. Hmmm I guess this makes it seem more like early 20s − 30. Either way 25 is definitely in peak range, and 18 typically too young(although people have made great discoveries by 18, like Galois. But he likely would have been more productive later had he lived past 20)