I play on lichess often. I can tell you that a lichess rating of 2900 absolutely corresponds to grandmaster level strength. It is rare for FMs or IMs to exceed a 2800 blitz rating. Most grandmasters hover around 2600-2800 blitz.
Hide
Karma: 22
The discussion on attack surfaces is very useful, intuitive and accessible. If a better standalone resource doesn’t already exist, such a (perhaps expanded) list/discussion would be a useful intro for people unfamiliar with specific risks.
This was excruciatingly frustrating to read, well done.
This is well-reasoned, but I have difficulty understanding why this kind of takeover would be necessary from the perspective of a powerful, rational agent. Assuming AGI is indeed worth its name, it seems the period of time needed for it to “play nice” would be very brief.
AGI would be expected to be totally unconcerned with being “clean” in a takeover attempt. There would be no need to leave no witnesses, nor avoid rousing opposition. Once you have access to sufficient compute, and enough control over physical resources, why wait 10 years for humanity to be slowly, obliviously strangled?
You say there’s “no need” for it to reveal that we are in conflict, but in many cases, concealing a conflict will prevent a wide range of critical, direct moves. The default is a blatant approach—concealing a takeover requires more effort and more time.
The nano-factories thing is a rather extreme version of this, but strategies like poisoning the air/water, building/stealing an army of drones, launching hundreds of nukes, etc., all seem like much more straightforward ways to cripple opposition, even with a relatively weak (99.99th percentile-human-level) AGI.
It could certainly angle for humanity to go out with a whimper, not a bang. But if a bang is quicker, why bother with the charade?
My first thought as well. IGF-1 exists for a reason. Growth is universally necessary for development, repair and function.
shift the
Minor edit - should be “shift in the”
It’s encouraging to see more emphasis recently on the political and public-facing aspects of alignment. We are living in a far-better-than-worst-case world where people, including powerful ones, are open to being convinced. They just need to be taught—to have it explained to them intuitively.
It seems cached beliefs produced by works like you get about five words have led to a passive, unspoken attitude among many informed people that attempting to explain anything complicated is futile. It isn’t futile. It’s just difficult.
In another of your most downvoted posts, you say
I kind of expect this post to be wildly unpopular
I think you may be onto something here.
You can fail to get rid of balls. All of your energy and effort can go into not allowing something to crash or fall, averting each disaster shortly before it would be too late. Speaking for ten minutes with each of fifty of sources every day can be a good way to keep any of them from being completely neglected, but it’s a terrible way to actually finish any of those projects. The terminal stage of this is a system so tied up in maintaining itself and stopping from falling behind that it has no slack to clear tasks or to improve its speed.
This is the salient danger of this approach. While valuable, it absolutely must be paired with a ruthless, exacting and periodic inventory of the balls that matter, otherwise your slack will be completely burned and you will die an exhausted and unaccomplished juggler.
It seems intuitively bad:
Capabilities and accelerationist-focused researchers have gone from diluted and restrained to concentrated and encouraged
Microsoft now has unbounded control, rather than a 49% stake
Microsoft cannot be expected to have any meaningful focus on alignment/safety
They are not starting from scratch: a huge chunk of their most capable staff and leadership will be involved
The “superalignment” project will be at best dramatically slowed, and possibly abandoned if OpenAI implodes
Other major labs smell blood in the water, possibly exacerbating race dynamics, not to mention a superficial increase (by 1) in the number of serious players.
All good practices. Although, isn’t this just “more metal”, rather than “less ore”? I imagine one would want to maximize both the inputs and outputs, even if opportunities for increasing inputs are exhausted more quickly.
How is such a failure of imagination possible?
It’s odd to claim that, contingent upon AGI being significantly smarter than us, and wanting to kill us, that there is no realistic pathway for us to be physically harmed.Claims of this sort by intelligent, competent people likely reveal that they are passively objecting to the contingencies rather than disputing whether these contingencies would lead to the conclusion.
The quotes you’re responding to here superficially imply “if smart + malicious AI, it can’t kill us”, but it seems much more likely this is a warped translation of either “AI can’t be smart”, or “AI can’t be malicious”.
I would happily play the role of B.
I do not have an established FIDE rating, but my strength is approximately 1850 FIDE currently (based on playing against FIDE rated players OTB quite often, as well as maintaining 2100-2200 blitz ratings on Lichess & Chess.com, and 2200-2300 bullet). I’d be available after 6:30 pm (UTC+10) until ~12:00 pm (UTC+10). Alternatively, weekends are very flexible. I could do a few hours per week.
I agree short/long time controls are a relevant, because speed is a skill that is almost entirely independent of conceptual knowledge and is mostly a function of baseline playing ability.
Edit: Would also be fine with C
[Question] Is there any reason to expect subjective continuity in “mind uploading”?
Strongly agree. To my utter bewilderment, Eliezer appears to be exacerbating this vulnerability by making no efforts whatsoever to appear credible to the casual person.
In nearly all of his public showings in the last 2 years, he has:Rocked up in a trilby
Failed to adequately introduce himself
Spoken in condescending, aloof and cryptic tones; and
Failed to articulate the central concerns in an intuitive manner
As a result, to the layperson, he comes off as an egotistical, pessimistic nerd with fringe views—a perfect clown from which to retreat to a “middle ground”, perhaps offered by the eminently reasonable-sounding Yann LeCun—who, after all, is Meta’s chief AI scientist.
The alignment community is dominated by introverted, cerebral rationalists and academics, and consequently, a common failure is to ignore the significance of image as either a distraction or an afterthought.
I cannot imagine losing this game as the gatekeeper either, honestly.
Does anyone want to play against me? I’ll bet you $50 USD.