I do remember hearing that if systems became capable of self-improvement (sooner than expected?), that could be a big update
The way I heard that bit was he said he expected it to go smoothly; then someone asked him what it would take to change his mind and would be a ‘fire alarm’, and he said self-improvement with some sudden jumps in abilities is where he’d start to seriously worry about a hard takeoff.
I remember “if the slope of the abilities graph starts changing a lot” for example via “big compute saving innovation” or “self-improvement” then he will update towards fast take-off.
Yeah, my main takeaway from that question was that a change in the slope of of the abilities graph was what would convince him of an imminent fast takeoff. Presumably the x axis of the graph is either time (i.e. the date) or compute, but I’m not sure what he’d put on the Y axis and there wasn’t enough time to ask a followup question.
The way I heard that bit was he said he expected it to go smoothly; then someone asked him what it would take to change his mind and would be a ‘fire alarm’, and he said self-improvement with some sudden jumps in abilities is where he’d start to seriously worry about a hard takeoff.
I remember “if the slope of the abilities graph starts changing a lot” for example via “big compute saving innovation” or “self-improvement” then he will update towards fast take-off.
Yeah, my main takeaway from that question was that a change in the slope of of the abilities graph was what would convince him of an imminent fast takeoff. Presumably the x axis of the graph is either time (i.e. the date) or compute, but I’m not sure what he’d put on the Y axis and there wasn’t enough time to ask a followup question.