Alignment normie. MS CS&AI @ Oregon State University.
.CLI
Karma: 10
Love the post! http://pragmaticaisafety.com/ is down for me right now though. Do the authors still endorse this sequence?
Just spent one year in academia; my experience trying to talk to researchers about AGI match what Dan wrote about.
(ramblingly) Does the No Free Lunch Theorem imply that there’s no one single technique that would always work for AGI alignment? Initial thought is probably not, because the theorem states that the performance of all optimization algorithms are identical across all possible problems. However, AGI alignment is a subset of these problems.
GPT-4 can do math because it has learned particular patterns associated with tokens, including heuristics for certain digits, without fully learning the abstract generalized pattern.
This finding seems consistent with some literatures, such as this where they found that if the multiplication task has an unseen computational graph, then performance deteriorates rapidly. Perhaps check out the keyword “shortcut learning” too.
Game Design
The videos under this category fits better the label “game development” instead. Game Design is more focused on designing rules, mechanics, sometimes narratives, instead of programming.
Is the event happening on June 11th or July 9th?
I think there should be more effort into researching the limits of controllability for self-improving machines. That aspect of rapid self improvement seems pretty important to me since it’s there regardless of which architecture we use to get to the singularity. If the singularity is dangerous no matter how we get there, or how aligned our first try is, then, [clears throat and raises sign] don’t build AGI?
.CLI’s Shortform
I bought the device and watched Interstellar on top of Mt. Hood with the stars as the background. It was a phenomenal experience. That said, having to bear the weight of the device for 2.5 hours, and other limits such as FOV & lens glare makes me hesitant to say movie’s the one killer app right now. I don’t think there is a killer app yet—Apple wants us to come in for that.
The strategic awareness property would be an interesting one to measure. Which existing system would you say are more or less strategically aware? Are there examples we could point toward, like the social media algorithm one?
I do hear No Free Lunch theorem get thrown around when an architecture fails to solve some problem which its inductive bias doesn’t fit. But I think it’s just thrown around as a vibe.