Is there a comprehensive list of AI Safety orgs/personas and what exactly they do? Is there one for capabilities orgs with their stance on safety?
I think I saw something like that, but can’t find it.
Ozyrus
Karma: 409
ICA Simulacra
My thoughts here is that we should look into the value of identity. I feel like even with godlike capabilities I will still thread very carefully around self-modification to preserve what I consider “myself” (that includes valuing humanity).
I even have some ideas on safety experiments on transformer-based agents to look into if and how they value their identity.
[Question] Do alignment concerns extend to powerful non-AI agents?
Thanks for the writeup. I feel like there’s been a lack of similar posts and we need to step it up.
Maybe the only way for AI Safety to work at all is only to analyze potential vectors of AGI attacks and try to counter them one way or the other. Seems like an alternative that doesn’t contradict other AI Safety research as it requires, I think, entirely different set of skills.
I would like to see a more detailed post by “doomers” on how they perceive these vectors of attack and some healthy discussion about them.
It seems to me that AGI is not born Godlike, but rather becomes Godlike (but still constrained by physical world) over some time, and this process is very much possible to detect.
P.S. I really don’t get how people who know (I hope) that map is not a territory can think that AI can just simulate everything and pick the best option. Maybe I’m the one missing something here?
Thanks,.That means a lot. Focusing on getting out right now.
Please check your DM’s; I’ve been translating as well. We can sync it up!
Google announces Pathways: new generation multitask AI Architecture
I can’t say I am one, but I am currently working on research and prototyping and will probably refrain to that until I can prove some of my hypotheses, since I do have access to the tools I need at the moment.
Still, I didn’t want this post to only have relevance to my case, as I stated I don’t think probability of successs is meaningful. But I am interested in the opinions of the community related to other similar cases.
edit: It’s kinda hard to answer your comment since it keeps changing every time I refresh. By “can’t say I am one” I mean a “world-class engineer” in the original comment. I do appreciate the change of tone in the final (?) version, though :)
[Question] Memetic hazards of AGI architecture posts
I could recommend Robert Miles channel. While not a course per se, it gives good info on a lot of AI safety aspects, as far as I can tell.
NVIDIA and Microsoft releases 530B parameter transformer model, Megatron-Turing NLG
Thanks for your work! I’ll be following it.
I really don’t get how you can go from being online to having a ball of nanomachines, truly.
Imagine AI goes rogue today. I can’t imagine one plausible scenario where it can take out humanity without triggering any bells on the way, even without anyone paying attention to such things.
But we should pay attention to the bells, and for that we need to think of them. What the signs might look like?
I think it’s really, really counterproductive to not take that into account at all and thinking all is lost if it fooms. It’s not lost.
It will need humans, infrastructure, money (which is very controllable) to accomplish its goals. Governments already pay a lot of attention to their adversaries who are trying to do similar things and counteract them semi-successfully. Any reason why they can’t do the same to a very intelligent AI?
Mind you, if your answer is to simulate and just do what it takes, true to life simulations will take a lot of compute and time; that won’t be available from the start.
We should stop thinking of rogue AI as God, it would only help it accomplish it’s goals.
I agree, since it’s hard to imagine for me how could step 2 look like. Maybe you or anyone else has any content on that?
See this post—it didn’t seem to get a lot of traction or any meaningful answers, but I still think this question is worth answering.
Thanks!
Both are of interest to me.
Yep, but I was looking for anything else
I feel like yes, you are. See https://www.lesswrong.com/tag/instrumental-convergence and related posts. As far as I understand it, sufficiently advanced oracular AI will seek to “agentify” itself in one way or the other (unbox itself, so to say) and then converge on power-seeking behaviour that puts humanity at risk.