CEO at AE Studio
Judd Rosenblatt
Karma: 1,298
There Should Be More Alignment-Driven Startups
Key takeaways from our EA and alignment research surveys
AE Studio @ SXSW: We need more AI consciousness research (and further resources)
Survey for alignment researchers!
That would be great! And it’s exactly the sort of thing we’ve dreamed about building at AE since the start.
Incidentally, I’ve practiced something (inferior) like this with my wife in the past and we’ve gotten good at speaking simultaneously and actually understanding multiple threads at the same time (though it seems to break down if one of the threads is particularly complex).
It seems like an MVP hyperphone could potentially just be a software project/not explicitly require BCI (though certainly would be enhanced with it). We would definitely consider building it, at least as a Same Day Skunkworks. Are you aware of any existing tool that’s at all like this?
You might also enjoy this blog post, which talks about how easily good ideas can be lost and why a tool like this could be so high value.
My favorite quotes from the piece:
1. “While ideas ultimately can be so powerful, they begin as fragile, barely formed thoughts, so easily missed, so easily compromised, so easily just squished.”
2. “You need to recognize those barely formed thoughts, thoughts which are usually wrong and poorly formed in many ways, but which have some kernel of originality and importance and truth. And if they seem important enough to be worth pursuing, you construct a creative cocoon around them, a set of stories you tell yourself to protect the idea not just from others, but from your own self doubts. The purpose of those stories isn’t to be an air tight defence. It’s to give you the confidence to nurture the idea, possibly for years, to find out if there’s something really there.
3. “And so, even someone who has extremely high standards for the final details of their work, may have an important component to their thinking which relies on rather woolly arguments. And they may well need to cling to that cocoon. Perhaps other approaches are possible. But my own experience is that this is often the case.”
The ‘Neglected Approaches’ Approach: AE Studio’s Alignment Agenda
And another interesting one from the summit:
“There was almost no discussion around agents—all gen AI & model scaling concerns.
It’s perhaps because agent capabilities are mediocre today and thus hard to imagine, similar to how regulators couldn’t imagine GPT-3’s implications until ChatGPT.”—https://x.com/kanjun/status/1720502618169208994?s=46&t=D5sNUZS8uOg4FTcneuxVIg
Right now it seems to me that one of the highest impact things not likely to be done by default is substantially increased funding for AI safety.
I got https://www.pinkshoggoth.com/ inspired by Pink Shoggoths: What does alignment look like in practice?
Right now it’s hosting a side project (that may wind up being replaced by new ChatGPT features). Feel free to DM me if you have a better use for it though!
This is a great point. I also notice that a decent number of people’s risk models change frequently with various news, and that’s not ideal either, as it makes them less likely to stick with a particular approach that depends on some risk model. In an ideal world we’d have enough people pursuing enough approaches with most possible risk models that it’s make little sense for anyone to consider switching. Maybe the best we can approximate now is to discuss this less.