Become a PIBBSS Research Affiliate
TL;DR: PIBBSS is hiring research affiliates!
PIBBSS is a research initiative facilitating work that draws on the parallels between intelligent behavior in natural and artificial systems, and leveraging these insights towards making AI systems safe, beneficial and aligned.
We want to support excellent researchers pursuing “PIBBSS-style” AI alignment research by providing them with tailored longer-term support: a full-time salary, a lively research community, operational support and more. The initial commitment is 6 months, with potential extensions to 12 or more.
To apply, click here. To be considered as part of the next iteration of affiliates, apply by November 5th.
At PIBBSS (“Principles of Intelligent Behavior in Biological and Social Systems”), our mission is to facilitate AI alignment research that draws on the parallels between intelligent behavior in natural and artificial systems. (For more details, see the section “About PIBBSS and our research priorities” below.)
Since PIBBSS’ inception in 2022, we have run two iterations of our 3-months research fellowships, as well as a reading group, an ongoing speaker series, and a number of research retreats exploring topics in line with PIBBSS’ research interests. Our alumni have gone on to pursue AI alignment research at places like OpenAI, Anthropic, the Alignment of Complex Systems research group, and academia, as well as through independent funding.
While we continue to be excited about the value of those activities, we also believe that there is more value to be had! In particular, we see potential in a program geared towards supporting longer-term, more sustained research efforts in line with PIBBSS’ research bet.
As such, we are delighted to launch our new PIBBSS Affiliate Program.
Goals and key features of the Affiliate Program
At its core, our goal in running this program is to counterfactually help produce substantial, high-quality and legible research progress on problems in AI risk, alignment and governance. The affiliate program aims to do this by supporting promising individual researchers in a highly tailored fashion to scope, develop, and progress on their personal research agendas.
Concrete success scenarios might look like (but are not limited to) affiliates publishing insightful research results; joining the efforts of existing research groups or founding new ones; identifying, scoping, testing and building traction on novel or neglected research avenues; maturing their personal research agenda and secure independent funding; etc.
The PIBBSS Affiliate Program has the following key features:
Affiliates receive a salary of 5,000 USD/month/FTE. We are open to paying higher salaries to exceptional candidates.
We are, by default, looking for affiliates who will be joining the program full-time. That said, we are open to adapting to affiliates’ circumstances where this seems appropriate, such as joining the program in a part-time fashion. We also have the capacity to offer support to people who do not want to leave their current research positions.
Affiliates benefit from a range of support structures aimed at helping them focus on and advance in their research endeavors, such as:
Quarterly research retreats bringing together affiliates as well as other alignment researchers frothe the m our extended network
Personalized research and strategic support, typically in the form of bi-weekly calls with your point person (with option to adapt frequency of nature of support to the ffiliate’s needs and preferences)
Access to the PIBBSS research community and extended network, including joining our slack space, tailored introductions, opportunities to participate in workshops, etc.
Financial and administrative support in order to e.g. visit relevant research conferences, get access to base models and compute where required for research, etc.
Access to office space depending on the needs/preferences of affiliates.
Note: A key feature of the program is to provide highly tailored support to our affiliates, according to what counterfactually helps them most make progress on their research. As such, the above list, while representative, is not necessarily fully precise or comprehensive.
The initial duration of the affiliateship is 6 months, at which point we review affiliates for a possible extension to 12 months. In exceptional circumstances, we will make initial offers of 12 months.
Our vision is for the affiliate program to provide a serious long-term home for promising researchers pursuing PIBBSS-style AI alignment research at time scales of 1 year and more, including the possibility of permanent positions.
We want to build on and strengthen the supportive, curious and dedicated culture that PIBBSS has built over the last two years. We care about good epistemic practices while navigating the intricacies of making sense of a complex world and particularly emphasize the value of epistemic pluralism in doing so.
At the current point in time, we orient to the affiliate program with an experimental lens—testing the hypothesis that this program can provide significant value, and learning about how exactly we can support our affiliates most effectively. Depending on what we learn, we might scale up or scale down, streamline the program or make it more tailored to individual affiliates, aim to have affiliates for time-limited or for open-ended periods, or do other things which increase the impact of the program.
Requirements of the Affiliate Program
The affiliate program is a very open-ended program with limited requirements. Here is what we asked from our affiliates:
Submitting a quarterly report reflecting on your research progress and prospective research plan.
The reports are meant to help the affiliate by providing an accountability structure and encouraging them to zoom out and reflect on a regular basis. They also help us understand what affiliates have been working on and whether there is anything we can do to support them better going forward.
Participating in our ~quarterly research retreats
A genuine commitment to making progress on questions related to AI safety, alignment and governance, and constructive, truth-seeking and kind participation in our shared epistemic culture
What are we looking for in an ideal affiliate?
Affiliates are selected for being in a strong position to pursue promising PIBBSS-aligned research directions in AI alignment in a highly self-directed fashion. As such, we are looking for (a mix of) the following characteristics:
Scholarly competence/caliber in your domain(s) of expertise
Strong prior understanding of AI Risk & Alignment
Evidence of good research judgment/taste
Evidence for good research productivity, including strong self-management skills
Promisingness of your proposed line of research
Ability to counterfactually benefit from what PIBBSS can offer
Note that, in our experience, great PIBBSS-style researchers have often had non-standard backgrounds. As such, we wish to encourage individuals with varied backgrounds and perspectives to apply.
About PIBBSS and our research priorities
PIBBSS is a research initiative facilitating work that draws on the parallels between intelligent behavior in natural and artificial systems, and leveraging these insights towards making AI systems safe, beneficial and aligned.
In essence, we believe that the study of complex natural and social systems can help us better understand the risk involved in implementing such behaviors in artificial substrates and improve our ability to develop AI systems such that they reliably display desired safety- and governability-relevant properties. (We describe in some more detail the nature of and some background assumptions going into our epistemic bet in this write up, as well as in this talk).
To provide more concrete pointers towards the type of research we’re excited about, here is a sample of the work that we have supported in the past:
Aiming to develop a principled understanding of the dynamics emerging from multi-agent interactions between AI systems, such as by drawing on the study of collective behavior in army ants or models from evolutionary biology and ecology
Exploring how to include understanding of multi-agent AI dynamics in developing model evaluations [forthcoming]
Exploring novel avenues for interpretability, such as A Geometric Approach to Interpretability and Singular Learning Theory
Building towards naturalized theories of agency, such as by exploring the role of abstractions in the phenomenon of life; testing the natural abstraction hypothesis in the game of life; thinking about the characteristics of agents that apply the intentional stance on themselves (“auto-intentional agents”); or applying a dynamical systems perspective on goal-oriented behavior
Tackling problems in decision theory, such as formalizing updateless decision theory, robustness to self-modification, or trying to bridge expected utility maximization and optimization
Developing more realistic account of values and practical reasoning, and exploring their implications for our understanding of the AI alignment problem [forthcoming]
Evaluating the potential and limits of existing legal tools for reducing catastrophic risks from AI
Testing the potential for productive epistemic exchange between AI alignment and other fields/topics, e.g. artificial life, basal cognition, morphological computation, active inference, critical social theory, etc.
Looking at philosophy of sciences (and the special sciences) to find ways to make our. research. efforts. more. productive.
Our fellows and collaborators have brought expertise from many domains to bear on the question of AI alignment, including but not limited to:
Ecology
Evolutionary and developmental biology
Systems biology
Biophysics
Neuroscience
Cognitive science
Linguistics
Legal theory
Political theory
Economic theory
Social theory
Complex systems studies
Cybernetics
Network science
Physics
Philosophy of Science
How to apply
Applications are now open. You can apply by following this link.
We accept new affiliates on a ~quarterly basis. To be considered as part of the next iteration, apply by November 5th.
We expect to hire 2-6 (median 4) affiliates over the coming 6 months.
About the application process: The details of the application process may vary between candidates in virtue of which information we think will most help us make well-informed decisions. In most cases, the initial stage (application form) is followed by one or more interviews to discuss your research in more depth. We may also reach out to your references, or in some cases, ask you to submit a work task of ~2-5h.
What city/country is PIBBSS based out of / where will the retreats be? (Asking as a Bay Area American without a valid passport).
We’re not based in a single location. We are open to accepting affiliates that are based in whatever place, and are also happy to help them relocate (within some constraints) to somewhere else (e.g. where they have access to a more lively research/epistemic community) if that would be beneficial for them. That said, I also think we are best placed to help people (and have historically tended to run things) in either London/Oxford, Prague or Berkeley.
If I were to be accepted for this cycle, would I be expected to attend any events in Europe? To be clear, I could attend all events in and around Berkeley.
Yes, as stated the requirements section, affiliates are expected to attend retreats and I expect about 50⁄50 of events will be happening in US/Europe.
What is the expected starting date of the first iteration of the affiliateship? (Or is it flexible?)
Starting dates might indeed differ depending on candidates’ situation. That said, we expect affiliates of this round will start sometime between mid-December and mid-January. We’ll be running a research retreat to onboard affiliates within that same time frame.