Thanks for sharing! If I had a penny for every article that—in hindsight—would have taken me 10% of the time/effort to write … lol
catubc
Karma: 159
Thanks for the great post. 2-meta questions.
How long did it take you to write this? I work in academia and am curious to know how such a piece of writing relates to writing an opinion piece on my planet.
Is there a video and/or Q&A at some point (forgive me if I missed it).
Hi TAG, indeed, the post was missing some clarifications. I added a bit more about free will to the text, I hope it’s helpful.
Hi Charlie. Thanks for the welcome!
Indeed, I think that’s a great way to put it “preserving human agency around powerful systems” (I added it to the article). Thanks for that! I am pessimistic that this is possible (or that the question makes sense as it stands). I guess what I tried to do above—was a soft argument that “intent-aligned AIs” might not make sense without further limits or boundaries on both human intent and what AIs can do.
I agree hard wiring is probably not the best solution. However, humans are probably hardwired with a bunch of tools. Post-behaviorist psychology, e.g. self-determination-theory, argues that agency and belonging (for social organisms) are hard wired in most complex organisms (i.e. not learned). (I assume you know some/a lot about this, here’s a very rough write up https://practicalpie.com/self-determination-theory/).
Thanks shminux. My apologies for the confusion, part of my point was that we don’t have consensus on whether we have free will (the professional philosophers usually fall into ~60% compatibilists; but the sociologists have a different conception altogether; and the physicists etc.). I think this got lost because I was not trying to explain the philosophical position on free will. [I have added a very brief note in the main text to clarify what I think of as the “free will problem”].
The rest of the post was an attempt to argue that because human action is likely caused by many factors, AI techs will likely help us uncover the role of these factors; and that AI-misuse or even AI-aligned agents may act to change human will/intent etc.
Re: whether bacteria/fish/cats etc have free will: I propose we’ll have better answers soon (if you consider AGI is coming soon-ish). Or more precisely, the huge body of philosophical, psychological and sociological ideas will be tested against some empirical findings in this field. I actually work on these type of questions from the empirical side (neuroscience and open behavior). And—to summarize this entire post in 1 sentence—I am concerned that most organisms including humans have quite predictable behaviors (given behavior + other internal state data) and that these entire causal networks will be constantly under pressure by both nefarious—but also well-meaning agents like (aligned-AIs) because of the inherently complex nature of behavior.
Great post Peter. I think a lot about whether it even makes sense to use the term “aligned AGI” as powerfull AGIs may break human intention for a number of reasons (https://www.lesswrong.com/posts/3broJA5XpBwDbjsYb/agency-engineering-is-ai-alignment-to-human-intent-enough).
I see you didn’t refer to AIs become self driven (as in Omohundro: https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf). Is there a reason you don’t view this as part of the college kid problem?