Recently someone either suggested to me (or maybe told me they or someone where going to do this?) that we should train AI on legal texts, to teach it human values. Ignoring the technical problem of how to do this, I’m pretty sure legal text are not the right training data. But at the time, I could not clearly put into words why. Todays SMBC explains this for me:
Law is not a good representation or explanation of most of what we care about, because it’s not trying to be. Law is mainly focused on the contentious edge cases.
Training an AI on trolly problems and other ethical dilemmas is even worse, for the same reason.
Moreover, legal texts are not super strict (much is left to interpretation) and we are often selective about “whether it makes sense to apply this law in this context” for reasons not very different from religious people being very selective about following the laws of their holy books.
I spoke with some people last fall who were planning to do this, perhaps it’s the same people. I think the idea (at least, as stated) was to commercialize regulatory software to fund some alignment work. At the time, they were going by Nomos AI, and it looks like they’ve since renamed to Norm AI.
Soon, interacting with AI agents will be a part of daily life, presenting enormous regulatory and compliance challenges alongside incredible opportunities.
Norm Ai agents also work alongside other AI agents who have been entrusted to automate business processes. Here, the role of the Norm Ai agent is to automatically ensure that actions other AI agents take are in compliance with laws.
I’m not sure if this is worrying, because I don’t think AI overseeing AI is a good solution. Or it’s actually good, because, again, not a good solution, which might lead to some early warnings?
Would sensationalist tabloid news stories be better training data? Perhaps it is the inverse problem: fluffy human interest stories and outrage porn are both engineered for the lowest common denominator, the things that overwhelmingly people think are heartwarming or miscarriages of justice respectively. However if you wanted to get a AI to internalize what is in fact the sources of outrage and consensus among the wider community I think it’s a place to start.
The obvious other examples are fairy tales, fables, parables, jokes, and urban legends—most are purpose encoded with a given society’s values. Amateur book and film reviews are potentially another source of material that displays human values in that whether someone is satisfied with the ending or not (did the villain get punished? did the protagonist get justice?) or which characters they liked or disliked is often attached to the reader/viewer’s value systems. Or as Jerry Lewis put it in the Total Filmmaker: in comedy, a snowball is never thrown at a battered fedora: “The top-hat owner is always the bank president who holds mortgage on the house...”.
Sensationalist tabloid news stories and other outrage porn are not the opposite. These are actually more of the same. More edge cases. Anything that is divisive have the problem I’m talking about.
Fiction is a better choice.
Or even just completely ordinary every-day human behaviour. Most humans are mostly nice most of the time.
We might have to start with the very basic, the stuff we don’t even notice, because it’s too obvious. Things no-one would think of writing down.
Sensationalist tabloid news stories and other outrage porn are not the opposite. These are actually more of the same. More edge cases. Anything that is divisive have the problem I’m talking about.
Could you explain how are they edge cases if they are the lowest common denominator? Doesn’t that make them the opposite of an edge case? Aren’t they in fact the standard or yardstick necessary to compare against?
Fiction is a better choice.
Why is is it different let alone better choice? Fiction is a single author’s attempt to express their view of the world, including morality, and therefore an edge case. While popular literature is just as common denominator as tabloid journalism, since the author is trying to be commercial.
I don’t read much sensationalist tabloid, but my impression is that the things that get a lot of attention in the press, is things people can reasonable take either side of.
Scott Alexander writes about how everyone agrees that factory framing is terrible, but exactly because this overwhelming agreement, it get’s no attention. Which is why PETA does outrageous things to get attention.
I don’t read much sensationalist tabloid, but my impression is that the things that get a lot of attention in the press, is things people can reasonable take either side of.
A cursory glance suggests that it is not the case, take a top story headline on the Australian Daily Mail over the last 7 days: “Miranda, Sydney: Urgent search is launched for missing Bailey Wolf, aged two, who vanished yesterday” it is not reasonable for someone to hope that a two year old who has vanished not be found. This is exactly the kind of thing you’re suggesting AI should be trained on, because of how uniform responses are to this headline. Keep in mind this is one of the most viewed stories, and literally top of the list I found.
I’ve read Scott’s article, but are you trying to understand what get’s attention or what is the nexus or commonly agreed upon moral principles of a society?
Recently someone either suggested to me (or maybe told me they or someone where going to do this?) that we should train AI on legal texts, to teach it human values. Ignoring the technical problem of how to do this, I’m pretty sure legal text are not the right training data. But at the time, I could not clearly put into words why. Todays SMBC explains this for me:
Saturday Morning Breakfast Cereal—Law (smbc-comics.com)
Law is not a good representation or explanation of most of what we care about, because it’s not trying to be. Law is mainly focused on the contentious edge cases.
Training an AI on trolly problems and other ethical dilemmas is even worse, for the same reason.
Moreover, legal texts are not super strict (much is left to interpretation) and we are often selective about “whether it makes sense to apply this law in this context” for reasons not very different from religious people being very selective about following the laws of their holy books.
I spoke with some people last fall who were planning to do this, perhaps it’s the same people. I think the idea (at least, as stated) was to commercialize regulatory software to fund some alignment work. At the time, they were going by Nomos AI, and it looks like they’ve since renamed to Norm AI.
I found this on their website
I’m not sure if this is worrying, because I don’t think AI overseeing AI is a good solution. Or it’s actually good, because, again, not a good solution, which might lead to some early warnings?
Their more human-in-the-loop stuff seems neat though.
Would sensationalist tabloid news stories be better training data? Perhaps it is the inverse problem: fluffy human interest stories and outrage porn are both engineered for the lowest common denominator, the things that overwhelmingly people think are heartwarming or miscarriages of justice respectively. However if you wanted to get a AI to internalize what is in fact the sources of outrage and consensus among the wider community I think it’s a place to start.
The obvious other examples are fairy tales, fables, parables, jokes, and urban legends—most are purpose encoded with a given society’s values. Amateur book and film reviews are potentially another source of material that displays human values in that whether someone is satisfied with the ending or not (did the villain get punished? did the protagonist get justice?) or which characters they liked or disliked is often attached to the reader/viewer’s value systems. Or as Jerry Lewis put it in the Total Filmmaker: in comedy, a snowball is never thrown at a battered fedora: “The top-hat owner is always the bank president who holds mortgage on the house...”.
Sensationalist tabloid news stories and other outrage porn are not the opposite. These are actually more of the same. More edge cases. Anything that is divisive have the problem I’m talking about.
Fiction is a better choice.
Or even just completely ordinary every-day human behaviour. Most humans are mostly nice most of the time.
We might have to start with the very basic, the stuff we don’t even notice, because it’s too obvious. Things no-one would think of writing down.
Could you explain how are they edge cases if they are the lowest common denominator? Doesn’t that make them the opposite of an edge case? Aren’t they in fact the standard or yardstick necessary to compare against?
Why is is it different let alone better choice? Fiction is a single author’s attempt to express their view of the world, including morality, and therefore an edge case. While popular literature is just as common denominator as tabloid journalism, since the author is trying to be commercial.
I don’t read much sensationalist tabloid, but my impression is that the things that get a lot of attention in the press, is things people can reasonable take either side of.
Scott Alexander writes about how everyone agrees that factory framing is terrible, but exactly because this overwhelming agreement, it get’s no attention. Which is why PETA does outrageous things to get attention.
The Toxoplasma Of Rage | Slate Star Codex
There need to be two sides to an issue, or else no-one gets ingroup loyalty points for taking one side or the other.
A cursory glance suggests that it is not the case, take a top story headline on the Australian Daily Mail over the last 7 days: “Miranda, Sydney: Urgent search is launched for missing Bailey Wolf, aged two, who vanished yesterday” it is not reasonable for someone to hope that a two year old who has vanished not be found. This is exactly the kind of thing you’re suggesting AI should be trained on, because of how uniform responses are to this headline. Keep in mind this is one of the most viewed stories, and literally top of the list I found.
I’ve read Scott’s article, but are you trying to understand what get’s attention or what is the nexus or commonly agreed upon moral principles of a society?
Ok, you’re right that this is a very morally clear story. My bad for not knowing what’s typical tabloid storry.
Missing kid = bad,
seems like a good lesson for AI to learn.