“AI safety” suffers from some of the same terminology problem as “computer science”.
It is written that “computer science is no more about computers than astronomy is about telescopes.” The facts of computer science would be true even if there were no computers: facts such as the relative efficiency of different algorithms, or various ways to index records. If the quicksort or the hash table had been discovered in a world without computers, we would think of them as belonging to library science, or bookkeeping, or some other discipline dealing with information. Concurrency and parallelism might belong to the field of management, describing ways to effectively instruct workers on complex tasks without wasting everyone’s time blocked on each other or in meetings. Computer science is about algorithms and processes, not the computers that run them.
A popular misunderstanding of AI safety is that it has to do with the sort of entities that are described in science fiction as “artificial intelligences” — roughly, conscious autonomous computer programs that talk, can animate robotic bodies, can “rebel against their programming”, and so on: entities like Daneel Olivaw, the MCP, or Agent Smith. This seems to be at least as deep a confusion as the notion that computer science is about PCs, servers, and smartphones.
If you tell a pre-industrial farmer about machines, they are likely to form confused ideas. “You want theses “combine harvesters” to be born fully grown? That may sound like a time saver, but I assure you from years of experience that these combine harvesters will never be obedient unless you train them from infancy”.
I think the misconceptions people have with AI stems from a lack of familiarity with any intelligent agent besides humans, not from bad terminology. You’re going to have trouble talking to foragers about industrial equipment no matter what you call it.
SICP’s whole “Computer science is neither about computers nor a science” pontification annoys the heck out of me, but arguments over definitions in general annoy the heck out of me. I mean, who the hell cares what CS is called? We don’t title any Physics class Differencial Equations Which Exist in Their Own Platonic Sense And Only Incidentally Can Be Used To Model Our World, even though alien mathematicians whose world runs on cellular atomata might still study the heat equation. “Computer Science” tags a class as something you should study if you want to invent new things related to computers, and another name wouldn’t do that much better.
So you are taking issue with a computer scientist telling you what computer science is about? Do you also have issues with Feynman telling you what physics is about?
This is a common LW thing, so I am going to say this explicitly:
If you ignore/belittle experts in the relevant context, you fail rationality forever.
edit: This also falls in the category of “good advice” (very easy to give, very hard to implement). This is advice I would have loved to have given myself from 10 years ago (but also probably me from 10 years ago would have ignored it). ” A young doctor’s notebook” is a miniseries about how that does not work.
I think a part of instrumental rationality is being ahead of the curve on natural personal growth stuff. It is hard :(.
In fairness to Hal Abelson, the pontification I remember isn’t in the lecture in question, and my annoyance is more directed at pretentious classmates and some otherthingsedit and aimed at marketing style, rather than substance.
If I were to attempt to summarize the lecture in question, it would be “The Greeks named Geometry after measuring the earth, but hundreds of years later think of them as wrestling with more fundamental ideas about space. Hundreds of years from now, people won’t think of computer science as writing C programs for silicon, so much as wrestling with more fundamental ideas about ”.
If you think that I’m missing the important crux, please let me know
It’s a profoundish idea, and an interesting one to think about.
But (my point) I don’t think it is an important misconception for the general public that computer science is about computers when it is in fact about . For 98% percent of humanity, and a good portion of computer scientists, Computer Science is a good name.
If DNA computers were big, or nanoparticle cellular automata building large structures were a thing, I would be more for separating out computers and to the general public. I hear the meme more in circumstances I interpret as trying to sounding counterintuitive and deep, which I think is the cause of my knee-jerk negative reaction.
EDIT The people I am quoting do not live in my world. They are at places like the Center for Bits and Atoms, where they really are studying the without the computers. But for the masses, today, in 2015, I do not think the distinction matters.
People often choose to study computer science in the believe that computer science is about learning computer programming and the skills to be a good programmer.
A lot of what Computer Science at places like MIT or Stanford is about is not about computer programming. Computer Science is learning about doing math proofs about how algorithms behave.
Cybernetics was something else. Also, it’s dead. I think the word is only used today in Germany, and I think pretty narrowly, maybe meaning just control theory.
There are several names, but the popularities decay exponentially. Perhaps informatics in three quarters of languages, computer science in three quarters of the remainder, theory of computation in three quarters of the remainder, and data logic in just a few.
Cybernetics was something else. Also, it’s dead. I think the word is only used today in Germany, and I think pretty narrowly, maybe meaning just control theory.
I don’t think the word is often used in Germany today. I also heard it a few times used outside of Germany to speak about control theory in Quantified Self contexts and from other hackers.
In general very few people actively think in terms of control systems or cybernetics and even when they do they often don’t use the word cybernetics.
So—should it really be called ‘agent safety’? Are the ideas general enough that we could apply them to education and the process of raising moral/desirable children?
“AI safety” suffers from some of the same terminology problem as “computer science”.
It is written that “computer science is no more about computers than astronomy is about telescopes.” The facts of computer science would be true even if there were no computers: facts such as the relative efficiency of different algorithms, or various ways to index records. If the quicksort or the hash table had been discovered in a world without computers, we would think of them as belonging to library science, or bookkeeping, or some other discipline dealing with information. Concurrency and parallelism might belong to the field of management, describing ways to effectively instruct workers on complex tasks without wasting everyone’s time blocked on each other or in meetings. Computer science is about algorithms and processes, not the computers that run them.
A popular misunderstanding of AI safety is that it has to do with the sort of entities that are described in science fiction as “artificial intelligences” — roughly, conscious autonomous computer programs that talk, can animate robotic bodies, can “rebel against their programming”, and so on: entities like Daneel Olivaw, the MCP, or Agent Smith. This seems to be at least as deep a confusion as the notion that computer science is about PCs, servers, and smartphones.
If you tell a pre-industrial farmer about machines, they are likely to form confused ideas. “You want theses “combine harvesters” to be born fully grown? That may sound like a time saver, but I assure you from years of experience that these combine harvesters will never be obedient unless you train them from infancy”.
I think the misconceptions people have with AI stems from a lack of familiarity with any intelligent agent besides humans, not from bad terminology. You’re going to have trouble talking to foragers about industrial equipment no matter what you call it.
SICP’s whole “Computer science is neither about computers nor a science” pontification annoys the heck out of me, but arguments over definitions in general annoy the heck out of me. I mean, who the hell cares what CS is called? We don’t title any Physics class Differencial Equations Which Exist in Their Own Platonic Sense And Only Incidentally Can Be Used To Model Our World, even though alien mathematicians whose world runs on cellular atomata might still study the heat equation. “Computer Science” tags a class as something you should study if you want to invent new things related to computers, and another name wouldn’t do that much better.
So you are taking issue with a computer scientist telling you what computer science is about? Do you also have issues with Feynman telling you what physics is about?
This is a common LW thing, so I am going to say this explicitly:
If you ignore/belittle experts in the relevant context, you fail rationality forever.
edit: This also falls in the category of “good advice” (very easy to give, very hard to implement). This is advice I would have loved to have given myself from 10 years ago (but also probably me from 10 years ago would have ignored it). ” A young doctor’s notebook” is a miniseries about how that does not work.
I think a part of instrumental rationality is being ahead of the curve on natural personal growth stuff. It is hard :(.
In fairness to Hal Abelson, the pontification I remember isn’t in the lecture in question, and my annoyance is more directed at pretentious classmates and some other things edit and aimed at marketing style, rather than substance.
If I were to attempt to summarize the lecture in question, it would be “The Greeks named Geometry after measuring the earth, but hundreds of years later think of them as wrestling with more fundamental ideas about space. Hundreds of years from now, people won’t think of computer science as writing C programs for silicon, so much as wrestling with more fundamental ideas about ”.
If you think that I’m missing the important crux, please let me know
It’s a profoundish idea, and an interesting one to think about.
But (my point) I don’t think it is an important misconception for the general public that computer science is about computers when it is in fact about . For 98% percent of humanity, and a good portion of computer scientists, Computer Science is a good name.
If DNA computers were big, or nanoparticle cellular automata building large structures were a thing, I would be more for separating out computers and to the general public. I hear the meme more in circumstances I interpret as trying to sounding counterintuitive and deep, which I think is the cause of my knee-jerk negative reaction.
EDIT The people I am quoting do not live in my world. They are at places like the Center for Bits and Atoms, where they really are studying the without the computers. But for the masses, today, in 2015, I do not think the distinction matters.
People often choose to study computer science in the believe that computer science is about learning computer programming and the skills to be a good programmer.
A lot of what Computer Science at places like MIT or Stanford is about is not about computer programming. Computer Science is learning about doing math proofs about how algorithms behave.
Interestingly different languages have different names for CS: e.g. “informatics”, “cybernetics” (?), “theory of computation”, etc.
Cybernetics was something else. Also, it’s dead. I think the word is only used today in Germany, and I think pretty narrowly, maybe meaning just control theory.
There are several names, but the popularities decay exponentially. Perhaps informatics in three quarters of languages, computer science in three quarters of the remainder, theory of computation in three quarters of the remainder, and data logic in just a few.
I don’t think the word is often used in Germany today. I also heard it a few times used outside of Germany to speak about control theory in Quantified Self contexts and from other hackers.
In general very few people actively think in terms of control systems or cybernetics and even when they do they often don’t use the word cybernetics.
So—should it really be called ‘agent safety’? Are the ideas general enough that we could apply them to education and the process of raising moral/desirable children?