Looking at Flare made me lower my estimation of Eliezer’s technical skill, not raise it. I’m sure he’s leveled up quite a bit since, but the basic premise of the Flare project (an XML-based language) is a bad technical decision made due to a fad. Also, it never went anywhere.
I haven’t looked much at Flare myself, might you explain a little more why it’s negatively impressive? I noticed I was a little confused by your judgment, probed that confusion, and remembered that someone I’m acquainted with who I’d heard knows a lot about language design had said he was at least somewhat impressed with some aspects of Flare. Are there clever ideas in Flare that might explain that person’s positive impression but that are overall outweighed by other aspects of Flare that are negatively impressive? I’m willing to dig through Flare’s specification if you can give simple pointers.
I’m rather interested in how Eliezer’s skills and knowledges grew or diminished between 2000 and 2007. I’m really confused. According to his description his Bayesian enlightenment should have made him much stronger but his output since then has seemed weak. CFAI has horrible flaws but the perspective it exemplified is on the right track, and some of Eliezer’s OB posts hint that he still had that perspective. But the flaccidity of CEV, his apparent-to-me-and-others confusions about anthropics, his apparent overestimation of the difficulty of developing updateless-like ideas, his apparent-to-me lack of contributing to foundational progress in decision theory besides emphasizing its fundamentalness, and to some extent his involvement in the memetic trend towards “FAI good, uFAI definitely bad” all leave me wondering if he only externally dumbed things down or just internally lost steam in confusion, or something. I really really wish I knew what changed between CFAI and CEV, what his Bayesian enlightenment had to do with it, and whether or not he was perturbed by what he saw as the probable output of a CFAI-ish AGI—and if he was perturbed, what exactly he was perturbed by.
I think jimrandomh is slightly too harsh about Flare, the idea of using a pattern-matching object database as the foundation of a language rather than a bolted-on addition is at least an interesting concept. However, it seems like Eliezer focused excessively on bizarre details like supporting HTML in code comments, and having some kind of reference counting garbage collection which would be unlike anything to come before (even though the way he described it sounded pretty much exactly like the kind of reference counting GC that had been in use for decades), and generally making grandiose, highly detailed plans that were mostly impractical and/or far too ambitious for a small team to hope to implement in anything less than a few lifetimes. And then the whole thing was suddenly abandoned unfinished.
I’ve looked at the Flare docs and been similarly unimpressed. Most of that is hindsight bias—knowing that the project remained (that I’m aware of) at the vaporware stage without delivering an actual language.
Some of the proposed language features are indeed attractive; the existing language that most closely resembles it is Javascript, which shares with LambdaMOO (mentioned in the Flare docs) the interesting feature of prototype inheritance (“parenting”).
Part of the negative impression comes from the docs being a catalog of proposed features, without a clear explanation of how each of those features participates in a coherent whole; it comes across as a “kitchen sink” approach to language design. Using XML as an underlying representation scheme being the most grating instance. The docs are long on how great Flare will be but short on programs written in Flare itself illustrating how and why the things you can do with Flare would be compelling to a programmer with a particular kind of problem to solve.
To give you an idea of my qualifications (or lack thereof) for evaluating such an effort: I’m an autodidact; I’ve never designed a new language, but I have fair implementation experience. I’ve written a LambdaMOO compiler targeting the Java VM as part of a commercial project (shipped), and attempted writing a Java VM in Java (never shipped, impratical without also writing a JIT, but quite instructive). That was back in 1998. These projects required learning quite a bit about language design and implementation.
It’s harder to comment on Eliezer’s other accomplishments—I’m rather impressed by the whole conceptual framework of FAI and CEV but it’s the kind of thing to be judged by the detailed drudge work required to make it all work afterward, rather than by the grand vision itself. I’m impressed (you have to be) with the AI box experiments.
I’m impressed (you have to be) with the AI box experiments.
I am confused and a little suspicious that he did a round with Carl Shulman as gatekeeper, where Carl let him out, whereas two others did not let him out. (If I misremembered someone please correct me.) Not sure exactly what about that feels suspicious to me, though...
The last three experiments had bigger (more than 2 orders of magnitude, I think) outside cash stakes. I suspect Russell and D. Alex may have been less indifferent about that than me, i.e. I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
Which suggests to me that as soon as people actually feel a bit of real fear- rather than just role-playing- they become mostly immune to Eliezer’s charms.
With an actual boxed AI though, you probably want to let it out if it’s Friendly. It’s possibly the ultimate high stakes gamble. Certainly you have more to be afraid of than with a low stakes roleplay, but you also have a lot more to gain.
I’m impressed (you have to be) with the AI box experiments.
I’ve previously been rather scathing about those:
Those experiments are totally unscientific—and prove very little—except that some people like playing role-playing games where they act as a superintelligence, and then boast about how smart they are afterwards.
That sounds like you are trying to rouse anger, or expressing a personal dislike, but not much like an argument.
The AI-box experiments have the flavor of (and presumably are inspired by) the Turing test—you could equally have accused Turing at the time of being “unscientific” in that he had proposed an experiment that hadn’t even been performed and would not be for many years. Yes, they are a conceptual rather than a scientific experiment.
The point of the actual AI-box demonstration isn’t so much to “prove” something, in the sense of demonstrating a particular exploitable regularity of human behaviour that a putative UFAI could use to take over people’s brains over a text link (though that would be nice to have). Rather, it is that prior to the demonstration one would have assigned very little probability to the proposition “Eliezer role-playing an AI will win this bet”.
As such, I’d agree that they “prove little” but they do constitute evidence.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay? Its merit is mostly to open places in Idea Space.
Eliezer Yudkowsky likes playing role-playing games where they act as a superintelligence, and then boast about how smart he is afterwards
Honestly, so do I. Have you ever played Genius The Transgression? Look, we all know he’s full of himself, he has acknowledged this himself, it’s a flaw of his, but it’s really really irrelevant to the quality of the experiment as evidence.
Where it does matter is that that trait and his militant, sneering, condescending atheism makes for awful, godawful PR. Nevertheless, I’ve heard he is working on that, and in his rationality book he will try to use less incendiary examples than in his posts here. Still, don’t expect it to go away too soon: he strikes me the sort of man who runs largely on pride and idealism and burning shounen passion: such an attitude naturally leads to some intellectual boisterousness: the expression of these symptoms can be toned down, but as long as the cause remains, they will show up every now and then. And if that cause is also what keeps him rollin’, I wouldn’t have it any other way.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay?
Not m’kay. IIRC, it was complete junk science—an unrecorded, unverified role playing game with no witnesses.
I figure people should update about as much as they would if they were watching a Derren Brown show.
Thank you, but I could have done that myself, I meant “explain, in your own words if possible, what aspects of who this person is are relevant to the discussion”. So, he’s a Real Life version of The Mentalist. That is very cool. Why shouldn’t people get extract useful knowledge from his shows?
Er. Hmmm… Look, I don’t want to sound rude, but could you elaborate on that? As it is, your post provides me with no information at all, except for the fact that you seem to think it’s a bad idea...
Well, if you are not familiar with Derren Brown, perhaps my original comment is not for you. Derren is a magician who specialises in mind control. He has some of the skills he claims to have—but to get that across on TV is often difficult—since it is not easy to reassure the audience that you are not using stooges who are in on the tricks.
The evidence is materially better than ordinary anecdote because the fact of the experiment was published before results were available. And it’s a demonstration of reasonable possibility, not high probability. It’s n=5, but that’s materially better than nothing. In fact, taking some reasonable low probability of the human failure rate, such as 1%, the p-value is quite low as well, so it’s a statistically significant result.
I remember when Eliezer told people about the AI-box experiments he had not yet performed, and I predicted, with high confidence, that people would not “let him out of the box” and give him money; and I was wrong.
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
Eliezer said that no such trickery was involved. But, he would say that in either case.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out. As somebody not affiliated with SIAI at all, I think my somehow being persuaded would count for more evidence than, for instance Carl Shulman being persuaded. Unfortunately, because I’m not affiliated with the AI research community in general, I’m presumably not qualified to participate in an AI-box experiment.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out.
For some time now I suspect that the argument that convinced Carl Shulman and others was along the lines of acausal trade. See here, here and here. Subsequently I suspect that those who didn’t let the AI out of the box either didn’t understand the implications, haven’t had enough trust into the foundations and actuality of acausal trade, or were more like General Thud.
When Eliezer was doing them, the primary qualification was being willing to put up enough money to get Eliezer to do it. (I’m not criticizing him for this—it was a clever and interesting fundraising technique; and doing it for small sums would set a bad precedent.)
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
If he had said that to me, I would have asked what evidence there was that his putting the fear of machines into people would actually help anyone—except for him and possibly the members of his proposed “Fellowship of the AI”.
Looking at Flare made me lower my estimation of Eliezer’s technical skill, not raise it. I’m sure he’s leveled up quite a bit since, but the basic premise of the Flare project (an XML-based language) is a bad technical decision made due to a fad. Also, it never went anywhere.
I haven’t looked much at Flare myself, might you explain a little more why it’s negatively impressive? I noticed I was a little confused by your judgment, probed that confusion, and remembered that someone I’m acquainted with who I’d heard knows a lot about language design had said he was at least somewhat impressed with some aspects of Flare. Are there clever ideas in Flare that might explain that person’s positive impression but that are overall outweighed by other aspects of Flare that are negatively impressive? I’m willing to dig through Flare’s specification if you can give simple pointers.
I’m rather interested in how Eliezer’s skills and knowledges grew or diminished between 2000 and 2007. I’m really confused. According to his description his Bayesian enlightenment should have made him much stronger but his output since then has seemed weak. CFAI has horrible flaws but the perspective it exemplified is on the right track, and some of Eliezer’s OB posts hint that he still had that perspective. But the flaccidity of CEV, his apparent-to-me-and-others confusions about anthropics, his apparent overestimation of the difficulty of developing updateless-like ideas, his apparent-to-me lack of contributing to foundational progress in decision theory besides emphasizing its fundamentalness, and to some extent his involvement in the memetic trend towards “FAI good, uFAI definitely bad” all leave me wondering if he only externally dumbed things down or just internally lost steam in confusion, or something. I really really wish I knew what changed between CFAI and CEV, what his Bayesian enlightenment had to do with it, and whether or not he was perturbed by what he saw as the probable output of a CFAI-ish AGI—and if he was perturbed, what exactly he was perturbed by.
I think jimrandomh is slightly too harsh about Flare, the idea of using a pattern-matching object database as the foundation of a language rather than a bolted-on addition is at least an interesting concept. However, it seems like Eliezer focused excessively on bizarre details like supporting HTML in code comments, and having some kind of reference counting garbage collection which would be unlike anything to come before (even though the way he described it sounded pretty much exactly like the kind of reference counting GC that had been in use for decades), and generally making grandiose, highly detailed plans that were mostly impractical and/or far too ambitious for a small team to hope to implement in anything less than a few lifetimes. And then the whole thing was suddenly abandoned unfinished.
I’ve looked at the Flare docs and been similarly unimpressed. Most of that is hindsight bias—knowing that the project remained (that I’m aware of) at the vaporware stage without delivering an actual language.
Some of the proposed language features are indeed attractive; the existing language that most closely resembles it is Javascript, which shares with LambdaMOO (mentioned in the Flare docs) the interesting feature of prototype inheritance (“parenting”).
Part of the negative impression comes from the docs being a catalog of proposed features, without a clear explanation of how each of those features participates in a coherent whole; it comes across as a “kitchen sink” approach to language design. Using XML as an underlying representation scheme being the most grating instance. The docs are long on how great Flare will be but short on programs written in Flare itself illustrating how and why the things you can do with Flare would be compelling to a programmer with a particular kind of problem to solve.
To give you an idea of my qualifications (or lack thereof) for evaluating such an effort: I’m an autodidact; I’ve never designed a new language, but I have fair implementation experience. I’ve written a LambdaMOO compiler targeting the Java VM as part of a commercial project (shipped), and attempted writing a Java VM in Java (never shipped, impratical without also writing a JIT, but quite instructive). That was back in 1998. These projects required learning quite a bit about language design and implementation.
It’s harder to comment on Eliezer’s other accomplishments—I’m rather impressed by the whole conceptual framework of FAI and CEV but it’s the kind of thing to be judged by the detailed drudge work required to make it all work afterward, rather than by the grand vision itself. I’m impressed (you have to be) with the AI box experiments.
I am confused and a little suspicious that he did a round with Carl Shulman as gatekeeper, where Carl let him out, whereas two others did not let him out. (If I misremembered someone please correct me.) Not sure exactly what about that feels suspicious to me, though...
The record of AI box experiments (those involving Eliezer) is as follows:
Experiment 1, vs Nathan Russell—AI win
Experiment 2, vs David McFadzean—AI win
Experiment 3, vs Carl Shulman—AI win
Experiment 4, vs Russell Wallace—GK win
Experiment 5, vs D. Alex—GK win
The last three experiments had bigger (more than 2 orders of magnitude, I think) outside cash stakes. I suspect Russell and D. Alex may have been less indifferent about that than me, i.e. I think the record shows that Eliezer acquitted himself well with low stakes ($10, or more when the player is indifferent about the money) a few times, but failed with high stakes.
Which suggests to me that as soon as people actually feel a bit of real fear- rather than just role-playing- they become mostly immune to Eliezer’s charms.
With an actual boxed AI though, you probably want to let it out if it’s Friendly. It’s possibly the ultimate high stakes gamble. Certainly you have more to be afraid of than with a low stakes roleplay, but you also have a lot more to gain.
I’ve previously been rather scathing about those:
That sounds like you are trying to rouse anger, or expressing a personal dislike, but not much like an argument.
The AI-box experiments have the flavor of (and presumably are inspired by) the Turing test—you could equally have accused Turing at the time of being “unscientific” in that he had proposed an experiment that hadn’t even been performed and would not be for many years. Yes, they are a conceptual rather than a scientific experiment.
The point of the actual AI-box demonstration isn’t so much to “prove” something, in the sense of demonstrating a particular exploitable regularity of human behaviour that a putative UFAI could use to take over people’s brains over a text link (though that would be nice to have). Rather, it is that prior to the demonstration one would have assigned very little probability to the proposition “Eliezer role-playing an AI will win this bet”.
As such, I’d agree that they “prove little” but they do constitute evidence.
They constitute anecdotal evidence. Such evidence is usually considered to be pretty low-grade by scientists.
LOL, yes, that’s why it weights little. But, see, it still gets to considerably shift one’s expectations on the matter because it had a very low probability assigned to its happening, as per Conservation Of Expected Evidence. Let’s just say it counts as Rational Evidence, m’kay? Its merit is mostly to open places in Idea Space.
Honestly, so do I. Have you ever played Genius The Transgression? Look, we all know he’s full of himself, he has acknowledged this himself, it’s a flaw of his, but it’s really really irrelevant to the quality of the experiment as evidence.
Where it does matter is that that trait and his militant, sneering, condescending atheism makes for awful, godawful PR. Nevertheless, I’ve heard he is working on that, and in his rationality book he will try to use less incendiary examples than in his posts here. Still, don’t expect it to go away too soon: he strikes me the sort of man who runs largely on pride and idealism and burning shounen passion: such an attitude naturally leads to some intellectual boisterousness: the expression of these symptoms can be toned down, but as long as the cause remains, they will show up every now and then. And if that cause is also what keeps him rollin’, I wouldn’t have it any other way.
Not m’kay. IIRC, it was complete junk science—an unrecorded, unverified role playing game with no witnesses.
I figure people should update about as much as they would if they were watching a Derren Brown show.
Who’s Derren Brown?
Derren Brown
Thank you, but I could have done that myself, I meant “explain, in your own words if possible, what aspects of who this person is are relevant to the discussion”. So, he’s a Real Life version of The Mentalist. That is very cool. Why shouldn’t people get extract useful knowledge from his shows?
Well, go right ahead—if you have not encountered this sort of thing before.
Er. Hmmm… Look, I don’t want to sound rude, but could you elaborate on that? As it is, your post provides me with no information at all, except for the fact that you seem to think it’s a bad idea...
Well, if you are not familiar with Derren Brown, perhaps my original comment is not for you. Derren is a magician who specialises in mind control. He has some of the skills he claims to have—but to get that across on TV is often difficult—since it is not easy to reassure the audience that you are not using stooges who are in on the tricks.
Oh! Right. Actually that’s a very apt comparison!
The evidence is materially better than ordinary anecdote because the fact of the experiment was published before results were available. And it’s a demonstration of reasonable possibility, not high probability. It’s n=5, but that’s materially better than nothing. In fact, taking some reasonable low probability of the human failure rate, such as 1%, the p-value is quite low as well, so it’s a statistically significant result.
I remember when Eliezer told people about the AI-box experiments he had not yet performed, and I predicted, with high confidence, that people would not “let him out of the box” and give him money; and I was wrong.
I still wonder if the conversations went something like this:
“If we say you let me out of the box, then people will take the risk of AI more seriously, possibly saving the world.”
“Oh. Okay, then.”
Eliezer said that no such trickery was involved. But, he would say that in either case.
I wouldn’t be persuaded to “let the AI out” by that argument. In fact, even after reading about the AI box experiments I still can’t imagine any argument that would convince me to let the AI out. As somebody not affiliated with SIAI at all, I think my somehow being persuaded would count for more evidence than, for instance Carl Shulman being persuaded. Unfortunately, because I’m not affiliated with the AI research community in general, I’m presumably not qualified to participate in an AI-box experiment.
For some time now I suspect that the argument that convinced Carl Shulman and others was along the lines of acausal trade. See here, here and here. Subsequently I suspect that those who didn’t let the AI out of the box either didn’t understand the implications, haven’t had enough trust into the foundations and actuality of acausal trade, or were more like General Thud.
When Eliezer was doing them, the primary qualification was being willing to put up enough money to get Eliezer to do it. (I’m not criticizing him for this—it was a clever and interesting fundraising technique; and doing it for small sums would set a bad precedent.)
If he had said that to me, I would have asked what evidence there was that his putting the fear of machines into people would actually help anyone—except for him and possibly the members of his proposed “Fellowship of the AI”.
Why are you sure he’s leveled up quite a bit since then? Something about his Bayesian enlightenment, or TDT, or other hints?