The quote was dashed by the poster.
Kawoomba
Karma: 5,169
Contains a lot of guesstimates though, which it freely admits throughout the text (in the abstract, not so much). It’s a bit like a very tentative Fermi estimate.
I truly am torn on the matter. LW has caused a good amount of self-modification away from that position, not in the sense of diminishing the arguments’ credence, but in the sense of “so what, that’s not the belief I want to hold” (which, while generally quite dangerous, may be necessary with a few select “holy belief cows”)*.
That personal information notwithstanding, I don’t think we should only present arguments supporting positions we are convinced of. That—given a somewhat homogeneous group composition—would amount to an echo chamber, and in any case knock out Aumann’s agreement theorem.
* Ironic, is it not? Analogous to “shut up and do the impossible” a case of instrumental versus epistemic rationality.
There are analogues of the classic biases in our own utility functions, it is a blind spot to hold our preferences as we perceive them to be sacrosanct. Just as we can be mistaken about the correct solution to Monty Hall, so can we be mistaken about our own values. It’s a treasure trove for rational self-analysis.
We have an easy enough time of figuring out how a religious belief is blatantly ridiculous because we find some claim it makes that’s contrary to the evidence. But say someone takes out all such obviously false claims, or take a patriot, someone who professes to just deeply care about his country, or his bat mitzvah, or her white wedding, or what have you. Even then, there is more cognitive exploration to be had there than just shrugging and saying “can’t argue with his/her utility function”.
The quote does some work in that direction. From a certain point of view, altruism is the last, most persistent bias. Far from “there is a light in the world, and we are it”—rather the final glowing ember on the bonfire of irrationality. But that’s a long post in and of itself. Shrug, if you don’t see it as a rationality quote, just downvote it.
Suppose now that there were two such magic [invisibility] rings [of Gyges], and the just put on one of them and the unjust the other; no man can be imagined to be of such an iron nature that he would stand fast in justice. No man would keep his hands off what was not his own when he could safely take what he liked out of the market, or go into houses and lie with any one at his pleasure, or kill or release from prison whom he would, and in all respects be like a god among men.
Then the actions of the just would be as the actions of the unjust; they would both come at last to the same point. And this we may truly affirm to be a great proof that a man is just, not willingly or because he thinks that justice is any good to him individually, but of necessity, for wherever any one thinks that he can safely be unjust, there he is unjust.
For all men believe in their hearts that injustice is far more profitable to the individual than justice, and he who argues as I have been supposing, will say that they are right. If you could imagine any one obtaining this power of becoming invisible, and never doing any wrong or touching what was another’s, he would be thought by the lookers-on to be a most wretched idiot, although they would praise him to one another’s faces, and keep up appearances with one another from a fear that they too might suffer injustice.
Glaucon, in Plato’s Republic
Is it ok to call people poopy-heads, but in a mature and intelligent manner?
Signs and portents …
Don’t knock it ’til you try it.
Truth had never been a priority. If believing a lie kept the genes proliferating, the system would believe that lie with all its heart.
Peter Watts, Echopraxia, on altruism. Well ok, I admit, not on altruism per se.
It is simply unfathomable to me how you come to the logical conclusion that an UFAI will automatically and instantly and undetectably work to bypass and subvert its operators. Maybe that’s true of a hypothetical unbounded universal inference engine, like AIXI. But real AIs behave in ways quite different from that extreme, alien hypothetical intelligence.
Well, it follows pretty straightforwardly from point 6 (“AIs will want to acquire resources and use them efficiently”) of Omohundro’s The Basic AI Drives, given that the AI would prefer to act in a way conducive to securing human cooperation. We’d probably agree that such goal-camouflage would be what an AI would attempt above a certain intelligence-threshold. The difference seems to be that you say that threshold is so high as to practically only apply to “hypothetical unbounded universal inference engines”, not “real AIs”. Of course, your “undetectably” requirement does a lot of work in raising the required threshold, though “likely not to be detected in practice” translates to something different than, say, “assured undetectability”.
The softer the take-off (plus, the lower the initial starting point in terms of intelligence), the more likely your interpretation would pan out. The harder the take-off (plus, the higher the initial starting point in terms of intelligence), the more likely So8res’ predicted AI behavior would be to occur. Take-off scenarios aren’t mutually exclusive. On the contrary, the probable temporal precedence of the advent of slow-take-off AI with rather predictable behavior could lull us into a sense of security, not expecting its slightly more intelligent cousin, taking off just hard enough, and/or unsupervised enough, that it learns to lie to us (and since we’d group it with the reference class of CogSci-like docile AI, staying undetected may not be as hard as it would have been for the first AI).
So which is it?
Both, considering the task sure seems hard from a human vantage point, and by definition will seem easy from a sufficiently intelligent agent’s.
Well, seems like a good enough test of mod response times.
Expected by whom?
Even if you didn’t personally steal a Native’s land, acknowledging your Western privilege means not forgetting that your current standard of life is partly dependent on a historic crime.
No more than your existence depending on some paternal ancestor raping some maternal ancestor, which stochastically also happened. Being neither a believer in kin liability*, and skeptical at best about collective guilt (for past events, no less), why should I—or you, or anyone—feel responsible?
(As an aside, just for the hypothetical: The Natives that were displaced could well be those tribes who previously themselves successfully displaced/replaced other tribes, no? Back the guilt ball rolls, to the first microbe. At least it can’t be triggered, not having a brain and all. Then again, that’s no protection for Tumblristas either.)
* Excluding otherkin liability, because otherkin are the epitome of what’s wrong with the world. When anything wrong happens somewhere in the world, the closest otherkin should be put on a public show-trial, incarcerated and/or have his/her rotary blades removed.
This reminds me of the HPMOR chapter in which Harry tests Hermione’s hypothesis checking skills. What you’re telling me is just what you’d expect if people were easily convinced of pretty much anything (with some caveats, admittedly), given social capital and social investment (which you have mentioned in your initial explanation). You have a specialized mechanism-of-action to explain your apparent success, one which indeed may not be easily adaptable to other ventures.
The problem is that it doesn’t explain the ubiquitous occurrence of people being convinced of pretty much any topic you could imagine (requiring more specialized theories). From organized religion, cults, homeopathy, nationalism, anti-nationalism, consumerism, anti-consumerism, big weddings, small weddings, no weddings, monogamy, polygamy, psychic energy, Keynesianism, Austrian economics, rationality, anti-rationality, the list goes on. It doesn’t matter that some of these happen to be correct when their exact opposite is also on the list, with plenty of adherents.
You have the anecdote, but looking at the human condition, I see plenty of data to the opposite. Though if you interpret that differently, please share.
I’m not satisfied if they agree today and disagree tomorrow.
Do you think that when (some time after your rationality talk with them) they display a bias in a real life situation, and you kindly make them notice that they did, that they’ll agree and have learned a lesson? It’s all good as long as it’s in the abstract, and a friendly guy who is sharing a cool topic he’s passionate about.
Which is good in a way, because just as the first rationality talk doesn’t stick, neither does the first e.g. “convert to your girlfriend’s religion”, usually.
Also, what, in your opinion, are the relative weights you’d ascribe to your success, in terms of “social investment / social strategy” versus your System 2/System 1 approach?
I would be interested in you actually trying the real, falsifying experiment: convincing someone of something obviously false (to you). It’s not hard, in the general case. Though, as you say, in recent years it has become slightly harder in some ways, easier in others: Far from creating one shared space, today’s interconnectivity seems to have led to a bunch of echo-chamber bubbles, even if Wikipedia is a hopeful sign.
Then again, Wikipedia exists. As does the multi-billion (and growing) homeopathy market.
You don’t think “convincing someone that homeopathy works” is controversial enough? Are you objecting to both political and non-political beliefs, and wouldn’t that make the initial claim, you know, unfalsifiable?
For reference, the initial mention was:
I find it much easier to change people’s minds about rationality than about, say, the NSA.
Take a non-political wrong belief then. Same applies to selling sugar pills, I’m sorry, homeopathy. At least some people are earning billions with it.
Also, guarded statements such as “political beliefs can reflect negatively (...) depending on their social circle” are as close to null statements as you can reasonably get. I could substitute “can help you become an astronaut” and the statement would be correct.
I’m not sure who in their right mind would argue against “can … under certain circumstances …”-type social statements. It’s good to qualify our statements and to hedge our bets, wary of blanket generalizations, but at some point we need to stop and make some sort of stand, or we’re doing the equivalent of throwing empty chat bubbles at each other.
So that makes it worth an investment of bit of focused discussion. Do you disagree?
It would be a rookie mistake to disagree with “this has value to me, so it is worth an investment”, which is nearly tautological (barring fringe cases).
What I disagree with is “I find it much easier to change people’s minds about rationality than about, say, the NSA.” If you’re at the right hierarchical spot on the totem pole and do all the right social signals, you can convince people of nearly anything, excepting things that could reflect negatively on themselves (such as rationality)*. The latter is still possible, but much harder than many of the bullshit claims which happen to reflect positively on someone’s ego. That you happen to be right about rationality is just, you know, a happy coincidence from their point of view.
If you’re up for it, I dare you to go out and convince someone of average reasoning skills of some bullshit proposition, using the same effort and enthusiasm you display for rationality. Then after that beware of rethinking your life and becoming a used car salesman.
* There is a special kind of shtick where the opposite applies, namely the whole religious “original sin, you’re not worthy, now KNEEL, MAGGOT!”-technique. Though that’s been losing its relevance, now that everyone’s a special snowflake. New age, new selling tactics.
#Civilizationalinadequacy
Every time I’ve invested an hour or two discussing some rationality topic with a newbie, they always came out (claiming to be) convinced. Of course I put in some work: I establish good rapport and compatible values, I adapt it to what they care about, I show them how they can use rationality to win—sure that helps. But rationality simply makes sense. I find it much easier to change people’s minds about rationality than about, say, the NSA.
The relevant data isn’t “I can convince people if I use all the social tricks in the book”, it is instead “how much easier is it to convince them of their cognitive biases than of XYZ”. Which makes it come down to this:
I find it much easier to change people’s minds about rationality than about, say, the NSA.
I would bet on the exact opposite. Those guys make sure your family are safe, done. What’s the other guy doing, still talking about goats and doors, isn’t he just saying he’s the smarter thinker (stupid elitist)? Well, I’m talking about knowing the people you love are better protected, having one less thing to worry about. Here are some pictures of people killed by the bad guys. Aren’t you glad we deal with that stuff, for you? Thought so.
(...) the people can always be brought to the bidding of the leaders. That is easy. All you have to do is tell them they are being attacked and denounce the peacemakers for lack of patriotism and exposing the country to danger. It works the same in every country.
Yes; your paraphrasing about covers it. Nicely done, if I may say so. Let me reemphasize that it was a minor point overall, but still one I thought worth mentioning (in passing), if only in a half-sentence.
I meant to say parentheses and just confused them with brackets (not a native speaker, or writer, for that matter). The point only being that a post in a “meta content—subject level content—meta content” format which sandwiches your important content in between meta remarks loses some of its saliency, parentheses or no.
You are doing fine, all the aspects we’re discussing are minor nitpicks. There is no need to worry about the correct tags, or even to overly fret about the amount of meta that’s along for the ride. Insight trumps in-group signalling. My remarks were about on the same order of importance as advising a really long post to include a “tl;dr” summary at the end. I often ready mostly the beginning of a post and the conclusion, to judge whether the rest is likely to be worth the time. In your case, that had the somewhat funny result of wondering what the hell your title was referring to, since all I saw was meta, hyperbolically speaking. So I read more of the middle parts to supposedly fill in the gap, imagine my surprise when I encountered a thoughtful and interesting analysis in there. So while it was my laziness more so than any fault on your part, that’s why I brought it up.
tl;dr: Your post is fine, now go write new posts.
The whole AI box experiment is a fun pastime, and educational in so far as learning to take artificial intellects seriously, but as real-world long-term “solutions” go, it is utterly useless. Like trying to contain nuclear weapons indefinitely, except you can build one just by having the blueprints and a couple leased hours on a supercomputer, no limited natural elements necessary, and having one means you win at whatever you desire (or that’s what you’d think). All the while under the increasing pressure of improving technology, ever lowering the threshold to catastrophe. When have humans abstained from playing with the biggest fire they can find?
The best case scenario for AI boxing would be that people aware of the risks (unlikely because of motivated cognition) are the first to create an AGI (not just stumbling upon one, either) and use their first-mover advantage to box the AI just long enough (just having a few months would be lucky) to poke and prod it until they’re satisfied it’s mostly safe (“mostly” because whatever predicates the code fulfills, there remains the fundamental epsilon of insecurity of whether the map actually reflects the territory).
There are so many state actors, so many irresponsible parties involved in our sociological ecosystem, with so many chances of taking a wrong step, so many biological imperatives counter to success*, that (coming full circle to my very first comment on LW years ago) the whole endeavor seems like a fool’s hope, and that only works out in Lord of the Rings.
But, as the sentient goo transforms us into beautiful paperclips, it’s nice to know that at least you tried. And just maybe we get lucky enough that the whole take-off is just slow enough, or wonky enough, for the safe design insights to matter in some meaningful sense, after all.
* E.g. one AGI researcher defecting with the design to another group (which is also claiming to have a secure AI box / some other solution) would be a billionaire for the rest of his life, that being measured in weeks most likely. Such an easy lie to make to yourself. And that isn’t if a relevant government agency doesn’t even have to ask to get your designs, if anyone of reputation tipped them off, or they followed the relevant conferences (nooo, would they do that?).