SilasBarta

Karma: 9,460

SilasBarta 24 Feb 2009 1:05 UTC
1 point
on: On Not Having an Advance Abyssal Plan
Correct me if I’m wrong, but wasn’t the problem that the government’s economic “experts” seriously believed that the worst possible thing that could happen was that the system needed a “liquidity” injection?

Well, now the major institutions are clamoring for “liquidity”, but that has by now become a euphemism for “free money in the form of a loan that no sane person expects to be paid back.”

SilasBarta 1 Mar 2009 0:10 UTC
3 points
in reply to: Emile’s comment on: The Most Important Thing You Learned
I agree about Engines of Cognition. It got me really interested in the parallels between information theory and thermodynamics and led me to start reading a lot more about the former, including the classic Jaynes papers. I think it gave me a deeper understanding of why e.g. the Carnot limit holds, and let me to read about the interesting discovery that the thermodynamic availability (extractable work) of a system is equal to its Kullback-Leibler divergence (a generalization of informational entropy) from its environment.

Second for me would have to be Artificial Addition, which helped me understand why attempts to “trick” a system into displaying intelligence are fundamentally misguided.

SilasBarta 7 Mar 2009 3:35 UTC
−7 points
on: Slow down a little… maybe?
Wait a second … what made you guys start using my standard of putting underscores in the blanks in names?

SilasBarta 8 Mar 2009 20:08 UTC
5 points
in reply to: AnnaSalamon’s comment on: Moore’s Paradox
I don’t think there’s one model that covers 1) and 2) like you’re saying. I think two very different mental processes are going on, and we only use the term “belief” for both of them because we’ve committed the fallacy of compression.

That is, “I believe (in) X” can mean either

1) My mental model of reality includes X.

or

2) I affiliate with a group that centers around professing X [so I’ve got a gang watching out for me and if you’re part of it we have a basis for cooperating].

So, I don’t think there’s one answer for your question, because you’re describing two different processes, with different methods and goals. Choosing beliefs type 1) is the process of seeking actual truth, while type 2) is the process of gaining power through group affiliation.

Or maybe Robin_Hanson’s cynicism is rubbing off on me.

SilasBarta 9 Mar 2009 22:16 UTC
9 points
on: The Wrath of Kahneman
Wait, what would I change? My sense of justice already implies amplifying punishment by the improbability of detection (more precisely, the inverse of the recovery rate). And I already get angrier about antisocial behaviors that were only caught by an improbable coincidence. (E.g. I dislike when police officers drive right past hard-to-find traffic violations to catch people who are speeding .)

Am I really that much of an outlier? I had no idea these intuitions were so rare, though occasionally I find myself having to defend them, so I guess I should have known.

SilasBarta 26 Mar 2009 3:16 UTC
1 point
on: Spock’s Dirty Little Secret
I enjoyed reading this, pjeby. It answered and tied together a lot of the things I’d wondered since I started reading about artificial intelligence. I won’t spell out the relationship between your post and the issues (this will be long anyway...), but I’ll list the connections I saw and what it brought to mind:

-How evolution’s “shards of desire” translate into actions.

-What it would mean to, as evolutionary psychology attempts to do, “explain emotions by the behaviors they induce”. And similarly, what feelings an animal would have to get in order to execute complex but innate activities (e.g. cats burying excrement, beavers building dams).

-That paper that got a lot of attention by talking about the five dimensions or moral reasoning which went into how in certain cultures they feel physically ill at the though of failing their duties.

-The issue of “men are more rational, women more emotional”. I had thought that it would be more accurate to distinguish by reductionist/holist, i.e., what we call “emotional” means basing judgments on a broader array of factors that are aggregated automatically by the brain through useful heuristics.

-The standard model of an “agent” in AI—whereby it takes in sense data, models its environment, makes predictions, and selects actions that optimize some utility function. This had long seemed to me like the wrong way to go about the problem, at least because of all the infinite regress one runs into. I figured that very simple mechanisms go through crude but effective versions of this (e.g. a mass/spring oscillating back to its equilibrium position), and emotions in the sense you mean are another way to build up to a non-regessing agent.

SilasBarta 13 Apr 2009 19:25 UTC
4 points
in reply to: Swimmy’s comment on: Awful Austrians
Sorry if I’m late to the party.

If the argument is that no evidence of any form could ever change the Austrian’s probability estimate of a certain theory, I charge the Austrian is either being overconfident or violating the conservation of expected evidence.

Not true; such a position can be perfectly Bayesian: You would just believe that P(E|~H) = P(E|H). In other words, “On average, learning E tells you nothing about H, and vice versa.” Such an H wouldn’t be very useful, but it’s Bayesian. (I had to point this out on a post by Bob Murphy critical of Bryan Caplan, but I can’t find it ATM.) All Bayes prevents you from doing is taking both E and ~E as evidence for H. That is, believing both that P(E|H) > P(E|~H) and P(~E|H) > P(~E|~H). That would violate conservation of probability.

By saying “it’s not a controlled experiment so it doesn’t count,” the Austrian misses out on a key insight about how people can be milked

Please refer to the OB discussion on the Allais paradox, where the participants’ actions were justified quite well by poster “gray area” early on, and never answered by Eliezer_Yudkowsky. Long story short: the choice one is making fundamentally changes depending on whether it’s one-shot or indefinitely repeated. The money pump doesn’t manifest until you actually offer the choice multiple times at which point people not surprisingly pick up on it. But that money pump is just not happening in the one-shot version; ergo, no money is being pumped and nobody is being cheated, nor liking it.

Also, I’d like to comment on the price control example. While I certainly don’t want to defend the Austrians, I also don’t see how this is a point against them: rather, it’s a confirmatory example.

he proclaims that price controls will lead to rationing by non-price means. But this is only true if the provider of the good in question is attempting to maximize profit; if the producer is willing to take a hit in the wallet out of the goodness of his heart for his customers’ well-being, as Mises’ tautological definition of self-interest allows, a small price ceiling could conceivably have no effect.

If the producer is continuing to sell at the ceiling rate, that is still non-price rationing because he is rationing the good based on who gets there first or who he likes most, etc, exactly the thing Austrians are warning about, and exactly the kind of non-trivial insight many people still don’t appreciate.

So I agree with the general thrust of this post, but this is a bad example.

SilasBarta 14 Apr 2009 2:44 UTC
5 points
in reply to: Swimmy’s comment on: Awful Austrians

But if people can be milked in the very short run (the Allais Paradox is of course not the only example)

But they weren’t milked in the short run. They weren’t milked at all! From their perspective, they got two free lottery tickets. Doesn’t sound like a milking to me. They never go through the hypothetical “implication” of the Allais Paradox, that they would start to trade opportunities intransitively, because they never have a chance to. A scientist just throws two choices at them, and concludes that they would make some other stupid choices that, it turns out, they never do.

I disagree that a producer who produces at a ceiling rate rations based on who gets there first. The producer could increase supply in response to the price control. The only thing limiting him from doing so is the size of his bank account.

Yes, and that’s exactly the Austrians’ point! If that is what turns out to happen, then the goods are being rationed by a) the size of producer X’s bank account, plus b) who is among the first n people to reach producer X before he exhausts the limits of his inputs. And that is—wait for it—non-price rationing. That is, it’s not the price of that good that ultimately “rations” who gets it, since there will be people willing to pay the ceiling rate who won’t get to buy at that rate.

So again, I don’t see how these two examples substantiate your case—they’re counterexamples.

SilasBarta 14 Apr 2009 17:51 UTC
3 points
in reply to: Swimmy’s comment on: Awful Austrians
Yes, you qualified with “small” … and the term you qualified was “price ceiling”. A small price ceiling amplifies this effect. Perhaps you meant to say “A price ceiling that is a small amount below the current market-clearing price”.

In any case, the claim is still wrong. To the extent that the seller reallocates inputs based on non-price-maximizing goals, then non-price rationing is happening. That is, when he ups production, he’s putting inputs to uses they otherwise wouldn’t have been put at their current prices. Non-price rationing has still resulted, just maybe not in the good that was price-controlled.

Yes, you can keep adding on more conditions that get the scenario to work out, but like I said, that would still make it a bad example, considering all that you have to add to delete the insight the Austrian attempts to give.

Ready to cry uncle on conservation of expected evidence and Allais?

SilasBarta 14 Apr 2009 21:19 UTC
0 points
in reply to: Swimmy’s comment on: Awful Austrians

I charged conservation of expected evidence or overconfidence. I’m plenty willing to concede that overconfidence is more likely. My idea was that any claim which takes the form, “No evidence could convince me otherwise,” is probably falling victim to at least one of the two.

If that’s what you’re saying, then the situation I gave you still doesn’t count as either. The situation was where you believe that P(E|~H) = P(E|H) for all E and a given H. I showed how it doesn’t violate CEE already. It also can’t be described as “overconfidence”. If anything, its’ under confidence since the person who believes it is deliberately setting it up so as to shy away from any possibility of being proven wrong.

Of course, this position also might not be what Austrians believe, since they claim that the insights (the H’s) from Austrian economics do constrain their expectations, but also that nothing can make them change their belief in them, which would then be a violate of CEE.

I’m asserting that the stability across large groups of people of Allais gamble preferences is evidence of a violation of rationality rather than a preference change. … Milking” (but not “money pumping”) is therefore an accurate diagnosis of what happens in the one-shot game.

And I’ve shown how that’s wrong. It’s not a preference change, and it’s not irrational. Rather, it’s a reasonable choice when you only get one shot at each choice. All the supposed “proofs” of irrationality or preference reversal require you to first assume multiple repititions of the game and the chance to trade one game for another, neither of which happen in the experiment. And again, they weren’t milked; they got two free lottery tickets.

I do not understand your point about price rationing. Prices reflect subjective values. If the producer’s motivation is “decrease price to the mandate,” how do the new prices on inputs not reflect that value? The economy rations on goods forgone, yes, but the producer in question is the highest bidder for the use of those goods, so the new prices still reflect people’s values.

No, they don’t. For the case of a price cap that is a small amount below the market-clearing price, the noble producer must up his production to handle the additional buyers, which requires him to implicitly sell his labor (to himself) below market prices. (If this were not the case—if his labor were really worth that much—the market price would not have been at its current level, which was the assumption.) Then there is non-price rationing because there is someone who wants to buy his underpriced labor but cannot.

SilasBarta 28 Apr 2009 21:13 UTC
5 points
in reply to: Richard_Kennaway’s comment on: What is control theory, and why do you need to know about it?
RichardKennaway, very interesting post. I actually specialized in control theory in graduate school, but didn’t finish the program. I must object to what you’ve said here, in that control theory most certainly does make extensive use of Bayesian inferenence, under the name of the Kalman filter.

The Kalman filter is a way of estimating the paramaters of a system, given your observations and your knowledge of the system’s dynamics. While it may not help you pick a good control input algorithm, and while the problems you listed there may not need such accurate estimation of the data, it is an integral part of finding out how much the system deviates from where you want it to be, and is used extensively in controls.

SilasBarta 28 Apr 2009 22:46 UTC
4 points
on: What is control theory, and why do you need to know about it?
Richard’s post is similar to something I was thinking about a few months ago. I tried to attack the problem of AI by looking at very simple systems that can be said to accomplish “goals” without all the fancy stuff that people typically think they have to put in AI, and and asking how that works.

For example, a mass hanging by a spring: it moves the mass back to its equilibrium position without doing the things listed in 2). But here, Richard is asking an easier question in 4), since he’s asking about systems that are specifically designed to track some reference, rather than systems that happen to do it as a consequence of their other properties.

In that case, the answer (about how an arational system accomplishes the goals of rationality) is pretty simple: the system has been physically set up in a way that exploits the laws of nature to create mutual information between the system and its environment. If you view Bayescraft as a way to increase the mutual information between yourself (hopefully meaning the brain part!) and your environment, then the system is in fact doing that, so it is not arational. Its design implements Bayesian inference.

In the case of the thermostat, the temperature sensor, via heat transfer, becomes entangled with its environment, a natural process that happens to have an isomorphism to the Bayes Theorem. Then, something else senses the reading, causing another set of effents that determines what temperature air to blow out.

The next question is why this mutual information is such that it keeps the temperature within a specific range, rather than making it spiral out of control. The answer to that part, as others have mentioned, is that the person who set up the system, chose rules that happened to work. That required another kind of entanglement with the environment, which does not need to be done again during the operation of the thermostat.

Well, as long as the assumptions it’s based on don’t change too much...
What links here?
- SilasBarta's comment on Without models by Richard_Kennaway (5 May 2009 4:07 UTC; 12 points)

SilasBarta 3 May 2009 21:37 UTC
1 point
in reply to: Vladimir_Nesov’s comment on: TED Talks for Less Wrong
Hm, I’m disappointed. He’s only able to talk very briefly about the Hurley model for what makes things funny and why evolution would produce it. All I got was something about the brain needing to provide a reward for “debugging”, or I guess “discerning the source of a problem”.

I’d like to read more about it, but when I googled “Matthew Hurley” and terms like jokes/humor, the only results were … links to that video!

Anyone know where I can read more about the Hurley model?

SilasBarta 4 May 2009 13:24 UTC
3 points
on: Allais Hack—Transform Your Decisions!
Can we please stop misreading the results of Allais experiments? As I tried to explain on Awful Austrians, you simply cannot draw the standard conclusion from the experiment, and this was explained early on in the first OB thread by gray area.

To summarize: the choice you are making fundamentally changes depending on whether it’s one-shot or repeated as many times as you want. The standard choice made by [EDIT typo] test subjects cannot be shown to be irrational unless and until you give them the choices repeatedly, in which cases they actually get the expected return. Otherwise, there is no money pump, no loss, no opportunity for being cheated.

As conducted, the experiment gives the subjects two free lottery tickets. If getting two free lottery tickets is exploitation, I don’t want to be empowered. (You can quote me on that.)

SilasBarta 5 May 2009 4:07 UTC
12 points
in reply to: Vladimir_Nesov’s comment on: Without models
This is exactly why I tried to restate the situation in terms of the more precise concept of “mutual information” in Richard’s last topic, although I guess I was a bit vague at points as to how it works.

So in the context of Bayesian inference, and rationality in general, we should start with:

“A controller has a model (explicit or implicit) of it’s environment iff there is mutual information between the controller and the environment.”

This statement is equivalent to:

“A controller has a model (explicit or implicit) of its environment iff, given the controller, you require a shorter message to describe its environment (than if you could not reference the controller).”

From that starting point, the question is easier to answer. Take the case of the thermostat. If the temperature sensor is considered part of this controller, then yes, it has a model of its environment. Why? Because if you are given the sensor reading, you can more concisely describe the environment: “That reading, plus a time/amplitude shift.”

Richard puts a lot of emphasis on how cool it is that the thermostat doesn’t need to know if the sun is shining. This point can be rephrased as:

“A controller does not need to have mutual information with all of its environment to work.” Or,

“Learning a controller, and the fact that it works, does not suffice to tell you everything about its environment.”

I think that statement sums up what Richard is trying to say here.

And of course you can take this method further and discuss the mutual information between a) the controller, b) the output, c) the environment. That is, do a) and b) together suffice to tell you c)?

EDIT: Some goofs
What links here?

SilasBarta 6 May 2009 17:59 UTC
0 points
in reply to: Richard_Kennaway’s comment on: Without models
FYI: I’m working on a reply to this, which is becoming long enough and broad enough that I’m going to submit it for the front page.

SilasBarta 12 May 2009 13:31 UTC
0 points
in reply to: MichaelVassar’s comment on: No One Knows Stuff
And likewise, can I apply for whatever position you’re interviewing these people for? (I mean talisman, not SiAI. I think SIAI requires such unreasonable skills out of me as “tact” and “not voicing why you think other people are idiots”.) I’m sure I’d be in the top 1%.

SilasBarta 12 May 2009 16:27 UTC
1 point
in reply to: MichaelVassar’s comment on: No One Knows Stuff
Wow! That’s tough! I don’t know if I could ever be more qualified than someone who nearly shut down the entire financial system! ;-)

SilasBarta 12 May 2009 17:34 UTC
2 points
in reply to: MichaelHoward’s comment on: No One Knows Stuff
Yeah, that is kind of tricky. Let me try to explain what Eliezer_Yudkowsky meant in terms of my preferred form of the Bayes Theorem:

O(H|E) = O(H) * P(E|H) / P(E|~H)

where O indicates odds instead of probability and | indicates “given”.

In other words, “any time you observe evidence, amplify the odds you assign to your beliefs by the probability of observing the evidence if the belief were true, divided by the probabily of observing it if the belief were false.”

Also, keep in mind that Eliezer_Yudkowsky has written about how you should treat very low probability events as being “impossible”, even though you have to assign a non-zero probability to everything.

Nevertheless, his statement still isn’t literally true. The strength of the evidence depends on the ratio P(E|H)/P(E|~H), while the quoted statement only refers to the denominator. So there can be situations where you have 100:1 odds of seeing E if the hypothesis were true, but 1:1000 odds (about a 0.1% chance) of seeing E if it were false.

Such evidence is very strong—it forces you to amplify the odds you assign to H by a factor of 100,000 -- but it’s far from evidence you “couldn’t possibly find”, which to me means something like 1:10^-10 odds.

Still, Eliezer_Yudkowsky is right that, generally, strong evidence will have a very small denominator.

EDIT: added link

SilasBarta 19 May 2009 16:56 UTC
5 points
in reply to: PhilGoetz’s comment on: Bad reasons for a rationalist to lose
Well, it looks like I’m an extreme outlier on this one, because I actually voted it down because I thought it got a lot wrong, and for bad reasons.

First of all, despite criticizing EY for “needing” things that would merely be supercool, matt lists a large number of things that would also be merely supercool: it just doesn’t seem like you need all of those chance values either.

Second, matt seemed to miss why EY was asking for all of that information: that presenting a “neato trick” that happens to work, provides very little information as to why it works, and when it should be used, etc. EY had explained that he personally went through such an experience and described what is lacking when you don’t provide the information he asked for.

In short, EY provided very good reasons why he should be skeptical of just trying every neato trick, matt said very little that was responsive to his points.