Eyal Roth

Karma: 6

Eyal Roth Mar 27, 2019, 10:33 PM
1 point
0
on: Report likelihoods, not p-values
I’m going to take the role of the “undergrad” here and try to interpret this in the following way:

Given that a hypothesis is true—but it is unknown to be true—it is far more likely to come by a “statistically significant” result indicating it is wrong, than it is likely to come by a result indicating that another hypothesis is significantly more likely.

In simpler words—it is far easier to “prove” a true hypothesis is wrong by accident, than it is to “prove” that an alternative hypothesis is superior (a better estimator of reality) by accident.

Would you consider this interpretation accurate?

Eyal Roth Mar 27, 2019, 10:17 PM
1 point
0
on: Report likelihoods, not p-values
I feel like this paragraph might be a little necessary for someone who haven’t read the bayes rule intro, but on the other hand is a bit off-topic in this context and quite distracting, as it raises questions which are not part of this “discussion”; mainly, questions regarding how to approach “one-off” events.

Say, what if I can’t quantify the outcome of my decision so nicely like in the case of a bet? What if I need to decide whether to send Miss Scarlet to prison or not based on these likelihood probabilities?

Eyal Roth Mar 27, 2019, 10:07 PM
1 point
0
on: Report likelihoods, not p-values
This “argument” by the “scientist” doesn’t IMO represent how a true experimentalist would approach the issue; they would not necessarily be so opposed to trying new ways of improving their methods, as long as it is done step by step without replacing the entire system over night (just like the “bayesian” explains in the next paragraph).

This is also a bit side-tracking as it opens up the topic of how much more “experience” computer scientists have given the simpler and much more reproducible systems they’re dealing with—especially in the modern commercial world—in contrast with natural sciences (I’m a programmer myself, so I’m a bit biased on this).

Eyal Roth Mar 27, 2019, 6:11 PM
1 point
0
on: Report likelihoods, not p-values
Not quite. The private state of mind of the researcher changes nothing. It’s only the issue of which question is asked.

In this case, the two questions are (a) what is the probability of such an event occurring after tossing a fair coin 6 times and (b) what is the probability of such an event occurring if a fair coin is tossed until it lands tails.

The meaning of the questions does not change, nor do the answers to them. It is only a matter of what question is being asked—which is obviously important when conducting a study, but is not so counter-intuitive (and much less confusing) when presented in such way (IMO).

Eyal Roth Mar 26, 2019, 11:03 PM
1 point
0
on: Laplace’s Rule of Succession
Broken link :(

Eyal Roth Mar 26, 2019, 10:23 PM
1 point
0
on: Interpretations of “probability”
I really love this example; it is one of the few I’ve managed to find online which actually helped me understand the differences between the approaches.

Eyal Roth Mar 26, 2019, 10:11 PM
1 point
0
on: Interpretations of “probability”
Why the capital letters? Is this suppose to refer or to link to something?

Eyal Roth Mar 20, 2019, 12:00 AM
1 point
0
on: Bayes’ rule: Vector form
I believe it is essential to explain why it is independent in the case of the bathtub example and not in the other examples.

In the bathtub example, the evidence presents an event which is directly described by the assessed trait; i.e, the fairness of a coin is directly concerned with the appearance of either heads or tails. In contrast, the definition of the degree of “spamness” in an email is not directly concerned with the appearance of a word in the email, but is rather concerned with the abstract concept of the meaning a person assigns to the email.

The appearance of a word in an email is hence only an attempt of estimating the degree of “spamness”, a proxy. In the case of a proxy, we need to consider the option that the proxy is flawed in a way which makes it so that the evidences are in fact dependencies of one another. This is not necessarily true, but it is possible, unlike in the case of hypothetical coins (in reality, a coin toss might actually be physically affected by the previous toss).

Eyal Roth Mar 19, 2019, 11:02 PM
3 points
0
on: Bayes’ rule: Vector form
I find the entire explanation described below very misleading and perhaps even largely incorrect. The workshop participants had it wrong mostly for two reasons:
1. They did not consider what is the likelihood of visiting a museum / workplace given any other alternative (mutually exclusive) relationship - not strangers but also not romantically involved; i.e, friends. Being acquaintances is not a relevant type of a relationship as it is not mutually exclusive with a romantic relationship (a pair can be both dating and working together).
2. They did not know the prior probability of an arbitrary pair of people being romantically involved. A naive assumption of 50% of them being romantically involved is wrong, and should be made by observing the proportions of romantic relationships in the population.
In terms of the previous coins-fairness example, they (a) only considered that one type of coin (fair) is 2 times as likely to turn up heads as another type of coin (tail-biased), but did not consider how likely are the other type of coins (head-biased) to turn up heads; and (b) they did not know the proportions of coin types in the bathtub.

The explanation below also fails to mention the important assumption that the trait being assessed in all of the examples (coins, emails, workshop) is constant and doesn’t change over time. It is important to mention because it may not be so trivial for every example, yet it reduces the complexity of the estimations tremendously. A coin is not expected to change its bias significantly over time, yet a relationship does, and so does the magnitude of “spamness” in a given mail for a given person (for instance, when I get older I may be more interested in pharmaceutical ads).

Eyal Roth Mar 18, 2019, 10:13 PM
1 point
0
on: Bayes’ rule: Log-odds form
Easier to grasp perhaps, but dangerously misleading. Increasing the likelihood of an event from 10^-100 to 10^-99 is very different and much less significant than increasing it from 10^-2 (1%) to 10^-1 (10%). I hope this is covered later in this guide.

Eyal Roth Mar 18, 2019, 10:07 PM
3 points
0
on: Bayes’ rule: Log-odds form
It is really confusing to apply one of the initial steps of a study as evidence to a prior which is the result (last step) of the same study.

Eyal Roth Mar 18, 2019, 10:05 PM
1 point
0
on: Bayes’ rule: Log-odds form
But that really gives a different magnitude to the evidence. Why not be consistent with the log base?

For example, if we were to use log base 2, the prior would be ~16.6 magnitudes strong and the evidence ~8. This means that the evidence would alter the prior by (slightly) less than half the order of magnitudes, where’s in the case of log base 10 the alteration is (slightly) more than half the order of magnitudes (5 vs 2.7).

Also, imagine the absurd choice of log base 100k. The prior would remain practically intact in terms of this kind of order of magnitudes.

Eyal Roth Mar 18, 2019, 9:01 PM
1 point
0
on: Bayes’ rule: Log-odds form
Wrong, they are exactly the same distances. I read the next paragraph so I get where you were going with this, but I find it confusing to start off with a blatantly wrong claim, especially when the next line compares 0.11 to 0.1 (11% to 10%) -- not to 0.100001 -- in order to describe how the significance of 0.00001 gets “lost in translation” when speaking in probabilities and not in bits.

Eyal Roth Mar 18, 2019, 8:06 PM
3 points
0
on: Extraordinary claims require extraordinary evidence
I really have a hard time understanding the point of this section.

What difference is there between calculating the posterior given an evidence—thus updating the future prior—and questioning the prior “in the first place”? Isn’t this the whole point of the process? to examine the prior and question it in case of an extraordinary evidence?