othercriteria

Karma: 420

othercriteria Oct 23, 2014, 5:47 AM
0 points
in reply to: Cyan’s comment on: Anthropic signature: strange anti-correlations
While maybe not essential, the “anti-” aspect of the correlations induced by anthropic selection bias at least seems important. Obviously, the appropriate changes of variables can make any particular correlation go either positive or negative. But when the events all measure the same sort of thing (e.g., flooding in 2014, flooding in 2015, etc.), the selection bias seems like it would manifest as anti-correlation. Stretching an analogy beyond its breaking point, I can imagine these strange anti-correlations inducing something like anti-ferromagnetism.

othercriteria Oct 21, 2014, 6:21 PM
0 points
in reply to: TheMajor’s comment on: What math is essential to the art of rationality?

To pick a frequentist algorithm is to pick a prior with a set of hypotheses, i.e. to make Bayes’ Theorem computable and provide the unknowns on the r.h.s. above (as mentioned earlier you can in theory extract the prior and set of hypotheses from an algorithm by considering which outcome your algorithm would give when it saw a certain set of data, and then inverting Bayes’ Theorem to find the unknowns.

Okay, this is the last thing I’ll say here until/unless you engage with the Robins and Wasserman post that IlyaShpitser and I have been suggesting you look at. You can indeed pick a prior and hypotheses (and I guess a way to go from posterior to point estimation, e.g., MAP, posterior mean, etc.) so that your Bayesian procedure does the same thing as your non-Bayesian procedure for any realization of the data. The problem is that in the Robins-Ritov example, your prior may need to depend on the data to do this! Mechanically, this is no problem; philosophically, you’re updating on the data twice and it’s hard to argue that doing this is unproblematic. In other situations, you may need to do other unsavory things with your prior. If the non-Bayesian procedure that works well looks like a Bayesian procedure that makes insane assumptions, why should we look to Bayesian as a foundation for statistics?

(I may be willing to bite the bullet of poor frequentist performance in some cases for philosophical purity, but I damn well want to make sure I understand what I’m giving up. It is supremely dishonest to pretend there’s no trade-off present in this situation. And a Bayes-first education doesn’t even give you the concepts to see what you gain and what you lose by being a Bayesian.)

othercriteria Oct 16, 2014, 5:58 PM
3 points
in reply to: IlyaShpitser’s comment on: What math is essential to the art of rationality?
You’re welcome for the link, and it’s more than repaid by your causal inference restatement of the Robins-Ritov problem.

Of course arguably this entire setting is one Bayesians don’t worry about (but maybe they should? These settings do come up).

Yeah, I think this is the heart of the confusion. When you encounter a problem, you can turn the Bayesian crank and it will always do the Right thing, but it won’t always do the right thing. What I find disconcerting (as a Bayesian drifting towards frequentism) is that it’s not obvious how to assess the adequacy of a Bayesian analysis from within the Bayesian framework. In principle, you can do this mindlessly by marginalizing over all the model classes that might apply, maybe? But in practice, a single model class usually gets picked by non-Bayesian criteria like “does the posterior depend on the data in the right way?” or “does the posterior capture the ‘true model’ from simulated data?”. Or a Bayesian may (rightly or wrongly) decide that a Bayesian analysis is not appropriate in that setting.

othercriteria Oct 16, 2014, 1:32 AM
3 points
in reply to: TheMajor’s comment on: What math is essential to the art of rationality?
Have you seen the series of blog posts by Robins and Wasserman that starts here? In problems like the one discussed there (such as the high-dimensional ones that are commonly seen these days), Bayesian procedures, and more broadly any procedures that satisfy the likelihood principle, just don’t work. The procedures that do work, according to frequentist criteria, do not arise from the likelihood so it’s hard to see how they could be approximations to a Bayesian solution.

You can also see this situation in the (frequentist) classic Theory of Point Estimation written by Lehmann and Casella. The text has four central chapters: “Unbiasedness”, “Equivariance”, “Average Risk Optimality”, and “Minimaxity and Admissibility”. Each of these introduces a principle for the design of estimators and then shows where this principle leads. “Average Risk Optimality” leads to Bayesian inference, but also Bayes-Lite methods like empirical Bayes. But each of the other three chapters leads to its own theory, with its own collection of methods that are optimal under that theory. Bayesian statistics is an important and substantial part of the story told by in that book, but it’s not the whole story. Said differently, Bayesian statistics may be a framework for Bayesian procedures and a useful way of analyzing non-Bayesian statistics, but they are not the framework for all of statistics.

othercriteria Oct 15, 2014, 11:30 PM
2 points
in reply to: TheMajor’s comment on: What math is essential to the art of rationality?

(Theoretical) Bayesian statistics is the study of probability flows under minimal assumptions—any quantity that behaves like we want a probability to behave can be described by Bayesian statistics.

But nobody, least of all Bayesian statistical practitioners, does this. They encounter data, get familiar with it, pick/invent a model, pick/invent a prior, run (possibly approximate) inference of the model against the data, verify if inference is doing something reasonable, and jump back to an earlier step and change something if it doesn’t. After however long this takes (if they don’t give up), they might make some decision based on the (possibly approximate) posterior distribution they end up with. This decision might involve taking some actions in the wider world and/or writing a paper.

This is essentially the same workflow a frequentist statistician would use, and it’s only reasonable that a lot of the ideas that work in one of these settings would be useful, if not obvious or well-motivated, in the other.

I know that philosophical underpinnings and underlying frameworks matter but to quote from a recent review article by Reid and Cox (2014):

A healthy interplay between theory and application is crucial for statistics, as no doubt for other fields. This is particularly the case when by theory we mean foundations of statistical analysis, rather than the theoretical analysis of specific statistical methods. The very word foundations may, however, be a little misleading in that it suggests a solid base on which a large structure rests for its entire security. But foundations in the present context equally depend on and must be tested and revised in the light of experience and assessed by relevance to the very wide variety of contexts in which statistical considerations arise. It would be misleading to draw too close a parallel with the notion of a structure that would collapse if its foundations were destroyed.

othercriteria Oct 15, 2014, 7:04 PM
0 points
in reply to: Lumifer’s comment on: What math is essential to the art of rationality?
Thanks for pointing out the Gelman and Shalizi paper. Just skimmed it so far, but it looks like it really captures the zeitgeist of what reasonably thoughtful statisticians think of the framework they’re in the business of developing and using.

Plus, their final footnote, describing their misgivings about elevating Bayesianism beyond a tool in the hypothetico-deductive toolbox, is great:

Ghosh and Ramamoorthi (2003, p. 112) see a similar attitude as discouraging inquiries into consistency: ‘the prior and the posterior given by Bayes theorem [sic] are imperatives arising out of axioms of rational behavior – and since we are already rational why worry about one more’ criterion, namely convergence to the truth?

othercriteria Oct 15, 2014, 6:18 PM
5 points
in reply to: TheMajor’s comment on: What math is essential to the art of rationality?
I would advise looking into frequentist statistics before studying Bayesian statistics. Inference done under Bayesian statistics is curiously silent about anything besides the posterior probability, including whether the model makes sense for the data, whether the knowledge gained about the model is likely to match reality, etc. Frequentist concepts like consistency, coverage probability, ancillarity, model checking, etc., don’t just apply to frequentist estimation; they can be used to asses and justify Bayesian procedures.

If anything, Bayesian statistics should just be treated as a factory that churns out estimation procedures. By a corollary of the complete class theorem, this is also the only way you can get good estimation procedures.

ETA: Can I get comments in addition to (or instead of) down votes here? This is a topic I don’t want to be mistaken about, so please tell me if I’m getting something wrong. Or rather if my comment is coming across as “boo Bayes”, which calls out for punishment.

othercriteria Oct 15, 2014, 5:28 PM
3 points
in reply to: Gunnar_Zarncke’s comment on: What math is essential to the art of rationality?
This is really good and impressive. Do you have such a list for statistics?

othercriteria Oct 15, 2014, 1:42 PM
0 points
in reply to: Douglas_Knight’s comment on: Rationality Quotes October 2014
The example I’m thinking about is a non-random graph on the square grid where west/east neighbors are connected and north/south neighbors aren’t. Its density is asymptotically right at the critical threshold and could be pushed over by adding additional west/east non-neighbor edges. The connected components are neither finite nor giant.

othercriteria Oct 14, 2014, 5:11 AM
5 points
in reply to: Sherincall’s comment on: Open thread, Oct. 13 - Oct. 19, 2014
If you want a solid year-long project, find a statistical model you like and figure out how to do inference in it with variational Bayes. If this has been done, change finite parts of the model into infinite ones until you reach novelty or the model is no longer recognizable/tractable. At that point, either try a new model or instead try to make the VB inference online or parallelizable. Maybe target a NIPS-style paper and a ~30-page technical report in addition to whatever your thesis will look like.

And attend a machine learning class, if offered. There’s a lot of lore in that field and you’ll miss out if you do the read-the-book-work-each-problem thing that is alleged to work in math.

othercriteria Oct 13, 2014, 2:58 AM
4 points
in reply to: Scott Alexander’s comment on: 2014 Less Wrong Census/Survey—Call For Critiques/Questions
But to all of us perched on the back of Cthulhu, who is forever swimming left, is it the survey that will seem fixed and unchanging from our moving point of view?

othercriteria Oct 12, 2014, 6:40 PM
1 point
on: LessWrong Help Desk—free paper downloads and more (2014)
Buehler, Denis. “Incomplete understanding of complex numbers Girolamo Cardano: a case study in the acquisition of mathematical concepts.” Synthese 191.17 (2014): 4231-4252.

Vélez, Ricardo, and Tomás Prieto-Rumeau. “Random assignment processes: strong law of large numbers and De Finetti theorem.” TEST (2014): 1-30.

othercriteria Oct 12, 2014, 3:17 AM
7 points
on: 2014 Less Wrong Census/Survey—Call For Critiques/Questions
In the mental health category, I’d love to see (adult) ADHD there as well. I’m less directly interested in substance abuse disorder and learning disabilities (in the US sense) / non-autism developmental disabilities, but those would be interesting additions too.

othercriteria Oct 11, 2014, 11:52 PM
0 points
in reply to: bogus’s comment on: Open thread, Oct. 6 - Oct. 12, 2014
I’d believe that; my knowledge of music history isn’t that great and seeing teleology where there isn’t any is an easy mistake.

I guess what I’m saying, speaking very vaguely, is that melodies existing within their own tonal contexts are as old as bone flutes, and their theory goes back at least as far as Pythagoras. And most folk music traditions cooked up their own favorite scale system, which you can just stay in and make music as long as you want to. For that matter, notes in these scale systems can be played as chords and a lot of the combinations make musical sense (often with nicer consonance than is possible with notes that have to respect even temperament).

What western art music and its audience co-evolved into (not necessarily uniquely among music traditions?) was a state where something like the first few bars of the Schubert String Quintet can function. The first violin plays a note twice, with the harmonic context changing under it, driving the melody forward, driving the harmony forward, etc. I should probably have said a non-static harmonic context to be more clear.

othercriteria Oct 7, 2014, 10:28 PM
0 points
in reply to: TheMajor’s comment on: Rationality Quotes October 2014
In something like the Erdös-Rényi random graph, I agree that there is an asymptotic equivalence between the existence of a giant component and paths from a randomly selected points being able to reach the “edge”.

On something like an n x n grid with edges just to left/right neighbors, the “edge” is reachable from any starting point, but all the connected components occupy just a 1/n fraction of the vertices. As n gets large, this fraction goes to 0.

Since, at least as a reductio, the details of graph structure (and not just its edge fraction) matters and because percolation theory doesn’t capture the idea of time dynamics that are important in understanding epidemics, it’s probably better to start from a more appropriate model.

Maybe look at Limit theorems for a random graph epidemic model (Andersson, 1998)?

othercriteria Oct 7, 2014, 9:40 PM
8 points
in reply to: sixes_and_sevens’s comment on: Open thread, Oct. 6 - Oct. 12, 2014
The idea that melodies, or at least an approximation accurate to within a few cents, can be embedded into a harmonic context. Yet in western art music, it took centuries for this to go from technically achievable but unthinkable to experimental to routine.

othercriteria Oct 6, 2014, 5:13 AM
3 points
in reply to: TheMajor’s comment on: Rationality Quotes October 2014
I think percolation theory concerns itself with a different question: is there a path from starting point to the “edge” of the graph, as the size of the graph is taken to infinity. It is easy to see that it is possible to hit infinity while infecting an arbitrarily small fraction of the population.

But there are crazy universality and duality results for random graphs, so there’s probably some way to map an epidemic model to a percolation model without losing anything important?

othercriteria Jul 31, 2014, 3:37 PM
3 points
in reply to: DanielLC’s comment on: Open thread, July 28 - August 3, 2014
This comment rubbed me the wrong way and I couldn’t figure out why at first, which is why I went for a pithy response.

I think what’s going on is I was reacting to the pragmatics of your exchange with Coscott. Coscott informally specified a model and then asked what we could conclude about a parameter of interest, which coin was chosen, given a sufficient statistic of all the coin toss data, the number of heads observed.

This is implicitly a statement that model checking isn’t important in solving the problem, because everything that could be used for model checking, e.g., statistics on runs to verify independence, the number of tails observed to check against a type of miscounting where the number of tosses don’t add to 1,000,000, mental status inventories to detect hallucination, etc., is left out of the statistic communicated.

Maybe Coscott (the fictional version who flipped all those coins) did model checking or maybe not, but if it was done and the data suggested miscounting or hallucination, then Coscott wouldn’t have stated the problem like this.

So, yeah, the points you raise are valid object-level ones, but bringing them up this way in a problem poser / problem solver context was really unexpected and seemed to violate the norms for this sort of exchange.

othercriteria Jul 31, 2014, 2:40 PM
0 points
in reply to: Stuart_Armstrong’s comment on: Why the tails come apart

I’m quite confident in predicting that generic models are much more likely to be overfitted than to have too few degrees of freedom.

It’s easy to regularize estimation in a model class that’s too rich for your data. You can’t “unregularize” a model class that’s restrictive enough not to contain an adequate approximation to the truth of what you’re modeling.

othercriteria Jul 29, 2014, 11:22 PM
4 points
in reply to: DanielLC’s comment on: Open thread, July 28 - August 3, 2014
When I know I’m to be visited by one of my parents and I see someone who looks like my mother, should my first thought be “that person looks so unlike my father that maybe it is him and I’m having a stroke”? Should I damage my eyes to the point where this phenomenon doesn’t occur to spare myself the confusion?