Tag: statistics

Differences in differences

The Nobel Prize goes to David Card, Joshua Angrist and Guido Imbens. If you seek their monuments look around you. Almost all of the empirical work in economics that you read in the popular press is due to analyzing natural experiments using techniques such as difference in differences, instrumental variables and regression discontinuity. The obvious way to estimate the effect of the minimum wage is to look at the difference in employment in fast food restaurants before and after the law went into effect. But other things are changing through time so circa 1992 the standard approach was to “control for” other variables by also including in the statistical analysis factors such as the state of the economy. Include enough control variables, so the reasoning went, and you would uncover the true effect of the minimum wage. Card and Krueger did something different, they turned to a control group. Pennsylvania didn’t pass a minimum wage law in 1992 but it’s close to New Jersey so Card and Kruger reasoned that whatever other factors were affecting New Jersey fast food restaurants would very likely also influence Pennsylvania fast food restaurants. The state of the economy, for example, would likely have a similar effect on demand for fast food in NJ as in PA as would say the weather. In fact, the argument extends to just about any other factor that one might imagine including demographics, changes in tastes, changes in supply costs. The standard approach circa 1992 of “controlling for” other variables requires, at the very least, that we know what variables are important. But by using a control group, we don’t need to know what the other variables are only that whatever they are they are likely to influence NJ and PA fast food restaurants similarly. Put differently NJ and PA are similar so what happened in PA is a good estimate of what would have happened in NJ had NJ not passed the minimum wage. Thus what Card and Kruger estimated the effect of the minimum wage in New Jersey by calculating the difference in employment in NJ before and after the law and then subtracting the difference in employment in PA before and after the law. Hence the term difference in differences. By subtracting the PA difference (i.e. what would have happened in NJ if the law had not been passed) from the NJ difference (what actually happened) we are left with the effect of the minimum wage. Brilliant!

LSD lowers priors

An important aspect of predictive processing is that each hypothesis generated by a level in the hierarchy is associated with a notion of confidence in the hypothesis, which in turn is based on prior expectations. Could psychedelics be altering our perception of reality by messing with this process? Friston and Robin Carhart-Harris think so. If psychedelics mess with prior beliefs, that might also explain why they cause one to hallucinate a reality that’s untethered from real-world expectations.

A Causal Sequence

Here is how I currently understand the relationship between correlation and causality, and the collective findings of meta-scientific research: a shockingly large fraction of psychological research and other fields is simple random noise which cannot be replicated ‘everything is correlated’—even things which seem to have no causal relationship whatsoever most efforts to change human behavior and sociology and economics and education fail in randomized evaluation in every field from medicine to economics, when we directly ask how well correlations predict subsequent randomized experiments, we find that the predictive power is poor all variables are part of enormous dense causal graphs ‘folk causality’ often performs badly, especially in extremely complex fields with ambiguous long-term outcomes

Self-locating uncertainty

Self-locating uncertainty is a different kind of epistemic uncertainty from that featured in pilot-wave models. You can know everything there is to know about the universe, and there’s still something you’re uncertain about, namely where you personally are within it. Your uncertainty obeys the rules of ordinary probability, but it requires a bit of work to convince yourself that there’s a reasonable way to assign numbers to your belief. In one sense, all of these notions of probability can be thought of as versions of self-locating uncertainty. All we have to do is consider the set of all possible worlds — all the different versions of reality one could possibly conceive. Some such worlds obey the rules of dynamical-collapse theories, and each of these is distinguished by the actual sequence of outcomes for all the quantum measurements ever performed. Other worlds are described by pilot-wave theories, and in each one the hidden variables have different values. Still others are many-worlds realities, where agents are uncertain about which branch of the wave function they are on. We might think of the role of probability as expressing our personal credences about which of these possible worlds is the actual one.

Base rate fallacy

The flawed reasoning behind the Replication Crisis

Medical students are now routinely taught the diagnostic importance of base incidence rates. Bayes’ theorem helps them properly contextualize test results and avoid unnecessarily alarming patients who test positive for something rare. To leave out that final ingredient, the Bayesian prior probability, would be to commit a fallacy of the same species as the one in the Sally Clark case. The crisis of replication has exposed the fact, which has been the shameful secret of statistics for decades now, that the same fallacy is at the heart of modern scientific practice.

Destroying psychology

interesting piece on the replication crisis in psychology, and what’s next.

Lindsay talks with psychologists all the time who aren’t eager to embrace the updated rules, and he understands why. “Our literature is packed with unreliable findings. And I can imagine if you hitched your whole wagon to a concept that doesn’t seem to be a real thing, that could be threatening.” Like Heathers, Nick Brown sometimes shakes his head at the reluctance among researchers to acknowledge what, to him, seems obvious. To continue to defend a system that’s churned out stacks upon stacks of hopelessly flawed papers, rather than to own up to the truth and try to fix it, seems pointless. “I don’t know whether they genuinely believe they’re doing the right thing or there’s a sort of doubt niggling at the back of their mind, but they don’t want to acknowledge it. Maybe the people who need to make those changes, in that deep, dark moment before they go to sleep, they think to themselves, ‘How are we going to get out of this?’”