12 January 2008 by John.

Jim Berger gives the following example illustrating the difference between frequentist and Bayesian approaches to inference in his book The Likelihood Principle.

Experiment 1:

A fine musician, specializing in classical works, tells us that he is able to distinguish if Hayden or Mozart composed some classical song. Small excerpts of the compositions of both authors are selected at random and the experiment consists of playing them for identification by the musician. The musician makes 10 correct guesses in exactly 10 trials.

Experiment 2:

A drunken man says he can correctly guess in a coin toss what face of the coin will fall down. Again, after 10 trials the man correctly guesses the outcomes of the 10 throws.

A frequentist statistician would have as much confidence in the musician’s ability to identify composers as in the drunk’s ability to predict coin tosses. In both cases the data are 10 successes out of 10 trials. But a Bayesian statistician would combine the data with a prior distribution. Presumably most people would be inclined a priori to have more confidence in the musician’s claim than the drunk’s claim. After applying Bayes theorem to analyze the data, the credibility of both claims will have increased, though the musician will continue to have more credibility than the drunk. On the other hand, if you start out believing that it is completely impossible for drunks to predict coin flips, then your posterior probability for the drunk’s claim will continue to be zero, no matter how much evidence you collect.

Dennis Lindley coined the term “Cromwell’s rule” for the advice that nothing should have zero prior probability unless it is logically impossible. The name comes from a statement by Oliver Cromwell addressed to the Church of Scotland:

I beseech you, in the bowels of Christ, think it possible that you may be mistaken.

In probabilistic terms, “think it possible that you may be mistaken” corresponds to “don’t give anything zero prior probability.” If an event has zero prior probability, it will have zero posterior probability, no matter how much evidence is collected. If an event has tiny but non-zero prior probability, enough evidence can eventually increase the posterior probability to a large value.

The difference between a small positive prior probability and a zero prior probability is the difference between a skeptical mind and a closed mind.

Posted in Statistics, Science | No Comments »

12 January 2008 by John.

An estimator in statistics is a way of guessing a parameter based on data. An estimator is unbiased if over the long run, your guesses converge to the thing you’re estimating. Sounds eminently reasonable. But it might not be.

Suppose you’re estimating something like the number of car accidents per week in Texas and you counted 308 the first week. What would you estimate is the probability of seeing no accidents the next week?

If you use a Poisson model for the number of car accidents, a very common assumption for such data, there is a unique unbiased estimator. And this estimator would estimate the probability of no accidents during a week as 1. Worse, had you counted 307 accidents, the estimated probability would be -1! The estimator alternates between two ridiculous values, but in the long run these values average out to the true value. Exact in the limit, useless on the way there. A slightly biased estimator would be much more practical.

See Michael Hardy’s article for more details: An_Illuminating_Counterexample.pdf

Posted in Statistics | No Comments »

12 January 2008 by John.

Here are three quotes on originality I’ve read recently. I’ll lay them out first then discuss how I think they relate to each other. 

C. S. Lewis from The Weight of Glory, as quoted in a blog post by David Rogstad.

No man who values originality will ever be original. But try to tell the truth as you see it, try to do any bit of work as well as it can be done for the work’s sake, and what men call originality will come unsought.

Larry Wall, creator of Perl, in his talk Perl, the first postmodern programming language.

Modernism is also a Cult of Originality. It didn’t matter if the sculpture was hideous, as long as it was original. It didn’t matter if there was no music in the music. Plagiarism was the greatest sin. … The Cult of Originality shows up in computer science as well. For some reason, many languages that came out of academia suffer from this. Everything is reinvented from first principles (or in some cases, zeroeth principles), and nothing in the language resembles anything in any other language you’ve ever seen. And then the language designer wonders why the language never catches on. … In case you hadn’t noticed, Perl is not big on originality.  

Paul Graham in the introduction to Founders at Work.

People like the idea of innovation in the abstract, but when you present them with any specific innovation, they tend to reject it because it doesn’t fit with what they already know. … As Howard Aiken said, “Don’t worry about people stealing your ideas. If your ideas are any good, you’ll have to ram them down people’s throats.”

If you strive to be original, you might achieve it in some technical sense, but end up with something nobody cares about. Strive for authenticity and excellence and you’re more likely to do something valuable. But originality isn’t appreciated as much in practice as it is in theory.

Posted in Creativity | No Comments »