23 January 2008 by John.

The following excerpt is a quote from Edsgar Dijkstra.

I think it was in 1970 that I gave my first talk in a foreign country on the design of programs that you could actually control and prove were correct. … The talk fell completely on its face. … The programmers didn’t like the idea at all because it deprived them of the intellectual excitement of not quite understanding what they were doing. They liked the challenge of chasing the bugs.

Taken from Out of their Minds: The Lives and Discoveries of 15 Great Computer Scientists. Emphasis added.

Posted in Computing | No Comments »

22 January 2008 by John.

C# verbatim strings and PowerShell here-strings have just enough in common to be confusing. The differences are summarized here.

Posted in Computing | No Comments »

21 January 2008 by John.

In his book The Nine, Jeffrey Toobin gives a few details of Supreme Court Justice David Souter’s decidedly low-tech life. Souter has no cell phone or voice mail. He does not use email. He was given a television once but never turned it on. He moves his chair around his office throughout the day so he can read by natural light. Toobin says Souter lives something like an eighteenth century gentleman.

I find it refreshing to read of someone like Justice Souter with such independence of mind that he chooses not to use much of the technology that our world takes for granted. I also find it encouraging to see examples of people who do not reject electronics entirely but selectively decide how and whether to use them.

I once had the opportunity to see a talk by the father of computer typography, Donald Knuth. Much to my surprise, his slides were handwritten. The author of TeX didn’t see the need to use TeX for his slides. While he cares about the fine details of how math looks in print, he apparently didn’t feel it was worth the effort to typeset his notes for an informal presentation.

I also think of the late computer scientist Edsgar Dijkstra who wrote with pen and ink, even when writing computer programs.

If you’re reading legal briefs by sunlight, your thoughts will not be exactly the same as they would be if you were reading by fluorescent light. If you’re writing computer programs by hand, you’re not going to think the same way you would if you are pecking on a computer keyboard. (And if you do use a computer, your thinking is subtlety different depending on what program you use.) Technology effects the way you think. The effect is not uniformly better or worse, but it is certainly real.

To shake up your thinking, try going low-tech for a day.

Posted in Creativity, Computing | No Comments »

21 January 2008 by John.

During WWII, statistician Abraham Wald was asked to help the British decide where to add armor to their bombers. After analyzing the records, he recommended adding more armor to the places where there was no damage!

This seems backward at first, but Wald realized his data came from bombers that survived. That is, the British were only able to analyze the bombers that returned to England; those that were shot down over enemy territory were not part of their sample. These bombers’ wounds showed where they could afford to be hit. Said another way, the undamaged areas on the survivors showed where the lost planes must have been hit because the planes hit in those areas did not return from their missions.

Wald assumed that the bullets were fired randomly, that no one could accurately aim for a particular part of the bomber. Instead they aimed in the general direction of the plane and sometimes got lucky. So, for example, if Wald saw that more bombers in his sample had bullet holes in the middle of the wings, he did not conclude that Nazis liked to aim for the middle of wings. He assumed that there must have been about as many bombers with bullet holes in every other part of the plane but that those with holes elsewhere were not part of his sample because they had been shot down.

Posted in Statistics | No Comments »

19 January 2008 by John.

Imagine this conversation with your doctor:

Your poor tumor. It has a chaotic blood supply. Parts of it get too much blood, other parts too little. We’re going to give you a drug to improve your tumor’s blood supply, making it healthier.

Before you run screaming from your doctor’s office, see if there’s a copy of the January 2008 issue of Scientific American in the waiting room. If there is, read the article Taming Vessels to Treat Cancer by Rakesh Jain.

Just as the cells in a tumor are abnormal and growing out of control, so are the blood vessels that feed the tumor. This lack of proper infrastructure inhibits the tumor’s growth, but it also makes it difficult to deliver chemotherapy to the tumor. This lead to the radical idea to make the tumors healthier in preparation for killing them.

So how would you go about improving a tumor’s circulatory system? By administering a drug that was designed to attack tumor vessels!

A new class of cancer drugs, antiangiogenic agents, has been designed to attack tumors by cutting off their blood supply. These agents haven’t been a complete success. Experience with one such agent, Avastin, shows that while it shuts down some of the blood vessels in tumors, it may make the remaining tumor vessels healthier. That’s bad news if you’re treating patients with Avastin alone. But when used in combination with chemotherapy, it’s just what people like Dr. Jain were looking for: a way to normalize the blood flow in a tumor in order to make it more vulnerable to chemotherapy.

More information, including videos, is available at the web site of Dr. Jain’s lab.

Posted in Science | No Comments »

19 January 2008 by John.

Hospitals often paint clowns on the walls of the pediatric wing assuming children like them. When someone finally asked kids whether they like clowns, they found that not one out of 255 kids questioned did.

See No Clowning for Hospitalized Kids for details.

Posted in Uncategorized | No Comments »

18 January 2008 by John.

Seth Godin has an interesting blog post today called The problem with perfect. Companies with a reputation for perfect service are only remarkable when they disappoint. Being interesting is a more viable business strategy than being perfect.

Posted in Business | No Comments »

18 January 2008 by John.

Bart Kosko in his book Noise argues is that thick-tailed probability distributions such as the Cauchy distribution are common in nature. This is the opposite of what I was taught in college. I remember being told that the Cauchy distribution, a distribution with no mean or variance, is a mathematical curiosity more useful for constructing academic counterexamples than for modeling the real world. Kosko disagrees. He writes

… all too many scientists simply do not know that there are infinitely many different types of bell curves. So they do not look for these bell curves and thus they do not statistically test for them. The deeper problem stems from the pedagogical fact that thick-tailed bell curves get little or no attention in the basic probability texts that we still use to train scientists and engineers. Statistics books for medicine and the social sciences tend to be even worse.

We see thin-tailed distributions everywhere because we don’t think to look for anything else. If we see samples drawn from a thick-tailed distribution, we may throw out the “outliers” before we analyze the data, and then a thin-tailed model fits just fine.

How do you decide what’s an outlier? Two options. You could use your intuition and discard samples that “obviously” don’t belong, or you could use a formal test. But your intuition may implicitly be informed by experience with thin-tailed distributions, and your formal test may also circularly depend on the assumption of a thin-tailed model.

Posted in Statistics, Math | No Comments »

17 January 2008 by John.

Here’s a web site where you can type in some TeX code, click a button, and get back a GIF with a transparent background. Handy for pasting equations into HTML.

http://www.artofproblemsolving.com/LaTeX/AoPS_L_TeXer.php

For example:

Posted in Math, Computing | 1 Comment »

17 January 2008 by John.

Everything is supposedly growing exponentially these days. But when most people say “exponential,” they don’t mean what they say. They mean “fast.” Exponential growth can indeed be fast. Or it can be slow. Excruciatingly slow.

If you earn a million dollars a day, your wealth is growing quickly, but is not exponentially. And if you have $100 in the bank earning 3% compound interest, you’re money is growing slowly, but it is growing exponentially.

Linear growth is a constant amount of increase per unit of time. Exponential growth is a constant percentage increase per unit of time. If you buy a pack of baseball cards every Friday, the size of your baseball card collection will grow linearly. But if you breed rabbits with no restriction, the size of your bunny heard will grow exponentially.

It matters a great deal whether you’re growing linearly or exponentially.

When you start a new enterprise — a company, a web site, etc. — it may truly grow exponentially. Growth may be determined by word of mouth, which is exponential (at first). The number of new people who hear each month depends on the number people who talk, and hearers become talkers. But that process can be infuriatingly slow when it’s just getting started. If the number of visitors to your web site is growing 5% per month, that’s great in the long term, but disappointing at first when it means going from 40 visitors one month to 42 the next.

How do you live on an exponential curve? You need extraordinary patience. While any exponential curve will eventually pass any linear curve, it may take a long time. If you’re making barely perceptible but compounding progress, be encouraged that you’re on the right curve. Eventually you’ll have all the growth you can handle. Realize that you may be having a harder time initially because you’re on the exponential curve rather than the linear curve.

How do you know whether you’re on an exponential curve? This is not as easy as it sounds. Because of random noise, it may be hard to tell from a small amount of data whether growth is linear or exponential, or even to tell growth from stagnation. Eventually the numbers will tell you. But until enough data come in to reveal what’s going on, look at the root causes of your growth. If you’re growing because customers are referring customers, that’s a recipe for exponential growth. If you’re growing because you’re working more hours, that’s linear growth.

Nothing grows exponentially forever. Word of mouth slows down when the message reaches saturation, when the talkers run into fewer people who haven’t heard. Rabbit farms slow down when they can’t feed all the rabbits. Most of the things we call exponential growth are more accurately logistic growth: exponential growth slows to linear growth, then linear growth begins to plateau.

How do you live on a logistic curve? Realize that initial exponential growth doesn’t last. Watch the numbers. They’ll tell you when you’ve gone from approximately exponential to approximately linear. Understand the mechanisms that turn exponential into logic growth in your context.

Posted in Business, Math | No Comments »

16 January 2008 by John.

Independence in probability can be both intuitive and mysterious. Intuitively, two events are independent if they have nothing to do with each other. Suppose I ask you to guess whether the next person walking down the street is left handed. I stop this person, and before I ask which hand he writes with, I ask him for the last digit of his phone number. He says it’s 3. Does knowing the last digit of his phone number make you change your estimate of the chances this stranger is a south paw? No, phone numbers and handedness are independent. Presumably about 10% of right-handers and the same percentage of left-handers have this distinction. Even if the phone company is more likely or less to assign numbers ending in 3, there’s no reason to believe they take customer handedness into account when handing out numbers. On the other hand, if I tell you the stranger is an artist, that should change your estimate: a disproportionate number of artists are lefties.

Formally, two events A and B are independent if P(A and B) = P(A) P(B). This implies that P(A | B), the probability of A happening given that B happened, is just P(A). Similarly P(B | A) = P(B).  Knowing whether or not one of the events happened tells you nothing about the likelihood of the other. Knowing someone’s phone number doesn’t help you guess which hand they write with, unless you use the phone number to call them and ask about their writing habits.

Now lets extend the definition to more events. A set of events is mutually independent if the probability of any subset of two or more events is the product of the probabilities of each event separately.

So, let’s look at three events: A, B, and C. If we know P(A and B and C) = P(A) P(B) P(C), are the three events mutually independent? Not necessarily. It is possible for the above equation to hold and yet P(A and B) is not equal to P(A) P(B). The definition of mutual independence requires something of every subset of {A, B, C} with two or more elements, not just the subset consisting of all elements. So we have to look at the subsets {A, B}, {B, C}, and {A, C} as well.

What if A and B are independent, B and C are independent, and A and C are independent? In other words, every pair of events is independent. Is that enough for mutual independence? Surprisingly, the answer is no. It is possible to construct a simple example where

• P(A and B) = P(A) P(B)
• P(B and C) = P(B) P(C)
• P(A and C) = P(A) P(C)

and yet P(A and B and C) does not equal P(A) P(B) P(C).

There are no short cuts to the definition of mutual independence.

Posted in Math | No Comments »

16 January 2008 by John.

Jonathan Schwartz announced on his blog today that Sun is acquiring the company behind MySQL.

Posted in Computing | No Comments »

16 January 2008 by John.

Here’s a great quote from The Logic of Failure on obscuring complexity.

By labeling a bundle of problems with a single conceptual label, we make dealing with that problem easier — provided we’re not interested in solving it. … A simple label can’t make the complex nature of the problem go away, but it can so obscure complexity that we lose site of it. And that, of course, we find a great relief.

Posted in Creativity | No Comments »

15 January 2008 by John.

In 1967, Stanley Milgram conducted the famous experiment that told us there are six degrees of separation between any two people. He gave letters to 160 people in Nebraska and asked them to pass them along to someone who could eventually get the letters to a particular stock broker in New York. It took about six links for each letter to get to the stock broker.

In 2001, Duncan Watts repeated the experiment, asking 61,000 people to forward emails to eventually reach 18 targets worldwide. The emails took roughly six hops to reach their targets, giving Milgram’s original conclusion more credibility due to the larger sample.

But a secondary conclusion of Milgram didn’t hold up. In Milgram’s experiment, half of the letters reached the target via the same three friends of the stock broker. These people were deemed “hubs.” In Watts’s experiment, only 5% of messages reached their targets via hubs. Watts’s thesis is that while hubs exist — some people are far more connected than others — they’re not as important in spreading ideas as once supposed.

See “Is the Tipping Point Toast?” in the February 2008 issue of Fast Company for more on Duncan Watts and his skepticism regarding the importance of hubs.

Posted in Uncategorized | No Comments »

15 January 2008 by John.

Sweave, mentioned in my previous post, is a tool for literate programming. Donald Knuth invented literate programming and gives this description of the technique in his book by the same name:

I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: “Literate Programming.”

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other.

Knuth says the quality of his code when up dramatically when he started using literate programming. When he published the source code for TeX as a literate program and a book, he was so confident in the quality of the code that he offered cash rewards for bug reports, doubling the amount of the reward with each edition. In one edition, he goes so far as to say “I believe that the final bug in TeX was discovered and removed on November 27, 1985.” Even though TeX is a large program, this was not an idle boast. A few errors were discovered after 1985, but only after generations of Stanford students studied the source code carefully and multitudes of users around the world put TeX through its paces.

While literate programming is a fantastic idea, it has failed to gain a substantial following. And yet Sweave might catch on even though literate programming in general has not.

In most software development, documentation is an after thought. When push comes to shove, developers are rewarded for putting buttons on a screen, not for writing documentation. Software documentation can be extremely valuable, but it’s most valuable to someone other than the author. And the benefit of the documentation may only be realized years after it was written.

But statisticians are rewarded for writing documents. In a statistical analysis, the document is the deliverable. The benefits of literate programming for a statistician are more personal and more immediate. Statistical analyses are often re-run, with just enough time between runs for the previous work to be completely flushed from term memory. Data is corrected or augmented, papers come back from review with requests for changes, etc. Statisticians have more self-interest in making their work reproducible than do programmers.

Patrick McPhee gives this analysis for why literate programming has not caught on.

Without wanting to be elitist, the thing that will prevent literate programming from becoming a mainstream method is that it requires thought and discipline. The mainstream is established by people who want fast results while using roughly the same methods that everyone else seems to be using, and literate programming is never going to have that kind of appeal. This doesn’t take away from its usefulness as an approach.

But statisticians are more free to make individual technology choices than programmers are. Programmers typically work in large teams and have to use the same tools as their colleagues. Statisticians often work alone. And since they deliver documents rather than code, statisticians are free to use use Sweave without their colleagues’ knowledge or consent. I doubt whether a large portion of statisticians will ever be attracted to literate programming, but technological minorities can thrive more easily in statistics than in mainstream software development.

Posted in Statistics, Computing | No Comments »