Wednesday, February 28, 2007

Tilting at windmills

Tyler recently blogged about the new book "The Black Swan" by Nissim Taleb.
Coincidentally, the Economist recently carried a story that governments are now seriously working on preventing what could have been the greates Black Swan of all- an asteroid colliding with the Earth.

The preferred way to deal with rogue asteroids is still a Hollywood-style heroic space mission to knock the object off course, or possibly blow it up—although almost surely as an unmanned effort. However, Edward Lu, of NASA's Johnson Space Centre, has recently proposed an intriguing alternative, which he outlined to the meeting in San Francisco. The idea is to send a satellite to hover alongside a threatening asteroid and, using the gravitational attraction between the two objects, to pull the asteroid gradually off course. Even though the force acting on the asteroid would be extremely small, the cumulative effect over a week or so should be enough to do the trick.

NASA no plans to test any asteroid-deflection scheme, but the ESA does, and they have a sense of humor:

the European Space Agency does have such a project in the pipeline. With an irony the organisers have not, perhaps, properly appreciated, it is known as Don Quijote. The mission will have two craft, Hidalgo and Sancho. As in Cervantes's novel, the servant will observe while his master makes a fool of himself by crashing blithely into the target asteroid. If all such asteroids turn out to be harmless windmills, it will have been a wasted effort. But if one does, indeed, turn out to be a threatening giant, future generations may be very glad of the Don's headlong charge

Edge .org has an interesting article by Nissim Taleb- first published in the New York Times. As he puts it

A black swan is an outlier, an event that lies beyond the realm of normal expectations. Most people expect all swans to be white because that's what their experience tells them; a black swan is by definition a surprise. Nevertheless, people tend to concoct explanations for them after the fact, which makes them appear more predictable, and less random, than they are. Our minds are designed to retain, for efficient storage, past information that fits into a compressed narrative. This distortion, called the hindsight bias, prevents us from adequately learning from the past.

Black swans can have extreme effects: just a few explain almost everything, from the success of some ideas and religions to events in our personal lives. Moreover, their influence seems to have grown in the 20th century, while ordinary events — the ones we study and discuss and learn about in history or from the news — are becoming increasingly inconsequential.

Consider: How would an understanding of the world on June 27, 1914, have helped anyone guess what was to happen next? The rise of Hitler, the demise of the Soviet bloc, the spread of Islamic fundamentalism, the Internet bubble: not only were these events unpredictable, but anyone who correctly forecast any of them would have been deemed a lunatic (indeed, some were). This accusation of lunacy would have also applied to a correct prediction of the events of 9/11 — a black swan of the vicious variety.

He points out that

There is a silly book called A Millionaire Next Door, and one of the authors wrote an even sillier book called The Millionaire's Mind. They interviewed a bunch of millionaires to figure out how these people got rich. Visibly they came up with bunch of traits. You need a little bit of intelligence, a lot of hard work, and a lot of risk-taking. And they derived that, hey, taking risk is good for you if you want to become a millionaire. What these people forgot to do is to go take a look at the less visible cemetery — in other words, bankrupt people, failures, people who went out of business — and look at their traits. They would have discovered that some of the same traits are shared by these people, like hard work and risk taking. This tells me that the unique trait that the millionaires had in common was mostly luck.

Note that he is himself a multi-millionaire trader.
More than a year ago, Tyler Cowen blogged about Philip Tetlock's book "Expert Political Judgement", quoting the book

...no matter how unequivocal the evidence that experts cannot outpredict chimps or extrapolation algorithms, we should expect business to unfold as usual: pundits will continue to warn us on talk shows and op-ed pages of what will happen unless we dutifully follow their policy prescriptions. We -- the consumers of expert pronouncements -- are in thrall to experts for the same reasons that our ancestors submitted to shamans and oracles: our uncontrollable need to believe in a controllable world and our flawed understanding of the laws of chance. We lack the willpower and good sense to resist the snake oil products on offer. Who wants to believe that, on the big questions, we could do as well tossing a coin as by consulting accredited experts?

What Tyler wrote then applies here as well

Assume that the experts are usually wrong in their novel predictions. The consensus views of a science still might be worth listening to. Economists cannot forecast business cycles very well, but you should listen when they tell you that a deflationary shock is bad news. Each new forecast or new theory is an example of individual hubris and in expected value terms it is stupid. But the body of experts as a whole, over time, absorbs what is correct. A large number of predictions creates a Hayekian discovery process with increasing returns to scale. Social knowledge still comes out ahead, and in part because of the self-deceiving vanities put forward every day. You can find that point in Jonathan Swift's Gulliver's Travels.

Monday, February 26, 2007

Culture matters

From an article in the Economist about how different countries treat gifted children:

And in Japan there is a widespread belief that all children are born with the same innate abilities—and should therefore be treated alike. All are taught together, covering the same syllabus at the same rate until they finish compulsory schooling. Those who learn quickest are expected then to teach their classmates.

In China, extra teaching is provided, but to a self-selected bunch. “Children's palaces” in big cities offer a huge range of after-school classes. Anyone can sign up; all that is asked is excellent attendance.

This from a New York Times article on how Japanese companies deal with slowdowns

If cost-cutting is necessary in Japan, there is a pecking order, says Yoshi Tsurumi, an economist at Baruch College in Manhattan and a consultant to Japanese companies. Dividends are cut first, then salaries — starting at the top. Finally, there are layoffs — if attrition is not enough to shrink staff.

“The matter of flexibility is important,” Mr. Tsurumi said, “but the Japanese notion is to retrain and transfer people within an organization.”

Tuesday, February 20, 2007

Mirror, mirror on the wall..

Over at Chatty Apes, Jason Wheeden describes a very interesting finding.

Measurable symmetry accounts for less than 1% of the variance in the attractiveness of women's faces and less than 3% of the variance of the attractiveness of men's faces. Before we went and checked, we had both believed the widely circulated story that symmetry was a big deal in attractiveness.

Of course, I have come across this story myself, and believed it. One of the reasons why people believed this theory is that there is a plausible "explanation" for why people find symmetry attractive. This is exactly why scientists are supposed to replicate the findings of others. Feynman said that Science is what we have learned about how to keep from fooling ourselves. However, at least in this case, the problem was not that others did not try and replicate the results.

So how did it happen that so many of us believed that symmetry was a big deal in attractiveness judgments? There were studies that said so, obviously. But when we looked at the details, it turned out that the initial studies showing big effects typically involved samples of less than 20 faces each, which is irresponsibly small for correlational studies with open-ended variables. Once the bigger samples starting showing up, the effect basically disappeared for women and was shown to be pretty low for men. But no one believed the later, bigger studies, even most of their own authors -- pretty much everyone in my business still thinks that symmetry is a big deal in attractiveness.

So, the first lesson I learned: Small samples are dangerous. They're so dangerous that we need to force larger samples. How? My solution has been to ditch the old p<.05 significance standard. Right now, most social scientists allow themselves to call something a real finding if there is less than a 1 in 20 chance that it's from random noise. It's a standard that arose before computers, in a day when scientists ran their numbers by hand and so just didn't run very many numbers. These days, all you have to do to basically ensure at least one significant finding is to measure 7 variables -- the resulting correlation table has 21 correlations, and, just at random, you'd expect at least one to be p<.05. Look at my dissertation or any of my published work (two new ones are coming out soon), and you'll find that I'm using p<.005 -- a 1 in 200 standard. I don't think anyone has noticed -- I haven't had any reviewers comment on it at all. But what the .005 standard does is force larger samples, leading to more stable estimates, leading to more replicable results. So that was my first lesson from the health-attractiveness paper -- false positives are easy to get and powerfully resilient to correction, so we need better studies in the first place, which means bigger samples, in which case the only way to police it is using better significance criteria.

It appears that the site has been down for quite some time- I am relying on the Google Cache of the post, and I am quoting extensively in case the cache becomes unavailable. This is one reason I blog, to keep a copy of all the stuff I find interesting- just in case the source disappears. Redundancy.

HT: Robin Hanson at Overcoming Bias

Wednesday, February 14, 2007

The Perverse Power of Praise

Arnold Kling links to a fascinating article in New York Magazine on the effects of praise. The gist is that children respond to incentives. If they receive praise for being smart, they try to look smart, even at the cost of becoming underperformers, or cheats. If they receive praise for the effort they put in, they work harder and get smarter.

For the past ten years, psychologist Carol Dweck and her team at Columbia (she’s now at Stanford) studied the effect of praise on students in a dozen New York schools. Her seminal work—a series of experiments on 400 fifth-graders—paints the picture most clearly.

Dweck sent four female research assistants into New York fifth-grade classrooms. The researchers would take a single child out of the classroom for a nonverbal IQ test consisting of a series of puzzles—puzzles easy enough that all the children would do fairly well. Once the child finished the test, the researchers told each student his score, then gave him a single line of praise. Randomly divided into groups, some were praised for their intelligence. They were told, “You must be smart at this.” Other students were praised for their effort: “You must have worked really hard.”

Then

the students were given a choice of test for the second round. One choice was a test that would be more difficult than the first, but the researchers told the kids that they’d learn a lot from attempting the puzzles. The other choice, Dweck’s team explained, was an easy test, just like the first. Of those praised for their effort, 90 percent chose the harder set of puzzles. Of those praised for their intelligence, a majority chose the easy test. The “smart” kids took the cop-out.

Why did this happen? “When we praise children for their intelligence,” Dweck wrote in her study summary, “we tell them that this is the name of the game: Look smart, don’t risk making mistakes.” And that’s what the fifth-graders had done: They’d chosen to look smart and avoid the risk of being embarrassed.

In a subsequent round, none of the fifth-graders had a choice. The test was difficult, designed for kids two years ahead of their grade level. Predictably, everyone failed. But again, the two groups of children, divided at random at the study’s start, responded differently. Those praised for their effort on the first test assumed they simply hadn’t focused hard enough on this test. “They got very involved, willing to try every solution to the puzzles,” Dweck recalled. “Many of them remarked, unprovoked, ‘This is my favorite test.’ ” Not so for those praised for their smarts. They assumed their failure was evidence that they weren’t really smart at all. “Just watching them, you could see the strain. They were sweating and miserable.”

Having artificially induced a round of failure, Dweck’s researchers then gave all the fifth-graders a final round of tests that were engineered to be as easy as the first round. Those who had been praised for their effort significantly improved on their first score—by about 30 percent. Those who’d been told they were smart did worse than they had at the very beginning—by about 20 percent.

Dweck had suspected that praise could backfire, but even she was surprised by the magnitude of the effect. “Emphasizing effort gives a child a variable that they can control,” she explains. “They come to see themselves as in control of their success. Emphasizing natural intelligence takes it out of the child’s control, and it provides no good recipe for responding to a failure.”

In follow-up interviews, Dweck discovered that those who think that innate intelligence is the key to success begin to discount the importance of effort. I am smart, the kids’ reasoning goes; I don’t need to put out effort. Expending effort becomes stigmatized—it’s public proof that you can’t cut it on your natural gifts.

Repeating her experiments, Dweck found this effect of praise on performance held true for students of every socioeconomic class. It hit both boys and girls—the very brightest girls especially (they collapsed the most following failure). Even preschoolers weren’t immune to the inverse power of praise.

Dweck's theories were tried out at the Life Sciences Secondary School in East Harlem

Life Sciences is a health-science magnet school with high aspirations but 700 students whose main attributes are being predominantly minority and low achieving. Blackwell split her kids into two groups for an eight-session workshop. The control group was taught study skills, and the others got study skills and a special module on how intelligence is not innate. These students took turns reading aloud an essay on how the brain grows new neurons when challenged. They saw slides of the brain and acted out skits. “Even as I was teaching these ideas,” Blackwell noted, “I would hear the students joking, calling one another ‘dummy’ or ‘stupid.’” After the module was concluded, Blackwell tracked her students’ grades to see if it had any effect.

It didn’t take long. The teachers—who hadn’t known which students had been assigned to which workshop—could pick out the students who had been taught that intelligence can be developed. They improved their study habits and grades. In a single semester, Blackwell reversed the students’ longtime trend of decreasing math grades.

The only difference between the control group and the test group were two lessons, a total of 50 minutes spent teaching not math but a single idea: that the brain is a muscle. Giving it a harder workout makes you smarter. That alone improved their math scores.

This research suggests that the "Self-esteem" movement is correct: how children feel about themselves is a major determinant of how they do in life. Correct, but in an unexpected way.

Dweck and Blackwell’s work is part of a larger academic challenge to one of the self-esteem movement’s key tenets: that praise, self-esteem, and performance rise and fall together. From 1970 to 2000, there were over 15,000 scholarly articles written on self-esteem and its relationship to everything—from sex to career advancement. But results were often contradictory or inconclusive. So in 2003 the Association for Psychological Science asked Dr. Roy Baumeister, then a leading proponent of self-esteem, to review this literature. His team concluded that self-esteem was polluted with flawed science. Only 200 of those 15,000 studies met their rigorous standards.

After reviewing those 200 studies, Baumeister concluded that having high self-esteem didn’t improve grades or career achievement. It didn’t even reduce alcohol usage. And it especially did not lower violence of any sort. (Highly aggressive, violent people happen to think very highly of themselves, debunking the theory that people are aggressive to make up for low self-esteem.) At the time, Baumeister was quoted as saying that his findings were “the biggest disappointment of my career.”

Children respond to praise, but they take into account not only what they are being for (for being smart versus for trying hard) but also why. As with any currency, too much supply leads to devaluation.

Psychologist Wulf-Uwe Meyer, a pioneer in the field, conducted a series of studies where children watched other students receive praise. According to Meyer’s findings, by the age of 12, children believe that earning praise from a teacher is not a sign you did well—it’s actually a sign you lack ability and the teacher thinks you need extra encouragement. And teens, Meyer found, discounted praise to such an extent that they believed it’s a teacher’s criticism—not praise at all—that really conveys a positive belief in a student’s aptitude.

In the opinion of cognitive scientist Daniel T. Willingham, a teacher who praises a child may be unwittingly sending the message that the student reached the limit of his innate ability, while a teacher who criticizes a pupil conveys the message that he can improve his performance even further.

Cheap praise can even corrode character
Scholars from Reed College and Stanford reviewed over 150 praise studies. Their meta-analysis determined that praised students become risk-averse and lack perceived autonomy. The scholars found consistent correlations between a liberal use of praise and students’ “shorter task persistence, more eye-checking with the teacher, and inflected speech such that answers have the intonation of questions.”

Dweck’s research on overpraised kids strongly suggests that image maintenance becomes their primary concern—they are more competitive and more interested in tearing others down. A raft of very alarming studies illustrate this.

In one, students are given two puzzle tests. Between the first and the second, they are offered a choice between learning a new puzzle strategy for the second test or finding out how they did compared with other students on the first test: They have only enough time to do one or the other. Students praised for intelligence choose to find out their class rank, rather than use the time to prepare.

In another, students get a do-it-yourself report card and are told these forms will be mailed to students at another school—they’ll never meet these students and don’t know their names. Of the kids praised for their intelligence, 40 percent lie, inflating their scores. Of the kids praised for effort, few lie.

When students transition into junior high, some who’d done well in elementary school inevitably struggle in the larger and more demanding environment. Those who equated their earlier success with their innate ability surmise they’ve been dumb all along. Their grades never recover because the likely key to their recovery—increasing effort—they view as just further proof of their failure. In interviews many confess they would “seriously consider cheating.”

And, of course, there is a biological basis to the ability to persist.

Dr. Robert Cloninger at Washington University in St. Louis located the circuit in a part of the brain called the orbital and medial prefrontal cortex. It monitors the reward center of the brain, and like a switch, it intervenes when there’s a lack of immediate reward. When it switches on, it’s telling the rest of the brain, “Don’t stop trying. There’s dopa [the brain’s chemical reward for success] on the horizon.” While putting people through MRI scans, Cloninger could see this switch lighting up regularly in some. In others, barely at all.

What makes some people wired to have an active circuit?

Cloninger has trained rats and mice in mazes to have persistence by carefully not rewarding them when they get to the finish. “The key is intermittent reinforcement,” says Cloninger. The brain has to learn that frustrating spells can be worked through. “A person who grows up getting too frequent rewards will not have persistence, because they’ll quit when the rewards disappear."

The article does not claim there are no innate differences between children. People have traditionally worried about how labelling can harm ordinary children when they compare themselves to the gifted, but even gifted children suffer from excessive emphasis on ability rather than effort.

Friday, February 09, 2007

Prizes for all

The New York Times recently published an article on how prizes could be used to encourage innovation. Prizes were common in the 18th century.

Most famously, the British Parliament offered the £20,000 longitude prize to anyone who figured out how to pinpoint location on the open sea.

Times changed.

Eventually, though, prizes began to be replaced by grants that awarded money upfront. Some of this was for good reason. As science became more advanced, scientists often needed to buy expensive equipment and hire a staff before having any chance of making a discovery.

But grants also became popular for a less worthy reason: they made life easier for the government bureaucrats who oversaw them and for the scientists who received them. Robin Hanson, an economist at George Mason University who has studied the history of prizes, points out that they create a lot of uncertainty — about who will receive money and when a government will have to pay it. Grants, on the other hand, allow a patron (and the scientists advising that patron) to choose who gets the money. “Bureaucracies like a steady flow of money, not uncertainty,” said Mr. Hanson, who worked as a physicist at NASA before becoming an economist. “But prizes are often more effective if what you want is scientific progress.”

Recently, though, prizes have made a comeback in the private sector- the article describes two: a million-dollar prize which Netflix is offering for any system that is at least 10 percent better than their own system (Cinematch) at predicting how many stars someone would give a movie, and the X Prize, which was created in 1996 as a $10 million purse for the first private manned flight into space. The latest prediction model is already 6.75% better than Cinematch, and there are 4.5 years to go before the prize expires. The X-prize was awarded just eight years after it was announced.
The advantages of prizes are two-fold:

They pay for nothing but performance, and they ensure that anyone with a good idea — not just the usual experts — can take a crack at a tough problem. Much to the horror of the leading astronomers of the day, a clockmaker ultimately claimed the longitude prize

Another day, another prize.

Richard Branson and Al Gore announced today a $25 million prize for the best way to remove significant amounts of carbon dioxide from the atmosphere.

The key insight is that the best way to handle a problem is not to impose a solution, but to provide people with incentives to come up with their own solutions.

Thursday, February 08, 2007

The psychology of power

Anders Sandberg blogs on new research that explores just how being in a position of power changes how we deal with others. The bottomline

it was found that high-power subjects also tended to assume other people had the same information that they had (the "telepathic boss" problem - the boss assumes that everybody knows what he knows and want). They were also less accurate than low-power subjects at judging emotional expressions. There were also anticorrelations between reports of general feelings of being in power in one's life and tendency to take other's perspective. Overall high-power people seem to anchor too heavily on their own vantage point and this impairs their ability to consider what others see, think and feel.

And

In general power bias would make the empowered people tend to think they have more support from others in their views than they have. Altruists in power would be even less concerned with individual variations in goals and values - i.e. they would tend to become more egalitarian and paternalist. Egoists in power would become more concerned about the ambitions of others, i.e. paranoid.

Thus, even altruists are corrupted- probably assuming that they are anyway doing it "for the good of all".

Perception

I have been reading "Stumbling on Happiness" by Daniel Gilbert. Its a good as they say it is.Two experiments on how our brains process sensory information jumped out at me.

Researchers tape-recorded the sentence 'The state governors met with their respective legislatures convening in the capital city'. Then they doctored the tape, substituting a cough for the first 's' in 'legislatures'. Volunteers heard the cough alright, but they heard it happening between the words- they heard the missing 's' too. Even when they were specifically instructed to listen for the missing sound, and even when they were given thousands of trials of practice, volunteers were unable to name the missing letter that their brains knew ought to be there and had thus helpfully supplied.

I wonder what would have happened if two coughs had been introduced- one overlaying the first 's' (as before)- and one in between the words (where the brain shifted the cough so we would "hear" the 's' that had been obscured).

Even cooler was this one

Volunteers listened to a recording of the word 'eel' preceded by a cough (which I will denote with *). The volunteers heard the word 'peel' when the it was embedded in the sentence 'The *eel was on orange', but the heard the word 'heel' when it was embedded in the sentence 'The *eel was on the shoe'. This is a striking finding because the two sentences differ only in their final word, which means that volunteers' brains had to wait for the final word of the sentence before they could supply the information that was missing from the second word. But they did it, and they did it so smoothly and quickly that volunteers actually heard the missing information being spoken in its proper position.

Lots more.

The case for limited Democracy

In this post, Chris Dillow argues that there is a place for non-representative institutions such as the House of Lords. He claims that

Representative democracy, then, would drive out virtuous people

and that

Maybe it shows the difference between political and market institutions. Market institutions are a way of ensuring that vices work for the public good, so bad men can do good things. Political institutions, however, have an adverse selection effect, ensuring that virtuous men can do no good.

An interesting argument, coming from someone on the Left.

Last night, I was watching a PBS documentary on one such non-representative institution: the Supreme Court of the United States. This episode was about how the Supreme Court stepped in to massively expand the rights of American individuals in the second half of the 20th century. What struck me was just how undemocratically the Supreme Court acted in most of these cases. In Brown v. Board of Education and Cooper v. Aaron, the Court unanimously ruled that segregation in public schools was unconstitutional. Segregation was hugely popular among the white majority in the Southern United States. The laws which were the basis of segregation were passed by democratically elected legislatures, and enthusiastically enforced by governments which were accountable to the electorate. In Miranda v. Arizona, the Court ruled that

detained criminal suspects, prior to police questioning, must be informed of their constitutional right to an attorney and against self-incrimination.

The Republicans argued that such laws needlessly tied the hands of the police, and this was one of the planks on which Ronald Reagan won the Presidency. These were cases in which one branch of the Government had to step in and restrain another branch, and the only reason the Court could do this was that they were not accountable to the electorate.
Why did the Legislature introduce these laws in the first place? Part of the answer may have been that they could thus pander to the prejudices of their electorate, while incurring none of the cost. Sometime back, Don Boudreax quoted an article by Thomas Sowell that suggests that this may have been true in the case of "Jim Crow" laws regarding public transport.

It was politics that segregated the races because the incentives of the political process are different from the incentives of the economic process. Both blacks and whites spent money to ride the buses but, after the disenfranchisement of black voters in the late 19th and early 20th century, only whites counted in the political process.

Katie blogged on the same article

Private owners of streetcar, bus, and railroad companies in the South lobbied against the Jim Crow laws while these laws were being written, challenged them in the courts after the laws were passed, and then dragged their feet in enforcing those laws after they were upheld by the courts.

Unconstrained democracy has a tendency to descend into demagoguery. To oversimplify, democracy is a tool to check the power of governments by making them accountable to the populace. A constitution is (among other things) a tool for checking the power of the people to use their elected government to tyrannize minorities.