Mathematical analysis of diversity
We’ve all heard of the wisdom of the crowd – for example, when guessing the weight of an ox, multiple people may be off from the true value in their independent guesses, but the average of all the guesses is very close to the true value. This is frequently used to show that multiple approaches to a problem is good for problem solving. But wouldn’t a group of professionally-trained oxen-weighers be better at arriving at the right answer? A paper published in 2004, and a book published this year, show mathematically the conditions under which that may not be true.
Groups of diverse problem solvers can outperform groups of high-ability problem solvers
Lu Hong and Scott E. Page
Proceedings of the National Academy of Sciences
November 16, 2004, vol. 101 no. 46, p. 16385–16389
The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies
by Scott E. Page
Princeton University Press, 2008, 448 pp.
In the paper, Hong and Page constructed a formal model showing mathematically that diversity can trump ability, and the conditions under which it does. Their model is similar to those that predict financial markets and voting patterns. What the model showed was that diverse groups of problem solvers outperformed the groups of the best individuals at solving problems. The reason is that the diverse groups got stuck less often than the smart individuals, who tended to think similarly. They also show how when making predictions, a group’s errors depend in equal parts on the ability of its members to predict and their diversity. This can be expressed as: collective accuracy = average accuracy + diversity. In the book, Page round out the mathematical model with game theory and experimental findings to give a more popular (readable) analysis of why diversity matters.
At GSA a couple of weeks ago, I was talking with a member of the faculty from my previous institution, who is now involved in an NSF ADVANCE activity to help increase the number of females in science in the state’s institutions. I sent him this article and I hope that it is helpful. Often, the final decision in hiring comes down to what the committee thinks is “a good fit,” where familiar = comfortable = not diverse. Sometimes this masquerades as “All other things being equal, we just think x person is a better fit.” What this research shows is that diversity of approach and experience is a merit, and may (should?) be considered an additional, positive factor when a hiring committee is looking for the best person to strengthen a department.
(Some material adapted from a NYT interview with Scott Page).