What does Nature’s Wikipedia study reveal?

This week Nature published a study on the reliability of Wikipedia in comparison to the Britannica encyclopaedia. 50 science articles were chosen from both encyclopaedias and send for “blind” peer review. The result from 42 returned usable reviews: Britannica turned up 123 errors, and Wikipedia 162.

In regards to this outcome, Nature titled: “Internet encyclopaedias go head to head.”

This is definitely a big success for Wikipedia and many have seen it in this way. But others interpret the study in a different way. The Register for example puts it this way: “Wikipedia science 31% more croncy than Britannica’s”. It points out that “there are errors and there are errors” and that no one should draw firm conclusions without a closer look at the reviews. The Register also suggests that Wikipedia will probably prove more unreliable in the fields of social science and culture.

These are excellent points and it would be too hasty to celebrate Wikipedia’s victory over Britannica. But the most staggering about this all is: why does Britannica turn out with 123 errors, an average of 3 errors per article? Does this mean Britannica is doing very bad, or does it only reflect that knowledge is difficult to grasp, constantly changing and more a matter of agreement than of cognition? If the latter would be the case, 162 “errors” in 42 articles wouldn’t be that bad.

And another point to think about: Presuming that knowledge is a product of peer production in the broadest sense, an encyclopaedia based on peer production might be the perfect tool to represent this knowledge. If it will be able to attract more and more qualified authors and get rid of those who deliberately vandalise the articles, future headlines might be something like this: “Wikipedia far ahead”.