The Reliability of User-Created Web Content: A Wikipedia Case Study
Cats: School, Internet|Editor’s Note: This is a research paper I’m turning in today for my Interactive Media Concepts Course (CIM 130) at Johnson County Community College. (A special thank-you to Wikipedia for inspiring my topic!) Enjoy!
Development of websites that enable its users to create and control the site content has brought about a wide range of social consequences. The community aspect of such sites creates a boon to communication and a thrill of networking with others to produce something new. However, user-controlled websites can also have disruptive consequences to online and offline society. For example, user-contributed sites such as YouTube.com and MySpace.com are now such popular resources for young people to search for new music that it is increasingly difficult for companies like Billboard to determine the most popular song for any given week (Kaufman). The so-called “Digg Effect” created by the social news website Digg.com can cripple individual web servers as thousands of users flock simultaneously to see the latest site linked by another user (“Digg”).
None of these can compare, however, to the difficulties presented by the online free encyclopedia, Wikipedia.org. While courts, journalists, and at least one peer-reviewed journal article (Burris)—not to mention online writers of every persuasion—regularly reference articles from the site, not everyone is so enamored with the website. Most notably, educators have been scrambling to ensure that students are not directly citing the website as a source for research papers (Torbati).
Wikipedia.org was founded upon the concept of the ‘wiki,’ which is defined as “a collaborative website whose content can be edited by anyone who has access to it.”(“wiki”). The site is a form of open-source online encyclopedia boasting millions of entries covering topics ranging from AA batteries to the band ZZ Top (Giles). Often the articles provide information about topics that are too recent to appear in print media (such as the aforementioned ‘Digg Effect’), or popular culture subjects that may not be discussed in-depth by more formal resources such as peer-reviewed journals.
The controversy surrounding Wikipedia centers on the manner in which content is created and controlled by the users of the site. The website is written and edited entirely by volunteers—called ‘wikipedians’—and its policy allows anyone to change the content on almost every page of the site (Giles). The only reason the website does not fall into a chaotic array of lies and misinformation is the number of wikipedians who take it upon themselves to police the site: they remove false, unsubstantiated, and even downright libelous content as soon as possible after it has been submitted. Despite their efforts, the lag time between posting content and correcting it has created some unfortunate side effects. For example, an article on Wikipedia suggested that a friend of the Kennedy family was suspected in the assassinations of John and Robert Kennedy. The friend in question happened to be a former editor of the USA Today editorial page, who wrote a scathing commentary on the rights of Internet users to remain anonymous, “even those who defame online” (Seigenthaler). Another Wikipedia article was written by a man who intentionally fabricated information about a non-existent sexual fetish relating to turtles (Torbati). While both falsehoods were corrected as soon as the editors were made aware of them, the damage was done. Most ironically, information about the fake fetish can still be found all over the Internet, all citing the original spurious article on Wikipedia.org (ibid.).
It is sometimes difficult to pin down the reliability of a source. Most accomplished researchers know a reliable source when they see one, but careful research is required to find a standard for the reliability of online source material. The Internet has posed problems for researchers for some time now. Writing in 1997, four years before Wikipedia was founded, Saliba and Shoemaker lamented the difficulties facing researchers searching the Internet:
In many cases, the criteria used to evaluate web resources is the same that is used to judge reference materials, but the very nature of the internet makes evaluating it as a reference source difficult and time-consuming. Many sites don’t list the author, let alone the sources used to compile the page or the organization that sponsors it…It is certainly true that we demand more from the internet than it is able to provide, so “good” sites are simply the best we can find on a needed subject at the time (Saliba and Shoemaker).
Such a sentiment may prohibit use of Internet resources for all serious research projects. However, the World Wide Web was still young in 1997, and many respected news outlets and peer-reviewed journals did not have a major online presence until more recently.
Scholars have taken great pains to establish ground rules for determining the quality of the information found on the Internet, and encouraging what is called ‘information literacy’ among students. As the American Library Association’s Presidential Committee on Information Literacy stated, “to be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information” (American).Information literacy is cited by Heil as the goal of her study in “teaching students about ways to evaluate Internet sites … [to help students] make more informed decisions about how they use the Internet” (Heil). Roger Riddell, online editor of eSchoolNews.com, quotes blogger and former teacher Tom Hoffman, noting that “students should be taught not to rely too much on a single source and to cross-reference sources against each other … Students should be careful to check the editing history of entries posted on sources such as Wikipedia” (Riddell). Collins outlined six criteria for evaluating content on the Internet: Content, Authority, Currency, Organization, Search Engine, and Accessibility. He recommended students ask themselves:
- Is the material at this site useful, unique, accurate, or is it derivative, repetitious, or doubtful?
- [What is the] credibility of the person(s) responsible for developing/providing the material?
- Is the site updated on a regular basis?
- Is the information easy to get to?
- Is the site available on a consistent basis? (Collins)
Certainly these questions are helpful in determining the quality of the information, but Carolyn Caywood points out that “few sites meet all criteria, so the benefits must be weighed against the lacks”(qtd. in Saliba and Shoemaker). Such bending of the traditional rules indicates a large step toward educators accepting online resources as good tertiary or even secondary sources of research information.
Wikipedia founder Jimmy Wales stated in a BusinessWeek Online interview that his goal for the online encyclopedia is to be “Britannica or better quality…we don’t reach that quality yet—we know that. We’re a work in progress” (Helm). Indeed, the very nature of the site is ‘a work in progress,’ but there are indicators that the wiki format is gaining further acceptance in the general population, and even among educators. McPherson cites the high readability level of Wikipedia’s articles (“Literacy”), and even recommends teachers use student-created wikis to promote student writing:
By integrating this new writing context into the school library curriculum, we validate the use of wikis in our students’ lives [and] open up opportunities to help students become motivated and powerful writers…Far from being the demise of the written word,…wikis may very well be the literacy technology that helps many of our current students learn to be effective writers (“Student”).
Since the concept of wikis is gaining support in education, it stands to reason that educators will become more familiar with the strengths and weaknesses of Wikipedia as a resource.
The journal Nature led “an expert-led investigation” in 2005 comparing the accuracy of Wikipedia science articles to the corresponding articles in Encyclopedia Britannica, and found that “the difference in accuracy was not particularly great”(Giles). Giles states: “The exercise revealed numerous errors in both encyclopaedias, but … the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three” (ibid.). The article goes on to note that “Wikipedia’s strongest suit is the speed at which it can [be] updated” (ibid.). If wikipedians continue their quest to impart accurate, up-to-date, unbiased, and free content to the world, it won’t be long before they find the technical means of achieving a level of reliability that will meet more and more of the information needs of scholars.
Those means are becoming evident even now. A recent Wikipedia ‘scandal’ has emerged, with Microsoft offering to pay an editor to keep a watchful eye on specific articles on Wikipedia “to fix purported inaccuracies in technical articles” (Bergstein). While Wikipedia saw this as a violation of its anti-bias policy, another similar arrangement may be in order to keep the articles error-free. Who better to check for errors than the experts themselves? After all, peer-reviewed journals have the same sort of error-checking by experts in a particular field.
In conclusion, Wikipedia.org will likely continue to provide information that is ‘good enough’ for general inquiry. However, without significant changes to its policy, Wikipedia’s avoidance of the perception of bias in its articles will prevent errors from being corrected by editors who may be experts in the subject but have an apparent bias. This lack of error checking will in turn keep scholars from trusting the site as a reliable source of information. The education establishment has already begun to change its collective mind about finding research information on the Internet in general, and the use of wikis in particular. The onus is on Wikipedia to find a way to peer-review its articles while keeping bias out of the picture, thereby finally producing a reliable online resource destination.
Works Cited
American Library Association. Presidential Committee on Information Literacy: Final Report. 10 January, 1989. Association of College & Research Libraries. 19 February 2007
Bergstein, Brian. “Idea of Paid Entries Roils Wikipedia.” USAToday.com 25 January 2007. 10 February 2007
Burris, Lilisha, contact. “Interpretation in Practice.” Canadian Journal of Dietetic Practice and Research 67.4 (Winter 2006): 3S. ProQuest Electronic full-text PDF. 15 February 2007
Cohen, Noam. “Courts Turn to Wikipedia, but Selectively.” NYTimes.com 29 January 2007. 8 February 2007
Collins, Boyd R. “Beyond Cruising: Reviewing.” Library Journal 121.3 (15 February 1996): 122-124. ProQuest Electronic full-text PDF. 15 February 2007
“The Digg Effect.” CBC News 20 December 2006. 08 February 2007
Giles, Jim. “Internet Encyclopaedias Go Head to Head.” Nature 438.7070 (15 December 2005): 900-901. ProQuest Electronic full-text PDF. 15 February 2007
Heil, Delilah. “The Internet and Student Research: Teaching Critical Evaluation Skills.” Teacher Librarian 33.2 (December 2005): 26-29. ProQuest Electronic full-text PDF. 15 February 2007
Helm, Burt. “Wikipedia: ‘A Work in Progress.’” BusinessWeek Online 14 December 2005. 10 February 2007
Kaufman, Gil. “Who’s Really #1? iTunes, MySpace Throwing Music Charts For A Loop.”
MTV.com 1 February 2007. 8 February 2007
McPherson, Keith. “Wikis and Literacy Development.” Teacher Librarian 34.1 (October 2006): 67-69. ProQuest Electronic full-text PDF. 15 February 2007
McPherson, Keith. “Wikis and Student Writing.” Teacher Librarian 34.2 (December 2006): 70-72. ProQuest Electronic full-text PDF. 15 February 2007
Riddell, Roger. “Wikis Test Students’ Research Skills.” eSchool News Online 25 January 2006. 21 February 2007
Saliba, Elizabeth, and Shoemaker, Kellie. “Brave the Wave: Using the Internet for Student Research.” The Universe at Your Fingertip: Continuing Web Education.
Seigenthaler, John. “A False Wikipedia ‘Biography.’” Editorial. USAToday.com 29 November 2005. 21 February 2007
Torbati, June. “Profs Question Students’ Wikipedia Dependency.” YaleDailyNews.com 7 February 2007. 8 February 2007
“wiki.” The American Heritage Dictionary of the English Language, Fourth Edition. Houghton Mifflin Company, 2004. 08 Feb. 2007. Dictionary.com.







February 22nd, 2007 at 11:16 am
you should have listed wikipedia as one of your “works cited.”
it would have been classic.
February 22nd, 2007 at 11:22 am
It really would have been classic. I actually had to go out of my way *not* to cite Wikipedia for anything, including its own article count, policies, etc. I was trying to beg the question without being infinitely ironic.
February 22nd, 2007 at 9:22 pm
hmm i wonder if i could do a wiki on myself?
February 22nd, 2007 at 9:22 pm
sometimes when i do a googel search i will hit a wiki ref, read it, then look at the other searh results and low and behold i will find a word for word article at say Ask.com or some other site. wtf?
yeah, i site it all the time in the blog cause it is just easy. but i don’t really trust much of what i find there. i am all for spreading bad info, i just don’t want to take it in myself.
February 23rd, 2007 at 11:21 am
you can’t. i tried to wiki “cade factor” and it wouldn’t let me.
maybe we can have a go and setting each OTHER up. that might work.
February 23rd, 2007 at 2:24 pm
1) Often other reference sites will cite Wikipedia (usually word-for-word) because they’re too lazy to come up with their own stuff. On the other hand, Wikipedia does what it calls ’scraping’ of other sites the editors find particularly good information about a subject. So if they were looking to put together an article on the ‘cade factor’ they might do their own google search and find Mike’s discussion of the cade factor, and if they liked it, they would scrape it onto Wikipedia and cite the original source…But someone would have to be interested in the cade factor; it would have to come into common parlance as a phrase or concept distinct from, say, Murphy’s Law, and someone with no particular agenda would need to be the one to add the article.
It all means nothing, however, if no one is even going to see the article. So if no one is searching Wikipedia for ‘cade factor’ there’s no reason to have an article there.
Chicken & egg.
Finally, they started a rule after the Seigenthaler scandal that newly registered users can’t create new articles. You can edit existing ones all you like until the time frame is up (I think it’s 4 days or something). So you could register, edit a couple articles you’re really an expert on, and eventually add ‘cade factor’. You’d want to link to that article from, say, Murphy’s Law, and vice versa, and make it a really well-done Wikipedia article…and even then, it could get wiped out in a heartbeat.
(In other news, if anyone wants to Digg this post, be my guest…)