This is the conclusion of a research study to be published in the next issue of Internet and Higher Education. I’ve emailed the authors to see if they’re planning on depositing the paper in their Institutional Repository… It’s only a pre-print right now so I can’t give you a complete citation, but if you subscribe to Science Direct you may be able to access the article here.
The title of the paper is A preliminary assessment of Google Scholar as a source of EAP students’ research materials, by Rena Helms-Park, Pavlina Radia, and Paul Stapleton, from the U of Toronto and Hokkaido University in Japan. They looked at the citations found in the research papers of 27 undergraduate ESL students using something called WATCH, an analytic website assessment scale, and found “no significant differences between sources obtained through Google Scholar and the university library’s catalogue of electronic resources. On the other hand, there were significant differences between Google Scholar and Google, as well as between electronic sources obtained through the library and Google, in key areas such as academic rigor ad objectivity.” Did I mention that none of the researchers are librarians?
I’ve got highlights all through the article but don’t want to quote, like, 50% of it here, so I sure hope they make it more widely available, ’cause I think you’re going to want to read this one. I was really pleased that they seem to be aware of the shortcomings we know about Google Scholar; Sullivan, Jasco and Notess are cited heavily throughout the paper.
Some implications for you and me: “Comprehensive instruction and discussion on critically evaluating research materials therefore seems to be more necessary than ever, preferably early in an undergraduate program through guided hands-on work.” and “With such quickly advancing refinements in search technology, educators urgently need to keep abreast of how newer generations view the process of research-based writing… In other words, the ease of the search process needs to be complemented by the hard work of critically assessing the credibility and quality of sources.”
I recently completed some research that made use of a questionnaire, and one of the questions we figured would be a throw-away really got me to thinking along these same lines. The question asked students if they agreed with the statement, “I usually can find all the material I need for my research through internet search engines” or something like that. We figured they’d know it was a librarian asking the question and everyone would disagree, but a few students agreed with that statement, and in discussing the results my colleagues and I were forced to admit that in fact search engines like Google Scholar and OAIster and possibly even Live Search Academic can, certainly for undergraduates, return enough quality information that in many cases students wouldn’t have to use a library database to get enough citations to write a quality paper. Getting to the full text may be another issue, but as far as indexing…
Interesting stuff…
Technorati Tags: Google, Libraries, Schoogle, Google_Scholar, Quality

Comments
3 Responses to “Google Scholar’s as good as the library”
Paul,
I may be on sabbatical but I am not adverse to a discussion of Google scholar, particularly when we have opportunity for debate. This article is extremely self-limiting with an n=27 but I find it lacking in one other respect: information discovery is not an either-or proposition for undergraduates (or even faculty for that matter). Librarians should never be advising the use of one source over another; doesn’t it seem more responsible to advocate for broad selection and use of information tools to introduce their attendent weaknesses and strengths?
In addition, the study conflates “using” a tool like Google scholar with the notion of blind faith, and lack of a critical attitude towards sources. That’s an awful lot to attribute to one search engine. Isn’t it more likely that undergrads want a quick fix and GS provides that better and faster than our OPACs? That being the case, we should be focussing our information literacy efforts on educating them about authority and less about why Google scholar will never be a proper index.
Google scholar is three years old, still in beta (Web 2.0?) and widely-used as a browsing tool. It’s one arrow in a researcher’s quiver; in a medical domain, where my work resides, it’s not as well-known as classic Google or PubMed. It has severe limitations, problems with duplication, bias, coverage, lack of authority controls – shall I continue? I won’t (read Jacso for the crankiest reviews in library science; Notess is better).
Do these reviewers work on reference desks (Notess does)? Do they know how undergraduates look for information, and how they browse? How social software is used to learn about the world of information?
The world of information is a big, unwieldly place. Librarians can make searching more coherent for our users and by showing them the range of what’s possible given their strengths as well as their limitations.
wow, excellent find! I’m sharing this with my colleagues here–it fits in with some discussions we’ve been having recently. Thanks for posting.
Google Scholar as good as the library?
This is the conclusion of a research study to be published in the next issue of Internet and Higher Education. I’ve emailed the authors to see if they’re planning on depositing the paper in their Institutional Repository… It’s only a