If you build it, will they learn?

I recently finished the following article: If you buld it, will they learn? Assessing online information literacy tutorials, by Elizabeth Blakesley Lindsay, Lara Cummings, Corey M. Johnson and B. Jane Scales, College and Research Libraries, September 2006, Vol. 67, No. 2.  Should be freely available online around March.

Here’s the abstract:

With the support of an internal grant, the Washington State University
Library Instruction Department was able to undertake an assessment
program to measure the use and effectiveness of online tutorials built
by the department. Students viewed four of the tutorial products and
were asked to perform tasks using these tutorials. They also answered a
number of questions designed to garner information about attitudes,
usage patterns and perceptions of library resources and services.
Results of the assessment activities and future plans for improving and
expanding our tutorial offerings are discussed.

I was sure hoping they were going to be able to lay out hard proof that the use of screencasts would lead to better results, but they didn’t.  Screencasts most definitely increased the confidence level of students in the study, but apparently didn’t really increase the test scores. 

I have a note in the margin wondering what would’ve happened had the students who participated in the study been compared to a sample of students who received similar instruction in a traditional f2f environment.  Which students (if any) would’ve shown improvement there?  The authors did point out that students who participated from off campus tended to perform a little better than those who were on campus.

A few interesting takeaways:  "Across all sections, one suggestion from students was to increase the interactivity of the tutorials".  And, "In additional, all of our tutorials and marketing effords could be enhanced by stressing how its use will ultimately save the user time and will be more efficient than a trial-and-error approach."  And they pointed out that when they moved the link to a tutorial from a separate tutorials page to instead live alongside the link to the database in question, usage went up from 10 visitors a month to 100 a month.  Makes sense.

Qarbon ViewletBuilder was the tool of choice for the screencasts used in this research.  They’re available at NetLibrary Tutorial and Using ProQuest to Find Scholarly Articles.  More examples of their screencasts are here.