Our approach to measuring science literacy is rooted in our definition of what we believe students who are scientifically literate should be able to do. As a result, our assessment challenges students with real-world problems whose solutions require them to reconcile potentially conflicting scientific evidence, while transferring skills in order to make sense of information that they have not previously been taught.

A major challenge is to come up with a task that is of sufficient distance from classroom experiences to require transfer while at the same time explicitly drawing on the skills they have gained in their courses. For example, students who have learned about the scientific process and the marshaling of scientific evidence in a microbiology course might be given a problem to solve about the cognitive benefits of musical training for children. While the content here is new to these students, the skills learned in microbiology should be applicable. Our view is that science literacy involves being able to apply their acquired skills in this new area (i.e. transferring their skills).

OUR
THINKING

Here's what we do:

• We have developed an instrument for assessing science literacy that consists of a single question motivated by a real-world problem. There is no right answer to the question.
  
• Equipped with a computer connected to the internet and paired with a partner, each student has up to an hour to do whatever research they like to develop an answer to the question. Each student is then asked to submit answers to a series of prompts that require them to describe the answer they came up with, and how they arrived at that answer.
• All of the questions follow a similar format presented in the form of two conflicting points of view about an important issue in the real world, such as the safety of oxybenzone in sunscreen, or the reliability of the body mass index as an indicator of health.
• Please contact us if you would like copies of our assessment materials.

Here are links to three other good assessments of science literacy at the college level:

The Experimental Design Ability Test (EDAT)
Sirum & Humburg 2011

The goal of this assessment is to determine whether students are able to design an appropriate experiment to evaluate an assertion. Two distinct assertions are provided for use at the start and end of a course, and there is a clear and simple rubric for assessing results.

Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments
Gormally et al. 2012

This article describes the Test of Scientific Literacy Skills (TOSLS), a very good assessment tool, featuring primarily multiple-choice questions, that explores whether students have a suite of skills in science literacy, based on a list developed by surveying science faculty from a variety of institutions

A Novel Instrument for Assessing Students’ Critical Thinking Abilities
White et al. 2011

In this broad overview, the authors outline major ideas about the transfer of learning and provide a toolkit of foundational concepts, including near versus far transfer, and automatic versus mindful transfer.

---

"[Outstanding teachers use] assessments to help students learn, not just to rate and rank their efforts."

— Ken Bain, What the Best College Teachers Do, 2012

Here we provide additional resources for using assessment to motivate, evaluate, and focus student learning in courses.

Student Assessment of Learning Gains (SALG)

This website provides a useful tool for developing a customized instrument to assess learning gains by students.

Critical Thinking Assessment Test

This website provides a variety of resources related to the Critical Thinking Assessment Test, a 15-question test that uses a short-answer format to assess how well students are able to use critical thinking skills to solve problems. The actual test must be purchased, but this article provides information about the test.

Science motivation questionnaire II

This website provides free access to a questionnaire that probes students about their motivations to study science. Versions specifically for biology, chemistry, and physics are also available.

Results of 20 years of testing the science literacy of undergraduates

In this provocative article, Impey et al. 2011 present the results of 20 years of assessing the science literacy of undergraduates, while providing the full assessment instrument, which consists of multiple choice and fill-in-the-blank questions. One important result: science majors score no better on average than non-majors do.

Campus Road, PO Box 5000
Annandale-on-Hudson New York
12504-5000 USA