Engage Toggle Menu

Measure

How do we measure science literacy?

Our approach to measuring science literacy is rooted in our definition of what we believe students who are scientifically literate should be able to do. As a result, our assessment challenges students with real-world problems whose solutions require them to reconcile potentially conflicting scientific evidence, while transferring skills in order to make sense of information that they have not previously been taught.

A major challenge is to come up with a task that is of sufficient distance from classroom experiences to require transfer while at the same time explicitly drawing on the skills they have gained in their courses. For example, students who have learned about the scientific process and the marshaling of scientific evidence in a microbiology course might be given a problem to solve about the cognitive benefits of musical training for children. While the content here is new to these students, the skills learned in microbiology should be applicable. Our view is that science literacy involves being able to apply their acquired skills in this new area (i.e. transferring their skills).

Our
Thinking

Here’s what we do:

  • We have developed an instrument for assessing science literacy that consists of a single question motivated by a real-world problem. There is no right answer to the question.
  • Equipped with a computer connected to the internet and paired with a partner, each student has up to an hour to do whatever research they like to develop an answer to the question. Each student is then asked to submit answers to a series of prompts that require them to describe the answer they came up with, and how they arrived at that answer.
  • All of the questions follow a similar format presented in the form of two conflicting points of view about an important issue in the real world, such as the safety of oxybenzone in sunscreen, or the reliability of the body mass index as an indicator of health.

Please contact us if you would like copies of our assessment materials.

Here are links to three other good assessments of science literacy at the college level:

The Experimental Design Ability Test (EDAT)

Sirum & Humburg 2011

The goal of this assessment is to determine whether students are able to design an appropriate experiment to evaluate an assertion. Two distinct assertions are provided for use at the start and end of a course, and there is a clear and simple rubric for assessing results.

View File

Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments

Gormally et al. 2012

This article describes the Test of Scientific Literacy Skills (TOSLS), a very good assessment tool, featuring primarily multiple-choice questions, that explores whether students have a suite of skills in science literacy, based on a list developed by surveying science faculty from a variety of institutions.

View File

A Novel Instrument for Assessing Students’ Critical Thinking Abilities

White et al. 2011

The authors describe a tool for assessing how well students can use critical thinking to resolve a complicated scientific problem about the causes of an unspecified disease, and report results. A major strength is the incorporation of messy evidence and conflicting results, and the availability of an explicit and clear rubric. The full set of materials for the assessment is available from the authors.

View File

"[Outstanding teachers use] assessments to help students learn, not just to rate and rank their efforts."

— Ken Bain, What the Best College Teachers Do

Here we provide additional resources for using assessment to motivate, evaluate, and focus student learning in courses.

Student Assessment of Learning Gains

This website provides a useful tool for developing a customized instrument to assess learning gains by students.

View Resource

Critical Thinking Assessment Test (CAT)

This website provides a variety of resources related to the Critical Thinking Assessment Test, a 15-question test that uses a short-answer format to assess how well students are able to use critical thinking skills to solve problems. The actual test must be purchased, but this article describes some of the details of the test.

View Resource

Science Motivation Questionnaire

This website provides free access to a questionnaire that probes students about their motivations to study science. Versions specifically for biology, chemistry, and physics are also available.

View Resource

Field-tested Learning Assessment Guide

This resource provides access to and information about assessment tools, including many sources of information about classroom assessment techniques like portfolios, concept maps, and attitude surveys. One particularly valuable component is a table that links specific learning goals with particular assessment tools.

View Resource

Results of Twenty Years of Testing the Scientific Literacy of Undergraduates

Impey et al. 2011

In this provocative article, the authors present the results of 20 years of assessing the science literacy of undergraduates, while providing the full assessment instrument, which consists of multiple choice and fill-in-the-blank questions. One important result: science majors score no better on average than non-majors do.

View Resource