Our approach to measuring science literacy is rooted in our definition of what we believe students who are scientifically literate should be able to do. As a result, our assessment challenges students with real-world problems whose solutions require them to reconcile potentially conflicting scientific evidence, while transferring skills in order to make sense of information that they have not previously been taught.
A major challenge is to come up with a task that is of sufficient distance from classroom experiences to require transfer while at the same time explicitly drawing on the skills they have gained in their courses. For example, students who have learned about the scientific process and the marshaling of scientific evidence in a microbiology course might be given a problem to solve about the cognitive benefits of musical training for children. While the content here is new to these students, the skills learned in microbiology should be applicable. Our view is that science literacy involves being able to apply their acquired skills in this new area (i.e. transferring their skills).