Evidence

As we engage in a process to develop a comprehensive set of variables meaningfully tailored to specific assessment contexts, we must adequately explore all factors that may be used to support our conclusions. This process of warranting evidence involves various kinds of documentation, ranging from student narratives of skill transformation to regression analyses of model efficacy.

The use of empirical research methods involves developing an evidence-based argument to provide validity to our findings. We may want to think in terms of the system of logic first proposed in 1958 by British philosopher Stephen Toulmin. The basic elements of Toulmin’s logic scheme, listed below, correlate with the inductive reasoning and evidence analysis that are cornerstones of a well-constructed and transferable model:

  1. Claim – the argument; what the study demonstrates
  2. Support – the nature of the logic upon which the claim is based
  3. Data – evidence; the kind of information on which the claim in based
  4. Warrant – assumptions and justification for claim based on the data at hand; inferences made that are either implicit or explicit
  5. Qualification – conditions, exceptions, or limitation of the claim

The figure below captures the essence of the model.

Figure 1: The Toulmin Model of Reasoning
Toulmin model

This framework, one that not only requires facts but reasoned justification, matches the persuasive nature of our assessment research. Through collaboration and mutual inquiry, we follow Toulmin’s lead and ask not just “‘What have you got to go on?’ but ‘How do you get there?’”

Evidence must include a well-developed research plan that considers sampling design, instrument selection, analysis procedures, and assessment impact. Through evidence-based analysis, we can support the warrants and claims in our arguments, qualifying them where appropriate.

We invite CPTSC members to provide us with the ways that they collect assessment evidence so that we may expand our community.

Suggested Resources on Evidence

American Educational Research Association. “Standards for Reporting on Empirical Social Science Research in AERA Publications.” Educational Researcher 35.6 (2006): 33-40.

Bernard, H. Russell. Social Research Methods: Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage, 2000.

Mislevy, Robert J.  “Validity by Design.” Educational Researcher 35.6 (2006): 33-40.

Toulmin, Stephen, Richard Rieke, and Allan Janik. An Introduction to Reasoning. New York Macmillian, 1979.

Toulmin, Stephen. E. The Uses of Argument. Cambridge: Cambridge UP, 2003.