What is an Assessment Blueprint, and Why is it Important?

Blueprints are foundational to assessments. As I stated in an earlier article, there are many factors to consider when you are preparing an assessment for your class or designing and developing a large-scale assessment. Once decisions have been made concerning the main purpose of the assessment (formative, diagnostic, summative) and consideration has been given to length/number of items/tasks, the next work to be undertaken is to establish an assessment blueprint or table of expectations. Jones (2014) explains that a blueprint is important for the following reasons:

• It indicates the number and/or proportion and type of test items/tasks (i.e., closed- and open-response) that are to be developed to measure the assessment’s content knowledge and skills.
• It provides a guide for item writers as to the learning expectations/objectives for which items/tasks are to be developed, as well as their levels of cognitive processing (difficulty levels).

The first two blueprint considerations apply to both classroom and large-scale assessment. Additional reasons apply especially to large-scale assessment:

• If comparability of assessments and their results from year to year is important, a blueprint will assist in ensuring consistency in terms of the content measured and relative test difficulty.
• If more than one test form is administered in a given year, there is greater likelihood that the forms will reflect the same learning expectations, and will therefore be parallel, if a blueprint is followed.
• While all strands or skill areas of a curriculum are likely to be assessed each year, it is unlikely that all curriculum expectations within those content areas can be measured in any single large-scale assessment given time constraints and the number of test items that would be required.

By documenting and tracking the learning expectations that have been measured, and using that information when targeting expectations for item development in subsequent years, you can ensure reasonable coverage of all measurable learning expectations over time. An added advantage of sampling from the curriculum in this way is that educators will not be able to teach only to selected learning expectations; they will need to cover all curriculum areas to ensure students are adequately prepared for the assessment. This discourages a narrowing of the curriculum. Furthermore, curriculum expectations that cannot be measured by large-scale assessment should be determined prior to developing the assessment blueprint and should be identified in it.

Blueprints may differ according to their level of detail. For instance, it is often useful to provide a general blueprint for public consumption that gives an overview of, for example, the number and/or proportion of objective (e.g., multiple-choice items), open-response items/tasks, operational items (those that count for a student’s outcome), and field-test items (those that do not count) if/when applicable. Test developers and item writers need more detailed information about the overall and specific learning objectives that will be targeted for the given assessment, together with required cognitive and difficulty levels (e.g., knowledge and understanding, application, critical thinking, problem solving).

It is useful to have the draft assessment blueprint reviewed by experienced educators and test developers prior to item development to ensure it matches the curriculum and demonstrates an adequate coverage of it. Involving psychometricians in the blueprint review is also important to ensure there are sufficient numbers of items for equating purposes, in order that reliable, valid reports can be generated (see Holland & Dorans, 2006; Reckase, 2009, for example [full citations provided below]).

More information about assessment blueprints can be found in my book: Large-Scale Assessment Issues and Practices: An Introductory Handbook (2014), which can be accessed at https://www.amazon.ca/Large-Scale-Assessment-Issues-Practices-Introductory/dp/0968485731.

RMJ Assessment offers a wide range of assessment services, which are outlined on our website: https://rmjassessment.com/large-scale-assessment/.

________________________________

Citations:

Holland, P.W., & Dorans, N.J. (2006). Linking and Equating. In Brennan, R.L. (Ed.). Educational Measurement, Fourth Edition (pp. 187-220). Westport, CT: American Council on Education and Praeger Publishers.

Reckase, M.D. (2009). Multidimensional Item Response Theory. New York: Springer-Verlag.