The Promise of E-Assessment

Computer-based testing (CBT) has been around, in one form or another, for approximately sixty years; the CBT trend has long been touted in the field of large-scale testing and measurement; and for the past three decades, the e-assessment industry has been predicting that CBT would supersede paper-based assessment. Advocates identify many potential advantages of e-assessment, including the following:

• Efficiencies in item and test development, as well as scoring
• Greater flexibility in test administration
• Enhanced authenticity (more closely mirroring classroom instructional practice and student learning)
• Use of innovative, technology-enhanced items (creating new ways of assessing students’ understandings and skills that have been difficult to measure using traditional paper-and-pencil tests)
• Higher levels of student engagement
• Increased test security
• Improved accessibility (providing accommodations/supports for some students with special education needs, who would otherwise be unable to participate in a paper-based assessment)
• Quicker turnaround of results
• Cost savings (related to services such as printing and shipping)

While it is true that CBT has become more widespread in recent years, many jurisdictions continue to rely on paper-based assessment. Given the promise of CBT, why has e-assessment not supplanted the paper-and-pencil approach on a broader scale? And what progress has the e-assessment industry made in realizing the perceived advantages of the approach?

In my conversations with those responsible for provincial and national assessments across Canada, there are two key takeaways:

(1) E-assessment is top of mind for all provinces/testing organizations, although they are at different stages in moving forward. For example, British Columbia has implemented online assessment for many years; in 2019 the Council of Ministers of Education, Canada (CMEC) implemented online assessment for the Pan-Canadian Assessment Program (PCAP); and all provinces are currently involved with CBT, to some extent, through international assessments such as PISA, e-PIRLS, and e-TIMSS). Two provinces are currently engaged in procurement processes for e-assessment services. All of the other provinces are at some stage in the discovery process. Some provinces, although recognizing that e-assessment is inevitable, have identified other priorities that must be addressed before seriously considering CBT; others are actively researching various aspects of e-assessment and intend to move forward in the not-too-distant future.

(2) E-assessment-related information is hard to come by.

An often-heard comment is that without going to the time, effort, and cost of releasing Requests for Information (RFI) and/or Requests for Proposals (RFP), it is challenging for jurisdictions to find information that supports their e-assessment decision making. Some common questions/issues they want answers to are as follows:

• Should we simply move our paper-based assessment to an e-format, or should we transform our assessment to an e-model (and be able to assess other knowledge and skills (e.g., global competencies)?
• What new item (question) types are possible with e-assessment? How are they developed? How are they validated? What are the costs?
• What are the implications for schools, teachers, and students?
• How can we make our e-assessments more accessible to students with special education needs (e.g., can assistive technology such as Kurzweil and speech-to-text and text-to-speech functions be incorporated into the e-solution)?
• Are paper-based and e-assessment results comparable (what are the mode effects)?
• What implications are there for test equating and standard setting?
• What are the school and school district infrastructure, computer equipment, and technology support needs of e-assessment?
• What will e-assessment cost?

Many jurisdictions struggle to know where they go from here in considering CBT. While recognizing the e-assessment industry must safeguard proprietary information, jurisdictions need access to more information to become better informed, as they consider the move toward computer-based assessment. RMJ Assessment has conducted a great deal of research in this field and can offer considerations and advice. Should you wish to learn more about the services we offer, please visit our website at https://www.rmjassessment.com/, contact us by e-mail at richard.jones@rmjassessment.com or call (647) 234-5346.