Many jurisdictions have either switched to computer-based assessments (also known as digital, e-, online or electronic assessments) or are considering transitioning their assessment program from paper-based to computer formats, and the benefits of modernizing assessments for examinees, teachers, administrators and other stakeholders are many, including the following.
Engagement
In recent years schools/classrooms have adopted technology-based tools for learning, and examinees are becoming increasingly tech-savvy preferring to do their work on computers as opposed to paper. Many believe that static, paper-based assessments are dull/boring; however, they tend to be more engaged with computer-based assessments that contain a variety of item types and use media such as maps, diagrams, photographs, animation, video, and audio. Engagement is advanced even more when technology enhanced items (TEIs), including game-like simulations, are used. In addition, examinees often experience test anxiety; however, if they have opportunities to take online practice tests to become more comfortable with test content and presentation, anxiety can be reduced with the computer-based format.
Authenticity
Authenticity in large-scale assessment is reflected by characteristics such as presenting examinees with real-world situations and having them demonstrate their knowledge, skills and abilities by applying their knowledge/understandings to complete complex tasks/solve problems. Unlike traditional paper-based assessments that typically emphasize the recall of facts and knowledge, computer-based assessment can present examinees with realistic situations (via simulations for example) to gather information about their higher-order thinking and problem-solving skills.
Equity & Inclusion
All jurisdictions strive to include as many examinees as possible in their assessment programs. Paper-and-pencil assessments present barriers to some examinees requiring accommodations and special considerations. Equity and inclusion are important lenses through which most jurisdictions approach large-scale assessment. When assessments are offered via the computer it is easier (compared with paper-based assessment) to adapt them to the needs of examinees who have, for example, visual, auditory and motor disabilities. Accessibility features include: the ability to change font size and background colour, magnification (zooming), the provision of text-to-speech and speech-to-text applications, screen readers and assistive technologies. Assessment administration via computer also facilitates adding extra time for examinees requiring this type of accommodation.
Item & Assessment Development
Compared with paper-based processes, working online makes it easier for an assessment organization to manage the tasks involved in developing items (questions) and creating assessments. For example, online work flows can be used to assign item development tasks, conduct reviews of draft items and provide approvals prior to adding them to an item bank (item development and reviews can be conducted electronically and remotely). Typically, all versions of an item are retained for easy reference. Once an appropriate bank of items has been developed (in terms of numbers, curriculum coverage and difficulty levels) online tools can be used to automate the creation of assessment forms. Online workflows track all steps in the development process and allow for accurate process documentation, so the organization has a clear picture of what has occurred.
Administration
Usually, paper-based assessments are administered at set times and locations because of the limitations resulting from their format. By definition paper-based assessments involve printing, shipping and circulating assessment materials on paper, which can be a tedious process and be prone to error, such as misprints, missing or damaged material. Furthermore, following administration of paper-based assessments the handling of examinee responses on paper is cumbersome and subject to error. Online administration allows more flexibility for the scheduling and administration of assessments (multiple administrations, testing on demand and remote testing are possible). Another potential advantage of computer-based assessment is that it allows for adaptive testing (adapting the content of the test to the knowledge/skill level of the examinee). Compared with paper-based assessment adaptive testing is both more efficient (best use of the examinees’ time, as they respond to items closer to their true ability level) and effective (provides a more accurate estimate of examinees’ true ability). Digital assessments provide test-takers with practice experience on the platform; test administrators have access to various supports (e.g., online, e-mail, telephone advice); and provided instructions are clear and straightforward, digital assessments significantly reduce the administrative burden for both the assessment organization, as well as teachers and administrators who operate local assessment sites.
Marking/Scoring
Paper-based scoring can be labour-intensive, inefficient and subject to error with regard to the management of scorers and related workflows, circulation and tracking of examinees’ constructed responses, record keeping, data analytics, the use of scoring data for the supervision and moderation of scorers and security of assessment materials. With computer-based assessment, machine-scorable items are auto-scored accurately immediately upon submission of the assessment. For responses that require human marking, online scoring platforms allow for efficient, streamlined and standardized processes that include identifying roles (e.g., scoring supervisors, scorers, expert scorers) and permissions for their respective authorities, automated routing and workflows for all marking roles and quality assurance procedures. The use of online scoring systems saves the assessment organization a great deal of time and effort, while also enhancing the efficacy, accuracy and security of marking and results. If conducted remotely, online scoring also represents substantial cost savings compared with centralized, paper-based scoring.
Feedback/Reporting
Accurate, reliable, valid and timely results are essential to all assessment programs. Stakeholders must have confidence in the data, because typical purposes include accountability (to indicate how, for example, the education system is performing) and improvement (to identify examinees’ needs, enhance instruction, establish and implement targeted interventions). Any question/doubt about the quality and integrity of the data/results may undermine efforts to use the data for both accountability and improvement. When assessment results are not released in a timely fashion their value is reduced for all stakeholders. In the case of large-scale, paper-based assessment, the time lapse between the administration of an assessment and examinees receiving results may be several weeks or even months. In the case of online assessments, machine-scorable items can be processed immediately and results made available very quickly (should the assessment organization wish to do so). With the more efficient online human scoring processes, results can be released more quickly than with the paper-and-pencil approach. Furthermore, digital analysis and reporting tools can generate comprehensive, detailed reports more efficiently than is possible with paper-based assessment. Consequently, clear, thorough and timely results for examinees can be made available to provide useful feedback on how they are doing and where improvements are needed (formative assessment) and for education systems, so that stakeholders can understand the strengths and challenges of the systems (accountability).
Data/Analytics
Another benefit of computer-based assessment over the paper approach is that additional data is available to provide a fuller appreciation of the examinee’s effort, approach and progress in responding to assessment items/tasks. For example, online assessments can identify for each examinee and by item, performance data such as the amount of time on task, the sequencing of steps in responding to a challenge, items flagged to return to later and changes to responses. The teacher, analyzing these types of data/information, can determine if there are issues/problems experienced by individual examinees or are common to groups of them. Such process data analytics can be used by the instructor to help examinees improve their learning and/or for the teacher to plan for future instruction.
Enhanced Security
Online assessment provides for greater security compared with paper-based assessment, because all examinee information, assessments and results are stored electronically. Only those people who are authorized to access given data/information are able to do so. Access management is much more challenging with paper-based assessments.
Scalability/Flexibility
Scalability and flexibility have increased with the advent of online assessment. In the case of paper-based assessments, administration occurs in physical sites such as test centres and schools. With online assessment, individual examinees can access assessments securely via the Internet (recently this has become a common feature as a result of COVID-19 and the closing of schools and/or the option of online learning). In remote assessment situations, assessment organizations must assess the risks related to the integrity and security of results and/or assessment content. Running secure remote online assessments may necessitate remote proctoring alternatives.
Environment/Sustainability
Electronic assessment is more environmentally friendly and has a much smaller carbon footprint than the paper-and-pencil approach, because it involves much less paper, printing and transportation.
Cost Effectiveness
Throughout the entire assessment process, online assessments have the potential for cost savings as compared with paper-based assessments. For example, assessments on paper include manual processes related to: item writing, item banking, field testing, test form creation, printing, distribution (transportation), securing invigilators, administering the assessment, return of test materials (transportation), scanning/data entry of machine-scorable responses, manual processing and tracking of scoring material, storage (and eventual disposal) of assessment materials and printing and distribution of reports (not to mention staff overhead to manage all processes). Because computer-based assessments vary so much in context (e.g., number of administrations, numbers of examinees, number and type of items, online versus offline, distributed versus centralized scoring, use of proctors) it is difficult to provide a generalizable percentage cost savings of online versus paper-based assessments. It is possible, however, to estimate cost savings associated with specific assessment scenarios, which will be provided in a subsequent post.
Many jurisdictions are struggling with decisions about whether/how to proceed with computer-based assessment. RMJ Assessment has experience in this field and can offer considerations and advice. Should you wish to learn more about the services we offer, please visit our website at https://www.rmjassessment.com/, reach out to us by e-mail at richard.jones@rmjassessment.com or call (647) 234-5346.
RMJ ASSESSMENT 