What is the Right Number of Test Questions?

There are many factors to consider when you are preparing an assessment for your class or designing and developing a large-scale assessment. Decisions about the number and types of items (questions) will depend on considerations, including the purpose(s) of the assessment, the use(s) of the information resulting from it, the amount of time for testing (given the constraint of available class time and the age of the students), and the amount of time (cost) required for scoring constructed-response items (in the case of large-scale assessment). Regarding purpose(s) and use(s) of assessment information, formative assessments (those that are ongoing during the teaching-learning process to provide students and teachers feedback to help them improve their learning and inform their instructional practices, respectively), should be shorter and frequently administered. Summative assessments (that are administered at the end of a program of study to evaluate what students have learned) will tend to be longer and less frequent. Another important factor to consider is the choice of assessment format(s). Item format choice (e.g., short/long constructed responses, problem-solving questions/tasks, objective items) should be based on the language of the curriculum learning objectives/expectations to be tested.

It is also important to note that several items/tasks are required to be able to report with confidence about student performance for any category of interest (e.g., skill areas in language or strands in mathematics). Dr. Mark Reckase, University Distinguished Professor Emeritus, Michigan State University, has used the following analogy to help understand this concept (expressed in my words).

Imagine a sandbox with an unknown object buried beneath the sand. If you push a single rod (probe) into the sand you may or may not touch the object. Even if you do, it is highly unlikely you could identify it from one probe. Only after using several probes can you discover the object’s shape. The same thing applies in testing. A single item or measure (probe) will usually not reveal much about a student’s knowledge or ability. Only after administering several items can one estimate a student’s achievement with any degree of accuracy.

Psychometrics can assist in determining, more precisely, the number of items required to confidently report on a category of interest. However, a good rule of thumb is that at least 10 well-constructed items are needed to report confidently and validly on any category of achievement, and only if they represent an appropriate cross-section of the relevant learning expectation (including an appropriate range of items difficulty) in the achievement category.

Further information about item and test development can be found in my book: Large-Scale Assessment Issues and Practices: An Introductory Handbook (2014), which can be accessed at https://www.amazon.ae/Large-Scale-Assessment-Issues-Practices-Introductory/dp/0968485731.

RMJ Assessment provides a wide range of assessment services, which are outlined on our website: https://www.rmjassessment.com/large-scale-assessment/.