With a career spanning more than 38 years in large-scale assessment, Dr. Mary J. Pitoniak is a noted psychometrician and expert in educational measurement who has recently transitioned into a high-profile leadership role in academic publishing and consulting. She holds a B.A., summa cum laude with Highest Honors, in Psychology from Smith College, as well as an M.S. and a Ph.D. in Educational Psychology and Psychometric Methods from the University of Massachusetts Amherst.
Dr. Pitoniak’s early career in assessment began at National Evaluation Systems (NES), a teacher licensure and certification testing company that was eventually acquired by Pearson. Her work at NES included test development, editorial production, project management, and proposal preparation.
After earning her graduate degrees, she joined Educational Testing Service (ETS), where she worked for more than 20 years. During her tenure, she held roles including Senior Program Administrator, Lead Program Administrator, and Strategic Advisor and Director of Special Projects to the Vice President of Statistical Analysis, Data Analysis, and Psychometric Research. In these roles, Mary contributed to numerous initiatives, notably serving in a leadership role on the technical advisory committee of the U.S. National Assessment of Educational Progress (NAEP).
She later served as ETS’s Senior Strategic Advisor to the Vice President of Research, continuing her involvement with NAEP. Subsequently she became an Executive Director, overseeing the Office of Disability Policy and the Office of Professional Standards Compliance, where she led audits of testing programs against the ETS Standards for Quality and Fairness.
Dr. Pitoniak’s expertise in assessment focuses on standard setting, test accommodations for students with special education needs and English language learners, and computer-adaptive testing. Throughout her career, she has conducted external evaluations of large-scale student assessment programs, served as a lead psychometrician on capacity-building initiatives in educational measurement, and provided training in standard setting and other assessment topics for international examination organizations. She has also delivered keynote addresses and served as a guest speaker at numerous events and conferences.
Dr. Pitoniak’s service to professional organizations includes her current role as a member of the Board of Trustees and past President of the International Association for Educational Assessment (IAEA), as well as past Vice President of Division D (Measurement and Research Methodologies) of the American Educational Research Association (AERA). She has also served as a member of the Coalition for Psychology in Schools and Education for the American Psychological Association (APA). In addition, she is the incoming Editor of the NCME Educational Measurement and Assessment Book Series.
Mary discusses the value and challenges of large-scale digital assessments, with a particular focus on the transition from paper-based to digital formats. She emphasizes the importance of carefully considering fairness, reliability, validity (including construct validity), security, and the skills required of test takers when moving assessments online.
She highlights examples from NAEP, where simulations and cognitive interviews were used to ensure that assessments measured what they were intended to measure. Mary expresses support for Vretta’s digital solution, noting that the company has invested significant time and effort in developing an efficient, effective platform that supports modernization initiatives across Canada and Europe.
In discussing the use of AI in assessment, Mary expresses concern about relying solely on generative AI without human oversight, given potential issues related to reliability, fairness, and validity. She outlines potential applications of AI in education and assessment, including enabling students to use AI to construct responses and to critique AI-generated material. Mary reflects on her background in educational measurement, standard setting, and training, emphasizing the importance of maintaining core principles such as reliability, validity, and fairness.
Dr. Pitoniak is a prolific author, having published numerous book chapters and journal articles. Most recently, at the National Council on Measurement in Education (NCME) Annual Meeting (April 2026), she received the NCME Annual Award for Exceptional Achievement in Educational Measurement, for the Fifth Edition of Educational Measurement (January 9, 2026), co-edited with Dr. Linda L. Cook. This major project took eight years to complete and involved contributions from 52 authors.
Reflecting on her distinguished career, Dr. Pitoniak takes particular pride in being recognized, along with Dr. Cook, as the first female editors of the historic Educational Measurement series. In addition, Mary values her ability to explain complex assessment concepts in accessible, everyday language, particularly in the context of developing countries.
Mary recently launched her consulting business: Pitoniak Educational Measurement. In her consulting work, she is particularly interested in providing training across all aspects of assessment, including item and test development, psychometrics, test marking, and test scoring; conducting standard-setting activities; and evaluating testing programs against established standards of best practice.
Away from her professional responsibilities, Mary enjoys training, working with developing countries, singing, and serving as a hospice volunteer. She also expresses her love of animals, particularly her Boxers. Please feel free to reach out to Mary at mary@pitoniakconsulting.com.
RMJ ASSESSMENT 