The Governor’s Office of Student Achievement (GOSA) has recognized Atlanta Public Schools (APS) for significantly improving auditing and security procedures related to administering the Criterion-Referenced Competency Test (CRCT). In an e-bulletin posted on the GOSA website, APS is acknowledged for implementing measures to increase the security of assessment data and testing materials.
In July of 2011, GOSA released a report that outlined its investigation into the 2009 administration of the APS CRCT exams; the report indicated that widespread cheating occurred in APS schools. Since the release of the report, under the leadership of new APS Superintendent Erroll B. Davis, Jr., the school system has established and enforced a number of testing security improvements, including:
- Securing testing materials before, during, and after a test is administered to students.
- Monitoring the inventory of testing materials via video surveillance.
- Creating a more stringent process for the distribution of materials, testing administration, and collection and shipment of completed tests.
- Developing a trigger report that identifies schools with mean scale score gains that are more than three standard deviations from the state mean scale score change.
- Crafting a CRCT test administration audit plan.
The test administration audit plan is essential to preserving the integrity of assessments. According to Dr. Rubye Sullivan, Director of Research and Evaluation for School Improvement at APS, “There is an important relationship between audit findings and the continuous improvement of the security process related to assessment.” Dr. Sullivan’s work that is critical to the security process.
The GOSA e-bulletin indicates that APS took additional steps to ensure that no breach of security occurs during the dissemination of tests, by establishing a testing materials chain of custody plan, which prevents unauthorized officials from handling testing materials. According to GOSA, APS’ process for the handling of testing materials is now the responsibility of a school’s principal and lead school testing coordinator (LSTC); both must sign legally binding verification forms when handling testing materials. In addition, the shipping and delivery of testing materials is also confirmed by the signatures of the principal and lead school testing coordinator.
Additionally, according to GOSA, After the principal and LSTC receive materials, they must sign and fax an acknowledgement form to the APS district warehouse testing center by close of business on the day of delivery. Materials must then be inventoried and certified by signature of the principal and LSTC within 24 hours. Logistics services drivers may only release testing materials to the custody of the principal and LSTC. If these individuals are not available, testing materials are returned to the test center, and an alternate day is scheduled for delivery.
While APS has established a number of security measures to prevent test tampering, GOSA suggests that the most significant difference between the former process and the current process is that test answer sheets are sealed in an envelope by a certified system administrator, assigned a serial number, and entered into a verification log, immediately after the testing period concludes.
In addition to the security measures established to prevent test tampering, Superintendent Davis expanded the APS Office of Internal Compliance (OIC) to conduct self-imposed audits of all testing practices and policies. When testing irregularities occur, the OIC reviews the irregularities to determine which cases require further review and possible investigation.
When asked about the new security measures to prevent test tampering and ensure the integrity of all assessments, Superintendent Davis outlined a number of enhancements that he and his team have implemented.
“We have developed a comprehensive ethics program which mandates that every employee participate in annual ethics training as a condition of employment. We have strengthened test security measures, including locked safe rooms, tighter chains-of-custody, and clearer test protocols, to prevent improprieties and tampering. Finally, we set trigger points that can result in automatic investigations of schools whose test scores show larger-than-normal, year-over-year changes. We are determined to ensure that our assessments serve children, not adults.”