|
THE STATE EDUCATION DEPARTMENT / THE UNIVERSITY OF THE STATE OF NEW YORK / ALBANY, NY 12234 _______________________________________________________________________________________ |
|
Office for
Elementary, Middle, Secondary and Continuing Education David Abrams, Assistant Commissioner Standards, Assessment and Reporting Room675 EBA 89 Washington Avenue Albany, NY 12234 (518) 473-7880 |
March 22, 2004
To: District Superintendents, Superintendents of Public
Schools,
Principals of Public High Schools,
Administrators of Nonpublic and Charter Schools
From: David
Abrams 
Subject: New Sampling Method for Pretesting and Field Testing of Regents Examinations
Good pretesting and field testing procedures are a fundamental component in building a high quality statewide testing program. In their October 2003 report to the Board of Regents, the Math A Panel made the following recommendation, which can be applied to tests in all subjects:
Sampling procedures for estimating item performance must be improved.
Sampling procedures must be revised to ensure that larger, representative, and more optimally motivated samples of students participate in pretesting and field testing of Math A items. Changes in regulations to require mandatory participation of sampled units would be one possibility; education and persuasion would be another. Enhanced auditing and monitoring of sample demographic characteristics and motivation by SED is also recommended to ensure stability of representativeness and comparability with operational data.
To address these recommendations and to ensure that we have the adequate pretest and field test samples that are needed, we sought recommendations from school officials across the State regarding how to increase school participation in pretesting and field testing. We received several recommendations repeatedly:
§ Give schools a schedule so they can anticipate when they will be expected to participate in pretests and/or field tests;
§ Ask all schools to do their fair share so that some schools are not over-burdened; and
§ Reconfigure the test forms so that they can be administered in one class period.
The Office of State Assessment took this input and developed a new pretest and field test sampling method that employs a multi-year sampling matrix. This sampling method randomly assigns every public high school in New York State, and those nonpublic high schools that participate in the state testing program to one of six strands. Each strand contains a stratified sample of high schools, based on needs-resource capacity definitions. Each Regents examination, other than those in Languages Other Than English (which will be treated separately due to their not being as widely administered across the State) was also listed and the resulting matrix assigns each pretest and field test to a strand. The field test and pretest matrix gives each school a maximum of three tests to administer during any spring field test period. This new sampling method will address the needs of schools in several ways:
§ It gives all schools advance notice of their pretest and field test responsibilities;
§ The matrix system depends on all schools participating in accordance with the strand schedule, thus sharing the burden of pretesting and field testing while ensuring an adequate sample; and
§ To the greatest extent possible pretests and field tests have been designed for administration in one class period. The exceptions to this are Comprehensive English, US History and Government, and Global History and Geography, which include extended response questions requiring more time. No school will be assigned more than one of these longer field tests or pretests per year and we will continue to look at this issue.
Each high school principal should identify the strand to which his/her school has been assigned from the alphabetized list that can be found on the website of the Office of State Assessment at http://www.emsc.nysed.gov/osa/hsgen.html. http://www.emsc.nysed.gov/osa/Important%20Notices/importantnoticearch/matrix.htm (Matrix) (The list, which is entitled Regents Examination FT Strand List, is organized alphabetically first by district and then by school within the district.) The assigned strand number for each school will remain in place through the 2009 field test period.
Participation of all schools in all three scheduled pretests and/or field tests each year is crucial. Full participation helps to guarantee that the data collected are truly representative of New York State’s student population. The State’s student population is carefully analyzed, by school building, to design a pretest and field test sample that accurately reflects our wide geographic, income and needs diversity.
The Office of State Assessment will pilot this system for the spring 2004 field tests and will continue to use this method as long as schools’ participation ensures sufficient samples. Within the next few weeks each high school will receive a letter from OSA identifying the school’s strand. When pretest and field test materials are shipped we will provide specific instructions for administering the 2004 field tests and pretests. In the meantime, if you have any questions, please send them to emscassessinfo@mail.nysed.gov.
Thank you in advance for your cooperation and support. If we all work together, we can ensure that New York will continue to have a high quality assessment system.
cc: James Kadamus