AG official feels problems may have skewed tests
Albert Gallatin Area School District’s top administrator said teachers had such trouble taking the tests in the state’s Professional Development Assistance Program that he questions the validity of reported results. “We’re going to look at the data for what it was intended, for professional development. Maybe even the state will give us some guidelines for analyzing this. But, we have to take into account the conditions under which they were tested and they had a good attitude. I question the validity of the information,” Substitute Superintendent Walter Vicinelly said.
Locally, only teachers from A.G. and Belle Vernon Area joined counterparts from more than 200 school districts, vocational and charter schools and intermediate units who initially took the Professional Development Assistance Program (PDAP), administered in December via computers and the Internet by the Educational Testing Service.
This was the first wave of the five-year $7.5 million test on reading and mathematics, based on the Pennsylvania System of School Assessment (PSSA).
Results posted recently at the state Department of Education Web site show A.G. teachers were more likely to fall short of rather than exceed the state norm. The district also was listed among those showing what the report considers significantly positive or negative results when compared to the state norms at the elementary reading level. The list quotes the district’s mean score was 27.615 of a possible 37 points, 2.36 points below the state mean.
A notation on each page of A.G.’s results, as well as the results of a number of other school districts, stated, “Because of technical difficulties during the test, school district’s completer rate may be below the level where the results are statistically valid.”
Education department spokes-man Jeff McCloud said after the results were released that the technical difficulties concerned the Internet forum in the first week of testing but did not invalidate what was posted. He said there is no retest for the school district, although an individual teacher may have a chance to retake the exam at a neighboring school district.
Vicinelly said the district was selected to be among the first to participate because of where it stands in the schedule of strategic planning, and school officials did what they could to get ready.
Among the efforts toward smooth testing, Vicinelly said, the district selected a week later in the available window to take the test with the hope any glitches would be worked out. He said they worked out a schedule over two days and at different times in computer laboratories and individual classrooms so that large numbers of teachers would not take the test at the same time, trying to avoid an overload on the system.
Vicinelly said the district’s technical people took the test first to be available for any technical problems.
Further, he said the district’s computer tech service, Computer Centerline, checked the system in advance to make sure it was ready and had someone in place to help out as needed.
No one refused to take the test, and Vicinelly said the teachers had a good attitude.
“We made every effort to cover all the bases. We thought we had it all covered,” he said.
However, Vicinelly said the computers were slow, and that meant text was slow in appearing or some was missing. He noted this was a timed test.
He said the district, at the advice of the education department, attempted to log off and then log back on, but that created a problem with trying to reuse a one-time password.
Vicinelly said the district documented its efforts and problems for the education department, which, he added, should take into consideration the technical difficulties.
He said that although the district will try to use the results, he was not happy with the system.
He said such a test could not measure intangibles in teaching ability, like the way a teacher motivates students or manages a classroom, and that it is not fair to make the comparisons among school districts.
The next wave of testing will occur in August, when seven other local school districts and three vocational schools are scheduled.
If Vicinelly has any advice for those who are next in line for PDAP, he said he recommends taking the practice test in order to become familiar with the format.
The PDAP information may be found at the education department Web site: www.pde.state.pa.us.