I hesitate to address the sensitive topic of international comparisons with school leaders who have to face the reality of leading schools on a day-to-day basis. However, I wasn't subjected to the kind of attacks on public schools, teachers, and principals that we have experienced of the past year. In the past, when NAEP or PISA results were released, we simply shrugged our shoulders and moved on. Today, however, our teachers and community expect us to respond when asked. In fact, our silence on this matter could be deafening.

That is why I put together some talking points for school leaders on the 2009 PISA results. I have drawn from a number of sources including the Organization for Economic Cooperation (OECD), which coordinates the international assessments and the Washington Post.

Background

• Begun in 2000, the Program for International Student Assessment (PISA) is a system of international assessments that focuses on 15-year-olds' capabilities in reading literacy, mathematics literacy, and science literacy.
• PISA is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries. 38 OECD nations and 28 partner nations participated in the assessment.
• PISA includes measures of general or cross-curricular competencies such as problem solving.
• PISA emphasizes functional skills that students have acquired as they near the end of compulsory schooling.
• The U.S. sample for the latest results includes both public and private schools, with 165 schools and 5,233 students participating in all. Schools are randomly selected and 15-year-old students within those schools are randomly selected.

2009 Results

• Reading: The U.S. average score in reading (500) was not measurably different than other OECD countries. U.S. female students scored higher than male students.
• U.S. 15-year-olds had an average score of 500 on the combined reading literacy scale, not measurably different from the OECD average score of 493. Among the 33 other OECD countries, 6 countries had higher average scores than the United States, 13 had lower average scores, and 14 had average scores not measurably different from the U.S. average. Among the 64 other OECD countries, non-OECD countries and other education systems, 9 had higher average scores than the United States, 39 had lower average scores, and 16 had average scores not measurably different from the U.S. average. 

• Math: U.S. average score in math was lower than the OECD average. Male students, in general, scored higher than female students. Since 2006, U.S. has caught up with 9 countries.
• U.S. 15-year-olds had an average score of 487 on the mathematics literacy scale, which was lower than the OECD average score of 496. Among the 33 other OECD countries, 17 countries had higher average scores than the United States, 5 had lower average scores, and 11 had average scores not measurably different from the U.S. average. Among the 64 other OECD countries, non-OECD countries, and other education systems, 23 had higher average scores than the United States, 29 had lower average scores, and 12 had average scores not measurably different from the U.S. average score. 

• Science: 12 other OECD countries had higher average scores than the United States.
• On the science literacy scale, the average score of U.S. students (502) was not measurably different from the OECD average (501). Among the 33 other OECD countries, 12 had higher average scores than the United States, 9 had lower average scores, and 12 had average scores that were not measurably different. Among the 64 other OECD countries, non-OECD countries, and other education systems, 18 had higher average scores, 33 had lower average scores, and 13 had average scores that were not measurably different from the U.S. average score.
• Male students scored higher than female students. Overall score was higher than 2006, and the gains in science exceeded those for math.
• The US is one of three nations that give more money to highly advantaged schools than to disadvantaged schools.
• Overall, private schools do better on PISA...until you account for SES.
• There are number of high performing economically disadvantaged schools in the US: "success is possible against all odds."

U.S. Strengths and Weaknesses

• U.S. students showed the best relative performance in answering questions that judged students’ ability to reflect and evaluate information. On that measure, the United States ranked seventh out of the 34 OECD nations.
• The weakest area for U.S. achievement was in accessing and retrieving information, for which students tied for 19th place with France.

Behind the Facts

• The PISA rankings are determined by nations’ average scores. "Some researchers have suggested, however, that average score comparisons are not useful: even presuming that the tests have some meaning for future accomplishment, average students are not likely to be the leaders in fields of mathematics and science."
• In the last administration of PISA, the United States has 25% of all high-scoring students in the world. Among nations with high average scores, Japan accounted for 13% of the highest scorers, Korea 5%, Taipei 3%, Finland 1%, and Hong Kong 1%.
• The fact that one of four high-scoring students came from the United States and the remaining high-scores came from the other 58 countries participating "suggests that many American schools are actually doing very well indeed."
• "Well-resourced schools serving wealthy neighborhoods are showing excellent results. Poorly resourced schools serving low-income communities of color do far worse."
• The U.S. had many more students scoring at the lowest levels; these kids likely can’t compete for the good jobs in the country."
• "Americans in low-poverty schools look very good, even in mathematics. They would be ranked third in the 4th grade (among 36 nations) 6th in the 8th grade (among 47 nations). This is important because while other developed nations have poor children, the U. S. has a much higher proportion and a much weaker safety net. When UNICEF studied poverty in 22 wealthy nations, the U. S. ranked 21st."
• The highest scoring countries have less diversity and less poverty.

PISA confirms what we already know. The U.S. is quite capable of producing top performing students in well-resourced schools serving middle class neighborhoods. Under-resourced schools in poor neighborhoods do not fare as well.

Resources:

OECD

Are today’s students prepared for the knowledge economy of the 21st century?

PISA: Who made the grade? (OECD)

Washington Post

Do international test comparisons make sense?

Hysteria over PISA misses the point