All Topics for this Forum
Topic: "Making sense of PISA"
Topic Posts
Topic started by: Brian Drayton on 12/12/13
"What does PISA mean to you, and for your work? The results of international tests of students' science and math knowledge get a huge amount of attention and commentary. Do they matter for your work, or the people you work with? How? Do you find a way to make use of these results"?
PISA -- what does it mean to you?
posted by: Brian Drayton on 12/12/2013 9:40 am
Do they matter for your work, or the people you work with? How? Do you find a way to make use of these results? Do you find that parents or others raise them as a concern?
My own attitude is pretty much in line with this video from the
American Federation of Teachers
http://www.youtube.com/watch?v=hf9UVg-TdH0
Another pair of reactions to the value of PISA
posted by: Brian Drayton on 12/14/2013 10:33 pm
http://www.washingtonpost.com/blogs/answer-sheet/wp/2013/12/03/a-pisa- contradiction/
Another report, this time from Hechinger-- the role of poverty
posted by: Brian Drayton on 12/16/2013 1:48 pm
PISA Results
posted by: Arthur Camins on 12/12/2013 4:12 pm
Could these both be valid claims?
Critique of overreaching evidentiary claims is essential, but the results of the debate may not move us forward toward effective solutions.
I wrote more on this topic for a Huffington Post article. PISA Results: A Chicken Little Moment:
http://www.huffingtonpost.com/arthur-camins/pisa-results-a-chicken-li_ b_4404925.html
Arthur H. Camins
Equity and focus problems
posted by: Brian Drayton on 12/13/2013 11:41 am
But schools can take more of a lead on "focus" as you define it, it seems to me -- when policy does not prevent that.
PISA and the federal role
posted by: Andy Zucker on 12/13/2013 11:38 am
Because I think that the No Child Left Behind Act institutionalized many of the wrong approaches to improving education, albeit in a bi-partisan manner (gee thanks, Ted Kennedy) -- remember that K-8 science education was pushed aside, for one thing -- I am skeptical of any future strong role for the U.S. Dept. of Education. The federal government is too blunt an instrument to trust in classrooms beyond a certain point. So, yes, PISA points to national needs in education, some of which -- like dramatic inequality in the U.S. among students and their families, or access to early childhood education -- are not really about K-12 schools. Yet we need to be cautious about prescriptive federal laws, especially in this highly partisan environment. How can our STEM education community encourage changes in the structure of U.S. schools, such as providing principals and teachers with more time for PD and work with colleagues? I wish I knew the answer.
Making Sense of PISA
posted by: Arthur Camins on 12/14/2013 10:14 am
One of the threats on the horizon for science education is a rush at the state level to put consequential science tests in place to assess the NGSS, not so much because science or engineering is valued, but because of requirements of new "value-added" teacher evaluation systems. So, we need push back at the state level with respect to test-based teacher evaluation systems to give teachers time to adjust curriculum and instruction and to revise NGSS as it is tried out in practice. More on that here: http://www.arthurcamins.com/wp-content/uploads/2013/04/NGSS_Wave-or-Ri pple2.pdf
Arthur H. Camins
"The Learning Tower of PISA" ?
posted by: Keith Sheppard on 12/15/2013 8:00 am
To put this in perspective, imagine reporting an average value for a "United States of Europe" (or Asia)- the result would be meaningless given the vastly different education systems that would be somewhat arbitrarily placed together. This is precisely what is being done by producing an average for the USA. As several commentators have noted the average values can result from widely disparate input data or outliers (controlling for poverty, for example, produces a completely different view of the results).
Making sense of PISA
posted by: Richard Askey on 12/17/2013 8:42 am
See page 100 for how well a few countries did on the PISA problems.
Shanghai had only one question where less than 40% of their students got it right, South Korea had eight, the US over 20.
The meaning of comparisons?
posted by: Brian Drayton on 12/18/2013 10:11 am
Thanks for this post, which adds some useful details to the discussion. What for you are the most important implications of this data?
Making sense of PISA-exam questions in China
posted by: Richard Askey on 12/19/2013 7:30 am
http://www.icme12.org/upload/submission/2034_F.pdf
Different exams are given in each area, and are developed locally. Shanghai is one of the areas, and according to the author, a math educator at East China Normal University, Shanghai's program is the most conservative of the areas studied in this paper. Some sample problems are given, and comments on the different areas are mentioned. Here is part of the conclusion: The tradition of mathematics education in China stress solid foundation of
basic knowledge and basic skills. It is believed that creative abilities and individual development is based on a sound mastery of basic knowledge and skills.
PISA does not stress basic knowledge and basic skills at any more than a low level. Shanghai shows that with a solid knowledge of them, and problems which encourage thinking and reasoning, it is possible for a much higher percent of students to do well than we think is possible.
Making sense of PISA-exam questions in China
posted by: F. Joseph Merlino on 12/20/2013 9:39 am
"It is believed that creative abilities and individual development is based on a sound mastery of basic
knowledge and skills."
The key word here is "believed". There is no causal evidence for such a belief.
Likewise
"a sound mastery of basic mathematical knowledge and skills could be a possible reason for Shanghai students' good performance in PISA mathematics 2009."
The key word here is "could." Such a casual connection is purely speculative in this paper.
Making sense of PISA-exam questions in China
posted by: Howard Dooley on 12/21/2013 10:14 am
I have only read the summary, so I apologize if these questions are overly simplistic.
PISA and other tests
posted by: Amy Cohen on 12/23/2013 3:22 pm
I am amazed that state are phasing in Assessment Consortium assessments for CCSSM almost before they have finished phasing in the materials allegedly aligned to the CCSSM. Surely the teachers I've worked with (N about 150) are very much concerned about that aspect of the CCSSM. Folks can hardly believe that they aren't "teaching to the (small number of released items from) the proposed tests."
Here is my favorite exchange on a released PARCC item.
Item shows a figure which is apparently a rectangle with one side labeled 3 and one side labeled 5. It asks for the length of the rectangle.
Mathematician to the math specialist discussing this on behalf of a state Dept of Ed. Would PARCC accept 3 as a correct answer?
Specialist to mathematician: Huh?
Me to specialist. A rectangle can have four sides of equal length. Does PARCC assume that "length of a rectangle" is the length of the longer side if one side is indeed longer?
Specialist: Please direct your concerns directly to PARCC's website.
me: Thanks
post updated by the author 12/23/2013
Another recent commentary
posted by: Brian Drayton on 1/14/2014 2:27 pm
http://www.tcrecord.org/Content.asp?ContentID=17371
OECD's PISA: A tale of flaws and hubris.
Yong Zhao analyzes, disbelieves PISA results
posted by: Brian Drayton on 4/7/2014 6:24 am
http://nepc.colorado.edu/blog/how-does-pisa-put-world-risk
(this blog post has links to 3 earlier ones)
P.S. He is not afraid to state opinions! But his analyses are often very compelling.
post updated by the author 4/7/2014
Making sense of American education
posted by: Andy Zucker on 4/7/2014 8:55 pm
PISA is imperfect, yes, but can it be useful nonetheless? When PISA reports that there is a greater correlation in the U.S. between poverty and test scores than in many other nations isn't that information that fits well with research by Linda Darling-Hammond and many others? PISA suggests American teachers are not well paid compared to many nations; is that wrong? When 26% of U.S. students don't reach Level 2 (out of 6) on PISA's math test, should we simply complain about poor sampling in Shanghai and excessive test prep skewing their results, or should we ask what we can do to improve U.S. math education? Yong Zhao may identify some real problems with PISA -- but what does he recommend we do to improve American education?
"Making sense" and PISA
posted by: Brian Drayton on 4/8/2014 7:35 am
I agree that even a very flawed data set can be used to ask important questions, such as those you raise. It is because those questions are not very prominent in the public discourse spurred by the PISA scores that dissenting voices need to be heard.
After all, it is highly unrealistic, given the complexities of the United States economic, justice, and education systems, to seek for a panacea. Such an illusory quest tends to mean that, while policy makers pursue the latest thing, other significant problems are demoted, and removed from the model that is being addressed. This is one contribution to the problems you allude to, which remain problems because schools are embedded in society, and student interventions are therefore limited in their capacity to overcome the consequences of that "situatedness."