April 07, 2004
Chicago retention studies
The Consortium on Chicago School Research at the University of Chicago has just released two reports on the "Gate" policy that sets rules for promotion and retention of students in 3rd, 6th, and 8th grade. The reports are all over the newspapers and the slant is that Chicago's policy is a failure.
Chicago Sun Times: Researchers blast policy of flunking kids, by Rosalind Rossi, April 7, 2004.
USA Today: Studies: 'Social promotion' fight didn't help in Chicago, April 7, 2004.
Chicago Tribune: Holding kids back fails too, study says. U. of C. report finds repeating grade no help, by Lori Olszewski, April 7, 2004.
New York Times: Studies in Chicago Fault Holding Back of 3rd Graders, by David M. Herszenhorn, April 7, 2004.
The New York Times writes:
The studies offer the most comprehensive examination to date of a large urban school system that adopted a stringent policy of holding back its lowest-achieving students based on test scores.
If that is true then it is sad. The studies are very limited in scope, although you wouldn't think it from the press reports. There are two reports and they are here:
Ending Social Promotion: Dropout Rates in Chicago after Implementation of the Eighth-Grade Promotion Gate, by Elaine Allensworth, April 2004.
Ending Social Promotion: The effects of retention, by Jenny Nagaoka and Melissa Roderick, April 2004.
The first study has a very narrow focus on drop-out rates. The second study takes a very slightly broader look, but the "effects" indicated in the title are still predominantly the drop-out statistics. The studies can be read as providing a negative assessment, but certainly not of the retention policy comprehensively.
It seems a reasonable hypothesis, one worth investigating, that retention has benefits for the academic achievement of students being retained because they will be placed in a class more in line with their achievement level. The first of the mentioned studies does not address this issue and the second study addresses it peripherally and concludes that the hypothesis doesn't hold up to scrutiny.
In both studies the focus is on drop-out rates (in the first one exclusively, in the second one predominantly), and the studies conclude that retention increases the probability that a student will drop out (this is by comparing a cohort that is retained with a cohort of similarly poor achievement that is not retained). Well, duh, I say. If two students of equal achievement level and equal age are placed in a situation where one is looking ahead to three more years of school and the other to four more years of school, the second one may well have better reasons to drop out, and in any case has an extra year to make that choice.
The studies do not seek to answer the question if the threat of retention offers motivation to perform better, and they do not ask if a retention policy benefits the students that are promoted by, perhaps, creating somewhat more homogeneous classrooms. Lots of other questions are not addressed.
Take away the pretensions of these studies and take away the overblown reporting, and one is left with the observation that low-achieving students are likely to remain low-achieving under Chicago's policies, both past and present. Well, that is a challenge for Chicago, but not necessarily an indictment of either their past or present policies. As a researcher one might just want to look beyond the drop-out rates and focus instead on what is taught and learned. Too bad that the Consortium on Chicago School Research didn't think that question very much worthy of their attention.
February 08, 2004
What are those reasons we study history?
I owe the following extended quotation to Sharon Collopy, who posted it somewhere with a challenge to identify the source. I should have been able to - I have it - but it is too long ago that I read it. How about the readers of this Blog?
"I agree with lobbyists for federal school aid that education is one of the great problems of our day. I am afraid, however, that their views and mine regarding the nature of the problem are many miles apart. They tend to see the problem in *quantitative terms*-not enough schools, not enough teachers, not enough equipment. I think it has to do with *quality*: How good are the schools we have? Their solution is to spend more money. Mine is to raise standards. Their recourse is to the federal government. Mine is to the local public school board, the private school, the individual citizen-as far away from the federal government as one can possibly go. And I suspect that if we know which of these two views on education will eventually prevail, we would know also whether Western civilization is due to survive, or will pass away.........
In the main, the trouble with American education is that we have put into practice the educational philosophy expounded by John Dewey and his disciples. In varying degrees we have adopted what has been called "progressive education."
Subscribing to the egalitarian notion that every child must have the same education, we have neglected to provide an educational system which will tax the talents and stir the ambitions of our best students and which will thus insure us the kind of leaders we will need in the future.
In our desire to make sure that our children learn to "adjust" to their environment, we have given insufficient opportunity to acquire the knowledge that will enable them to *master* their environment.
In our attempt to make education "fun," we have neglected the academic disciplines that develop sound minds and are conducive to sound characters.
Responding to the Deweyite attack on methods of teaching, we have encouraged the teaching profession to be more concerned with *how* a subject is taught than with *what* is taught. Most important of all: in our anxiety to "improve" the world and insure "progress" we have permitted our schools to become laboratories for social and economic change according to predilections of the professional educators. We have forgotten that the proper function of the school is to transmit the cultural heritage of one generation to next generation, and to so train the minds of the new generation as to make them capable of absorbing ancient learning and applying it to the problem of its own day.....
Read the full Blog entry, "What are those reasons we study history?".
January 12, 2004
Hillary Rodham Clinton's battles for education
New York's senator Hillary Rodham Clinton sent a letter to her constituents reviewing the challenges and opportunities of the past year. Among her concerns is, naturally, education. See what she has had to offer, and please be advised that this all that she had to say about education.
January 9, 2004
The 108th Congress has adjourned for the year, and I thought it was a good time to update you on the past twelve months. It has been an active session full of challenges and opportunities.
Like many of you, my top concerns this past year were economic security, homeland security, and national security. At the same time, I have continued to fight on other vital fronts, concentrating on efforts to improve health care, promote education, and protect the environment. It has been a tough battle. But I have been working hard to represent the state of New York and fight for the interests of New Yorkers.
I would like to conclude this update by telling you about an innovative, yet simple program that helps both New York students and apple farmers. Teaming with General Mills this summer, we launched the "Apples for Education Program." Students across New York State can "harvest" stickers from New York State apples and place them on posters in their school cafeterias. Schools redeem the posters for cash through General Mills' "Box Tops for Education Initiative." This program exemplifies what we can do when we work together: industry and education, business and government, students and farmers. It shows us that together, we can all prosper.
The past year was filled with challenges and opportunities. I am certain that 2004 will be as well. There is much to do for New York State and for New York families. I hope you will join me in making our New York community the best it can be.
Hillary Rodham Clinton
Thanks to Elizabeth Carson of New York City HOLD for forwarding this.
November 14, 2003
Education wars brewing in California
Education wars brewing, by Jill Stewart, San Fransisco Chronicle, Nov 14, 2003. In this superb article Jill Stewart expresses her unease about the incoming team of Arnold Schwartzenegger and his designated education secretary Richard Riordan.
FOR YEARS, I hammered Gray Davis as a weak leader and a bad governor. [...] [But] Davis excelled at one thing. He stopped a high-pressure crowd of educators and politicos hell-bent on reversing big advances that have ended 25 years of academic freefall in California's schools.
[...] Every year, opponents of reform bring forth politically motivated legislation to roll back reform, and the Democratic-led Legislature shamefully approves it. Every time, Davis vetoes the anti-reforms.
Indeed, Davis strengthened reforms adopted by the Board of Education under Gov. Pete Wilson. Davis' own Board of Ed backed rigorous academic standards that are tracked through testing so the public can see how well their schools teach subject matter.
Awful districts such as Los Angeles Unified saw student achievement in math and reading skyrocket after introducing Opencourt explicit phonics and English immersion, and retraining teachers who learned zilch at teacher colleges. But cities such as San Diego fought the reforms, and their student achievement tanked.
Stewart continues with a description of the battles that lie ahead with the anti-reformers, and she explains her unease with Riordan.
Riordan is pro-reform but doesn't grasp the Sacramento Education Wars. When I spoke to him, he did not volunteer details I believe should be on the tip of his tongue. I'm worried he will be drawn to trendy uber-discussions while the Legislature turns back the clock. [...] His job is to prevent anti-reformers from ramming California's education miracle back into the dark 1990s.
November 08, 2003
Shark Blog on K-12 in Seattle
Shark Blog, by Stefan Sharkansky, reports from Seattle on "Current Events, Smarter Investing and Fatherhood". Shark Watch has been keeping close watch on the local school board, the latest school board elections, and the recent superintendent search. Stefan Sharkansky's articles are rife with links, and I advise the reader to visit the original. It is a great Blog. Here are a few links to recent entries on education, with just a snippet from each.
Seattle School Board (Nov 5, 2003).
The Seattle School Board has been captured by the loony-toons slate of Sally Soriano, Brita Butler-Wall, Darlene Flynn and Irene Stewart.
[...] Two of the new board members (Soriano and Butler-Wall) were endorsed by the Green Party, whose goal is to "transform pre-K-20 education in Seattle in alignment with all 10 Key Values of the Green Party of Seattle, through research, education, and advocacy". [...] Both Soriano and Butler-Wall propose eliminating the high-stakes WASL test, because the "use of WASL scores to label racial blocs of students as failures also constituted institutional racism" and/or because it reinforces the "public's perceptions about the 'failure' of our schools".
Putting the "achievement gap" in perspective (Nov 3, 2003).
Darlene Flynn, who has been endorsed by the teachers' union, might find it easier to blame an abstract bogeyman like "institutional racism" than to roll up her sleeves, understand the actual issues and propose actual solutions to actual problems.
Brita Butler-Wall and her nutty delusions (Nov 1, 2003).
It might be tempting to some people to blame an abstract bogeyman like "racism" for the underachievement of some children. But Brita Butler-Wall either suffers from delusions, or she doesn't bother look at the school system's actual data, or both.
First of all, although the population of Seattle is only 8% black, about 25% of the school systems' administrators are black. If that's attributable to institutional racism, it's probably not the sort of racism that Brita Butler-Wall is fantasizing about.
Second, the school system's performance data [large PDF] indicates that by a number of quantitative measures (e.g. high school GPA, graduation rates, attendance rates, test scores, expulsion rates), Asian students (especially Japanese, Korean, Chinese and Vietnamese) outperform white students, and Latino students are doing better than black students. How this is explained by a lack of "recognition that people come from a variety of cultures" or a lack of books in foreign languages, is a big mystery to me.
Sally Soriano (Oct 31, 2003).
Soriano is right to be concerned about the disparity in achievement between different ethnic groups, but I am unpersuaded by her understanding of the problem or its solutions: [...]
The district can tackle this problem by reducing class sizes, giving teachers more time to collaborate with each other and embedding the curriculum with awareness of racism, sexism and classism, Soriano said.
I have no idea what "embedding the curriculum with awareness of sexism" actually means, or what it has to do with the achievement of black students. Furthermore, under the Soriano regime, we would never know whether or not we were making progress on the achievement gap, because:
Sally Soriano said the use of WASL scores to label racial blocs of students as failures also constituted institutional racism.
Naturally, Soriano is endorsed by all the elements of the clueless, lunatic left, including: Green Party of Seattle, Seattle Post-Intelligencer and both weekly newspapers.
Seattle Superintendent Saga (Oct 6, 2003).
The attempt to recruit a superintendent for Seattle public schools continues to spiral out of control, as the hysterically self-interested teacher unions have trashed the only qualified candidates with the active assistance of the local newspapers. The second of the four finalists withdrew her name from consideration today, writing that
increasing polarization of this process makes it difficult to accomplish a common vision for educational leadership.
the school district's search consultant added that
all the candidates have been publicly humiliated and harassed. They felt the media coverage was very unprofessional.
Now, the only candidates left standing are the very impressive former Cincinnati superintendent Dr. Steven Adamowski and the less capable (but union favorite) Dr. Evelyn Williams Castro.
Seattle has a new superintendent (Oct 7, 2003).
The Seattle superintendent selection saga ended tonight, at least for now. The school board voted 6-1 to name interim superintendent Raj Manhas as superintendent on a one-year contract with an option to extend for a second year. I attending this evening's school board meeting. It was less a business meeting for a public authority than a group therapy session. It reminded me of the house meetings at the co-op where I lived my sophomore year of college.
Houston drop-out statistics and education 'miracle'
Education 'Miracle' Has a Math Problem. Bush Critics Cite Disputed Houston Data, by Michael Dobbs, The Washington Post, November 8, 2003 (front page). The article reports on the fraudulent drop-out statistics in the Houston school district during the tenure there of now education secretary Rod Paige.
This image of integrity is not supported by the Washington Post article, or by earlier reporting on the Houston drop-out statistics. The WP article reports on clear fraud at one high school and continues:
Conceding that individual "indiscretions" may have occurred in a school system that serves more than 200,000 students, Paige described the Houston Independent School District as "the most evaluated school district in the history of America." He said he places great stock in the credibility of an accountability system that demands quantifiable results from administrators, teachers and children.
"The whole system for me rode on integrity," Paige said.
An investigation by state auditors showed that at least 14 other Houston high schools, including Austin, reported unusually low dropout rates in 2000-2001, although there is no evidence administrators falsified data. By reporting a dropout rate of less than 0.5 percent, school principals increase their chances of winning bonuses of as much as $10,000 and earning top accountability ratings for their campuses.
After years of relying on dropout statistics as a key component in their annual accountability studies, school officials concede that they were worthless all along. "The annual dropout rate was a crock, and we're not [using] it anymore," said Robert R. Stockwell Jr., the district's chief academic officer.
In this matter, and writing from quite a distance without local knowledge, I am a lot more sympathetic to the school officials than to the Texas Education Authority officials. The TEA decided that schools must report drop-out statistics, but of course schools have no good way of knowing which students have moved to another district, switched to another school, or have really dropped out - and the school officials may well decide that it is none of their business and not worth their time to try to track down students that have left.
The WP article also pays attention to score inflation in Houston on the Texas 10th grade test - a very important benchmark in their high school accountability system. I am not surprised and not disturbed that many students are held back in 9th grade. The WP article, however, mentions cases - not clear if they are isolated or part of a pattern - of students that are held back twice in 9th grade and then advanced to 11th grade, thus not polluting the 10th grade test scores.
October 30, 2003
Educational reform in Quebec
Grading policy overhauled in Quebec. Larry Braden posted this item from the Montreal Gazette (October 29, 2003) to a mailing list. Quebec's educational reform, which was started in 2000 in grades 1 and 2 of the elementary schools, is approaching secondary schools, and some observes are befuddled.
"The policy is definitely progressive and brilliant," said Elizabeth Therrien Scanlan, executive director of the Quebec Association of Independent Schools. "But it still isn't very clear."
And that appears to be a supporter speaking. The reform includes grouping two grade levels together into two-year cycles, and this was started in 2000 when Grades 1 and 2 became Cycle 1.
Evaluation: Evaluation is no longer just about giving out marks, or determining whether a student has learned his or her stuff. Now, evaluation is defined as a way of helping students learn better.
Students can only be held back at the end of a two-year cycle, rather than at the end of the school year. And students are to held back only in extreme cases, such as missing classes for most of the year.
It seems that the fuzzies are in charge of education in Quebec.
October 04, 2003
UC Berkeley admissions policies
UC Berkeley Admissions Scrutinized. The Los Angeles Times reports on a confidential report on UC Berkeley admissions. According to the Times the study found that hundreds of highly qualified applicants were rejected in favor of freshmen who were 'marginally academically qualified'. It is apparently a preliminary analysis:
The report was prepared at the request of regents Chairman John J. Moores. It is based on university data, but contains extensive analysis that primarily was written by Moores. The report does not attempt to explain the reasons for UC Berkeley's admissions patterns. It does not break down admissions by race, ethnicity or socioeconomic status, nor does it measure changes over time.
We take note of that disclaimer. The study nevertheless finds that overall, the admissions process at UC Berkeley "might not be compatible with [the school's] goal of maintaining academic excellence." For defence of the process the LA Times article quotes the head of admissions committee, an education professor, who says that the campus is in full compliance with the regents' stated policy on admissions. The admissions office finds other indicators of academic strength, notably high grades.
The study relies primarily on SAT data for its conclusions, and UC Berkeley certainly does well to look much beyond the SAT (the SAT-I, that is) in assessing its applicants. But I think that if they really mean to look for academic strength then a better measure than school grades would be performance on some collection of high quality standardized tests. Please see also my July 18, 2003, Blog entry UC Regents approve new admissions test policy.
September 28, 2003
SAT scores and HOPE scholarship in Georgia
Splashed over the front page of the Atlanta Journal Constitution (Sun Sep 28) is the headline: Perdue wants to add SAT score to HOPE requirement.
As was extensively reported last month Georgia ranks 50th in average SAT scores, and this is a matter of some local embarrassment. (The state ranks 15th in participation rate, so the low score is not as bad as may be thought at first.) A few days ago I reported here on the response of Georgia's Superintendent of Schools, Kathy Cox, to the low SAT scores, and I explained why I think that her focus on test preparation misses the mark. In the present AJC article, reported by James Salzer, Georgia Governor Sonny Perdue voices his views.
In an interview with The Atlanta Journal-Constitution, Perdue argued that forcing students to obtain a minimum SAT score to earn the HOPE scholarship - combined with the only current requirement, a B grade average - would help boost Georgia's dismal national education rankings. [...]
"Knowing that there is a perception of grade inflation, I got to thinking about where our SAT scores would be today if, 10 years ago, they had been a component of the HOPE scholarship," Perdue said Friday. "My theory is, we wouldn't be 50th out of 50 states."
The question is, he said, "Are we willing to think of this [HOPE] as a merit-based scholarship which addresses a serious issue of lagging behind in SAT scores?"
[HOPE - Helping Outstanding Pupils Educationally - provides financial assistance for study at eligible Georgia public and private colleges and universities, and public technical colleges.]
Governor Perdue's suggestion looks as misguided as the policy of Superintendent Cox with regard to the low SAT scores, and I don't think the proposal is advisable policy for the HOPE scholarship either. With regard to the SAT scores the state will have to focus on instruction in grades K-8, and then not expect an effect on the scores next year, or the year after that. (Please note that "SAT" refers in all the local reporting to the SAT-I; the traditional math and verbal test with focus on aptitude.) For a college scholarship it looks not unreasonable to take into account SAT scores, which are generally held to be a good predictor of college success, but one should not confuse the issue and present it as a way to improve high school education. For the latter purpose, and it would be a fine objective for Gov. Perdue, one should consider scores on some proper test of school learning. The SAT-II subject tests come to mind, or something like the traditional New York State Regents examinations, or some good quality statewide high school exit exam.
The AJC article raises the familiar sensitive issue and reports the Governor's response.
"States have seen a much lower percentage of minority students qualifying [for scholarships] when they add test scores," said Gary Henry, a Georgia State University researcher who has studied the HOPE program for much of its 10-year existence.
Perdue said that doesn't have to happen in Georgia.
"I think that's an example of falling prey to the bigotry of low expectations to say that African-American children can't be successful on the SAT," the governor said.
That is a rather harsh response to something that wasn't said. As the AJC article points out, the average score of Georgia's white students was 1035 and that of the black students was 852.
Henry, the GSU researcher, offers this perspective.
"The SAT is not based on the curriculum that students take during high school," he said. "I think the SAT [requirement] would undermine students' current incentive to work hard all four years of high school and further, it would potentially put the whole scholarship program in question."
I'm with Henry on this matter, and I think that Governor Perdue's suggestion to include the SAT score among the HOPE requirements is poor policy. This balloon should not fly. It won't fly, I'm convinced, but this for an entirely different and irrelevant reason. If the SAT becomes part of the HOPE requirements then more students will take the SAT and this will be a burden on the average scores. I think that the Governor will have second thoughts about the matter, and the suggestion to add the SAT score to the HOPE requirements will quietly disappear.
September 21, 2003
Biology textbooks in Texas
In Search of Intelligent Life at the SBOE, by Michael King (Austin Chronicle, Sep 19, 2003). The Texas State Board of Education devoted a marathan session (the first of two) to a hearing of arguments for and against the proposed adoption of high school biology textbooks. The proposed adoptions represent mainstream science, and the anti-Darwinian forces claim that the adoptions are factually in error by being insufficiently critical of evolutionary theory. The article reports on the testimony and the testifiers, and also provides a good background on the relevant rules for textbook adoptions.
Core curriculum at HarvardStart making sense, by Patrick Healy (Boston Globe, Sep 21, 2003). Speculations about the direction of reform of the core curriculum at Harvard. Subtitle: "Critics say Harvard's curriculum fails to provide rigor, coherence, and basic knowledge. [President] Larry Summers is on a mission to change all that."
Today, every aspect of the curriculum is on the table -- the Core, the concentrations, faculty-student advising, students' writing skills, and even their speaking abilities. But there is a widespread belief on campus that the president is hoping most of all to transform the Core and its perceived emphasis on methods over content.
"When we consider the importance, embodied in the core, of exposing students to 'ways of knowing,' I hope that we will think more rigorously about the level of mastery we ask of our students, and more flexibly about how we let them acquire it,'' Summers declared at commencement.
It is an interesting article, placing the present curriculum reform in the context of the earlier core curricula created under the direction of James Bryan Conant (1945) and Derek Bok (1978). The present direction appears to be away from Bok and towards Conant.
SAT scores and Standards revision in Georgia
Bloggage has been a bit light on this page for the past month. I've been busy with a move from New York to Atlanta, and from New York University to Emory University.
What's up with K-12 education in Georgia? Well, the state came in last on the SAT scores (this is old news; it was all over the Georgia press in late August) and the state Superintendent of Schools, Kathy Cox, knew she had to do something.
As we all know, the SAT (more precisely, the SAT-I, which is the one at issue) primarily measures aptitude, and the formal schooling required for the SAT is concentrated in K-8. It looks as if education in Georgia has a problem with basic skills in middle school and before, and an appropriate response for Ms. Cox would have been to say "thank you, thank you, thank you" to the ETS and College Board for putting the finger on the spot, and then to work very hard to understand what is wrong and what must be done differently in the early grades. Instead, in a press release of August 26 Ms. Cox announced that in order to improve SAT scores statewide the education department would do four things: (1) [we'll get to that]; (2) expand availability of Advanced Placement classes; (3) increase participation on the PSAT; (4) institute professional development on PSAT analysis. This response, with its depressing focus on test preparation, rather misses the mark.
Item (1) in the press release is the following:
1. A Revised and Strengthened State Curriculum
Work on the Quality Core Curriculum (QCC) will be complete this fall, resulting in what Cox calls a "world-class curriculum that will establish high standards, maintain clear expectations, and place our schools and students not just at the top of the southeast, but at the top of the nation and the world."
I'm observing the work on that QCC revision for mathematics, and Ms. Cox's description is hubris of the highest degree. The process is managed, if not manipulated, by the National Center for Education and the Economy (NCEE), and will result in their kind of "performance standards" that emphasize "authentic work" instead of basic skills. These new standards will do nothing to lift education in Georgia. I'll have more to blog about this in due time, when the new standards are released in draft form. In the mean time I point the interested reader to a posting by Donna Garner and a follow-up by myself on a Georgia teachers email list.
No more F grades in Britain
It's official: you can no longer fail your exams reports the Daily Telegraph (UK). The British Qualifications and Curriculum Authority has directed that the "F" grade be replaced by an "N" (for Nearly) on the national curriculum exams. In a related piece of verbal inflation, Right and Wrong are replaced by Creditworthy and Not Creditworthy. The chairman of the Campaign for Real Education described the changes as "political correctness gone stark raving bonkers".
Talk of the Town on NCLB
Making The Grade, by Malcolm Gladwell (The New Yorker, issue dated Sep 15, 2003). In this Talk of the Town comment Gladwell takes on the NCLB's "Fordist vision of the classroom as a brightly lit assembly line, in which curriculum standards sail down from Washington through a chute, and fresh-scrubbed, defect-free students come bouncing out the other end". The article makes a useful point that in school ratings both the best and the worst performances are predominantly small schools, but this is presumably because small schools are more affected by statistical fluctuations. The article also points to the perverse incentives of the NCLB Act; an issue that was addressed with more care by Chester Finn in a Gadfly comment, A field guide to low academic standards, last year. Gladwell's concluding observation, that learning cannot be measured as neatly and easily as the devotees of educational productivity would like, misses the point, I think. Students can document their learning perfectly fine when applying to college, and States could make a better effort.
August 28, 2003
Poor marketing of NYC school reform
The New York Times has a misguided report by David Herszenhorn on NYC mayor Bloomberg's and schools chancellor Klein's poor marketing of their reforms.
When Mr. Bloomberg laid out the bulk of his education plans in a speech in Harlem in January, his proposals were received with general enthusiasm, even winning the initial support of the teachers' union president, Randi Weingarten.
But in the weeks after the mayor's speech, the administration failed to build the momentum, officials said, and instead became embroiled in an arcane debate over whether the proposed literacy curriculum had a strong enough phonics component.
Sadly, it seems entirely possible that mayor Bloomberg and chancellor Klein view the debate over how to teach reading as just arcane and petty. Their defence of the curricular mandates that chancellor Klein imposed on the New York City schools is never based on the substance of the specific choices, but only on the claimed need to have a unified curriculum throughout the city. A Freedom Of Information Law request done for New York City HOLD showed that there is no documentation, not even for the Department's internal reference, of the rationale for the specific choices of textbooks for reading and mathematics.
August 27, 2003
Interim report on NYS Regents Math A
Readers may recall the controversy over the difficulty of the June 2003 New York State Regents Math A exam. The results of the exam were tossed for juniors and seniors, and a panel was appointed to study what went wrong. For reference, here are links to Commissioner Mills's earlier press release and the charge to the Math A panel. Also for reference, my critique of the New York State Regents Math A exam.
The Math A Panel has now produced an interim report, and it is receiving plenty of press attention. (Go to Google News and do a search on 'regents "math a"'.) The best summary that I've seen is that of Karen Arenson in the New York Times.
The panel's interim report deals with only a very limited part of the charge, and deals with it in a disappointingly limited way. The panel clearly thought it was important to have a recommendation out before the start of the school year about a rescaling of the test. I am surprised that they only found the time to compare the June 2003 and the June 2002 instances; in the 6 weeks that they've worked they really might have had a serious look at, say, the past 6 instances of the exam, and this both in a qualitative and a psychometric way. Who knows, maybe the June 2002 exam was exceptionally easy.
In fact, though, the conclusions of the panel regarding the difficulty of the June 2003 instance match all the informed speculations that I've seen, including my own speculations: Parts 1 and 2 of the exam were in line with previous instances, and parts 3 and 4 were more difficult. For my critique I looked at August 2002, January 2003, and June 2003; and found June 2003 the hardest and January 2003 the easiest.
The interim report does not specifically criticise any officials or any actions, but I draw from it the conclusion that inexcusable errors were made in the development of this June, 2003, instance of the exam. In my earlier commentary I quoted an article by David Hoff in Education Week in which he quoted deputy commissioner James Kadamus as saying that the June, 2003, exam had more problem-solving questions than previous exams, because the state is gradually raising its expectations. I wrote then that this is a remarkable statement, because all previous reports indicated that the added difficulty of the June exam was unintended and had taken the Department entirely by surprise.
Now here is Karen Arenson, writing on the basis of the interim report of the Math A panel:
Based on field tests before the actual test was administered, the Education Department expected the average score on the June test to be 46. The expected average for the test given a year earlier was 51 slightly higher, but still below the score needed to pass, which is 65 for students who entered ninth grade in 2001 or later, and 55 for everyone else.
Did commissioner Mills know that the average scaled score of the June, 2003, exam was expected to be 5 points lower than that of June, 2002? (Arenson is mistaken, of course, to describe 51 as "slightly higher" than 46; the difference is large.) Public indications are that Mills did not know this.
I am still surprised that the error of the added difficulty was made in such a blatant way. For myself I had been speculating that a subtle error would have been made: the department might have used for its psychometric evaluation of the difficulty of the test a rather different population of students than the population that really matters. They might have had a test population with lots of bright 9th and 10th graders, and perhaps for that group the difficulty of the June 2003 exam was in line with earlier instances, while for the struggling seniors the added "problem solving" (i.e., aptitude oriented) focus of the exam would have posed more severe problems. But apparently the department did not make a subtle error; they were just completely wrong and out of control.
August 24, 2003
NYC schools chancellor Klein under fire
New York City schools chancellor Joel Klein has been under some heavy and well deserved fire recently for his curricular policies. This blog entry is based on articles and opinion pieces by James Traub, Sol Stern and Andrew Wolf; and on the Web pages of New York City HOLD.
On August 2 the New York Times educational supplement offered New York's New Approach, by James Traub. (The original article has gone off-line, and the link is to a copy.) Traub focusses on the literacy part of New York's "Children First" initiative.
[...] All New York elementary and middle-school students will have lengthy "literacy blocks" each day to focus on reading as well as writing skills. Teachers will read books aloud, engage in "shared reading" with the whole class, "guided reading" with smaller groups and "independent reading" from classroom libraries whose books will be carefully calibrated by skill level.
[...] Here was a form of teaching that built on the child's innate knowledge and love of learning, required virtually no rote instruction and permitted children to acquire information and understanding as a painless byproduct of pleasurable activities. It sounded delightful. But would it be effective?
Traub presents Klein as perhaps an unwitting captive of the city's
liberal consensus on pedagogical issues, and presents the deputy
chancellor for teaching and learning, Diana Lam, as the real force
behind the progressive pedagogy. Traub himself has no sympathy for
the direction chosen by chancellor Klein:
Read the full Blog entry, "NYC schools chancellor Klein under fire".
August 16, 2003
On travel, no Blogging for now
This August is a busy month for travel for me, and I don't expect to have anything new to offer here until September. For frequent education news, please check out Number 2 Pencil and Joanne Jacobs. I also recommend my standard Google education news search. For a collection of articles of some longer term interest please see the left column on this page. My Links, Articles, Essays, and Opinions Web page is a more extended, annotated version of same. Happy reading!
August 09, 2003
TAKS science problems
Two days ago I commented on a questionable item in the 10th grade TAKS mathematics test, for which scores were revised. The associated TEA press release refers also to a controversy over some science test items, and states that, upon review, these items were found to be correct. It is fascinating to see the items and to see how they are judged to be correct. The TEA (Texas Education Authority) put out an Additional Information Regarding Released Science Items for the spring 2003 testing cycle. Four controversial items are discussed.
Grade 5 Science, Item 13. Item 13 asked students which two planets are closest to Earth. Among the possible answers: Mercury and Venus, and Mars and Venus. The correct answer varies over time, and the question is plainly wrong or crazy. To add insult to injury: the intended answer was Mars and Venus, but on the day the test was given the correct answer was Mercury and Venus. Nevertheless, the TEA insists that for the purpose of the 5th grade test the question had only one correct answer - to wit, the wrong answer.
Grade 10 Science, Item 50. Item 50 looks crazy to me - they seem to be testing in a most convoluted way that the student knows that the element symbol K stands for Potassium. The TEA discussion indicates that the item is factually wrong to boot, but they insist that it is valid just the same.
Grade 11 Science, Items 11 and 45. Question 11 asks for the force exerted by a jumping frog on a leaf. The force has two components: one due to the weight of the frog and the other due to its acceleration. These are to be added vectorially, but the direction of the jump is not given. The TEA insists that therefore the correct treatment of the question must ignore the weight of the frog. Obviously the question is wrong and the TEA is wrong to insist that it is correct. Question 45 concerns a hypothetical situation in which a force is exerted on an object but no work is done. The question asks what can be concluded, and the intended answer is that the object is and remains at rest. This is wrong; the force may be perpendicular to the direction of motion. The TEA insists in effect that students don't know that, and that therefore the TEA's intended answer is, for the purpose of the test, the unambiguously correct answer.
The TEA has a bit of a quality control problem, obviously. In connection with the earlier 10th grade Math test problem Kimberly Swygert asked if the pre-testing might not have found the error. The same question could be asked for these science test items, but I think that it is too much to ask of the psychometric process that it correct for blunders of this kind.
I suspect that for many patently wrong questions students will nevertheless do what the TEA expects of them. The pernicious effect of the bad test items is indirect. It creates among the students and the public an impression (a correct impression) that the TEA doesn't have its house in order; that questions can't be read to mean what they mean; and that one should always be prepared to second-guess the clear meaning of a question.
A closing remark: the New York State Regents testing division has similar quality control problems. I remind the reader of the earlier discussion about the June 2003 Regents Math A exam, and my related Critique of the New York State Regents Mathematics A Exam
August 08, 2003
Advocacy research on NYC District 2
The Educational Policy Archives added a very interesting pair of articles earlier this week. The main article is Research or "Cheerleading"? Scholarship on Community School District 2, New York City, by Lois Weiner of New Jersey City University. This is followed by Reforms, Research and Variability: A Reply to Lois Weiner, by Lauren B. Resnick of the Learning Research and Development Center at the University of Pittsburgh.
New York's CSD2 covers Manhattan below Central Park (except for a small area near the Williamsburg bridge), as well as the East Side up to a boundary varying between 96th and 100th street. During the period covered in the cited articles the superintendents of CSD2 were Anthony Alvarado and (after Alvarado moved to San Diego) Shelley Harwayne. District 2 is a hotbed of whole language and constructivist mathematics teaching and is much celebrated in certain circles for those reforms. The principal cheerleaders from the education research community are Lauren Resnick and Harvard's Richard Elmore.
Anyone familiar with the District 2 area of Manhattan will know of the tremendous changes in the demographics and the quality of life that have occurred there over the past 25, and also the past 10-15 years. Chelsea, TriBeCa, and Alphabet City and surrounding areas east of 2nd Ave are all unrecognizable now from the way they looked in the 1980s. The distractions of urban crime and decay have been very much reduced over the same period. Any honest investigation into the success of school policies must pay careful attention to the effects of these demographic changes. Researchers Elmore and Resnick note the changes, but otherwise pay scant attention to them, as is documented well in Weiner's article.
The reply by Lauren Resnick is of interest in its own way. Resnick characterizes Weiner's article as "... at once an analysis of data on demographics and achievement in Community School District Two (CSD2) in New York City and an attack on the research strategy (and by implication the research ethics) of the High Performance Learning Communities (HPLC) project that I co-directed, along with Richard Elmore and Anthony Alvarado". Resnick also writes (in the abstract):
The intent of the HPLC investigation was always to link scholars and practitioners in a new form of research and development in which scholars became problem-solving partners with practitioners. There are important issues about how to profitably conduct such "problem-solving" research. These issues are worth substantial attention from the communities of researchers and practitioners as collaborative research/practice partnerships proliferate. Serious studies of such partnerships are needed, going well beyond the anecdotal attacks offered by Weiner in her article.
Or, to paraphrase: "Sure, we were engaged in advocacy research. Let others do the proper work." Guilty as charged, is my assessment.
August 07, 2003
TAKS 10th grade math problem
As reported in the Houston Chronicle the Texas Educational Authority yesterday announced a revision of the scores of the 10th grade TAKS exam that was given this spring, because of an error in one of the questions.
Readers with some knowledge of high school trigonometry may find it interesting to see the problem. The question is reproduced in the Houston Chronicle, or one can see the TEA original (look at question 8). The question shows a drawing of a regular octagon, indicating the inscribed radius as being 4.0cm and the circumscribed radius as being 4.6cm. The question is what is the perimeter of the octagon to the nearest cm. The choices are 41cm, 36cm, 27cm, and 18cm.
The data are contradictory: an octagon with inscribed radius 4.0cm has circumscribed radius about 4.33cm. Taking the 4.0cm and 4.6cm at face value a student might reason that the perimeter of the octagon is somewhere between 2*pi*4.0cm and 2*pi*4.6cm, and this leads to the answer 27cm in the multiple choice format. Or the student could apply trigonometry and obtain perimeter 26.5cm by starting from the given inscribed radius or 28.2cm by starting from the given circumscribed radius. A fourth approach is to use Pythagoras's theorem on a right triangle that has hypothenuse 4.6cm and one right side 4.0cm; then one finds that the circumference of the octagon must be 36.3cm. That (or rather, 36cm) was the intended answer.
According to the TEA press release, "item eight on the 10th grade math test could have been read in such a way that the question had more than one correct answer". That is putting a very kind spin on their blunder - there is in fact no reading of the question under which it has just one correct answer. It is amazing that the TEA would have this test composed and reviewed by people that fail to recognize that one cannot arbitrarily specify both the inscribed and the circumscribed radius of a regular polygon. According to the TEA press release: "Each test item goes through a rigorous review process that includes a field test of the items and two separate review sessions by professional educators who have subject-area and grade-level expertise and who are recommended by their district." The TEA didn't mean that as an explanation, but for me the "professional educators" part goes a long way just the same.
[Addendum, Aug 09. Please see the figure accompanying question 8 in the exam. The line segments that I described as inner and outer radii are not, in fact, identified as such in the figure or in the question. They meet at a point that certainly appears to be the center of the octagon, but that is not labelled either. There is, therefore, a reading of the question under which it has a single correct answer. Under that reading the given data are all correct, the special point is not meant to be the center of the octagon, and the figure is simply distorted in what happens to be a highly misleading way.]
July 28, 2003
HS for gay, lesbian, bisexual and transgender
School's Out, by Carl Campanile (New York Post, July 28, 2003).
The city is opening a full-fledged high school for gay, lesbian, bisexual and transgender students - the first of its kind in the nation, The Post has learned.
Operating for two decades as a small alternative program with just two classrooms, the new Harvey Milk HS officially opens as a stand-alone public school with 100 students in September.
[...] The Hetrick-Martin Institute - the gay-rights youth-advocacy group that manages and helps finance the school in conjunction with the Department of Education - has hired the school's first principal.
[...] [Principal] Salzman said Harvey Milk will be an academically rigorous school that follows Schools Chancellor Joel Klein's mandatory English and math programs. It will also specialize in computer technology, arts and a culinary program.
The New York Post claims the story as an exclusive, and my first reaction was to wonder if it would hold up. However, there does exist a Hetrick-Martin Institute in New York City and it is indeed home of the Harvey Milk School, which is at present offering some path to an alternative high school diploma. They are going mainstream, then. Meanwhile also New York Newsday has picked up the story and they quote Mayor Bloomberg as to why it is a good idea.
July 27, 2003
The Daley generation. A Chicago Sun-Times special report, July 27-28, 2003, in collaboration with the University of Chicago Consortium on Chicago School Research. The report deals with the progress of the 36,000 children that started first grade in a Chicago public school in 1995, just after Mayor Daley won control of the school system. Those that stayed on track have just completed eighth grade. Sunday's article focusses on grade promotion and retention, and Monday's article will look at why middle class children are still leaving the system at a high rate.
Education Secretary Defends School System He Once Led, New York Times, July 27, 2003. Based on a conversation with secretary of education and past superintendent of Houston schools Rodney Paige. The "Texas education miracle" has received plenty of well-deserved scepticism over time. Presently in the news are the result of an audit of drop-out figures from Houston public schools. According to the article, Houston as a whole reported a 1.5 percent annual dropout rate, though education experts estimate that the true percentage of students who quit before graduation is nearer 40 percent.
(Clearly the two rates, 1.5% and 40%, are not directly comparable. Is it 1.5% of the high school population or of the entire school population? For how many years should we count those 1.5% to obtain the reported percentage of students who quit before graduation? I am guessing 5 or 6 years. Even then the discrepency between reported and estimated drop-out rates is substantial.)
College Board Scores With Critics of SAT Analogies, LA Times, July 27, 2003. About the elimination of the analogies section from the SAT in response to demands by University of California president Atkinson. The famous "oarsman : regatta" question that appeared on the SAT several decades ago is hashed up as representative for the analogies questions. I trust that Kimberly Swygert will have something to say about this in her Blog on testing. [Addendum, 07/28: she has indeed, and Joanne Jacobs too.]
Is 'minimally adequate' education good enough?, The State (SC), July 27, 2003. My answer to the question in the title would be "Yes, of course, just barely", but a group of South Carolina school districts is arguing otherwise and wants a circuit Judge to order the state government to provide additional aid, beyond present additional aid, to rural and poor communities.
July 26, 2003
Shelley Harwayne retires
The New York Times and other NYC newspapers report the retirement of Shelley Harwayne. As superintendent of New York City's community school district 2 Shelley Harwayne was among the most visible proponents nationwide of the educational reform movement associated with whole language reading instruction and constructivist mathematics teaching. In the new governance structure that took effect just the start of this month Shelley Harwayne held the position of superintendent for Region 9: the largest region by number of schools in the system and encompassing most of Manhattan including the old CSD2. Ms. Harwayne is retiring to deal with health and family issues.
Shelley Harwayne has written at least 6 books and is a frequent speaker at national events; for example, keynote speaker at the NCTE Whole Language Umbrella Conference in Nashville (2000) and at the National Conference of the Reading Recovery Council of North America (2002), and giving the opening talk at the NCTE Whole Language Umbrella Conference in Bethesda (2002). Before becoming superintendent of community school district 2 Ms. Harwayne was the founding principal of the Manhattan New School. One of her books, Going Public: Priorities and Practice at the Manhattan New School (Heinemann, 1999) is based on that experience, and offers insight into the educational philosophy that guided District 2 and that has been influential throughout the NYC school system.
My own interest is mathematics and science education. Last year I read Going Public with that perspective and used it for a Web article, Shelley Harwayne and Mathematics. The present contribution is based on that longer article.
Ms. Harwayne's book has one chapter where one may look for academic ambitions of the school: Chapter 6, Talking Curriculum and Assessment. The issue of mathematics education covers about half a page in that chapter, and there is nothing at all about science education. In the half page about mathematics Shelley Harwayne describes how she marvels at what her children are able to do, such as renaming numbers, seeing patterns in hundreds charts, and performing great amounts of mental math. With little attention to algorithms her students understand how knowing that 6 x 7 = 42 helps you to know what 60 x 70 is, what 12 x 7 is, what 3 x 7 is, and so on. Observing the teaching of mathematics she realizes how little she knows and how much there is to learn.
Ms. Harwayne's limitations in mathematics did not prevent the CSD2 superintendency from taking a very active and damaging interest in mathematics instruction, removing curricular choices from the schools and teachers and imposing a sequence of reform mathematics curricula throughout the District that are roundly rejected by mathematics professionals. These curricula include TERC: Investigations in Number, Data, and Space in grade school, Connected Mathematics Program (CMP) in middle school, and Mathematics: Modelling Our World (COMAP) in high school.
The educational reform in district 2 gave rise to an opposition, and especially to New York City HOLD: an advocacy organization for parents, educators, mathematicians and others focussed on improving the quality of mathematics education in New York City schools. In spite of the efforts of NYC HOLD and others, at present the District 2 philosophy holds sway throughout the New York City school system.
July 22, 2003
Accountability in the NYC school system
Ronald Brownstein writes a Washington Outlook column for the Los Angeles Times. The column of July 21, 2003, Failing Schools Need Courses in Readin', Writin' and Accountability (also here in case the LAT link disappears) made some laudatory references to the performance of NYC schools chancellor Klein in his focus on accountability. I think it useful to provide a counterpoint.
Ronald Brownstein wrote:
Joel I. Klein, the accomplished attorney who was Mayor Michael R. Bloomberg's unconventional choice as schools chancellor last year, understands he can effectively educate the 1.1 million students in his care only if he shatters the cozy arrangements that have kept the New York City school system focused more on providing jobs for adults than on opportunities for kids. After 11 months on the job, Klein has the scars to prove his commitment to that cause. [...]
In conversation, it's apparent his greatest challenge is to impose more accountability for results on principals, teachers and the rest of the school system's 150,000 employees. "In public education," he says in a measured understatement befitting his days as a federal prosecutor and an assistant attorney general under Clinton, "the normal merit approach to service is very limited."
It's achingly ironic that Klein's headquarters is now in the 19th-century courthouse built near City Hall by legendary political boss William Tweed. Tweed's power rested on a patronage system that guaranteed jobs for even his most unqualified supporters. Klein presides over a $12-billion system whose work rules and union contracts make it dauntingly difficult for him to fire even the most incompetent. Last year, Klein initially hoped to remove 50 principals in woefully under-performing schools; he was only able to dismiss one. Just 132 of the system's 78,000 teachers last year were removed for inadequate performance.
The article goes on to argue that, besides accountability, also more resources, and specifically federal funds, are required in order to help schools make the grade.
Brownstein is correct to point out that so far there is little to show for chancellor Klein's focus on accountability. Where our new chancellor has made an impact is in the choice of a system-wide mandated curriculum for reading and mathematics, and here the result is entirely negative. In a secretive Children First process a K-5 mathematics curriculum, Everyday Mathematics, was selected that was twice rejected in the California textbook adoptions process. For reading the chancellor and his deputy decided upon "instruction based on classroom libraries" (what may safely be understood as "do as you like whole language") supplemented by a program that has phonics in the name but that was severely criticized by reading researchers; later a further supplement to the supplement was identified. Recently the chancellor also decided to preserve New York City's failed bilingual education program. There will be a relentless focus system-wide on the subjects of reading and mathematics, with no apparent concern for science, history, or the arts. Finally, chancellor Klein's personnel decisions to-date, including his choice of deputy chancellor for teaching and learning and his choice of which of the 32 local district superintendents to promote to one of the 10 newly created regional superintendent positions, give not much hope for his future focus on accountability.
For ongoing commentary on the state of mathematics education in New York City, please visit New York City HOLD.
July 21, 2003
High school exit exams under fire
Here is a sensible Op-Ed on California's decision to delay the high school exit exam requirement: Why the exit exam got held back instead of failing kids, by Daniel Weintraub (SacBee, Sun Jul 20 2003).
The California Board of Education's recent decision to delay the impact of the state's new high school exit exam was a disappointing but necessary tactical retreat that should ultimately advance the long-term goal of accountability in the public schools. If the ed board hadn't backed off, the legal dogs would have sued the state on behalf of thousands of students in the class of 2004 who would have been denied diplomas after failing the test. Their argument: These kids never got a fair opportunity to learn the material on which they were tested. Unfortunately, they are probably right.
The issue continues to play in New York State as well. The difficulty of the June 17 instance of the Regents Math A exam was misjudged and the exam had an unexpected high failure rate. Commissioner of Education Richard Mills might have responded by lowering the passing score, but instead he tossed the exam entirely. It must have given him great relief to get it off his back and not have anyone fail the Math portion this year, because he went a step further and extended the good news to juniors as well: any junior that sat the exam in June has it waived as a graduation requirement. The Board of Education has now named a panel to investigate the Regents Math A exam and respond to 9 specific questions. My take on the exam is in this critique of the Regents Math A.
In other news on high school exit exams, the Boston Herald has this:
STIRRING STORY: Tracey Newhart, an aspiring chef who has Down syndrome, failed her MCAS test, which could keep her from studying cooking at Johnson and Wales University.
The original is pay-per-view at the Herald. The story is also here in the Cape Cod times. How long ago was it that a grade school diploma, or at most an eighth grade education, would be the normal requirement for studying cooking?
Which brings me to this news item from Australia. Drop-outs can be successful, by Farrah Tomazin (The Age (AU), July 21, 2003).
A study tracking nearly 8000 students has found that many teenagers who do not finish year 12 earn more money and have higher job stability than those who stay but do not go on to university. Geoff Masters, chief executive of the Australian Council for Educational Research, which released the study, said the findings contradict the common belief that early school leavers tended to struggle after dropping out. 'It's often believed that students who leave school before the end of year 12 are at risk of not making a successful transition from school into the workforce,' Professor Masters said. 'But when you compare them with students who finish year 12 but don't go on to university, the early leavers are more likely to be working full-time, have a degree of job stability and be in a job that they like.'
It sounds plausible to me. The trend to demand a high school diploma for everything and look down on those that leave high school without a degree mainly reflects a steady lowering of standards for the eighth grade education.
July 18, 2003
UC Regents approve new admissions test policy
The San Fransisco Chronicle reports on actions by the UC Regents at their meeting of July 16-17. A 25% fee increase, a ban on romances between faculty and their students, and then this.
In other action, the regents approved a new admissions test policy to require students entering UC in 2006 to take a new SAT I that includes a writing exam, or the ACT with an added writing component. In addition, students will have to take two SAT IIs in subjects such as history/social science, English, mathematics, laboratory science or a foreign language. They currently take three SAT IIs: math, writing and one of their choice.
UC initiated the changes in its testing policy in 2001 after President Richard Atkinson said that the SAT I did not test what students were learning in the classroom. After he called for UC to scrap the test, the College Board decided to revamp it, dropping the analogies, changing the math and adding a writing component.
Under the new UC requirement, the SAT II exams will no longer be weighted twice as heavily as the SAT I.
"I think this is sending a message to the schools on the importance of writing at an early age and focusing on certain mathematics skills," Atkinson said. "I think we have accomplished a great deal."
Now, I recall Atkinson's call to drop the SAT-I requirement, and I recall when in response the College Board announced that it was changing the nature of the SAT-I by adding a writing component. Some notable critical comments at the time came from Stanley Kurtz in NRO and from Heather Mac Donald in City Journal. I also recall comments by Columbia University President Lee Bollinger.
As I understood the issue, the claimed intent of Atkinson and the California Regents was that the UC system would rely for its admissions decisions more on subject oriented tests and less on an aptitude test. Looking at the whole enchilada as the SF Chronicle presents it, however, I don't see this happening in a significant way. Previously the applicants submitted a two-part (three-hour) SAT-I and three one-hour SAT-II subject tests and the subject tests were weighted twice as heavily as the SAT-I (I assume it means the subject tests together carried twice the weight of the SAT-I). Now they have a three-part SAT-I and two SAT-II tests, and I guess that they are weighted equally. It looks like a marginal change, and it is hard to discern if it increases emphasis on subject matter at all. Applicants no longer have to take the SAT-II mathematics test. Moreover, as I understand it, the standardized tests continue to have less weight than high school grades.
I'm sympathetic to an increased emphasis on standardized subject matter tests in college admissions decisions. Let universities move to requiring their applicants to submit test scores for at least English writing, English reading and literature, mathematics, two or three sciences, one foreign language, U.S. history, and world history. Leave it to the applicants to choose in each case what test best fits their background: an SAT subject test, an AP or IB, or something like the New York Regents. Use those tests scores in addition to the high school transcript - it will offer a mechanism to keep high school grades honest at the same time.
July 17, 2003
Women in Mathematics
In the Chronicle of Higher Education Robin Wilson (login required) writes about programs boosting the number of women PhDs in mathematics. She explains the effort by:
Women in the field are particularly at risk: In 2002, 42 percent of the undergraduate mathematics majors in the country were women, but only 31 percent of those who earned Ph.D.'s in math that year were women, according to the American Mathematical Society. And only 13 percent, or 127, of those who earned doctorates were female U.S. citizens. In the professoriate, in 2000 only 17 percent of those tenured in math at four-year institutions were women.
Seems to me that 42% undergraduates and 31% of doctorates are very respectable numbers. Sure they can grow--given that there are enough women interested in PhD in math. The argument of only 13% US citizens is spurious. In her eagerness to show the female "disadvantage", the author strays to bigotry. What would satisfy her? Prohibiting legal residents to attend PhD programs? Numerus clausus for foreign students? 50% US female citizens plus 27% non-citizen women?
This seems like yet another attempt to find "disadvantage" where there is none. Should we now go and check the percentage of women in journalism or medicine? In teaching? Bookkeeping? Engineering? Clearly there were professions that in the past discouraged women (or men). However, there should be a limit to such bogus equity argument. Isn't it only natural that certain disciplines are more or less attractive to different sexes? 42% and 31% look pretty decent when viewed in this light.
The real problem is that our education system does not prepare enough US students to study math, science and engineering. That is why those programs, and the graduate ones in particular, have plenty of foreign students. Both male and female.
July 16, 2003
Apples to apples evaluation of charter schools
The Manhattan Institute Center for Civic Innovation has just released the Working Paper Apples to Apples: An Evaluation of Charter Schools Serving General Student Populations, by Jay P. Greene, Greg Forster, and Marcus A. Winters (July, 2003). In the report the authors confuse the ability of schools to improve themselves with their ability to improve their students, as this Web contribution will explain. It is an elementary error that completely invalidates the report.
I remark that in February, 2003, the same authors produced a report Testing High Stakes Tests: Can We Believe the Results of Accountability Tests?. I wrote a Web review of that report in which I explained that the authors confused the predictive power of a high stakes test with its validity as a measure of student learning. That too was an elementary error that completely invalidated the report. (In both cases the report's conclusions are plausible, but that is besides the point.)
The present Apples to Apples report sets out to compare the
performance of charter schools with that of public schools serving
similar populations. (Given the wide range of educational policies in
place in charter schools as well as in public schools I'm not sure
that the question is all that interesting, but let's accept the
question anyway.) In order to compare similar schools, the report
focusses on charter schools that serve a general student population,
and the control group of public schools is formed by taking for each
charter school the nearest public school that also serves a general
Read the full Blog entry, "Apples to apples evaluation of charter schools".
July 14, 2003
'Early College' high schools
The New York Times has a curious report today: 'Early College' Gains Ground in Education, by Karen Arenson (July 14, 2003). The article reports on two new schools in NYC that belong to a national wave of "early-college high schools". An excerpt:
Neither new school will require an admissions test, as Stuyvesant High School, the Bronx High School of Science and three new high schools opened on CUNY campuses last fall do.
Many high schools already offer their students college-level work through Advanced Placement courses or classes at local colleges, but those who take them tend to be the strongest students.
These early-college schools aim to make such advanced curriculum the norm for every one of their students, not just the handful at the top. The concept is based on the notion that less accomplished students - including those in danger of dropping out - are capable of handling more difficult work, and that more of them will graduate if they are challenged more. That idea appeals to school officials at a time when as many as half of ninth graders are dropping out before graduation in many cities.
The refusal to have an entry test, the dismissal of Advanced Placement classes, and the focus on students at risk of not graduating in the first place all lead me to wonder just what kind of view these schools have of college-level work. I'll venture a guess: lots of projects and independent work; not much learning; and it won't be certified by college level exams.
Academic bar lowered to get schools on track, Arizona Republic, July 14, 2003. "Arizona isn't alone in lowering passing scores on standardized tests and setting up dual rating systems to help schools meet tough new student achievement goals. [...] This year, Arizona will lower its proficiency rate for the math portion of Arizona's Instrument to Measure Standards, the big state test. [...] Texas has lowered passing scores for third-graders in math and reading. [...] Colorado used to have a four-tiered state accountability system, but to appease new federal rules, lowered its standard of 'proficient' to include what used to be deemed 'partially proficient.'"...
Edison Schools Announces Merger Agreement with Management Team and Liberty Partners to Take Company Private, PRNewswire, July 14, 2003. "Edison Schools (Nasdaq: EDSN - News), the nation's largest manager of public schools, announced today that it has signed a definitive merger agreement with a company formed by Chris Whittle, its Founder and Chief Executive Officer, and an affiliate of Liberty Partners, a private equity firm based in New York City"...
Analysis: Darwin's defenders inspire debate on education board, Lawrence Journal World (KS), July 14, 2003. "If evolution plays a major role in Kansas politics during the next 18 months, Darwin's defenders - not Darwin's detractors - will have revived the debate"...
Putting the brakes on auto shop, SD Union Tribune, July 14, 2003. "Hampered by spiraling costs, inflexible curricula and a culture that places more emphasis on college-prep courses, districts have cut back on teaching trades in classes like auto shop. The result, educators say, is that the student who needs a shop class more than a philosophy course may be suffering. And the automotive industry - and every motorist - is feeling the consequences"...
(Sources: Google and Education News.)
Charter schools and segregation
The Civil Rights Project at Harvard University has produced a new study, Charter Schools And Race: A Lost Opportunity for Integrated Education (PDF), by Erica Frankenberg and Chungmei Lee. Please note as well the Executive Summary and the press release. According to the executive summary:
This report details a disappointing set of findings regarding its central question: charter schools are largely more segregated than public schools. Segregation is worse for African American than for Latino students, but is very high for both. In some states, white student isolation in charter schools is as high as that of African Americans. The problems reported here may not be due either to the intent or the desires and values of charter school leaders. They may reflect flaws in state policies, in enforcement, or in methods of approving schools for charters.
It is a strange study, meant primarily, it seems to me, to be quoted in the press. I cannot imagine that this work has anything to offer to a serious student of schools policy.
The situation is that charter schools are found disproportionately in inner cities. It may be of interest to ask why this is so, but the report doesn't address that issue. Taking it for a fact, then it is only natural that charter schools would disproportionately have a high minority population. The study authors might have avoided the loaded and pejorative language of segregation to describe the situation that charter schools are, in their demographics, not much different than public schools in the same geographic environment.
The authors could, of course, have undertaken a sincere study to investigate if charter schools have a special appeal to Black and Hispanic students, beyond what one would expect on the basis of the location of the schools. I don't know if it is a particularly interesting question in this generality - given the diversity of charter schools, as of public schools, it may be more interesting to focus on specific schools or districts rather than on charter schools generally - but in any case, the authors didn't think to ask.
The report was featured in the Boston Globe on Sunday. (That article quotes Frankenberg as saying that charter schools remain more integrated than public schools, but the press contact for the Civil Rights Project describes that as a mis-quote. Should have been: more segregated.) Joanne Jacobs covers the study with reference to a book that she is writing about one overwhelmingly Hispanic charter school. The Center for Education Reform has also paid attention and has put out a press release criticizing the study. Thanks to Education News for the first pointers to this study and the press coverage.
July 10, 2003
NAEP 2002 writing results
The National Assessment of Educational Progress (NAEP) Writing 2002 report is being released today. The test involves a sample of 4th, 8th, and 12th grade students across the nation. For 4th and 8th grade there are also samples for individual (participating) states and for subgroups, e.g. by gender or ethnicity. NAEP has assessments in reading, mathematics, science, writing, U.S. history, civics, geography, and the arts. NAEP results need to be interpreted with caution. The tests are, I think, set at a rather low level, and there are problems with varying exclusion rates for students with disabilities. However, the tests serve as a benchmark for calibrating States' results on their own 4th and 8th grade assessments. The results of the 2002 writing assessment will no doubt be all over the news, so with that in mind, here are some pointers to the official sources.
Overview: What is NAEP?. Links to descriptions of the individuals tests of NAEP; the governance and organizational structure; clarification of the national and the state NAEPs, and of the main NAEP vs. the long-term trend NAEP.
The Nation's Report Card. NAEP main site at the National Center for Education Statistics.
Writing 2002 Major Results. The main page for the results of the 2002 writing assessment.
Statement by the Associate Commissioner of NCES on the release of the 2002 writing assessment.
Executive Summary of the 2002 writing assessment.
July 08, 2003
Occasionally I may feed some education related terms to the Google News search engine and post here whatever I find of most interest.
Ed board to discuss science testing standards, Topeka Capital Journal, July 8, 2003. "Evolution could become a hot topic again for the State Board of Education. Board members planned to discuss Wednesday whether they want to review science testing standards in place for the past two years, which make evolution an important topic for students to learn. The alternative is a limited, internal review of how students are performing on science tests"...
Drop the control, monsieur, Telegraph (UK), July 2, 2003. "Charles Clarke and his French counterpart have much to learn - about how not to run schools, says Anthony O'Hear." Centers on a book by French minister of education Luc Ferry that was distributed to all schools.
Exit exam likely to be postponed for 2 years, The Mercury News, July 8, 2003. "California's high school seniors have been told since they were in eighth grade that they would be the first class to have to pass an exit exam to get a diploma. Now, the State Board of Education appears poised to deliver a revised message: You're off the hook. The board is expected to vote Wednesday to delay enforcing the high school exit exam requirement for at least two years"...
Sometimes high school just isn't enough, Christian Science Monitor, July 8, 2003. About members of the first graduating class of Bard High School Early College in New York City. The school compresses high school in two years and offers two years of college level material. [I don't quite see how this differs from offering plenty of AP/IB classes.]
NEA vows to undo President Bush's education programs , USA Today, July 6, 2003. "Wrapping up its annual meeting Sunday in New Orleans, the 9,000 delegates to the National Education Association crowded special kiosks to telephone and e-mail members of Congress, asking them to amend or reject provisions of Bush's No Child Left Behind education reform law. [...] The delegates also approved a resolution aimed directly at its core, saying generic, 'norm-referenced' standardized tests should only supplement classroom tests"...
PU president Tilghman on teaching science
Today's New York Times science section has a conversation with Shirley Tilghman, a distinguished molecular geneticist and since 2001 president of Princeton University. The conversation dwells for some time on issues of Women in Science and then touches briefly on science education.
Q. How would you change the way science is taught at universities?
A. I think we do not teach the introductory courses appropriately. Right now, we just teach all the basic facts of chemistry, physics, biology or mathematics. Then, we teach a few basic principles. By the third year, we finally tell the students what is interesting about all of this. I think we should break the pyramid. We should begin with the most exciting ideas in chemistry, physics, biology and how you go about studying it. What are the things you need to know? We should only teach what students need to know in order to understand what those are.
Q. Would you teach science by changing science education into a "great ideas of science" course?
A. Absolutely. I'd like to see us teaching more than a canon, a collection of facts, but why this is exciting, why is the exploration of nature one of the most wonderful ways to spend one's life.
All this without a hint of regret that even Princeton University students should have to be babied into an appreciation of science. I wonder if it is really so.
July 06, 2003
Update on NYS Regents Math A
New York State Education Commissioner Richard Mills is under much pressure because of the high failure rate on the recent (June 17) New York Regents Math A exam. The latest and earlier instances of the exam and the associated scoring keys and conversion tables are posted on the Regents Examinations Web site, under the link to Mathematics A. Procedural information related to the exam is posted at the State Assessment site under High School General Information. I know of two reviews of the exam on the Web. There is my own Critique of the New York State Regents Mathematics A Exam, and there is an Analysis of the June, 2003, Administration of Physics and Math A Regents done by the New York State Council of School Supervisors (NYSCOSS).
In connection with the Math A flack the director of the testing division at the NYS Education Department was reassigned and chose to resign, but this is not presented as a cure for any problem. The State Assembly and Senate will hold hearings, according to a report in the Rochester Democrat and Chronicle:
[State Assemblyman Steven Sanders, D-Manhattan], who chairs the Assembly education committee, and Sen. Stephen M. Saland, R-Poughkeepsie, who chairs the Senate education committee, plan to hold public hearings for people to express their feelings about high-stakes testing in light of an estimated 37 percent passing rate on the June 17 Math A Regents exam. Those hearing dates have not yet been set, but Sanders said Rochester, Albany and New York City might be host cities.
Read the full Blog entry, "Update on NYS Regents Math A".
July 04, 2003
Can Education Schools Be Saved?
Can Education Schools Be Saved? was the title of a one-day event at the American Enterprise Institute on June 9, 2003. Among the contributors were George K. Cunningham of the University of Louisville, KT, Lisa Graham Keegan of the Education Leaders Council, and J. E. Stone of East Tennessee State University (and founder of the Education Consumers ClearingHouse).
From the Remarks by George Cunningham:
There are two major competing philosophies in education. One asserts that teachers should focus on increasing their students academic achievement. The other dismisses the importance of academic achievement and instead defines good teaching as the creation of a classroom atmosphere that eschews explicit instruction in favor of giving responsibility for learning to the students. The two approaches are incompatible and there is really no way to create a compromise between the two. The question left unanswered is who gets to decide between the two. Legislatures, governors, and the federal government through NCLB have declared that academic achievement should be paramount. The faculties of education schools and the national organizations that support them have decided otherwise. We will have to await the outcome of this contest, but it looks like the education schools already are ahead on points.
Closing recommendations from the Remarks by John E. Stone:
1. As you make decisions about teacher training and certification, bear in mind that colleges of education have a vision of teaching and learning that is at odds with the public's educational priorities. They have revised and reformed themselves many times over the decades, but the outcome has always been the same--another permutation of the same basic doctrines.
2. The ability of a teacher to produce achievement is not something that the colleges should be trusted to judge for themselves. If policymakers want colleges of education to respect the public's priorities, they will have to independently audit the student learning gains produced by newly minted teachers. Contrary to what is often assumed, it is possible to fairly and objectively judge this outcome. Tennessee has been doing so for the past 10 years.
3. The colleges of education need competition. Their virtual monopoly on training and certification has not well served the public. I think the Department of Education's emphasis on the subject-matter preparation of teachers is a step in the right direction. The key issue, however, is to allow individuals to become teachers without having to undergo training in the untested and often fanciful practices that are too often taught in schools of education.
A contributor on the other side was David Imig of the American Association of Colleges for Teacher Education. He had the misfortune that day of being very much in the news in a negative way in connection with the leak of the ABCTE teacher test, and his remarks were delivered for him.
This seems a good occasion to remind readers of another recent article in the spririt of the contributions by Cunningham and Stone. The article is Ed Schools in Crisis, by Martin Kozloff (October 2002).
New OECD PISA Report
The OECD Programme for International Student Assessment has updated its PISA 2000 report, Knowledge and Skills for Life. The new report, Literacy Skills for the World of Tomorrow - Further Results from PISA 2000, expands PISA 2000 with results from 15 middle income countries. The following is from a summary by the UN News Centre (July 1, 2003).
The report analyses data collected in 2002 from 15 mainly middle-income countries and economies - Albania, Argentina, Brazil, Bulgaria, Chile, Hong Kong-China, Indonesia, Israel, Latvia, Liechtenstein, FYR Macedonia, Peru, Romania, the Russian Federation and Thailand - with data collected in 2000 from nearly all 30 OECD members and first published in 2001.
OECD countries are Australia, Austria, Belgium, Canada, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Japan, Republic of Korea, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic, Spain, Sweden, Switzerland, Turkey, the United Kingdom, the United States.
Among the non-OECD economies, students in Hong Kong-China emerge as star performers, achieving overall scores in reading proficiency equivalent to those of students in the top OECD countries, after Finland, Canada, New Zealand, Australia, and Ireland. Along with students in Japan and the Republic of Korea, they are also ahead of the rest, on average, in mathematical and scientific literacy.
On the other hand, students in Latin America are well behind. Peru has the largest proportion of students (80 percent) at Level 1 and below, indicating that students are having serious difficulties in using reading as a tool to advance and extend their knowledge and skills in other areas.
One can expect some agonized reporting from Israel in the coming days with regard to their educational system. As Israel Insider summarizes it, Report card on Israeli education: F:
In reading comprehension, Israel ranked 30th. The category was led by Finland, Canada and New Zealand, with the United States in 16th place. In mathematics, Israel ranked 31st, with Hong Kong, Japan and South Korea taking top marks. Canada was listed 7th, and the U.S. was listed 20th in math. In the sciences, Israel ranked even lower, in 33rd place. The top marks went to South Korea, Japan and Hong Kong, with Canada in 6th place and the U.S. in 15th place.
Only students from Latin American countries, Bulgaria and Albania received consistently lower marks than Israeli students in the international survey.
PISA assessed mathematics, science, and language skills, but the present report is focussed on literacy. When PISA was first in the news I wrote some comments on the PISA philosophy and also a review, Mathematics in the OECD PISA Assessment. My view is that PISA is about as fuzzy a test as can be imagined, and certainly for mathematics quite unsuitable to evaluate student learning. Just the same, the results will have some correlation with student performance and are not entirely ignorable.
June 21, 2003
New York Regents Math A
The Regents Math A exam that was given on June 17, 2003, has received much negative press attention. For example: New York Daily News, Test Mess Threatens Diplomas. New York Newsday, Math Test Too Tough?. New York Post, Testy Teachers Blast 'Too Hard' Math Exam. New York Times, This Year's Math Regents Exam Is Too Difficult, Educators Say. Rochester Democrat and Chronicle, Huge Numbers Fail Math Test. Buffalo News, Many Seniors Fail Crucial Test. (The links will disappear, but the titles are clear enough.)
I don't know that there is any serious analysis of the exam to be found yet, so the following brief comments may be of interest.
The June, 2003, Math A exam is not yet posted on the NYSED Web site. The previous instance of the exam was in January, 2003, and that one is posted; follow the link to the Math A exams here. I have a FAX copy of the recent Math A exam, but not of the scoring rubrics.
The format of the test is identical between January and June. There are 20 multiple choice questions worth 2 points each, and 15 open response questions, 5 at 2 points each, 5 at 3 points each, and 5 at 4 points each, for a maximum score of 85.
Question 14 on the June exam is plainly faulty, and the scoring rubric has already been changed to allow two answers. The question asks: "If the expression 3-4^2+6/2 is evaluated, what would be done last?" Could be either addition or subtraction, but in the initial rubrics only addition was considered correct.
The wording of several other questions makes it plain that the exam was not proofread by people with adequate mathematical training. This was also the case with the January exam. Examples: Both the January exam and the June exam ask for the "inverse" of a statement of the form "if A then B". I am inclined to assume that this concept of the "inverse" of an implication exists in the curriculum guide, but I am sure that many professional mathematicians would have to guess what is meant. The given answers (multiple choice) make it clear that it is not the negation.
Another example of a poorly worded problem: January Question 7 and June Question 20. They are very similar and have the same flaw. June Q20 asks: "How many different five-member teams can be made from a group of eight students, if each student has an equal chance of being chosen?" The "equal chance" bit does not belong in the question.
There are some minor irritants. The tests (January as well as June) use "equivalent" where mathematicians would use "equal". June question 4 asks "Which of the following does *not* have rotational symmetry: trapezoid, regular pentagon, square, circle?" A mathematician would not be happy with this formulation, although it is clear which answer is intended. (The circle has complete rotational symmetry, the square and regular pentagon have symmetry with respect to rotations over a multiple of 90 or 72 degrees, and only the trapezoid has, in general, no rotational symmetry at all.) The data analysis questions, on the latest exam as well as on earlier instances, are a further source of mild irritation. Mathematicians tend not to care about the "mode" of a data set, but it is on the curriculum and students can learn what it is. Likewise for reading a stem-and-leaf display and reading a box-and-whisker plot.
Part 4 of the June test, especially, has several multi-stage questions that are probably as much a test of intelligence as of learning. (I don't mean this as criticism.) I have not made a careful item by item comparison between the January and the June tests, but it looks plausible to me that the June test is indeed more difficult. Without the scoring rubrics I wouldn't try to say more.
I am convinced that the Regents Math A should be a predictable exam of which the level of difficulty is carefully matched between instances. It is possible that the June, 2003, instance failed on that measure, and the State Superintendent should study that very quickly and decide if an adjustment of the passing score is in order. The press reports, however, give a wrong impression. The exam is, on its own, not unreasonable and not wholly out of line with earlier instances.
[Addendum, October 24, 2003. TheJune, 2003, and earlier instances of the exam are posted on the Regents Examinations Web site, under the link to Mathematics A. Procedural information related to the exam is posted at the State Assessment site under High School General Information. I posted a Critique of the New York State Regents Mathematics A Exam on my Web pages, accompanied by a Detailed Critique of specific items on the June 2003, January 2003, and August 2002 exam instances. The New York State Council of School Supervisors produced an Analysis of the June, 2003, Administration of Physics and Math A Regents. I summarized the issues in a Blog entry Update on the Regents Math A. Commisioner Mills and the Regents appointed an independent panel to review the Mathematics A Regents exam. This panel provided a Report to the New York State Board of Regents and the New York State Commissioner of Education in October. At the same time the Association of Mathematics Teachers of New York State produced a Math A position paper for the New York Regents. Even before public release of the independent panel report commissioner Mills recommended and the Regents enacted changes in the future administration of the Math A exam. These changes are described in an October 2, 2003, press release (released October 9 or so): Four Policy Decisions on Assessment.]