"To qualify for a national merit scholarship, students in different states have to take the same exam, but they don’t have to get the same scores to win."--Valerie Strauss

We know that it is impossible to compare schools and students in different states based on the results on state tests. For example, the highest performing district in one state had 67% of its student score at "proficient" or above, while one of the schools in another state identified for SIG funds as a "low-performing" school had 82% of its students score at "proficient" or above in reading. In another example, one of the lowest performing high schools in one state, would be among the top performing high schools in ten states.

Now we learn that we also cannot compare National Merit Scholars in different states.

Here are the highlights of Valerie Strauss' piece in the Washington Post, which I have filtered for review by busy school leaders:

3.5 million high school students take the Preliminary Scholastic Aptitude Test (PSAT) as a preparation for the Scholastic Aptitude Test (SAT).

In addition to serving as a practice test and indicator of college readiness, the PSAT is also used as the qualifier for the prestigious National Merit Scholarship Qualifying Test.

"About 50,000 students qualify based on their PSAT scores, and that number is whittled down to about 16,000, who become semifinalists (the 34,000 others get letters of commendation). About 8,500 are named finalists, eligible for a scholarship of varying amounts, based on test scores as well as other criteria including academic performance."

However, "the initial cutoff scores separating the possible winners from the definite losers are not the same in each state."

"Students can wind up winning with lower scores than students who didn’t make the cut in their state." For this reason, the University of California system decided to stop funding National Merit Scholarships in favor of other "merit-based" scholarships.

States With Highest Cut Scores: D.C. New Jersey, and Massachusetts

States With Lowest Cut Scores: North Dakota, West Virginia, Wyoming

National Merit Scholars can live in one state, but go to school in another. The state where the school is located gets the credit. For example, "National Merit winners in the past who have been listed as coming from Washington D.C. go to private schools in the city, but live in Maryland or Virginia."

Cutoff Scores by State

"An unofficial list of cut scores for the class of 2012 assembled by the Web site College Planning Simplified, which provides free college planning advice, shows these."

Alabama 211


Alaska 212


Arizona 213


Arkansas 205


California 221


Colorado 215


Connecticut 220


Delaware 217


District of Columbia 223


Florida 214


Georgia 218


Hawaii 216


Idaho 211


Illinois 216


Indiana 214


Iowa 210


Kansas 214


Kentucky 212


Louisiana 209


Maine 212


Maryland 221


Massachusetts 223


Michigan 210


Minnesota 215


Mississippi 205


Missouri 213


Montana 209


Nebraska 209


Nevada 209


New Hampshire 216


New Jersey 223


New Mexico 210


New York 219


North Carolina 217


North Dakota 204


Ohio 214


Oklahoma 209


Oregon 216


Pennsylvania 215


Rhode Island 213


South Carolina 211


South Dakota 206


Tennessee 214


Texas 219


Utah 208


Vermont 217


Virginia 220


Washington 220


West Virginia 204


Wisconsin 209


Wyoming 204

Posted by Mel Riddile at 08:45 AM | Permalink

A late school start aimed at tourism revenue undermines the efforts of college-bound students and parents.

After a friend of mine personally witnessed the horrific traffic in the Washington, D.C. area, he said, "the people here must have a really high tolerance for this craziness."

Recent articles in area papers reminded me of three major barriers to college readiness and admission. Late school openings, university admission policies for out-of-state and foreign students, and the misuse of honors courses represent barriers that sabotage years of work and effort on the parts of college-bound students and their parents. My friend was correct. These people must have a high tolerance.

Barrier #1: Late School Starts

Like many other areas of the country whose population consists of a high percentage of college graduates, the D.C. area, with several of the wealthiest large counties in the nation, demands that its schools and school systems prepare students for admission to the most competitive colleges and universities, particularly those in-state schools with lower tuition rates. In fact, test prep programs begin in elementary school preparing students for admission to the regions elite public magnet high school for the gifted, Thomas Jefferson. It is widely believed that admission to Jefferson guarantees admission to the State's most competitive and highly regarded university, The University of Virginia (UVA).

The reality is that attendance at Jefferson may actually reduce the chances of being admitted to Virginia, because the school's admission policies and desire for a diverse student body will not permit admission of a large number of students from the same high school. Actually, students who take a rigorous course of studies, which includes Advanced Placement (AP) or International Baccalaureate (IB) courses, at any of the area's surrounding high schools and who achieve at high levels probably have a statistically better chance of admission to UVA than do the "elite" Jefferson students.

Parents operate under the assumption that taking a heavy dose of Advanced Placement or IB courses and obtaining a passing score on the exams translates into earning college credits, which not only increases the chances of college admission but also ensures a significant cost savings, because the students will graduate sooner.

The problem is that Virginia's late school opening date makes it much more difficult to earn better scores on AP and IB exams. Virginia students are placed at a distinct disadvantage by a State law, called the "Kings Dominion law", which was passed in 1986 to protect the tourism industry. The law prohibits schools from starting before Labor Day. Because both AP and IB exams are administered beginning in the first full week of May, Virginia students have two weeks less instruction and preparation than many of their counterparts around the country, most of whom begin school in the third week of August. Virginia students start behind and the teachers spend the entire school year trying to catch them up.

Officials in Loudon County (VA), the nations' wealthiest large county, get it.  According to a Washington Post article, "Loudon (County) officials argued that starting in August would give students more time to learn what they need to know for Advanced Placement and state Standards of Learning exams, which are generally administered in May or June. “The more we can do to prepare our kids in advance of those tests, the better off they’ll be,” said School Board Chairman John Stevens (Potomac)."

On one hand Virginia has made a sizeable investment in a system of high stakes testing and accountability that is one of the most rigorous in the nation. On the other hand, Virginia law places its own students at a disadvantage on national and international assessments. According to the report, Virginia's position was succinctly expressed by "Senate Majority Leader Richard L. Saslaw (D-Fairfax). He said sending students back to school early would cut into Labor Day weekend’s substantial sales tax revenue, and he hasn’t heard a compelling argument for giving up those dollars."

Other Peoples' Kids

Virginia's late school start date undermines years of parental encouragement and involvement and puts students at a two-week disadvantage when it comes to AP and IB exams. Would any of us set up a system that places our own child two weeks behind before the school year even begins? I think not.

Next: Barrier #2: Admission policies at state universities

Posted by Mel Riddile at 02:26 PM | Permalink

by Stuart Singer, The Teacher Leader

When No Child Left Behind (NCLB) was first presented in the early years of the Bush Administration it represented a significant shift in educational policy.  End-of-course exams like Virginia’s Standards of Learning (SOL) were soon to become a measure of a school’s success.  The initial requirement of a 70% pass rate in English and Math appeared challenging but reasonable.  Of course, most educational professionals acknowledged that such a level could only be a starting point.  When the SOL exams became a barrier to graduation, no one would have been satisfied with such a low target.  As the Fairfax County Public Schools (VA) Coordinator of Math told me at a department chair meeting at the time, “We have to be striving for something closer to 100%.” 

In the first few years before NCLB went into effect, most systems struggled to reach the 70% level but each year brought higher scores and by the time this requirement had become mandatory the majority of schools were deemed to be making Adequate Yearly Progress (AYP).  In slow, realistic increments the benchmarks began to rise.  These initial goals continued to be well within reach.  

The collision of idealism and reality

There is, however, a time bomb planted deep inside the legislation.  The necessary pass rate for AYP reached 89% last year, a point that became difficult to attain for an increasing number of schools especially in the sub-groups of Special Education and some minorities.  But more importantly, there was a lack of common sense at the endpoint of this relentless march upward. 

In 2014 the requirement for AYP will become 100%.  While such a percentage would literally leave no child behind it also empowers certain students to hold an entire building hostage by choosing to fail a test or at least not giving it their best effort.   

Why would a student not give his best on a standardized test?  Because some tests, although the results are included in the school’s data, are not considered barrier tests for the student.   It doesn’t matter what the student makes, but it can be devastating for the school’s statistics and their AYP.

Surprisingly, as 2012 testing comes into focus, there has been plenty of activity but little effort to address this reality.  Instead of facing the obvious—some children are going to be left behind regardless of the effectiveness of a school— band aids are being applied to the program.  In lieu of adjusting impractical expectations, special temporary exemptions are being issued.  Schools are being given additional time to reach an unreachable goal. 

The irony in all of these machinations is that if the majority of schools did attain test results with no failures, it would be likely that the methods of assessment and/or grading would have to be considered suspect.  Virtually every outcome of 100% is a source of concern.  When Saddam Hussein would win an election with 99.1% of the votes everyone knew the results were rigged.  Garrison Keillor’s Lake Woebegon children are all “above average”, a statement which is intended to be satire, not reality.  Expecting a typical school to have a perfect pass rate is equally ridiculous.  The result of such expectations will be to place inordinate pressure on the school staff and test creators to find ways to pass even the most recalcitrant of students.  In such a case the validity of all test scores will be diminished.  In addition, the recent spate of cheating scandals would likely increase in the wake of such outrageous expectations.  Ask a successful teacher what they would think if every student in all of their classes made an “A” on their final exam.  The answer would reflect on the quality of the test and/or whether it had somehow been compromised. 

Listen to the coach

Vince Lombardi wrote, “Always strive for perfection.  Though you will never achieve it, you may pass excellence along the way.”  Apparently, the authors of NCLB did not read the coach’s book.  Everyone wants all students to be successful.  But this is an aspirational goal, not a realistic one.  Any 100% rule can neither be implemented nor enforced.  However, instead of changing the rule to reflect authentic academic success for as many students as possible, there has been a flood of exceptions, exemptions and excuses.  It is time for a more realistic approach.  While it is true that a rising tide will raise all ships, exam grade inflation will not do the same for actual student performance.   A requirement of a 100% pass rate will surely lead to a disastrous scenario.

Posted by Mel Riddile at 08:17 AM | Permalink

By Stuart Singer, The Teacher Leader

The key to creating an illusion is to distract the viewer.  Draw the attention away from the sleight of hand and the audience will believe they have seen magic.  Apparently this technique is now a key element when dealing with the analysis of standardized test results.  The plan appears to be to use some new and often outrageous assertion to distract the public and “abracadabra” many of the problems inherent in the No Child Left Behind (NCLB) and end-of-course standardized testing magically disappear. 

The latest form of deception is an idea being floated by educational leaders in Virginia.  They are considering a proclamation announcing that a “pass advanced” on the state’s Standards of Learning (SOL) exams is an indication of college preparation.  The word “advanced” in most contexts indicates a significant level of accomplishment; in this case, however, it should not be confused with readiness for post high school study. 

It is actually more of a numbers game

At first glance a “pass advanced” might appear to be a significant achievement.  It requires a score of 500 or more on a test scaled to go from 200-600 (400 is required for passing).  But as in other forms of magic, these numbers are an illusion.  Like many other end-of-course exams being used for NCLB, the SOL is a four-option multiple-choice test with no penalties for wrong answers.  Consequently the actual range of scores is not nearly as large.  The laws of probability decree anyone answering 50 such questions would start with 12 or 13 correct responses simply by random guessing.  In 2011 a passing score of 400 on the Algebra 1 exam required 23 correct answers.   As a result of that scale, every student begins with a score of at least 340.  Thus the real possible range is 340-600.  Suddenly a tally of 500 does not seem quite so “advanced”. 

Forty-four correct answers will earn a student a 500.   Even if this were an exam with open-ended questions and penalties for wrong responses, mastery of only 88% of the curriculum is hardly college-level work.   But with a multiple-choice, no penalty format, 44 accurate responses represent much less.  A few quick calculations reveal that if a student can answer 42 questions, probability will produce the missing two from the remaining eight.  Now the mastery level is down to 84%--a “B-“on most grading scales.  Even those numbers are a bit skewed.  If an individual can eliminate one or two potential answers in a question the likelihood of a successful “guess” increases exponentially. 

Adding to these misperceptions is the limited nature of such forms of questions.  They cannot require multiple-step responses or demand a true demonstration of mastery of the most complex or intricate aspects of a subject.  They can only ask questions that have reasonably simple answers.  It quickly becomes clear that based on almost any analysis, a pass “advanced” on these tests is not a predictor of college success. 

To get quality, you need quality

As Mel Riddile discussed in a previous post, tests made on the cheap are susceptible to both cheating and inflated results.   If Virginia and other states want to administer tests that are indicators of future educational success, they will need to move away from the current easy-to-grade and inexpensive formats and invest in exams that will accurately measure a student’s mastery of a class.  Until then, any claim of academic prowess based on the results is nothing more than an illusion and distraction.

Posted by Mel Riddile at 06:41 AM | Permalink

by Stuart Singer, The Teacher Leader

The test scores for the Washington D.C. schools are in and while they are generally disappointing what is more informative is the fact they clearly demonstrate one of the fundamental causes of low academic performance—poverty.

This is not a new or original discovery.  Mel Riddile organized data clearly indicating that the U.S. scores on the most recent PISA tests were far more about socio-economic issues than poor teaching.  But these numbers from the Nation’s Capitol place an exclamation mark on his assertions.

A quick tour of Washington D.C.

I once heard someone say that zip codes are the best predictors of standardized test scores.  Three of the eight wards in the District of Columbia reinforce the validity of that comment.

Ward 3:  has a median household income of $97,690 and less than 4% of families with minor children are below the poverty line.  

Ward 7:  More than one-third (34%) of families with minor children live in poverty, and the median household income is $34,966.

Ward 8:  The poorest of the city’s eight areas where two of every five children (40%) have incomes below poverty level and the median household income is $31,188.

A comparison of scores

The elementary and secondary math pass rates in Ward 3 indicate that students scored more than 40 points higher than those in Ward 7 and 50 more than Ward 8­. Test results in different subject areas parallel those of mathematics.

But there are other areas that demonstrate the imbalance between rich and poor.  Using the city’s own criteria for teaching excellence, the IMPACT evaluation, the vast majority of the higher rated educators gravitate to the wealthier areas.  Only 71 of the top 663 teachers in the system worked at 41 schools located In Ward 7 and Ward 8.  That represents an average of less than two per school.  In the ten Ward 3 schools there were 135 of these educators placing on average more than 13 highly effective teachers per building.

A vicious cycle of failure

These statistics underscore what is already known.  Students at poor schools do not perform as well as those at wealthy ones.  It also emphasizes that in an era of accountability based in large part on these results, a preponderance of the top educators in a district will migrate to the more well-to-do buildings.  School leaders need to recognize this disparity and address it by offering incentives for top level administrators and teachers to work at low-performing schools. These could include both financial rewards and a different and enlightened approach to measuring progress in test results in areas that traditionally do poorly.

Otherwise, the academic story in Washington D.C. will continue to be a microcosm of entire country.

Posted by Mel Riddile at 09:13 AM | Permalink

Master teacher, Sherry Singer, taught Advanced Placement Biology for approximately fourteen years and then switched to International Baccalaureate (IB) Biology for another fourteen years. Sherry taught for much of her career in what National Geographic Magazine called "the most diverse high school in America." When I need an expert teacher's opinion, Sherry is one of my "go to" people. 

I was looking for a veteran teacher's take on the recent cheating scandals. So, I asked Sherry, "You have probably read about the cheating scandals in DC, Atlanta and elsewhere.
Were you concerned about cheating on IB Exams?  Here is why I am asking. It seems to me that cheating is more of a problem on inexpensive, easy-to-score, multiple-choice tests than it is on a well-constructed assessment that require students to write and explain their answers.

In her own words, here is Sherry's take on cheating and quality assessment:

"You are correct.  I was never worried about cheating in my IB classes.  My classroom tests were always at least 50% essay or data analysis.  These types of assessments are almost impossible to cheat on. 

The IB has clearly thought out ways of discouraging cheating on their end-of-course exams.  Dave, our IB Coordinator, would always recruit IB teachers to be invigilators during the exams.  But we could never invigilate exams in our own subjects.  So I could not have helped students with answers even if I wanted to!  Back packs, cell phones, etc. were left outside the testing area.  No one was allowed to talk after entering the testing area.  One of my favorite IB exam stories was one of my students became sick during the exam and actually threw up in the testing area.  One of the invigilators took him to the restroom got him cleaned up and he came back to the exam, finished the exam and received a score of 6 out of a possible 7.  IB makes students tough! (He is a cardiologist today.)"

The Bottom Line

If students can copy and text answers to each other, and, if teachers can erase incorrect responses, we should admit that these inexpensive tests with poorly constructed questions tell us very little about what students know and what they are able to do. Why are we wasting so much money going through the motions? Why not spend the money on quality assessments that are much better indicators of learning? Instead, we end up spending valuable education dollars on test security and on investigating and firing cheaters at the same time we are laying off teachers and increasing class sizes.

We know how to construct high-quality assessments that, in Sherry's words "are almost impossible to cheat on." How can state and district officials look at themselves in the mirror and rationalize and justify firing teachers and principals and closing schools on the basis of the cheapest assessments money can buy? Bad karma?

I could have sworn that goalie Tim Thomas was talking about the evaluation of principals when he met with the press following the Boston Bruins victory in the Stanley Cup Finals. Thomas, whose meteoric rise from minor league obscurity to Most Valuable Player (MVP) in the National Hockey League Finals, had a reaction that reminded me of how I felt when I received an "all exemplary" evaluation a few years ago.

When the press asked Thomas how it felt to be the MVP, he matter-of-factly responded that 'if I get off to a bad start next year, the MVP may buy me some extra time, but I know that if I don't produce, I will be gone.'

I felt no joy from my A+ evaluation. Most high school principals have learned over time that when something good happens, something bad will come along to bring you back down to earth. When our school was honored with a Presidential Visit, to the surprise of my staff, I decided to hold the regularly scheduled Faculty Advisory Committee meeting that same day. When asked why, I responded, "They will make certain that I am brought back to earth."

Why no happiness from a stellar evaluation? First, I felt more of a sense of relief than joy. I had taken a big career risk going to work in a high-poverty school just as our State (Virginia) decided to become a high-stakes accountability state, and I had paid a heavy personal price for it. I didn't have a good night's sleep for over five years.

Nevertheless, nothing could have been more rewarding than working with an outstanding group of teacher leaders, all of whom had years of service in that school, to turn the school from the ranks of the condemned to that of the commended. Second, the opinion of the people in our school was more important to me than the evaluation mainly because the evaluator had no idea what I actually did. Finally, and most importantly, like Tim Thomas, I knew that the evaluation wasn't worth the paper it was printed on. I knew that I served at the pleasure of the Superintendent and the School Board and that I could be removed and replaced on a whim. Something could go wrong the next day, and, if it was expedient to do so, I would be unceremoniously thrown under the proverbial bus without hesitation.

My attitude toward the current discussions about better ways to evaluate principals is simple. Bring it on, because it doesn't matter. As one of my first principals told me, "you can delegate responsibility, but you can never delegate accountability." The principal is solely accountable for everything that goes wrong in a school, and believe me, there is a lot that can go wrong in a school.

Being a principal is similar to being a coach in a professional sport. When things go wrong, it is the coach's fault. When things go well, it was because the team had good players. In school, when things go wrong, it is the principal's fault. When things go right, it's because the school has great teachers.

The Bottom Line

The principal must answer for everything that happens in a school. The evaluation methodology will not change the culture, which is set up to squeeze those in the middle--the school principal. True, over time, principals can earn some extra credit that may buy a little extra time, but a principal can and will be removed at a moment's notice irrespective of their most recent evaluation.

Instead of focusing on evaluating principals, the emphasis should be first on building the capacity of current principals to do their jobs and secondly on reducing principal turnover.  However, building capacity and reducing turnover will require more effort than I believe the so-called experts are willing to exert or fund.

Posted by Mel Riddile at 08:03 AM | Permalink

by Stuart Singer, The Teacher Leader

The Virginia Standards of Learning (SOL) exams, which are representative of many of the end-of-course assessments utilized as barriers to graduation throughout the country, have plenty of room for improvement.  In previous posts I have made a number of specific suggestions for upgrading the SOLs.  That said,  I must admit that a new “improvement” advanced recently  in the “Washington Post” by some of Virginia’s educational leaders rendered me virtually speechless.

“Several prominent Virginia superintendents are pushing the state to give (SOL) standardized tests months earlier in the school year, a shift they say would reduce the impact of testing on classes and free teachers to offer more meaningful lessons.

“A change in the testing calendar would allow more time to teach ‘key 21st century skills that are linked to college and career readiness,’ the superintendents wrote to state Superintendent of Public Instruction, Patricia I. Wright, in April.”

The new plan would allow those students who fail the exams in the first semester to retest one or two times later in the year.

Not nearly as simple as it looks

There is no question that for many students end-of-course exams are outrageously easy.  Over an eight year period in the honors Algebra 2 classes at my former school, no student ever failed the corresponding SOL.  And certainly a portion of our students, based on our internal assessments, could indeed pass the exams by the end of the first semester.  However, at least in terms of the math curriculum, the people making these proposals are not accepting certain realities.

A close inspection of the May, 2011 test scores for my former school is enlightening.  During that session 28% of the Geometry students in the building did not pass the SOL.  One fifth of the Algebra 1 and Algebra 2 also failed.  What would those numbers have been if the test had been given four months earlier? 

An episode that occurred several years ago may help answer that question.  The district leadership was in turmoil because the scores on the Algebra 1 SOL the previous year had dropped dramatically at nearly every school.  At a department chair meeting the concern of the math coordinator was palpable. We were tasked with going back to our schools and trying to find the root cause of the across the board decline.  I asked my lead Algebra 1 teacher why it had happened and she instantly replied with a single word—“snow”.  We had lost two weeks to inclement weather that year and that translated into two fewer weeks of review.  Those ten school days were sufficient to have a negative impact on the students' performance.  This new plan would be the equivalent of three months of snow.

Another factor that cannot be measured precisely by statistics is the damage done to the fragile confidence of the most at-risk students when they fail a test that they are woefully under-prepared to take.   How many students who fail the exam in January, but could have passed in May, will be too discouraged to continue? Conversely, what is the motivation for students who have passed the exam in January, to continue the course with the same intensity as they exhibited prior to the exam?

Pros, cons, and something in-between

The Virginia superintendents are very clear about why this change would help successful students.  One of them explained, “Our students are bored because they’re not doing the hands-on kind of learning that they’re great at.”  Unfortunately, they are a bit vague when it comes to their plans for the students who do not pass the test.  According to the “Post”:

“Testing Virginia students in the first half of the school year would also allow time for teachers to tailor remediation efforts to those who do not pass the exams the first time.”

No details were forthcoming about what this “remediation” effort would look like.  It was not designated as after school, during school, a weekend activity or within the daily class schedule.  What was clear was that regardless of the format, the teachers will be responsible for the implementation.  If the ultimate goal is to create a better academic environment for students, placing ambiguous and unfunded responsibilities on teachers is misguided.  Any remediation plan that puts all of the pressure on teachers without providing any clear program for the additional time necessary for execution is destined to fail. 

Making this into a good idea

The apparent goal of this proposal is to diminish the importance of the end-of-course exams by administering them in January.   While this may indeed have validity for many, if not most students, there are other students who, given less time to prepare for the exams, will become unacceptable collateral damage.   

Ironically early testing could be an excellent approach if it were properly utilized.  One possibility is to test the students in January and establish who has passed the tests and who has not.  Then move those students into two different groups.  Those who have succeeded can move forward and enjoy the “21st Century Education” so desired by the various school leaders.  The ones who do not pass would be placed into special courses that would focus on the subject matter needed for success.  While some may negatively categorize this idea as tracking and draconian, the truth is that these tests are barrier exams for graduation.  It is not hyperbole to state that failing these exams could possibly have negative life-long consequences for some students.   Developing classes specifically to ensure that students pass exams that are required for graduation is the best course of action for all involved.  Clearly, moving all tests to the first semester without adequate planning for all students is not.

Posted by Mel Riddile at 11:09 AM | Permalink

Is the title of this Washington Post article a rhetorical question? Of course students will blackmail teachers and schools. I know they will because they already are holding teachers and schools hostage all over the country and the so-called experts don't have a clue.

Why? Because, while many states are perfectly willing to close schools, fire teachers, and fire principals, they are not willing to hold students accountable for the results of their state assessments. They are not willing to take the political risk of making state assessments barriers to graduation. They lack the moral courage to stick with their stated values about the need for a quality education for all, because they don't truly believe in their own students. They fear that many students are not capable of achieving at higher levels. Legislators are running from the inevitable political fallout that will result from what they fear will be dramatically declining graduation rates.

Fear not all ye faint of heart! One state, Virginia, stuck to its promise and required all students to pass six of eleven end-of-course exams, which were used to calculate adequate yearly progress. Despite the most rigorous standards and assessment system in the country, graduation rates and proficiency rates continue to improve.

It is common knowledge to anyone who has worked in a high school in the last decade that students routinely "flag" or "Christmas-Tree" tests. Teachers and principals in many states have based their careers and the fate of their schools on the hoped for, good will of their students. Unfortunately, even if 90% of the students put forth 100% effort, which is unlikely, the remaining 10% will undermine even the best efforts of teachers and school leaders to raise test scores.

One principal in another state lamented to me that, despite having one of the best college admission rates in his entire state, his school is now on the "needs improvement" list because his seniors thought it would be funny to draw Mickey Mouse on the answer sheets on last year's assessments. The joke was on the school.

From another perspective, many of these students have been repeatedly tested since the third grade and they have reached the breaking point. They are sick and tired of spending long hours taking tests from which they never receive feedback. By high school, they have figured out that these tests have no direct effect on them. Colleges will never see the results and their grades will never suffer. Consequently, if they don't "feel" like taking the test seriously on a given day, they don't. So what? Well, the principal gets fired and the teachers are transferred that's "so what." Meanwhile, those seniors are away at college having a good laugh at the expense of their alma mater back home.

The Bottom Line

Unless everyone in a system including state officials from the governor on down, district offices, schools, principals, teachers, and students are held accountable and hold themselves personally accountable, there is no system of accountability. What we have in many states is not a system of accountability but rather an excuse to scapegoat convenient targets of opportunity--principals, teachers, and "failing schools."

Posted by Mel Riddile at 10:04 AM | Permalink

by Stuart Singer, The Teacher Leader

It would seem logical that a local community college would be an excellent source to evaluate the quality of the education high school students were receiving.  Kristen Amundson, a former member of the Virginia House of Delegates and current communications manager for Education Sector, relayed some chilling data concerning the shortcomings of Virginia schools in a recent article in the Washington Post.  According to Ms. Amundson:

“At NVCC (Northern Virginia Community College) 2,913 of the 4,719 freshmen (62 percent) require remediation in one or more subjects. In other Virginia community colleges, as many as 80 percent of students arrive on campus needing at least one remedial class.”

Over the years Northern Virginia Community College has received national respect for the success of its students.  As a part of the Virginia Consortium students who earn grades of “A” or “B” in courses at the school can transfer those credits to any four-year state university.  Many talented individuals use two years at NVCC as a low-cost alternative for the first half of their college degree which they can complete at prestigious schools such as the University of Virginia and Virginia Tech. 

A disturbing imbalance

The fall schedule for NVCC reveals a major problem.  At the Alexandria campus there are 32 sections of Math I.  This is a non-college credit class that is designed as a remedial course that covers topics that should have been learned in high school.  Meanwhile Math 151, the freshman entry-level math course requiring a prerequisite of Algebra 2 and Geometry has a total of 11 sections.

The big lie

The state of Virginia thinks so highly of its Standards of Learning (SOL) end of course exams that the governor opted out of the Department of Education “Raise to the Top” competition because he believed the state’s standards far exceeded those of the federal government.  When one looks at the graduation requirements for the state, that contention seems valid.  To earn a diploma in the state students must pass three math courses at or above the level of Algebra 1.  In addition, they must pass at least one math SOL and in most cases two in order to “verify” those credits. 

On paper these requirements would appear to preclude any of the problems being encountered by NVCC.  The prerequisites for Math 151 are basically identical to those of a Virginia diploma.  And yet to meet the needs of the incoming freshmen, almost all of whom are products of Virginia high schools, there are three times as many remedial math courses as college-level.  The answer may be in how the state defines “mastery”. 

In May, 2011 in order to “pass” the Algebra 1 SOL exam a student must correctly answer 23 of 50 multiple-choice questions.  While on the surface that requirement might seem low (it equals to a 46% score) the reality is much worse.  On average random guessing on four-option multiple-choice questions will result in correct answers one-fourth of the time. Thus, a student can earn a passing grade by knowing the correct answer for 15 of the 50 questions (30%) and then by guessing on the remaining 35 questions net 8 or 9 more which will then total at least the required 23.  Though this level of knowledge could hardly be considered “mastery”, it does receive the label “passing”.  Even more disturbing is the fact that a few years ago that required pass number had been 26.   Is it really that surprising that a student with these credentials is struggling as a college freshman?

It is time to demand more

Sadly, the Governor is correct.  The state of Virginia does have comparatively high standards.  The problem is they are clearly not nearly high enough.  Instead students are receiving a potentially dangerous mixed message. They are pushed to go faster by taking Algebra 1 in middle school and abolishing all non-honors courses.  When too many begin to falter the requirements for defining “success” are lowered. The result is a transcript full of wonderful sounding classes and a plethora of graduates with woefully inadequate academic skills. 

It is time to stop looking for semantic shortcuts and begin the difficult process of demanding more.  The mere act of placing wallpaper over the cracks in our academic walls is not enough. 

Posted by Mel Riddile at 08:25 AM | Permalink

by Stuart Singer, The Teacher Leader

Evaluations that have value

Academic success has been directly linked to the quality of a school’s teaching and administrative staffs.  Ideas are constantly being presented for improving educational personnel.  But singular approaches such as merit pay, improved evaluations, and easier termination policies will not individually accomplish that goal.  What is needed is a multi-faceted program that will address all of the shortcomings in the current system. Previously, eight fundamental areas were presented that need to be addressed in order for districts to be able to hire, improve and retain the most talented educators.    

This is the fourth in a series of articles that will detail the steps needed to implement those improvements.  In earlier posts hiring practices and effective interviewing techniques have been discussed. Part 4 will focus on the evaluation process.

Changing more than the format

Nearly every proposal for improving teacher evaluation revolves around the use of data particularly standardized tests.  While that is an important item for consideration, the most pressing reform may well involve redefining who is best qualified to actually perform those evaluations.  The first step in true evaluation reform should be to create a new set of individuals who are responsible for this portion of the process. 

Not the best choice for the job

In most school districts the primary responsibility for this process falls to the local administrative team and there are many assistant principals who are well equipped to review the work of a staff member.  But even though the vast majority undertake this time-consuming task with a high level of commitment, they have an overwhelming array of day-to-day responsibilities. For example, the AP who evaluated the math teachers at my former school was tasked with the discipline of more than three hundred freshmen, administering (and evaluating) three different departments, implementing the testing plan for state barrier exams, hall duty between change of classes, supervising several extra-curricular and athletic events and interviewing candidates for vacancies.  In addition there were the almost daily emergencies that always arise in a high school.  A majority of administrators have similar job descriptions.  It is not surprising to find the time to evaluate teachers falling toward the bottom of this lengthy “to-do” list. 

Creating professional evaluators

A better option for school districts is to train a group of master teachers to become full-time, system-wide professional evaluators.  Such individuals would be significantly better equipped to accurately assess the skill of an instructor than administrators who in all likelihood were not hired primarily for their evaluation skills.  The money required for forming such a cadre could be offset by a reduction in administrative staff and an improved evaluation process.

An even more important argument for this innovation revolves around fairness and consistency.  School-based evaluators cannot help but be somewhat biased by their daily interaction with the staff.  Within a building there is an intuitive sense of which individuals perform well and which perform poorly in the classroom.  These reputations are rarely based on quantitative data; they are the result of comments by students, parents, faculty and other subjective experiences.  Minimally these unsubstantiated ideas can influence the amount of effort spent on an evaluation.  If pressed for time, an argument could be made to shorten the observation of a teacher who is widely “recognized” as being outstanding.  Conversely, negative sentiments from the school community can result in closer scrutiny of less well respected educators.  In either case a measure of fairness is compromised. 

A team of district-based evaluators would eliminate this problem and would also create consistency throughout the system.  The evaluations at school A could be compared with confidence to those at school B.  A number of issues ranging from merit pay, transfers and the termination of contracts could be resolved more reliably.      

What would the process contain?

Here is one hypothetical fix for the ineffective and unproductive teacher evaluations that are sadly typical.  Three professionals would form a teacher’s evaluation team.  One (generalist) would be a highly trained observer who is thoroughly versed in the fundamentals of good teaching.  Another (curriculum specialist) would have similar training but would have taught in the subject field being observed.  The third would be an assistant principal at the school. The generalists and curriculum specialists would be required to have at least ten years of successful teaching experience in addition to extensive training in observing and interpreting classroom activities.  Successful retired teachers could be an outstanding and economical talent pool for these positions. 

The actual evaluation process would be intense.  At least five formal observations would be required.  The generalist and specialist would have two announced and two unannounced.  Additional observations can be done when necessary or desired.  A local assistant principal would perform one unannounced visit.  This experience would familiarize the AP with the teacher being evaluated.  The results of this observation would not be included in the final document but should allow the administrator to better understand and interpret the input from the other team members.  All observations would encompass an entire class period.  The two announced would be videotaped which would become a central component in post-observation conferences.  The videos would also be available to the other members of the team.  When appropriate, standardized test scores and other pertinent data would be included in the overall assessment.  Evaluators will, of course, be carefully trained in analyzing such data and how to utilize it in a fair, accurate manner.  At the end of the process, the three observers would meet and create the overall rating that would then be shared with the teacher.   The primary source of the final conclusions would be the two observers; the role of the AP would be to coordinate the process and provide further input if needed.

The next steps

The purpose of an evaluation should be to both determine the quality of one’s performance and to construct approaches to improve and enhance skills.  Ultimately it should also be a tool in determining pay, advancement and termination.  Those will be the focus of the next part of this series.  

Posted by Mel Riddile at 01:09 PM | Permalink

As measured by gains in student achievement, teachers, even experienced, mid-career classroom instructors, improved as a result of their participation in a formal evaluation process. An Education Week report cites two studies conducted in Cincinnati, which has an extended history using a formal evaluation system consisting of four formal observations. In Cincinnati, teachers are given a rating relating three standards: classroom practices, classroom management, and questioning and discussion techniques.

In addition to discovering that 1. Teacher participation in a formal evaluation process improved student achievement, the first study concluded the following:

2. Classroom Management Improves Math Performance

The study found that "while overall teaching practice was the best predictor of student achievement, classroom management was more highly correlated with better math performance."

3. Questioning Improves Reading

Teacher use of open-ended questions was more highly correlated with student performance than classroom management.

In a second study, also conducted in Cincinnati, student performance not only improved in the year that the mid-career teachers were being evaluated, but the improvement in student performance continued and even increased in the years following the evaluation.

It is important to note that in the Cincinnati evaluation system teachers are not evaluated annually and that the evaluation process does not use a value-added component. In addition, the Cincinnati teacher evaluation system connects to a career ladder for teachers, which may be a motivating factor.

Implications for School Leaders

- Teacher evaluation is not the most enjoyable part of a school leader's job. However, knowing that teacher evaluation improves student achievement makes the process more meaningful.

- If school leaders want to make a difference in student achievement, teacher evaluation is a no-cost way to do so.

- Teacher evaluation is most effective when the teachers are clear on what behaviors will be evaluated, and if both teachers and principals have had extensive, multi-year professional development. Cincinnati was chosen precisely because both factors were present. In addition, many states and districts are proposing similar evaluation models and Cincinnati's decade long experience should help inform future practice.

- Even more encouraging is the fact that student achievement continues to improve in the years following a meaningful evaluation process of even the most experienced teachers.

- It makes sense that questioning improves reading, but I wonder why classroom management makes a bigger difference in math classes. Perhaps the sequential nature of math demands continuous attention and student engagement--students miss out if they miss a step in the process.

Posted by Mel Riddile at 11:52 AM | Permalink

by Stuart Singer, The Teacher Leader

In an essay published in the Outlook section of the Washington Post, John Sener has decided that by successfully making 18 of 20 free throws in a gymnasium he has, using the same criteria used by analysts of standardized testing, successfully proven that he is a better basketball player than Michael Jordan.  Using similar logic he dismisses the poor performance of American students on the PISA test as irrelevant.   

When in doubt, ridicule

 When my former school posted outstanding scores in the state’s standardized tests I was surprised by the number of questions other educational leaders raised about our “ethics”.  Mel Riddile would explain to me that when you have poor outcomes you have two options—work harder and smarter or find a way to bring the competition down.  Unfortunately, the latter approach appears to be the one favored by Mr. Sener.

His argument is that standardized tests in general and the PISA international test in specific are inaccurate indicators of the quality of a country’s educational system.  He begins with sarcasm and then drifts into the surreal.

“Once you truly understand the awesome power of test scores, you will embrace them, as I have done — especially after realizing how standardized testing proves that I am a better basketball player than Michael Jordan.

“Don’t laugh; I have the test results. I read something in a blog somewhere about how MJ recently made 16 out of 20 free throws in a friendly shooting contest. Pretty good, but I thought I could do better. So I went to my local gym and practiced and practiced until I achieved my aim: 18 out of 20 free throws! I’ll send you the video, if you like. (Or you could do what most people do with PISA scores and simply take my word for it.)”

Making the basket; missing the point

Based on his free-throw shooting (real or otherwise) Mr. Sener reaches several conclusions about the PISA test scores in the United States.

“You may argue that it’s not a fair comparison, but that’s what so great about this — simply use the same rules we apply to judging PISA scores, and it’s perfectly fair.  So what if it’s not a head-to-head competition? PISA’s not a head-to-head competition. The students take the tests at different times in different places under different conditions. Heck, they take the reading test in different languages.”   

His second explanation of the poor performance of U.S. students is their lack of interest.

“…what makes you think that American students take PISA seriously? When I tested my teenage son’s knowledge of the PISA exam, he just looked at me quizzically, since he’d never heard of it…Do you really believe that every student who takes the PISA has the same amount of practice?"  

To assess for yourself whether increased practice would affect the outcome of US students’ scores on the PISA tests, go to http://pisa-sq.acer.edu.au/ .

Not all air balls

Mixed into the misguided basketball analogies Mr. Sener does make some excellent points which should be emphasized.

“Standardized tests don’t measure most skills, yet opinion leaders and policymakers constantly tell us how America’s education is going down the toilet based on those scores...There is no place in standardized tests for creativity...You would be wise to ask these questions, even though standardized tests don’t care about curiosity, either.”

Ignored problems do not go away

There is no question that standardized testing does not answer all of the questions of how to measure learning and good teaching.  I have long argued that the Standards of Learning (SOL) exams given in my state (VA) did not indicate mastery of a subject and the method of administering the tests was poor.  But I also knew that though imperfect this new accountability was a step in the right direction.  Prior to such tests there were virtually no quantitative measures of the relative performances of students from classroom to classroom, school to school or district to district.  These results clearly indicated discernible patterns that, if used correctly, could be of great value. 

While this standardization did not equate to the level of precision that would be optimal, it did offer critical insights into the quality of teaching.  In every school the staff forms subjective conclusions as to which teachers are effective and those that are not.  During the ten years I observed SOL testing (VA) the results of these exams closely matched these informal evaluations.  Based on substantial data, the students of certain teachers routinely outperformed others.  While such statistics can and were misused, they did provide a limited amount of quantitative proof of student comprehension, weaknesses and the quality of the work of their instructors.

These outcomes were not enough.  The testing methods need to be improved to better reflect the actual knowledge acquisition.  They must demonstrate a legitimate understanding of a wide range of material.  This process is still in its infancy and far from a finished product.  The potential for improvement is present if the willingness to keep an open mind is maintained.

But simply ignoring any measurement that indicates a serious problem in American education is reckless.  A country where more than three of every ten students drop out of high school and only 30% attain a college degree is hardly in a position to dismiss a poor global performance with sarcasm and ridicule. 

Note: At the high school level, Virginia administers eleven end-of-course (EOC) exams, which are used both as barriers to graduation and to calculate adequate yearly progress (AYP). Only a few states use EOC exams for accountability purposes and as barriers to graduation.

Posted by Mel Riddile at 10:11 AM | Permalink

Background: Nationally, two-thirds of students are not reading on grade level by the fourth grade, the earliest year of testing in the National Assessment of Educational Progress (NAEP). That proportion rises to four-fifths for low-income children, according to NAEP results released last year.

A recently released national study indicates that students who are not proficient readers in third grade are significantly more likely to drop out. "Students who don’t read proficiently by third grade are four times more likely to leave high school without a diploma than proficient readers."

It's Poverty Not Stupid (3-6-8) "Poverty compounds the problem."

Students who have lived in poverty are three times more likely to drop out or fail to graduate on time than their more affluent peers;

• If they read poorly, too, the rate is six times greater than that for all proficient readers.
• For black and Latino students, the combined effect of poverty and poor third grade reading skills makes the rate eight times greater.
• Poverty troubles even the best readers: Proficient third graders who have lived in poverty graduate at about the same rate as subpar readers who have never been poor.

“We will never close the achievement gap, we will never solve our dropout crisis, we will never break the cycle of poverty that afflicts so many children if we don’t make sure that all our students learn to read,” said Ralph Smith, executive vice president of the Annie E. Casey Foundation.

Specifically, the study found:

• One in six children who are not reading proficiently in third grade do not graduate from high school on time, a rate four times greater than that for proficient readers.
• The rates are highest for the low, below-basic readers: 23 percent of these children drop out or fail to finish high school on time, compared to 9 percent of children with basic reading skills and 4 percent of proficient readers.
• The below-basic readers account for a third of the sample but three-fifths of the students who do not graduate.
• Overall, 22 percent of children who have lived in poverty do not graduate from high school, compared to 6 percent of those who have never been poor. This rises to 32 percent for students spending more than half of their lives in poverty.
• For children who were poor for at least a year and were not reading proficiently in third grade, the proportion of those who don’t finish school rose to 26 percent.  The rate was highest for poor black and Hispanic students, at 31 and 33 percent respectively. Even so the majority of students who fail to graduate are white.
• Even among poor children who were proficient readers in third grade, 11 percent still didn’t finish high school. That compares to 9 percent of subpar third graders who were never poor.
• Among children who never lived in poverty, all but 2 percent of the best third-grade readers graduated from high school on time.

The study concluded that improvements are needed in the following areas:

1. improving the schools where these children are learning to read
2. helping the families weighed down by poverty
3. better federal, state and local policy to improve the lot of both schools and families
4. aligning quality early education programs with the curriculum and standards in the primary grades
5. paying better attention to health and developmental needs of young children
6. providing work training and other programs that will help lift families out of poverty.

Essential Question

Can high schools or middle schools afford to wait until students arrive at their doors with reading problems?

• The development of math and literacy skills is a PK-12 issue, not an elementary issue, not a middle school issue, and certainly not a high school issue.
• Vertical articulation between all levels is one key to improving literacy skills.
• Curriculum alignment is another key.
• Cross-content literacy instruction (Common Core ELA Standards) and whole-school literacy initiatives are another key.
• Keep in mind that, even if under-resourced students are proficient by third grade, they must have direct, explicit literacy (reading, writing, thinking, discussing) instruction every year thereafter or they will not progress.
• Literacy skills predict future math performance, which, in turn, predicts future college completion.

Posted by Mel Riddile at 10:11 AM | Permalink

The following was excerpted from a Fairfax County Times article:

"In response to an opinion from Virginia Attorney General, Kenneth Cuccinelli, the Fairfax County School Board (177,000 students) voted Thursday night to refund Advanced Placement and International Baccalaureate exam fees to students.

The School Board had approved the test fees during its budget discussions last year as a way to cut costs. The estimated cost to refund the fees, school officials said, is $2 million."

State Constitution Guarantees a Free Education

"On Jan. 8, the Attorney General wrote that the school system's practice of charging AP and IB students a testing fee violated a provision in the state constitution that mandates access to a free education. Students were charged $75 for testing to help balance the school system's budget.

To comply with the Attorney General's opinion, the school system had the choice to either make the test optional for students taking AP and IB classes or to resume covering the testing costs and refund the previously collected fees, said Superintendent Jack D. Dale."

Principal Points

1. Schools in many areas charge fees and most states have provisions in their constitutions referencing a "free education." Does this mean that any and all fees violate the state constitutions?

 2. AP and IB courses are electives. Students voluntarily enroll in the courses. It makes sense that no fees could be charged for required courses, but what about electives?

3. Placing fees on tests is a particularly regressive way to balance a school budget. These fees discriminate against under-resourced students and the schools who serve them. Fees have little or no impact of resourced students. For middle class families, test fees are an inconvenience. For under-resourced students, the testing fee prevents them from taking the course. The last time these fees were assessed, our IB participation dropped by 20% even though the students were guaranteed that we, the school and the PTSA, would cover all fees for any student in need. The impact of test fees on course participation parallels participation in the federally subsidized lunch programs in high schools, many students are simply embarrassed and do not enroll. By the way, in the case of our school, as soon as the fees were eliminated, our participation levels returned immediately.

The rich get richer and the poor get poorer!

Posted by Mel Riddile at 02:06 PM | Permalink

by Stuart Singer, The Teacher Leader

The teacher’s lament was plaintive.  He had just been informed that the principal wanted to meet with him about an undisclosed topic.  “I know exactly what he wants to talk about.  It’s my grades—too many “F”s.   I am working harder than I ever have.  I have created a dozen different interventions.  But I can’t change my grading policies.  I know what my students need to be successful.  They are failing because they don’t know the required material.  I’m just giving them what they have earned.”  Another teacher had told him that she had recomputed her grades to avoid “the meeting”.  He then closed with the inspiration of this post.  “I’ll tell you what it is.  It’s grade inflation by intimidation.”

Not as simple as it looks

Virtually every day a new proposal is introduced to save public education in the United States.  Too many of these plans are more focused on reducing failing grades than on the creation of more successful students.  The mere act of lowering the number of “D” and “F” grades does not automatically improve academic performance.  To the contrary, an argument could be made that by giving inflated, inaccurate assessments students are being mislead as to their skill level and misplaced in future courses. 

The grades teachers assign are an important component in assessing their work but it is not a precise tool.  Like every other statistical measure it must be utilized by someone who understands all of the factors at play in any particular classroom.  The broad conclusion that an individual “gives too many poor grades” can be a dangerous oversimplification.

A long inflationary cycle

The presence of grade inflation is not a new phenomenon.  More than 25 years ago teacher evaluation meetings at my school always began with an examination of the “D/F rate."  An assistant principal would look at a printout giving the teacher’s percentage of students receiving grades of either “D” or “F”.  Unfortunately those raw numbers did not always tell the complete story.  For some students a grade of “D+” was a triumph over adversity and past history; for others a “C+” was a reflection of little or no effort.  But for the teachers the former would count as a negative statistic while the latter was of little or no concern.  Without understanding the complete context, individual grades and D/F rates mean little.  But the message, however misguided, was clear—better grades indicate better teaching.

This pressure was relentless.  During a discussion of the topic in a math department meeting one teacher stated the obvious.  “Listen, we are all math people here.  We have the capacity to construct our grading in whatever way they want.  Just tell us what percentages to have and we can make it happen.”  Whether she was serious or not, her contention was accurate. 

Over the years due to a variety of influences the inflation of grades has occurred.  In a typical suburban high school in 2011, a GPA of 3.0 which in theory translates into a “B” which is above average will rank a student in the middle of their class.  Though nearly all measures of achievement clearly indicate little or no improvement by today’s students, at least on paper their academic performance is clearly on the rise.

Parents join the fray

When the AP Biology teacher walked into the parent conference, she was confident that her position on the matter to be discussed was secure.  The student had missed the scheduled mid-term exam to go on a family ski trip.   Prior to the vacation he had presented a form for a prearranged absence.   The teacher had checked the block that indicated she did not approve.  In addition she had written on the paper what she had told the student in person—he would have to take the exam on his first day back at school.  This timing was appropriate since it was in keeping with the class policy she had established on the first day of school and until he took the exam none of his classmates could review their results.  

Though the student had accepted these stipulations, when he returned he informed the teacher that he was not prepared.  She insisted he take the test as per their agreement.  His score was low and resulted in his grade dropping from a “B-” to a “C” for the quarter.  

Some parents are more equal than others

The student’s father was a highly-regarded physician in the community.  During the preliminary discussions at the conference he appeared disinterested in the arguments being presented.  After about ten minutes he held up his hand and said, “Let me be clear.  I really don’t care about all of these details.  I just want to know what it is going to take to make this grade into a “B”.  I don’t care if it is a retest or just exclude him from the mid-term or whatever, let’s find the answer.”  The teacher refused to alter the grade and was asked to leave the meeting.  Later that day the principal informed her that the “C” had been changed to the desired “B”. 

An issue of fairness

Based on this change, the teacher was faced with an ethical dilemma.  Giving this student a “B” would lessen the value of the grades earned by the other students in the class.  Consequently, she decided to proportionally raise everyone’s grades.  Once again, while no additional learning had actually taken place, on paper every student in the course had improved their academic performance.  Not surprisingly, as the details of this episode spread throughout the staff, teachers began to have second thoughts when enforcing valid policies that might result in lower grades.

A time and a place

Many school districts have designed creative methods to give the appearance of reduced disciplinary referrals, absenteeism and dropout rates.  Too often grades are also just another statistic to be manipulated in order to give the appearance of improved success.  There are some teachers whose grading indicates either an inability to correctly assess student performance or ineffective instructional skills.  Such situations must be addressed directly and corrected immediately if education is to be successful. 

But the danger is when lockstep grade expectations are placed on an entire faculty or school district.  There should be cause for concern when a conversation with a teacher who has years of successful experience begins with “you are giving too many bad grades”.   Grades are meaningless if they are not a legitimate reflection of student learning.  The key to truly improving grades is to ensure that students are gaining mastery of the subject matter.  When knowledge acquisition is attained, D/F rates will decline.  And more importantly, they will indicate real academic progress.  

Posted by Mel Riddile at 03:37 PM | Permalink

by Stuart Singer, The Teacher Leader

Like Mel Riddile, I saw the recent “60 Minutes” segment featuring the TEP School (The Equity Project) in New York City.   The opening tease about a school offering teacher salaries of $125,000 drew my attention.  The segment itself was riveting.  The discussions of hiring, firing and tenure policies in the report were persuasive.  The passion of the teachers and the school’s founder was palpable.  The stories of 80 to 90 hour work weeks were a testament to dedication.  The scenes with the students interacting with their instructors were exhilarating.  The evaluation process with peer involvement and regular administrative visits was inspiring.  I turned to my wife, a retired high school teacher, and announced “I think these people are on to something.”  That enthusiasm was significantly diminished during the final two minutes of the segment. 

Despite the high pay, long hours, classroom enthusiasm and attention to detail, student academic scores fell in the school’s first year.  Of course, one year is hardly a fair trial period but an actual drop in such results is a valid point of concern as well as a reason for some reexamination.

A benign dictatorship

I recently heard in a discussion on NPR that a global dictator would be required to ensure that the world would productively address climate change.  The rationale is that with an all-powerful individual making the decisions, the necessary policy adjustments could be put into place without the intervention of special interests.  It is a parallel analogy to the account that a dictator was the only person who ever successfully made the Italian trains run on time.   Democracy can be so messy.

The TEP model is also a one-man operation.  Its founder and principal, Zeke Vanderhoek, hires and fires the teachers, observes their classes on a regular basis and makes all policy decisions.   Because of his complete control he has been able to creatively increase salaries, easily remove underperforming teachers, eradicate tenure and quickly implement program changes.  All of these factors are positive steps toward improving education.  Plus, as a result of his immense power, his teachers do not have to deal with the same type of bureaucracy other educators face in traditional schools.  If they need an answer or wish to make a suggestion, they need only walk down the hall.   

What has gone wrong?

Working 80 hours a week is impressive.  That amount of time is a clear reflection of commitment but not necessarily a formula for success.  As much as I would love to play in the Major Leagues, even if I spent 12 hours a day, seven days a week in the batting cage, I am not going to successfully hit a Roy Halladay fast ball.  Likewise, a teacher who works twice as many hours is not automatically twice as effective.  Every experienced educator has worked with people who arrive ten minutes before the first bell, leave shortly after the buses and are remarkably successful teachers.  On the other hand, some of the most unproductive educators I have encountered spent voluminous hours in the building.  While no one is quite sure what they were doing, there was no doubt as to its ineffectiveness.  A school’s success is measured by student performance not by the time cards or paychecks of the teachers.  Though it is still early in the process the overriding question remains:  Why is the immense sacrifice of time by the TEP staff not yielding better results?

A potential answer could be found in the administrative structure of the school.  There is no question to the commitment of Mr. Vanderhoek.  It is readily apparent that his primary desire is to create a positive learning environment for the students and teachers.  But he is not unique.  There are multitudes of educational leaders whose passion to find a winning educational formula would match his.  And the majority of those individuals understand that the best recipe for success requires multiple ingredients.  (Here is an additional perspective by an NYC parent.)

Good education needs cross pollination

Portions of the TEP model should be replicated throughout school systems in the country.  Similar effective hiring, firing and evaluation policies must be created to form the best teaching staffs possible.  Highly qualified principals have to clearly possess the institutional power to implement their vision of academic excellence.  But this path needs to include a diverse and significant amount of outside input.  Teacher opinions should be constantly sought.  Other programs should be observed and studied.  Stories of success and failure must be shared at every level both vertically and horizontally.   Strategies that have been proven to work need to be utilized and refined; those that do not should be discarded. 

The lessons of the TEP School’s first year are that relying on only one person’s interpretation of best practices can thwart maximizing potential outcomes.  And when teachers are working 90 hours a week and significant student progress is not occurring it is a clear warning that something is terribly wrong.

Posted by Mel Riddile at 11:02 AM | Permalink

by Stuart Singer, The Teacher Leader

There are few lightning rods in the educational landscape of 2011 that rival the debate on the role of testing in the evaluation of student, teacher and school performance.  However, another perspective on this topic was addressed in a recent Mel Riddile post which discussed research indicating that frequent testing had a positive impact on learning.

According to Dr. Riddile, “A recent study summarized in Science magazine and reported in a New York Times article titled To Really Learn, Quit Studying and Take a Test may be a key to unlocking some keys to the teaching and learning process.”  This discussion does not concern the end-of-course barrier exams that are the focal point of most educational conversations.  The research revolves around the use of testing within a teacher’s daily lesson planning.  The study found “practicing retrieval produces greater gains in meaningful learning than elaborate studying.”  As Dr. Riddile notes, “In other words, the simple act of taking a test may improve learning better than any other studying technique including note taking and concept mapping.”

Perhaps the most compelling conclusion noted revolved around the retention of information.  “The Times article went on to say, The research, published online Thursday in the journal Science, found that students who read a passage, then took a test asking them to recall what they had read, retained about 50 percent more of the information a week later than students who used two other methods. One of those methods — repeatedly studying the material — is familiar to legions of students who cram before exams. The other — having students draw detailed diagrams documenting what they are learning — is prized by many teachers because it forces students to make connections among facts.”

The view from the classroom

For forty years I taught high school mathematics.  For the last thirty-eight I employed a teaching technique that paralleled the views expressed in those studies.  Whether the subject was General Math, Algebra 1, Algebra 2 or Pre-calculus I created a classroom strategy that was clearly focused on the concept of frequent and consistent testing.   It was a plan that was simple and direct. 

The centerpiece of the plan

Every class period included a quiz.  It always contained relatively simple questions that could be completed in ten to fifteen minutes.  Questions would be graded on a “right or wrong” basis with little partial credit involved.  It would be the math equivalent of a short-answer, fill-in-the-blanks question.  As the previously noted research found, the regular testing of information led to a number of extremely important outcomes.  Not only did the students retain the material better, they were also clearly aware of their academic status in the class.   A daily evaluation of one’s performance means no one is surprised by their ultimate success or failure.  The teacher also benefits from having a barometer of student learning in every class period.   A quiz that results in a significant number of poor grades requires more work on the topic.  One that indicates overall comprehension allows an educator to move forward with confidence.  Since it is critical that these papers be returned the next class meeting, they must be easy to grade.  The best utilization of time for the teacher is to be able to grade one set of papers while the next class is taking their quiz. 

A systematic approach

My overall classroom strategy was to introduce every topic in three consecutive classes.  The daily quiz was a key component of that plan.  This approach was used regardless of the level of the math or whether the school utilized a block or non-block schedule.  On day 1 a topic would be presented to the students.  An explanation of the concept would be followed by examples and then homework would be assigned to give the students practice.  Day 2 would begin with a review of the homework.  After that review was completed and all questions were answered, a quiz would be given.  Designed to cover this one concept, it was based on questions similar to those found on the homework.  On day 3 the quiz would be returned and reviewed.  

This philosophy was explained in detail to the students on the first day of school.  A typical class would be divided into four segments.  Part one was returning the quiz from the previous session and discussing any questions.  The next segment was reviewing the homework assignment.  Often a worksheet would follow to ensure understanding.  At the conclusion of that conversation the class was given a quiz.  The fourth and final element of the period was devoted to the next topic which would be then practiced in a homework assignment. The next class would be structured in the same manner.   By following this schedule every topic was discussed in three consecutive classes.

It sounds so boring

Obviously, such a highly-structured approach could be a formula for boredom.  Though the basic plan never changed, the challenge for the teacher was to create variety within the segments.  On some occasions I would have my “A” students write the quiz solutions on the board.  An “A” student was anyone who received a grade of “A” on that particular quiz.  Students quickly perceived this opportunity as an “honor” and since all students at one time or another would have a perfect paper I would take care throughout the year to have as many different students as possible receive this recognition.  It was stunning to watch otherwise sophisticated 18-year-olds become giddy when they had a chance to demonstrate their math prowess.  On other occasions, I would personally focus on any problem that was missed by a significant number of students. 

The review of the homework was also approached in different ways.  Volunteers would be solicited on some occasions; other times students were assigned problems.  A third option would have me do the work.  The practice worksheets could be presented as individual work, group projects, contests, or puzzles.  The outcome was always the same—practice—but the methods would vary from day to day. 

The introduction of the new topic would also be open to a variety of educational strategies.  Lecture, group discovery, question-answer and any other method available would be employed on different occasions.

Students love structure

People are most comfortable when they have a familiar routine.  When students feel comfortable in a class they become more confident.  By the end of the first week of school, my students understood the process and knew what to expect each day.  There were no surprises.   At the end of every year I would give my students the opportunity to complete an anonymous evaluation of the course.  When asked for the aspect that contributed the most to their success, the daily quiz was selected more often than all of the other options combined. 

The sincerest form of flattery

Over the course of my career a number of teachers adopted my “daily quiz” approach to teaching.  These individuals taught in courses all across the curriculum.  Many reported not only improved learning but also better communication in terms of student performance.  My wife, an associate Biology professor at a junior college, has successfully used the same strategy with her students. 

Clearly from my perspective those research studies are truly on to something.

Posted by Mel Riddile at 11:28 AM | Permalink

New research may help school leaders with two important challenges that they face on a daily basis. First, in these tight budget times with fewer teachers, larger classes, and fewer resources, how do we improve student performance? How do we do more with less? What are some no-cost ways that we can improve our schools?

Second, given the complexity of course content, particularly in high schools, how do we enhance our skills as instructional leaders? How do we give meaningful feedback to teachers that will enhance their instruction even though we may have little or no background knowledge regarding the content of the course? For example, how do we give feedback to a world language teacher when we have never studied the language and cannot understand a single word they said in the lesson?

A recent study summarized in Science magazine and reported in a New York Times article titled To Really Learn, Quit Studying and Take a Test may be a key to unlocking some keys to the teaching and learning process. However, to find those gems, school leaders need to read between the lines.

Practicing Retrieval

When I read the abstract, my first thought was that this study would serve to support and defend the current obsession with standardized testing. The study concludes "practicing retrieval produces greater gains in meaningful learning than elaborative studying." In other words, the simple act of taking a test may improve learning better than any other studying technique including note taking and "concept mapping."

Furthermore, the researchers concluded that testing might enhance learning far beyond the recall of simple facts. They report "retrieval practice is an effective tool to promote conceptual learning about science."

The Times article went on to say, "The research, published online Thursday in the journal Science, found that students who read a passage, then took a test asking them to recall what they had read, retained about 50 percent more of the information a week later than students who used two other methods. One of those methods — repeatedly studying the material — is familiar to legions of students who cram before exams. The other — having students draw detailed diagrams documenting what they are learning — is prized by many teachers because it forces students to make connections among facts."

Counterintuitive?

Students who used intense review, also known as "cramming" for a test, as well as other popular methods to aid recall such as "concept mapping" or "mind-mapping" believed that they would have better recall of the content. On the other hand, those who took a test after reading a passage believed that they would remember less. In reality, the reverse was true. Ironically, those who took the test and believed that they had learned less actually learned significantly more than their hard studying counterparts.

The real proof of learning

The only evidence of learning is remembering. Notice that I didn't say "memorizing." Remembering is the key. In this case it is the practice of remembering (retrieval) that improves learning. Think about it, if a student cannot remember the essential concepts of the lesson, did the student really learn it?

“I think that learning is all about retrieving, all about reconstructing our knowledge,” said the lead author, Jeffrey Karpicke, an assistant professor of psychology at Purdue University. “I think that we’re tapping into something fundamental about how the mind works when we talk about retrieval.” The Times reported that "several cognitive scientists and education experts said the results were striking.

The researchers divided the students into four groups. One group simply read the content. The second group read and studied the text in four five-minute bursts. The third group used a widely popular strategy known as "concept mapping." The fourth group read the passage, wrote a free-form essay, reread the passage and then took another practice test.

A week later the students were re-assessed and "the students in the testing group did much better than the "concept mappers." They even did better when they were evaluated, not with a short-answer test, but with a test requiring them to draw a concept map from memory."

The experts were surprised by the results of the study. They cannot explain why retrieval testing helps. "The Purdue study supports findings of a recent spate of research showing learning benefits from testing, including benefits when students get questions wrong. But by comparing testing with other methods, the study goes further."

This is a Big Deal

Cognitive psychologist, Dan Willingham indicates “It really bumps it up a level of importance by contrasting it with concept mapping, which many educators think of as sort of the gold standard. Although “it’s not totally obvious that this is shovel-ready — put it in the classroom and it’s good to go — for educators this ought to be a big deal.”

It Throws Down the Gauntlet

Howard Gardner, an education professor at Harvard who advocates constructivism — the idea that children should discover their own approach to learning, emphasizing reasoning over memorization — said in an e-mail to the Times that the results “throw down the gauntlet to those progressive educators, myself included.” Educators who embrace seemingly more active approaches, like concept mapping,” he continued, “are challenged to devise outcome measures that can demonstrate the superiority of such constructivist approaches.”

More Testing?

After reading between the lines, my initial reactions to this article turned out to be unfounded. This study does not promote or denounce standardized testing. Nor does the study promote memorization or rote learning. This study simply supports quality classroom instruction, but how?

Look 4s for School Leaders

Closure and Learning - The focus of instruction is not what teacher teaches but what the students learn. The close of every lesson should focus on what the learner has learned not what the teacher has taught. The question is how does the teacher know that the students have learned and mastered the lesson unless there is some type of formative assessment--quiz, test, or activity.

Remembering - The only evidence of learning is remembering. When observing a lesson ask yourself how does the teacher know that students will remember what they just learned?

Checks for Understanding - Teachers should pause frequently during a lesson to check for understanding. How frequently? As a rule of thumb, teachers should check students understanding approximately every fifteen minutes, which approximates the attention span of the average adolescent. According to the Science study, one of the most effective checks for understanding is the quiz used as a formative assessment. Teachers can pause and ask students to write a summary or take a brief quiz on what they just learned. Immediately re-teaching a concept to a classmate may also be used to test practice retrieval.

Timing is critical. When it comes to recall, tomorrow is too late. Teachers need to check for student understanding before students leave the classroom each day.

Feedback - "Feedback is the breakfast of champions." Unless students practice recall (retrieval) and get immediate feedback they will not remember.

Defined Instructional Practices - Some students absolutely need a highly structured classroom room environment characterized by identifiable instructional practices, smaller units of instruction, more frequent assessments, coupled with frequent and immediate feedback. However, students who can function equally as well in low or highly structured classrooms are not penalized in any way by the use of structure. In other words, when in doubt, use a more structured approach.

Formative Assessments - How often should students be assessed? How frequently students are assessed or asked to practice retrieval depends on their familiarity with the content and the student's level of mastery. When students are introduced to new content or when they are struggling with a particular concept, they should be assessed more frequently. For example, the skills of proficient and advanced readers need only be assessed annually, while students reading at the basic level or below basic need to be assessed regularly. Frequent assessments mean more feedback. A quiz or summary essay at the close of a lesson will do more for student recall than extensive homework assignments.

Mapping - Instructional strategies like "concept mapping" are effective, but they work better if they are used as part of "practice retrieval." The act of creating a "concept map" in and of itself does not improve learning unless the student makes use of the map as a part of the "practice retrieval" process. Teachers should show students how to use the concept maps to review for a test and not assume that the students know how to do so.

What this study really says to school leaders

This study emphasizes the critical importance that school wide defined instructional practices, which include frequent checks for understanding, play in the learning process. When the teacher asks students to reflect on the lesson by practicing retrieval and the students receive immediate feedback, learning improves by as much as 50%.

Next: Checks for Understanding

Posted by Mel Riddile at 09:50 AM | Permalink

Several years ago the College Board announced a dramatic departure from past procedures and began requiring Advanced Placement (AP) teachers to submit a syllabus for approval. I viewed this as an attempt by the College Board to protect both it good name and brand. Up to that point, a course could be called AP, but the content of the course was strictly up to the discretion of the teacher. In theory, the AP course was designed to prepare students to take an AP exam. However, some schools were gaming the system by "offering more AP courses," but, in reality, many of the students enrolled in those courses never sat for the exams. Of those who did take the exams, the national rate of students scoring a passing grade of 3, 4, or 5 plummeted to approximately sixteen percent.

I remarked at the time the AP was trying to look more like the International Baccalaureate (IB), and I predicted that the trend would continue to intensify in the near future. According to the New York Times, that is precisely what is happening and I view this as a positive trend that will dramatically change the quality of the AP courses and take much of the guesswork out of the process for both teachers and students. However, these changes will stress out veteran AP teachers, who have become accustomed to doing their own thing. This means problems for school leaders.

Some of the changes, which take effect in 2012 and 2013 include:

• Reductions in the amount of material that teachers need to teach and that students need to know for the tests
• Providing a curriculum framework for what courses should look like
• A move away from memorization to a focus on bigger concepts and more analytic thinking
• A move away from multiple-choice responses to an emphasis on thinking and written expression
• More hands-on activities and experiments intended to help students think more
• An emphasis on going into greater depth on fewer topics, allowing students to "experience problem-solving, controversies and the subtleties of scholarly investigation.”

Instead of drawing my own conclusions about these changes in philosophy, I practiced what I frequently preach. I asked an expert teacher. In this case, I happen to know a master biology teacher who has a unique perspective. Sherry Singer actually taught AP Biology for approximately fourteen years and then switched to IB Biology for another fourteen years. In here own words, here is Sherry's take on the changes.

"Having taught both AP (Advanced Placement) and IB (International Baccalaureate), it is very clear to me what is happening.  The AP is moving more and more to the IB model.  This is a very good trend in my view. 
When I used to teach AP (and it sounds as if it hasn't changed very much if at all), I was frustrated by the amount of information that was to be taught in one year.  It necessitated students coming in before or after school, during lunch or on Saturdays just to meet the basic requirements of the course.  I could really empathize with the teacher who said that she hoped that she could retire her "swift marches through the organ of the day".  They only required 12 labs but we needed several days each to do them well and even at that, we never really had time to discuss the results or what could have been done to improve the lab.  The tests were largely multiple choice and the essay questions were so broad that it was difficult to determine exactly what the AP was looking for in an answer.
The first and most important difference with IB was that I could have two years.  (Something that the AP still apparently does not permit.)  What a joy to be able to have the time to perform labs well and actually discuss the results.  The IB requires laboratory work but the instructor can choose which labs will be performed and they must be at least partially developed by the student.  In other words, the student actually does research rather than performing a canned lab that was developed by a university professor.  Obviously, the benefits to the students working in this kind of laboratory situation where problem solving was critical transformed lab days for me and my students.  From the article, it sounds like the AP is going in this direction by having students develop their own hypotheses and figure out how to test them - very IB!
The other change that was mentioned was that they are going to give teachers a detailed syllabus stating what topics need to be taught and which can be left out.  If they do as good a job as the IB, this should be a great help to AP teachers.  I found the IB syllabus to be a huge help when designing my course.  (In fact I used to copy the IB syllabus and give it to the students to help them study for the exam.) If a topic, term or concept was not in the syllabus it was NEVER on the IB exam.  It allows teachers to have more time to develop concepts in depth.  I remember the first few years that I taught IB, luxuriating in the extra time to do labs well and have meaningful, in-depth discussions with students without feeling the clock ticking behind me. 
Finally, the IB tests were only 40 multiple-choice questions.  The other questions were data-based or essay.  The data-based questions often dealt with data that the students had never encountered before. This was done by design and was intended to help the graders determine which students could take their knowledge of biology and make sense out of totally new data - something that scientists have to do every day.  The third type of question was essay.  But the essays were always divided into sub-headings, which helped the students organize their answers and ensured that they delivered the information that the test designers were looking for." 

What school leaders should consider

• The changes in AP Biology and other science courses courses will require appropriately equipped labs, which, in tight budget times, may be difficult to implement.
• "Another concern is how well teachers — across the full range of A.P. subjects — will adjust to an approach that will require them to give up some control and let the students dictate more about where the class discussions go." Like IB, these changes in AP will require a considerable amount of professional development. As Sherry Singer once told me, "We are going to have to totally change the way we teach science."
• Both AP and the new Common Core assessments will require more writing by our students, which is known weakness in most high schools. We cannot wait until students take their first AP course to begin emphasizing writing skills. Ditto for critical thinking skills.

Posted by Mel Riddile at 01:59 PM | Permalink

According to a report by Catherine Gewertz of Education Week, a study released today by the Center on Education Policy, indicates that testing at the high school level will increase.

• More states are using assessments as barriers to graduation and requiring other exams that are not linked to graduation. Twenty-eight states now have such requirements, up from 26 in the 2009 report.
• Three quarters of the nation’s students now attend schools in states that give exit exams.
• Twenty-three states currently give end-of-course tests. Only seven use them as exit exams, but another 10 plan to begin doing so.
• States increasingly are requiring students to take a college-entrance exam—the ACT or SAT—or a workplace-readiness test such as WorkKeys.
• More states also are requiring or considering some form of portfolio assessment.

Posted by Mel Riddile at 05:33 AM | Permalink

"There are three kinds of lies; lies, damn lies, and statistics."--Mark Twain

The release of the 2009 PISA results this past week has created quite a stir and has provided ample fodder for public school bashers and doomsayers who further their own philosophical and profit-motivated agendas by painting all public schools as failing. For whatever reason, these so-called experts, many of whom have had little or no actual exposure to public schools, refuse to paint an accurate picture of the state of education.

Secretary of Education, Arne Duncan, should be providing the nation with a proper vision and focus for public education. He knows our challenges all too well. He confirmed that he gets it when he recently wrote me saying, "We must build a culture nationally where great educators ... choose to work with children and communities who need the most help." I believe his message is sincere and heartfelt and it is spot on. However, overstating a problem in order to increase the sense of urgency around school improvement is just as bad as understating the problem.

This week, Duncan had a golden opportunity to use the PISA results to provide focus for our education efforts and to point us in the right direction. Instead, he dug himself deeper into the pseudo-reformers' hole--more charter schools, more reliance on competition and free-market strategies, more testing, more use of test scores to evaluate teachers, more firing of principals and teachers, more closing of low-scoring schools--when he said, "the PISA scores released this past Tuesday were "a massive wake-up call," because the scores show American students holding relatively steady in the middle of the pack of the developed nations taking the international exam.

There is, however, someone who recognizes that the data is being misinterpreted.  NEAToday published remarks from National Association of Secondary School Principals Executive Director, Dr. Gerald N. Tirozzi, that have taken "a closer look at how the U.S. reading scores on PISA compared with the rest of the world’s, overlaying it with the statistics on how many of the tested students are in the government’s free and reduced lunch program for students below the poverty line." Tirozzi pointed out, “Once again, we’re reminded that students in poverty require intensive supports to break past a condition that formal schooling alone cannot overcome.” Tirozzi demonstrates the correlation between socio-economic status and reading by presenting the PISA scores in terms of individual American schools and poverty.  While the overall PISA rankings ignore such differences in the tested schools, when groupings based on the rate of free and reduced lunch are created, a direct relationship is established.

With strong evidence that increased poverty results in lower PISA scores the next question to be asked is what are the poverty rates of the countries being tested?  (Listed below are the countries that were tested by PISA along with available poverty rates. Some nations like Korea do not report poverty rates.)

Leveling the playing field

A more accurate assessment of the performance of U.S. students would be obtained by comparing the scores of American schools with comparable poverty rates to those of other countries.

Schools in the United States with less than a 10% poverty rate had a PISA score of 551.  When compared to the ten countries with similar poverty numbers, that score ranked first. 

In the next category (10-24.9%) the U.S. average of 527 placed first out of the ten comparable nations. 

For the remaining U.S. schools, their poverty rates over 25% far exceed any other country tested.  However, when the U.S. average of 502 for poverty rates between 25-49.9% is compared with other countries it is still in the upper half of the scores.

Mathematically Speaking

The results of the latest PISA testing should raise serious concerns.  However, the overall ranking of 14th in reading is not the reason to be concerned. The problem is not as much with our educational system as it is with our high poverty rates. The real crisis is the level of poverty in too many of our schools and the relationship between poverty and student achievement. Our lowest achieving schools are the most under-resourced schools with the highest number of disadvantaged students. We cannot treat these schools in the same way that we would schools in more advantaged neighborhoods or we will continue to get the same results. The PISA results point out that the U.S. is not alone in facing the challenge of raising the performance of disadvantaged students.

Additional observations from PISA results:

·      Shanghai, China topped the list with 556 but is not included in this analysis because Shanghai is a city not a country and because only 35% of Chinese students ever enter high school and because "when you spend all your time preparing for tests, and when students are selected based on their test-taking abilities, you get outstanding test scores."

·      Of all the nations participating in the PISA assessment, the U.S. has, by far, the largest number of students living in poverty--21.7%. The next closest nations in terms of poverty levels are the United Kingdom and New Zealand have poverty rates that are 75% of ours.

·      U.S. students in schools with 10% or less poverty are number one country in the world.

·      U.S. students in schools with 10-24.9% poverty are third behind Korea, and Finland.

·      U.S. students in schools with 25-50% poverty are tenth in the world.

·      U.S. students in schools with greater than 50% poverty are near the bottom.

·      There were other surprises. Germany with less than half our poverty, scored below the U.S. as did France with less than a third our poverty and Sweden with a low 3.6% poverty rate.

·      Having recently listened to Sir Michael Barber talk about the amazing progress of the reforms in the United Kingdom, I was absolutely shocked to see that the UK, with 25% less poverty, scored below the U.S. average.

The Real Meaning of PISA: It's Poverty Not Stupid

If the so-called experts would have honestly and responsibly reported the PISA results, we might now be on the road to responsible school improvement instead continuing down the road of "reform de jour."

President Bill Clinton is famous for his campaign slogan, "It's the economy stupid!" When it comes to student achievement and school improvement, it's poverty not stupid! Researchers report that perhaps the only true linear relationship in the social sciences is the relationship between poverty and student performance. While there is no relationship between poverty and ability, the relationship between poverty and achievement is almost foolproof. To deny that poverty is a factor to be overcome as opposed to an excuse is to deny the reality that all educators, human services workers, law enforcement officers, medical professionals and religious clergy know and have known for years.

PISA reports average scores. The problem is that the U.S. is not average. While the U.S. is the top country in global competitiveness, we also have the highest percentage of students living in poverty and, regretfully, poverty impacts test scores.

To Secretary Duncan, poverty is not an issue that educators must address. At least he won't admit it in public. Apparently, he wants to take away all the excuses from teachers and principals. When I met with Duncan, I asked him if he had read the book or seen the movie, Blind Side. He indicated that he had and that he had enjoyed it very much. I reminded him of the pride and sense of accomplishment felt by the teachers in the private school attended by Michael Oher. In their minds, they had performed a miracle. I pointed out that, in high-poverty schools, a Michael Oher is the average student. In schools like ours we have hundreds of students like Michael Oher who depend on our school for everything including food, clothing, and emotional support.

The Bottom Line

School improvement is not an event. It is an ongoing process that has no end. As a principal, parents and community members would repeatedly ask me, "When can we stop our comprehensive school-wide literacy initiative? I would answer, "We will stop emphasizing reading, writing, thinking and speaking when our parents repeatedly complain that their children are reading too fast with comprehension that is too high and when our students' writing skills are so superior that they are regularly winning Pulitzers and other literary awards." Smiles would erupt throughout the audience. They got it. They understood that literacy skills can always be improved and so can our schools.

There are three compelling reasons why we must improve our schools:

1. We have a moral and ethical obligation to provide every student with the best education, the kind of education that we would want for our own children.
2. In a knowledge economy, the country with the best-educated populace will have the highest standard of living.
3. Every dropout as well as every graduate who is not prepared for at least some post-secondary education and training is and will continue to be an economic and social burden on their local community and on this nation for their entire life.

The challenge of ensuring that each and every student is a life-long learner prepared to contribute in a global community is daunting enough. We don't need more hyperbole, particularly from those education insiders who should know better. For those of us who are deeply committed to improving the performance of every student, this rhetoric is counterproductive because it seriously erodes our ability to hire teachers, obtain resources, and gain the confidence and support of our communities.

We count on our leaders to provide focus and direction. Sadly, our education leaders don't trust us enough to tell us the truth. The problem is that we will never solve a problem that our leaders refuse to admit even exists. The comparison of PISA scores by poverty clearly identifies our strengths and challenges as a nation. Our schools with less than 50% poverty) are some of the best in the world. Our extremely high-poverty schools, with over 50% poverty, are among the poorest performing internationally.

Instead of labeling all schools as failing, we must find a way to raise the performance of our students in under-resourced schools. Instead of looking to low-poverty countries like Finland for direction, we should be looking to take what we already know about educating students in high-performing, high-poverty schools like our Breakthrough Schools and scaling up their successes across the nation. We continually look for gold in other countries when, all along, we are sitting on Acres of Diamonds.

Truthfully, you and I know all too well that Secretary Duncan, who led schools in Chicago, is aware of the relationship between poverty and student achievement, but he doesn't trust us enough to tell us the truth. He is afraid that we will use poverty as an excuse and that we will forget about our disadvantaged students. Ironically, by not acknowledging poverty as a challenge to be overcome, Duncan is forgetting about our disadvantaged students. Duncan needs to deliver the message that all our students deserve not only access to an education, but access to an excellent education. He needs to repeatedly remind us that, when it comes to school improvement, it's poverty not stupid.

Posted by Mel Riddile at 12:13 PM | Permalink | Comments (2)

I hesitate to address the sensitive topic of international comparisons with school leaders who have to face the reality of leading schools on a day-to-day basis. However, I wasn't subjected to the kind of attacks on public schools, teachers, and principals that we have experienced of the past year. In the past, when NAEP or PISA results were released, we simply shrugged our shoulders and moved on. Today, however, our teachers and community expect us to respond when asked. In fact, our silence on this matter could be deafening.

That is why I put together some talking points for school leaders on the 2009 PISA results. I have drawn from a number of sources including the Organization for Economic Cooperation (OECD), which coordinates the international assessments and the Washington Post.

Background

• Begun in 2000, the Program for International Student Assessment (PISA) is a system of international assessments that focuses on 15-year-olds' capabilities in reading literacy, mathematics literacy, and science literacy.
• PISA is coordinated by the Organization for Economic Cooperation and Development (OECD), an intergovernmental organization of industrialized countries. 38 OECD nations and 28 partner nations participated in the assessment.
• PISA includes measures of general or cross-curricular competencies such as problem solving.
• PISA emphasizes functional skills that students have acquired as they near the end of compulsory schooling.
• The U.S. sample for the latest results includes both public and private schools, with 165 schools and 5,233 students participating in all. Schools are randomly selected and 15-year-old students within those schools are randomly selected.

2009 Results

• Reading: The U.S. average score in reading (500) was not measurably different than other OECD countries. U.S. female students scored higher than male students.
• U.S. 15-year-olds had an average score of 500 on the combined reading literacy scale, not measurably different from the OECD average score of 493. Among the 33 other OECD countries, 6 countries had higher average scores than the United States, 13 had lower average scores, and 14 had average scores not measurably different from the U.S. average. Among the 64 other OECD countries, non-OECD countries and other education systems, 9 had higher average scores than the United States, 39 had lower average scores, and 16 had average scores not measurably different from the U.S. average. 

• Math: U.S. average score in math was lower than the OECD average. Male students, in general, scored higher than female students. Since 2006, U.S. has caught up with 9 countries.
• U.S. 15-year-olds had an average score of 487 on the mathematics literacy scale, which was lower than the OECD average score of 496. Among the 33 other OECD countries, 17 countries had higher average scores than the United States, 5 had lower average scores, and 11 had average scores not measurably different from the U.S. average. Among the 64 other OECD countries, non-OECD countries, and other education systems, 23 had higher average scores than the United States, 29 had lower average scores, and 12 had average scores not measurably different from the U.S. average score. 

• Science: 12 other OECD countries had higher average scores than the United States.
• On the science literacy scale, the average score of U.S. students (502) was not measurably different from the OECD average (501). Among the 33 other OECD countries, 12 had higher average scores than the United States, 9 had lower average scores, and 12 had average scores that were not measurably different. Among the 64 other OECD countries, non-OECD countries, and other education systems, 18 had higher average scores, 33 had lower average scores, and 13 had average scores that were not measurably different from the U.S. average score.
• Male students scored higher than female students. Overall score was higher than 2006, and the gains in science exceeded those for math.
• The US is one of three nations that give more money to highly advantaged schools than to disadvantaged schools.
• Overall, private schools do better on PISA...until you account for SES.
• There are number of high performing economically disadvantaged schools in the US: "success is possible against all odds."

U.S. Strengths and Weaknesses

• U.S. students showed the best relative performance in answering questions that judged students’ ability to reflect and evaluate information. On that measure, the United States ranked seventh out of the 34 OECD nations.
• The weakest area for U.S. achievement was in accessing and retrieving information, for which students tied for 19th place with France.

Behind the Facts

• The PISA rankings are determined by nations’ average scores. "Some researchers have suggested, however, that average score comparisons are not useful: even presuming that the tests have some meaning for future accomplishment, average students are not likely to be the leaders in fields of mathematics and science."
• In the last administration of PISA, the United States has 25% of all high-scoring students in the world. Among nations with high average scores, Japan accounted for 13% of the highest scorers, Korea 5%, Taipei 3%, Finland 1%, and Hong Kong 1%.
• The fact that one of four high-scoring students came from the United States and the remaining high-scores came from the other 58 countries participating "suggests that many American schools are actually doing very well indeed."
• "Well-resourced schools serving wealthy neighborhoods are showing excellent results. Poorly resourced schools serving low-income communities of color do far worse."
• The U.S. had many more students scoring at the lowest levels; these kids likely can’t compete for the good jobs in the country."
• "Americans in low-poverty schools look very good, even in mathematics. They would be ranked third in the 4th grade (among 36 nations) 6th in the 8th grade (among 47 nations). This is important because while other developed nations have poor children, the U. S. has a much higher proportion and a much weaker safety net. When UNICEF studied poverty in 22 wealthy nations, the U. S. ranked 21st."
• The highest scoring countries have less diversity and less poverty.

PISA confirms what we already know. The U.S. is quite capable of producing top performing students in well-resourced schools serving middle class neighborhoods. Under-resourced schools in poor neighborhoods do not fare as well.

Resources:

OECD

Are today’s students prepared for the knowledge economy of the 21st century?

PISA: Who made the grade? (OECD)

Washington Post

Do international test comparisons make sense?

Hysteria over PISA misses the point

Posted by Mel Riddile at 11:54 AM | Permalink

by Stuart Singer, The Teacher Leader

Previously, Mel Riddile has asked whether there is a schism within high school faculties as a result of the continual emphasis on standardized testing.   He quotes a teacher who believes there is such a problem—the core teachers feel they are receiving an unfair level of scrutiny while the non-core believe they are being ignored. 

A perfect storm of discontent  

There is little doubt that the public spotlight is squarely on a school’s standardized test scores and the teachers whose students produce them.  Schools are accredited, ranked, and publically evaluated based on the results of a few tests in a limited number of subjects.  Virtually every conversation concerning teacher evaluation begins, and too often ends, with a desire to use student test scores as a major component. With the misguided encouragement of educational leaders some newspapers have ranked teachers based on their students’ results. Virtually every discussion of merit pay includes student scores as a primary consideration.

It is not surprising that this fixation on certain numbers has split teaching staffs into two separate camps.  The pressure on those who are responsible for tested classes is immense and often oppressive.  As it ratchets up during the year, these teachers become increasingly resentful of their colleagues who do not have to deal with similar issues.  Intensifying these sentiments is the irony that so much attention is being given to a group that represents less than one third of a typical high school staff.

These ingredients result in the great disconnect within the teaching community.  If test results are a must in evaluation and merit pay, what does that say about those who do not produce such scores?  Many teachers are forced to ask some tough questions.  Do policymakers mistakenly believe that all educators produce such scores?   Or worse, do they feel that creating an evaluation for those teachers is not a critical concern?  Likewise, should they be excluded from merit pay consideration?  Regardless of the reasons, the message they hear is clear—if you do not create standardized test data within your classroom, you are relegated to a lesser status.

This educational caste system gets worse.  It turns out that not all standardized tests scores are equal.  NCLB focuses on only the results of math and English exams.  The exclusion of science and social studies from this mandate places the teachers of those subjects in a difficult position. While they have the same curriculum pressures as math and English colleagues, their work is clearly not considered as important. The state of Virginia has affirmed this stance.  While a score of 400 on the end-of-course exams is required for a “pass”, the state established a special policy for students in the two non-NCLB curricula—if a student takes the exam twice and scores at least 375 on one of the attempts, they are eligible for a “locally verified credit”.  The teacher and school are still charged with a failing score but a student who has mastered less than 35% of the material is declared successful. 

No cures but some ways to help

While it is impossible for any school to totally alter the educational culture, there are steps that can be taken to lessen the chasm between core and non-core teachers.  As Dr. Riddile noted in his post, the implementation of a literacy program at his school was woven into the entire curriculum.  This approach was the result of both necessity and design.  In order to be successful, this initiative needed to be incorporated into all subjects rather than implemented by the English department alone.  By asking every discipline to embed literacy into all of their classroom activities, staff members had an equal stake in a critically important program.  Every meeting, email or memo in regards to this program was appropriate for the entire staff.

The school took a number of additional steps to mitigate this problem.  The goal of each of these concepts was simple—demonstrate to all teachers that every class was important.  These included:

The end of year testing schedule did not negatively impact non-tested classes. Many schools adopted testing schedules that would involve massive disruptions to all classes for an extended period of time.  Considering that more than two thirds of the classes in our building did not have end-of-course exams such an approach was deemed to be unacceptable.  Our test schedule was designed to ensure that every class would meet during the entire testing window.  The length of the periods would be altered to allow appropriate time for the exams but no class was lost.  Also since testing was done within the period the subject was taught, no students were pulled from other rooms to test. 

Discussions of SOL (state assessment) issues were limited to staff members who were directly affected.  At department chair meetings, the core department chairs would meet as a group after the conclusion of the topics pertaining to all subjects.  Likewise, faculty meetings were focused on topics of general interest.

Another possible solution

Authentic success in almost any endeavor is the result of an outstanding team effort.  Improvement in standardized test score is no exception.  Why not acknowledge this achievement as a school-wide effort?  Instead of awarding merit pay to individual teachers whose students exceed certain standards, a practice that can cause divisions within a staff, reward the entire school when the student body attains prescribed benchmarks.  Either give a bonus to all staff members or make the award in the form of extra funding for the school and its programs.   While such an approach would not end all friction among teachers, it would certainly be a good starting point.

Posted by Mel Riddile at 09:04 AM | Permalink

by Stuart Singer, The Teacher Leader

For years Mel Riddile and I shared the belief that there were a number of factors that could have a significant impact on the success or failure of students on standardized tests.  When scheduling the Virginia Standards of Learning (SOL) exams we took great care to create a positive test-taking environment.  Now there is research that indicates such concerns were justified. 

A recent study of the results of SAT testing in Indiana has found that each change between daylight savings and standard time adversely affected student performance.  Indiana was chosen because counties in the state could independently choose whether or not to participate in daylight savings.  This option created data from two different groups of students—those who changed their clocks and those who did not. 

The Journal of Neuroscience, Psychology, and Economics found that students who lived in areas which used daylight savings had scores that were consistently 16 points lower than testers who did not have a time change.  A loss of this magnitude could be critical and should serve as a warning to anyone responsible for planning test schedules.  

Some points to consider

While the SAT is different in many ways from end of course barrier exams, the lessons to be learned from this research are clear.  If a simple one-hour shift can demonstrably cause negative consequences, it is incumbent on school leaders to take great care in creating a plan for large scale testing.  Here are some suggestions that worked at my school.

Keep the testing areas small.   Many schools in our district would place hundreds of students in large testing areas such as the cafeteria or gymnasium.  As a direct result of their size these locations would have high levels of confusion, distractions and poor discipline.  To avoid those complications we designed our testing to place students into classrooms, computer labs, or small common areas.  The rule was that no more than three classes (about 75 students) could be placed at any one site. 

Maintain a testing schedule similar to the regular school day.  Rather than alter the normal sequence of classes we built our plan to work within the framework of the traditional day.  Classes met in the same order as they had all year.  The testing periods on a particular day would be lengthened to provide additional time.  No class whether it was an SOL course or not was eliminated during the entire testing window. More importantly this procedure ensured that students who had a tested course during the second block of the day took their exam in the second block of the day.  Schools that implemented special schemes for testing would often lose all sense of continuity.  Students who took a subject in the afternoon might take the exam in the morning or a morning class might be tested after lunch.  Could such a variation cause lower scores?   Think daylight savings time.

Have teachers proctor their own students.  Another advantage of maintaining the normal class sequence is that it allows teachers to work directly with their classes.  Special schedules will usually preclude this possibility.  There are many good reasons for keeping teachers and students together.   Students feel more comfortable asking questions of someone with whom they have a personal relationship.  Teachers will do whatever is necessary to ensure that their students have the best possible circumstances to test.  And there are far fewer discipline issues when proctors know the individuals in the room. 

Place classes from the same curriculum in a testing site.  The untimed SOL exams took dramatically different amounts of time to complete.  A typical Algebra 1 group would take more than two hours to finish while most Biology 1 students would be done in an hour.  Whether a school’s policy is to let early finishers leave or to require that those who are finished wait for extended periods of time, the academic climate in the testing area will be negatively affected.  Also, the initial verbal directions for tests can vary from one discipline to another thus adding to the confusion.  And finally, teachers from the same department are more likely to be able to work together smoothly.

Create the testing schedule well in advance.  In order to plan effectively teachers and students should be informed of the schedule as early as possible.  Whether an exam is on day one or day seven of an eight day testing window is significant.  Our goal was to always distribute the schedule at a department chair meeting two months in advance of testing and answer questions at a faculty meeting prior to the dates.

Not easy, but critical

A testing program that maximizes performance is especially important for certain groups of students.  Individuals in ELL or those with special needs are continually straddling the line between success and failure on standardized tests.  Every advantage or disadvantage a school provides in the design of its testing program can change the trajectory of these young lives.   Such an approach is often not the most convenient choice for the adults in the building.  A student-oriented scheme requires intricate planning, extensive explanations, multiple revisions and demands full participation by every staff member.  But these concerns are a small and reasonable price to pay for our students. 

Posted by Mel Riddile at 11:09 AM | Permalink | Comments (1)

Here's a news flash. The National Assessment Governing Board has convened an ad hoc committee to study the motivation of 12th graders to take the NAEP assessments. I can save NAGB a lot of time and money. Ask any high school teacher or principal and they will tell you. If you are testing 12th graders, and the tests don't matter, forget it. It is a waste of time and money. However, like most issues in education today, no one bothers to ask anyone who works or has worked in a school. After all, what do they know?

Wake Up People

Kids across the country are "christmas-treeing" tests, not because they are not motivated, but because they are smart. This is not a protest against tests and testing. Students know what tests matters and what tests don't.

As I have reported in the past, teachers and principals across the country are being victimized by state testing programs that holds teachers and schools accountable but not students. The fate of public education in those states rests on the good will of the students. If students feel like trying, they do. If they don't feel like trying, they simply "Christmas-tree" the test by drawing patterns on the answer sheets.

I worked with one district in which all the high school principals were fired or replaced and hundreds of teachers fired or transferred on the basis of student test scores and that state had absolutely no student accountability.

In yet another state, a high school principal lamented that his students inexplicably decided that they were not going to put forth their best effort on the state tests. Despite the school sending record numbers of students to four-year colleges, the high school was placed on a state list of “low-performing schools.”

I worked in a high-pressure, high-accountability state that held high schools accountable by using eleven end-of-course exams to calculate adequate yearly progress. However, students were also held accountable. The tests were used to award course credit and as barriers to graduation. In this context, everyone, students, teachers, and administrators took the tests seriously. In the early days of the state program, only the schools were held accountable, and it was difficult to get the students motivated to take the tests.

Been NAEPed?

Three years ago, our school was "chosen" to participate in NAEP testing. A team for the state DOE descended on us and took complete control of the selection of students as well as the testing logistics and conditions. Our seniors were "randomly" selected to participate. The DOE team decided that testing would take place in the gym, a location that no one that was serious about the test would have chosen. The DOE team not me, the principal, delivered the charge to the students. I have no idea how our kids did, but I was certain that I didn't want my job to hang on the results.

No Feedback, No Effort

"Feedback is the breakfast of champions." Students know that neither they nor anyone else will ever see the results of their NAEP test. They know that their school will not be held accountable. They know that the results will not reflect on their teachers.

Don't Count = Don't Care

Diane Ravitch understands when she writes, "The students know that the tests don't count, that there are no individual scores, that no one will ever know if they did well or poorly, and they are not motivated to do their best. 

The public does not realize that NAEP is a sampling test, and it is not given to every student. They also don't realize that no student takes the entire test, only a portion of it. The seniors may not know that they are part of a national sample, but they know that this test will not affect their grades, their likelihood of graduating, or their plans after high school."

Others wrote comments supporting Ravitch's position.

Senioritis

"As a recently retired high school principal, I can attest to the fact that 12th graders, infected with "senioritis," do not care about the NAEP tests. They have mentally moved on from high school. In Arizona, until seniors had to pass our state assessment tests in order to get a diploma, they did not care about it either. If we are to use the NAEP tests, we will have to come up with something that is in the students' self-interest."

Fed Up With "Testing Nonsense"

"Diane is right that these particular tests are a waste of money. My high school senior and her friends are fed up with all the testing nonsense they've endured for years."

Sick of being guinea pigs

"Way back in 1983, as a highly motivated, successful high school senior at a competitive science/math boarding school, I was given the same standardized test several times over the year. By the last time we took it, my friends and I were deliberately choosing wrong answers because we were sick of being guinea pigs."

Christmas-Treeing Defined

Diane Ravitch gets the last word. "NAEP tests don't matter. And seniors know it. They doodle on their test papers, or they select answers with a pattern, like all B, or all C, or ABCD/ABCD. Or they leave questions blank, without even bothering to make a guess. The government should stop wasting money on this test in this grade, and the usual critics should turn their fire elsewhere."

The Bottom Line

If you work in a high school in a state that has no accountability for students, you will have to do everything possible to earn the good will of the students. Hopefully, you and the staff have a big "emotional bank account" from which to draw.

Posted by Mel Riddile at 08:07 PM | Permalink