What the latest revelations on test cheating really mean
A year ago, I was one of four academics or test specialists that advised USA Today and its affiliated Gannett newspapers when it conducted a multistate analysis of irregularities in assessment data. These journalists worked with an excellent dataset – it had data at the level of individual students, which allowed us to ensure that erratic patterns reflected changes in the scores of individuals, rather than changes in the composition of a school (due to changes in enrollment at a school, for instance). The USA Today dataset also allowed the reporters and analysts to reveal erratic patterns in test scores and erasures rates from year to year.
The resulting USA Today story was therefore able to present a thorough and comprehensive analysis that resulted in the identification of relatively few schools at which there was a very high likelihood that systematic cheating was taking place. The journalists responsible for this story were later recognized with the Philip Meyer Award for investigative journalism. The series of articles resulting from this study prompted a federal Department of Education investigation into the testing practices at District of Columbia schools and it also resulted in a tightening of security around testing.
Given my past role in reviewing data and methods used for detecting systematic cheating, I was delighted to have the opportunity a week ago to review Ohio assessment data that was being used as part of a national study released today by The Atlanta Constitution-Journal and affiliated Cox newspapers. My review, however, yielded serious concerns about the data used, the methods of analysis employed, and the conclusions drawn. I shared these concerns with journalists at the Dayton Daily News, which is one of the Cox affiliates involved in this story.
To be clear, the Cox analysis may accurately detect irregularities in assessment results from year to year. But my own analysis of the data suggests that these irregularities are less likely due to actual cheating than due to mobility in student population (recall the lack of student-level data). Although the Cox news articles on this study offer a disclaimer that their analysis does not actually prove cheating, this disclaimer should be expanded considerably.
In short, here are some of my concerns about the methods:
- As noted, the analysis is based on school-level data and not individual student-level data. Accordingly, it was not possible to ensure that the same students were in the group in both years.
- The analysis of irregular jumps in test scores should have been coupled with irregularities in erasure data where this data was available.
- The analysis by Cox generates predicted values for schools, but this does not incorporate demographic characteristics of the student population.
- The limited details available on the study methods made it impossible to replicate and verify what the journalists were doing. Further, the rationale was unclear for some of the steps they took.
Yes, there may in fact have been cheating in some of the schools and districts flagged in the report. But it seems likely to me that most of those flagged were not in fact engaged in cheating. If a more thorough and rigorous analysis had been conducted, the number of flagged schools and districts would have declined substantially. With a more focused list of schools and districts, the journalists could have then contacted the schools to see if there were reasonable explanations for irregularities detected.
The resulting news story appears to be intended to be alarmist, implying that cheating is rampant in our schools. It is fortunate that the journalists in Ohio at least have restrained from reporting the names of the specific schools flagged, since suspicions would have been unfairly cast on hundreds of improperly flagged schools. The irregularities in such schools likely arose simply because there was a large change in the actual students taking the test from year to year.
Given that the methods used were much more likely to identify schools with high mobility, it comes as no surprise that charter schools are highly represented in the flagged schools and that the Houston Public Schools also garnered considerable attention. Recall that Houston was heavily impacted by the influx of students relocating there after Hurricane Katrina in 2005, which is the starting year for the Cox analysis.
We all need to be concerned about cheating and its implications. At the same time, we need to be leery of sensational attempts to secure headlines with weak and incomplete analyses.
The increasing focus and reliance on standardized tests to evaluate schools and teachers is resulting in cheating. That’s probably inevitable. But it’s also probably minimal. The bigger problem is a more serious type of cheating – one that’s perfectly legal and apparently acceptable. Students are being cheated of a broader education that emphasizes a balance of creativity, extracurricular activities, foreign languages, higher math and science skills and other opportunities due to the over-emphasis on testing for basic math and reading. In this sense, a fixation on testing cheats not only our students but also their communities and the future employers who will depend on their creativity and can-do problem-solving. And our democracy is certainly cheated when our youth are unprepared for healthy civic engagement.
Yes, it is important for reporters and others to seriously pursue stories about schools engaged in wrongful practices. The groundwork done by the Cox reporters is part of that (although I wish they had pursued their investigation further and more carefully before publication). But we as a nation are missing the forest for the trees. No cheating on tests is as serious as the cheating done by the tests.
See the story at http://ow.ly/9RS98
By Gary Miron, professor of education at Western Michigan University who has extensive experience evaluating school reforms and education policies. Over the past two decades he has conducted several studies of school choice programs in Europe and in the United States, including nine state evaluations of charter school reforms. Before coming to Western Michigan University, he worked for 10 years at Stockholm University in Sweden.
Invest to help meet the needs of students
Eddie Murphy’s new film, “A Thousand Words,” was released on March 9. Judging from the reviews, it will not be a box-office hit, and I will probably not pay a babysitter to watch my kids so my wife and I can go see it. However, the premise of being able to speak only a thousand words before dying, is intriguing. In my daily routine of teaching, spending time with my family and interacting with my colleagues, I am sure I wouldn’t last for more than a few hours. With only a thousand words, I wouldn’t be able to express all the things I want to say to all the people I want to say them to. Even if I cut my conversations to the most minimal interactions, I would still quickly run through my quota of words.
The theme of having many valuable ways to use resources, but far too few resources at hand, is something that educators deal with every day. With state funding cuts and levy failures, programs of study are eliminated, class sizes are increased, and jobs are cut in school districts everywhere. There isn’t enough money allocated for districts to accomplish all their goals. It’s not about “trimming the fat,” since there often is no fat to trim.
With the current funding crisis in mind, it’s hard for me to think about where, specifically, I’d like to see more money allocated for schools, because every facet of public schools is in need of more funding. We could use more teachers, newer computers, improved facilities, more supplies….the list is endless. Despite this long list, if I had to prioritize, there are a few integral areas where I would like schools to invest to help meet the needs of students.
I would start by investing in the guidance office. Many students come to school with emotional burdens that need to be addressed before they can focus on learning. Children are dealing with their parents’ divorces, family money troubles, social issues and substance abuse. Increasingly, the responsibility of dealing with these problems has fallen on schools, which fortunately are filled with many caring people. Unfortunately, guidance counselors, the people who are best trained to offer advice and solutions, are overburdened with handling standardized tests and scheduling students. Elementary schools share counselors between several buildings, so when a student needs guidance, a counselor may not even be on site to talk to the student.
Continuing with the idea of helping students deal with the difficult issues they face in life, I would invest in resources that would help students develop life skills. Many students leave high school with a solid understanding of how to write a good essay for their freshman composition class in college. However, far fewer go off to college knowing how to manage their time and money. Even fewer go off to school with an understanding of what path they should pursue towards earning a degree. They major in business because it “sounds good,” not because it is an area of strength or interest. We can’t assume that parents are going to provide kids with direction. Part of my high school’s mission statement is that we will “…prepare our students for responsible citizenship, lifelong success, and productive employment in a global community.” In order to attain our mission, we must invest in programs that help students acquire life skills.
Not only would I invest in the high school curriculum, I would also invest in the primary grades. I’m not an expert in the elementary-school world, but my wife teaches first grade and my daughter is in first grade, so I get their insights daily. From that knowledge, I would invest in smaller class sizes in kindergarten up to third grade. I realize that class sizes need to be reduced at every level, but it’s most crucial in the early years. When I hear about the wide range of knowledge students have when they enter school, I can’t help but believe the children would be better serviced with a student-to-teacher ratio of 15 to 1 maximum. When there’s a group of kindergarten kids that includes children who can read at a second grade level and students who can’t recognize their name in print, I can’t imagine how hard it is to meet all their educational needs simultaneously. By reducing the number of students per class, each child can get the necessary help and also feel challenged at a time when learning and being in school is new and exciting.
Just as trying to express yourself in only 1,000 words for a lifetime is a nearly impossible task, so is trying to adequately invest in all the crucial programs that schools offer. Hopefully, state and local funding will soon increase, so that, like Eddie Murphy’s movie, we can have a happy resolution.
By Dan Greenberg, Sylvania Education Association
The Foundation for Student Learning
Each year in the spring, Brookhaven High School always has a crop of education majors completing their student teaching with our students. During the week that the Ohio Graduation Test is administered, our class schedule is put on hold and our soon-to-be teachers see the real-life impact of standardized testing on our students.
At some point during the week, after testing has concluded for day, I always ask our student teachers the same question.
“If one of these students doesn’t pass the test they took earlier today, whose fault is it?”
After posing the question to them, I tell them there’s no right or wrong answer, that I’m interested in hearing their opinion.
Some answer quickly, others take a bit longer. As a group, their collective answer to the question always incorporates the usual suspects. A few say parents are to blame, while others say the students’ teachers are at fault; other student teachers place the blame of potential failure on the students themselves. Interestingly enough, in the many years that I have been posing my query to Brookhaven’s student teachers, not one of them has placed the blame for a student’s potential failure on elected officials.
While I tell the pre-service teachers that there is no right or wrong answer to the question, there has to be a right answer because our students’ future is on the line. Each child’s success bin the classroom is something that we as teachers are deeply committed to.
For me, I believe the right answer to the question is D, “all of the above.”
I understand that I am my students’ teacher, but my efforts alone are not enough to help my students be successful—everyone must work together for this to occur. In fact, all of us—teachers, students, parents and our elected officials should be held accountable for our students’ success.
I could be the greatest teacher in the world, but if a student in my class chooses not to take my class seriously, if their parent can’t guarantee that their child attends school regularly or if our elected officials don’t consider students’ needs when making education policy decisions, the impact I have on my students could be neutralized.
School finance reform might not be an ideal topic for party conversation, but it is a very real concern for Ohio’s more than 600 traditional school districts. Ohio is the only state in the nation without a permanent school funding plan, and recent news reports indicate we won’t see a new one proposed until next year. Whatever plan is proposed must put students at the forefront and invest in classroom priorities that build the foundation for learning.
If we as a state are able to ensure that parents, teachers, students and elected officials are working together and truly make it a priority to invest in whatever is necessary to build the foundation for our students’ learning, there’s just one thing left that could hold our students back. We need to make sure that there is a caring, committed and qualified teacher in each of our students’ classrooms. Just because I stayed at a Holiday Inn last night doesn’t mean I can safely practice medicine, nor does attending a six-week training course impart the skills necessary to be a teacher to those who want to be one for a few years.
As a state, we’ve got a lot of work to do, and we can do it the right way or the wrong way. The right way to do it is by working together and putting our students’ success first and foremost.
By Phil Hayes, Columbus Education Association
March 2012 Ohio Schools
- IN THIS ISSUE
- Attendees at the fourth annual OEA collective bargaining conference prepare to rebuild union power as collective bargaining advocates
- Putting Our Power to Work
- Legislative update, Association news, and more
Moved recently? Contact the OEA Member Hotline to update the address on file at 1-844-OEA-Info (1-844-632-4636) or email, membership@ohea.org. Representatives are available Monday-Friday, from 8:30 a.m. to 6 p.m. | OhioSchools — Past Issues
Oh Yes, We’re Social — Join the Conversation!
Mentorvention and Student Success
As a Browns fan, I find little excitement in watching the Superbowl. My most compelling reasons to watch the spectacle a few weeks ago were:
- Tom Brady was the quarterback for my playoff fantasy football team.
- Julian Edelman, a fellow Kent State alum, was playing for the Patriots.
- Madonna’s half time show
- The commercials
It ended up that reason number four was the best of all, because it became the basis for my American Literature lesson Monday morning.
I have been teaching literature analysis, contrasting several adolescent short stories with Catcher in the Rye, and I want my students to understand how tone is expressed differently in each work. When I saw Chrysler’s new ad, narrated by Clint Eastwood, I immediately knew how I could use it in my class. The next morning, I showed it to my students and we had a lengthy discussion about how the commercial used images, music and words to create a defiant, determined and hopeful tone. From this engaging discussion, we turned to our attention to the way tone is established in literature.
Even when watching the Superbowl, my classroom is in the forefront of my mind. It’s part of my commitment to connecting with and teaching every student. That commitment is why I get so excited every year to make improvements on my lessons from years past, in order to have make them better and more compelling for my new crop of students. It’s why I spend my summers at teaching conferences and in my classroom. It’s why I spend my Sundays tucked away from my family in the basement office, so I can grade essays and create lesson plans for the week.
My investment of time is not the only way I work to make sure all students succeed. Every year, I examine the novels I teach, considering what pieces of literature are most apt to engage my classes. I found that my tenth graders were not enthusiastic about reading a series of classic novels, like Lord of the Flies and A Separate Peace. Many did not complete the daily reading assignments or show willingness to participate in discussions. I decided to go beyond the classics and merged Jodi Piccoult’s novel Nineteen Minutes into my curriculum. Although I don’t see the novel as an outstanding piece of literature with underlying symbolism, like Lord of the Flies, it deals with bullying, school shootings and interpersonal relationships. It’s a suspenseful, high interest book, which kids understand, and can easily relate to. It makes students reflect on the way they treat each other.
Along with the new literature, I created a real-world connection, by showing clips of various news clips about bullying incidents, and had students write reaction journals, in class, about the stories. These assignments were graded on completion, and by giving class time, I was assured that all students would do the assignment.
This unit was a success, and what was most rewarding was that I had several students admit that Nineteen Minutes was the first assigned novel they had ever finished.
The place where my commitment to student success is most evident is in my mentoring. If students aren’t feeling emotionally balanced at school or have anxieties and unresolved issues, it doesn’t matter how good my or my colleagues’ instruction is, because students are not focused on class. For that reason I teamed up with colleagues and we created a program we coined “Mentorvention.” We encouraged staff members to reach out to students who seemed to be in need of a positive adult relationship or who were struggling academically. We planned a connections day, where we invited all the mentored students to a day-long workshop where they had opportunities to talk with their peers and adults outside the school setting. Students let down their guard, shared personal stories and even shed some tears. Their connection to their teachers, peers and school, grew and, for some, grades and behavior improved.
Beyond the formal mentoring program, I try to foster strong connections with my students by sharing personal stories about my family or my own high school experience, usually at the beginning of class. It gets the students attention and makes them more willing to talk about their lives in class and in writing assignments.
Making the commitment to helping all my students succeed is one that takes energy, creativity and time. It may mean giving up a Sunday afternoon to grade papers and make lesson plans, but it’s a far more rewarding way to spend an afternoon than watching the Browns.
By Dan Greenberg, Sylvania Education Association