Gates Foundation study We’ve figured out what makes a good teacher

Even as most of the nation’s 15,000 public school districts roll out new systems to evaluate teachers, many are still struggling with a central question: What’s the best way to identify an effective educator?

After a three-year, $45 million research project, the Bill and Melinda Gates Foundation believes it has some answers.

The most reliable way to evaluate teachers is to use a three-pronged approach built on student test scores, classroom observations by multiple reviewers and teacher evaluations from students themselves, the foundation found.

“We identified groups of teachers who caused students to learn more,” said Thomas J. Kane, a professor at the Harvard Graduate School of Education and principal investigator of the Gates study, also known as the Measures of Effective Teaching project.

The findings released Tuesday involved an analysis of about 3,000 teachers and their students in Charlotte; Dallas; Denver; Memphis; New York; Pittsburgh; and Hillsborough County, Fla., which includes Tampa. Researchers were drawn from the Educational Testing Service and several universities, including Harvard, Stanford and the University of Virginia.

The large-scale study is the first to demonstrate that it is possible to identify great teaching, the foundation said.

Researchers videotaped 3,000 participating teachers and experts analyzed their classroom performance. They also ranked the teachers using a statistical model known as value-added modeling, which calculates how much an educator has helped students learn based on their academic performance over time. And finally, the researchers surveyed the students, who turned out to be reliable judges of their teacher’s abilities, Kane said.

They used all that data to identify teachers who seemed effective. And then they randomly assigned students to those teachers for an academic year.

The teachers who seemed to be effective were, in fact, able to repeat those successes with different students in different years, the researchers found. Their students not only scored well on standardized exams but also were able to handle more complicated tests of their conceptual math knowledge and reading and writing abilities.

Researchers found that multiple classroom observations of teachers by several people — a principal, a peer, an outside expert — result in the most accurate assessments. Many school districts currently rely on observations by just one person, usually a principal.

The Gates Foundation hopes that states and school districts will use the research to create evaluation systems to help teachers improve, not just in hiring and firing decisions, said Vicki Phillips, who directs its college-ready education programs in the United States.

Denver is already doing so, said Tom Boasberg, superintendent of the Denver public schools. “There’s not some clear dividing line in the middle, with some folks on one side who are clearly not effective teachers and some on the other who are clearly effective,” he said. “You have a lot of folks in the middle who want to get better. The key is to use multiple measures and feedback to help them get better in this enormously complex job.”

For decades, teacher evaluations were little more than a formality in most school systems, with most educators getting top ratings based on little more than a principal’s checklist. Tenure, rather than student achievement, largely determined whether a teacher was rehired at the end of a school year.

But reformers have been pressing for evaluations that judge teachers at least in part on how well their students perform on tests. The Obama administration has accelerated that change by requiring states to adopt such evaluation systems to compete for Race to the Top funds or to receive waivers from No Child Left Behind, the federal education law.

Critics have said that some of the new evaluation systems place an unhealthy emphasis on test scores.

Randi Weingarten, president of the American Federation of Teachers, said the findings from the Gates study “reinforce the importance of evaluating teachers based on a balance of multiple measures of teaching effectiveness, in contrast to the limitations of focusing on student test scores, value-added scores or any other single measure.”

Teacher Quality Not Linked to Advanced Degrees or Experience

An advanced degree and multiple years of teaching experience don’t necessarily make for a better teacher. A new study from the Tennessee Department of Education shows that teachers with these qualities, which traditionally increase teacher salaries, do not increase what children learn over teachers without them.

“Previous research has consistently shown there is little to no correlation between teacher graduate degrees and effectiveness” as measured by what children learn in a school year, said Kelli Gauthier, a Tennessee Department of Education spokeswoman. ”Similarly, research has shown that teacher effectiveness is not correlated with experience after the first five years in the classroom. This study reinforces those results. We have highly effective teachers who have master’s degrees and highly effective teachers who do not. We have highly effective teachers with many years of classroom experience and highly effective teachers with relatively few years in the classroom.”

This study is “important news,” said John Chubb, a Hoover Institution fellow and author of The Best Teachers in the World, because, unlike economists, education authorities have hardly studied the relationship between teacher quality and inputs like degrees and experience.

“At a time when school districts are struggling to make ends meet with fewer state and local tax revenues, it is especially important that districts ensure their compensation systems are aligned with the goal of student achievement, and therefore reward the most successful teachers—regardless of degrees or experience—and not pay based on factors that do not predict student success,” he said.

Skewed Incentives
The common practice of tying salary hikes to advanced degrees and extra college credits means “teachers tend to take the fastest, cheapest route to earning a master’s degree,” said Kate Walsh, president of the National Council on Teacher Quality. “A far better system would be one which rewards teachers for being effective, with teachers themselves identifying the specific coursework that would help them become more effective.”

Salary schedules also push bright young people away from teaching, Walsh said, because they are confident in their abilities and want to earn more for high performance rather than receive “relatively small incremental raises over the next 25 years.”

As of 2012, 20 states required student achievement to count as a significant or the most significant factor in judging teacher performance, according to NCTQ.

The study and body of related research emphasizes the importance of Tennessee finding ways to select, reward, and retain teachers based on effectiveness rather than meaningless characteristics,

Gauthier said. It also suggests the state needs to support current teachers better and help improve teacher performance over time, she said.

Chubb is not sure whether the study will make much difference in state policy.

“Years of academic research have not persuaded policymakers to alter compensation practices; here’s hoping a study from the Tennessee Department of Education will,” says Chubb.

Putting the education in educational apps

New apps developed for children come online every day and many of them are marketed or labeled as “educational” — but how can we tell which of these thousands of apps will actually help children learn? A comprehensive new report published in Psychological Science in the Public Interest, a journal of the Association for Psychological Science, integrates research from scientific disciplines like psychological science, linguistics, and neuroscience to provide an evidence-based guide that parents, educators, and app designers alike can use to evaluate the quality of so-called “educational” apps.

Since the iPad was introduced just five years ago, over 80,000 educational apps have become available in the Apple app store, which means apps are being developed far faster than the scientific community can evaluate them, say report authors Kathy Hirsh-Pasek (Temple University), Jennifer Zosh (Penn State University, Brandywine), Roberta Michnick Golinkoff (University of Delaware), James H. Gray (Sesame Workshop), Michael B. Robb (Fred Rogers Center at Saint Vincent College), and Jordy Kaufman (Swinburne University of Technology).

The full report and accompanying commentary by communications researcher Ellen Wartella (Northwestern University) are available free to the public online.

While scientific research examining specific features of individual apps may be scarce, scientists have amassed a wealth of knowledge about how children learn and this knowledge is directly applicable to the assessment of new forms of digital media, including apps, the authors say.

In their report, Hirsh-Pasek, Zosh, Golinkoff and colleagues present a comprehensive review of research from many disciplines related to the science of learning, offering a set of four evidence-based principles that can be used as guide, both by developers creating new products and by parents hoping to find high quality games for their children.

The researchers conclude that “educational” apps best support learning when they are:

  • Active in a way that requires mental effort and not just swiping
  • Engaging, not distracting
  • Meaningful in the context of a child’s life
  • Socially interactive because children learn best with others including parents and peers

“These four principles can help us distinguish apps that masquerade as educational from those most likely to engage children in an educationally meaningful experience,” the researchers say. While not all of these principles are necessary, the more an app promotes these types of learning experiences, the greater the educational value of the app will likely be.

The researchers point out that many apps feature content that seems educational, like letters or numbers, but this doesn’t mean that they have true educational value. For example, studies have shown that e-storybooks that contain lots of “bells and whistles” can distract young children from attending to and learning from the actual story. These findings suggest that apps that feature attention-grabbing sound, movement, or other visual elements may be more distracting than they are engaging, and are unlikely to serve as effective educational tools.

“There are a number of great apps in the marketplace,” the researchers note. “One of the major reasons we sought to complete this project was to empower a much wider audience to be educated consumers and developers of apps. Since we embarked on this project, we have been approached by a number of the leading app developers who seem excited to include more of the scientifically informed processes in their product lines.”

With the science of learning at their core, Hirsh-Pasek, Zosh, Golinkoff, and colleagues believe that the next generation of apps will be able to fully realize their potential as effective and engaging educational tools.

Elementary Students Held Back In Early Grades Often Do Not Get Special Education Plan

Many children who are retained in kindergarten, first or third grade for academic reasons do not subsequently receive a document outlining the individualized special education services they should receive, according to a new report.

Each year, 5 percent to 10 percent of American students are retained at the same grade level, according to background information in the article. One in 10 students age 16 to 19 have repeated a grade. “Some of these students may require special education services at the time they are retained, in subsequent years or both,” the authors write. “One approach to supporting a child with low academic achievement is the provision of speciala education services, as indicated in an Individualized Education Program (IEP). An IEP is a legally binding document describing a child’s special education services and is developed after the child has undergone a special evaluation and has been determined eligible for services.”

Eligibility for an IEP varies from state to state, but under the Individuals With Disabilities Education Act, every American child has the right to an evaluation. Michael Silverstein, M.D., M.P.H., of Boston Medical Center, and colleagues studied 380 children nationwide who were retained in elementary school for academic reasons (300 in kindergarten or first grade and 80 in third grade). The children were followed up through fifth grade.

Of the children retained in kindergarten or first grade, 40 (12.9 percent) had an IEP on record during the year they were held back, 60 (18.2 percent) received an IEP in the next one to five years and 210 (68.9 percent) never received an IEP. Twenty (18.9 percent) of the third-graders had an IEP during or before the year they were retained, 10 (8.8 percent) received one in the next one to two years and 60 (72.3 percent) never received one.

Children retained in kindergarten and first grade were less likely to have an IEP if they had a high socioeconomic status or lived in the suburbs rather than rural areas. “Among kindergarten/first grade retainees with persistently low academic achievement in math and reading, as assessed by standardized testing, 38.2 percent and 29.7 percent, respectively, never received an IEP,” the authors write.

“Although debates about the value of grade retention abound, the practice, in and of itself, has never been demonstrated to be an effective intervention relative to subsequent academic achievement or socioemotional adjustment,” the authors write. “Therefore, some experts in the field believe that retention should be accompanied by focused individualized assessments of children’s special education needs. Although our results do not definitely demonstrate that retained children have been denied their rights to such assessments, they raise the question of whether the potential special education needs of retained children, particularly those who demonstrate persistent academic difficulties, are being addressed consistently.”

This study was supported by a training grant from the Maternal Child Health Bureau.