Report: In Some Urban Districts, Science of Reading Limits ‘Robust Comprehension’

Four school districts in major urban areas using the science of reading found while students are grasping basic literacy skills, limitations toward deeper comprehension still exist, according to a new study.

The “Robust Reading Comprehension” report, conducted by nonprofit research organization SRI, examined literacy instruction in districts in Texas, Maryland, North Carolina and Virginia that have been using materials rooted in the popular phonics-based literacy approach for at least five years. 

Through numerous classroom observations, teacher surveys and interviews with district officials in Aldine Independent School District, Baltimore City Public Schools, Guilford County Schools and Richmond Public Schools, researchers found a majority of reading lessons lacked “depth” – meaning foundational skills were mainly limited to working on single words rather than reading them in sentences. 

Get stories like this delivered straight to your inbox. Sign up for The 74 Newsletter

Comprehension lessons in later elementary grades also mainly focused on completing a task, such as identifying a main character, rather than using a text for discussion and understanding its purpose.

“You’re not able to really think about the unpacking of a complicated sentence. You’re not thinking about really intentional vocabulary instruction or the building of kids’ word knowledge over time,” said Dan Reynolds, one of the lead authors of the report. “Ultimately, how should we be framing kids to read? Are we teaching our K-4 kids that reading is just tasks? Are we teaching them that they just need to label stuff and fill out graphic organizers?”

In recent years, nearly every state has passed science of reading laws, including many that have limited the type of programming and instructional materials a school can use – a move that has drawn some criticism that it’s too restrictive and that the instruction faces its own limitations.

The report defined surface literacy skills as a student’s ability to complete tasks and understand texts based on their literal meeting while robust instruction would further push a child to understand, evaluate and synthesize what they had read for its significance. 

The study said its “comprehension observations alone are more rigorous than nearly all studies conducted in the last 50 years.” It’s not expected to be representative of reading instruction across the country, Reynolds said, but “we have four big districts in four different states, and we saw this pattern happening in all four of them with three different curricula.”

The study also found that teachers struggled with implementing comprehension-focused learning materials and said many times the curriculum was too dense, required substantial planning or may not have been developmentally appropriate. Professional development opportunities for these educators were also limited.

Researchers reported less than a quarter of observed comprehension lessons were engaging in robust learning. More than two-thirds of the lessons focused on “surface-level” comprehension. 

“It seems that these curriculums are designed to build knowledge and they don’t develop meaning, and so then why read about the Civil War or about insects?” said Katrina Woodworth, director at SRI’s Center for Education Research & Improvement. “The point is to both teach reading and to build students’ knowledge base so that they have more scaffolding for future learning of both content and meaning.” 

The SRI researchers also found that many review tools that measure comprehension don’t make a distinction between surface-level and robust instruction and skills. So, while educators are tasked with meeting a baseline standard, like having a child compare and contrast a text, it may be “unintentionally encouraging teachers to focus on surface-level goals,” the report said.

Without distinction, it weakens instruction for students and can later manifest as a skills disadvantage, Reynolds said.

“Districts had done so much to get the kids all the way there [with literacy], but it was losing voltage in the end,” Reynolds said. “If we can actually shift the way that districts are thinking about improving their comprehension instruction, they can take that all the way home and deliver really high quality comprehension instruction because so many pieces are already in place.”

Reynolds and one of his fellow co-authors, Sara Rutherford-Quach, said they saw glimpses of “magic” in the classroom when students understood a passage in wide-ranging contexts, which is the type of instruction they’re hoping to see districts incorporate more of in early grades.

“The kids were way more engaged,” Rutherford-Quach said. “Surface-level is important and necessary in some cases, … but it really is fundamentally different when you start talking about meaning and making it matter to the kids, and you see that they’re invested in it.”

Reynolds added that it’s unlikely robust comprehension could make up 100% of lessons in the classroom, but “we are thinking that if we can shift that needle from 24% robust lessons up to 50 or 60, then that would be a real catalyst for comprehension growth.”

The report recommended district leaders create “a shared vision for robust comprehension and define what it means for students, teachers, schools and the district,” and align how to best measure the extent of learning. It also called for better professional learning structures that could help model and rehearse robust comprehension work. 

Previous reporting from The 74 found the percentage of recent high school graduates who lack “robust” comprehension skills is the highest it’s ever been, according to 2023 data. The sooner districts can engrain literacy skills that go beyond just explicit tasks, the easier it will be as they continue through the K-12 system, Reynolds said.

“I see the distinction between surface level and robust comprehension as critical to comprehension in fifth grade, but I also see it in the kids when they’re in 12th grade. Surface level comprehension and robust comprehension is the difference between a two on the AP exam and a three,” he said.