A new independent analysis has found that learning loss among Hawaii’s public school students during the COVID-19 pandemic has ranged from moderate to severe, and the state will need a “multiyear effort” to catch them up to where they should be, a consultant told state Department of Education officials on Wednesday.

The analysis by the Dover, N.H.-based Center for Assessment found that the impact of the pandemic on students’ performance in English language arts was “mostly moderate to large,” and the impact on their mathematics was “mostly large to severe,” state Schools Superintendent Keith Hayashi wrote in a memo that summarizes the report.

However, the good news is that the results of Hawaii’s recovery efforts so far have been “exceptional” compared with those in some other states, Damian Betebenner, senior associate at the Center for Assessment, said in a Honolulu Star-Advertiser interview Wednesday.

“Of the dozen or so states that I’ve looked at … the recovery, or the rates of learning, that Hawaii is demonstrating this most recent year are the most impressive I’ve seen,” Betebenner said.

Betebenner is in Hawaii to present the center’s preliminary findings to state public school leaders. He discussed his findings with top state Department of Education administrators on Wednesday, and is scheduled to participate in a presentation during a special meeting of the state Board of Education at 11 a.m. today at the fourth-floor boardroom in the Queen Liliuokalani Building.

The board hired the center to conduct an analysis of the impact of the pandemic on academic performance and the extent to which the Hawaii DOE’s recovery efforts are bringing about intended results. The DOE has been allotted the lion’s share of the state’s Education Stabilization Fund federal funds to address pandemic learning loss, with about $639.5 million total in Elementary and Secondary School Emergency Relief funds, also known as ESSER.

The center says on its website that it works to “help state and district education leaders design, implement, and evaluate assessment and accountability practices in order to improve student learning and increase equity in important educational experiences and outcomes.” A DOE spokesperson did not immediately know the cost of the study.

According to Hayashi’s memo, Betebenner said the COVID-19 pandemic “caused the most significant educational upheaval in the history of the United States. Even after three years, students and educators continue to experience the effects of the pandemic in various aspects, including academics, emotions, physical well-being, and finances.”

Hawaii students’ performance on the 2022 Smarter Balanced Assessment was scrutinized in the study. Commonly known as the SBA, the mandatory assessments are traditionally given annually to Hawaii public school students in grades three-eight and 11, and are designed to measure whether students are “on track” for readiness in college and/or career.

The latest round of SBA results indicated that Hawaii students are starting to climb out of the pandemic slump, but are still lagging behind pre-pandemic levels — which were already lower than the DOE wanted.

Hawaii students proficient in math had been at 43% in the last year of testing before the pandemic started, plummeting to 32% during the worst of the pandemic, then rising to 38% in the 2021-2022 school year.

Students proficient in English language arts had been at 54% before the pandemic, sliding to 50% in 2020-21, then inching up to 52% last school year.

The report said that “Hawaii students learned at a faster rate post-pandemic than they did before the pandemic,” and that “mathematics needs a steeper recovery than English language arts.”

Betebenner said the data suggests Hawaii public schools have been effective at increasing the “speed” at which Hawaii students are learning.

But he recommended viewing the 2022 results as a “first installment” and added, “the state (and) stakeholders should be looking at this being a multiyear effort to catch those students back up.”

There was no single “silver bullet” in a particular strategy or program, Betebenner said, but schools that maintained stability in their faculty and staff, and emphasized their existing core strengths rather than shifting widely, tended to do better. The center is still conducting case studies of schools with especially strong performances amid the pandemic.

Other study findings include that academic recovery during a crisis is stronger when school leaders use a more decisive leadership style, and that “strong leaders” at schools have found ways to get needed funds for their students’ challenges in the pandemic, Hayashi wrote. “This finding highlights the importance of resourcefulness and creativity in securing funds for necessary programs and services during a crisis.”

Cheri Nakamura of Hui for Excellence in Education, a statewide coalition of stakeholders committed to improving public education in Hawaii, said the DOE and its principals, teachers and staff deserve credit for “their hard work and dedication to lift up our students during and post-pandemic.”

However, broader test data stretching from 2014-2022 “show a flat line for meeting standards prior to the pandemic, a dip during the pandemic, and a recovery after the pandemic, but not to the levels of pre-pandemic,” Nakamura said in an emailed request for comment. “It also shows that the achievement gap between non-high needs and high-needs students is persistent throughout the period.

“In sum, the system has not improved during these eight years,” Nakamura said. “We are concerned about the lack of progress prior to pandemic and what DOE will do differently to change the trajectory going forward.”