Pandemic Impacts on Mental, Physical and Educational Outcomes for American Children: Session One

REBECCA JACK: Hi, everyone. Thank you so much for having me. I'm truly honored to be here and to be in this room. And thank you to Dr. Harris for such a wonderful introduction to the day. I hope to continue some of those themes as I talk about my work.

I'm Rebecca Jack. I'm an economist and a PhD candidate at the University of Nebraska-Lincoln, and I'm presenting my work on schooling mode and test scores, which I did along with Claire Halloran, James Okun, and Emily Oster as part of the COVID-19 School Data Hub.

And so, to start off, I want to set a big picture and just show what happened. This is the data piece. This will walk us through what children experienced during the pandemic in terms of schooling. And then I will go into our results about test scores.

And so, to set the stage, we all remember this. In March 2020, schools closed immediately in response to the onset of the pandemic. And I think we all initially thought this was going to be a few weeks. We'll go back. For our local schools, it was spring break, so it felt just, OK, this is an extended spring break, and then we'll go back, and it'll be fine. But, instead, schools remain closed. And so for a lot of students, this was all of those big end-of-the- year activities that they were looking forward to and proms, graduations, kindergarten graduations, all of those things, suddenly they weren't able to have or at least have in the same way they would have otherwise.

And then we entered summer 2020. And throughout that entire period, schools and districts were faced with this very immediate policy decision of what they were going to do in the fall. And they had to decide with very little information, very little data at that point in time, what they were going to do for the next school year.

So we saw, throughout that summer, a wide variety of plans. And I'm going to bucket those into three things. But each of these really contain a large variety of things that schools implemented. So the three buckets we're going to talk about are the schools that opted to do fully virtual education. Those are the ones that did not reopen the school buildings and had schooling online, on Zoom or other platforms. And sometimes, this varied by the age of the student. But largely, these are the schools that did not reopen.

And then you had some schools that implemented some hybrid methods. And so these could look like half-days, where you had some of the students attend in the morning and a different group of students attend in the afternoon in order to try to mitigate spread by keeping these groups of students separate. Or some had partial days, where some groups of students would go Monday/Wednesday, and other groups of students would go Tuesday/Thursday or something like that. But all of these plans were a way to let students have some in-person schooling while keeping students separate to mitigate spread.

And then the final group of schools are the schools that opened in-person, which you can think of as more or less normal. But we know nothing during this time was normal. And so, usually, these opened with a virtual option, so parents could opt out of school if they wanted for their kids. And they typically had other mitigation policies in place, like masking, again, distancing, social distancing, and then pretty strong exclusion policies if the child had COVID or someone in their family had COVID.

And our favorite COVID word-- school closures were unprecedented, and all of these decisions were made without knowing any of the short or long-run consequences, and these are beyond those immediate health consequences, which were very important at the time, both for the students and their families. But beyond that, we didn't know the consequences in terms of students' education, their mental health and well-being, the impact on parents, their ability to work, childcare, all of these things. All these decisions were made without knowing any of these consequences.

And, at the time, there was really no data on what was happening. And so, in response to the need for information, our group, the COVID-19 School Data Hub, collected information on the policies that schools were implementing. So we collected this at the school district level to make it manageable from state education agencies. And we built a publicly accessible database of school district policy decisions during the 2020 to 2021 school year.

And I really want to emphasize policy here. This is not a measure of what families were opting into or out of. And, in most places, parents did have a choice to opt in or out. But this is the policy that was set by the school district.

And so what did that look like? You can see, we have data from most states. There's a handful of missing. This is September 2020. The dark blue are the school districts that opted to in-person education, and the yellow districts are the ones that opted into hybrid. And the red districts are the ones that opted into virtual schooling. So this is a picture of what that opening of schools looked like in fall 2020 after that summer. And you can see quite a bit of variety in what districts chose. The coasts especially largely opted into virtual education or some hybrid. And the middle of the country and Florida down there were all largely in-person.

And then we can track this over time, which gives us this really interesting picture of what students experience throughout the year because another important point is that this is not a one-shot policy decision. Schools were changing their policies over time depending on what was going on in their area.

So we can look at October, November, and then if you remember, in 2020, there was that large spike in COVID cases in December, and you can see that pretty clearly in the map here. A lot of districts switched to virtual or hybrid at that time during that spike. And then, moving into January, that spike was kind of dying down, and we had more movement back into hybrid and some in-person.

And now, leading into the spring, you can see, slowly, school districts reopening across the nation with some hybrid remaining and a few pockets of virtual school. But comparing that to where we were at September 2020, there was a lot of change over this school year for students, a lot of uncertainty. Students had just a wide variety of experiences during this year.

So the first question that we ask with this data was how did district-level schooling mode impact educational outcomes? And, again, this is only one of many, many outcomes that matter for students, and we will hear about a lot of those today. But educational outcomes are an area where we do have existing data that we can look at. And so that's where we started.

Specifically, we studied state standardized test pass rates for grades 3 through 8 in both English learning arts and math. Those are because grades 3 through 8 are pretty consistent across states, the grades that are tested, as well as English and math are subjects that are widely tested across all states.

And now we were able to include 11 states in this analysis at the time that we wrote this based on these following requirements here. So first, spring 2021 test data was available at the time of writing this paper. Now most states have it available, but at the time, there were only a few. We also required them to have at least two prepandemic years of test score data available. That way, we were able to compare the districts from pre-COVID period to what we see post-COVID, which, well, it's not really post-COVID, but in the 2021 year.

We also require that they had no significant changes to the test content, test structure, or score cutoffs during this period so that we can make those direct comparisons between pass rates before and after. And then, finally, we had the requirement that they did not have excessively low participation on the spring 2021 test.

So, as you might expect, participation went down on these tests of across the board. Again, not all students were even in the building at this point in time. And so participation wasn't what it would have been. But some states were excessively low for a variety of reasons. They had different policies in their state. Some of these states had 40% participation or 60% participation, and it was just too low to make any kind of accurate comparison.

So this is to give you an idea of which states are included in the analysis. You can see that we cover most of the country, most of the regions in the country. We're definitely missing the West Coast over there. But for what we had, we have pretty decent coverage of states based on those requirements.

We have a few additional sources of data that we use here. So the first is we pulled school district-level demographics by the share of enrolled students from the National Center of Education Statistics. And all of those demographics are taken prior to COVID, so from 2019 generally, because these things will also change during COVID. These include shares of race and ethnicity, free and reduced price lunch, and English language learner status to get a picture of the student population in each school district that we're studying. In addition, we have county-level COVID-19 case counts from USA Facts. Different areas of the country experienced COVID spikes at different times, and so we wanted to make sure that we had some information about what they were experiencing locally in terms of COVID levels. So we include that data as well.

We also have county-level unemployment shares, and that's to help us capture some of the economic conditions. This is from the Bureau of Labor Statistics. And as we know, COVID impacted everything, including employment, and so we also wanted to control for that because kids, parents employment will impact their outcomes as well. And then, finally, we include county-level 2020 election vote shares. That's to help us measure the political leanings of the area because it did influence the policies that the local districts were opting into. And so we want to make sure we have information on all of these things that impacted what decisions school districts were making.

So now to give you a picture of the states, specifically in our data set-- you can think back to the map that I showed you before-- these are the number of school districts in each state that we study and the average number of years of test score data that we were able to include for each state. Most of them, we have close to five years, but that's not true of all of them. And then the breakdowns of the shares of time spent in person, hybrid, and virtual-- and you can see, right off the bat, that there's quite a bit of variation in what students experienced based on where they lived, similar to the map that I showed before.

You can look at Virginia down there-- 9.7% of the time spent in person. Compare that to Wyoming, where 86.5% of the time was spent in person. So, again, students had vastly different experiences based on where they lived during COVID.

And I also want to show you the demographic breakdowns by state here to show that we're also looking at a wide variety of different student populations here. This is back to Dr. Harris's point of disaggregating the data. These are different groups of students that we're looking at. And so there's a variety in the racial breakdowns of the districts, the share of free and reduced price lunch, share of English language learners, depending on the state that we're looking at.

And so just looking at the trends, right off the bat-- so these are the test score declines across different groups. And we're going to walk through this graph. So what I've plotted here are the changes in pass rates year to year. So we're comparing, for example, spring 2019 and calculating the change from 2018, or spring 2018 and calculating the change from 2017. And then we also have that COVID year, spring 2021, where we're calculating the change from 2019 because there weren't test scores in 2020. And even if they were, those would have been impacted by COVID. So that's the comparison pre-COVID to the COVID year.

So those open circles, the light gray open circles, are all of the changes in pass rates prior to COVID. And you can see those float around 0. So year to year, there are not a lot of changes in pass rates for students in the school district year to year before COVID. And then the dark gray circles are the COVID year, and every single group is negative. So it's very clear off the bat that every single student was impacted here educationally. Test scores dropped for everybody, no matter what they experienced, no matter where they lived.

To give you some numbers here-- so look at that top row, the top dots there, the overall. Everyone on average-- test scores decreased by an average of 12.8 percentage points in math and 6.8 percentage points in English language arts, which is substantial, very large. And you compare those, again, to those the changes pre-COVID, the open circles and the overall, which float solely around 0. There's typically no change.

We see a lot of variation by state, similar reflecting what we saw with the in-person and virtual and hybrid-- again, the location matters a lot, a lot of variation there by state. Virginia-- the outlier. And in-person that I point out before is also an outlier in those math test scores.

And then, right off the bat with the trends, there's this clear pattern that less time spent in person led to lower test scores. So that's that in-person bucket there on the graph. And the 0 to 25 group is the group that spent very little time in person. And the 75 to 100 group is the group that spent a lot of time in person during that school year. And in math, especially, you can see that just really stark trend where those who spent little time in person had the largest test score drops, and those who spent more time in person had the smallest test score drops. And that's also true in English language arts, although not quite as stark as math.

And if you look at demographic patterns too, we can see, again, that not everyone had the same experience during COVID. You can see those school districts with large shares of Black students-- so that's the high group there, the bottom one-- they also had the largest drops in test scores. And they were also the group of students who were most likely to not have in-person schooling available to them. And we see similar patterns with Hispanic students, free and reduced price lunch students, and English language learner students.

And so all of these groups that are potentially historically underserved are also the groups who saw the biggest impacts during COVID. And these disparities are just a really important part moving forward and talking about policy moving forward that not all kids were impacted the same during COVID with school closures. And it's important to understand who was most impacted.

And then the goal of our paper was to isolate the impact of these school closures on test scores, controlling for everything else. So I'm going to walk through that briefly here. But the way that we did that is we just estimated a standard panel regression. We do that separately for math and ELA. And so we are going to measure the impact of the percent of time spent in person and the percent of time spent in hybrid schooling, and compare that to the percent of time spent in virtual schooling to get those estimates of the impact of school closures, essentially, on students.

And then we want to be able to look within a small geographic location so that we're making these comparisons among districts that look pretty similar to each other. So in order to do that, we do three separate things. We either compare districts within a state or within a commuting zone, so those urban areas that are close to each other, or within county. And so, in the paper, we have results for all of those.

Today, the results that I'm going to highlight are the ones for the commuting zones specifically, but they all look pretty similar. And so we also control for year in school district fixed effects-- all of this to say we are trying to compare districts that look very similar in everything except for schooling mode so that we can really isolate what did that schooling mode do to test scores, controlling for everything else.

And, in addition to that, and for the same reason, we control for the school district demographic measures, COVID-19 rates, unemployment rates, and political leaning of the area. So again, all of this is to make those comparisons strictly about schooling mode, comparing students who were in similar districts to each other.

And you can see that the findings are quite stark, quite large. So, again, with the trends, we saw everyone decrease. But school districts that had full in-person schooling available to their students had significantly smaller declines compared to those who only had virtual schooling available to their students. That difference is 13.4 percentage points in math and 8.3 percentage points in English/language arts, which is quite substantial differences in these test scores.

We also found something similar with hybrid schooling. It's not quite as strong, which is what you would expect. They had some time in person, but not as much as those schools that had full in-person. And so, again, the school districts with hybrid schooling available had smaller declines compared to those with only virtual schooling available. The differences are 7.2 percentage points in math and 5.4 percentage points in English/language arts. So, again, alone, already quite large-- not as large as the in-person.

We do the same thing, estimating each grade level separately as well. And we don't have as much power for those calculations. I'm not including the numbers, but I wanted to highlight that the results were quite similar, and we do see at least a suggestion of stronger impacts in the younger grades, which also not surprising-- you would think that a third grader on Zoom and an eighth grader on Zoom would have very different experiences, and the older students might be a little bit more equipped to handle online education compared to the much younger students.

We also run it separately, interacting with the demographic variables. So we can tease apart those differences for these different demographic groups, and the group that really stands out when we're doing this is those school districts with larger-proportion populations of Black students. And we find the impact of in-person schooling to be even larger for those students-- so, again, highlighting the disparities that existed during this time and the differential impact on different populations of students.

I want to highlight a few additional impacts and considerations before I conclude that I think are very policy relevant moving forward. So one thing that we controlled for in our regressions but we saw pretty clearly in the data is that school enrollment and test participation both dropped in the 2020 to 2021 school year. Again, not surprising but, like I said, I think, very policy relevant.

In terms of test participation-- declined more for historically underserved students, including students of lower SES backgrounds, students of color, and ELL students. And so these results you can think of in that context where some of these students who we would really like to know information about are the ones that were not tested that year for a variety of reasons. And so you could almost think of our results as this lower bound in what students experienced.

And I think, very importantly, again, talking about policy moving forward, public school enrollment declined during this period. And, as we're watching, it hasn't fully recovered. And it's kind of unclear where these students went in some cases. This was especially large for kindergartners, which we don't have test scores for kindergartners, so they're not in the study. But there was a lot of delaying school entry for students. There was a lot of switching to homeschool or to private school. And so as we look forward and as we move forward, there is a real need to understand what happened to enrollment, where these students went, where they are now, are they going to come back to the public school system? There's a lot of open questions here, I think, in terms of policy.

And I also always like to emphasize that standardized test scores are only one small piece of the puzzle, both educationally and in terms of all the impacts that students experience during this time. It's important and it's useful because we have information on it, we have data on it, and we can measure it. But it is a small piece, and I think that's part of why we're here today, is to talk about all of these other impacts that students experience that are very important.

So, just to summarize, it's very clear that all students experienced test score declines during this period of time. And, again, it's what we would expect. This was a time of upheaval. Students lives were not consistent. You wouldn't expect them to not have test score declines. But students who experienced less in-person schooling in the 2020 to 2021 school year had larger decreases in standardized test scores quite significantly.

And there were clear disparities in what students experienced, both the schooling mode that they had available to them-- for example, Black students were less likely to have in-person school as an option-- and in their test scores as a result of that. And I think it's clear that, in terms of these test scores and educational outcomes, virtual schooling was not a good substitute for in-person schooling. I think that's clear.

This is not a statement about what we should have done or what we might potentially do in the future-- hopefully we never experience this again, but if we do. But it is clear that educationally, virtual schooling is not a good substitute for in-person schooling. And so moving forward, talking about policy, we really need to focus policy on those students who had the longest school closures during this period of time. They were the ones who were most impacted by the policy decisions during COVID.

And, finally, the last policy piece that I really want to highlight is that it is really important to have good data collection systems in place. This is truly vital for real-time policy decisions in times of crisis. To circle back to where I started, when local leaders were making these decisions in real time, there, was very little information about how to do this, first of all, and what the impacts on students would be. And having data collection systems in place make it a lot easier to collect that information and to find closer to real-time answers about what students are experiencing and the impacts on them. And so yeah, one of my most important policy pieces is data collection is very important. And yeah, that's all I have for you today. Thank you so much again for having me.

KRISTEN BROADY: So I want to thank Rebecca Jack for that presentation. And I'm going to ask Sofoklis Goulas, who is a fellow with the Hamilton Project at the Brookings Institution, to join us-- can have a seat at this front table. I have a few questions.

SOFOKLIS GOULAS: [INAUDIBLE]

KRISTEN BROADY: Yeah, go ahead-- after his policy reflection.

SOFOKLIS GOULAS: Thank you. I will be quick. I think Becky covered a lot of ground on the policy implications of this paper. And so I will skip the findings. In broad terms, we see large declines in passing rates in the pandemic year in both math and ELA, and we see lower declines in places that had fully in-person instruction. And we see larger losses among districts that had a higher share of Black and Hispanic students.

I want to jump directly to the policy implications, and I think I will pick up exactly where Becky left off and the need for data systems, the need for testing, because if we do not test, we do not know how students are doing, how can we address the problem. So, first of all, we need testing. And even with testing in place, we might still-- you will not have full participation in those tests, which means that whatever measures we have right now of learning losses may underestimate the magnitude of the problem because we have, probably, positive selection into testing-- those that have suffered the loss do not even show up to test.

A key policy implication refers to the inequalities because the pandemic was a cost in terms of opportunity. We missed the opportunity to make progress in closing the gaps because learning gaps means that the students did not progress, which means that the gap in achievement became even larger. So this is important to realize that even if there weren't any gaps before, there are now because of COVID. And if there were gaps before, they are even greater now because of COVID.

This means that we need to search for solutions among approaches that accelerate the pace of learning, meaning approaches that help students learn faster within a year, to make sure that they cover more than they would in a normal situation to cover the COVID-related loss, plus any achievement gap we have among racial groups.

And this may be challenging because the school system we currently have may not be able to accelerate too much. So we will talk more about this soon. But I want to emphasize this to foreshadow a little bit.

Now because of the closures and because of the learning losses, we might be seeing, right now, families not being very happy with public schools, meaning that they may be looking for alternatives to traditional public schools specifically. And this translates to the learning losses being part of a bigger puzzle that includes enrollment losses.

So as we are trying to find ways to address the learning losses of students, we need to also address other problems, like the engagement, the friction in the handshake between families and schools because it is not only that the students did not learn that year, it is that they miss the opportunity to make progress in closing achievement gaps. It is that the families were not happy with the way some districts handled the pandemic.

This means also that the policies, that the enrollment losses we see right now, first of all, are not a phenomenon that happens randomly across these districts but may be associated with the experience families had and the communities had during the pandemic and the magnitude of learning losses in those communities.

This was a very quick policy reflection so we can move on to the questions.

KRISTEN BROADY: [INAUDIBLE] Rebecca, join us. And I sent Peter a message a minute ago. I wonder, can you go back to Rebecca's- - there was a slide that I have a question about, or can I do it here? OK, I'm going to say what I want to-- I had some prepared questions, and I'm still going to get to those. But your slide led me to one more that I want to see [INAUDIBLE] Thank you so much, Peter. This part was not planned, but I saw something that I want us to address.

PETER: Remember-- you go ahead, and you can seek for it.

KRISTEN BROADY: So here, I guess I understood the in-person, the hybrid, and the virtual. And I don't know if anybody else saw this or had this question, but I was looking at the closed altogether, and I feel like that changed. So there's one right there, and a couple up there. So this is 2020 to 2021, and then still nothing there. But here, what happened?

REBECCA JACK: That's a good question. I don't know if I have an immediate answer to that.

SPEAKER 1: Depending on what state you're in [INAUDIBLE]. So, for instance, Illinois tends to run a very full year-- further into the year than Indiana does.

KRISTEN BROADY: OK, that is helpful. It was like, I saw that, and I'm thinking, why is that and what does that have to do--

REBECCA JACK: And, I mean, I think we can point out to you that a lot of these are rural schools and probably smaller districts overall, and [INAUDIBLE] these are averaged by month, too. And so this doesn't mean that they were necessarily closed all of May 2021, just majority of it.

KRISTEN BROADY: OK, that is helpful because I'm thinking, like, what is going on? And then I guess that, too, makes me wonder about the seasons and how, when it's warmer or cooler, what was going on with COVID and meals and, yes, that schools in the South are open completely different time periods than those in the North. So that caught my attention.

So I want to start with, do we know why school districts with lower baseline test scores and a higher share of Black or Hispanic students offered less in-person learning during the pandemic? Because it seems like those are the schools that provide even more resources to those students that really need them. So do we know why that happened?

REBECCA JACK: I think one element is that these are the communities that were hardest hit by COVID, and so I think, sometimes, the mitigation measures there were much stronger in order to protect the community as a whole. And some of it was in response to parents and what they wanted and what mitigations they wanted for their kids. But I think there's still open questions about that as well, definitely.

KRISTEN BROADY: So I guess my thought-- I'm going to come to you-- but my thought was that Black and Hispanic people were more likely to be in jobs that were face-to-face, that were high-risk, that are essential workers, putting them at higher risk of getting COVID, and then passing it on to their families and more likely to have multiple families living in the same household. So that's kind of where my mind went. But I wondered if there were other reasons. Yes, sir-- can you introduce yourself and ask your question?

AUDIENCE: Mic, or I can just speak out loud?

KRISTEN BROADY: Well, we do have some people online, so using the mic--

AUDIENCE: Good morning or afternoon, everyone. My name is Kevin [INAUDIBLE]. I am a sole practitioner management consultant, and I have space in the vertical for government and nonprofits. And so I have clients-- part of the reason I'm here is to try and learn more and help them think about the way forward.

To your question, what I found with some of the clients that I've worked with that are trying to through this is the multigenerational household and the essential worker and where they landed economically that kind of drove what ended up happening and what the families were comfortable with. As you could probably imagine, in some households, one parent, single parent maybe had multiple jobs, and the primary caretaker was the grandmother or grandparent or some other type of relative that had a close familial bond who were more adverse and subject to COVID. And so they took those extra precautions. So at least that's what I saw, not in the level of detail that you guys analyzed, but at least consistent themes across my accounts. Thank you.

KRISTEN BROADY: Thank you very much. I think all of this just made me think that you have these students, again, that need these resources more than anyone, and because of all of these reasons, did not have them. I focused on that. And I guess it related to all of the research on labor and how it impacted people long term. But that's a conference for another day. So why do you think average test score losses were larger in math as compared to English language and language arts?

And I guess, for the economist-- I don't know, maybe it's just me, but I feel like it's easier to understand math. I know this is counter to probably everyone else in the world. But when I think about English and learning arts, you need to hear the words, hear the pronunciations, have the conversations with your classmates and with the teachers. So I believe you, obviously. Dr. Harris talked about evidence-- we see the evidence. But why?

REBECCA JACK: I do think as an economist, it does feel like math should be the easier one to-- and, I mean, for my kids, I could support them in math in a very real hands-on way. But I think, in a lot of families, that's not the case. And the way that we talked about it and I think about it, is at home, when you're isolated at home and have nothing else to do, it's probably pretty easy to pick up a book and read and continue reading and at least building some of those skills on your own. But I think there are probably fewer children who will pick up a math workbook, and there are-- they do exist. I was one of them. That's why I'm an economist. But fewer children who will pick up that math workbook and do that on their own. And so I think that's probably part of what helped keep the reading scores up a little bit.

KRISTEN BROADY: I'm going to invite Sofoklis, and then I'm going to go to Dana and then Diane.

SOFOKLIS GOULAS: Of course, it is easier to pick up a book than a book of exercises in algebra. And, for sure, well, English is easier to practice in normal communication than math. So it is understandable that the pandemic would have caused the greater interruption in the progress of students in their math education than in their language skills.

KRISTEN BROADY: That makes sense. Yeah, Dana was--

[INTERPOSING VOICES]

AUDIENCE: I was just going to say something quickly about this point. First of all, this paper is magical. And I could look at these maps all day. I'm so grateful that somebody captured this great data on the policy decisions. What I was going to say on my wish list as I was looking at the more scatter plot graph that you showed was that I wish we could disaggregate it by learning style because I think we know that kids-- some of them learn the way you just described, like, you need to hear it. Some they need to see it. Some they need to experience it.

And I think, depending on how-- I mean, I have two kids with very different learning styles who responded really differently to online learning. I wish we could do it that way because maybe it would tell us more about why the math gaps were bigger. I did just text the 21-year-old and say it was harder to learn-- you weren't wrong. You can't learn calculus from bed. Anyway, sorry, go on.

AUDIENCE: I was going to say that this is actually a really common finding in the education literature. Small classes impact math more. Charter schools impact math more. It seems like maybe reading is more of a function of the stock of all of your inputs over time. And math is more about flow is sort of our best-- but I think we don't really understand why. But it is something that we find time and time again across education interventions. It just seems like math is easier to gain and easier to lose.

AUDIENCE: [INAUDIBLE] in front of the computer. And so you are missing [INAUDIBLE] the teacher passing out the paper or passing out [INAUDIBLE] so we could be as simple as not just having regular tools that you see every day [INAUDIBLE] why we may have seen [INAUDIBLE]

KRISTEN BROADY: That's definitely helpful. So my next question is we hope that there will never be an event like this ever again. But what policies would you recommend to better prepare schools for possible future large-scale schooling disruptions, whatever they may be? And I guess that can also be reflected in changes in technology or whatever else may disrupt education. What did we learn here in terms of learning loss, ways to prevent it, and ways to stop these gaps from happening and continuing?

SOFOKLIS GOULAS: I can start. First of all, I think we're still trying to figure out how to come out of this situation. So it is hard to imagine that we are ready to face the next COVID because we haven't really figured out how to come out of this one. And I feel that everybody is trying to use all the tools we have, all the knowledge we have, all the experience, all the evidence to design proposed policies that mitigate the learning losses.

And I suppose that the COVID situation, like a crisis, drained more places that didn't have some kind of reserves-- reserves in resilience, reserves in leadership, reserves in all kinds of preparations from equipment to protect students and families from the health crisis, preparations to help students get their meals at school, preparations to help students get the materials needed to continue learning, even technology.

So it seems that if we had to face another crisis like COVID in next week, we would try to pile up on things that we felt we missed the most. Do we have the basics for learning? Do we have the basics to keep feeding families? Do we have the basics to stay healthy, to stay resilient, to stay strong mentally and physically? So it feels like we need to create some capacity for crisis in order to be able to deal with crisis.

REBECCA JACK: Yeah, I think along with that, what we've heard already today and we'll talk a lot about today, schools really are a hub in society for a lot of resources beyond just education.

And I think acknowledging that and recognizing that and then utilizing that in times of crisis instead of just maybe shutting them down completely and shutting off access to these, I think we could be more creative about ways to get students in buildings safely and keep some of that social interaction in a safe manner and keep their attachment to these hubs that are really vital for some students in terms of feeding them and sometimes clothing them, all these different things, keeping those attachments stronger in times of crisis and some kind of in-personness in a way that's safe could really help keep everyone at a safer level.

And I think a lot of districts did this, to be clear. And a lot of leaders worked really hard to keep these kinds of structures in place. But I think, in some places, students were kind of lost here and isolated and trying to prevent that for as many students as possible, I think, would be really important moving forward.

KRISTEN BROADY: [INAUDIBLE]

AUDIENCE: Yeah, your resident warrior here-- there will be another pandemic. There will. And so the question is, do we have the will to-- and I think there is a preparedness playbook. We just have to have the will to implement that and make sure that every district, every jurisdiction has a preparedness playbook that works for them. So folks should be thinking about that now. There should be table exercises right now. But we will have another pandemic. We can be prepared.

But look what they've just experienced in certain areas regarding a weather disruption. So I think the key is-- and we know some of those folks are probably going to have to go to virtual learning before they can get the schools rebuilt-- so I think the key is to build up the infrastructure. And it's not an either/or. It's a both.

And there will be weather disruptions, other disruptions, and we want to have the infrastructure for virtual learning. I don't know if it happened here, but in Georgia, broadband-- and in my hometown of West Virginia-- broadband was a huge issue, so they had to deploy buses to neighborhoods so they could even have the capability of virtual learning. So I think those are some of the things that we need to think about as far as our playbook going ahead.

And I just do want to note that there are some kids-- and again, this may be learning styles-- but even some patients that I had who actually did better with virtual learning because they were bullied at school. And so then they didn't have to-- of course, they had, again, all the other advantages. They had broadband. They had a quiet place to study at home. They didn't live in-- most of these folks didn't live in multigenerational families. There's nothing wrong with that. It's great, sometimes. However, when you have more people might have less space to have a quiet space to learn. So all of those factors that we've discussed-- and we just need to make sure that we're having those conversations-- but I do want to note that some kids actually preferred it and learn more virtually, but they had a lot of advantages-- privilege.

SPEAKER 2: We do have one question online. I want to encourage people online to pipe in as well, so I did want to ask it. And I think it's been touched on a little bit, but want to know if there's information about how test scores and other variables might have been impacted by the parent's job, the style of, probably, the question of essential workers, and maybe not necessarily from this study, but other information about that.

REBECCA JACK: Yeah, I think that's a really good question. I know there's been work on parents' labor force, like the impacts on parents' labor force participation and things, but that specific interaction between parents and test scores, I don't think we really have any clear information on that. But I think it is a very important question. And I know, anecdotally, we heard a lot of stories about some students, especially students from wealthier communities, who were more privileged. Their parents were able to outsource a lot of this education and fill these gaps in ways that other students did not have access to.

And I think some families had the ability to have one of the parents step back from the workforce and provide more of that support at home educationally, and others didn't. So I think there is a clear interaction there, and I think it's a really interesting question. I also don't think we really have the information to speak to that yet.

SPEAKER 2: I guess I have more of my own questions. We're going to get Kristen off-schedule. She'll be mad at me. But do we have a sense of what it would take to make up for the learning loss in terms of resources? And I don't know if educational production functions are well enough understood, but if we did summer school for everybody for a year or something, would that be the order of magnitude or what would take?

SOFOKLIS GOULAS: Yeah, [INAUDIBLE]. And, well, we have a public school system that generates this knowledge. So it is conceivable that if we extended the duration of how long people stay in that school system, it would help them recover.

That's one approach to go about it. Another would be to inject additional resources in the existing system to accelerate the pace of learning. And this can be through tutoring. We hear about tutoring a lot these days.

Another would be to provide more resources to teachers to make them more effective and improve the consistency of that effectiveness to make sure more-- because a key challenge of the pandemic is that the classrooms became more diverse. So it is harder for the same teacher to help all students in the classroom now. So providing more resources to the teacher might be a way to go about to address those losses.

KRISTEN BROADY: Well, I want to thank Sofoklis and Rebecca for joining us, and Rebecca for your amazing presentation. We are going to take a 10-minute break and just a bit of housekeeping. There are restrooms down the hall that way for anyone who needs that. At 11 o'clock, we are going to come back together and hear from our very own Kelli Marquardt, who is going to talk about ADHD diagnosis during the pandemic.

And just to go through the rest of the day, we will have lunch, so please plan to stay for lunch. After that, we're going to hear from Dr. Jenssen, who is going to talk about one of my favorite topics, which is obesity and, particularly, changes in childhood obesity. We'll then hear from Dr. Schanzenbach, who is going to talk about the safety net, what was done, what can be done. And then we're going to have an amazing panel discussion. So we hope that you will stay for the rest of the day, but enjoy a quick break.