All right, it's 2.50. Welcome back to our session. I'm Jeff Freimler, endowed chair for geology of the solid earth in the department of earth and environmental sciences at Michigan State University. And so we have two speakers in this upcoming session. Jonathan Arno from UCLA and Beth Pratsotala or Beth P.S. as often known for short from Central Washington University and Beth is also has a long-term association with UNEVCO. And so I accidentally told John that he had 20 minutes earlier today. It was supposed to be 15, but anyway, we have time for both of those talks. And then after that we'll have a little bit of questions after each talk if we have time and then we will have a panel, about 30 minutes of panel after the talks. So our first speaker is going to be John and so if we can get him queued up, we'll move right into the first talk. Okay, I'm sure we'll get this queued up very shortly. Okay, can you see or hear me? Yes, I can see you. We can now see you on the main screen. So you should be able to share, are you doing that?  I am, I'm gonna share my desktop. Okay, great, we'll turn it over to you then. Okay, and let's start my talk. Can everybody, does this come through? Does my desktop working? Yes. Can somebody verify that? Okay, great. Okay, and yeah, my apologies for getting the timing wrong. I thought it was 20.  So today I'll be talking about an experiential, maybe philosophy to geoscience engagement. And this is through a project we're calling DIY Namics. And really the approach here is what comes before quantification, right? We need students to become quantitative. It's an important aspect. But before we can get quantification, what do we need? And my argument is that we need really engagement. And we need buy-in. And so to get that buy-in, we need approaches to engage broadly, students of all levels and all ages. And my approach is that typically the best way to do this is with an experiential steam argument. And so that's a big capital A in steam. That is the R really comes through importantly in that first engagement event. And so my argument then is we need to actively lead with the art, with the beauty. And once people have bought into that, then the math will come later. Okay, so what is this? Okay, so what's the content for these engagement? For engaging, what's the content that's critical? And my argument is it's whatever interests the person taking part in the engagement. Usually for me it comes down to whatever flummox is my students. If I have something that's well known to be difficult for my students, maybe that's an aspect where I will specifically focus to try to engage them with something experiential that they can really dig into. However, and then at what level do you do this? My argument again is to get things started, it's whatever you would first naturally wanna take part in. And then it takes a lot of brainstorming. What's the topic? What's the level? And that's not just brainstorming with yourself, that's with other colleagues of yours. And for me, it's often with grad students. And that's, I find really rich for everybody. Okay, so now what are the mechanisms for engagement? For me, I'm gonna argue there are three at least that we tend to focus on demos. After demos, we then go to videos. And then the last thing I'm gonna try to argue for today is DIY do-it-yourself demos. And that requires both of the above and more. And as you move down that list, you're increasing both in complexity and in effort and hopefully also in scalability, both the ability to reach more and more people. Okay, so let's first just do demos. Here's a quick movie I'll show from an outreach event. We have one of our big demo of rotating tables. I study rotating fluid mechanics and planetary settings. And I'm gonna, let's just play the movie. Here's a demo. Okay, and in a minute or two after this, we let it, actually it's for 30 seconds. We let the turbulence that the participants have generated with their hands. We let it die out of it and we see this. Okay, so arguably this demo was fairly engaging for the students taking part. You hear the, there's one person nearby talking about how maybe things look like Jupiter. You also see, actually you can still see in the bottom of the image right there, there's one person who's, she's still got her finger in the water. I don't know who's engaged more. There's a lot to simply dragging your finger through water. This is simply water and food coloring on a rotating table. And yet immediately we've drawn in a lot of people. There's a great deal of buy-in. Okay, that's wonderful. You might say that's engaging for the people taking part, but also the person running that event was a UCLA undergrad. And by the end of the day, he also was at a very different level of understanding the concepts underlying that experiment. Okay, another way we go is with videos. And so we've got a YouTube page, just youtube.com spin lab UCLA. And this is a very different direction to go. This is now where we've generated about 40 videos to date. And they cover a great array of topics, typically they're fluid dynamics because that's what we do in the lab. And the idea here is that we can now disseminate far more effectively, right? With demos that we're doing either in class only or at an outreach event, we're only talking to a few people honestly. So for that outreach, so for the video I showed in the prior slide, that's at UCLA's annual outreach event that gets about 5,000 participants, maybe 1,000 come through our part of campus. And we've done that for 10 years. So maybe that talks to 10,000 people total. And with videos, we can get much further. We can do the same experiments, hopefully with some explanations, some instructables. And I know other faculty members use those in their classes and other people watch these as well. So hypothetically they scale further than just my group doing a demo. So let's just show one of these to get the idea across. We'll show, I actually have the slides still from this. So we'll look at my slides. I'll just show you a video we made of a time lapse of a evaporative convection in just a one meter container of water. And so this video has very brief, in this case, directions or what's being shown, one meter tank, about three centimeter depth of water. The fluid contains barbissol. So shaving cream that we mixed with water and basically extracted the flakes from the barbissol. And there's a link to a paper where you can do that. We also have an instructables on how to do that. And then we put a GoPro above the experiment and we just filmed it for about two hours. And it looks like that. Okay, so it's interesting because one, there's a lot we can talk about there. One, this immediately sets up that when you do this by hand, there's truly nothing to see. It doesn't look like the fluid's moving. It's definitely moving far less than a millimeter per second. And yet if we can control time through this time lapse, then you see really complex motions. You see, I haven't actually calculated if it's turbulent, but you see a lot going on. You see large-scale structure. You see the evolution of that structure over time. And so there's a lot we can talk about with students and let's put, again, order to think a way to visualize what it's doing and get creative with how we film it. Okay, the other part of this is that my whole team gets to take part in this. The filming was done with Susanna Horn, who was a postdoc at the time. The barbissol was made by grad students who found Sue and Ashna Hargirwal. The time lapse was, again, done by Susanna. And then I just keynoted it and put it onto YouTube. And we have many of those films. This is supported by NSF. Humbly, this is often one of the better parts of my reviews for my science grants. Now, so now we've talked about demos. We've talked about doing demos and then videoifying them, hopefully, so that they can go to a much larger audience. I now wanna go to one last topic, which is the idea of DIY demos. Because, again, scaling-wise, seeing movies is not the same as doing. And so we've got a new project called DIY Dynamics and we've got a GitHub site for this. And there are about 15 people on this at six different universities. And here's an image of the GitHub page. And the idea is that we're trying to create affordable and easily accessible demonstration setups for teaching geophysical and earth and planetary fluid dynamics. And so the key here is that we generated about a $50 do-it-yourself kit for building a rotating table to simulate atmospheric and oceanic flows. So the image you see there at the top left shows our little Lego drive system. It's a little Lego motor attached to a wheel. The wheel then makes contact with an oxo-lazy Susan. And then we add a little sidewall or put a little tank on the lazy Susan and we can do rotating fluid dynamics. And so there in the bottom image, you can see we've created a quote-unquote cold north pole there at the center. That is simply a can of frozen tomato paste. There's a layer of water and we've put cold food coloring around the cold can and it undergoes what's called a baroclinic instability. So it goes unstable like the polar vortex does. And there are two important aspects here. One, the students can build the Lego system in about 15 minutes. We've got videos online to show that. And so you'll see there's the table. There's a link to the table and we've got a video. That's Norris Coo building the Lego system. It's a three minute video. We've also got PDF somehow to build that system if you don't want to do the video. In addition, we've got a blog with a number of entries and we've got also a link for teaching with a bunch of instructions. But if you look on the blog here's, for instance, how the table was used by Dr. Miriam Glesner at Keel University. She's an oceanographer. And this was a departmental colloquium she gave where she set up four of our Lego tables and did four separate experiments. And just had everyone mix around. And our grad students I believe were running each of the demos. That's at a very high level. However, we can take the same system. There's our little Lego drive.  And you can see there's a beautiful green vortex there. That's in a seventh grade class in Los Angeles. And I guess actually what I still want to get across here is students and teachers can easily order this kit. All the web, everything's on our website for what you would click to order this system. And then you can build it yourself. And then we've got directions for how you would carry out various projects and various experiments. We're trying to make as we do this kind of a hierarchy of experiments. It's $50 for the Lego table. We've got about a $500 space record player. And then at the top end, this is a Gonzales. And he has designed with us and fabricated about a $5,000 kit. Might be a little less. We're still working to get that down. But this is a one meter table. And it rolls, it's got its own battery. You just roll it in and, oh, sorry. It just rolls in. And there again, you have all the tools you need to do experiments with students. And the sides are cut on the bottom so it'll roll through some of the tighter doors in our department. And so here's a movie showing that in action. And again, this is doing another Barrett Clinic and Stability experiment. You can see these huge vortices that form. That's a lot like what the vortices that come down and slap into Minnesota in the middle of winter and make people uncomfortable. One fun part of this is you can also see right here we put on a string of thermometers. These are thermistors and they're wirelessly connected to my iPad. So we were looking at temperature signals and recording temperature signals while doing this experiment. So we could correlate what we were seeing in the dye to what the temperature was doing. And it's interesting because when these big cold vortices come down the temperature goes down but when the cold structures go down that means warm stuff has to fill back in. And so we would also see spikes up and down in temperature and the students could actually see that. Okay, last thing I just wanna talk about in time of in our present COVID remote universe I've been asked about how do we do engagement and how do we do experiments and DIY work remotely? And so I'll just give it again just an example. Here's an ocean and a cup. A grad student in my group is TAing oceanography right now and he got asked how do I get, normally we would do experiments, talk about ocean dynamics and ocean structure. So we don't need to do much here. Here's a time lapse of a cup of coffee with some cream in it and you can already see there's sub layering and we wanna get students thinking about structure of layers within the ocean. And I took a time lapse and you can see we have stable layers and we also have lots of convection dynamics in the hot coffee. And he asked how do I actually do that myself? So here's another one I just made yesterday, again a time lapse. You can see there's the hot water being made. Then I used tea, so there's the tea and then I added some cream. And so the upper layer is not a terrible analog for the mixed layer of the ocean and you could think of the creamer layer as the deep water and what's interesting is there's turbulence in that convecting upper layer of coffee and you could, I hope, see, I'm gonna show it again. I hope you could see that very naturally it starts to form sub layers. And so the students have a lot to possibly talk about. At the, right there we just missed there were some really rapid oscillations and they damped out too fast, but there you can see the sub layering. The sub layering is robust, I mixed it, it comes back. I mixed it again, it started to come back and then I mixed it a third time. So one of the ideas for them is to simply have students in a way to create a best movie, right? Just every, let's all see who can make the best movie to explain these processes. And if you want, you can spend a lot of time on those actually, another question is what time scale is the best time scale to think about a set of processes? So here I'm gonna show the same experiment now in coffee, same creamer, but now filmed at real time. And now you see very different dynamics. You see a large scale fundamental mode of oscillation and it's this kind of low period internal wave variability that greatly affects climate and we can then get students talking about climate dynamics based on long period internal dynamics of the ocean. And so we don't have to do much to get students actively doing their own experiments. What are other possible tools for remote engagement?  Do students have remote sensors? They do, it's called a cell phone. Most cell phones have GPS, accelerometer, pressure sensors, light intensity and magnetic field sensors all built in. One of the best ways I know of to easily take data off your cell phone is with Google Science Journal which is an app that gives you access to all of that data. And you can use that on a cell phone, you can download that also to an iPad. And the data is wonderful. How else can we get students DIYing things remotely? We can be teaching programming with scratch remotely, we can be teaching Python remotely, it's free, anyone can download it. Lastly, I would mention Minecraft because many, many students now have Minecraft on their computers. And Minecraft really is inherently geological, I would argue, right? My kid plays it, he's always talking about obsidian. He's talking about all these different rock types. And you can connect Minecraft to Python and you can program in Python. And so actually we could be programming geology problems into Python. We could be building in erosion laws. We could be doing lots of interesting things. I don't know that that's been done yet but I'm just brainstorming things that we could do remotely that students already kind of have access to. Lastly, I would mention Arduino's. These are little microprocessors. They're about $20 for expensive ones. And this is a great way again to be doing experiments remotely that people can do in their kitchens. I do this in an advanced computing class for undergrads. In this case, everybody builds their own, each of those little red things is an Arduino. They've each built a sixth temperature sensor array. They're dumping it into a big vat in the middle of petroleum jelly. And we're gonna put that into the fridge and look at a thermal wave going into the petroleum jelly. So we're actually in a way trying to model cooling of the ocean lithosphere. How to set up one of these systems is already online. I had an undergrad make instructables for how to do this. It's actually not hard. And so it's interesting because it immediately gets students taking data on a problem they know, cooling lithosphere. And they're in doing so, learning advanced computing, learning hardware and doing moderately hard theory all at the same time. And they get very into it. They really buy in once they've built their own deacquisition system. Yes, this is all in my class, but it's very portable to a remote system. Okay, so engagement before quantification. You can see there at the end, engagement quickly turns into quantification as we get people computing and doing experiments. For me, it's demos and it's videos. Then it's do it yourself, right? Then it's take action. I'm hoping with the DIY, DIY NEMEX project that this could scale even better than the videos we will see in time. I'd also like to argue that these concepts hold for numerical modeling as well and basically across fields. It's a little harder, possibly, for to get people to get a hands-on feel for numerical models, but if energy is put into creating a good, gooey interface or even a good Jupyter notebook, then I really feel like these can be largely parallel systems. And a lot of it then just comes down to brainstorming with colleagues and students. Okay, thanks very much. All right, thanks, John. So yeah, in the interest of time, I'm gonna go straight to Beth and we'll go to our second talk, Beth Prasatala from UNEVCO, quantitative skill development in the Geosciences, examples from existing projects.  And yeah, if you can do 15 minutes, we'll have a little time for questions and then we'll move to our panel. Thanks. Yeah, he'll try. I was told 20 minutes too, so I've been trying to think of ways to shave it off. Don't worry if it's not, we'll get to the panel when we get to it. Okay, well, thank you for inviting me to participate in this meeting. Quantitative skills are a really important aspect of developing a ready Geoscience, including Geophysics workforce and I'm glad to be a part of it. Among the other things, I run an undergraduate curriculum initiative for UNEVCO. UNEVCO runs the NSF's Geodetic Facility and so quantitative skills for undergrads are an important part of my work with UNEVCO because they're important to the Geodetic community. I also do a lot of work involved in situating STEM learning in a societal context and working on Geohazard education. Today I'm gonna talk about several projects that involve helping undergraduates to develop quantitative skills in Geoscience. One project that I'm heavily involved in and several others I've observed or have influenced our work. I just wanna make sure that, and I'll be touching on a few things that other speakers and panelists have mentioned that hopefully is informative. Let's see which way does my cursor want to advance. Oops, there we go. There we go. Okay, so the contents of the talk will be, I'll talk about Geodetic Tools for Societal Issues, the GetZee project, which I'm the project manager for so I'm very involved. Earlier projects that include quantitative skills in Geoscience that influenced or informed things we did in GetZee. I was not involved in those but I'll tell you about them a bit. And a new project that's ongoing called the EDI project that I'm slightly involved in and then some takeaway thoughts. So Geodetic Tools for Societal Issues, Undergraduate Teaching Resources. I'm really talking on behalf of a large group which includes a variety of co-authors listed here and more than 20 authors and technical advisors that's collaborative between UNAVCO and other universities and supported by CERC and N-A-G-T. The history behind GetZee is that Geoscience and Geodesy community members had great ideas and were already doing some things. There was a lot of other community members at the same time asking the facility, the NSF facility for more help with teaching on geodetic topics and support with the teaching resources. So UNAVCO collaborated with community members to make this happen and the GetZee project came from this. The overall mission is to develop and disseminate these materials that feature geodesy data and quantitative skills on critical issues such as climate change, water resources, natural hazards. It is a sister project to the larger integrate project that was headed by CERC. GetZee has had four NSF grants since 2013. Two are the exploratory ones are over now and the two others are still underway. In total, 13 developed modules of sort of like two weeks each. That's, you know, a squishy size, but it's both introductory and major's level and both classroom and field. So I think everybody knows, but just to make sure Geodesy is the science of accurately measuring the earth's size, shape and orientation and the mass distribution on the earth and how they change with time. So traditionally this was pretty much like surveying because it was just hard enough to know how far Germany was from France. But in the last decades, Geodesy has really burgeoned into a toolbox of techniques for better measuring the earth and the changes over time. If you look into this toolbox, so these are the data types that we're including. It's including GPS, INSAR for regional deformation, high resolution topography, be it from LiDAR or from photogrammetry, structure from motion, strain meters, tilt meters, creep meters, gravity measurements, especially the gray satellites and sea level and ice altimetry from satellites and some extent airplanes. So working with Integrate, the guiding principles for Getzy were to address one or more grand challenge facing society that Geodesy, Geoscience can help with, make use of authentic, incredible, in this case, Geodesy data, improve student understanding of the nature and methods of science, like how we do science, how we communicate about it and develop student ability to address interdisciplinary problems. And I think of this as applying Geoscience learning to social issues. It's not enough to just say that earthquakes, yep, they knocked down buildings, but having students use data, geodetic data to then inform how you would look at infrastructure vulnerability and make mitigation plans for a community. Increase student capacity to apply quantitative skills. And this is an emphasis that Getzy has more strongly than Integrate. We do follow the backwards design principle using the assessment and development model that was moved forward by Integrate, which is starting with the learning goals and the learning, more granular learning outcomes, deciding, well, how will they know if those goals are accomplished? You do your assessment strategy, then you find your teaching materials and instructional strategies to match. Everything was pilot tested by both authors and a third non-author and revised prior to publication. Just a quickie on the topics that we tend to cover. These are the intro level modules ranging from volcanoes to climate change, water resources, landslides, majors level modules, earthquake hazards, again, climate change, water resources, landslide hazards, floods, I didn't get that one in here, but at a higher level than the intro and we do have two majors level field modules as well. So the quantitative components were not envisioned as a big, like, oh, let's cover all quantitative skills. We really let it come from what the authors thought made sense based on their particular discipline and the learning goals. The quantitative learning is embedded in the need or task of addressing a societal issue and the quantitative learning is intimately tied to working with real, i.e. messy data. Some examples of what that actually looks like is the intro level time series and rates came up a lot. So even students that weren't comfortable at the beginning of the module had a lot of chances to work with it. Also things such as reading quantitative images, simple, more graphic oriented vector work. At the majors level, spreadsheet analysis came into many of the different modules from data analysis to sort of like a strain analysis using it for more complicated calculations. GIS work for landslide susceptibility or surface generation from a survey, modeling, earthquake modeling from INSAR deformation, flood modeling from topographic and other parameters. Other things that were included were statistics and anomalies and trigonometry, estimations and detrending, but I should note that calculus differential equations in linear algebra might be mentioned to some extent, but the students were not doing them as a central focus. That was a plan that's just how it happened based on what the authors decided. What one of these might actually look like, this is the intro climate change one, start out with a reading on Bangladesh and the human cost of a sea level rise, then there's three units related to data, may a gravity or GPS, and then finally looking at two case studies of the New York area and Southern California and how they're going to have to address and mitigate for sea level rise. For the GPS strain and earthquakes, it's similar starting out with looking at economic losses from and human losses from Tohoku, Japan 2011, moving on with the GPS data here, in particular looking at triangles of GPS stations to understand the overall strain in an area, and finally, students are able to pick at least a somewhat tectonically active area that they're interested in, such as my grandmother has a cabin in Tahoe and I'm interested in the earthquake hazard there, and then they have to propose mitigation that would be appropriate for a community once they've learned that. So that's the sort of things that the Get See project has done and it is, of course, relevant to the topics of this committee. Now I'll go into some of the earlier projects that I wasn't involved in, but I think help inform how we did things. So I wanna just mention that Jen Wenner is the person who I talked to a bunch about this in preparing this section and she's been involved in all of these and Kathy Manduka who is on the panel and you can ask her more questions about this, she knows more about it than I do. But in 2002, there was a workshop that brought together geoscientists and mathematicians teaching quantitative skills in a geoscience context and they developed five good ideas that have shaped a lot of projects since then and that is place concepts in context, a disciplinary context use multiple representations within that context work in groups that's been shown to work well in STEM across the board use appropriate technologies and do in-depth projects problems that last more than one day or come back to problems in the same sort of concepts more than one time. CERC has a variety of site guides including this one on quantitative skills, thinking and reasoning. And so this is just a snapshot of some of the things that are there, collections and guidance within the larger CERC site. So developing quantitative reasoning, teaching quantitative skills in the geoscience, that's one that Jen Wenner was involved in, spreadsheets across the curriculum and the math you need when you need it. This is been mentioned I think by two of the earlier speakers and so conveniently I was planning to talk about it. This project was run by Jen Wenner and Eric Baer and it is a series of tutorials that are designed to be asynchronous but parallel to a main class and it sort of relieves the introductory instructor from having to teach the math but giving the students access to it who need it. And I'm trying to put the URLs up here if anyone is wanting to follow along. So when you go to the math you need when you need it place and I have taught with this a little bit back when I was a faculty member, it starts with a page that introduces and explains the concepts and then a set of practice problems with different geology applications for the same concept and then depending on the instructor's choice a set of quiz questions that can have a higher grade value or not. And these are the topics that were picked through input from the community and looking at typical introductory concepts. Oops, so the, sorry, I have to move some things around on my screen. So this project particularly emphasized the first two of the big ideas that's placing it in a disciplinary context and using multiple representations. And so an example for this would be like I calculate rates, calculating changes through time. This top half of the page is sort of the what is it, the math side and what are some of the overview ideas. And then the second page that you can jump to is actually example problems. And so you can see that rates have lots of applications in geology, be it temperature change over the day, magma generation per year or movement and stream channel meanders, like all of those can have rates. All right, so the findings according to Jen was that overall the students were happier doing more math with better explanations than less math with little explanations. So it's not the math, it's the feeling confused and lost about math. That's more problematic. It did help level the playing field so that students who were lowest were the ones that gained the most ground. So the lowest quartile underwent 44% normalized gain and the students that if it was optional, the lower students who availed themselves of the math you need when you need it came up and the lower students who did not avail themselves of it did not. So it's not the course itself, it really is these additional tutorials and 10% of the users that they were aware of were majors level courses. So unfortunately, our majors still need these pretty basic skills too sometimes. The last project I'm going to mention is Eddie, which I've been involved with as an author, not as somebody running it. And Eddie is environmental data driven inquiry and exploration. The project goals, oops, I have to move some stuff around on my screen. The project goals overall, maybe I should back up and say, this is using really big data, publicly available large data sets and having students work with those. So that first off, they're learning to manipulate these larger data sets. Think like, well, what questions can I ask once I'm in this data set? Developing reasoning on statistical variation, engaging in scientific discourse and understanding the nature of environmental science. So there's been, they're currently in the second phase of the project. In the first phase, there were 10 modules. This gives you a sense for some of the variety that was there. I'm involved in the second generation, if you will, on a GPS data related module. Can't get enough GPS. And one thing that I just found this, I brought this in here because I found this Venn diagram to be super helpful. And this is thinking of quantitative reasoning over just quantitative skills. So math skills, quantitative skills are awesome and we need them, but they're much more powerful and much more useful if you take it to the level of quantitative reasoning, which is bringing in your critical thinking skills and this disciplinary or real world context. So bringing these things together and trying to pull out what I would say are some of the main takeaways. Backwards design, start with defining your goals and outcomes of if we are doing programs together, projects together in the future, start with your programmatic goals and how you'll figure out if you accomplish them. And then if you have teaching resources or programs within that, have backwards design for those as well. Aim for quantitative reasoning with the skills combined with critical thinking and real world societal context. Provide a variety of examples of a given math skill applied in different geoscience applications and consider whether something like the math you need, majors edition would be beneficial, not just the intro. Thank you. All right, great. Thank you, Beth. I think we've got time. If there's maybe one question, well, actually I think we're gonna skip the questions. We'll lump that into the panel. I think that's gonna work better. So thank you, John and thank you, Beth and we should transition to the panel. So John and Beth will be joined by Michael Hoopenthal, John Tabor and Kathy Manduka. Let me just briefly introduce them. So Michael is senior education specialist at the Iris Consortium. He was an earth science and physics teacher in the Maryland public school system. Also worked at NASA's Goddard Space Flight Center as an earth system science education specialist. John Tabor is director of education and public outreach at Iris Incorporated Research Institutions for Seismology. And so he's been involved in a wide range of formal and informal education activities at a variety of grade levels. He was also on the leadership team for the undergraduate education integrate project which has been mentioned. Kathy Manduka is director of the science education research center, CERC at Carleton College that's been mentioned multiple times today. She leads their work to improve education guiding projects to completion, developing new directions, raising funds, managing the staff. And in addition to all of that, she was the executive director of the National Association of Geoscience Teachers from 2007 to 2019. So Michael and John each have a slide or two and I think Kathy does not have a slide. So if the panelists could just maybe give us just one minute or two minutes at most, then we'll move on to questions from the committee and the audience. So this is Michael's slide, so I'll turn this over to Michael. Great, thanks Jeff. So I just wanted to jump on to or follow along on a couple of things that Beth mentioned that I think tie in well with this slide. So running the undergraduate internship program, I think a few things that we know work really well from these sorts of in-depth experiences is that these are highly engaging for students. They create a reason for students to want to learn and apply the skills, these quantitative skills that they've learned because the scenario creates the, so what or why should anyone care about making use of some of those skills? And I think the other piece that's important about this is that these really are cognitive apprenticeships. They're chances for students to learn that quantitative reasoning that Beth mentioned as they have a chance to watch expert researchers make decisions about how and why to apply various statistical methods and make those sorts of choices explicit for the students in ways that they might not otherwise get to see in coursework. And then the final piece that I wanted to note here about the program was something that ties more into the first session about this. But over the history of the program, what we found is that we've received more applications from geoscience majors. However, throughout time, what we've observed is that geoscience majors generally are accepted at a rate that tends to be lower than those of math, physics, and geophysics majors into our program. And that's certainly because of the quantitative nature of the research that they're doing. But the other part of that I think ties into something that was mentioned in that first section, which is about advising. And often we see geoscience students who don't start taking quantitative courses until later into their careers. So advising tends to also be an important point. Okay, great. Let's see, do we have the next slide for John? Okay, so John? Sure, so I'd just like to talk about one project that Iris is working on right now which falls on directly to what Beth is just talking about. That's, this is basically built on the foundation of CERC's Integrated Project and what Getzi's been doing. And you'll see a lot of words that are very similar to what people have been talking about all afternoon. We're trying to provide context for analytical skills so that students see a reason why they should learn those analytical skills. We're trying to do it through taking introductory geophysics which is usually taught at a higher level, often junior senior level, and have it brought down to a level where you can have it in an intro course at a freshman or sophomore level but still have it quantitative enough that students see the reason for using those quantitative skills. And what we're doing that by something we're calling urban environmental geophysics. And so those modules are things like using GPR to look at infrastructure in the city, using resistivity to understand water pollution near a city or using shallow size of refraction to figure out what the site response might be when an earthquake occurs in a city. And we're also trying to target underrepresented groups. So we're trying to again have problems that are associated with something near where they live and they can see how geophysics can be used in trying to solve those problems. And we're also looking at trying to get these modules into physics courses and into chemistry courses because in a way we're trying, I think we hope they can be used in geoscience courses as well but we think like for geophysics really it's trying to engage the physics students to see that geophysics is also a potential career path. And so for example, we're trying to work with HBCUs to see if we might get some of these modules into HBCUs that have physics and math and chemistry but may not have a geoscience department. All right, thanks John. And I'll call on Kathy in just a minute. So give her a moment since she doesn't have a slide if she wants to collect thoughts for a moment. Let me give some instructions to those of you who are all of our attendees. As before, put questions into the Q&A box and then I can ask questions and direct them to the panel. If you have a question for a particular panelist, if you could identify them in the question. If it's a general question, I'll just address it to all of the panelists. We tried in the first session to use the raise your hand feature but that sort of worked but sort of didn't work. So I think if I don't call on a particular panelist, I think panelists feel free to sort of jump in and if there's two people trying to speak at once, I'll play referee and get you to you in order. So let me, Kathy didn't have a slide but let me turn this over to Kathy for just a minute or so. And Kathy can just say a few words. Everybody talking about all the great work that's taking place and it gives me more ideas for things I might comment on than there's time available. But I wanted to throw out one thing that hasn't been talked about too much and emphasize another one. So we haven't talked too much yet about the role of education research in informing these activities. The guiding principles that Beth put out that came from the early 2002 workshop actually came from mathematics education research. And that's the oldest of the disciplinary-based education research groups and is a very well-established field and has a lot of insight to offer to us as we move forward. The second piece of the research that I think we have some good insights into is about the whole problem that having a quantitative skill doesn't mean that you know how to use it and Sharon called out both of those as goals for the employers in geoscience. And we know one of the things we do know, so the people who understand that or who study that ability to use it are the people who study the transfer of learning to application. And one of the things that we know from that is that to be able to use it requires practice and the more practice and the more diverse the opportunities for practice are the more likely the more you build your ability to use your quantitative skills in diverse kinds of problems. And I think that's the piece of research that really underpins this notion that we have to have quantitative skills in all of the geoscience classes. We can't just stick it in one course and think that people will take away the ability to solve problems. There's a middle piece that we don't have very much research on and that's how the various models that people are developing and have developed over time for supporting students' use of quantitative skills. And we've seen a whole bunch of examples of those today. How they support learning and how well they support learning. And I just would shout out the work of Kim Kaston's and Krumholt Hansel who have analyzed the integrate materials, several of them for the different kinds of designs that people are using to teach students to look at data and that I think we're gonna need more research in that kind of realm in order to really be able to look comprehensively at the kinds of resources we're developing and understand not which ones are better necessarily but how they're working and therefore what kinds of circumstances and students they'll best support. And then the second thing I just wanted to say is that going back to the shout outs for Matthew Neid and some of the other points that have been made is that it's not enough to be able to develop quantitative skills for some geoscience majors. We need to be able to think about how to develop them for students from a wide variety of backgrounds that come in with different talent for different kinds of problem solving. And that just really enhances the problem that we're facing, right? Because we have to be able to adjust to help people who have different backgrounds. We have to be able to provide different kind of practice for different kinds of students with different kinds of talents.  Okay, thanks, great. So we're open for questions. Again, those of you participating if you could type questions into the Q and A box we're monitoring those. In the meantime, actually I'd just like to invite the other panelists. I saw some heads nodding as Kathy was speaking and just like to invite you to add on to that momentarily here or not. Well, actually we have a question. So let's move to that. So this is a question from Mike Brzezinski and he asks, has there been an evaluation of what factors control the adoption of these more quantitative educational resources? In essence, I'm curious how well these development efforts will be adopted at the scale we need to prepare students appropriately. So yeah, what can we say about the, what is influencing adoption or lack of adoption of these tools by educators in practice? Well, one of the things, I don't have an answer yet I guess I would have to say, one of the things that Getzi is asking for is now that materials are out there. Please, if you use them, tell us what you used and why but it's a slow process to get people sort of professionally developed about the modules and then actually use them and then actually take our survey afterwards. So maybe I would have more quantitative data on that after about another year. Can you speak to them, Matthew, when you need it? I mean, I know they did a lot of studying of the use patterns and stuff, Kathy. I can't speak to the adoption. I mean, I know that math you need has been well adopted among the, I mean, in the sense that there are a lot of people using it. I don't know what the actual penetration is relative to the people who could be using it. I know for the, so the integrate materials have been well adopted and are still seeing the last time I looked very strong increase in use with the majority of people who are adopting them now adopting them from the website with little or no introduction to them. So you could ask the question, that doesn't speak to the quantitative piece at all but you could ask, but it's also a behavior that's not been seen in very many cases. So you could ask what's supporting that adoption and I would argue that two things are supporting it. One is that there's very good information about how to use them well designed. They're easily, they're very well supported but there's also a lot of information about who's using them and how to adapt them to other different situations. I think that's important. We see people commenting on that. The other thing I think is true that we have less information about is that there's a large group of people who are involved in integrate and can talk about those materials to other people and they have, so they make a big groundswell of trusted sources that can help disseminate information. And I would argue that if we really wanna see a transformation in quantitative, teaching of quantitative skills, we're gonna have to mount a community scale effort that really goes at all three levels that are needed. One is we've talked about as curriculum materials, the second in supporting faculty and second is about departments and the third is about advising and that will have to be integrated with a research component to help us make sure we know what we're doing. That we're doing no harm. Right, are there other thoughts from our panel on the same question or follow-up? I don't know the answer to my specific question which is, are quantitative activities different than regular activities? We have some data on that. What we could, I mean, we have the data to be able to look into that question. Okay, I think Michael was about to speak. I was just gonna say that, I think we can think about this in terms of just changes to education systems broadly. And so we need, this will require some professional development. It's gonna be because the traditional model is content first. And so we're looking at shifting from a content first, geo science content first, to integrating this quantitative aspect to it. So that's going to require some changes in thinking about course design and curriculum materials. So it came up in the first discussion today about juggling that trade-off between how much time is being, content that's being covered versus some of these skills and processes that we're beginning to think about. And there's models for that from even K-12 or middle school education. Okay, great. Actually, I've got a question that comes from Cindy Ebinger. And so she says, some colleagues have argued that increasing quantitative skills or increasing the quantitative expectations might lead to decreasing numbers of majors. That is, if we're driving away students by making them do things that they find hard. But are there actually data to support that concern? And this is separate from whether we require particular courses, but do we have a sense of anything other than anecdotes that do we actually attract students because we're doing things more quantitatively? Or do we actually drive people away or do we simply help them bring their level up? Do we have some data? I don't know the answer to that question in a paper-siding sort of way. I think that the majority of work that's gone into investigating and is at the introductory level. And I think a lot of it is anecdotal, but there may well be some studies. I think for me, the bigger question is, what is the role of an introductory geoscience course and what role does it have to play in developing what we now know are critical quantitative skills across the population for living an effective life and being well-renumerated in your job? Is it appropriate to offer classes with no quantitative component in a science regardless of what that does to the enrollment? But I'm on the extreme end in my view there. I don't know anything about the major's level and it's been a long time since I've read the math you need when you need it papers. But I do think they found that student feedback was like, oh, if it's not so confusing, I don't mind it so much. I think at the intro level, again, that that level of way they were surveying the students, they were not finding that it was a deterrent. It was a relief, I think, overall. They've also been doing a lot of work on affect and helping students overcome math anxiety and how that relates to students feeling successful in quantitative classes. Okay, great. We've got some questions coming in. So let me go to one of the next ones here. It's from Karen Viscopic. What are some strategies for helping faculty to understand the importance of transferring learning and seeing their own role in helping students develop quantitative skills rather than expecting other departments like math to do the job? So this was discussed a little bit in the previous panel to some degree. But yeah, do we have more thoughts? I think one of the issues in the previous panel where this was discussed was in putting these problems in geoscience context, you're introducing them to these new quantitative skills in the context of things that they already have some understanding about and that they have some interest about. And then that helps them in developing those skills rather than something that is done completely divorced of any context that seems relevant to them. Does anyone have any further? Yes, John. Yeah, well, perhaps maybe this is a little outside the way we've been talking about everything inside a geoscience department, but maybe we also need to be looking at it the other way around how do we help math departments provide geophysics example or geoscience examples, the same thing with physics, that maybe we need to be doing a better job convincing the math department that they'll have, even if they have more majors, even if it's more majors or just more engaged students that we can provide them with some very good examples. Okay, great. So I've got a couple. Oh, sorry. I was just gonna say, I think the math you need, I mean, professional development sort of, I think is one answer to that question, right? I know the math you need when you need it went along with significant amounts of professional development for those sort of deep implementers. I can just add that I don't wanna dominate the conversation so I'm trying to be quiet, but this problem is not a problem just in the geosciences. So the approach that John is talking about has been picked up in physics and biology where of course the numbers are a lot bigger, but the physicists are now developing a physics course for biologists. I don't think we can get that kind of a response, but we ought to be thinking in that direction. It's also true that quantitative reasoning is of concern not just to geoscience departments, but to institutions as a whole because quantitative reasoning is now recognized as an important undergraduate level outcome, not just a science outcome. And the quantitative reasoning and numeracy initiatives have spent a lot of time writing about what numeracy is and why it's of importance to democracy and mounting campaigns on campuses to support faculty and learning about quantitative skills. And so I think there's a good model there that we could use. They've been quite successful. Okay, great. I've got a couple of questions here for Jonathan Arno and I'm gonna try to combine the two questions. So one is we've talked about evaluation and assessment and so on. Do you have any data or have you looked into how to sort of assess the enhanced learning outcomes associated with the demos and in particular for how that helps students get to quantitative skills and any ideas about some specific skills you would assess in order to do so. And then the other question is also on a practical level in terms of using these in class, how much is the entire class session designed around the demo or are they sort of more interludes in between other parts? So how do you build these into individual class sessions and maybe again get to the question about assessment? Well, it looks like Jonathan is frozen here. Well, can you hear me? Yes. Can you hear me? All right, great. So I haven't taken data yet in any meaningful quantitative way. I'd be interested in learning to do that for them we've been developing demos in a sense on a by-need and what my students are asking for. However, demo like ours have been evaluated by John Marshall at MIT and they have shown they did a fairly big experiment was done and they found that it's were much able to automatically address what the means after doing the demos versus simply learning this concept which was going nowhere and often does go nowhere. In terms of, I can't see the quest mark, Bane. Yeah, I think one question was about specific skills. I mean, if we were to do assessment what do you have some specific skills we might be aiming to assess and then the other part was about integrating these into the design of a class period. Right, so for some of them, I actually do both. For some things I'll do a 20 to 30 minute demo. They always take a while and they always take a while because this often have a million questions. They almost always slow on the class which is to me one of the best parts and they force us to really interact. And what I like is when you do a demo there's a kind of a rising effect. Only everyone is and has patience. We're trying to run the demo to figure out what we're even seeing. That said, that I'll run a topic people in an appreciative of what a topic is in the formal or quantity or using our dwindling. I am building the hoarder class around doing an experiment where we do cooling of a half space. And that we ran that experiment probably four times. And we change it. What do you learn? Learning how are you taking data? How are the computers controlling everything? You're learning all these hands-on computing skills and learning how to take real-time data, plot it and then later analyze it. For me it's worth a quarter. There's one whole core based around one topic and one experiment. Okay, thanks, Jonathan. Yeah, just so you know, it does seem to work a little bit better with your video off. I think it's just less bandwidth. Yeah. Okay, so we... My apology. No, no problem. It's our new world. So another question from James Knapp. This is I think more aimed at the whole panel here. There have been a number of comments today about the role of academic advising in the curriculum. However, the recent trend at many universities is towards quote, professional advising, which I think often means a staff person, but you know, they're distancing the faculty in their own experience from the students, having someone else basically doing the advising. Do we have any constructive comments on how to navigate this trend? Or how should we be doing it better? Well, the first order comment would be, make sure that whomever is doing the academic advising is very well informed about why quantitative skills are important, what the, you know, give them Pranody's career maps so that they know how to help students with different questions navigate the careers. You know, I think there's for challenges with advising are not just about who gives it, it's about how much, you know, advising is a high touch, high interaction activity. And so different schools are using different strategies to make sure there's adequate human resources to be able to give advising. It's really important that whomever is in charge of that has the information they need to give good advice. Yeah, I mean, that's just it. Go ahead, John. I just add to that, I think that there are also a lot of, already a lot of things online that students can watch to learn more about careers. And so I think the, since students are often used to absorbing materials now via video, that the more career type videos that also emphasize, here's an exciting career and here's the skills you need to get to that career. I think the more than that helps the advisors too. Yeah, no, this also fits in with my own experience where in my case, it's mostly been students who have finished their undergraduate degree or applying for graduate school. They find they're interested in doing these things that are in geophysics and then they find, oh, well, I stopped at calculus one and I need two years of math to be prepared for graduate school or something like that. So yeah, I think it is, there's a lot of cases, I think where students get engaged in something later in their undergraduate career and if it is something that requires a certain quantitative basis, they sometimes have a hard time catching up in the time that's available for them. So yeah, I agree, it's really important that they find out about things and find out about, and also I think the more we are integrating some of these things earlier into the curriculum, the less of a jump they have to make, essentially. All right, let's see, one more question about advising here. Barbara Tewkesbury asked, or actually maybe a comment here. Also, geo-faculty can do valuable individual and group advising in the context of courses even at the intro level. This could be totally separate from formal academic advising with an advisor assigned by the university. So yeah, I think that's absolutely correct. We still have time to take another question or two. I've got one queued up here from Andrew Newman. If we are fundamentally changing our undergraduate programs to be more quantitative, should we also be considering changing the names of our majors? There's a stigma, at least at my university, that geology isn't a hard science. So is the name has some connotations? Well, I guess I'd say that's why some schools have a geophysics major that is separate from a geomajor. Now, I realize that's not the answer to show the geosciences also requires a lot of analytical skills, but that's one answer that some schools have taken, I think, or want to approach. Okay, we have some other thoughts. We're out of open questions. We're just past our nominal time, but we can, if need be, continue on for a couple of minutes. Do we have anything further? Any last thoughts from the panel before we go to the next break? I have a thought that comes from this last discussion, which is we tend to think about undergraduate education and graduate education and K-12 education all as separate things. And particularly if we're thinking about diversity and equity, I think there's a big role for thinking about how we support students, no matter to whatever level they wanna get to, no matter what happened to them in the last level. So I think if we start to think more about the total pathway, how do we support students in successfully getting through the pathway that they want? Not to say everybody needs to go to graduate school, but we think a lot about now, about bridge programs and supports for helping students get from K-12 into college. And I wonder if we shouldn't be thinking just as hard about the college to graduate school transition, given that we have a diverse system that's always gonna have people coming out with different sets of skills and because people change their mind in terms of what they wanna do. And it seems like a terrible ways to tell somebody they can't go do geophysics because they didn't know that when they were freshmen. And I don't mean to be really critical because I know for a fact that graduate schools and the geosciences have often supported people switching from, for example, physics into geophysics or physics into geosciences, but thinking about how we help students switch from a non-quantitative approach to geology to a geophysical approach or whatever it is they need to be supported and I think is worth thinking about. Okay, great. So actually I'd like to leave with one last question which comes from Cindy Ebbinger and sort of relates to our new reality we're living in right now. So right now we are being forced to do new things in new ways because we can't do them in the old ways. So does this perhaps create some opportunities for revisions to the curriculum as we discover what works and doesn't work in terms of ways of doing REUs, field work, even lab classes and regular classes. So how can we act to motivate change as we maybe find some solutions in our current crisis? Well, oh, go ahead, John. Can I go to Beth, please? Go ahead. Beth, go ahead. Okay, so the field camp directors and other people in the teaching field camp or field skills and the current scramble to try to provide things for the community that Sharon responded on a question this morning. I know that within those discussions people have talked about how they really hope that we can take from this foray into completely remote learning, take some of the value from that into serving students with different physical abilities. It's been happening, but this is ramping it up because we will have a lot more remote opportunities and people have also talked about how we may be better at preparing students for the field work, meaning that once they're in the field you're spending a bit less time just dithering around and be able to get a bit more down the road of critical thinking in the field. Those have been proposed. Let's hope that they happen, but people have recognized those possibilities.  So I was just gonna say that the geo-RU community has been working really hard together to figure out what to do. And so a number of the RU's are gonna run as virtual RU's this summer. And I think that, so that's, I mean, I'd say that people are acting as quickly as possible and they are being motivated. And I think there will be some really good ideas and techniques that come out of this summer because they're looking at how to maybe pair students together and pair faculty together so that they can actually still carry out their research over the summer while even though it's totally very virtual. Okay, great. Let's thank all of our speakers and panelists and thank you for some really interesting talks and a great discussion. We're not gonna take our break. It's not quite 30 minutes long. I think we are coming back as scheduled at 4.30 p.m. So we've got about 20 minutes break. So we'll take that break and come back for our wrap up at 4.30 Eastern time, bottom of the hour, whatever time zone you happen to be in. Thanks a lot.