So thank you. Is the microphone sound OK? Yes. OK. So today I'll be talking about hurricane forecasting and risk communication. And I'll be talking about the past and the present and then looking into the future about where some of the changes are happening today. So before starting with the main part of my talk, I wanted to review some basic concepts about hurricanes to help people follow along in the presentation. So this shows the Saffir-Simpson Hurricane wind scale, which represents different types of hurricanes. So on the left here, you can see the category of hurricanes one through five, where five is the strongest and one is a relatively weak hurricane in terms of the wind speeds. You can see how the wind speeds grow up as the category increases. And on the right-hand side here, I've put some example hurricanes that some of you may have heard of in the past and that I'll be talking about today, just to give you a reference point of how strong these hurricanes were at Landfall. I remember when I first started learning about hurricanes and going to hurricane research talks. People were always talking about different names of hurricanes and everyone seemed to know which hurricane happened when and where it was and everything like that. So you don't have to feel like you have to remember these hurricanes. I'm just going to give you an idea in case some of them you remember or want to look up later. And so hurricanes are a type of tropical cyclone. So in the Atlantic, strong tropical cyclones are called hurricanes. And you can see here at the bottom that before a hurricane becomes strong enough to become a hurricane, it's a tropical cyclone called the tropical storm or tropical depression. So on the right here, this shows a satellite picture of Hurricane Irma from this last season, 2017. The wind rotates counterclockwise around the hurricane and you can see the eye that's in the center of the hurricane. And this shows tropical storm Irma. So the same storm Irma earlier in its lifetime before it become a hurricane. So you can see it still has a similar sort of circulation, but it doesn't have a well-formed eye yet. And the circulation isn't as strong. So this tropical storm later became a hurricane. So that's how hurricanes form. They're usually tropical depressions, tropical storms, and then they can strengthen along the Saffir-Simpson scale. So I wanted to talk about how tropical cyclones tend to move across the ocean because that's valuable context for what I'll be talking about today. So this shows the tracks of the 2017 storms in the Atlantic, in the North Atlantic. And as you can see, the TD here stands for Tropical Depression and the TS stands for Tropical Storm and then hurricanes. And then a major hurricane is one of the stronger hurricanes on the Saffir-Simpson scale. So you can see here that tropical storms and hurricanes tend to form along the coast of Africa as waves, and then they strengthen, and they move westward across the tropical Atlantic. And then they often go north and then recurve across the Atlantic as they strengthen. And then some also either travel into the Gulf of Mexico or they strengthen in the Gulf of Mexico. So this gives you the general idea of the tracks of hurricanes they tend to go like this in the Atlantic. So this is a picture of the United States. So kind of scrolling in a little bit, you can see Texas down here and Florida and the east coast of the US. And I wanted to give you some geographical reference for some of the storms that I'll be talking about today. So I'm going to start with the Galveston Hurricane, which happened in 1900 here in Texas, south of Houston. And then I'll also talk later about Hurricane Ike, which happened in 2008. And then Hurricane Harvey also hit this region this past season. After the Galveston Hurricane, I'll go back and talk about Hurricane Camille, which affected the Louisiana and Mississippi coastline in 1969. And then some of you probably remember Hurricane Katrina that affected Louisiana and Mississippi about a decade ago. Then there was also Hurricane Irma that I just mentioned from this past season, which affected Florida. And then Irma also affected the Georgia and South Carolina coast in North Florida, as well as the Caribbean. And then Hurricane Maria, which affected Puerto Rico and other areas of the Caribbean in 2017, so this kind of gives you reference of the different kinds of storms. And then Hurricane Sandy, I'll talk about briefly today, which affected the New York and New Jersey area in 2012. Actually, officially, Sandy was not a hurricane at landfall. It had transformed from a tropical system into a non-tropical system. But it was a hurricane shortly before landfall. And it was equivalent to Category 1 of the Safferson scale. And as many of you probably remember, caused major damage in the New York and New Jersey and surrounding areas. So I wanted to start by talking about what is called the Quiet Revolution of Modern Weather Prediction, as described by a paper by Bauer et al. in Nature, the Nature Journal. So in the meteorological community, we're very familiar with the fact that weather forecasts have improved a lot in the last decades, and they're continuously improving. But many members of the public don't think about this. But if you think back 10 or 20 years, you can really understand how much forecasts have improved. The things that we couldn't forecast 20 years ago really can be forecasted today. And so this diagram shows this. On the x-axis, we have a measure of skill in weather forecast modeling. So as you go up along the scale, it increases in scale. This is output from weather models, which are used by forecasters. And you have the year along the x-axis. And you can see here that if you look at a forecast that would say five days in advance, the skill is steadily improving over time. And then a five-day forecast now is about as skillful as a three-day forecast was 10 years ago. So this indicates the improvements in forecast skill that are steadily happening. And I'll talk about some of the reasons underlying that in a minute. And these improvements have really transformed weather forecasting, communication, and decision making. So if you think about the kinds of decisions people can make in terms of hurricanes, which I'll talk about, or tornadoes, or winter weather, or all those kinds of things, of course, not every storm is forecasted well. There are still forecast errors that I'll also talk about. But in the last 20 or 40 years, there's really been a revolution in how well we can predict whether forecasts underlies what we'll be talking about today. So now I'll go back in time and illustrate that with examples from hurricanes. So this is about the Galveston hurricane, which affected Texas in 1900. And this is an excerpt from a journal that calls it the West Indian Hurricane of September 1900. This was before hurricanes had the naming convention that they do today. So this talks about how measured by loss of life and property and the pressure drop in the barometer at Galveston, the hurricane of 1900 was a severe storm that ever occurred in the United States. On Galveston Island, more than 6,000 people were drowned or killed, and the estimated value of the loss was $30 million, and that's in $1,900. So it was a devastating hurricane. So if you think back to 1900 and wonder what kind of storm warnings were available, they did have some warnings. This shows some quotes from a description of the storm written by Isaac Klein, who is the main forecaster there at the time. He said, storm warnings were timely and received a wide distribution. They had storm flags flying. So this is an example of what the storm flags looked like. This is actually from just a few years ago. This is not a picture of 1900, but this is the flags that they still use today to indicate to mariners and to local populations that there's a hurricane. Of course, back then, that was a major form of communication. There was no television or internet or anything like that, cell phones, of course. They did have telephone, so they had a person at the forecast office who was constantly on the telephone, giving on information. People were calling in, and then people came to the Weather Bureau seeking advice. Isaac Klein talked about going through Galveston. So the public was warned over telephone and verbally, and this all happened the day of the storm. So the storm affected Galveston that evening. And during that day, they were using the phone and using mouth-to-mouth communication to communicate that the storm might come. And people were advised to seek secure places for the night. So they did have some warning. The storm had previously affected Cuba, and there was some debate after that over where it was going because they had no satellites. They had no observations over the ocean, unless a ship happened to pass through and then go back to land and talk about the storm and tell people about it. But there was a ship in the Gulf of Mexico that had observed the storm a couple days before it made landfall, so they knew that it from Cuba had passed into that region. But they had no idea how bad it was going to be and really what was safe. So this shows kind of the state of the art of the neurological observations that were available at that time. This is from a barometer, a pressure tracing, that's shown in the same article. And you can see the pressure drop as the eye of the hurricane approaches. And this shows the readings. So this is the data that they had. In fact, it says in the article that the person who took these pressure readings as the pressure was dropping so fast did this because he was worried his instrument wasn't working. So he took these readings very quickly, very quickly as the storm was approaching. And then they had observations of the sky. They would look and look at the clouds. And in this article, there's also descriptions of the clouds in all of the areas in the Caribbean that had observations available. And so they also talk in this article about how the usual signs which herald the approach of the hurricanes were not present in this case. The brick dust sky was not evidence in the smallest degree. So they were looking for this brick dust sky that they thought would be the sign of an approaching hurricane for a couple of days and didn't see it. So that was kind of the state of the art of how people knew a hurricane was coming in those days. But they did have some warning. So what happened when the hurricane hit land and its winds hit land? So Isaac Klein in his description, he talks about how at around 6.30 PM there were strong winds and the roofs of houses and timbers were flying through the streets as though they were paper. And he was talking about people killed because of that. And then he had actually, his house was on high ground in Galveston, which he thought was a safe location. So he had his family there, as well as told other people to shelter there. But what happened is his own house was overrun by water. And he and his family were left floating on timbers in the ocean for a few hours. So he talks about estimating the tide and how he tried to estimate even after it surpassed his house. And this is the storm surge, which I'll talk about later, that comes in with the storm. So these different hazards, the strong winds, as well as the flooding that comes with the hurricane. And this shows a picture of Galveston afterwards, really. In many areas, there was devastation. Everything was destroyed. So in this article, Isaac Klein talks about how they had the hurricane of 1875 in mind. And that's why they thought certain areas were safe. But no one dreamed that the water would reach the height observed in that storm. They couldn't have imagined that the height would or take his own house and basically most, even the high ground in Galveston. And that they will continue to make improvements. And then he talks about how a seawall, he thinks that would have broken the swells and would have saved much loss of both life and property. And this will come up again later in my talk. They did actually build a seawall that has protected Galveston from some storms, but it doesn't protect Galveston from everything. So the idea about the seawall goes back to this 1900 storm that was so devastating in the area. So if we fast forward about 70 years, I'm going to talk about Hurricane Camille. Hurricane Camille was one of the first major hurricanes in what we call the modern satellite era. So when there were satellites that were able to see the ocean and see the storms beyond just a ship or an island happening to pass through a storm. So this is a newspaper front cover after Hurricane Camille made landfall. And the reason I picked this article is because on the right-hand side, you can see here, it says Spiroagno is to view storm damage. I thought that was interesting because you never hear about Spiroagno doing anything. And then also, I mean, at retrospect, being someone who's sorry, who was born after he was vice president, but then it said below that President declares four parishes, major disaster areas. And for a while when I first saw this, I was worried that Spiroagno was president, and I totally misunderstood all of history, but he was just vice president. It talks about Nixon down below. So anyway, the vice president went out to view the storm damage. So it's also an indication of how in some ways things are similar today. So a storm happens. They declare disasters. They send the president or the vice president out, and they're going on how important they think it is, I guess, or the political will in that part of the country who's more likely to vote for which kind of person. But so if you are interested in the types of warnings that were available at this time, they were basically text-based warnings. So this shows some examples of the warnings that were issued around this time. This first one is about two days before landfall. And it talks about the hurricane. They anticipate the hurricane will affect the Northwest, Florida coast. And then there's this statement here that no information has been received near the center of Camille since early this morning. And it's 1 PM. Another reconnaissance plane is approaching the storm, and more details will be available later this afternoon. So they did have reconnaissance aircraft then, where they would fly aircraft in and towards the storm so they could get extra data. And so this is another advance beyond the satellites that they had beyond 1900. This next forecast right here is about 12 or 15 hours before the storm made landfall. And they anticipated earlier on that the storm was going to hit Florida, but about 12 or 15 hours before the storm made landfall, they realized it was gonna go further west and affect New Orleans and the Mississippi coastline. And so the people in those areas had about 12 or 15 hours of warning, and they evacuated many people during that timeframe. It's interesting actually, the amount of warning available to this population is only a few hours longer than was in the Galveston hurricane. So there was more information about what the storm would do and more knowledge and of course better at ways of disseminating information. And so the forecasters afterwards in reviewing this storm wrote about how they had 15 hours of warning for most of the segment of coastline affected, and that the forecaster basically the state of meteorological science at that time. So they said, we would have liked to have more warning, but this is really where meteorology, what meteorology is capable of at this time. So this shows what they did have radar, so what radar looked like in those days. So these are radar images on the left. You can see they look really different from the radar images you see today. You can see the eye of the storm and they were pretty excited about that, that they could track the storm as it came in. And then this shows a composite that someone produced looking at the radar across the different areas where the radar was available. And this shows a satellite loop from that time. It's actually a reconstructed satellite loop. So at the time they had individual frames of satellites, but they weren't able to view it in motion. Someone reproduced this later. And so they would have the satellite and there would be someone looking at a screen that would be able to see something and then call someone and tell them about it. So they did have satellite that was able to tell them there was a hurricane, but you can see that compared to the satellite images available today, it was pretty crude. This is the kind of communication they used back in those days. This is radio communication, which they said was very effective. They did have television. They did even affect color television back then. And so this shows an example of a broadcast from the Emergency Operations Center in New Orleans. So when the storm was approaching that day, they did have broadcasts on television telling people that they should evacuate, discussing the evacuation and so on similar to today. Although of course the broadcasts were much coarser and they had less information. They didn't come out as far in advance as they do today. So now we fast forward again to the 2017 hurricane season. So this shows what the satellite imagery looks today. And of course you can view loops of these as well. This shows three of the major storms of the 2017 season. This is Irma as it approaches, goes to the Caribbean and approaches Florida. And then this is Hurricane Jose and Katya. So this doesn't even show some of the other major hurricanes of the 2017 season, Harvey and Maria and other storms. But this shows what the satellite imagery looks like today. So as far as the forecast go of the storm today, this shows an example of one of the most commonly used forecast images for hurricanes. It's the cone of uncertainty graphic that's produced by the National Hurricane Center. And so the black line with the dots in the middle shows the anticipated track of the storm. And as you can see, it goes, looking for my mouse, come eventually. It's anticipated to hit Florida and then go north. And then the white represents the uncertainty. So where the track of the storm might go. The red represents the areas that are under warnings. And this graphic has been around for 10 or 15 years. And there were many critiques of the graphic over the years that people keep still using it because they like it a lot because it communicates at least some major things about a hurricane. But one of the criticisms was that you couldn't tell how big the hurricane wind field was. So this shows the hurricane wind field. So the ideas are supposed to superimpose that over where the storm is going to kind of anticipate where is at risk. Because otherwise you don't know around this dot here, sort of which areas are at risk. So this is one of, as I mentioned, one of the flagship products of the National Weather Service. And compared to, say, 1969 or even 1900, there's forecast information available much further in advance and it's updated much more frequently. So this graphic here is the same as I just showed. And this is about 12 or 15 hours for landfall. So in the keys. So similar timeframe to the other storms where they had the warning. And then of course there's forecast later in time as the storm evolves. And if you go back in time, you can see that several days in advance, really the storm was forecasted to go to the west and to approach Florida. And this is about five days in advance. You can see that the forecast were actually pretty good in this case. There was pretty good knowledge that the forecast, the storm was likely to affect Florida. It just wasn't clear which part of Florida. And of course, since Florida has a large population. Oops. Since Florida has a large population, that is a lot of, that can change evacuations which areas need to evacuate a lot. So of course the National Hurricane Center and the Weather Service produces kinds of graphics. And then there's all kinds of other images that are produced in the media. So this shows some other examples of the cone of uncertainty graphic that were available in the media. So there's all kinds of other kinds of graphics interpreting the same kind of information. And then there's information about other hazards like tropical storm force winds, storm surge flooding, of course all kinds of different graphics and text messages are related to that. And of course today we have people that are getting information much more complex ways than they were in 1969 or even 10 years ago. People can look up information on their computers, on their cell phones, on their tablets, as well as on TV and get information through social media and through people that they know or don't know at all or just met from across the world. So not everyone of course is getting information through these ways, but a lot of people are. And so it really illustrates how science and technology have really changed hurricane forecasting over the last 100 years as well as hurricane risk communication. So I wanted to talk briefly about why weather forecasts have become more accurate, some of the reasons behind that. So this graphic here shows the global observing system today. So if you think back to 1900 when there were observations of pressure and of the clouds, now we have satellites, we have aircraft, we have ships, we have weather balloons, all networked together. We can get information very rapidly from all of these different kinds of observations. We also have sophisticated computer weather models where here you see an example of a weather modeling grid. So what the weather model does is it takes information and puts it on a grid like this and then it uses equations and integrates those forward in time to predict the future. And then a key part of this prediction process is what's called data simulation where it takes all the data from these observations and combines it with other information to produce the initial conditions for the computer modeling. So you have this data simulation that uses the observations to create the initial conditions for the model that are then integrated forward in time. And of course that all happens on supercomputers, so as computers get faster and technology gets better, you're able to do more with the observations and the predictions. And then in conjunction with this, there have been a lot of advantages in scientific knowledge, both using all of these kinds of data and information as well as this scientific knowledge feeding back into the advances in forecasts. So together, all of these different factors mean that today's weather forecasts are more accurate, more detailed, they're available further in advance and they have better representations of the confidence or the uncertainty. So as I'll talk about in a minute, weather forecasts are inherently uncertain and so part of modern weather forecasting is estimating how uncertain the forecast is or what we can say with confidence. And they're also continuously improving as our technology advances, as our computers advances, and as our knowledge and our ability to model these things advances. So I wanted to talk briefly about uncertainty in weather forecasts because it's important for the context of the rest of my talk. So some of you may have heard of Ed Lorenz and Lorenz Attractor and Chaos Theory, which Ed Lorenz was a meteorologist, but he was also one of the founders of Chaos Theory. So when he was doing some simplified atmospheric modeling in the 1960s, he sort of found some things that ended up really contributing to Chaos Theory. And one key aspect of this is that trajectories that are initially close in a physical state will diverge in these kinds of chaotic systems. So Ed Lorenz found this in the atmosphere and this is one of the reasons why weather forecasts are uncertain, that even if you have initial conditions that are really close to each other, so you have initial conditions of your model that are very close to the atmosphere, the real atmosphere, but not exactly like it, that will diverge from the real atmosphere over time. And you'll end up with a forecast that's different than the actual atmosphere. So what this looks like in weather forecasting is often depicted in terms of these ensemble forecasts or spaghetti model plots where you have different trajectories from different models or the same model started with different initial conditions. So you can see here on the right, this shows the initialization of Hurricane Irma about a week before landfall. And in this case, it's different models. These all represent different atmospheric models, but you can also do this with the same model in different initial conditions. And you can see that they track together closely for a while and then they diverge from each other. So this shows about seven days out the different ways that the different models think the hurricane might go. And it's related to this divergence of trajectories in these simplified systems connected to chaos theory. So if you move forward in time, you can see that here's the spread in the ensemble forecast about seven days before landfall. A few days later, you can see their spread is still there. It's still kind of around the east coast of the US, but the spread is smaller. And then a couple of days before landfall, it really narrows down to West Florida. So you can see that this uncertainty is evolving with time and it's dependent on the situation. And so one of the advances in modern weather forecasting is to be able to predict this uncertainty as well as predicting the best likelihood of what might happen, predicting the uncertainty around that. And kind of when you can say with confidence that the hurricane is likely to hit West Florida. So it's also important to know that hurricanes are about more than wind. And the examples I talked about really illustrate this. So when the hurricane approaches, a lot of people think about strong winds as the main hazard. And that near the eye of the hurricane, that can cause a lot of damage as well as flying debris. So as the winds hit and they hit one thing, trees or other people's houses start to fall apart, that can also create a lot of damage. Another major hazard that's less well appreciated often is the coastal flooding from storm surge, which as I talked about was a huge hazard in the Galveston hurricane. So these two diagrams on the right show what storm surge looks like. So the storm surge is basically water that is pushed by the hurricane winds and the pressure from the ocean over to land. And this simulation on the bottom right shows a simulation of storm surge as it affects the coastline from some modeling that we've done. And you can see the inundation come from the ocean to the shore. There can also be inland flooding from heavy rain. So in Hurricane Harvey, this was the major hazard in the Houston area. There was storm surge and strong winds near the area where the hurricane initially made landfall. And then later on, there was inland flooding from heavy rain in the Houston area and other areas. And this coastal flooding from storm surge and inland flooding can interact as well. So if you have water coming from the coast and then you have a river coming down, you can have flooding come from the interactions among those effects. And sometimes the coastal flooding can really go miles inland if the land has a very low slope. And you can also have tornadoes that are embedded, especially in the eye of the hurricane or the outer rain bands. So tornadoes can be a risk even within the high winds of the hurricane. And then of course you have all of the secondary effects. So all of these initial hazards that cause disruption to power, transportation, communication, medical services, sewers, water supply, all those kinds of other things, all that critical infrastructure. And that can also cause a lot of damage and misery and so on. So you can see this if you look at the major causes of loss of life from US tropical cyclones. So this was an analysis that was done about five years ago. And you can see that the major cause of the deaths from tropical cyclones during this era was to storm surge and then rainfall flooding. So winds do cause some deaths but storm surge and rain are some of the major causes of death. And often what happens is there aren't that many hurricanes that cause a lot of deaths in storm surge. But when there is a big storm surge, it does kill, it can kill a number of people. And then people have also analyzed the indirect deaths. So the deaths that occur from when people evacuate, they can die saying car accidents or for other reasons. They can die from loss of power, heart attacks, all those kinds of other things. So this on the right shows the different kinds of secondary effects from infrastructure failure and other kinds of things. And then of course there are complex situations like Hurricane Katrina where people die for not really clear reason. There are also convolutional factors. And I think as people analyze Hurricane Maria in Puerto Rico this year, we'll find a number of deaths like this where it's really a combination of factors. It's not the immediate impact of the hurricane so much as the things that happen afterwards and the lack of services, a lack of effective response to help people. So I've talked about the advances in hurricane prediction. So now I wanna talk about some of the advances that have been made in hazards research and risk communication research that have helped to reduce hurricane impacts. So this diagram on the bottom shows what's called the Four Faces of Emergency Management or the Hazards and Disaster Cycle. So this talks about an event that happens and after the event there's a response phase where people are coming in to rescue and kind of clear the roads and things like that. Then there's a recovery phase where the community is rebuilding and trying to get schools open and businesses reopened and people back in their homes. There's a mitigation phase where they're trying to mitigate for the next storm. So for example after the Galveston hurricane there's a mitigation phase where they say okay, what can we do, can we build a seawall that will help us next time and so on or stricter building codes or other kinds of things. Then before an event happens there's preparedness and warning phase where there might be a warning of an upcoming event and people are preparing and then an event happens and you go back through the cycle again. Of course in the real world all of these overlap but this is kind of one way that people think about hazards and how they affect society. Of course the big reason we're concerned about this is because the event can have significant negative impacts. So they have these hazard impacts that are death, damage, disruption, misery, all kinds of other things. And people can take protective action such as evacuating or boarding up their homes or so on that can help reduce these hazard impacts. So the purpose of the hurricane forecast and warnings from the societal perspective is that they provide information that can help people make better decisions that protect themselves from hazards. So this is how hurricane forecast and warnings are connected in with hazards and decision making and impacts. That provides a simplified view. If you look at a lot of the risk communication research that's happened in the last 30 or 40 years what's really key in this is how people interpret and perceive the risks and there are other kinds of beliefs that affect the protective actions that they take. So people get information and they combine that with their experience. There are other kinds of knowledge. What do they think about risk? Do they like to evacuate? Do they have a car? How much money do they have? Do they have a place to go? All those kinds of things that affect their protective actions. And so with that in mind a lot of the research that I do focuses on this piece, the communication. How can we take the hurricane forecast and warnings and communicate them in ways that help people interpret the risk, understand the risk and take better actions that can help them reduce the negative impacts. And so the way we do this is we go backwards we think about the actions that people can take and that we would like them to take to protect themselves and how they interpret the information and use that to inform the creation communication of information. So because of this there's been a huge revolution in the last 10 or 15 years in the meteorological community in terms of interdisciplinary research to understand and improve the creation and communication of forecast information. And the work that we do is interdisciplinary in the sense that it's in the context of the current and potential forecasting capabilities. So we work closely with atmospheric scientists and some of us are atmospheric scientists where we really think about what are the current forecasting capabilities and what's possible in the future and really connect that with what people can do and what they need to know to be able to make decisions and bring that all together. Some of the research methods that we use include surveys that shows an example of a survey that we conducted a number of years ago that was in this case it was an online survey where people could answer a series of structured questions that we can then analyze to tease out things like how do they respond to different messages? Why did they respond in the ways that they do? How do different kinds of people respond? Those kinds of things. We conduct interviews. So we talk to people face to face and we ask them again a structured series of questions where we can then analyze the data to understand why they're thinking the way they do and how they respond to different kinds of information or what kinds of information improvements would be really valuable to them in the future. We conduct focus groups which are a series of guided conversations with a group of people where we can talk with them about about what they would think about different kinds of information or the kinds of decisions they made in a storm and so on. And this shows an example of several of my colleagues conducting a focus group where they were presenting different kinds of risk information to a group of people and discussing with them what they would think about it, what they liked about it, what should be improved and so on. We also do studies embedded within the surveys and interviews and focus groups where we change messages and see how people respond. So this shows an example on the right where we did some work with the National Weather Service where they were concerned about some communication on their webpage. And what we found was that actually the biggest problem people had was not the problem the weather service initially thought but was that they had difficulty when they were looking at the webpage seeing if there was some kind of warning or severe event they needed to be concerned about right now. So we tested these two different ways of communicating this information that are shown in yellow, this red box around the forecast, the current forecast, as well as this text below that people could click on. And we tested different ways of communicating this and found the one that worked the best which the weather service then implemented. And then we can also do computational modeling. So this video shows an example from a model that we've produced that simulates people getting information and exchanging information with each other and making decisions as a hurricane approaches. This example is actually from when Hurricane Irma was expected to go along the east coast of Florida. That's when this was created. So that's why it goes there. And then also another area that we've been working in for the last five or 10 years is analysis of social media data, particularly data on Twitter because that's publicly available unless people decide to make it private. So this graphic shows the number of hurricane keyword tweets. One of the ways we can collect data from Twitter is by collecting the tweets that mention either hurricane or the word hurricane in Spanish as well as the hurricane storm names and certain places that are affected. So you can see in this graphic kind of the ebb and flow of the tweets that talk about hurricanes as the season progresses and as the different storms make impact. And then within that, there's a lot of different things going on. This is in the millions, this Y-axis. This is a million tweets down here and this is up to nine million tweets per day. So it's a lot of data. But you can also find things within that where people are saying things like this is a forecaster who communicated these graphics about the risk of the storm and someone responds right back saying, could you please tell me if I need to get my family out of this mobile home and says where they are? So you can really see from this what people are thinking about doing, how they're interpreting information, all those kinds of things as the storm approaches. So I've talked about kind of where we've come in the last hundred or more years in hurricane forecasting and communication and kind of what are some of the areas of research going on now and how advances in science technology have really made a tremendous revolution in terms of providing more accurate and more detailed weather forecasts further in advance as well as estimates of uncertainty. I've also talked about how advances in interdisciplinary research have improved weather risk communication in ways that support people's risk interpretations and their decisions. And now we'll talk about some of kind of the key areas in weather forecasting and risk communication right now and going into the future. So the first one I'm going to talk about is work to improve forecasting and communication of hazardous conditions and impacts given predictive uncertainties. As I mentioned before, weather forecasts are inherently uncertain so this makes it challenging to predict exactly what's gonna happen a long time in advance but people also need to know what's gonna happen to them in order to make decisions and so we'll talk about that now. So to do that I'll talk about some research I did with interviews after Hurricane Ike which happened in 2008. So on the right here you can see the area that Ike affected and where we conducted interviews. So this is Galveston Island which is where the 1900 hurricane made landfall. So we did interviews in Galveston and then in this area nearby on the coast called Kima and we were interested in Hurricane Ike because it was a large category two storm so category two is not that strong on the Safferson scale. It's still a strong storm but not the winds aren't as strong as they could be but because Ike was so large geographically it covered such a large area it was predicted to produce a very large storm surge so it was predicted to produce a 20 foot or higher storm surge in the Galveston, Texas area and this was a big concern because the seawall is 17 or 18 feet high and so they were concerned that the water would go over the seawall and flood Galveston and there were reports that a lot of people in Galveston weren't evacuating so in advance of the storm the weather service and the local community officials were trying to get people to evacuate and the weather service even issued a message that said if you don't evacuate and you live in these kinds of areas you will face certain death or you may face certain death that's how concerned they were and so these messages were broadcast and we were watching this where we were in Colorado and thinking oh no what's gonna happen and so this graphic this picture on the left shows what did happen in some areas you can see this one house standing and basically everything else is gone this house had been rebuilt after a previous hurricane of course this house was not livable it was filled with mold and water and mud and debris and everything else but it was still there so what actually happened was the storm track changed a little bit and this devastation happened further to the east and so Galveston did not experience 20 foot storm surge it did experience a significant storm surge but it didn't go over the seawall what happened was Galveston's an island so the waters came around and went from the other side and flooded Galveston from the other side so we went down to Galveston and Kima as I mentioned we interviewed people who were in those areas that were cleaning up their homes we didn't actually go to areas like this because there weren't any people there because they couldn't be there when we were back there a month or six weeks after the storm but there were people in Galveston and these other areas that were coming back and fixing up their homes so we interviewed people about how they got information what they thought about that information how they decided what to do what they thought about the risks to kind of see what had happened and see if there were ways where we could use what is known in the weather community about how the forecast can be improved to improve the risk communication so this shows some examples of how hurricane forecast information was used about 10 years ago so on the left you see some results from a question we asked about which information sources people used so you can see that television especially local television was a very popular source of information and it still is today when there's a hazard people often have preferred local broadcasters that they will watch and so on to learn about the storm people watch satellite and cable TV radio people at that time at least still use the newspaper there were a fair number of people using the internet even back then 10 years ago this source right here is cell phone or PDA which is like a palm pilot or a blackberry for those of you that remember back in those days 10 years ago before that most people had smartphones so there were still a few people using those of course this has changed a lot in the last 10 years the availability of information from these sources and of course people used family and friends and other sources and then we asked people how frequently they checked forecasts several days in advance you can see here that about a third of people so they check forecast constantly so we're really getting a lot of forecast information and actively interpreting it and deciding what to do so if you think back to the way I talked about the past storms and the way the sort of hazard field has thought about warning communications say 30 or 40 years ago it was really this vision that there is a warning people get that warning and they decide what to do they might ask around and so on but it's really this sort of linear process there's an official source that provides a warning and then people decide what to do but you can see from this picture that people are actively using information they're getting a lot of information on their own they're really making decisions about their own risk and our data also talks about this so people do pay attention to evacuation orders some people do but a lot of people are really evaluating their own risk and making decisions I remember one person who told us that he heard the forecast of storm surge was 15 feet and he said my house is at 11 feet above sea level I'm more than four feet tall I'm good so people are really thinking about information and making those kinds of decisions and he was fine so he was right I guess but this is sort of an example of how active the use of hurricane forecast information is today so people are using the forecast information they were getting a lot of information about the storm as it approached days in advance what do they think about that information how did they make decisions about what to do so we found that many interviewees believed their property might be at risk so they got the information they really thought that they might be at risk and most of them prepared their property before the storm arrived and most of those people prepared for winds they boarded up their home they put up their hurricane shutters they did other things that would prepare their home for the strong winds and only about 25% of people in these two areas which were all of which were flooded prepared for flooding and a lot of those said they didn't prepare adequately for flooding so they might have taken something from the ground floor of their home and moved it up to the top of bookshelf but the flooding was so high that it flooded everything and so it was really this tragic example of people who did have information about the risk and they did try to prepare but they didn't know the right thing to prepare for and they didn't really know how to prepare far enough in advance and so if you look at what happened during like these are some pictures that my colleague and I took after the storm made landfall when we were there this shows the Galveston Seawall right here and this is on the ocean side of the seawall so you can see that the water came up here and sort of devastated this building as I mentioned it didn't go over the seawall so it didn't flood this area this area was just affected by winds but it just came around the other side and then it flooded up to seven feet of water and through a large portion of Galveston and so this shows what's left after that if you had a ground floor home and it was flooded up to seven feet you had all this debris lying around in your home everything up to seven feet in your home was flooded and moldy and ruined and if you only have a one-story home then that's basically everything so some people we talked to lost everything and they'd really tried to do their best before the storm to prepare so there are some different reasons that as I note here a lot of people told us they never expected flooding to be so bad they never dreamed of seven feet of water and that goes back to the 1900 Galveston hurricane you remember that quote that's very similar people just couldn't imagine the flooding would be that bad at their home so there are different reasons that people thought the flooding wouldn't be that bad there was different kinds of flood mitigation that had happened including the seawall but other mitigation with the river channels and so on people thought that that mitigation had reduced the risk of flooding they didn't know that this kind of storm surge could flood most of the island and then another big issue was that because storm surge is difficult to predict far in advance the hurricane forecasts are coming out five, seven even more days in advance you can see the storm approaching on satellite people know what hurricanes associated with strong winds but the and so people who are knowledgeable about flooding know that Galveston is at risk of storm surge flooding from a big hurricane but the specific forecast for the storm of storm surge flooding didn't come out until maybe 24 most 48 hours in advance of the storm and by that time a lot of people had left so people said I heard these forecasts of flooding but I was already gone and already prepared and so this really motivated some of mine others interests in trying to figure out how we could issue these forecasts of the hazards further in advance so people could make better decisions and protect themselves so this shows some examples from some of our other research about the same idea so as I mentioned you know in these interviews we hear these people say things like I never dreamed of seven feet of water and this is an example from a focus group talking about storm surge we knew about the risk in our neighborhood but the way it came in this time it was just mind-boggling they couldn't have imagined it would be that bad and then we have surveyed data where people in response some of the information that we've tested they said they would want more specific numbers like wind speed and height of storm surge so just having a general area at risk or that it might have storm surge is not enough for them to know what to do and then they want this local information so they want to know what's going to happen in their area because often the forecast cover a large geographical area because that's sort of the state of art of forecasting these specific kinds of hazards and this shows an example from our Twitter data here on the left where someone that's in our data set gets a tweet that from a broadcast or who's on Twitter a broadcast meteorologist and the broadcast meteorologist says storm surge may reach eight feet with 10 to 20-foot waves on top and the rockaways are set for major flooding so this forecaster is saying listen it's going to be bad if you live in the rockaways so this person who lives in the rockaways this is an area of New York City he retweets this showing that he's got it and then he tweets right back saying I'm two blocks from the beach to find major flooding please so the forecaster thinks he's communicating pretty clearly this is going to be bad and this person is like hey what do you mean for me so these are all examples of how people want to know what kinds of hazardous conditions and impacts they will experience and this raises important questions about how do we forecast this information and how do we communicate it given the limits of predictive scale so it's difficult to predict locally specific information about these hazards far in advance and so how can we provide this information to people in a way they can understand so I'll show some examples of some work that we're doing now to test this so on the left here this is a map of potential storm surge flooding and this is similar to the kinds of maps that are often conveyed right now from the National Hurricane Center and the media that shows areas that might experience flooding of different heights you can see it covers a large geographical area we have you can blow these maps up and see like in your region what kind of flooding might you experience people still had trouble kind of translating what this information might mean for them so we tested things like with this scale of flooding showing what this one to three feet or three to six feet or six to nine feet might mean and some people really felt like this was useful they could see okay well here my car's fine at one to three feet you know it's not under water here yeah I should have laughed a while ago that's not good of course not any kind of message is perfect some people had concerns about this like they would say things like well that isn't like my house that's not going to affect me so a lot of it does depend there's no silver bullet that answers everything but this is one kind of communication that we tested another that we tested is simulation this simulation shows from our storm surge modeling how surge evolves in a neighborhood over the course of about 12 minutes you can see the water coming in and then inundating the cars and the buildings and so on and as one person said in response to this information he said I think a map couple with the video would make a bigger impact because you can look at a map all day you can try to find your street but this is definitely telling you how fast is going to happen so we did this in response to some of our earlier focus groups where we found that some people just didn't know that they wouldn't have had a day or you know six hours once the flooding started happening this really helped them understand that the flooding could happen very quickly of course the challenge with this is at the time scales that people are making evacuation decisions two or three days in advance we can't tell which locations we can't tell people which locations are actually going to experience this what we can say is there's a risk of this in these different locations and if you're in one of those locations as the storm gets closer we'll be able to tell you this is the kind of thing that could happen okay so I've talked about the advances in science technology and the improvement in weather forecasts as well as the advances in our study research and improving weather risk communication and that work is still ongoing today to improve all of those things and I've talked about improving forecasting and communication of hazardous conditions and impacts given predictive uncertainties and now the last thing I'm gonna talk about today is kind of this effort these efforts going on now to forecast and communicate in ways that connect with people's capacities in the complexity of the modern information environment and by the modern information environment and its complexity I mean the complexity of all the information I talked about that's available today where really we're inundated with information if you have a hurricane approaching and you wanna go seek information you can find hundreds about thousands of graphics and textual descriptions and images and videos and so on with information so there's all this information we communicate in all kinds of complex ways you can quickly go on Twitter or Facebook or other kinds of social media and ask one of your friends what's happening what do you think is gonna happen and have a friend of a friend of a friend rapidly respond to you and really use that as a way of interpreting the information and getting new information so this really complex modern information environment that we live in how can we kind of interact with that to forecast and communicate in ways that leverage that but also recognize that not everyone is interacting with the internet and social media and all of those things in the same way or sometimes at all so to illustrate this I'll talk about some analysis that we've done with social media data from Hurricane Sandy we also have some ongoing analysis from hurricanes from this last season so as we all know the internet and social media and other technology like cell phones and those kinds of things have really transformed the way in which many people access information exchange and use information and one example of this is I remember my phone broke about a year ago when I was actually in the airport about to leave on a trip and when I got back my phone still wasn't fixed and so one of my kids wanted to have a play date with someone else and I had to think like what would I do before I texted the parent would I call them and I was like email that's what I would have done email and I was like geez what did we do when I was a kid you would call on the telephone or you'd just show up at their house right but I couldn't even remember back to a few years ago when I would have just emailed the parent so that just an illustration of how fast technology changes and how fast it changes how we communicate and how we interact with information in addition to changing how we communicate social media provides a rear lens into how people interact with and respond to information as a hazard approaches and arrives so we as I mentioned before we do interviews we do focus groups we do surveys you can try to access people when they're evacuating using a telephone or people have done this by being on the roads where people are evacuating at a rest stop where they stop to go to the bathroom but it's really hard to access people when they're in the middle of an event really thinking about what to do and of course they're very busy they're packing their things they're deciding what to do and so on and so social media data internet data where people are posting about what they're doing in real time it provides this lens into what people are actually thinking at the time and how that evolves to the extent that they're telling us that and so this data is really rich as far as providing us a lens and things that we couldn't really understand very well before because we just didn't have the data to understand it other than a few people here and there and the extent to which they remember it after the fact and can describe it to us and the results illustrate the dynamic interactive nature of weather communication and decision making in today's world and if you think back to the 1900 Galveston storm it paints this picture of people talking to each other on the phone interacting with each other but they were doing that all in person and face to face and very occasionally over the phone it's very different today people can do all those same things but they can communicate with many people much more rapidly and a much wider diversity of people and can instantaneously communicate with someone across the globe it's really a different picture of how people can communicate so I showed this model earlier of how we study hurricane forecasting and risk communication it goes from risk information to how people think about that information how they interpret it and their beliefs about the situation and how they decide what to do and so this is often the kind of framework in which we test our messages we change one piece of information and we see what happens but really what we see in the Twitter data and in our other kinds of data is that all of these things are interconnected people are doing all of these things at the same time they can get new risk information very rapidly and they can interpret it and respond to it and often these are interconnected and you often can't separate them out so to show some examples of what we see about this modern weather communication system in social media data we do see evidence of the things I talked about before so people are mentioning hurricane forecasting warnings these are examples of tweets and this one on the bottom right here shows someone talking about landfall being expected Monday night at 6 p.m. so often they don't reference a specific forecast but this person wouldn't have an idea that when the storm was expected to arrive without forecast information so there's evidence of people getting forecasts and then there is evidence of people deciding what to do so this person is evacuating and they're telling us they're leaving Far Rock and they're giving us some little emojis hand emojis to tell us how they're thinking about leaving Far Rock which is the neighborhood that they're leaving in this case people also have a lot of they're giving us a lot of knowledge about what other kinds of information they use so they're using evacuation orders which we know and actually what we can see in this data is as soon as the evacuation order is issued there's a lot of tweets about it so people are getting that information and they're thinking about it people also look at what other people are doing so here's an example of someone saying I can hear the tap tapping of people boarding up their windows so they're listening to what other people are doing and using that as information about what they should do and then also using their other observations of the physical environment so what can they see what kind of clouds do they see do they, is it raining all those kinds of things what do they smell as evidence of whether the risk is real or not and so they're using all these different kinds of information in these complex ways and getting new information all the time they're also responding in much more complex ways in addition to deciding whether or not to evacuate or to board up their home they also need to pack and do other things so this person is talking about what they're packing and that's a kind of before you evacuate you have to pack and get your things ready a lot of people are exchanging information with each other so they're asking each other what's happening there where they are and what they're doing this is an example of that and then of course people exhibit a lot of what we call coping behaviors this is the example of cooking a lot of people cook big meals and they tell us about them they watch movies, they hang out with their friends they drink, all the kinds of things and so people are doing these other kinds of things where either they already have evacuated and they're trying to sort of help themselves cope with the stress created by their risk or in some cases and the research and risk communication shows this sometimes people can be so overwhelmed by their fear of something if they don't think they can do something effective they will engage in these coping mechanisms such as humor, this is also an example of humor people make a lot of jokes and things like that to try to minimize the threat so they can kind of get through it and then I just have one example here these interpretations, perceptions and beliefs but this is an example of a person who is saying my block doesn't ever flood so I'm fine, I'm just worried about something flying through a window so she's saying I'm worried about wind it's not flooding and there are a lot of examples of people talking about how they're interpreting the information and what they think the risks are and how that's influencing their decisions so we put this together and we get this really complex picture of course any given person has all these things going on at one point in time and they're evolving in time and sometimes we can see this using these data and then they're interacting with all kinds of other people so this really complex system where it's all these complex things going on with any one individual and that's evolving in time and interacting with lots of other individuals in their environment and as well as across the world that they're communicating on the internet or social media. So I'm gonna close here with reviewing my main points and kind of talking about them as a vision for the future not only for hurricane forecasting and hazard forecasting but for weather communication in general and weather forecasting so there will continue to be many advances in science technology that are providing more accurate more detailed weather forecasts further in advance and a lot of the research happening here at MCAR and at universities and other kinds of places is really focused on advancing this science technology to improve weather forecasting and improve different kinds of weather forecasts and also in NOAA and the National Weather Service there's a lot of people working on this and this will continue to improve and we'll see a lot of technological advances in this going forward. There've also been advances in interdisciplinary research that have improved weather risk communication and this is a growing area in the field right now and I think this will continue to improve as well and bringing these two together to really improve the forecast in ways that can improve communication. And I've also talked about two of the major areas that people are starting to work on now that I think are important going forward and the first is to improve forecasting and communication not only of the weather or of the hurricane that's coming and how strong its winds are gonna be but also the hazardous conditions and impacts associated with that. So where will there be flooding? How much flooding can there be? Where will there be power outages? Where will people not be able to access services for a long period of time? Those kinds of things and that kind of advance more specific advanced information can really help people prepare that can also help governments and communities prepare. So hopefully we have less of these catastrophic events like the Katrina's and the Maria's where people are left without services for a long period of time and it's really, there wasn't a planning to kind of get services available to repair things quickly later. But of course it's important to do this given the predictive uncertainty. So when you try to predict more specific conditions and more specific impacts further in advance as well as more location, oops, location information. When you try to do all those things there's more uncertainty. And so it's important to keep those predictive uncertainties in mind so that we don't over predict over predict confidence that in these kinds of things that we're going to try to predict. And then also they talked about towards the end to forecast and communicate in ways that really connect with people's capacities. They really meet people where they are in terms of the risk information they have, the ways they're getting that information and the people in their social networks that they're communicating with, how they're interpreting information, their prior experience and so on to really meet people where they are and communicate with them and collaborate with them to provide the information that's available in the scientific community in a way that can really help them understand the risk they face and make the best decisions given their circumstances. So I will stop there and take any questions. Thank you. Thank you. Thank you. Thank you.


Trinite Warrior 13 was looking at, I think, 18 experiment initiatives, different, not really different technologies, but different ways of applying existing technologies, tactics, techniques, procedures, and what we call non-material solutions. It's not about getting the latest sensor, the latest weapon, spending money to buy new things. It's trying to figure out how to use what we have more efficiently and more effectively. It's based on warfighter capability gaps that are presented by both Joint and Navy operational commands, notably Pacific Fleet, Commander Pacific Fleet, Commander Seventh Fleet, Commander Fifth Fleet, U.S. Air Force, and then the Office of Naval Research and various other sort of Navy stakeholders, and they say we have this capability gap. We need to be able to do something. Can we do something in the dot-mil-pf field with this doctrine of operation technology, material, planning, logistics, or personnel to solve this problem? We've been focusing on anything other than technology in terms of acquisition of technology. Experimentation is important to the Navy in many ways. One way currently that's very important is in the current fiscal environment, experimentation is a way to identify products or technology that need to go forward to the fleet and those products that need to be cut off and not waste any more resources on. Trident Ward 13 is a good example. There's 19 separate initiative experiments almost brought together in one venue and in that package we're reducing costs for the Navy.


Hello, Mod's gone. I'm very thankful to be back here with all of you today. This is one of my favorite events, because it's the only place I know where we can get this many nerds together in the same room. So I'm really happy to be back here. Today, we're going to be talking about something I call key performance factors. And it's something that I've been using a lot within the last couple of years to make better decisions over developing strategy and implementing audits. And so basically, to get started with, I wanted to tell a story because the last time I was here at Moscon, the biggest criticism I got from my talk was that I didn't give enough examples. And so this presentation is kind of based around two examples, really. I don't want to make that same mistake again. I'll make new mistakes this year, though, probably. So basically, let's start off with a story. Our first case study was a large e-commerce website. And they had around 75,000 product pages. They had already done SEO for quite a while. And they kind of knew what they were doing already. They had executed a lot of best practices. And a lot of their technical SEO was already pretty good. Their performance was not bad either. A lot of the product listing pages were ranking pretty well, product category pages as well. But they really wanted to more emphasis on the product pages themselves because they thought that that was one area that they were missing out on inorganic search. And so they came to me and they asked for an SEO audit and a new strategy. And so I did what I normally do. I did this huge audit for them. It had about 50 or 60 recommendations within the report. And each recommendation was prioritized high, medium, low, which is typical for most SEO reports. And then after I was done with the report, I sent it to them. And we did a meeting. And I reviewed the report with them entirely with their management team and technical team leaders. And they said to me, OK, this looks great, Joe. We understand why we need to be doing this. This looks good. But we're going to need to take some time to develop a project plan to implement the audit. And they said, would you be willing to answer some more questions after we do that? And I said, yeah, absolutely. Just email me. And so I got this email from them about a week later. And they said to me, Joe, we put together this project plan, we think, but we're still kind of at lost at which items on your audit should we work on first. And I said, well, the high priority items, obviously, right, like they're the most important. So do the high priority ones first. And they said, well, it's a little more complicated than that, and you're going to need to give us a little more direction on that. And so I said, OK, let's meet. And we had a meeting. And they pull up these particular items out of the report. And I had about, I don't know, maybe 10 high priority items. But these three right here were the three that they were really concerned about. And they said, which one do we do? And I said, all of them. Because why am I putting them in the report if I want you to ignore them, right? Optimization is something that we want our clients to be fully optimized, not halfway optimized or 75% optimized, with it all the way, right? I'm not satisfied until all the work is done. And they said, OK, we understand that. And we understand why we need to do each three of these things. But they said, you want us to create unique content for product pages. We have 75,000 product pages. And by the way, when I say create unique content, I mean they needed content. Like the only thing they had on the product pages was like the product title and the product image and the price. That's it. There was no product description, no specifications, no unique features, nothing. And I said, you need content on these product pages. And they said, OK, well, that's 75,000 product pages. We can do it. And we will spend the money to do it. But if we do that, we won't have anything left in the budget to execute the other three items on this list. And I said, OK, well, let's look at the other two then to see if we can do something easier with those. And they said, what about improved page speed? I mean, they had some pretty bad page speed scores. They had a very old legacy e-commerce system that hadn't been updated in quite a while. And I mean, that is always a difficult task to work on that task. But it's not unheard of that you can make headway in it. And they said, well, we took your recommendations on how to improve page speed to our tech team. And they told us that basically in order to get the performance levels that I was asking for, they would need to recode the entire website. And unfortunately, as a developer, that made sense to me. I thought, well, that's right. They need to recode it or get a completely different content management system. And that would take another chunk of money and maybe even a year of work time to execute that recommendation. And so they said to me the same thing they said last time. They said, we will do it. If you say to do this, we will do it. But it can be the only one on this list here. And I said, OK, all right, let's calm down, OK? This last item here is improve internal linking. That's not hard. It's just links, right? Just put some links on the page. You're fine, right? And they said, it sounds easy, but I gave them four different recommendations on how they could do this, OK? They said, if we put internal links specifically on product pages or other pages of the site, the points of product pages, we need to interface with our design team. We need to discuss it with our back end engineers to develop the right logic that should be applied to those links so that we're not duplicating links across 75,000 pages. And we need to get verification through our conversion rate optimizers as well. Because when you add more links into an e-commerce site, it can throw off your conversion rates and you're not careful. And they explained to me that's three different teams of people that they have to allocate resources for. And again, that is more money and more time. And they said, like they did before, we will do it if you say so. But that's the only thing we can do out of this list. And at this moment, I felt like I was a part of one of those cheesy Facebook memes where they say, pick one breakfast food, and you can't have any of the others. And I was kind of just frustrated at this whole process. And I said, well, I'm going to need some time to think about this. Let me do some analysis and get back to you on which one of these three I think you should go with. Later in this talk, I'm going to show you that analysis and show you how I made the decision of which one they should go with.  And this is something I always knew, but I constantly forget this as a consultant. It's that SEO is fundamentally easy to understand. I could sit down with a client in an hour and explain the fundamentals of SEO and they understand why they need to do each thing and how it connects to the bigger picture, right? Execution is a lot, lot harder. And as a consultant and sometimes agency people, we forget that. We forget how incredibly difficult it is to execute the recommendations that we make. In-house people, you guys know it because that's your life, right? But we forget that a lot. And so I think that this talk, hopefully, will help us understand easier ways to make better recommendations for execution. The next case study, though, was a large law firm. This is a big law firm. They had locations in several states and they were targeting local and national terms. So they had, like, a lot of the similar terms that you might imagine a law firm would have, personal injury terms, things like that. But they also did a lot of integration law. So they were actually able to do immigration law nationwide. And so they were attracting, you know, clients from all over the country to do immigration law as well. They only have about a few hundred pages, but quite honestly, a lot of their competitors didn't have much more than that either, so that wasn't really a big deal. They had worked with a local SEO agency for about a year. But they came to me because they started to suspect that there was something wrong with their SEO work, right? Because after a year of working with this local SEO agency, they weren't getting any results. And they understood that SEO took time, but they weren't really sure how much time it should take, you know? And so I looked over what they had done and in the first year of working with this local agency, they had new content development. They built a bunch of new landing pages related to their service offerings, their practice areas. They had fixed a lot of page speed issues. They were using WordPress. And so WordPress can be kind of hit and miss with page speed, but there are a lot of quick fixes for WordPress page speed issues. So they had done a lot of that work already. And they had done link building too. And they still had not seen much results with that. This client had went through the typical road map to results.  This is the kind of thing that pretty much every company goes through when they're attempting SEO.  That's usually where I come into play. I do that usually. And then they work on implementation. So that usually takes around two to three months to do. And then hopefully, if everything's been done correctly, they should start to see results in four to five months, maybe later. Of course, every company is different. Every business that works for the company is different. But generally speaking, it's this three-step process that a company has to go through to start to see results. Here's the thing though. Like, results are the main thing, right? And if they're not getting results out of this process, then it can get really problematic for everyone involved. And a lot of times, when I am in this position where I have to tell clients that it takes time to see results, I'm reminded of that old saying that SEO is a marathon and not a sprint. All right? And that is so true. SEO is a marathon, but not a sprint. The only thing is, the thing that SEO has never mentioned when they say this, is that even marathons have a finish line. And also, every race, whether it's a sprint or a marathon, you're supposed to be competing against someone. So if SEO is a marathon, who are you racing against, right? And the cold, hard truth is that you're racing against other channels. And I mean, I'm gonna be just clear about this. I've had lots of managers when I worked in agencies and other places tell me, no, Joe, you're not competing with Page Search, okay? Like, you're supposed to work with Page Search. And I love people that work with Page Search. By the way, raise your hand if you worked in Page Search. Okay? Here's a secret. Don't tell anyone. I'm super jealous of you guys. Okay? Because you guys get results so much quicker, right? I'm in month three or four and the clients ask me, where are the results? Why have our rankings not changed? What's going on? Page Search is out there improving their conversions, improving their ROI.  Okay? Like, come on, man. How many outreach emails do I have to send before I can get rankings to change, right? So, it's really kind of frustrating. It's frustrating for me, but it's also frustrating for business owners, right? Because if we've convinced them that SEO is a marathon, I don't care how much they believe that. In four or five months, they're gonna be looking at the other channels and say, well, it might be a marathon, but the other channels are winning. And the longer that SEO takes to get results, the harder it is to execute, okay? Because if we're not delivering results in a timely fashion, the momentum is lost. That's for a small group of work. Clients are excited and they're jazzed, right? Month four or five, if they're not getting results, they start to lose interest and they start to lose confidence in the process. And so, adding to that last lesson that I learned, SEO process can be hard to execute and always urgent to succeed. We always have this urgency to show results. And if you've worked in an agency before, you know this. You know that every month, you guys have to send reports to the client, maybe weekly, that show ranking reports. And it's almost like a nail-biting situation where you have to show positive results in some way or another. Because you wanna keep that momentum going. You wanna keep them interested and engaged in the process in order for you to do good work for them, right? But here's the good news. We can use data to make execution easier and results faster. And that's what I'm gonna talk about today, one way that we can do that. So let's go back to those recommendations because we're gonna use these recommendations as an example of how I solve this problem for the client and how I think you guys can use that same type of analysis in the future. So I had to, I went back to the drawing board and I thought to myself, where did I go wrong with this audit, right? Because in my opinion, these are all high priority items and they should be new in all of them. But here's the problem that I had and this has kind of changed the way I do things a little bit, is that I added these to the audit because I knew that they were important, right? But what's important for SEO isn't always impactful, all right? And what we need to be focused on as SEOs and just no matter where you're at, consultant, agency or in-house, we need to be focused on what is impactful and what is gonna provide the biggest impact because if we can nail down what is most impactful, then results come quicker, we can prioritize our strategies faster and the client will see results more timely. So the mistake that we make is not like any fault of our own. This is kind of like how we're trained to think, right? Because when we think about what's important for SEO, here's what typical SEOs usually look for. They look at things like raking factor studies, right? We look at things like algorithm analysis, which is like looking at the algorithm changes and the data that is impacted from those. We look at things like what search engines say.  And then we also look at what SEOs say, right? Because we have a community of people that like to share information and ideas. And this is all great, like all of this stuff is super important and we can learn a lot of important information from this and we should not lose sight of any of these things. But the problem is that modern search engines today analyze each domain in a much more subjective manner, right? So all those things I looked at on that last slide, these things right here, this is sort of like information that's sort of like a one size fits all, right? But nowadays search engines are much more subjective. They have things like artificial intelligence, machine learning, knowledge graph, query intent, you know, location, entities, all of these things will make the analysis of each domain different. So they don't judge every domain in the same way anymore.  Because every domain has a unique characteristics about it that the search engine understands in a unique context. And so we can do that by using key performance indicators and key performance factors. And that's what I'm gonna show you right now. So we all kind of know what key performance indicators are. These are metrics that measure a type of performance and they evaluate the success of a particular activity or effort. Most every company has their own sets of KPIs that they like to use. These are some just examples of different KPIs that are typically used for SEO. And I get clients all the time that are new to SEO and they ask me, what KPIs should we use? And I'm like, whatever makes sense for your business, right? You just need to focus on the ones that are closely aligned with the business objectives. But for this example, for this presentation, we're mostly gonna talk about impressions and average positions. You can get these from Google Search Console and that's where I got them for this talk. The next thing we're gonna talk about though is something I call key performance factors. And this is something that I've been working with. I coined the term, I think. I was gonna call it key performance enhancers, but performance enhancers are steroids and steroids are drugs and drugs do not help your SEO. They might make you feel better about your SEO. But they will not drive results. So this is key performance factors. These are things that you can measure that influence your KPIs, right? And so things like number of backlinks, word count on the page, your internal link scores, your page speed scores, percentage of duplicate content and crawl errors. And so for this presentation, we're gonna look at these three right here, word count, internal link score and page speed score. You're probably asking yourself, well, why did I pick these three KPIs? Well, the way that you wanna decide which KPIs to use is to really think about the problem that you're trying to solve. And so in this example, we're trying to solve these art recommendations. Now, these are the same art recommendations from that first case study. I wanna find out the impact that each one of these will have on our KPIs. And so what I did was, I looked at each one of these recommendations and I thought which KPF is gonna align most closely with that recommendation, right? So that first one is create unique content for product pages. That would be your word count, improve performance and page speed. That's your page speed score and improve internal linking. That's an internal link score. The internal link score that I'm using for this is coming from a screaming frog, but if you are a user of Moz, I think the closest thing would be your page authority metric. Page authority uses a lot of other things, other than internal links, but I think it would work well in this instance if you wanted to replicate this. So basically, these are our metrics for the example. I'm gonna show you. Our KPIs are impressions and average positions and our performance factors are word count, page speed score and internal link score. After you identify your KPIs and your performance factors, you need to put together your data set. And so this is the data set that I use. This is obviously an example data set, but that first column has all of the URLs or pages that we're analyzing. The next two columns are our KPIs, impressions and average positions. And then the next three columns after that are our performance factors, which is our page speed scores, word count and internal linking scores. You can really easily pull all this data together using a screaming frog, one of my favorite tools and integrate it with various APIs that integrate well into the tool. Okay, so I need to provide a warning before we go any further, all right? On this next slide, I'm gonna talk about correlations. And if you've ever spent any time in the SEO industry or the SEO community, you know that one of the favorite things that SEOs like to say is that correlation is not causation, okay? I am well aware of that, all right? Please do not harass me on Twitter about it later, okay? Let me tell you what correlations are for a moment. Correlations are not causation, but they also do not rule causation out. And in fact, they are probably the most likely case for causation, okay? So the reason that SEOs hate correlation studies is because the original intent behind them in SEO was to try to find ranking factors, and that is causation. So legitimately, yes, they should have been kind of skeptical of that. In this scenario, we are not assuming that these KPFs are causation, but instead we wanna look at the relationship between the KPIs and the KPFs. And we're gonna show you how to do that right here. So let's say that I wanna know the correlation between internal link score and impressions. And all I've gotta do is take my gas set that I've just assembled here, and this is in Google Sheets. And I've got a key in this formula for correlation. And that first metric there is the B column, and that's the impressions. You see it's B2 through B1000. And then that second one is the internal link score, so that's F2 through F1000 right there. And what this is gonna do is it's gonna give us the correlation between those two metrics. And so we can do that for each of our KPIs and each of our KPFs. And if we do that, we'll get something like this. This shows us the correlation data between each of those things. So like you can see here, I've organized this chart by impressions, so we can see the highest correlation for impressions for this client was internal link score, the word count and page speed was last. Average position, the highest correlation was word count, and then it went to internal link score and then page speed score. So what this tells me is that we're not doing page speed. We're not gonna advise them to do page speed because for both of these metrics, page speed is the lowest correlation, right? We can take a look here, which one should I choose? Well, if they care about impressions, the winner is improved internal linking because that internal link score has the highest correlations to impressions. If they care about average position, the winner is to create unique content for product pages. So this is a good example of how you can use this type of analysis to make smarter decisions over the recommendations you give and the impact of those recommendations will have on the client's success. Some other examples, though, about how you can use this for other things. Title tags, if you're optimizing title tags, your KPIs might be clicker rate and average position. The performance factors could be title tag length, but is the title tag same as H1 or split test format? Those last two KPFs would be simply marked in the spreadsheet as like a binary number, like a yes or no, so it'd be like one and zero kind of thing, and that would get the correlation for you. Another example would be page speed. KPIs could be conversion rates. So maybe you don't care about page speed for SEO, you care about it for conversions and UX, but you can look at conversion rates or bounce rates for your KPIs. You can look at your speech insight score, core web vitals or JavaScript execution time for performance factors. Another example would be for links. So you want to look at like your KPIs could be average position, impressions, number of ranking terms. Your performance factors could be number of backlinks, number of internal links, page authority, if you're using loss. So the closing thoughts here is that key performance factors can help you better understand the most important elements to your SEO process. But if for nothing else, I hope they inspire you to have a greater understanding of each domain's unique SEO thumbprint. Thank you very much.


Why do teams find it so hard to make decisions together? Well, in this next video, you are going to learn a simple framework that's going to help you understand how groups essentially make decisions together. This framework is also going to help you understand why collaboration essentially breaks down in most teams. The video that you're about to watch is a shortcut from a two hour live training video I run recently, where I really dive deep into the topic of the dynamics of group decision making and I gave some really useful tips on how facilitators can help groups through the decision making process. If you're interested in joining or participating in more events like this one, be sure to join our free community called Facilitator Club where we will be posting upcoming events. And in these sessions, in these live training sessions, you can ask any question you want about facilitation or workshopping and the AJ and SMART team are going to be there to answer those questions for you. Okay, I don't want to waste any more of your time. Let's just jump straight into the video. So why is collaboration broken? Before we try to understand this, we first need to understand how groups make decisions. So we need to understand the decision making process. And this is where we're going to get a bit complicated and go into a bit of theory, but this is going to be really useful for you and your future workshops and future facilitation gigs. Yeah. That's a word. But no, understanding the decision making process and a lot of people don't really go too deep into this. Understanding it is actually a core competency. That's a hard word to say. For every facilitator, any leader, any person who wants to work with a group, you need to understand how they make decisions. It's incredibly important.  What camera do I look at now? This one? The other one. Okay, the other one. It looks so good actually. I'm going to be teaching you about the dynamics of group decision making, how people make decisions together. Now, so this is more of an idealized model. She's trying to write, oh no, oh my God. Yeah, we're grand. We're fine. This is more of an idealized model. And before we go into it, I want to draw something first. Can people see me? Yes. Okay. I'm so worried. Okay, right. That looks weird. So before we go into the idealized model of how groups of people make decisions, I want to explain to you two types of thinking that most people that every person actually engages in when they're engaging in problem solving or decision making. And that is the first one up is divergent thinking. Divergent thinking is all about, this is how I teach as well. It's manic. Divergent thinking is all about generating multiple ideas and multiple, yeah, multiple options to solutions. So this is the really fun phase of thinking. You can be incredibly creative. And then we have convergent thinking. Jeannie Mack, look at this. It's so cute. Going down. It's good. Convergent thinking then is where you need to start narrowing down your ideas, refining your ideas. So this is more of like an opening to let's say a workshop or if you're in a meeting, you're going to be generating multiple ideas to a certain solution or challenge.  Okay, so we're going to talk about the idealized model of decision making. What would happen in an ideal world if people could actually collaborate? So people come together. There's a new topic or challenge they want to try work on. I need my fans over here. And. Oh, no, not those type of fans. The actual fans. Oh, thank you so much, Jacob. Chaos. Thanks. That's actually lovely. Yeah. So in the divergent thinking zone, what people might think happens is that loads of ideas are generated here. And they're trying to solve a particular challenge. Loads of ideas. Each circle is representing an idea. And then they kind of come into this middle zone and they're listening to each other. Everyone's staying focused. Everyone is being really nice and they're not being taken away from the discussion. No one is being very dominant or loud or taking over any of the conversations. And then they nicely move in here to the convergent thinking zone where they reach a decision point, like we kind of said earlier. I feel like I'm repeating myself. But yeah, this is in an ideal world, but this isn't how we work and how human brains work. What normally happens in this zone here, in this middle zone. Use a different color. Oh! Say the name. Oh no! Okay, but I will be getting to the name. Don't worry. So what happens in this middle zone, I'll tell you exactly what we call this in a second, is that people actually go on tangents. People go off track completely. They can't kind of stay focused. They're getting distracted. Their energy levels are dropping. The more introverted people don't have a space to speak up because the more loud people, the more articulate people are so dominant that the introverted people's ideas are being sabotaged. So lots of really good ideas are being lost in this zone. Because yeah, there's no space for people to properly get out their views and express their views. So what Sam Caner calls this zone, and he acknowledges it, he calls it, I know Ellie, I'm getting to it, he calls it the groan zone. The groan zone, perfect name for it, but we decided to call it the messy middle because it is, it's chaos. It's absolute chaos.  And what this framework is actually called is the diamond of participatory decision making. I'm gone off track. No, I'm not. But no, it's incredibly important for facilitators, people leading groups, people working with groups to understand that this middle zone actually exists. It's actually a core competency to be able to understand this because understanding it means that you can figure out different methods, different skills, different techniques to help participants through this messy middle and to the convergent thinking zone. And what normally happens here, people really, really feel stuck here, unmotivated and that's when projects are prematurely abandoned, that's when teams usually kind of give up and say, look, the problem is too challenging, let's get back to this next week and they actually schedule in another meeting and the meetings could go on for months and months and months on one challenge just because they don't have somebody to help them through this very, very tricky, messy middle groan zone. I guess the two terms really, really confusing. But don't worry, we as facilitators can help people through this phase by using different skills that I'm gonna be teaching you in a second. Different techniques to help people stay focused, stay energized, they'll be using different techniques to make sure that everyone's listened to and they have a space to share their ideas so these poor introverted people can get their ideas out. Also, they're managing the discussion to make sure that everyone has a space to share their perspectives and I'm going on. You know what I mean. So, yeah, knowing this framework is really important as well when you're running a workshop or if you're running a meeting, we like to show this framework to participants at the very beginning because it kind of sets, oh, that's such a bad word. It sets expectations and lets people know how they're gonna feel because feeling stuck in here, feeling a bit like there's loads of miscommunications happening, feeling like the process is spinning out of control, that's a complete normal part of the decision-making process and telling participants this is really important because it kind of relieves a lot of pressure off these participants as well. They can kind of relax and go, all right, it's gonna feel messy, but that's normal. We'll eventually be able to get out here and come up with a well-defined solution to our challenge. So yeah, that's a tip. Use this framework at the beginning of your workshops to set expectations but also understand it deeply because this is incredibly important for you as a facilitator to know because knowing that this exists, yeah, it's a game changer going on. Oh, you're back! Oh, hello, welcome back. I hope you enjoy that video. If you're interested in watching more live training sessions like that one, be sure to join our free community called Facilitator Club and the link is in the description below. Get in there, be cool and join us. Okay, we're curious. We'd love to know, does your team also struggle with collaboration and making decisions together? Cause if they do, we'd love to hear about it. Share your experiences in the comment section below. Now, please leave because I need to clean my TV. Bye! Thank you so much for watching. I'll see you in the next video. And you're going to understand why facilitators are so important in helping children. In helping groups. In helping teams.