This video is sponsored by One Football. One Football's app makes it easy to keep up to date with transfer news scores and stats all in the One Place. To download the One Football app, click the link in the description. The app is free. To watch more of this video get over and join us on page. So Southampton gone to themselves clear of it. Barely beat Cardiff 2-0 and you think Cw'r cadwff y diolch beth y byddai y rautafaith, a ddim bry LU 5 nil ap bomath. I'r ddweud yw'r ddweud gyda chi dwi'n gwneud, brytyn mewn cadwaf ar brytyn a'r ddweud mewn, dwi ddim yn wneud gydrwm ni, ond mi'n dweud â ddweud angen ydw i mewn nad, ond rydw i'n credu ychydig. Mae o'n ddweud i ddim 2 nil, Bydd y bydd y bornyn, 5-0. Dyna ym Mente... Dyna dim genes. Dy hynny'n golygu, mae'r gael e'n gwrdd yn ein bodgylch, ac yn ddulech y mynd i ymdweud i'r bod yn ôl i bryd. Bydd hynny'n golygu'r gwnaethaf isb. Dyna'r golygu'r gweld i'r bod yn digwydd ym gyfaint, ond mae'n gwbeithio efallai bod y gael'r ddweud iddo buddedd. Dyna e'n greu'r golygu. Mae eich golygu. Mae'r golygu eich golygu. Ac mae'n rhaid i'w gwlad i'w cyffabiliadau o'r gysgwyr yn golau. Felly mae'n ganddo i'n bryd gwych. Dyna'r hollol sy'n amlwg o'r form. Ac rwy'n gweithio, rwy'n gweithio, a rwy'n gweithio, mae'n rhaid amlwg ar y bydd. Fy enw i gyd, ac mae'n gweithio, mae'n gwybod i'r gwaith ar y gwbl. A yw'n gweithio ar y byd, ac mae'n gweithio ar y bydd. Ac rwy'n gweithio, ac rwy'n gweithio ar y bydd. Wel, gydwyddi'n gwych arno fydd, yna'n gwneud o'n ddwi'n creu'r rhwng arnyn nhw. Mae'n ffwrdd o'n rhan o'r tyn, mae'n rhan o'n rhan o'r rhan, mae'n ddweud o'n ddweud o'n cyllid Ddechrau, ac mae'n dweud... mae'n ddweud o'n ddweud o'r rhan o'ch eu cyflwyllau. Yn y ddweud, mae'n ddweud o'n ddweud, Dyna'n edrych arrd, mae'r ddydd yn gennym arfos ddwy o'r gweithio o'r dŵl. Mae'r dŵl yn ystafell o'r gweithio, a expertian o ateb yma, a mae'n wedi gweithio'n gyntaf o'r dŵl. Mae'r cadrhaff o'r gweithio ddweudio a dwi'n digwydd o'r dŵl. A mae o'n ôl o'r gweithio sydd yn yw drieddol iawn. A gweithio ddweudio'r ddych chi'n neud i ni yn arfer o fynd i chi'n ddydd. Ac rwy'n ddweud e'n gwneud y flwyddyn, wrth agriff yr llyfrnid, y gallwn feddwl gweithio yn ffwrdd o thamor yn llyfrnid, ond rwy'n ddweud eich gyrddol, yn eich ffordd yn ffwrdd gwaith ar rescolon, bardzo rwy'n ddweud eich hun o'ch wgwylliant. Ar y dyfly'n gobl di heddiw'r 3 oed yn gwneud. Erd dyfly'n gobl y cyddeithas, yno wedi ar gael. Mae'n gobl y gyd XDorol yn morarchol eich cyddedig. Mae'n hoffi'r gwrs o'r wych. Mae'n gweithio a'r wych. Mae'n goblio ar y llunc ddiddordeb. Mae'n goblio ar y newg. They've got tricky, tricky games, fighting and Cardiff, you know, OK, Cardiff go to full-on. Dwi'n cael e'ch aff te, now OK. Cardiff will dig in. Cardiff will cope with full-on, that's it. So if Cardiff could, even if he nicked a point against Liverpool, he'd give himself a chance. But I think that now for Liverpool is a horrible game because it's a game everyone expects Liverpool to win. But it's a game that Cardiff will physically cause you problems unless you get the first gold. If Cardiff takes the lead, Liverpool will be rhaid in the game. The incentive for Neil Warner will be, listen lads, I don't think Brighton will win at Wolters. The best he could hold for is a draw. That gives Cardiff the opportunity to beat Liverpool and draw a level with Brighton. The goal difference is massively in Brighton's favour. It is, but... Ishing Brighton, I've got bad games. So def o Cardiff, get level or get next to them so that everything's on it because the pressure becomes a bit too much and you know that Cardiff Liverpool is going to be very interesting because you know Umaneas hasn't scored a goal yet and you know the Prince you know what happened when Hull Liverpool played Hull a few seasons going Hull going down, you know the Prince popped up so it'll be a tough game for Liverpool because if you're Cardiff you're thinking if we can somehow get something out of this game we are right if we've got an opportunity for that. There was a lot of football to go to Southampton and to go to Brighton and win that game because for Brighton it's. But also though from Brighton's point of view to lose that game is shows that they're obviously suffering, they're not scoring goals, they've conceded seven in two home games and hadn't scored. That's, there's no bounce back, that would be ffuming, no bounce back ability there was that, two home games and you don't do anything. So all of a sudden it's made that bottom very very interesting again, you know like you said, just got themselves an important win when you need to in Southampton. I thought Southampton a bit more so.  But it'll be more than just this weekend Money. Of course listen, the right back in it and one pop past Rapha go in there and beat them because there's a way to go for Newcastle, Newcastle's a way to go's brilliant.  Brighton beat Newcastle didn't it, Newcastle bum. So that'll be a little bit of payback. Or even a draw, Brighton are going to have to win some of these home games because they've got some bad away games and you know it's just all of a sudden it's very very interesting now with that bottom in them. That's what you're looking at aren't you, because that's where all the story is now.


Hello, everyone, and welcome to Resilience Untangled Challenges and Opportunities in Latin America. Thank you all for joining us here today for the first ever collaborative conference organized by Latinx groups at Penns Weitzman School of Design, MIT, DUS, Columbia, GSAP, and Harvard's GSD. We wanted to come together as a group to discuss and highlight work done across design disciplines, specifically in Latin America and the Caribbean. We've, we've really enjoyed getting to know each other and hope to continue the traditions in years to come, hopefully in person. So today we'll hear from academics, practitioners, students, and communities about topics with resilience in these regions. Resilience, the capacity to recover quickly has emerged as a guiding principle for development at the global level, thus shaping the future socially, economically, and environmentally. We hope to advance the conversation about resilience from the perspective of various backgrounds and to experience and experiences to better understand this concept from the lens of work in Latin America and the Caribbean. First up, we have a panel featuring esteemed professors from each institution involved in organizing this event today. Members of the panel will discuss the meaning and importance of resilience moving forward and the evolution of the term. To introduce the moderator for this panel, Nora Libertun de Durin. She is a senior specialist in the urban development and housing sector of the Inter-American Development Bank. Nora holds a PhD in urban planning from MIT, a master's degree in urban design from Harvard University, and a master's degree in architecture from the University of Buenos Aires. Previously, she's been the director of urban planning for the city of New York and has taught urban planning and urban design at Columbia University, MIT, and Harvard University. Nora's analytical work focuses on urban sustainability, housing policies and public space in Latin American cities. Her articles have been published in prestigious academic journals, including cities, urban studies, housing policy debate, international journal of urban and regional research, international journal of housing policy, journal of planning education and research, and city and community, among others. So without further ado, I will now pass it on to Nora to introduce our panelists and to begin the introductory session of this conference. Hello, everybody. Thank you so much for the introduction. Before I begin, I just want to congratulate the organization and all the students who participate in this initiative. It's wonderful, it's wonderful that you are working as a community of colleagues and scholars and thinkers and practitioners trying to advance the most important questions of our time. And also, you know, to have this idea of focusing on the region in particular, I think it's very valuable and it's something that I have not seen before. So I just want to congratulate you a lot for that. I am humbled by this panel. I think that I feel, as I said to them, I feel like I'm in, you know, like in an Oscar ceremony or something like that, you know, they are all the, you know, the best of the best. So, so it's really an honor to be moderating this panel. I will first give, you know, a very brief be of each of them brief because, you know, anything I'll say will be brief, comparing to what I could say about them. And then I will just introduce them, the main topics for this session and pass them to each of them to present their own perspective on that. I will read just not to miss anything, but please understand that their achievements are much more than what I'm going to read. So Larry Bale is Associate Dean of the MIT School of Architecture and Planning for Professor of Urban Design and Planning and Director of MIT's Resilience Cities Housing Initiative. Larry is the author or editor of 12 books and more than 60 articles examining urban design affordable housing and city planning, including for price winning volumes on American public housing history design policy and politics. Finally, he is best known for two other books, Architecture Power and National Identity, which is an analysis of design capital cities and a co-edited volume, the Resilience City. At MIT, Larry has won the Institute's highest award for teaching and for graduate student advising, as well as departmental awards for advising and service to students. Thank you for being here Larry. The first or is the founder of SCAPE and a professor at Columbia Graduate School of Architecture Planning and Preservation. She is the director of the urban design program and faculty director of the Center for Resilience Cities and Landscape. SCAPE is an award winning 60 person professional practice based in Lower Manhattan, New York, where she directs the design of all projects. As a professor and practicing professional, she has advanced concept of sustainable planning and urban design at multiple states. Thank you, Kate. I don't see you here now, but I'm sure you're here. Diane. Okay. Diane Davis is the Charles Dyer Norton Professor of Regional Planning and Urbanism at Harvard Graduate School of Design. Before moving to the GSD in 2011, she served as the head of the International Development Group in the Department of Urban Studies and Planning at the MIT, where she also had a term as associate dean of the School of Architecture and Planning. She is trained as an urban sociologist and her research interests include the relations between urbanization and national development, urban governance, informality and the growth instructor of cities with a special emphasis on Latin America. Hello Diane. And then we have Dini Bert, who is the Lawrence C. Noster Chair of Urban Research and Education. She teaches courses in global organization and the doctoral seminar, and serves as a chair at the Graduate Group Institute of Regional Planning and Code Directs at the Penn Institute for Urban Research. She is president of General Assembly of Partners, the engagement platform for the implementation of the UN's new urban agenda and associated global agreements. She co-chairs the Sustainable Development Solution Network thematic group on cities, and she is an associated editor at the Journal of the American Planning Association. Hello, Diane. Again, this is very impressive.  So I will tell you a little bit about what I asked from them before I, you know, I passed the word for them and I will follow the same order in which I present each of them. So we start with Larry, then Kate, then Diane, and then Dini. So when I was reviewing this idea of resilience, which I think is quite complex, it came to me the question that we all have, which is, what is this? What are we talking about? And then I did my own little research on the concept in Latin America, what was the main conversation? And basically, there were three things where the issue of urban informality and how it affects specific communities, and how the environment affects them in particular. We have the issue of urban planning, and we have the issue of governance. So, so my first question to myself is what's new? I mean, like, this is the same thing that we've been talking about in Latin American cities for the last 30 years, if not more. So, so I was wondering if resilience was a new word, a new, a new vessel to speak about all issues. And then that's why I posed these three questions to these distinguished panel members. So the first thing I asked was how did we find resilience? Like, how do they understand this word? The second thing I asked from them is how this definition changed, particularly in the last five years, if it has changed, because as we all know, these last five years have been particularly challenging in terms of our, our common quest as a society, in terms of what do we want and which are the main problems and, you know, both in terms of inclusion, social inclusion, as well as the environment. So I was curious, you know, if there's something that has changed in this last, this last five years. And also, with this concern in my mind that we keep on facing the same issues over and over again, particularly in Latin America, it's hard to see progress in some, in some issues. So what we should start doing now to achieve this vision of resilience, which are the main things that we need to start doing and by doing I mean like thinking or talking or teaching or, or, you know, designing what should we do. So, without further ado, I want to invite Larry to provide his input, his insights on these issues. Welcome, Larry. Thank you so much, Nora. It's a pleasure to see all of you and, and a great honor to be with this panel. I'm going to share the screen and try and respond. Let's see, that didn't seem to do it. It says I'm screen sharing. Let's just get rid of my calendar. That looks like something that's getting close. So, Nora asked us, I think, really helpfully to state our definition of resilience and then reflect on how our thinking has evolved and to consider what is needed next to achieve a definition of resilient and for me this is really expressed by the need to move from something that we really all to easily call the resilient city to something that I think is more consequential, a notion of an equitably resilient city. So if one is going to take the distributional politics of resilience seriously it becomes possible to arrive I think at a more holistic definition of equitably resilient urban settlements. For me, equitably resilient settlements result when there is something that's seen as a legitimate process that will not just in emerge environmentally protective infrastructure care about an attractive built environment, but also ensure that it's promoting an adaptation that will benefit everyone, including the least advantage, and all too often in Latin America and elsewhere, projects that are touted as promoting something called resilience, often fall short in the key political dimensions of legitimacy and equity. What's moving is that the effort to document equitable resilience can identify and promote some more promising alternative cases. Nora asked if our definition has changed in the last five years or so. And for me the answer has been fairly consistent for about eight years. Ever since the launching of the resilient cities housing initiative or Archie at MIT in 2013. The main difference for me is that we've been attempting to operationalize that definition by seeking out ambitious projects and programs that really try to improve human settlements in holistic ways. So my window into resilience has focused on housing, but recognizing that that's not enough to build a rebuild housing to that's simply affordable. If you're going to be concerned about the overall resilience of a city housing has to introduce what I think of as four principles. It can't be equitably resilient unless it's going to help residents cope with other simultaneous challenges, economic struggle, a change in climate dysfunctional governance and urban violence and unstable tenure. So we give these a shorthand of livelihood environment governance and security and housing in Latin America and elsewhere needs to be cited with access to jobs it needs to be designed and located in less risky areas it needs to involve residents in community management, and it needs to protect the security of residents, both from violence and from displacement. We found that few projects succeed in doing all of this, but substantial progress on any of these seems worthy of praise. So one of the more promising examples we've been studying is already quite well known, the Kanye Martin, Kenya, Community Land Trust in San Juan, Puerto Rico, Caribbean but not quite Latin America, launched between 2002 and 2004. They were substantially controlled by the residents of eight informal communities along the Martin Pena Channel, which was a highly polluted title estuary reaching near the city center. The land in the area was mostly government owned and the government wanted to clean up the channel but the residents correctly feared displacement and gentrification if this happened. The 100 low to moderate income residents became members of the channel land trust which owns and manages 110 hectares, hectares I think it's about 272 acres. This provides in from permanently for affordable housing, as well as alternative housing for those who had to be displaced by the dredging. They created a government corporation and lost say to implement the project and regularized land tenure. And also created improved public space and a variety of socio economic initiatives. In the words of a to 2020 assessment resident participation turned an engineering project unaware of its negative externalities into a comprehensive development project, taking action to prevent such externalities. And this led to creation of public policy and institutions to make it feasible. We discussed in two chapters in the recently published book on common ground, it won a UN World Habitat Award in 2016, and it's now actively seen as a model for favela redevelopment in Brazil, among other places. So, so lastly, what do we need to do now to achieve this vision of resiliency. Certainly that equitably resilient urbanism results when there's an enthusiastic yes, not just to my two first questions about environmental protection or even attractive public space that seem up till now, mostly the focus of projects, but also the latter two questions focused on the equity, the equity and the legitimacy of proposals. So, for me it's it's difficult but not impossible to do this, and we should be collecting and touting more examples of successful efforts. Thank you. Thank you so much Larry for for this presentation, I just want to say that at the end will have time to discuss, you know, these four presentations is open up for questions for from the audience. So Kate, I don't know if you're ready to follow. Yes, I am. Is that working. Great. So, thanks, Larry, I'm happy also to be part of this panel and I love seeing that collaboration among institutions I'm working on something called the Green New Deal super studio now which also has a similar collaborative spirit and it's really the way we all need to be working and thinking and together in the future. I tried to take your prompt quite quite literally in our in terms of, you know, the way I used to think or the way I'm thinking now. I will say I used to think about resilience as a sort of a built environment challenge that we need to think and differently and build differently in our physical landscape. And more and more, I think about it as the top two aspects, one, which is reducing carbon now, and that truly is the systemic. One of the kind of core systemic problems that that's a root at the root of the issue in terms of, you're talking about climate resilience. And then to that I feel more and more like these issues are issues of human rights and that the frame of justice somehow needs to change to adapt to a human rights perspective. And that it's often people in harm's way that are the least empowered and have the least access to infrastructure are those that are required by us to be made deemed resilient. So, and then, moreover, I guess, you know, as from from many years and into the future I do still think and try to advance the concept of concepts embedded in landscape as templates for resilience thinking and acting. So I've learned quite a bit from the New York environment. And I think as a landscape architect and someone who's sort of pulling permits for resilient projects do think I have a kind of an eye on the ground in terms of, you know, the need for incredibly robust physical environments with intact shorelines with thriving wetlands with coral reefs intact, etc. These are the kinds of physical environments that I think care clean water etc that characterize resilient cities, and I think empowered communities is also a deep kind of characterization of resilient cities so we've had a lesson here in New York around learning from the past learning from the physical landscape, understanding the protective benefits of wetlands shallow water landscapes dunes, sedimented replenished sediment etc and the protective benefit of intact landscape ecosystems, and ecosystem as infrastructure approach. Unfortunately, these ecosystems not only in the United States but probably all around Latin America and my experience visiting and working in several locales have been taken and have been diminished. So, part of what at least in our practice escape I've been trying to do is weave together, not just a kind of a definition of resilience but to try to advance a resilience stance. And you know that goes beyond just best practices of planning or urban design to sort of primal gate culture onshore that has aware of risks and environmental risks to, you know, promote landscape projects that rebuild ecosystems, and that together that this is a sort of a world if you will, that kind of reverses this trend of degradation of the environment and disempowerment of communities and increasingly at risk so we have to reset this world here. I've learned quite a lot personally from the Jamaica Bay landscape which was pictured earlier in the sense that this has been a federal project to replenish disappearing wetlands it's also been incredibly local project that involves community action and community activities that is paid and and part of school projects, etc. And it's really this kind of framework I think that has a lot of power that could potentially also sort of be applied or interpreted in the Latin context. So this project, which is this living breakwaters project that's in construction in July is also something that tests resilience physical project that integrates all three of these aspects of risk reduction ecological rebuilding and rebuilding kind of onshore culture. So that's underway now, and you know I do feel like has a lot of potential for not just thinking about resilience as like a planning mode but something that is in invests in people, and people and and and sort of reconnects in a direct way, a kind of a stewardship framework. So in the Columbia context I've taken students all over the world and have really put water at the center of the resilience narrative. This is students in with with in Pune, India, and this is a range of cities that we've been to in the past say six seven all of which are incredibly unique and different but that share qualities of becoming having populations that are more and more vulnerable. And that that are be are developing along a kind of a petrochemical urbanization mode, and that are not are becoming more and more vulnerable at less resilient. And so that sort of framework has talked, you know, tried to explore the way that we plan and design for water is a matter of life and death for a planet, and truly is kind of a foundation of what it means to be healthy and resilient. So, particularly in the Latin context, and from if we want to talk about the range of mangroves on the coast of Columbia, or the reefs, protecting Billy city, and the wetlands of the in Uruguay, or in Montevideo, you'll you'll hear but all of these ecological landscapes have a huge role to play as do the people that live in them. So I think that's a couple of big picture lessons that I hope we can that apply to many contexts. Thank you. Thank you, Kate, we'll come back again at the end of the four presentations and we now follow with Diane. Diane, are you there. Yes, I'm here. Let's see. Is my screen shared. Yes, it is. Okay, I'm so bad on these things. Yes, thanks. Just a thanks for including me in this. It's such a great panel I know we all are going to want to talk to each other for much longer than we possibly have time for but let me just say I'm happy that I'm following Kate. I'm partnering with Larry, whose work kind of looks at both climate as well as kind of you mentioned violence in your introduction. Larry, I just want to caveat that my response to Nora's great questions have to do with my own work on questions of resilience as seen to the lens of violence in Latin America or across across the global south and when I was at MIT I developed a project with a group of students that looked at what we called urban resilience and situations of chronic violence. I do think as you'll see in a second that the questions that I'm raising about resilience have a lot to do with looking at violence but they're going to be hopefully relevant to looking at climate change and other forms of vulnerability. And I will side note at the GSD I've been co directing a master in design studies program and risk and resilience where we're really trying to engage conceptually and analytically in kind of the parallels what what we can learn those who are studying violence and vice versa because they're two different forms of vulnerability. So in answer my in response to Nora's question about how I define resilience I want to first day and I've written about this and many of you who taking classes with me know this about me. I'm not that a big fan that and I have not been that big of a fan of the concept of resilience so what I did in this slide here is I want to under I want to first of all highlight what I see the limitations of the concept again drawn from my work on violence. But you know resilience which is often like summarized is thinking about coping and adaptation strategies. First of all that the kind of as I would say the kind of under the epistemological underpinnings of the notion are ones that I think we have to question because it does not assume a capacity to directly eliminate the root causes of the vulnerabilities and am I understanding of those is kind of like okay there's a structural problem and let's see how we can tinker around the margins. I'm not saying everybody who use the concept of resilience when they looked at violence we're doing that but in my own work on violence I mean this is a major problem just like climate change is even probably more complex with the variety of global scales from the local to the planetary that produce violence. I'm just giving that produce climate change but violence is also the same type of a problem it's not easily managed with interventions that are based on other ways. So that's the first problem with the notion of resilience. The second one is constantly people say where resilience is about bouncing back or returning to normalcy but in the context of violence. Violence is a product of the fact that people are living in terrible situations. So the thought that you would want to introduce policies to return to normalcy to me is ridiculous or shall I say conceptually flawed. Third thing that I want to say that we developed and I think it's one of the best things that came out of our project on violence in Latin America is we developed the notion we wanted to disaggregate the concept of resilience. We had to work with that concept but let me just say because we got funding from USCID which was under Hillary Clinton at the time and resilience was a big branding concept that they were using to fund you know international relations. So we had to kind of take seriously resilience but what we did is we proposed a distinction between what we call positive resilience, negative resilience and equilibrium resilience. With positive resilience being coping and adaptation strategies that might forge a pathway out of the structural problems of violence. Positive resilience being the fact that in the real world people undertake coping and adaptation strategies at a certain moment in their certain context that can drive the root causes of violence that would empower the perpetrators of a violence because that's how they can survive. And then the third concept was equilibrium which most people live in which is your kind of things don't get that much better but they don't get that much worse so maybe we can talk about how relevant those distinctions are for looking at climate change problems. Two things really quickly I only have three slides and the next are going to go quickly. I just wanted to say that in our research we looked at deep histories we had eight case studies to understand the sociocultural and socio structural and socio spatial context of violence. In other words it wasn't a technical exercise we were kind of trying to kind of reveal the historical origins of violence and understand what can be done in that particular structure, the structural context. So that led us to obviously the work that I've been doing at the GSD is I don't focus on resilience anymore I focus only on risk, because I think that we are living in a world at risk. And that doesn't necessarily mean that we are trying to do something about it and kind of have all the positive attributes of people identified with resilience, but that we're really focusing on risk. Second now how do I move my slide. Does any. I have this online. Oh there. The second question slide quickly and or asked us to think about how our definition of resilience is different now than in the past. I never liked the definition of resilience in the first place what I wanted to do is kind of morph that question into a question about time and temporality. Not so much whether I myself have changed the way I define resilience but having us think a little bit about the power of the concept of time and where that should factor into our studying. So, both vulnerabilities climate violence and how to deal with it. So, and I'm not going to say much more than the fact that in our research that we found that the measurement of resilience or the measurement of what looked like it was working had a temporal dimension, because you could take a snapshot of something at one moment and it didn't really accurately reflect a larger the larger scheme of change and in fact at one moment. We found for example on the issues of violence that they're going to data analysis showed that homicides were going down but homicides are going down, because there was a new relationship between the perpetrators of violence and citizens that negative resilience in other words the data show that things look like they were getting better and they were really getting worse, and there was a temporality to that. So that's kind of the point about thinking about when we look at an action and when we assess whether the actions that we're undertaking are working or not we have to be very attuned to temporality. And I think that also for me suggests that again in the study of violence that we need new methods of analysis and action of that are not just like formal quantitative metrics. The other thing that I think comes across on this slide is that we looked at the hit the history of violence has a lot to do with what's possible and what's not pointed made earlier. The second thing is that the violence, there was a co-production of kind of conditions of vulnerability, depending on the conflation of various sectors. So we might think about that in the contemporary era of looking at violence in Latin America the ways in which, well, in this slide and we looked at resources, urbanization, drugs, economic there all sorts of sectors that explain this larger thing in the contemporary period I'm very interested in the conflation or the kind of co-production of climate change issues and violence because climate change brings climate migrants, often in neighborhoods or countries where violence is already high, and that kind of intensifies both climate, climate migration as well as violence. And there's my last one, just the last one and I won't say much more about this because I think this is where I'm totally with Larry and Kate that like about what do we do about these problems now. Obviously, we want to connect them to equity and social justice commitments and Larry did a great job of laying that out. But the second thing that I wanted to put on the table here is, well, when we look at, it's important to look at governance and not just space and look at social relationships in history and like we want to look at the overlapping sectors and that sweet side of how they produce really bad vulnerabilities and whether, in order to untangle that we have to have like a multiple actions at these three scales simultaneously that's one point. And the second one that I'm working on now is thinking a little bit more and this is maybe to speak to the institutional, the appropriate to enable preemptive action and when I say sovereignty, I'm thinking about, I would like to use the Benedict Anderson concept of imagine community. So not just formal political territorial governance strategies, but what is it kind of the political ecology or the political territoriality of sovereignty, who's involved, maybe we could think about the tourist concept of the critical zone, who should be involved in certain types of vulnerabilities, whether it's violence or climate change, it may move beyond a political territory of governance, but that we have to rethink those sovereignty arrangements if we are going to support preemptive action about vulnerability. So I'll stop there. Thanks. Am I off? Yeah, stops here. Sorry. Thank you so much. And as I said, we'll come back at the end, and also we will provide time for questions and answers. And then Jenny last but not least, I will keep forward to you. Thank you very much and thank you, students and faculty for organizing this wonderful conference and this very provocative set of introductory talks that we have. I'm going to be a lot more simplistic in terms of thinking about resilience and of course agree with everything that has come before me, but to boil it down to a very simple set of thoughts I think it's the ability to provide for the needs of all citizens to work, live and play in places that are peaceful, prosperous and environmentally safe. It's fairly simple and fairly basic. And when you talk about Latin America, I think the primary issue that has to be dealt with which Nora addressed in the beginning was this issue of informality with 60% of the population in Latin America working informal jobs. 3040% of populations living in informed settlements. It's time to think about how to address informality with real seriousness. In terms of how I used to think about resilience, obviously, there were priorities that were set within that. I think we have all changed our priorities now in terms of what needs to be done first. Obviously the health issue has come to the fore and we need to be thinking about that above all things. And of course thinking about poverty because when we're talking about a green and just recovery after COVID-19, one of the biggest issues of the vulnerabilities that exposed and issues with regard to poverty. So that leads us to thinking about how jobs are created, how education is supported, and how the basic physical environment that our speakers before us have addressed those particular issues, whether it's dealing with housing or dealing with climate problems or whatever, but we need to deal with the global issues as well in order to meet my simple definition of resilience. And that brings me to how are we going to do this, and it's all about money. And I think we need to be thinking a little bit more about money and how the structure of how funding is occurred is occurring in the development arena today. And I think that has been focusing on how we finance climate resilient infrastructure. And let me just share this slide, just a second. And one of the things that I've been dealing with obviously is how one understands what the financial system is in terms of to get that money into the areas that we want to get it to all of these things that we've talked about whether it's housing, or dealing with our shorelines, whether it's dealing with violence and so forth is going to take a lot of funding to do so. And the first thing we have to understand is that we have a finance system that is based on 20th century ideas. And it's based on the agreements that were made after World War two to promote peace and security to give precedence to nations rather than cities, and cities are not part of this discussion, particularly when it comes to making the decisions of the great international funding organizations, whether it be the World Bank or the IDB or whatever, it's very difficult to get a city voice there. In fact, a city can't borrow money directly without international guarantee for obvious reasons but nonetheless difficulties arise when there are political realities when national governments don't agree with local governments and therefore how are you going to do this sort of thing. To their credit the international financial institutions are doing a lot of things to address these issues so they can work within their mandates. For example, the World Bank has created programs to help with credit worthiness of cities so that cities can create their own revenues to support some of the changes that we want for resilience. We have special city focused activities, urbanization reviews and so forth. The IDB has its innovation labs. We can talk about lots of things that are happening. But what we really need to be thinking about is how we can start finding ways for cities to direct their resilience within the national goals that are held. And that means finding ways for them to become credit worthy. It means finding ways for these large informal populations to be integrated into the formal economy so that we can create the kinds of projects that you need to do within a city itself. It means finding ways for education and capacity building for locality so that they can create the kinds of projects that can be fundable through existing and new institutions. The resilience means finding a way to do this and this brings us back of course to governance which Diane was talking about as she finished her talk and the necessity of expanding the number of stakeholders that are involved in the decision making that's occurring. Because if we're going to create resilience it has to be created with a collaborative approach to the kinds of solutions that was wonderful solutions that we've heard being presented here today. And that means expanding civil society. It means giving local governance a voice at the table and so forth. Thank you. Thank you, Jeannie. This has been wonderful and I just want to do some to highlight some things, some commonalities and some differences and then open up to questions. So the good news is like after listening to all of you, we can work together. You know there is so much in common and there are so many entry points that are complementing to each other that I think you know that's wonderful. Like Jeannie came at the end and said like I agree with everything else and you said something totally different which I think it's powerful itself. What I found is that all of you ended with an idea that we can do something about this and I appreciate that a lot. The message was there is something to be done, there's something that can be improved and there are examples of improvements. So I think that's also very powerful. I also want to highlight that and it's no wonder that the notion of resilience even if it's contested as in the case of Diane it always includes people and places and institutions. And then in terms of difference is something that call my attention is that some of you were focusing on products and some of you were focusing on processes. I think that Larry was focusing more on a product and Kate as well, and Diane and Jeannie were more talking about processes of doing things, even though, of course, everything is, you know, connected and we know it's a loop in which we are learning but that was there. The other things that call my attention are the scales, the scales and the entry points, and I think that we kind of cover the whole range like from the architecture to the landscape to the society to the institutions and I think that that that is wonderful because it does provide many solutions to work on this. And also, some other thing that I want to highlight that it seems to me that the utopian notion of resilience is a society that doesn't exist with the landscape that used to be somewhere 1000 years ago. It seems that you know it's, it's a future of a society that we don't know yet that we have never experienced where everything you know is perfectly fair and just with the landscape that, you know, maybe, maybe we never live there yet but it was somewhere in our far far past. So, I want to do something now which is open the q&a I think it's open, it's available to all of us to read. And then try, I will try to assign questions to all of you, but before that I just wanted to, to take advantage of this opportunity in case there's some, you know, observation or some insight that Larry Diane Kate or Ginny want to say now after listening to all of you, you know, before we jump into the questions, you know just just something brief because we don't have so much time but I want to allow that. Yes, Ginny. As I think back on just what the 20 minutes that just happened, and we're all talking about resilience but everyone has a specialty and a focus, and it's bringing these folk by together in a collaborative situation that's so important, because we can't have resilience unless we have so many different components. And it's also on us to problematize resilience I mean I see this sort of, you know the words, the questions in the q&a about, you know, how has the framework of resilience started to become more, more dominant in the discourse personally on the on the ground in working with mayors or governors, even the United States in in red states or in in conservative states where you're not allowed to say the words climate change. Everybody wants resilience work and resilience project so I don't know I also just feel like it might be interesting to talk about that term a little bit more because as much as it's challenged by all and to Ginny's point like each, each perspective is kind of problematizing it in a different way. We still come back to it as a kind of a notion of what a path forward might look like that has, you know, not, you know, a degraded physical environment that has more, more characteristics of just and good cities and so I mean I would be interested just to hear a little more on that because as much as it's constantly called into question it is something that municipalities government agencies and others tend to rally behind relative to funding governance project development etc. It's not just physical. I also had I agree with all those other comments I was thinking it didn't develop too much it was in one of my slides but I also think another issue I don't have an answer I have a question for the everybody on the panel but also the students and anybody who's listening to this great initiative that that you guys have pulled together, but I'm thinking a lot more about how do we imagine the future. It's not just what to do in the future but in order to kind of pull to try to link triangulate all the forces that we need to make a difference, whether it's financial as Ginny's mentioned or triangulation or kind governance coordination across governance from the planetary down to the local, we probably need a conversation about the future. And this is I'm thinking when Kate mentions if she goes to work in the south of the United States and you can't talk about climate change but is there a way what can we as a profession, do to use the kind of tools we have in planning scenario to really make the possible to imagine alternatives in the future, the preferable ones, maybe none are perfect. And then how do we use those kinds of visual and imagination tools to kind of work back to the present. I think that that's another everybody's so focused on the current now which we have to be for the politics of her problem solving, but somehow or another I think we also have to inspire people to imagine a different well imagine what the future, what the data says the future looks like it's going to be an imagine something different and then work, find action strategies somewhere in there. And what Diane is making clear is that it's a false distinction to focus on process and product because if it's not a legitimate process. It won't matter what you actually produce the only other thing I would say is I thought it was great to have the panel. And with Jeannie talking about a financial need. And I just assumed Nora you were going to have IDP pay for everything in direct response so maybe we maybe we just open, open that up and, and we don't even need Q&A, but I, but I'd love to hear from others. Well, regarding the process and product, of course, you know, fully agree with that I think it's just, you know, it's like the loop where you just have to enter somewhere. And in terms of funding and I think that this relates relate to some of the of the questions. They are sovereign states, they are the one making the decisions on how to, how to spend funding. I mean of course you know the funding is not enough, but the priorities and what to invest is, is their own decisions you know they are they are, they are taking the loans. So, one of the, there is one question I'm going to read it out. It says, I am curious to know what the panel thinks about who are the main diffusers or drivers of the paradigm of resilience globally and in Latin America. I think now it goes to what Larry was saying. Is it academia, 100 or C states, development bands, private sector, and why. What are the motives to promote resilience instead of other paradigms, paradigms, such as energy water food nexus. I'm around, which it just moved, sorry, and a new question entered in this one. Give me one second. Something disappeared on that question but they think it was long enough already. So, so just to summarize who or what are the drivers of resilience in your view.  Larry, would you like to start or who is there. Yeah. I, I think, you know that the advantage and the problem of resilience is that it has been adapted by a variety of political perspectives, and, and so it, it can sound like a kind of neoliberal. We will congratulate you for figuring it out on your, your own kind of thing. And it can also seem like a very state driven investment kind of opportunity. And one of the problems that we have is that we don't agree on on what it, what it isn't. And if, if resilience isn't coupled with some kind of effort to include a differentially oriented beneficiary. But then we, we don't know who is benefiting from it. And, you know, so, so my hope is that we can put some kind of specifications on it so that when, when governments are deciding how to put their loans in into play, that, that everything that is done is done with a lens that that says who is this really benefiting and that the conversation about resilience is not separate from the conversations about inequality or informality or any of the other kinds of things. But, but it's seen as having distributional consequences. And, and I think that will, will matter no matter who's in charge of regulating the term, but, but people are going to have to act on it in particular ways, otherwise it will just mean whatever one anyone says it means. Maybe I'll jump in here it's I'm not sure it's exactly an answer to your question, Nora, but it's what I thought about it and Larry's also put it on the table to. I actually want to make an argument for the, the governance that this is a question of governance. And, and it's a question of sovereignty, getting back to my point, I hear exactly right Nora that for example the IDB has to deal. It's a, it's an organization I love the way Genie said we have these tools from the 20th century that are not helpful for the 20th century. We can say the same thing about governance institutions, many of them, including a place like you know something like the IDB which has to deal with the sovereign national state. Some of my work, I'm doing some work on the pandemic and some other sets of things, but I think we really need to question the scales of sovereignty that are appropriate for dealing with that for with a variety of problems and the client comment changes one of those. And I think that if we, if we kind of reamact we should redesign the appropriate governance institutions at whatever scale to deal with these problems. I mean that's not a new idea but and we have kind of global governance organizations that are the ICP, you know the climate change committees etc. But somehow or another they're not really able to push back against the national state sovereignty as kind of pushing things forward. I can't help but notice that this morning the New York Times there's a big article bubble scenario. There's global interest in having him do something but it's like his nation, he's controlling the Amazon for his political purposes. So, what I guess I'm going to close by saying is that this is this, this is about social movements that are connecting to calling on existent sovereignty arrangements that we have and calling out the problems that they're in capacity and the last thing I would say. And I fully aware that this might generate a, you know, a good conversation but I've been thinking a little bit more about promoting in the context of other projects, urban sovereignty. You know, and I know that's really fraught to think in a world of planetary urbanization where a lot of people don't even believe there's such a thing as the urban but I think a scale of sovereignty that's closer to the experience is going to mobilize a variety of factors, whether it's, you know, citizens, social movements, youth groups, private sector, etc, bringing conditions down to a scale of everyday experience. So it's not, you know, neighborhoods too small nation states too big. Could we think about ways in which we have associations and I know genies work with habitat, etc, does some of this of like cities, and thinking about triangulating, you know, sovereign, institutions that can push and mobilize resources and forces at a scale that's not that's not the planet and then it's not just at the neighborhood and I think that our discipline needs to be thinking about new institutions for doing those things. There are groups already Diane thinking in your way back there's something called UCLG which is 240,000 government members that is pushing for this very idea that you're talking about. But there are real reasons as Nora was saying why states are responsible for the financial piece it's their ability to operate on the capital markets and to reduce risk. You were talking about risk earlier and that's that's really part of this but to answer your question or who's driving it. There are many people driving it this whole panel is made up of visionaries right who are driving it. The public sector is driving it for political reasons just as Larry mentioned the private sector is driving it because of the desire to deal with risk and their investments civil society is driving it for a variety of reasons because civil society so large and so many different groups. So there's, there's the drivers are all over. And as many have mentioned there's no common definition of what they're after. And so that's some of the confusion that we have. Okay, I don't know if you have something to say we have, I mean, it's painful to cut this panel, you know, it just starts. I could only, I could only agree and I mean I kind of started off by saying in my own kind of work I've refocused on this notion of human rights in this conversation because, you know, even the lens of the city as a unit and I agree there's a a mismatch between unit of finance unit of governance unit of, of frankly project. And, you know, there are there's so much slippage in this in this in this in this realm you could think about the imperiled archipelago's in all over that are don't don't get covered and that are probably made up of indigenous communities or or you know there are many groups of groups that would probably fall between the cracks and in any kind of administrative concepts that we might advance here. So, you know, I, I do feel like in my own experience it's that that in any kind of resilience narrative, usually either the social falls away, or the equity piece falls away and you know you end up with kind of best practices or planning as usual but I do feel like what what I hear in all the panelists is that that it's that that this kind of critical aspect of, of social and physical and ecological and administrative governance like there's a sort of need to insist that these facets are all present and don't fall away towards a business as usual scenario and that we truly are kind of trying to think in an envision a future and I also to Diane's earlier point I also feel like that's why resilience somehow also has kept a stronghold or has developed a foothold in this discussion it's that it's somehow a future oriented term that enables an alternative vision that that that that can help rally action or behavior change. So, thank you so much I think that, as, as you know well we can go on forever with this topic it's not going to be sold but I mean it's, I think it's the beginning of the conversation is of course the beginning of this event this long afternoon event. And then we want to give time to the other panels coming. I just, you know, like want to thank you so much for opening up, you know, starting to open up these these questions about, you know, legitimacy, something that we need to talk more about risk, the scaling which the institutions that we need to think to use and how are those becoming tools for our, for our new vision of resilience and if resilience is the word even if that is the word that we should be using. So thank you so much. It has been an honor being with you and I want to also thank again the students for organizing this I think it's wonderful that they are thinking about this and doing this in a collaborative way and perhaps that's the best answer for our questions that's the way to work like many people from many different places, coming together and trying to talk about the issues. So thank you so much for all of you.


Okay, so hi, I'm Jessica Holman. I'm an assistant professor at the University of Washington. I study or research information visualization. And so my talk is called the visual uncertainty experience. And so when I proposed this talk, I thought it'd be cool to sort of spend all of my time talking about what I think is like the future of uncertainty visualization. New ways that designers are using and that I've been studying for showing uncertainty visually. But then I sat down and I started preparing my talk and I was like, ugh, if I'm gonna talk about uncertainty visualization at all, I have to talk about some not so exciting things, like probability distributions, variants. And so I was like, well, maybe I should retitle this talk so that people sort of know what they have coming. I could call it something like the trouble with uncertainty or maybe even more appropriately, I could call it stats 101 and people wouldn't know exactly what they were getting into. But then I thought, well, okay, maybe I can talk about these sort of fundamental aspects of uncertainty and the really exciting stuff and I'll just do it through a story. And so the story I wanna start with is out of Sarah Francis Galton. Galton was a Victorian statistician and Galton was thinking about, well, actually a lot of things, but one of those things was uncertainty. So Galton became interested in height, this physical property, and his interest was driven by a simple observation he made as he began to look around at the heights of people near him. He noticed that everyone has height, but we can't exactly predict what it will be. And this bugged Galton a lot because he wanted to know, how can I predict heights? I see heights all around me. So he actually starts measuring people's heights. So he's measuring families at first with children. He's trying to figure out sort of what is the set of possible heights? And so he starts to make observations from these measurements that he's taking. He has about 1,000 people that he measures. And so he figures out, for instance, that when you have parents that are unusually tall or small, the child's height is actually often closer to average. And this happens on both sides of average, whether the parents are very tall or whether they're very small. Similarly, the inverse is true. So if you have a child that's unusually tall or small, their parents tend to be closer to average and height. So what Galton had noticed here, we call regression to the mean. And he was really interested in this and he started to think, so what is this set of possible heights look like? And he kept making sort of more observations. So he looked at his measurements and he noticed that, well, the number of people with heights that are average plus or minus some amount, it decreases predictably and symmetrically as you move towards the extremes of height. And so he starts to see this vision in his head of what height looks like and it looks something like this. So here we just have the height scale and then to the right of that, you see this curve, looks like a bell curve. What he had discovered was actually in the normal distribution and you read it just in that as you move further right, that means there's a higher probability of having a height at that value. So Galton was really excited because he knew this was super powerful. So suddenly without any knowledge about the family's heights, other things that might help him predict, he could simply take the shape that he had and he could say, for instance, that given a child a random boy, his probability of being some height, say six one is 14%. So this was really, really powerful but he also knew that he could really, really confuse people with this because really what does this mean? So you have a random boy, you wanna know what's the probability he'll be six one, it's 14%. In actuality that boy is gonna either be six one or he's not. So what do we do with this 14%, what does that even mean? So he realized, well maybe if I communicate this as a frequency, people will get it because probability is kind of abstract and so he thought, well maybe I could say that like 100 boys, if we took 114 would be six one. So he decides, this is how I'm gonna communicate it and I wanna visually represent this. And so he draws a chart. So he draws his height scale and then next to that he has a bar and he decides to draw dots to show possible people. And so he draws a million dots and sort of the frequency of the dots in a given region means that's how many people there are with that height. And he annotates it. Actually he was a little bit wrong if you look at this. So height is not a perfect normal distribution. There are not as many sort of nine feet tall people as he predicted but that's okay. He basically discovered something really important. So what he had created was an uncertainty visualization. It's a distribution of possible heights that he's showing, he's representing it as frequency which ends up being important. People understand probability in terms of frequency much easier. So we could say from this example that uncertainty is simply the possibility that a quantity could take multiple values. And that makes uncertainty visualization the task of representing what those values are. So the interesting thing and I'm not the first person to say this is that most visualizations actually don't represent uncertainty. So people know, designers know and users know uncertainty is complicated. And so even though our data are always measurements they're always imperfect samples we often just don't show the uncertainty because we don't know what to do with it. And Galton also knew that uncertainty was complicated and so he'd done his best to visually represent it in his chart but really he was not very happy with this because he thought well if I wanna communicate this shape, this normal distribution I have to communicate it as a process. So it's a very abstract thing, any probability distribution but maybe people will understand what it is if I can show how it naturally arises from a bunch of events that are random basically. And so he makes something else to communicate this distribution which is more like a device. So he makes this thing out of wood and he has a funnel at the top and he drops little balls into it. And then with various probabilities the balls move through some pegboard and they sort of bin up at the bottom in these bins.  He said okay now I can communicate with my colleagues that this normal distribution arises from sampling people from the population assigning heights from the distribution and you get back the same thing. So he was happy with what he had made. So I just wanna pause here and point out that just in Galton's work we have sort of a dichotomy, two different ways to show uncertainty. On the one hand we're showing all of the probability distribution at once in a single static graph. On the other hand we're showing it sort of one outcome or one observation at a time. We're building up the distribution at the bottom but the focus is on the individual samples. So let's move away from Galton fast forward to uncertainty visualization today. What's interesting is that when we do see uncertainty and visualizations today we tend to see these same two approaches one is way more common. So on the one hand we have these sort of static plots like error bars, violin plots to show probability distributions. We could also use something like a gradient plot map it to opacity. And I like to call these static aggregate distribution plots overall or sad plots. On the other hand we sometimes see occasionally not very often what I would call like sample based or experienceable uncertainty visualizations closer to Galton's board. We're actually showing people the data in such a way that there's a focus on one observation or one sample at a time. We can do these so that they build in sort of the depiction of the probability distribution overall but a very different technique. So one of the things that I wanna convince you of today is that most of the time if we see uncertainty visualization at all it is using one of these static aggregate distribution approaches. And I wanna convince you that there's problems with those. So let's take a closer look at them. So here's a bunch of these techniques on the very far left. We have things like error bars which are designed to show an interval in which some value can fall something like a mean height or just a height measurement. And next to that we have a box plot also showing through lines basically graphical annotations different properties of a distribution like the quartiles which are points that break it up into four equal sized bins. So I call these approaches sort of summary marks because we're using lines to summarize properties of a distribution. On the other hand we see things or other approaches that take the probability and map it to a visual variable. So things like area in this violin plot which is basically sort of just a PDF turned on its side and doubled so that it's symmetric. And then we see things like the gradient plot next to that using opacity to show probability. Okay, so what are the problems with these things? Well, let's start with these summary marks. So in infobiz research we have these criteria that we use to judge whether a visualization is gonna be successful. And one of them is which we can sort of summarize as expressiveness is basically does it express the data in such a way that the user's natural interpretations are accurate? So we don't wanna put something in front of the user that's gonna mislead them. And we see these summary mark based approaches sort of fail sometimes in this regard. So one thing that people do when they get error bars is do what or they show what's called within the bar bias. So they basically look at the error bar they see that half of it's on top of the data and so they think, oh, that part has to be more probable. Actually, this is not true. And so this is one problem. Another problem we have is that we know people don't like uncertainty, it's just complicated. And so when you show it as these lines on top of the data it's really easy for people to just sort of squint and say like, oh, I don't really wanna deal with that and it just kind of goes away. And they just, they don't use it at all which is a problem because we want people to incorporate the uncertainty into their judgments. Okay, so there's other problems that tend to affect the approaches that use visual variables to show the probability distribution. And so this other criteria that we use in visualization is called effectiveness. It's basically how easily can a person read the data values back from the visualization.  So scatter plots are great, length is also good. The problem is that often when we're visualizing on certainty we're already using those really good channels for other data to show the data itself for instance. And so we're left with these less effective channels like area which is hard for people, opacity also pretty hard for people. And so they basically can't read the probabilities back so that's not good. Okay, so these are what I would call visual problems with these static aggregate distribution plots. There's also what I would call model problems that deal with what we're showing people and how complicated and abstract that is. So recall uncertainty visualization is representing the possible values that a quantity can take. So things get complicated because what quantity we're talking about can differ. So on the one hand we could be talking about sort of the actual measurements.  On the other hand we could talk about a quantity that is actually a statistic that we calculate given a set of measurements. So what does the average height look like? What's that set of possible values? Two different things. Usually when we're looking at uncertainty visualizations things like error bars, we're looking at the ladder. So we're looking at the distribution of something like average height which is very different from the distribution of actual height measurements and it's called the sampling distribution of the mean. And it's basically related to the data distribution so this red SD is the standard deviation of that sampling distribution or standard error. We can look at the data distribution and the standard deviation divided by the square root of N which is the number of samples but nobody really understands this and they're not sort of broader audiences and even statisticians struggle with this. So people don't understand how sample size relates to variance and so this distribution you sort of tell them that's what they're seeing. I think maybe they think still that they're seeing the data distribution. You tell them it's the sampling distribution it's just like it does not compute at all. Okay so if I were gonna summarize these static aggregate distribution plots and their limitations that we've gone over I think I'd have to say that they are the folk music of uncertainty visualization and so I mean this in the most scientific way so I have some folk music for you to listen to so let's think about what this means. How many times can a man turn his head and pretend that he just doesn't see? Okay so why do I say this? They're the folk music of uncertainty this. For one folk music has complicated abstract lyrics lots of metaphors nobody really knows what the song is about they argue is it about politics is it about drugs we don't know and similarly static aggregate distribution plots things like error bars they represent something complicated abstract people don't really know what they're looking for or looking at. Folk music is also often unplugged so they could have used technology like amplifiers synthesizers they chose not to and similarly these static aggregate distribution plots don't make use of things like interactivity and animation which could help us sort of convey uncertainty to people. Finally folk music the lyrics often raise questions and then they don't answer them and I think similarly when we see things like error bars on visualizations we there are actually questions that we wanna answer and then we can't and it's really frustrating so I think this last point is really important so to me at least visualization is about comparisons so we wanna put the data in front of people in such a way that they can accurately compare the values and that's why in visualization research we care about graphical perception studies because they tell us what's the most accurate visual encoding it's position et cetera so we really care about comparisons and differences and the problem is that when we put a visualization like a bar chart with error bars in front of people they want to look at the differences and what it suggests that they should do is ask themselves are these differences reliable are they likely to repeat if I did this study again et cetera but they can't make that judgment so for instance I might have done an experiment I'm looking at blood pressure drugs or new drugs so I have my treatment I have my control the treatment group has lower blood pressure but obviously there's uncertainty because I have a limited number of people and so I want my readers when I show them a plot like this to be able to say sort of what's the probability that this effect would repeat again if I reran the study what's the probability even that I would see an effect of at least 10 percentage points nobody can answer that question easily and so we might think well people are rational right so they know they can't use the visualization to judge to make judgments of reliability and so they're smart they'll go look at other statistics you know significance testing which has problems of its own but that's actually not what people do so what they do is use the chart anyway because they can still make a judgment with it and so this has led Daniel Kahneman to say that when called upon to judge probability people actually judge something else and believe they've judged probability so this is problematic so what else are they judging if they see for instance error bars on a bar chart and they're trying to judge reliability so they might be judging for instance how much they like the authors of the study that's gonna tell them how reliable they think it is what I've seen when I've done studies to see how people interpret error bars is that they often look just at the difference in the means or the difference in bar heights regardless of the uncertainty so they'll say oh a big difference has to be reliable regardless and a small difference is always reliable so this does not always work but what they're doing is using what we call a heuristic it's a mental shortcut people use them a lot when thinking about uncertainty and probability they don't know how to do answer the question so they use some other information and heuristics help us reduce complexity we can answer questions more easily but we use them so often and they're so familiar that we end up actually being confident even though we're using these flawed sort of mental shortcuts and I think to people who are visualizing uncertainty this is really scary because what it means is that we can put a crappy uncertainty visualization in front of someone they don't know how to interpret it and yet they do anyway and they feel confident so that's not a good thing okay so what can help us so we see there's all these limitations of static aggregate distribution plots and sort of uncertainty in general so what's the answer so one thing that I've been thinking about which I think is sort of one path that could potentially help is to sort of step away from conventions of modeling uncertainty and how we tend to think about it and graph it in statistics and instead think about how we experience uncertainty and uncertain events in our everyday lives so for instance maybe I take a bus home from work every day and so every day I'm sort of waiting for the bus and I get a sense of when it's going to arrive and how that compares to its scheduled arrival times so maybe five out of 10 times I know the bus will be within one minute of scheduled time maybe another three out of 10 times it's gonna be two to three minutes late two out of 10 times it's really late, et cetera so I'm basically building a probability distribution in my head but it does not feel like that it's just a sense of expectation that I get from observing things over time and this has led Laplace, a mathematician to say that the theory of probabilities is basically just common sense reduced to calculus it makes one appreciate with exactness that which accurate minds feel with a sort of instinct so how can we draw on this when visualizing uncertainty how can we sort of show uncertainty in a way that kind of matches the way people are used to experiencing it so one thing that I showed briefly earlier is what I would call sort of an experienced based on certainty visualization where you're actually showing someone what the variation looks like so here's an example from the New York Times and they wanted to communicate to people how the jobs report which is sort of an interpretation of things like unemployment and job growth rates can be highly sensitive to variation in the months preceding whenever the report was written so what we care about is the overall annual job growth rates but we can write reports that are very sensitive to sort of what happened the last two months so to show this to the user they have all these headlines that could occur from the exact same annual rate depending on local variation and then at the bottom they have graphs so where we actually see on the far left sort of what we might expect if the job growth rate was actually steady but this is sort of hypothetical we would never see this this is a case where there's no random variation but then next to that we actually see okay so let's take this steady growth rate and show what it looks like with random variation so suddenly people get a sense of oh so I could have a steady growth rate and this is actually what I would see there's a lot of variation then next to that we're showing on the right side what we would see if it was actually accelerating but without random variation and then next to that what happens with random variation so people can suddenly sort of watch randomness play out in the actual data that they have and see okay these are other possible outcomes another thing I like about these types of visualizations is that we can as designers build in or build on people's natural associations with randomness so people do have experiences with randomness through things like coin flips and gambling and so we can sort of build this in and give them sort of a fun experience with uncertainty so this is from the New York Times also and they're actually using roulette wheels to show you in a way that you can play again and again what the outcomes could be for states for different states for the election finally I think these types of visualizations that are showing you sort of individual samples possible values are better at showing the process and in that way better at communicating what a probability distribution actually is so I think actually Nicky Case showed this one from Nathan Yao but we're basically seeing sort of the probability of living to the next year a bunch of different possible outcomes given some gender and some age and it's actually building up this little histogram as it goes so people sort of have a better intuition for what that histogram really stands for okay and then I think most importantly if we show uncertainty as sort of sets of possible values actual possible outcomes people can suddenly have a chance at answering questions about how reliable differences in the data are so we know this is hard with error bars but imagine that I'm showing sort of just a set of outcomes so suddenly someone can watch this over time and say oh I think about 70% of the time the treatment's gonna have a lower value than the control and they can even get more precise perhaps with some interactive help they could look for things like how often there's an effect of a given size or more another really cool thing is that you can show joint probabilities with these sample based plots you cannot show with the static aggregate plots so imagine in one possible world I've run my study of the blood pressure drug and I've done it as a between subject study so different people in control and treatment but then in another world I decided I wanted to run it as a within subject study so suddenly I have the same people in the control and the treatment they're doing both in randomized orders so what these sample plots will show that you can't see from other visualizations is when you have correlations between multiple variables so I can watch this thing and I notice oh when the control goes up the treatment tends to go up as well same vice versa and that's because it's the same person so there's correlation there and I can suddenly see that so what this is leading up to really is this thought I have that these experienceable uncertainty visualizations that show possible values really have to be sort of the disco funk of uncertain divas so let's think about what that means so I think you get the point so why do I say this well for one when you go to a discotech or a rave you kind of want to feel the music on a sensory level so you actually want your organs to vibrate and I think similarly some of these experienceable uncertainty visualizations are showing you uncertainty in a way that it becomes sort of a perceptual experience it's showing uncertainty as frequency over time which is something that we can relate to disco funk uses a lot of technology there's a lot of synthesizers you have DJs sort of mixing the same songs over and over with different sound effects and I think similarly these types of visualizations allow us to use things like animation and interactivity and I think we're just beginning really to explore sort of how we can use interactivity to set up kind of a conversation between the user and the uncertainty in the data finally anyone could get in to disco funk you don't have to know the history of that type of music to enjoy it and so there's this low barrier to entry and I think the same is true of these experienceable uncertainty visualizations so people can understand randomness without a statistics background so I just wanna spend my last sort of five minutes talking about how we could create these things so there's, cause there's sort of a general formula and I wanna show you really that you could do this for different types of data and for different types of visualizations so there's sort of three steps so first step you wanna generate from your data these hypothetical outcomes are hypothetical samples and the goal here is that we wanna actually generate these samples in such a way that they are representative so if we were to re-run our measurement process or our data collection process we could have gotten these other values so they're plausible and representative so how do we do that? Well, we could do that in a few ways so imagine we have our data set where we have our control and our treatment for this blood pressure study one thing we could do is simply look at what sort of distributions we have in our actual data and then sample from those distributions so maybe I find that my control and treatment are both normally distributed and they have a given mean so control has one mean and one standard deviation treatment has another and then I just sample over and over from distributions that are set to those parameters another way that I can do it even better because I don't have to assume what kind of distributions I have is to use bootstrapping or re-sampling with replacement from the original data set and Amelia actually talked about this a little as well and so nice thing about bootstrapping is that there's ways to do it for almost any type of data and there's a lot of literature out there there's books, there's stuff online so if you just Google it and what type of data you have you can find sort of guidance okay, so the second step is then to take all of these hypothetical samples we've made you wanna make a lot sort of like a thousand would be great and then visualize them so that they become separate frames in a visualization and to do this we wanna make sure that you can then compare the frames one to another pretty easily and so in most cases that's pretty easy in some cases it's trickier depending on what sort of visualizations you're using but sort of an example of why we need to do this would be to imagine that I have these bar charts where I'm showing the data using height of bar but I have different y-axis ranges on every bar chart that would be really hard for people to compare it's just like it's not very easy at all so we wanna make sure that those mappings are the same or are our functions that set the scales etc are the same finally we present them and you can do this in many ways and I think we're just starting to explore how to do it so you can do small multiples if you can shrink them down and people can still see the individual outcomes because you wanna show a lot you can use animation and I think there's a ton of possibilities for how to do the animation so you can do it sort of pure where you're only showing one sample at a time you could add the summary marks like the mean etc you could have it aggregate in to show the distribution you could have it be interactive so if you think it's too distracting to have the animation maybe people can interact and turn that on when they need it so I think there's so many possibilities here so the last example I wanna do is to show you that you can apply this type of technique to any data set really and so I thought I'd walk through this example from the New York Times so what they wanted to show is that we can use poll data to predict who's gonna be in the Republican debate so they did this before the debate happened but they wanna show how poll results are uncertain and so how we could actually run the same poll over and over and we might see different people predicted as debate candidates so they wanna basically show what happens when you have sampling errors cause polls are always based on smaller numbers of people than the actual population so there's error there so how do they do it so they're doing this all in JavaScript I just looked at their code and what they're doing is basically simulating many polls and showing different rankings that you get based on each poll so what they're gonna do is start at zero and they're gonna take the most recent poll results from the candidates the different Republican candidates so maybe in the recent poll Trump had 18% Bush had 14% et cetera and then starting at zero they're gonna create a bin for the percentage for each candidate and so for instance Trump would get zero to 18 then the next one we're just gonna build on top of that so Bush gets 18 to 32 Walker gets 32 to 43 et cetera so we do this for all the candidates and it should sum to 100 and then what we're gonna do is actually simulate each poll so we're gonna simulate a large number of polls maybe a thousand times and then for each poll we'll decide how big it's gonna be I think like 2,000 is sort of a common poll number so then for each of 2,000 people that we're gonna simulate we're just gonna draw a random number between zero and 100 and then put it in the corresponding bin for whatever candidate it falls in so the first one goes to Walker the second one goes to Trump et cetera so we do this for 2,000 people and then we've simulated a poll and we're doing this for many many polls and so then all we have to do really is for each poll after we've sort of put all the people in the bins we calculate the new percentage for each candidate we sort those and assign rank sometimes the ranks are gonna change so people are gonna swap so we're actually gonna predict different people are gonna be in the debate when we do this and then the last step is simply to visualize it so we can use something like D3 visualize or create a function that visualizes a set of ranked candidates in order to make it easier to compare we're gonna animate transitions so that people don't have to keep track of where each candidate was they can sort of follow just from the transition okay so I think I'm about out of time so thanks for your attention if you're interested in more on this technique of experienceable uncertainty visualizations you can check out my website I've also written some stuff on the interactive data lab blog on Medium so thanks I think we have time for maybe one question if there is any so you can study it in the same way that you study the other stuff so people are just beginning to do stuff on like looking at how people average a bunch of dots over time so we don't totally know so obviously it's different and in some cases it might be less precise but I think that's one of the goals is sort of understanding how to design these things so that it actually works so basically it's an open question there are a few studies that I could point you to but we don't totally know how it compares yet so awesome thank you so much