We are joined once again by pre-court co-director Sally Benson, who will be moderating our Stanford Educators panel. She is joined by three Stanford deans. We have Jonathan Levin, dean at the Stanford Graduate School of Business, Deborah Sats, dean of the School of Humanities and Sciences, and Jennifer Wittem, dean of the School of Engineering. So, Sally, over to you. Okay. Well, terrific. So, to all of you and the energy of Stanford and Slack program, I hope you've been enjoying it. It seems by your participation in the chat function, you're highly engaged, which is awesome. And thanks to Alicia for that great talk. So, we're really lucky today to have three of the deans from Stanford University joining us. And one of the most fantastic things I think about Stanford is that students, even though they're enrolled in a particular school and even a particular department, that they often take classes across the entire university. So, it's particularly relevant to have these deans here. Unfortunately, two other deans that we invited, Steve Graham from the School of Earth, wasn't available as well as Dean Moeller, the dean of research. But thank you, John, Deborah, and Jennifer for joining us. So, just to let you know a little bit more about this class of incoming graduate students, 33 are from the GSB, 67 are from engineering, 17 are from H&S, and eight from Earth. And we even have someone from the medical school. So, very diverse cohort of students that we have this year. So, to kick things off, what I would like to do is ask the first question, and really for all of you, what kind of energy-related teaching and research is taking place in your school, just so we can broaden the knowledge base for the students here? And maybe we'll start with John, if that's okay. Sure, thank you, Sally. And I really appreciate the chance to be here. It's great to see so many GSB students who are interested in energy and sustainability participating in this. And welcome all of you to Stanford. It's great to have everyone and get started with what's going to be quite an extraordinary year. So, energy at the GSB, it's actually hard to know where to begin, because there's lots going on. And I think on the education side, some of you are probably, who are GSBers are probably thinking about the EIPER joint degree program. That has become one of our most popular joint degree programs. The interest in both energy and climate sustainability has just been going up and up. And on the faculty side in classes, we are now offering a whole set of electives around climate. And a lot of faculty have gotten interested in sustainability issues over the in recent years. And that actually spans across all different types of perspectives on management. So, we have faculty in operations who are really interested in issues like sustainable supply chains. We have folks who are in our political economy group who are interested in government policy and regulation around energy, around climate, around sustainability. We have faculty in our economics group who are interested from an economic perspective. We just hired a finance professor about a year ago who is interested in basically long dated securities and securities that are perpetuities or last for a long time. And when you have long dated securities, it turns out that that the effects of climate change matter for how they're priced. So, he's gotten into thinking about climate, but from a completely different perspective. And then we, one of the features at the GSB is that we often, much of our teaching, particularly electives, is done by practitioners who we bring in from outside. So, we teach with our 10-year line faculty and sometimes to teach on their own. And we've increasingly been trying to bring people in who have an energy background or who have an interest in sustainability from a business perspective to offer classes that would I talk about business models that will support the energy transition or that have explicit sustainability goals. So, you know, it's a big topic of discussion at the school and I think you'll all experience that. And it's a really important discussion. And if you think about, you know, where the, what is, what are the, where is the potential to drive really important change in energy and in sustainability? I think we all understand that public policy, a carbon tax or something like that would be the natural global solution. But in the absence of something like that happening, it's going to have to come from the private sector and from business innovation and from different business models that are going to drive change. And I think that's something that, you know, we were talking about and thinking about all the time at the GSB. Okay. Well, terrific. Thanks very much. How about moving over to Deborah? So, Echo, John's welcome. We're really glad and excited to welcome you to Stanford. It will be a challenging year. And the way I'm thinking of the year, it's a year we all need to have a kind of double focus. There's a lot going on in the world outside that is stressful and challenging and exhausting. And we need to focus on riding the waves and making sure we all get the support we need at the same time that we need to move forward with the very important teaching and research missions of the university and to really focus in. And that's going to be a challenge. But we are committed, I think, as a school and a university to making sure that the, you know, transformational, innovative, exciting, creative work of our grad students and our faculty and our postdocs continues. And we're thrilled that you're here. So let me say a little bit about the school of humanities and sciences. We're a super broad school, unlike many of the other schools. We're not unified around a kind of broad topic area. But we span, you know, people who work on dark matter and dark energy to people who work on medieval poetry. And it's quite an array. We have 23 different departments and 23 different interdisciplinary programs. But energy and sustainability and managing the environmental crises we face are really interdisciplinary and are going to call on expertise from many different types of people from economists to political scientists to biologists and chemists to historians. And you, it's really not, I think, of energy and the environment in general, you know, it's not the primary possession of any one area of knowledge, but we'll really need all of us working on this because there are a lot of reasons, some purely technological, some political, some ethical that have made finding solutions so difficult. And so the school really, I can't say every department contributes equally to the study of energy or has it essential, but it really, you know, it ranges from obviously people in our chemistry department working on catalysts and catalytic, you know, agents that might, you know, lower the costs of certain and make more efficient certain kinds of processes, chemical processes, people working on batteries, on creating sustainable plastics, on fundamental research, not oriented to one particular area, but that we think will advance our understanding of molecules and how they come together and ways we can intervene. As like John in the business school, we have a bunch of economists who are working on policy issues from questions about optimal taxation to emissions trading policies to thinking about the regulation of air quality is very much a factor right now, as you know, in California. And we have historians who work on the conditions that have led governments to choose sustainable or unsustainable paths of agricultural development and what we can learn from them. And of course, we have biologists who study, you know, various kinds of energy sources from biological processes and biological materials. So it's, there's a tremendous amount to offer. I think from the school, we're all excited about the possibility of some new structure of school of sustainability and some structure that helps bring more of us together. Because one of the things that Stanford is great at is creating phenomenal research, but kind of disparate. And there's so much going on often, it's really hard to keep track of what there is. And we have an opportunity here to really become much more than the sum of our parts. And so we're excited about that possibility. But please, there's lots of stuff in the School of H&S related to energy research and courses that deal with these issues. Okay, thank you very much. It's so incredibly broad, the School of H&S and so much to contribute. So maybe we'll move over to Jennifer Wittem, the Dean of the School of Engineering. Hi, everyone. I'm Jennifer. Get ready with your raise hand button if you're joining the School of Engineering as a new graduate student, because I'm just going to do a little survey. Don't raise it yet. Oh, sorry. Just get ready. I'm going to be just doing a survey because I'm going to guess that we probably have in this group people from all nine of our engineering departments. We'll see. So there's 60 some of you, as I understand, 60 something engineers in this cohort. So when I look at what goes on across the School of Engineering, we do have nine departments and I'm going to enumerate them in a moment. There's really two areas, I would say, maybe three, that there's a tremendous amount of cross cutting work and a tremendous number of faculty working in applications in those areas. And they probably won't surprise you. One of them is human health and a full one third about the faculty in the School of Engineering actually have joint projects with people in the medical school. And the other area that's just really prevalent and becoming more so every year is the area of energy and sustainability. And I think close to a third of our faculty are actually also working in some aspect of energy. And so I was just going to mention our departments and for each one that what they're doing just very briefly and also just take a survey and how many of you are coming into material science? Raise your hand, do a raise hand and yeah. So not surprisingly, quite oh wow lots of materials science. So there's all kinds of things going on with batteries and materials materials for batteries and for energy storage. What about chemical, let's let those hands. What about chemical engineering? I'm sure we have some chemical engineers and raise your hand if you're a chemical engineer. Chemical engineering, just a couple that's a little surprising to me because a lot going on in chemical engineering as well. And in our material science and chemical engineers often work very closely together many partnerships, they're similar areas. And again, sort of like Deborah mentioned many people working in catalysts and so forth. What about bioengineering? Anyone here from bioengineering, a raise hand? All right, so that's next year we got to get some bioengineers in here because they're also, you know, they're people starting to work on biofuels and I think there's quite a close relationship with some of the work going on with plant biology as well. What about civil and environmental engineering? I hope we've got a great, awesome, there you know we have our civil and environmental engineering departments are combined. On the civil side there's a great interest now in sustainable urban systems. So just thinking about the, well the world was getting urbanized and it still is as a world though some small pockets I think in the U.S. are unurbanizing a bit but just looking at energy use and sustainability in urban areas is a huge area of our civil engineers and of course our environmental engineers are interested broadly in energy distribution and more broadly in environmental areas. What about mechanical engineering? Anybody from mechanical engineering? Great, oh quite a few and I'll just pause while you unraise and ask about electrical engineering. I hope we have some electrical engineers there as well, we do. So and across ME and WE of course people are also working on batteries and just I like to think of the people in those departments as working on energy efficient everything. So good group there. All right I am very hopeful that we've got some people from management science and engineering, anybody from management science and engineering? There we've got a lot of work now on energy policy on optimization. Our management science and engineering department really does go broadly from policy areas all the way over to hardcore operations research and all of those relevant to energy. Computer scientists we all know software is super important in all kinds of aspects of energy. How many computer scientists do we have here? Just one. So that's an area for growth next year as well and then the last one I'm going to ask about is aeronautics and astronautics. Do we have anybody from aeroastro because there's good that's great because there's some really interesting work now going on in energy efficient spacecraft and even like fleets of microsatellites that is I think is truly fascinating. So sorry to do the inventory there but we are a school that's small enough that we can do an inventory of our departments and again with the theme being we've got people across all nine of our departments working in the energy area and of course they're also working together collaborating with each other. I'm sure you've heard over and over how collaborative Stanford is and and we really are that's very low barriers to collaboration and we have it within the school and across the entire university. So welcome to Stanford Engineering. Well terrific thank you all very much that's a great overview. So moving on to the next topic that addressing challenges in global energy access and climate solutions requires an interdisciplinary approach and systems thinking. So the question is this for you how do you think about this and what opportunities are provided in your school for developing these perspectives and skills and maybe we'll go in reverse order maybe Jennifer will will start with you. Well I guess I just pick up where I left off which is that you know this is absolutely a global challenge and a challenge that touches every part of the university and so interdisciplinary is key we've been moving in the school of engineering towards what we call catalyst grants where we have groups of faculty across many broad areas coming together and we seed those groups with fairly significant funding to get started on projects that will be very hard to start at that scale and we have also partnered with the Precord Institute in doing this type of thing as well and it will I think we'll see more and more of that there's more of an appetite. So I do think one of the one of the issues with a global challenge of this scale is that sometimes in the university setting it is hard to launch a group that's of the scale that you need to really get going and prove that you're going to make progress. It can't just be a bunch of small individuals with their smaller medium-sized grants and so a lot of on the research side we're very interested in assembling those groups and working across the university to do that. Okay terrific okay thank you Deborah. So I'm just gonna I think basically echo what Jennifer just said. The barriers to interdisciplinary collaboration here are as low as I've ever seen. The problems around energy and sustainability are not problems whose solution can rest with any one discipline. As I said it's not just the technical set of questions we face but a bunch of policy questions. There's some ethical aspects to what's fair. There are you know many dimensions to the challenges we face around energy and climate. A couple of the initiatives coming out of the long-range planning of the university I think have a also an opportunity to contribute. So I think of again the sustainability school is a still a work in construction you know in theoretical construction but it will be a way to knit these different groups and faculty across all of Stanford's seven schools together in an exciting way. One thing in that's a university priority that's housed in the School of Humanities and Sciences is the Stanford Impact Labs that seeks to bring social scientists together with data scientists together potentially with scientists with natural scientists in partnership with NGOs, communities, state governments, city governments around naughty problems and climate is definitely one of the problems that that group you know it'll be open to grants teams applying and so far we've had teams from all over the university being chosen and there's a lot of interest in those space of environmental problems. So I think some of where the university is going around catalyzing to go back to catalyst research and figuring out ways to increase the power of the research we do and bring it to the public really fit with you know the interdisciplinary nature of a lot of the social problems we're grappling with today. Okay thank you. So John how about you? Well I would second everything that that Deborah and Jennifer just said about interdisciplinary research and and also just add that you know when you when we think about a problem like energy transition leading to a lower carbon world you know I think we all understand that there's there's technical aspects to that problem there's going to need to be a lot of technological breakthroughs in areas like batteries and clean energy otherwise just can't happen it's a political problem because the almost truly has to be some spur from regulation it's a business problem and it's a leadership problem it's so all of those things have to come together if we're going to have a dramatic change in in energy and climate over the 21st century and Stanford has all of those things they're all they're all here on campus expertise in in every one of those things and I think one of the challenges for students is just finding them and and Jennifer and Deborah alluded to this is just how do you how do you connect with all of that expertise across the campus how do you connect with students who are in all of those those areas and I think you know one of the things that's great about an event like this is it's just a way to start to see where different things are located and who the different students are and and so forth and I would just encourage you to try to seek out some of those those things as you go through the the program yesterday I was we ran a summer entrepreneurship program for our MBA students who are between year one and year two over the summer and yesterday I was at the final presentations and one of the things that really struck me in seeing the student presentations was how many of the GSB students who had had had an idea and were pushing it forward over the summer had had found someone or a group of people to work with from other Stanford schools it was and whether it was from the engineering school or the medical school or different different places around campus that was really really striking how many partnerships had had formed in that way around really interesting types of ideas around energy or healthcare or whatever it was I think you know that's and that's going to be a little harder this year frankly because we won't have the chance for serendipitous meetings on campus so it's going to take a little more proactive effort to get out there on you know to figure out ways to have those connections but that is an amazing opportunity about being at Stanford you know with with all of these different lenses that you can tackle come at a problem from so I you know encourage you to take advantage of that opportunity in every way that you can yeah so so just a follow-up is that the ignite program that you're talking about oh well I can actually this was a different program that we just set up different for that that we just set up as a experiment during COVID which was the Bosa Chan innovation internship program which was a program for GSB students to get funded to go that entrepreneurial part of the summer but we do for students outside the GSB we run a program called ignite which we ran virtually this summer and we'll run again during the year and it's a it's a wonderful program this is a program where you can you basically get a it's an introduction to management and entrepreneurship and it's targeted at students in other schools around the GSB and you can either do it part time during the year or you can do it over a month in the summer and I encourage you know invite everyone who's outside the GSB to apply particularly if you've got an interest and you've got an entrepreneurial banner even if you don't but you're just sort of intrigued to think about how work you're doing might eventually be commercialized or how you could be you know have that capability in your in your back pocket yeah I know that's a fantastic program I've had a number of my students take it so I highly encourage all of you to look into that so so just a little bit more I think the the theme you got you know regarding this question about addressing you know having getting an interdisciplinary perspective is the Precourt Institute has a number of initiatives there's the bits and watts initiative which is about grid modernization and it integrates all the way from fundamental innovation of a better a better gadget all the way through to business models and so forth and and there's one on natural gas the natural gas initiative there's the storage x initiative which we just launched about six months ago and we also have a new carbon dioxide removal initiative that's coming along so I highly encourage you to find a way to get engaged with those as a way of developing this sort of systems thinking we also have a fantastic class called the energy ventures class that is taught by two entrepreneurs and it's Joel Moxley and Dave Danielson and typically about a third of the students are from the business school a quarter maybe from the law school and then the other half from engineering and humanities and science and so forth so so seek out those opportunities because I think it's really by experience that you'll learn those things um I was going to ask you all about this new school of sustainability but you've all sort of touched on that so I'd like to jump to really the next uh the next question which is you know what advice do you have for incoming graduate students as they begin their studies and and given that these COVID-19 days perhaps you could sort of address something really specific do you have any suggestions because we've all been living with this now for about six months and and I think we we all have our our favorite solutions but what don't do what don't we start with with Debra you know an easy question uh you know graduate school is a amazing wonderful uh opportunity of a kind that's just a gift uh to have these years to pursue uh your interests wherever they leave you and uh to have the chance to find out for yourself what matters to you and to push it in these new directions we're counting on you uh so it's it's an incredible opportunity it's also even in the best of times uh can be uh challenging uh can be isolating can sometimes feel lonely and I think all of that is heightened at this moment um in the COVID world I mean I I have spent a lot of the last five months on zoom and I will say you cannot zoom the things that really matter uh you know the energy uh what John referred to is those serendipitous conversations and meetings that you have with people uh there's no substitute for those in-person collisions uh so we're in a second we're not maybe second maybe third best world but I would urge you to find ways to in this time be connected reach out take advantage of resources take advantage of the people go you know to zoom meetings or classes and really try to create community find out you know so one thing we have done better on the internet of you know it's easier to survey all the different options that are going on and take care of yourselves because this is just a really challenging time for all of us and it's exhausting and it's really really important like it's the backdrop to your everything you do in the world of knowledge and education that you are uh you know you're in recently good health and good spirits and maintain a kind of hope um in a in a very challenging time so we have a lot of resources on the campus there are university resources to help graduate students um I or somebody else could send those around you can also go to the VP GE the vice provost for graduate education website a lot of the departments and I hope all of them are stepping up and really trying to create community gatherings where people exchange not just ideas and research but also remind themselves why it's valuable for us to be together the I think I remember talking about this sort of at the beginning of the crisis and John I remember you're saying and I completely agree with this that the crisis has focuses focused us so much on what really matters and what makes Stanford great is the people who are here and what they're able to do together um by being here together and we just have to find as best we can in this hard time ways to keep doing that um and know that you know we'll get through this and uh we will have those serendipitous meetings and uh and conversations that blow you away and change the direction of your thinking that will happen again and some of that can happen on zoom it's just not uh ideal you know one thing I would say is I I think in a way even you know despite all the challenges that that COVID has brought this isn't this is an amazing time to be in school the world is just going through an incredible period of change and on many different dimensions we're having this incredible acceleration of comfort with technology and and and this reckoning with societal issues racism and but also things like thinking about other types of inequities in society around health and education and the and the effects of of climate change and you know the world the world is not going back to January 2020 it's going to be different when we get through this experience and hopefully it's going to be better and and I you know the our challenge is to sort of figure out how how will we make it better when we come out of COVID and and not not necessarily how will we how will we get back to what we had before and so you know in a way at a time like that it's just a great time to be able to to be able to invest in yourselves and to think and to reflect and to sort of figure out you know what is the path that you want to chart for yourselves when when we get through this it's a great time to to launch your your career whether it's going to be a career in research or in business or in policy or whatever direction you end up you end up going in this group's going to go in all different different directions so I mean that way I'm I'm I'm very optimistic about the experience you're going to have and I think Deborah's point is just spot on in terms of you know the the work that is going to have to go in this year to to making sure that we realize the what is so great about Stanford which is the people and and making sure that that you get the opportunities to meet people and to form relationships that are going to last and to learn from everyone on the campus even though some of that learning is going to have to be in this remote technology mediated environment but hopefully we'll we'll get through that too and and be able to get back to to doing what the things that we love in person so I'm excited for all of you I think I think you're actually at a great time in many ways this is like the perfect time to be starting graduate school much as that might be counterintuitive so you know I just I think putting in the effort to try to make sure you meet people and form relationships and like Deborah said you know just understanding that when things do go wrong this year and they're big challenges you will not be the only person who is feeling that way on campus there's going to be a lot of people who who are with you in spirit and and there to help you okay thank you that's uh those inspiring words how do we make it better when we come out of this so so Jennifer over over to you yeah as always going third is difficult because I'm you know want to say similar things to what Deborah and John said but actually they kind of got me thinking one way you could go into this year is saying wow this is the moment in my life where I've you know I'm coming to Stanford and it's entirely ruined because of this you know virus and everything else going on and how could my first year of graduate school be ruined or you can look at it as wow this is a time when I'm going to be able to make a difference in how we come out of this and of course it's much better to take that second viewpoint and graduate school I always tell people I think graduate school can be some of the best time of your life you're sort of through some of the anxieties of undergraduate you've probably know more or less what you want to do with your career you're getting to specialize you're getting to contribute you're part of a cohort and so I just think it's a really exciting time so if any of you are thinking about being a professor enjoy graduate school because once you're a professor boy you've got a lot more responsibility so but it's I mean being a professor is awesome as well but anyway graduate school is a great time you know on on the personal side if my recommendation if you are living on campus is to get outside you can see I'm outside this is not a fake backdrop this is my own backyard I live near the foot of the dish trail for those of you who have become familiar so get outside walk the dish walk around we're very fortunate that we're in California and have good weather once this smoke goes away but you know I'm thinking this is going to be a year where people are going to be outside a lot for the year or try to get outside and we're going to be able to do that because of our simply because of our weather and I've been taking walks every evening and fairly often walk through the graduate student dorms including the the brand new ones and I see a community building there so you know put on your mask and go outside and meet other graduate students if you are living on campus that's what I would do and you'll meet some really interesting people so I do think that even though we have a lot of restrictions there are you're if you are here you're in a setting where you can still meet people if you're not local I hope you're in a okay situation and you try you know we're going to have all kinds of opportunities for you to connect virtually so again you know take advantage of everything Stanford has to offer even in this unusual time okay well thank you for that good advice you know I'm going to put you all on this spot this whole conversation has made me realize that it's a little bit more difficult to get to know people during these times and we have to take every opportunity to try to do that and of course the three of you have very important leadership positions at the university but you're also incredible academic scholars and and I just wanted to ask each of you to spend you know maybe two three minutes just describing the kind of research that you do the kind of questions that you're passionate about that have you know driven you to get to the point where you are and and we'll start with you Jennifer and apologies for springing this on you but I it's such a great chance for people to get to know you all yeah so no I'd be happy to uh happy to to talk about that briefly I'll start by saying that I was a music major undergraduate so I I'm the only dean of engineering anywhere with a bachelor's degree in trumpet performance I'm pretty sure about that um but and that takes a lot of energy but anyway I did switch over to computer science so I am a computer scientist and my research area when I began my career was considered one of the most boring areas of computer science database management um all we did then was build systems that handled large amounts of data and how boring could that be and at the time that you know the the people who are interested in using large amounts of data were people like banks and enterprises and boy has that changed in my career so it's been really exciting for me to watch my own research go from this sort of sort of traditional area of and again I was more on the systems building so building systems for nontraditional types of data into something that is so important across every area it's just it's been wonderful to watch that the the need for data data systems data analysis tools data visualization tools which is my area to all of those areas to blossom so that's that's where I've come from and obviously data is huge to making advances in energy all all aspects really okay thank you so so Debra how about you okay uh so uh maybe I'm you know the furthest out here I'm a philosopher I'm a political philosopher lots of people don't really understand even what philosophers do but actually philosophers are you know we're all nascent philosophers uh and what philosophy political philosophy does is it responds to current problems and crises and confusions in the culture are things that are in contention that we're not sure of and it tries to clarify what the values and concepts are how they're related to each other why they matter as a way of trying to resolve and see whether there's a resolution for some conflicts that people are trying to deal with conflicts between very abstract concepts like equality and liberty you often hear that they can't be reconciled this is you know large literature trying to explore the space for reconciliation my own work has really been focused it's just about equality I wrote I think my first book which really looked at the reasons for thinking that equality as a generic or abstract or simply income focused uh you know a project was too narrow and that there were real differences between when and where equality matters so that equality for example in life expectancy or in access to opportunities is quite different than the fact that we're all unequal in height and so and people respond quite differently to different kinds of equality and I tried to come up with a kind of an analysis of why that might be so and what values might be driving that right now I'm involved in a five-year project that's being led by uh Angus Deaton who's a economist at Princeton uh Nobel Prize winner which is working on something called the Deaton report on inequality in the 21st century and mostly economists but I'm writing the normative chapter about where equality matters and why and what are the different arguments and how do you respond to people who think equality doesn't matter at all what really matters is poverty what are the kinds of arguments that you can give and how does this relate to some of the directions that economic research should go in and this is connected to climate although I myself don't work on issues about climate but clearly there are lots of considerations in the policy space especially when we think globally about fairness in the distribution of up costs and benefits in the context of climate change yeah so so Debra just a quick reminder that we actually did a project together I work on carbon capture and storage and there was an ethical issue um you know there are many ethical issues you know should we be doing part two in carbon capture and storage and you had two fantastic postdocs who we we worked together and that was a real treat so thank you for that um so now we'll uh John what about you what uh what excites you what is what was your academic path my background is as an economist and uh in fact I started at Stanford in the economics department and was there for 16 years before I moved to the business school and in within economics I worked I worked on it well I worked on a bunch of different things but I most of my research was in a field called industrial organization which is about research innovation r&d competition regulation of different industries and I particularly focused on technology industries and so actually by field has now become since I have less time for research has actually become incredibly interesting because people are rethinking with the rise of the large technology firms the people are really rethinking the way government should regulate technology and the the the relationship between large business and and government and and you can see the political divisions there so there's a political aspect to it as well as a an economic aspect to it I sometimes wish I had more time to be thinking and writing about that because I think it is such an interesting problem particularly since we're here in Silicon Valley it's a really pertinent problem for for us here I also worked on other one of the other areas I worked on was was a field called market design which is basically had a set up market mechanisms to accomplish different types of goals so designing auctions or matching markets or different kinds of incentive mechanisms and in fact the last the paper had an opportunity this last year to write a paper which proved to to be timely which is that I wrote a paper of a Christmas about creating incentives for vaccine development it's done some work that I had done probably 13 or 14 years ago to set up a program to accelerate the introduction of pneumococcal vaccine in in the developing world so project I did with an economist at Hartford named Michael Cramer and we wrote a paper over Christmas about a retrospective on that program how we set it up and and how it had worked and and then of course it I mean not anticipating that that vaccines would be on everyone's mind come come March and so that that proved to to be the model we had set up called an advanced market commitment proved to be something that people have been thinking about a lot about now in the context of COVID so Michael's been doing a lot of advising of different different groups including the WHO and and the European Union and the White House and so forth so it's it's nice to feel by one one degree of separation that research is is still relevant in the world yeah well anyway those are really exciting stories and for all of you you know incoming graduate students I I hope you heard this common theme that people pursue their academic interest and and you know over time the impact of these areas grow and all of a sudden you find yourself at the epicenter of of a whole new wave of innovation so it's one way to think about is is you want to be in a situation where you're skating to the puck and so what are those areas those really innovative new areas that that eventually will flourish and make a tremendous difference so now we want to open it up to questions we don't have too long but so what the way we'll do this is just please raise your hand and and we will call on you great thank you very much well firstly thank you to all the deal and Sally as well as an ex team I guess for the warm warm welcome to Stanford my question was just really around the EIPA program you mentioned that it's particularly popular these days and clearly very relevant for the audience at the moment do you have any plans to open that up to other graduate programs for instance the MSX program of the GSP I can I can take that one because it relates to a GSP cross degree program so the EIPA program is is students are able to enroll in that from a number of other programs at Stanford including the RMBA program in the law school and and so forth there I think with the plans in coming out there's now plans coming along to create a school of sustainability at Stanford that likely would be the home of the EIPA program and I think there has been some discussion at least in some of the things I've been in about what would be the natural evolution of that program could it expand it's been very successful would there be more opportunities as as Stanford has a bigger footprint in sustainability so I think that's you know there will be some discussion over the coming year about what you know can that program be expanded it's been a wonderful and successful program already and so it's and there's probably more opportunities I don't know that there's I'm not aware of any specific plan at the moment to expand it but I but it's certainly in the air as as something that people will want to think about with the new school it'd be fantastic if we could clearly but that's very exciting that there's a new school of sustainability as well it's a fantastic development I've been following through the question you know we're in the middle of Silicon Valley so so not only is the campus at stealth a you know incredible opportunity to be there but you know how can students take advantage of the place that they're in now I know it's a little bit hard right now with COVID-19 but you know if we roll that back and we we imagine that you know this is past how how can that enrich the graduate student experience and that I'll open up to all of you and whoever wants to go first just go first yeah I'll just jump in and say you know even in COVID times we have very close connections with Silicon Valley with companies a lot of companies some of them sponsor our research a lot of them are have our graduates working there or leading the companies the people at companies are very interested in giving seminars and meeting students hiring them for internships so there is a very close collaboration and in some ways some of our seminars and people's interest in speaking at them and the participation in them has increased quite a bit so one you know one positive effect of COVID is everybody's at home they're not traveling and so we've seen increased participation so there's going to be a lot of events that cross your email a lot of them and I just say pay attention to those events pick ones that interest you go to them sometimes you know if there's questions or opportunities to meet speakers just take advantage of everything that that we offer because even in this current mode there's they're going to be a lot of opportunities I would say to make connections here from meet people in in Silicon Valley well I'll just highlight one of the programs the Precourt Institute offers it's the Tomcat Center organizes the opportunity to work in energy startups in Silicon Valley and there's summertime appointments and they're really really interesting you are you know typically in a very tiny company that's in this very early stage of trying to get off the ground and they're very exciting because you kind of do everything you're you know incredibly important part of the team so I encourage you to take a look at that we also have a fantastic internship it's now going to be called the Schultz energy fellows program where we place people with local government so we place people in Palo Alto Sacramento San Francisco and you work directly with the leaders of these government organizations and really understand what what public service is in terms of a career so do we have any more more questions oh is it okay if I ask a question of course um this is uh really I think for Dean Sats but anyone's welcome to um to opine I just wondered if you had any thoughts on sort of where we are with our current political moment and partisanship and kind of how that all relates to to climate and energy as a political philosopher ask me after in November we'll see fair enough I you know I think it's a challenging moment um and uh I will say that uh one of the things hns is doing that I'm very excited about and you should all um but it's I think now posted on the hns web page and we'll be pushing it out to the community uh under uh the leadership of our new senior associate Dean and the social sciences Rana Bramitsky we're running a um fall uh weekly series on problems in democracy that'll look at everything from healthcare to climate uh to racial injustice to inequality and it will showcase faculty in the university I think we have 39 faculty lined up from the law school the business school the engineering school hns it's really fantastic and uh part of what we're trying to do is you know again evidence-based analysis trying to also understand why in the light of evidence sometimes people disagree what extent those disagreements are about uncertainty about the evidence to what extent it's about different values and uh ranking different values differently so people will make different trade-offs to what extent it's involved in uncertainty so it'll be offered as a one-unit course for undergrads but it will be open to everybody with a Stanford ID and um I think this will be you know a great moment to sample lots of phenomenal research that's going on in the university around everything from voting to political polarization to racial justice um and uh and hopefully spark a lot of really good conversations okay we'll take one last question this will be from Hisu and then we'll wrap up so Hisu why don't you ask your question uh yeah so I'm a I'm an incoming physics phd student so no human science but I have a lot of interest in like political theory like um and like history and those kind of kind like things that are very much unrelated to what I will be researching which is material science um and so what are some of the ways that like uh students who are doing research in a very unrelated field kind of get involved with like what's happening in the other kind of side and also like learn more about more active research in that area um so there's some great classes offered in the philosophy and political science department around uh you know everything from why democracies succeed and fail to what are the ethical values at stake in democracy versus hierarchy uh uh how do we think about uh you know uh you know debates about justice so I would look at and there are a lot of grad offerings as well as the kind of mixed undergrad grad courses we do there's also a lot coming out of again some of the um long-range planning so uh there is an ethics and technology and science initiative uh there's something we're calling a hub that's brings together people from the medical school and people from the sciences from computer science from political philosophy who are interested in ethical and political questions uh the co-leaders of that initiative are Margaret Levy who's a political scientist and runs our Center for Behavioral for the Advanced Study of the Behavioral Sciences up on the Hill and uh Rob Breach who is a political philosopher and they're doing some really wonderful things and the aim of this is actually he sort of bridge the gap between our incredible work that goes on in the campus in technology and science with some incredible work going on in ethics and political philosophy and to find ways of bringing that into dialogue um there's a an undergraduate course talk uh co-talk uh probably uh a very popular successful course on ethics politics and computer science uh that try is taught co-talk by an ethicist a policy person and a computer scientist trying to help students move back and forth and understand both all of those different languages um and be able to think you know we shouldn't get into thinking i'm a technologist so i don't need to think about these other questions of ethics uh there are lots of points of uh ethical decision making that is involved in the conduct of scientific research and wherever you come down on those issues you want to be you know able to be reflective on it and have the opportunity to get feedback so it sounds like there are lots of opportunities anyway thank you to to one of the three of you for joining us this was an incredibly great and exciting panel uh so and thank you for sharing your own personal personal stories as well and with this i think we will wrap up this session and uh thank you so much


Hello, my name is Rob Hirschfeld. Josh McKinney and I are the co-chairs of the Deaf Core Committee for the board where we have been on a long road to define what OpenStack Core is. We are probably halfway down the journey. And today's talk is going to be about what we're doing, how we're doing it more importantly and most essentially how people here can get involved, which is really what we want to have happen and how we're defining core.  So if you have any questions about anything, including like Docker and what it is, that's not the right question. Everything else. Everything else. And if you get a fire small arm when we say the word Docker, tell us so we can duck. Go ahead. And so, but if you do have a question, please raise your hand. I think Josh and I both prefer an interactive style and if we're going to answer the question in the next slide we'll tell you. So this is the boilerplate for Deaf Core. We've been talking about Deaf Core for years, a lot of time. And so we're not going to present you with the history in this presentation. We're going to try and do just enough information for you to understand what it is and figure out where we're going from there. So it's important to understand this. Deaf Core sets the requirements for all OpenStack products by defining designated sections of code, must pass tests and capability, and that these definitions will use community resources, involvement to drive interoperability by creating minimum standards for products labeled OpenStack. Sorry, this is the only slide I'm going to read. I hate reading slides because this is really important. I was going to go through and highlight the words that I thought were important on this slide and it's just everything. Every single word. This is Deaf Core distilled to 37 words. So the really important caveat here, and we're on the board, the board controls commercial use of the mark. The other uses of the OpenStack trademark are protected in the bylaws. So the community is always allowed to describe themselves as OpenStack and the code itself, the integrated release can always be described as OpenStack too. So what Deaf Core covers is only using the word OpenStack in your product, and everyone who does that has a license from the foundation, and that license requires them to do some things in this particular case, Ship Core. And product, we get, people get hung up on this, product includes distros, product includes cloud services, product includes anything that you are doing to make money off of OpenStack. Appliances. Appliances, ecosystem, products, right? There's actually marks for that. Those are controlled. This is how to patch it to the software, anybody can use it. This is how the OpenStack brand means something. If we don't control it, actually this is where the side note. If the board does not control the use of the mark, we actually can lose control of the mark. So it's very important for the success of the foundation that there be controls on the mark. That's why we're doing it. All right. So this is how we've organized ourselves, this is the overview of what we're going to cover. There was a lot of debate as to whether Def Core needed to be split up into many subcommittees and working groups, and we didn't do that. Actually we did. Two co-chairs.  We've been moving as slowly as possible so that Def Core will become boring over time, and then it becomes easier to, it's actually a very tedious process. The stipulation of any board activity that it has to be boring. It has to be boring. So we define a bunch of principles ahead of time, we'll cover the principles. We use those principles to create criteria in terms of how do we select what capabilities will be part of Core. We boiled all of the tempest test of which there are hundreds and hundreds down into capabilities, which are groups of tests, and then we are producing a scoring matrix for every release, applies the criteria to the capabilities and sizes, which ones are in Core. I think we need like a flowchart for that. So here's the idea, and I really like to stress this. This is not the way you're used to thinking of Core in the past. So if you're used to OpenStack in the bylaws, it actually says Core is Project 1, 2, 3, 4, and 5, and that's not what we're talking about. After all this process we figured out that's not the way we think it's best to define Core. What we're doing is we're defining Core by the tests, the features, the capabilities tests are an expression of that in the OpenStack that are defined by the community. So the community defines tests, those tests reflect features and capabilities of the product. That's what we use, and that means that a project like Nova isn't 100% Core, only the capabilities that are must-passed tests are Core capabilities, and that allows Nova to continue innovating and changing things and adding new APIs and stuff like that. It doesn't lock all of Nova into being a fixed thing that can't change. There's a really simple example of this, and it's one of the problems we set out to solve. Nova itself has always been in the core definition, but none of the extensions were. And yet almost everything that people do with Nova requires extensions. So having key pairs, key pair auth was an extension. Floating APIs was an extension. Most of what people did with Nova from a technical standpoint wasn't required to be in anyone's product. And when it wasn't in there, OpenStack didn't work the way people expected it to. And the goal is to make OpenStack useful and interoperability is the ultimate destination here. So we have to use the things that people find most useful and define those as what's core for the product. Seems pretty straightforward, it's really been exciting to do it. The challenge is, we don't have a slide for dead puppies, I meant to put a dead puppy slide in here. That means that there will be features and capabilities that people write that is in core, and that's okay. It doesn't mean they're not important, it just means that we're not controlling them from interoperability perspective. We'll talk about that. These are the ten principles. We're not gonna read them. It took six months. To put these together and then get a board vote on them. It ended up being a unanimous board vote, which was amazing. Rob actually traveled around the country to meet with OpenStack user groups to talk about what we were trying to do and how we were gonna do it. The most important piece of this that I will emphasize is that Defcore ended up being two things. One is the tests and the other is the must pass sections of code.  GPL means you have to use all of it. But we're using the trademark license to require the community to collaborate in certain cases. And what code is designated is controlled by the technical committee. But as a simple example, the Nova API is a designated section. If you're gonna use Nova, you've gotta ship that piece. The drivers are not. So if you have a different crazy hypervisor, whatever you wanna use, maybe it controls robots, awesome. You can use that code, you can include it in your product, no problem. And you don't have to include driver code for hypervisors, you don't ship. And we have some slides about that specifically. Sorry. The one thing I would, before we jump to the next slide though. The reason why I traveled so much to talk about these and I continue to really work with users and I love to get feedback about this is we are being very careful that what we do in defining core drives the culture of OpenStack. That drives positive behaviors that we wanna see. We intentionally chose tests because after talking to a lot of people, we found that encouraging the use of tests to determine core creates a culture of creating tests. And it creates commercial incentives to improve the testability of our product. We spend a lot of time thinking through the implications, that's why we go slow. So that when we roll out core, everything we do creates positive virtuous feedback cycles to make OpenStack stronger. Very deliberately chosen. If you have questions and wanna talk about it, it's one of the things I spend a lot of time actually thinking through and tuning and you'll see that as we go through the capabilities. This is what Josh was talking about with designated code and tests. We put this together to make it very simple, try to make it very simple. So there's two things in the matrix. There's designated code, which is code that's defined by the technical community is must be included. And the TC has a set of principles, draft principles, I don't know if they've finalized it. I think they ratified them. They did it? Awesome. Principles that define how you know what is designated code and what's not. So when you're writing, working on a project, it should be very clear reading those principles. If the code you're working on is designated or not, it's things like, did you mean it to be replaceable? Does it have multiple implementations? Things like that. And then the must pass tests, right? So we have, we pick these tests. And if you have designated code and must pass tests and you're using, you've implemented, remember, commercial use. So your product uses the designated code and passes the test. You can get a license. You can call it OpenStack. If you pass the test but don't use the designated code, there is a future plan to maybe have a OpenStack compatible mark. It doesn't exist today. So today, in order to use the mark, you've got to use the code. And we'll probably keep it that way for a while. There are folks like Sinefo that have done total re-implementations of OpenStack and said, hey, can we call this OpenStack? Not really, no. Maybe in the future. Maybe in the future. If you are using the code and it doesn't pass tests, you are misconfigured. And if you are in neither camp, get ahead. Why are you even having this conversation? You took off my more emphatic no. Yeah, well, there was a much more, yeah. OK, no, no, no, click. We'll just fix it here. OK, we're not going to read these ones either. Let's just say that this is the main work product of the DEFCOR committee. We had a lot of long meetings to come up with. How are we going to score every capability? And the goal here is to be as objective as possible. It's tough to get in a room with a bunch of arguably vendors and say, how are we going to decide which capabilities everybody should have in OpenStack and which ones don't matter? And we wanted it to be data-driven. We didn't want it to be emotional. Also bear in mind, this is criteria and scoring that the board uses to make a final decision. It's not that the score is an absolute measure of it's in or out. The board can use their best judgment if they have to. But so far, we haven't had to, which has been a good sign for the criteria. And what was interesting in coming up with these is we came up with the criteria. We actually started with like 17, and we've boiled them down a little bit after hours of discussions. We didn't plan them to be in these four categories, four top categories. They actually naturally fell out into this. And if you go back to the original, even before we did the principles, these four categories reinforced the cultural emphasis that we thought core should have. So it's really nice to see, you do a lot of discussion, you talk to people, and after time you come up with things that actually reinforce what your principles and goals are, even after doing it from an implementation. I do sort of miss criteria 13 and 14, which we dropped. 13 was about admin APIs. And we had to decide that things that use admin APIs can't be core because you can test them on private cloud software, but you can't reliably test them on public clouds. And number 14 was if it weighs more than a duck, it's a witch, which was called the duck criteria. That one, we couldn't figure out how to score anything on, so we had to take it out. And the ducks were not floating, that's the problem. Moving on. It is worth noting that we, these are release by release criteria. So the purpose here is that we're sharing these with you. If you look at these and say, oh, you missed something obvious, please, the criteria will be adjusted, release by release, people often get caught up. We're not trying to go and make Juno decisions today or things like that. It's still time, it's always time, but right now is a really good time for you to look at what we've done, give us feedback. That's actually our call to action at the end. Okay, we stated this before, but we took all the Tempest tests and we grouped them into capability, actually Troy Toman did all the work. There are almost 800 tests as of Havana. We can't score 800 tests, we would go insane. So we rolled them up into capabilities, we ended up with 75 capabilities that we could score. And this is an example, one block snapshots means these two tests pass. Some of the capabilities have dozens of tests, some of them have only one. And our expectation is the technical community will own the capability grouping, not the board. This is now stored in a JSON file in the refstack project, so if you want to argue about how they're rolled up as some of the PTLs do, awesome, we can do that in a review queue. Oops, hey, this is rebooted, I might be back. All right, this picture is a cheat sheet that I put together, because we haven't, this is the first time we did the scoring matrix, so it's a little bit not user friendly. Our UX experts haven't weighed in yet. So you can actually go Google Defcore cheat sheet and find this matrix, and that'll show you, sort of take you through it. I don't want, we don't want to spend a lot of time in here, but capabilities boiled down into test groups, test groups were scored on these 12 criteria. Then this, we actually did a histogram of the scoring, that broke into three pretty strong group categories of, these were pretty clearly the must pass category, these were the maybe category, and these were the now we don't think so category, without a lot of shades of gray, in the nice W shaped histogram, like statistics. So please review this, we don't have time here. Okay, this is the call to action part, and we're gonna get into teacup and ref stack next, but just the, it's that entire three of the 12 criteria rely on community feedback, and so it's, are these capabilities widely deployed? In other words, do public clouds support these APIs? If they don't, there's no point in us saying they're in core because people can't expect them to work. Are they used by tools, meaning are there, pass tools or dashboards or orchestration tools that consume those APIs, and are they available in client libraries? Fog, Bodo, JCloud, et cetera. So if you have an opinion, if you write one of those client libraries, or you work on one of those tools, and you say, hey, we really need this API, bring that to our attention. It'll bump that score up in the next pass. Yeah, and this is from, I think I'll speak for Josh too. That criteria is the criteria, right? We have a balanced criteria, I think we're actually gonna be able to keep them all 12 off and balanced, but it is really important to us, and I can speak for other members of the board because interop is what we're trying for. This is how we know that we're actually passing for interop, and so everybody needs to help us collect this data because we want to be able to stand up in front of the technical community, in front of the user community, in front of press and analysts, and say, this is what's getting used, this is what's important, and then come back and say, and don't change it, or if you change it, it's gotta have these really good reasons to change it because it's gonna start breaking people. Until you've given us the data and helped us collect it, it's much harder to have that conversation, and then you get into a design meeting where they have incredibly strong technical reasons for changing an API. Always do. No idea of the actual cost in the community, and we wanna give them that cost so that we can make better decisions about it. All right, we should go quickly through the examples so we have lots of time for questions, but one of the things that we did- Let's pause for a second for questions because it's a good breaking point. Anything confusing about that? Okay. We have more pretty pictures, yeah? Oh, there we go. So I go through the testing with one configuration, so I get the branding, or do I have to tweak it, or do I have to pass it down for all the different configurations I could deploy? It's a really good question. I think it has been debated hotly. The current thinking is, if you're selling a software product and you have a reference architecture that represents your typical configuration, you can certify against that and say, yeah, I get the branding because my reference architecture configuration will pass tests. Right. We actually do look for tests that don't have a specific, that would fail in different configurations and those get scored down so there's some capabilities stuff for that. It's a great question. It's something to watch for. Yeah. There's still open debate at the board as to whether we need a broader family of marks over time and you end up open stack certified for private cloud deployments for the medical industry or something. I don't know. Until we have the data, we don't know what extra marks are, so we like to talk about kicking the can down the road. We called this cycle, we named the cycles for the def core. This was the elephant cycle. We're trying to eat the elephant. There's a couple of analog ways we use the elephant analogy, but we don't try and, so we intentionally don't solve problems that we don't have enough information for. You named these, not me. So we thought it would be really helpful to come up with three specific examples so that people could get a feel for the impact of what it would look like and then this ends up being pretty specific. You want to do banana cloud, now I'll do this rocket. Simple example, we had to go to the board and say if everyone today had to match def core criteria, who would be impacted? So let's, we have an imaginary service provider runs banana cloud. They use Nova, they ship the designated sections, the tests all pass, that's green. They implemented the Keystone APIs, but they don't actually ship Keystone, which most service providers don't, but because Keystone doesn't have any designated sections, that's also fine. They also ship Trove. Trove isn't in core, they get karma for doing it, awesome, doesn't interfere with their use of their mark, but it doesn't matter if other people don't use Trove. And again, they use Horizon, Horizon's not in core. Actually, yeah, I modified this incorrectly, I'm sorry. They don't use Horizon, was they? No, it was Horizon. Also doesn't matter because Horizon isn't in core and it turns out that's deficiency in tests. There aren't really any tempest tests for Horizon. No tests? It can't be in core if there aren't any tests. And so all of the technical, I've actually had now conversations with some of the PTLs and they're like, thank you, we now have a stick to beat my contributors up to get tests in. And now commercially it is important if you're paying a developer to work on a project and they are not contributing tests along with their code, there is no avenue to get into core. It's not necessarily the only important thing, but we want to reinforce that culture. I'm always gonna go back to the culture card. Mm-hmm. Oops, you and I are fighting. You use the code. I have the remote, go ahead. So Sprocket is a private cloud, right? So somebody who's using the OpenStack code, but chose not to implement, I think key pairs was the example we have. We're gonna post this as a blog post after we get it a little bit edited and make sure some of the statements in there like these don't freak people out. But Sprocket ships as much of OpenStack as they possibly can with, they missed one thing that they needed, they have to add that, so that becomes a thing. They don't implement Swift in our example, they use Ceph, which is pretty common in private cloud implementations. We are currently discussing whether or not Swift has designated code or not. And so once that's resolved, we'll be able to answer this question. If Swift has no designated code, but say Ceph can pass the Swift API tests, rock on, you're good. If it's 100% designated, then you need to ship Swift. And if it's 10%, then we have to figure out what percent that is and Swift could use that. I think my expectation is OpenStack projects should have a degree of designated code and we'll eventually have a target percentage. And then they use Heat, great. There are no tests for it, but there will be some designated code in Heat. Hyper-V, Subdriver, no designated code, and doesn't have tests specifically. I think actually it does have tests. So they're all in good shape for there. And this is one of the things. Shipping products that have OpenStack projects at our core, we love that. That's great. It doesn't make it bad. We're just all about use of the brand and mist. So this actually isn't impacted by Def Core at all, but we're deliberately trying to engage with the client library community because if they need APIs, we can't deprecate those. And if they're consuming APIs that are outside of Core, it's a good indicator that maybe they belong in Core, right? So this fictitious API client called Mist, they should tell us what they're doing. And if they add tests to Tempest for capabilities they're using, that's very helpful because then we can consider those in the future. I think I'm gonna rename this one Meow because I like the graph. Yay, end the tools. So this is a good place to pause again in case there's a question about those things. Does that help? This is the first, the major feedback we got when we did the first presentations of this was, so what does it mean? Does this help people understand a little better what the impact is? I'm seeing there's a question back there. Okay. So I'm gonna repeat your question rephrasing a little. So the idea is can people fake it out and cheat? So here's, I love that question because we get asked that a fair bit. And the answer is yes until your customer runs the test suite that you said you passed on that cloud and fails it. And so one of the things, and this is actually a great setup for this, one of the things that we're doing in making these tools generally available, and T-Cups is a big part of this, making it incredibly accessible for people who aren't OpenStack developers to use this information and share this information is that if a vendor is saying one thing and publishing it to get the mark and then their customers try and repeat that experience and don't have the same experience, then that's gonna come out very, very quickly. If they truly did it in a way that hid everything and they created this black box and stuck in code that they shouldn't have stuck in, we're not trying to be an enforcement agency. If they can pass all the tests for interoperability and somehow make it work behind the scenes. The other thing is there are not an infinite number of vendors using this process. So again, this is a process that applies to licensees.  So there's only 300-ish companies that are in that bucket and not even all of them are licensees today. So we have a small ecosystem with a lot of peer pressure. We're not super concerned about policing it because the reality of getting caught out is pretty extreme. There's 17,000 members of the foundation. It'll be pretty public if you were cheating. All right. So we feel if once again kicking the can down the road, if that becomes an issue, then we'll have to take action against it. But we hope that what we've got to show you with the tools and tests make it very hard for somebody to sort of say one thing and then ship something different. All right. Rev stack. Okay, so we bought this domain name. We built a community portal. The idea being when vendors certify by running Rev stack the code, they end up with a scorecard that lives on Rev stack the website. And this is so that when folks are thinking about using OpenStack, either software or services, they can go to one website and say, okay, which capabilities are actually gonna be available broadly? So if they're thinking about hybrid or they're thinking about avoiding lock-in, they can say, okay, if I consume these APIs, I know that I can go and buy OpenStack from other different people and I can get it from this public cloud or that public cloud and what I'm using will work. I'm not getting locked into using an API that is only supported by a small ecosystem of vendors. Conversely, the vendors can also then highlight that they are beyond the bleeding edge, that they're already supporting Trove, that they're already shipping heat, that they're whatever those other sets of capabilities are, they can highlight that in their scorecard. And they might be able to use that to say, hey, if you're really excited about what's coming in OpenStack, we're already shipping that for you. We also wanna be able to show how people's compliance has improved over time and particularly for, again, the APIs and tools community to consume that. And I wanna reinforce that we're not trying to post just the must-pass tests, right? That's 15% of the base. The goal is to show every test, right? Or at least every positive test, because we suspect people are gonna prefer to post only their passing tests, probably. But the idea here is that that will allow us to see up-and-coming capabilities that are widely deployed in Interesting OpenStack. So if a lot of people are interested in Trove and we start seeing the Trove test passing, that's data that indicates that it's on track to get into core and then the market can actually start to implement that more often in their clouds and then it becomes an interoperability candidate, which is really what we're driving. If we get this data, we make better decisions about what to highlight, what to include. That's the whole point. One thing we have skipped over earlier on is talking about the timeline. And I think it's because we've given this talk so many times, we forget pieces. What we're doing right now is living in the past. So the entire matrix that we just scored is for Havana. And obviously, Icehouse just came out, right? We are hoping to have the Icehouse matrix done within 90 days and then by Juneau have the scoring matrix for core done at the same time as the release. And at about the same time of Juneau to be enforcing those requirements against vendors. So right now this is advisory to the existing vendors to say, hey guys, it looks like your product or service doesn't meet the definition of core. Six months from now, we're gonna be having a tough conversation about your license and use of the mark, so you probably wanna do whatever remediation is required. And I'm hoping that in Paris when we certify Juneau we will also retroactively certify or at the same time certify Icehouse and as an advisory and say, look, you can actually go and test your products right now and see if the Icehouse ones are in compliance. So the other piece of that sort of preliminary period is that we're not publicly posting the scorecards on RefStack. So the tools available, the vendors can use it, they can generate their own scorecards and use it for understanding where they're at. But we're not gonna expose everybody's dirty laundry when they haven't had a chance to work on it.  It is really important, really helpful for people to come in and say, yes, I think this makes sense. Because the vendors, when the community says these things make sense, I understand them, the vendors then start to listen. Cause you're ultimately the influencers the vendors care about. And so we'll go back and call to action why that's important but read them, respond to them. And if you don't like them, that's okay. We wanna hear what's wrong, we're constantly adjusting this process constantly. And so we're very receptive to feedback, positive and negative. In some ways the negative helps us the most. So do not think, I'm not interested. They don't wanna hear it, we do. We've made a lot of changes over time. All right. You wanna take this one? I'll take this one. So RefStack and I've been working really closely with the code side and actually Piston's been has some people forces here. David should be here too. Yahweh, IBM, Yahweh, IBM, some people from Dell and I've had a whole bunch of people come up in the summit to get more involved have been helping with the code side of RefStack. So this drives the public site. But what we found is a lot of companies have been building Tempest results UIs. They run Tempest as part of their normal process both for building deployments or for actually testing like building their own gate. And they wanna be able to take those results and start comparing them and see if they're improving over time as they tune their deployment engines. And they were repeating the work. So that work is synergistic with RefStack from the UI perspective, the public UI. And so we have a lot of people who are investing in it because they wanna use it internally. It's really helpful to have the same thing that you're gonna get scored on publicly and privately and that's perfectly good. We also have folks using it to validate deployments, right? When they're out doing a customer deployment of OpenStack and they just run, it's a real quick way to look at all passes green. This is the configuration matches what you were expecting. That's what the Dell team's actually been doing that for several releases. It's our final site validation test. And there is this caveat because we keep talking about Docker and T-cup. There's no Docker in RefStack. Just everyone's clear. We did not introduce a Docker dependency to a chunk of OpenStack which I know a number of people were nervous about. So we're gonna talk about Docker and T-cup next. And the enthusiasm for Docker, we did not do that. But T-cup does use Docker. So T-cup is a very small, small sliver of what we're doing, but it's an important one because it's the community facing piece. So our goal, so RefStack is set up to create community feedback. Run the Tempest test against your product where we're talking about cheaters, catching them. To do that, we need to have people who never don't know how to get cloned, don't know how to do a PIP install. They have never set up DevStack and are never going to. We need them to be able to use this. And more importantly, we need our community members not to spend hours on the phone helping them figure out why they have a Python dependency or issue that they couldn't resolve. So what we did was we took the RefStack project and we put it in a container because that's highly portable, very repeatable, and it's a much smaller surface to support. And T-cup, so T-cup is Tempest in a container, and then I added Upload and Probe because I love the phrase. It's an acronym. Yeah, it's an acronym, yes. It's a acronym. It's an acronym. Yes, it is a acronym. Speed of cheating. Yes. And so that's the idea with T-cup. And so the way it looks, what it does is the flow, you basically put your credentials in, test your cloud, after a test is completed, it uploads the results to the RefStack database. Pretty straightforward. You could point it to another database if you want. That's the flow. What it looks like is this. You get the Python file, minimal dependencies by design.  You run the Python file. It takes a while. It runs the entire Tempest test suite. It builds the container, it runs the test suite, it uploads the results. And there's a manual mode so that if we have to help people, they can do it, or if they just want to run the tests manually and see the results, they can do that too. But it's mostly selfish for us because we don't want to have to play IT support for everybody who's trying to figure out their scientific Linux variant that I haven't used yet. Okay. Is there any questions on that, by the way, before we skip straightforward? Yeah, there's one over there. So this isn't true at all. So, and this is where people get really confused. Okay. So the question was, does this mean we can't test anything that has specific hardware drivers? So just be clear. The container does not have to be in your cloud, right? Tempest doesn't actually have to be anywhere near your cloud. What Tempest is running is API calls against a cloud somewhere else.  And so I just run teak up on my laptop, but it's pointed at a cloud somewhere else. That cloud can have hardware drivers. It could have a magic nova driver that uses robot arms, which I still think is a cool idea. You can point it at a public cloud. And that's deliberately what we want to do is have community validation that public clouds are providing the APIs that they claim they are in different availability zones and regions, whatever else. So there's no limitation on the kind of open stack. What we did have to limit is that probably your admin API tests are going to fail unless you have admin credentials. And so we've said we're not gonna have any admin APIs as part of core. And the reason is, I don't have admin credentials on the HP cloud. I would like to have admin credentials on the, I think that'd be really fun, but they won't give them to me. And since this is true for most community members, they don't have admin credentials necessarily on any clouds. We can't get the level hybrid experience for interop on that side. And that reminds one things I keep taking for granted. Our goal is to be able to test private clouds too. Ref Stack originally, if you go back in time, was about probing service providers' clouds where there was access. And T-Cup came out of saying that there's more private clouds that we want to test. If we want interoperability, we need to help a company who has bought a cloud and installed it, get that, get that, test it, get it running. Because once again, we want, I'll call Piston as an example, if a customer buys a piston cloud, that customer without any help from Piston should be able to test that cloud, right? The CIO should be able to sit down in their desk, crank up their, open their Linux desktop, and run T-Cup without ever having installed anything else. And we felt like that was a really important thing because that's where we're gonna drive interoperability. And a lot of companies, we talk to, have multiple private clouds that can actually test interoperability between their own clouds, between different versions of their clouds, ends up being a really useful thing for us to collect the data about those sites and systems much more broadly than just probing external interfaces to service providers. I mean, our original thesis was if you were a software vendor, you would have to stand up a copy of your product, literally a public cloud of your private cloud software in order for RefStack to test it. And that didn't, nobody was willing to do that. Some of us couldn't. Yes, some of us, some of us are not allowed. You're sad. Any other, did you have another question? Yeah. This question came up yesterday. So the question is, Tempest is non-deterministic. What then? So we're not, sorry, we're not part of the QA team, although there's a debate to say should RefStack actually join OpenStack QA or be put into the OpenStack Info program. If we were part of the QA team, we would have to figure out how to fix this. Since we are a customer of the QA team, I would really like them to figure out how to fix this. That's a totally unfair answer. We had lunch with them yesterday. We're collaborating, we're collaborating with them, and there's an element of, hey, you guys are encouraging tests and test use. Oh my God, you're encouraging tests and test use. So, this is the whole thing, right? I'm a huge fan of test-driven development and more tests and automated tests, so we're really creating a lot of pressure in the community by design to encourage the test coverage, the test stability of OpenStack. Ultimately a good thing, short term, we're gonna expose gaps. I mean, and the other thing is if a test can't pass reliably, probably that capability can't be core because it also means the capability isn't reliable, or at least not, or the test is not exercising the capability adequately, and we should look at better tests. We should make that a requirement. We did. It's probably expressing one of the criteria. No criteria, we didn't. No, and test reliability is totally a requirement. So, the resolution process. Let's repeat the question. What's the resolution process for criterion designate a test? Disputes. So, right now it's a deaf core committee. I think it will probably always be the deaf core committee, although we will cycle through other people because if Rob and I spend this much time in this many meetings, people never do anything else. Sometimes we're gonna sunset that. The deaf core committee has been expanded to include some members from the TC. So, we have TC liaisons, which has been really helpful. Let me be much simpler. We hadn't worked that out yet, but I think it's entirely a great question and what we will do is we will set up at least two public meetings to review the final list and send out information about it to the community list so that people can participate in a community review of the list. So, thank you, yes. We try to engage the community as much as possible and that's the exactly type of feedback that we want. We've done this before when we went through the principles to get community feedback. There will be two meetings, one favorable to Asia time zones, so evening in the US and one in the morning that's favorable to European time zones. We've done this as normal for us, but you're right, we haven't scheduled it yet. Well, then actually it's really good to highlight that we've done a lot of outreach with the board and we've done a lot of outreach with the gold members. We've not actually been talking to the rest of the corporate sponsors and licensees, although the foundation staff I think has been reaching out to them to keep them in the loop. All right, we need a lot of help. Oh, sorry, another question there. Last question. Versioning is fun. So we spent a lot of time this week with the Tempest team. There's a blueprint for Branchless Tempest which is designed to be able to test backwards across other releases with the current branch of Tempest. It's not done yet. Right now, RefStack has a separate set of capabilities, JSON files for each release. So there's a Havana one, there's et cetera. And we use the Tempest branch as of that release. So you're really only certifying against the test weed as it existed when that release was cut. There's a lot of work still going on inside RefStack to figure out how do we deal with stable? What about when we introduce new tests in stable that prove the capabilities are actually broken? What do we do there? From a license standpoint, the belief now is if you certified as of that release, you're fine. And if new tests get included later on into stable that prove that those capabilities don't work, you did pass at that time, it's fair game. But it is a, the Defcore release capability set is a per release statement. And vendors will be expected to certify their release each time as they come out with a new product on that release, they'll be expected to make a statement about that release. So that's gone back and forth. I'm happy to take the question, but we'll answer it in a second. We are technically out of time. So we're gonna wrap, we need to wrap up. There's a lot of discussions about changing the mark. So the board just, sorry, the foundation was just presenting to the board and has some more, has some more work to do to present to us about simplifying the mark and creating less marks, right, compatible and powered. The license requires vendors to declare what version they're shipping, including service providers as well as, but it's not incorporated in the mark. It's an additional requirement. So the name mark versus the brand mark versus the versioning pieces are still in the license, but they're not all incorporated in a very simple way yet. We've gotten the feedback pretty clearly. Fewer marks is better. So it's gonna be an uphill battle to add any additional marks. There's training, compatible and powered, right? That's the new, the new coming trend is along those lines. Actually I wanna flip back to the, we'll post this, but please, please help us make this process work, bring up concerns, bring up issues. If nothing else, just say, hey, this made sense to me. We'd love some plus ones. That's gonna help roll this process faster because if we only get silence, we go slower. If you want us to go faster, and most people we hear do, we need the positive, we need the conversation, the dialogue, that actually accelerates this process more than leaving the room and just saying, oh, it's nice, I like it, but assuming that everything's on track. Speed comes from community consensus. We're out. We're out of time. Thanks. Thank you all very much.