I'm a cooperative extension specialist in ecological restoration at the University of Arizona and I promise you purposely made this like super opaque like what am I going to talk about tonight, but it's going to be awesome. And so before I start talking about what I'm going to talk about I want to tell you a little story so as a cooperative extension specialist probably or hopefully most of you are aware that cooperative extension has to work with stakeholders to help address their management goals. Okay, so I work with a lot of stakeholders in the state of Arizona, farmers, ranchers, backyard gardeners, BLM government, all kinds of folks to help them solve their land management concerns like that. So one of the things that I heard a lot when I was going around talking to people was that there was no sort of single source, single resource that folks could go to if they questions about restoration there's lots and lots of stuff out there right but there's no one place that people could go that Arizona.edu right down and it's this one stop shopping online in place for all the resources you might need if you want to do restoration in there. Okay. And so there's all kinds of information about kinds of cool stuff and people love and then folks started going to the site and they're like you know what we need, we need local resources we'd love if there was a map, and we could click County on the map and find local resources of seed purveyors and all the resource map, it does exactly what I just said so you can click on one of the counties and you get local resources. People saw this and they were like that what dude a is doing down the road in terms of land management it would be really awesome if there was a map that had descriptions of local restoration projects, so that I can go and check and see what people did and learn from other people. And I was like that is a great idea. So I created this really nice form for folks that you know ask them information about their restoration project. And then I sent it out to, I don't know, something like 500 people that I interact with fairly often. Okay, and I waited, and I waited, and these forms weren't coming back what was coming back to me were emails that people are like this is awesome. This is awesome. This I love this effort but you know I'm not comfortable talking about my recent restoration efforts because this thing failed, or oh this is awesome I can't wait to see this but I have nothing to contribute because the last couple things didn't exactly hit the mark we wanted. And over and over again, I kept hearing from people who are like, This is something we need I want to know what other people are doing but I don't want to share my work, because stuff didn't come out perfectly. And what I thought was really weird about this whole thing is that stuff often doesn't come out perfectly right particularly not in restoration and revegetation right restoration reclamation revegetation is really really hard to do. So rarely do we meet all of our benchmarks right, we might be successful in some areas, but not in others. And in addition to that we are human, and to air is human to forgive divine by Alexander Pope like we make mistakes, right. But I think a major issue in our field is that we don't talk about our mistakes. Okay, and if we don't talk about our mistakes, we can't help, we can't learn from one another. Okay, we're going to be reinventing the wheel. And then we are not talking other people about our mistakes, and then hearing back from them and then maybe helping us solve some of our challenges. Okay. So, what I want to do today is talk all about mistakes, and I'm going to talk about my own mistakes, and Alexander Pope is going to help me and if you guys don't know who Alexander Pope was he was this sas that kind of famous but he also was really into gardening so I just like Wikipedia like wormholing and I learned more about Alexander Pope than I ever thought I would but Alexander Pope was going to help me talk all about how I messed up and hopefully at the end of this talk you don't walk away with being like wow she is terrible at her job. I hope you walk away with something like oh okay people mess up, and there are things you can learn from it and now I'm going to be a little more comfortable talking about it, because I really think there needs to be like a regime shift in our field, where we start becoming more comfortable talking about our mistakes now I understand. This is a little weird right sometimes if you're being paid to do restoration you don't want to lose clients right you don't look better if you're in research right you need to come out with positive results so you can publish them and then get your next promotion. So I get all that, but I think we need to be a little more forthcoming with what doesn't work for us so that other people can learn from our mistakes and also so that we can develop more of a sort of culture of sharing. So that we can also learn how to do things better. Okay, so what I'm going to do tonight is I'm going to tell you all about how I messed up in lots of things that I've done. I'm going to talk to you about how I've made assumptions about what works. I'm going to talk to you about how I haven't accommodated for some uncomfortable truths ecological truth, and then how I've overlooked simple for fancy which is like the worst, the worst. Okay. And first, I'm going to talk about how making assumptions about what works, and I'm not terrible at my job I swear. Okay, so I work in Arizona. Mostly I work in desert grassland systems which are super arid and there's some of the hardest to restore successfully. And there's lots and lots of reasons just like probably any place that you guys do ecological restoration. There's like a litany of things that are keeping you from being super successful with your restoration right and in arid land systems this is just a list of some of them right weeds soil types may context dependency. And rain comes soil chemistry all this stuff there's tons of things that can block us from being successful in air and systems however precipitation is by far the dominant thing. Okay, if you don't have rain plants don't live long it's like this you know common thing so precipitation is super important and this was highlighted in this recent paper by Nancy Shackleford and colleagues where she took all kinds of published studies on ecological restoration in arid land systems and look what they found as you go up in a ridity index so as you get more water, restoration success increases. Oh my God. And so this wasn't too surprising but it just shows something that we kind of all know right, but something else that this study also showed was that lots of the things that we make decisions about who to put down how much to put down when to put down where to put those things are really really important for driving success also not particularly surprising right. So this is showing for example as you increase seed rate, you get more plants, who to know. As time goes by you have less plants things die and the bigger the seed mass the more plants are going to have okay so this stuff is pretty well known but this is highlighting that after precipitation for the most part decisions or or the the decisions you're making the things you're doing on the ground are really important, or are they. So I did a study and when I say I almost everywhere here it's going to be someone else so it's going to be mostly my students in my postdoc so anything you really like probably someone else did anything that you're like this is garbage I probably did but Hannah Farrell was a PhD student of mine and she was working on a natural gas pipeline in the altar valley in Arizona which is ultra valleys about 45 minutes southwest of Tucson. And this natural gas pipeline got installed and what they did is they first scoured the top five inches of topsoil. So all that delicious topsoil that's got the organic material in the seeds and all that stuff they scoured it all and they put it to the site. And then they dug the trench they put in the pipeline. They covered the trench and then they put the topsoil back. Okay, so Hannah. Oh, and this is what it looked like it looked like a moonscape. And this was in an area that was graced. Okay, so Hannah was interested in understanding what restoration approaches might be important for successful re vegetation of this natural gas pipeline. So she had a couple of different treatments she had this topsoil treatment so remember the company that installed this they took the five inches of topsoil. They scoured it all that's exactly how they did it. I don't know they use the machines they scoured it all they put it to the side and after they installed the pipeline, they put it back. Okay, so she had just a topsoil treatment topsoil and seed. So we asked the local ranchers about what kind of native forage they liked and we created a forage mix and we put out native seed.  I'm going to talk about the livestock exclusion treatment that's not kind of part of where I want to go. But if you're interested in that I can talk to you about it later. Okay. So this is what she found now there's a lot going on here but I'll orient you to the graph so on the top graph we have plant species richness on the bottom graph we have total vegetation cover, and I just three things listed on each graph which got cut off that stinks. Okay, well this is the first year, second year, third year so just this is early and this is later. And the orange all the orange bars are control. Okay, so there's no topsoil no seed. Then you have this little lump that's supposed to look like topsoil. And then you have these little seeds of seeds. Okay. And in the first year, after we deployed our experiment. So over here, what we find is that what I would have thought that when you put seed down you get more plants. Okay. Before this experiment, if you ever asked me, like, does passive restoration or does restoration without the inclusion of seed or plants work I would have been like, almost never know. Okay, almost never does passive restoration area land systems work. And look at that. Our first year that data goes with what I said, I know everything. Unfortunately, we decided to follow us for more than one year. And what we found afterwards is that there was no significant difference in either plant species richness or vegetation cover any year after the first year. So the green and purple bars that are showing just the topsoil topsoil seeded. There was no difference. Okay. So essentially what I had always thought what I had always recommended to people always was that if you want successful restoration, you need to add seed was totally blown apart by this study. Look, Alexander Pope is very disappointed me. Okay, I made this assumption about what works because in like, I've been working in restoration for like 15 years but that only means I've worked on what like what like 30 projects, which is a lot, but it's not all of them. So I had made this assumption and that assumption pushed my recommendations to people. So I was making recommendations to people that they buy very expensive but we love native seed I want to insult any native seed people here but like very expensive native seed and put it down I always made this assumption because I was under the assumption that you had to do that well that is clearly not true. I would still say in lots of situations you want to add seed or you want to put in plants. But clearly that's not always the case right. I was wrong people I admit it. I've been running again. So in the desert, people like to install solar panels. Okay. So something like this often actually looks something close to this okay.  And there's this weird dynamic in the Southwest about like revegetation, revegetating under solar panels. One, it's hard because it's just hard to like get in here to revegetate with machines and stuff. You don't have things growing you don't want the plants like touching the solar panels. And the other thing is that there's this widespread belief that in the desert. So we're talking about the Sonoran desert. You can't seed under these panels, because there couldn't possibly be native desert plants that are adapted to being in like the sun when it's like right here. So really, these plants can't grow under the shade. Right. I've never seen any studies on this, but this is the the general assessment and I was under this. This assumption as well. So then we did a test, and given this test is not out in the desert it's on a super contrived place but it's a test nonetheless. So we did on the roof of the building I work at which there is so this is kind of the roof. It's kind of weird but there's these big solar panels that have been installed on the roof and we're like, Oh, cool. A study system that I don't have to drive around to go get there so under in this whole area we seeded a bunch of native Sonoran desert plants plants that you might expect to be totally adapted and require the sun to be on them so that they could photosynthesize right. It is after we seeded we installed a bunch of plots. So these plots we installed I don't know why that says for that should say five, I made a mistake, another mistake. So we installed five plots that were under the solar panels, all the time. These plots were shaded 24 hours a day. Okay, we had five plots that were half shaded by the solar panel and half unshaded. And then we had five plots that were completely outside of the solar panel. Okay, this is kind of what it looked like. You can see so the solar panels were fairly continuous but they did have these little rays of light. And that's what the plants look like now I'm not going to show you a graph of the data I'm just going to show you a picture. And plants grow under solar panels like I think so. So this, these are all the native plants that we seeded plus like a couple of randos that I was like where did these things come from I don't know birds are dropping them in or whatever. But you can see that under the solar panels in almost continuous shade. Native Sonoran Desert plants were doing, not just fine, but really well, there was really high cover they were all flowering doing all the kinds of things and then out here in this wimpy land, things weren't doing so well. Again, I made this assumption based on just like kind of talking with people looking at people but I didn't see anything to suggest otherwise I was making an assumption that you just cannot do ecological restoration under the solar panels totally totally wrong. I think, again, it's not this silver bullet I think that there are situations where maybe restoration isn't warranted or it might be a little harder but this is suggesting that it can work. All right, not accommodating for uncomfortable truths let us continue along this pathway of my failures. Okay, so usually I'm working the Sonoran Desert usually looks like this super beautiful it's full of plants and up. And sometimes it looks like this so a border wall has been put up along the Mexico and us border and lots of regions. When you put up the border wall fence they have to take out a lot of the plant material so that they can move big machines to install the fence. So, lots of times you need to do restoration after barter wall installation to get back some of the, some of the plants and usually actually don't do restoration right here because you have to do this thing called the view shed. So people have to be able to see right next to the border wall but there's just lots of degradation that goes on with the installation of the border wall fences. So, one of my master students Amy Gill was interested in this and we were working in a park where an area of the border wall was installed, and about 4000 agave pulmarize were destroyed in the installation. So agave pulmarize a really important native plant in the Sonoran Desert, it's really, really important culturally for the native peoples there. It's important ecologically so there's this endangered long nose lesser bat I really should know I don't know I should remember the name but there's this bat that's endangered that relies on the flowers of the agave and it's economically important as well because you use it to make tequila. So it's a super important plant and they grow super, super slow. So she was interested in doing a restoration study and so actually you probably can't see it but this is a photo of the area where we did it you might be able to see the lines but anyway in this region right here we're about 4000 of these agaves were destroyed. So, we conducted a study in collaboration with some other groups to install agaves for restoration agaves are like notoriously difficult for like all the people in the region are like. But they're really important so we need to figure out how to get them in the ground. So, Amy was interested in looking at how to understand biotic and a bio factors important. So what are the things that might drive the success or failure of agave restoration.  Okay, and then she had a bunch of treatment so she had shading we did not use umbrellas we use shade cloth. We had herb or exclusion out there the herb or is there are rabbits but have alinas like particularly like these what once agaves are really big like in this picture.   Okay, so the area in which we installed our experiment was overrun by this non native grass called layman's love grass.  And we took it out in two different ways okay we hand pulled and we use herbicide so those are two different treatments. Okay, oh, here we go hand pulled. These are the pictures I decided to use I don't know what I'm thinking okay hand pulled and herbicide to get rid of the weeds okay. So this is what the data looks like. So up here we have the number of live agaves. Down here we have the number of leaves. Okay, so that kind of tells you something about growth. And then all the bottom here you can see the different treatments so see if for control we had an agave we didn't do anything we excluded have alinas we exclude have alinas and we shaded did it did it did it. Okay, don't worry about what's going on the bottom, I will orient you. Okay. So what do you see. Well, it looks like over here you have the highest number of live agaves when you exclude have alinas or you do some kind of shading or for something like that. And that makes sense because we know that have alinas like to go in and munch the little baby ones and shading is really important because although agaves are a native Sonoran desert plant, the little baby ones need to be shaded a bit because the sun can actually like bake them and and kill them. Okay. Okay, that we know, and everyone's known, not a big deal. Here's the weird thing. Over here, we found that some of the lowest numbers of live agaves. So the greatest mortality the greatest debt was actually in places where we remove the weeds. I don't know about you guys, but what I was taught and how I operate is that weed management is a central component of successful ecological restoration, because if you got the weeds in there, they are going to take all the nutrients and the shape like the space and the sun and all the resources that those growing plants that you want there they're going to take them all. So they're going to out compete the natives. So weed management is a super central component of ecological restoration. However, this is showing that that is actually not the case at all. Again, our little agave is that we put in had the most debt when you got rid of the weeds. You guys don't like the Alexander Pope thing. I thought it was cute. Okay, whatever, I'll take it out from the next talk. Okay, so this is weird, right? Because essentially, before I get to that, essentially what this data was suggesting is that it's a waste, not only is it a waste of resources to remove the layman's lovegrass that not native from the site before you put in your plants, but it's bad. It is against or reducing your likelihood to be successful. And that is super uncomfortable for like a restoration person to be like, okay, maybe weeds aren't that bad. But you know what people have known this for a long time. So this is a paper that came out, I don't know, recently. And this is just showing the diversity or the number of flower visitors of native pollinators on different plants. Okay, so higher is better. And the orange is native plants and the blue are non native plants. Okay, so all this graph is showing you is that there are times where non native plants are providing pollinator services. Okay, pollinators are important. And maybe weeds aren't all that bad. So I'm not going to use this and start going out and telling people like weeds are great. I love weeds. No way. But what this did is totally change my approach and saying like weeds are bad and always take like 30% of your resources and allocate it to weed management blah, blah, blah, blah. That's not always the case. In some cases, particularly when you're dealing with agaves, you can take those resources that you were going to allocate to weed management, allocate them elsewhere. Maybe buy more agaves. I don't know buy yourself a nicer field truck. But the uncomfortable truth here is that this like thing in restoration that like weeds bad. I mean they are bad but in some cases they're not too bad or in some cases they help you achieve your goal. Like this really uncomfortable truth that I had to come into, you know, anyway, okay, and how have I messed up the last one overlooking simple for fancy. Oh, this is the worst. This is the greatest sin. Okay, um, so I live in Arizona, if you haven't already picked that up and I live over here in Tucson. And over here is Phoenix or capital. And there is a corridor called the I 10 quarter it's one of the only parts of I 10 that goes north to south, crazy. And it's the highway you take to get from Phoenix to Tucson or to not Tucson to Phoenix. It's also one of the most dangerous areas of highway in the United States.  And the reason for that is that this is what the road looks like all the time so they put the road in this area. And that to either side of it for miles and miles and miles there's graded farmland ag land that's been left. Okay, there's nothing growing up there, nothing, nothing.  And for whatever reason people like to continue driving when they can't see what's in front of them but that's a whole nother story. So, you get these dust storms and, and, and it causes a lot of car accidents is terrible. Because of that there's tons of people associated with this road from researchers to Arizona Department of Transportation to farmers to all kinds of people who are interested in doing restoration to the, to the sides of the road so that they can get the soil in place so that it doesn't blow and then cause these accidents. Okay, and people have spent a lot of money, like an absurd amount of money, trying to restore these areas with like zero success, not like 2%, like zero success. So I come in with a group of people. And I'm like, I know what you guys need. You need some fancy stuff where we get like atmospheric physicists and soil physicists and all these different people to do like interdisciplinary work so that we can attack this problem from all sides. And they're like, you know what you're talking about. I was like, I know, I didn't. So we try this experiment where we're like, all right, we're going to put up some fences and we're going to have some plots.  So we're like, okay, the first problem is is that these soils. These are have been abandoned for like, I don't know, a really, really long time like the soils are like almost inert they're like super depleted of beneficial bacteria and fungi and like that are really important not only for the plant but for producing the stuff that like six the soil together. So we're like, we need to take like a soil perspective. Okay. And they're like, we never did that before and we're like, oh, this is so fancy.  Okay. And then we bought all the stuff and we sprayed it and we like talk with my probability colleges and we put all this instrumentation out we're like this is totally going to work. I don't remember which plot was which but it didn't work one of these is the control one of them is the treatment and it was a total failure. We got nothing at all to grow out there, or did we is there stuff growing out there. You guys see something on the ground. Alexander Pope sees it stuff grew out there, not in our plots, but up against the fence. And I was like, huh, that's funny. I shouldn't have said that, you know why because I've been doing work for years on this very thing. I do work on above ground structures for arid land systems because we know that an airline systems, particularly in the desert. Hey, when you get monsoons. I don't know if you guys have ever experienced monsoon, but it's literally like a wall of water you can get like an inch of water in like 30 minutes it is insane they're beautiful. But when you get this wall of water the water moves and it picks up everything with it. So you get this thing it's called sheet flow movement of water. Okay, so the water moves across the landscape it's picking up seed it's picking up organic it's picking up all this delicious stuff. And it just moves it wherever it wants, unless it hits something. If it hits something it drops it. Okay, and so I've been doing a lot of work on this like super old school I mean people have been doing this for like 4000 years of putting out these rock lunas or these brush piles. And essentially all you do is you put this above ground stuff down the ground and then you just wait and after monsoon, what starts happening is if the water comes in this direction and it hits this. You drop all that delicious organic material and seeds and then you're going to start getting stuff growing here, and it works. And I've done this for years. Why did it not occur to me to try this in that other site. This is another thing I work on I work on these things called con mods okay they're con modifier I don't remember what con is for con. And essentially they're just these super easy little metal X's and you put them out on the landscape and they're supposed to serve the same purpose you put it on a landscape so that when water moves through the landscape or even wind. And it's moving organic material and seed.  There's a little bit of shading which of course is always good so it reduces soil temperatures at the surface and increases just slightly soil temperature. Okay, and so you get nicer areas for seeds to germinate and then once they're growing, you have all that organic material that got trapped there so that they have all these nutrients to access to grow faster. I do this work all the time. Okay, this simple work, all the time. It did not occur to me to try any of this stuff at our site. Why because we were trying to be fancy man and it didn't work it like was just absurdly. It didn't work. That's all I have to say. So, you know, now we're trying all this other stuff where we're trying to put in another. Okay, we're trying to put in fences and con mods but I overlooked simple and simple like this whole field of stuff that I work in constantly that I know that works that is less expensive, and it's easier to do. I overlooked it because I was blinded by like fancy microbial stuff that we can buy online and then dig a pit and put a sensor down and, you know, and the point of this is not to like, you know, say that fancy stuff doesn't work because I'm sure there are times it does whatever. The point is, is that I was blinded by the fancy so I ignored my knowledge and I got blinded by something else and it turned out pretty badly because nothing worked. And so I think it's just important for us to remember particularly for things you know a lot of the that rock work I was doing it stuff that we know that people have been doing for 4000 years you know people have been doing for 4000 years probably works I don't know, but I overlooked the simple for fancy and it was to my detriment. Okay. Okay, that's that's actually all I have so if you take anything from this talk. Again, I hope it's not that I'm terrible at my job because I'm kind of okay at it, but the point is that I make mistakes all the time and all these mistakes were totally preventable. If I wasn't blinded by fancy things, or if I was a little more open minded about things that would work. Okay, or if I did more tests, or if I talked to other people, but I didn't I made that mistake. So I won't make the mistake again. Right. And hopefully you guys won't make those mistakes because you just heard me talk about them, and that is why we need to become a little more open as a field in talking about things that didn't work. Okay, none of us are perfect and also restoration and reclamation is extremely hard. We're never going to get it 100% all the time, ever. Okay, so it doesn't mean anything bad about us or about our abilities. It just means that you had to learn something else and if you share with other people you're more likely to get that knowledge that you can make it better for the next time. Okay. That's all I got. Thank you. Are there any questions. Oh, before I go on I just want to prep you all. I'm going to be here tomorrow morning. And guess what's going to happen. We're all going to share. Okay, I'm going to stand here uncomfortably if I have to. And I'm going to ask people to share their own stories. Okay, and you could share it about your friends Steve if you if you want but be ready, because that's what's happening tomorrow morning. Okay, anybody have any questions. Yes. The question was how long were was the scraped five set five inches of topsoil held before they re put it on it was about a year. Yeah, love pimples love me some pimples. Yes, so the one thing that has stayed consistent throughout everything I've ever done is that the more heterogeneity you have the better now I am also coming from a place that very often I'm working in the super degree landscapes where getting anything to grow is like seen as a win even if it's a non native, because you just need stuff holding on to the soil first. Obviously that's not the goal the ultimate goal is something else so you know we've worked with divots. Have you heard of ramps. So there's this thing run by the USGS it's called ramps. The R stands for restoration. Yes, I don't remember what the rest of it stands for but it's USGS ramps if you put that into Google you'll get to it. Anyway, the point of it is we have this coordinated. And this has to do with the question I promise we have this coordinated set of studies I think there's 27 now they're all over the Southwest and it's exact same very very large scale experiment. We're testing the same stuff and we want to see what opera and it's across already gradients, we want to see what works across all sites and etc etc. We're checking so we're checking the con mods we're checking all kinds of things we're checking divots and divots always always works because particularly in the area Southwest when you have any sort of divot in the ground that's where not only water will collect so it's it's So moisture maintains higher for longer periods of time but you also get the collection of the organic material and the seeds and usually even if it's a really small to be you get slight shading, which also increases soil moisture. So adding heterogeneity on onto a landscape, I think is a really, really good way to go. Anyone else. Yes. So go back to the question there were there any responses. I got probably like three and I sent out 500. So I probably got back emails of people that are like this is great I don't want to do it probably like 50 of those emails, and then the rest of people are like, Elise Gorn is not interested to lead. Yeah, that's a great question so in. There's a lot of organizations that are doing this thing so like the US Fish and Wildlife for example. I don't know if it's in the south or just the Southwest but they have this thing called CCAST, which is. It sounds for either but it's it's US Fish and Wildlife CCAST and it's a, what they do is that they find people have done restoration reclamation, they interview them and then they get like a little blog about it and it's all this information of like what were your goals tell me about the site what worked what didn't blah blah blah and then you could go on to a map, and it's all kinds of stuff it's riparian it's, I don't know if they have much reclamation but it's. And it's exactly that and so, and I've known about efforts like that in they had one like Orange County California and there was one in like the Northeast so there's people doing it here and there, but I don't know anyone that's doing like a nationwide folks it's it's pretty expensive students to like do to like find people and then interview them so it's like who was the money to do I mean US Fish and Wildlife I guess but there's no grants to do this kind of thing most of Eco Restore I like created based on like the you know last bits of money from other grants. It's like a really hard thing to fund but it's super important right. Is there someone back there. No. Yes. I love weird questions. It's not a weird question that's a brilliant question and yes so the question was, you know this whole thing with the, the, the solar panels obviously the plants were doing really well. Very likely because the soil surface, the temperatures were lower and the soil moisture was able to maintain. There is a whole like field of research it's called agrovoltaics, and it is growing food under solar panels, and it works and the in the three times the production under solar panel, the exact same inputs. And there's some people in Flagstaff they're doing so they're growing biocrusts under those things and then they're putting that back out. So biocrusts are really important for holding on to the soil and for inoculating the soil. So brilliant and people are doing it and if you're into it, there's people doing it. Yeah. Yes. Oh yeah, I probably should have mentioned that. The question is that weird study where we removed the weeds and the agave did so poorly. What happened. That was an excellent question. So two things. When you don't have the weeds, the guy they are much easier to find by the hovelina. So there was way more hovelina damage and without the wheat so the, the, the weeds were something called layman's love grass really tall grass, and the agaves are really small so we were actually acting as shade from the sun. So we were noticing way more herbivory on the areas where we took the weeds and way more like sun damage. So it's like, you can measure sun damage on these things. You know, we didn't measure that that's a great. That's certainly a possibility we didn't measure that. So maybe it is. Yeah, absolutely not. So the question was about anecdotal evidence and how do you. So, so people on the field know way more about what's going on than almost anyone else. Right.  How do you get those people to record the information is huge cashier information they have so that we can leverage it to build better restoration and I don't know the answer to that. So the way I do it personally as cooperative extension because it's the best job in the world. I actually am like the bridge between research and application and so I talked to a lot of the people who have the anecdotal evidence and then I facilitate events where people can share these kinds of things or I collect data and surveys or do this kind of thing and then I share that with some of the restoration researchers. But beyond that I'm not exactly sure I mean I think events like this where people are talking and sharing are really important. I don't I don't know I mean most of the folks I work with that are non researchers I'm always like write me a blog post like let's get this information out here and the people understand we are like I do not have time for that, or I'm not going to pay for it and I'm like I get it. It's a hard enough to crack. I think also there's a there's a need to get people in academia who often are doing some of the formal testing these kinds of things to understand the value of people on the ground because academia is like, oh I'm just going to like test the restoration thing and I don't actually care like what happens afterwards because they're not trained to care and they're not rewarded to care. So also getting them because when you when you get the people who are conducting the research or that some of the formal research anyway, and ostensibly getting out to the masses, even though they don't. If you get them to care, then they will start doing things like showing up to more management relevant meetings or actually going out and talking to some people in the field, they can get more of that like two way flow of information. Anything else. Yes. So the question is do I think it's changing this this getting academic research kind of more relevant, I guess the field and I would also agree that it's getting better and I think part of it is that a lot of academic grants now, like force you to be interdisciplinary and force you to have something called like a broader impact like what is the value of this outside of your lab. And so people are like, oh, like this has to matter outside of my life, I shouldn't make fun of these people but I'm one of them so I can put. I'm in academia, but so people are like oh like I have to think about like how the I'm not going to just do an experiment write it up into a journal that nobody reads and then then I'm done it's like maybe I should attend a manager meeting and present my work maybe I should write a blog post for a special issue, a special interest magazine maybe this and this and this and so I think it's almost being forced on through the granting agencies I mean even like NSF who they don't actually care about anything applied or that's my opinion, they have these these things in there where you have to write the broader impacts or the, the like utility on the ground. And so it's forcing people to think about it. Any other questions. I hope you all spend the rest of your evening thinking about the failures of your friends Steve, so that you can come back tomorrow morning, because I'm going to ask you and if nobody says anything I'm just going to stand up here and stare, and it's going to be super awkward for everyone involved. Okay. Thank you very much.


Federal Judicial Center Orientation Series for United States Bankruptcy Judges. A word of welcome to the Federal Judicial System with the Honorable Rhea W. Zobel, Director of the Federal Judicial Center and United States District Judge for the District of Massachusetts. I am Rhea Zobel, a United States District Judge and currently Director of the Federal Judicial Center. On behalf of the Center, I am pleased to welcome you to this initial orientation seminar for United States Bankruptcy Judges. I have already met some of you at the Center and look forward to meeting all of you at the second phase of the Bankruptcy Judge Orientation, a week-long program in Washington. I know that at this point in your new career, your main interest is in learning and understanding your responsibilities. In that connection, I would like to speak with you briefly about the Federal Judiciary, its governance, and the agencies that will be of service to you. The national policy of the Federal Judiciary is the sole responsibility of the Judicial Conference of the United States. The Chief Justice chairs the conference, its members are the Chief Judge of each Circuit Court of Appeals and the Court of International Trade, and the District Judge elected from each regional circuit. The conference meets twice a year in Washington, but much of its work is done through some 25 committees composed largely of circuit, district, bankruptcy, and magistrate judges. These committees consider issues relating to the Court's budget, human resources, space, security, and automation, as well as amendments to the civil, criminal, bankruptcy, and appellate rules. The Federal Judicial System is too large and too diverse, however, to be administered solely by a group of people sitting in Washington. That is why much of the system's operation is the responsibility of the Judicial Councils of the Circuits. A Circuit Council is chaired by the Chief Judge of the Circuit and includes an equal number of Court of Appeals and District Judges. The Councils monitor the state of business of the courts and the circuit, including any backlogs. They approve district court operating plans in such areas as juror utilization and court reporters, and they play a key role in acting upon complaints, charging a judge's disability, or misconduct. The Chief Judge of the Circuit also has the statutory duty to call a conference for all judges, usually with members of the bar and at least every other year. These circuit conferences are to consider ways to improve the administration of justice. They are also a valuable bridge between bench and bar and a source of ideas and innovations. Two agencies within the Judicial System are primarily responsible for serving you throughout your judgeship. The Administrative Office of the U.S. Courts, usually known as the AO, might be described as the Federal Courts Housekeeping Agency. It is managed by a director who is appointed by the Chief Justice after consulting with the Judicial Conference. The AO administers the federal court's budget. It is responsible for space and personnel matters. It supports the judiciary's automation programs and its libraries. And it collects and publishes extensive statistics on all aspects of the judiciary's activities. The annual report of the director further describes the work of the AO and the various ways that agency can be helpful to you. And now a word about the FJC. The Center is the federal court's agency for continuing education and research on judicial administration. Its policies are set by a board, chaired by the Chief Justice. The board's members are six judges elected by the Judicial Conference of the United States, as well as the director of the AO. The Center provides orientation programs, like this one, as well as numerous seminars, workshops and other educational programs on a wide range of substantive and managerial topics. Center programs are developed to serve the needs of judges and their staffs. We will send you information about particular programs for judges as they are scheduled. Our annual report, a copy of which we earlier sent to you, provides details about various facets of our work. You have also received our catalogs of publications and of audio-visual media programs. Some periodicals will come to you automatically, but all items in these catalogs are available to you. If there is anything you need and we can provide, just ask us. If you have any comments or suggestions, just tell us. I end as I began by welcoming you to this orientation seminar. In the larger sense, I want to welcome you to the Federal Judiciary. Membership is a high honor. It is also a serious responsibility. Those who have come before you have served with courage, diligence and dedication, and they have kept this traditional system viable, strong and independent. Each of you has now been given a unique opportunity to do your part and to serve our country. I know you will find it challenging and rewarding, and I wish you well.


Howdy greetings. It's quite the game collection behind you He said See couple others. I know Looks like fun If only I had time to play them, that's the problem Sure. Yeah, I also generations poster. I think we would get along. Yes. Yes, we would get along. There's also a spaceship discovery there and the Lego Gemini 5 Well, I see the captain's hat on top of the mini fridge. I'm questioning that one. Oh, that's a Marine Corps hat That was my that was my cover from when I was in the Marines make sense It's on top of the beer containment unit that seems appropriate The desk rum unit Emily, I just want to let you know they were all brutal with me last week when I was trying to leave this meeting Just real rough with me Just want to get that out there especially Chase. Oh Just just kidding just kidding She knows that's untrue. You've just revealed yourself That was my tell I Quick reminder for everybody joining us The agenda is linked in the chat Please be sure to add yourself in the attendance if you have any updates Please put them in parentheses after your name. If otherwise just put no update if you're a new member Go ahead and put new and we'd like to have you introduce yourself Gonna give everybody a couple more minutes. I will also need two volunteers as scribes I have a question on the scribe thing Is the is the expectation that it's a kind of word-for-word note-for-note or is it really just minute minute notes?  So we definitely don't need folks to type it out So really it's high-level notes But was discussed and kind of decisions that are going on Okay, that's kind of what I had defaulted to you, but I didn't know if I was being a rebel or no or in line I believe we had that updated in the role and the repo when we defined what a scribe does In one of my first times I was like, what is the expectation? I took down every single thing. I was like typing real fast You weren't the only one I did it too. I think I got word-for-word on my first couple meetings as a scribe All right, we are gonna go ahead and get started. So A quick reminder to everyone this meeting is being recorded and posted to YouTube shortly there after your participation in these meetings Is an agreement to abide by the SIG security code of conduct which can be located in our repository All right Pushkar you've got an update for issue 480 Let you go first Yes, so some of the folks from last meeting might remember We it's almost now a month since we public Published the security white paper version one So obviously we have gotten good feedback so far. We made some minor updates, etc There is a read me a PR also in words What I was wondering is If any of the folks who have read it or who have shared it with their colleagues Wanted to create a process working with everyone that would allow us to have a good set of retrospective for every version that we will publish and I've created a Issue on get up for it. Just pasting the link So the I the only ask from To ask rather from every one of you is If you have any feedback in terms of how could we build a process? Please add a comment to the get-up issue and the second one is potentially I'm thinking Emily open for feedback similar to the white paper Meetings we could have a couple of meetings at least in the new years after the holidays Just before this six security meeting And kind of discuss with the folks who are interested in working together on this So if you want to work with me on this DM me your email address so I can send to zoom And then after that we can set up something once everyone is back from holidays Yeah, and comment on the ticket as well for that and push guard. There's a pr out if number 477 about maintenance of The white paper and what that process looks like from updating the documentation within the repository So if you could take a look at that So I had a question if I may So, um, can you please clarify? What is the objective? I'm not sure I got the complete essence of it So is it just working on updates or what what exactly is the objective here? Yeah, it's kind of similar to how in agile After a spring to be kind of do retrospective of how what worked well what could be better And then based on the feedback that we receive from everyone who were contributors as well as Who are consumers of the white paper? We will get some ideas for what the next version should look like do we need to change a few things? Like do we need to reduce the content or was there something that was missing Or something that people thought would be useful to add more details about And those kind of ideas we can put it together as the first retrospective of version one and then While we're doing this we build a process for how to do the same kind of retrospective for future versions Okay, thank you. Pushkar. So I think there are two aspects to this if I understand it, right? One is how could because for a lot of us who contributed to this How could we make that process better? And second is from a content perspective I think we should keep those two distinct as we proceed through this I feel Yeah, then I think that's fair point We can do that What I want to kind of see is Give a voice more importantly to the silent voice in a way Who are not really in the group today that we discuss every day or every week with But mostly are like end-user consumers who read the paper But have some thoughts and we haven't had a chance to listen from them So from CNCF help maybe with the marketing team If we could get some kind of a survey out as well is what I was wondering would be maybe a good idea And I think from that perspective, I also feel like we need to give it. This is like brand spanking you, right? I don't even think like even yesterday. I did a webinar on Linux foundation where I amplified that message So I think all of us need to ensure that this is amplified even more Which is where I open another ticket where we should do a webinar To really get the message out before we are able to bring in that feedback So that give a little bit more time for people to actually consume All the content there and and so that we can feed it into the process that you're suggesting Yeah, I agree and maybe the webinars would become another data source for feedback Where after the webinar people will read it and give feedback Or to ask questions, which would help us in retrospective So, yeah, I'm between that awesome Okay Sounds like you guys are moving forward and that's great Go ahead It sounds so grown up. I know right um All right, so it looks like we have a new member And I apologize if I do not pronounce your name correctly Fasal Yeah, uh, so hey, good afternoon everyone. Good morning. Wherever you are My name is fessel razzak And uh, yeah, I I'm a new member here And just to give some introduction about my background Um, I I'm basically a software engineer And I also have a phd degree in automation systems from cure in italy Uh, currently I am in tronto canada And I've worked for a cyber security vendor in united states Um, and basically I deal with machine identity management. I'm a lead cloud security analyst there Basically, I my main role these days is that I go to different customer sites And I analyze their environments their infrastructures and then I basically make recommendations to them Uh regarding how they can manage their machine identities Uh, my introduction to this forum was that I read the security paper That came out the white paper that you guys wrote And well, I basically read the paper and and I deal with the customers as well So I have some background in that Information as well, uh, I want I actually joined this group because I wanted to see a couple of things Added more to that security paper regarding machine identities as well And also how people are doing their, uh CI CD pipelines actually right now in in a different customer site So I wanted to bring that knowledge here and probably in the next version Or if not a next version something else in the future, I would want to see a few things Um kind of changed Uh, and uh, yeah, that's why I'm here and let's see how it goes Well, awesome to have you join us today Yeah, right and then Next up, I'm sure like we all want to talk about solar winds and what happened there But before we get into that lovely conversation that I believe mark would like to kind of drive um, we have an announcement So for those of you that are new or some of you that have been around for a while typically our sig has technical leads That are they kind of drive a lot of the direction Um with the co-chairs and help initiate some of the activities and make sure the sig is moving in the right direction Um, I used to be a technical lead. I'm now co-chair Brandon Lum is a technical lead as is Justin Kappos. So Given that I am no longer a technical lead and the community is growing very quickly getting larger We've got lots of really awesome projects that we want to have upcoming um, the co-chairs Have nominated to our talk liaisons our new technical leads um So I would like to kind of welcome aradna ash And andres as the new technical leads for sig security joining justin kappos and brandon lump Uh Don't they have to be voted by the cnc ftsc first. So We're just going to open the voting but um, we agreed to do that today. Yeah Yes so Hopefully, we should be able to get this them all approved by january. Yes. We're very excited about it. So who's the third person there? andres vega okay Grace up uh Let's talk about solar winds mark. Do you want to kind of Start that Well, sure. This is it's early. Maybe it could be argued. This is too soon to talk about it but I think it's a great use case for the problem of securing the supply chain and You know the update framework is something that a number of us have poked Either using or have talked to the developers And participants in that project, you know, it's a cnc f project It's you could argue, you know a central piece of securing software in In the foundation to be able to secure the flight the supply chain The other thing is that you know from my point of view We tend to attract a and this is self-congratulatory, but we we attract a pretty high level of Competence in this group and in others and what that means from a attack surface point of view to use the the mitre terminology that smart people can Put smart bugs into software that you know end up being Compromises of both Not just the the tool in the case of solar winds here, but the whole Institution in which we are trying to stay engaged on building frameworks to build secure software and calls the whole thing into questions So the you know the the mechanics of how this happens get diluted as it bleeds out into the public space and becomes a You know both they slam against the cloud platform in general against self-regulated institutions like this one And I think about crowdsourced software kind of writ large into that that space So, you know, I have some views about this, but I really just wanted to open the floor to What our thoughts about this might be, you know, what are we doing the best we can do? You know, is this the case of you know lightweight adoption by commercial entities to things we're already doing well? Is it you know weak socialization of best practices that we think people have already adopted or is this really just the You know reality we haven't come to terms with that undermining the tools is really the most Sophisticated and best attack surface that we should have been ready for this It's it's lovely. I think to imagine that we should have been ready first. I think we all knew about coming. Um I think we're all humans. We're all lazy, right Um, I I found an example somewhere recently where someone was they didn't want to wait for a pull request to be merged into a project So they forked it. They did a build and then they included that binary in their docker image right And we're going to get riveted in two weeks because it's just a temporary thing and that was four months ago. So it's it's It's going to be ongoing. It's I think it comes down to how can we make the security as easy as possible for folks to use Um, so that they don't roll eyes when we ask them to adopt it no I mean there is a one of the I don't know weaknesses that I see here is a kind of an instinctive preference for vulnerability scanning as the security framework You know this it should have been a lesson when the asus update thing happened. I think that was uh early in 2019 we're probably some others before that but It's very hard to get away from that, especially if you have siloed teams in your enterprise That who do nothing but that and that is a You know, that's a busy making enterprise trying to keep up with that and it's not just a case of patching everything, right? It's a risk-rated gradation of things. That's a non trivial effort We have to acknowledge that that's the reality of it But I do think there's a tendency to say well, I passed the scans. It must be okay And then there's the other thing that we deal with in finance is Uh, how do you vet your third party? software in general your third party Contractors partners even some of your customers, you know, baby suppliers indirectly to you or exposed platforms and you have You know semi federated trust or maybe even more tightly federated trust That you are assuming is in place, but you really can't do assurance testing on any of that I think part of the problem is is that right you're talking about the problem where How do you I trust the software vendor that they implemented a process the way that I want them to Right and when all I get out of that process is a signed image where Their pki probably was exposed on an ftp server for two years, right? um So I think being able to bubble up that metadata that went into the build process In a standardized way is going to be the way forward And that's where I think this community can help out is what does that standard look like and how do we distribute that among the enterprises? How do we contribute to that? Is it what does the scope of it look like right? What is the minimal scope and what what would an ideal system look like? They're just the tip of an iceberg, right? So we're trying to do like I think it's happens every year more or less uh, loudly in a press depends on Uh, how many and how big companies it's been involved into accident But this is just like prevention of this and securing software supply chain Against such compromises is just like a prevention tactics But we should think about like defense and that like Presume that this is happens already or happening in your system. How would you be able to? Detect and find it, right? uh, I think so There there should be strategies like An assumption should be that this is happened. So if it happens, how would you know about it? I think part of the challenge has been that historically we've relied On self attestation for these kinds of things to Suggest that the supply chain is secure so we've seen some movement for example with cmmc to go in and to Try to bring some auditability into this and some third-party ways of attesting I think if there is a way to automate some of that Can probably reduce some of the cost and the load Um But that would imply an integration of the supply chain and Some of the work that i'm doing with With some other standards groups is trying to tackle this kind of a problem So i'm sure folks in this group are aware of some of those activities, but We have to pick and choose I think where we think we can play a role in this we we can't boil the ocean on this Um, it's it's a big problem if we can automate key parts of this I think would be a at least a step forward. Thanks And There've been enough problems. I think recently in in a cicd and if you guys subscribe to any any channels where you Get a like vulnerabilities. I think lots of them coming in different cicd systems and a plugin so um a good points that i've seen I don't remember it's been on twitter or likes or internal discussions related to this but there is a Feeling that's we do pretty good job or like descent on a product security site. So it's pretty hard to Get access or compromise systems through this but say cd historically been something that's Sitting somewhere and would strap in all our systems, but we paid less attention to The security of the system system as well and then that get an easy target nowadays for Uh bad guys that's trying to get into our system. So Maybe spending more attention for and cycles for Securing cicd systems as well as internalists also a good idea Just pulling on that if we can make the economics of this work just again the the issue here should be that it can't be prohibitively expensive for the supply chain members to participate in this like a really low key like, you know, somehow There's got to be a way to automate this is is kind of what i'm coming to I think Can I ask a question? There's a group maybe um I'm sure everybody probably knows fair and does some amount of Or attempts to quantify risk right for prioritization budget allocation all that kind of stuff But you know kind of some of the interesting thing here for me is when you When your crown jewels are not necessarily financial, right? Like if you are the cornerstone of civic responsibility then How do you Quantify that in a fair like system, right? How do how does that work if everything is meant to boil down to dollars? You know, I've had the same thing where I worked at a a nonprofit where we did a significant amount of human rights and activist work and You know the the push and pull there on What you couldn't couldn't do and who you couldn't couldn't protect them And what you couldn't couldn't guarantee was it was pretty exhausting But a lot of our models on where and when to spend money come down to you We can do it so well and after that there's insurance But not everything is recoverable, right if it's not financial. It's not recoverable so you know the whole idea of security being a part of the risk management space and It seems like that requires some acknowledgement that like We don't have all the answers and we don't necessarily know What are the most valuable things to protect always which is clearly the thing that you need to know to start So, I don't know that's a long ramble, but that's where I'm at with it I think there's points there though, right? It's so the first thing I think when the comment was around Cost of making this cheaper people secure. I don't think they're talking about a dollar cost There's all sorts of different aspects of that, but um Let's see where you're going with oh, yeah, no, I don't think cncf wants to get into risk management So I think that that's I hear what you're saying there But I think we need to bring this back a little bit think about what can we do to actually Either make recommendations to other cncf projects as my guess around Is there a standard we can either use or create or modify something existing that is easy to implement? It has that sort of cheap cost And is automatable for those who want to do that Is that sound approximate or am I Off the field No, I hear you I think the cncf's already engaged in risk management But I don't know that they want to inform anyone else that's for sure, but really I was just shooting the breeze right like What the ultimate impact or loss will be is oh hugely totally agree This is it's an interesting possibility, you know, yeah risk is in there I mean the reason I wanted to bring it up with this group Isn't correctly if I'm wrong about this but there's a de facto pressure is not the right word instinct For cncf to cannibalize its own projects That's a good thing if the projects are good, you know use the authentication tool that's in there You know use the the audit tool use the red team Tool that's already in the landscape, right? That's sort of an implied Uh, you know, if you're cut that's an implied coming to the party thing But what that says I think about security is The probability then of infecting, you know across the landscape with Reused compromise components is greater in the cncf than in many other organizations where You know, some of it is outsourced some of it is built-in house using Internal dev ops practices. So but here we have this sort of open source uber all is kind of uh instinct. Am I am I wrong about that? No, you're right I think there's a lot of There's still a lot of diversity in open source I don't think there's like I think there's a lot of different choices made by Different cncf projects and I don't think it's as uniform as that. I think there are More good open source choices you can make potentially than closed source ones in many cases. I mean there are more Probably more choices with closed source, but not all of them are necessarily good I mean you at least get the opportunity to make a better informed decision I mean, I guess kubernetes is the The obvious thing to call out in this, right? Is there something that we can add or integrate into The security assessments that we're currently doing to either increase visibility and awareness Of how these things can happen or to help teams think more about how to prepare to Potentially address this or mitigate it either by providing end users with behavior signature of their projects so like When it's deployed, this is what it should be doing and looking like and having that separate from the actual build and release Or is there is there something that we're already doing and we just we haven't Reached enough saturation within the landscape I think Emily you took the words out of my mouth. I mean, I was I came back to What can we do right and one thing that we can do is the assessment framework and in the assessment framework Call out Very explicitly these best practices, right? And and that's one of the things that I think I had raised when I participated in a couple of assessments Where we're calling out a few things and a lot of those projects have done their own pen testing and assessments and that's what they provided But we were had no way of evaluating a particular control if you will or a particular Recommendation or something. So maybe if we added Some sense of for every single project that goes through the seek security assessment that we can categorically say we've evaluated Against these controls these And we've also it's also been actually how do you say evaluated against that control? But we've always provided that and it's just been a discussion But we've never had the and please for correct me anybody correct me if i'm wrong But we've never had the feedback to come through to say yes We've gone ahead and checked to make sure that you know, we handle secrets There are encryption keys and you know those kinds of things Does that make sense or do I just go off on a ramble? So Vinay, I just wanted to offer some thoughts. I think having you know, any kind of a Way of assessing security is a good thing But many times if we start to fall back to a manual way of doing this then it doesn't scale So whatever approach we end up taking has to be Automatable, so if for example, we're going to check in on secrets management Specifically, what are we going to check in and and I think it's going to help us to kind of drill down On what are the key things and we can build this over time But just get the ball rolling and iteratively if we can tighten this a little bit I think would be kind of the approach to take Yes, that's a good point. I mean nothing is manual. I completely agree. Absolutely Here was If comparing like the same safe projects versus Individual projects managed by one person Generally The overall sense is I have more confidence On cncf projects security because of the process involved from sandbox to graduation Security assessment done by the group here But it's sometimes Uh, not a zero to hundred percent. I mean zero or one Boolean decision So maybe in the assessments, I don't know if it's already done. We should mention a few things that We haven't assessed for so we generally share that this is in scope. I would imagine But sometimes maybe it is worth saying that Supply gene security was not in scope of this assessment. So that when people are Not blindly trusting that the assessment was complete Uh versus knowing that okay, this was not in scope. So we need to do our own due diligence for So we have a couple of those things documented that are needed to be done in the in the updates So for anyone that's not familiar I'm going to do another plug here for Brandon the security assessment process It's performed by the SIG is currently undergoing some changes Based off of feedback from the last five assessments that we did as well as some Community involvement and some of the other things that we've noticed over time while working through these assessments So there is at least one pr out that has a recommendation for like new documentation that federal aligns with the current talk phases for cncf either sandbox incubation And graduation as well as there's an updates one coming out. There's benefits associated like so explaining To project teams. What what are they going to get out of it when they're coming to us? And we're asking them for all of this information and what what are their end users get out of it? So there are plenty of issues On making these updates Check them out on the repo. There's actually a security assessment label. I believe that you can click on So if you have ideas definitely add them to those issues I think some of what we had talked about in that working group is a lot of these things From my perspective, I'm not sure entirely how much of it is We're going to be evaluating teams against a standard So to speak or against Best practices so much as providing them educational information about like these are the things that they really should be focusing on This is where they should be paying more attention Because realistically if they're not a security project, they're probably not thinking about it In the forefront of their mind and we've to date really done closer looks at the security project like with spiffy inspires assessment and In totos looking at them from a supply chain But not all projects are security-minded. They're not security focused And we'd like to be able to help them out in that area so that the entire landscape becomes more secure But how does that look what it what are What are we in? What are we going to be on the hook for providing in in terms of education and documentation? This one specifically would suffer supply chain based right and so that's I mean That's obviously the low hanging fruit aspect or no somebody's nodding no on this. I just want to make sure I'm No, it's it's not the only software This is this is where it started Right Like my my whole point like you cannot solve this problem with only one thing And back to Emily's point Uh, we have multiple projects in Science here right but Some of them might not be there, but if you think about this problem holistically like who Like here has everything all tools to be able to protect and detect this Uh, I can openly say that we don't and this is like why we're looking into this problem Like what are the tools here? That's why I prefaced it with that. I'm sorry to interrupt it Like that's why I prefaced the whole that's part of it Like is that the financial part of the software supply chain? There's obviously runtime, you know capabilities There's things you need to do at a node level. I mean, I think we're all in agreement It's it's not just one one thing, but it's basically like I said some type of standard Which I think Emily you I think you're You know also kind of alluded to as well. It's some type of base standard We can say here here you are These are kind of what you should be doing for each of the projects We do that somewhat with the assessments that should be almost a playbook as well post that says okay going forward Because people's like, okay. I just escaped this assessment. Cool. I'm signed off Then what happens is a year later something like this happens And they're like, well, you know, I don't have this set of like list of things that I need to do in my world Right, so that's what I think is is probably missing is some type of at least ongoing You know process for those types of things and I'm sorry to cut you off. You like No, no, no, no worries. Just just to finish my thought I think that's that's a good idea to have users example like hey, this this is what happened This is what we know and this what you probably should be doing and these are the tools that would be able to Not completely solve this problem But have a certain mitigation to prevent this or if it's not if you cannot prevent this and how you probably can detect this and that's what also help like polish and landscape and Find out what what gaps we have and what tools we need and what which projects that might not be in there that we need to engage with And I think now to a certain extent the core infrastructure best practices batch Does measure for a number of these things and it's something we look for in the assessments We require projects to attain the best practices batch if not the silver it's also something that the toc looks for the promotion of any project from Like intake or promotion from one state to the other Assessments by nature. There's like a lot of variables. Not many things are constant But like one thing we've come to is like hey, how are you managing secrets? How are you managing rotation of keys? I know Justin was very Diligent in asking well, you're using tough great We cannot just check box and like give you passing callers for this thing How is stuff actually being used like what version of tough? How does this interact with with all those different things and that does uh Demand a lot of like well taking a closer look at this things but like for starters If we extend either the cii best practices or we do like a security best practices batch We could measure for like at least common denominator things That should be in there And just forming a thought around that I want to put that there like thinking I like that a lot. Yeah, I like that a lot and uh the cii batch kind of if we were to How amenable would it be for us to come up? Of course, this is uh, and this goes to al-taz's point Which is all our fully automated, but how? Costly in both from resources and all those kinds of things would it be for us to come up with here are the 10 security checks And then we give that badge. It's like security checked badge for a lot of these projects and a constant maybe which goes back to chase's point Which is 3000 5000 10000 and regular assessment. Is that a too high a bar? I think so because um, it is not one time thing again You have to do yearly reviews or periodic reviews of those products and re-certify them as well Because new threats will come and new vulnerabilities. Who's going to keep up all the updates to them? That becomes quite challenging It's almost like a certification and again might might mean more work for us But i'm just thinking in general like there's an assessment that you do to get a project over the hill, right?  But then also we need again. I mentioned this playbook It's not like something you drop off and say go It's like look there's a link to the best practice of the sick security has that you need to put in place from Your supply chain from your runtime from your whatever it might be You know that they can refer to and you know, again, it doesn't absolve Sick security from all this but at the very least it gives us these best practices. I believe that's help more helpful than what's in place now So is that something There's an appetite for to expand from the white paper on So the white paper was intentionally intended to be like the my first cloud native security architecture kit How do we take that concept and break it down even further to more concrete actions that we want to see projects and In and entire architectures take when they're looking across the ecosystem at what's going to be their orchestrator What's going to be responsible for their service mesh? How are they managing identity? I think that's a great great starting point. I think we can Expand from the paper, but there'll be lots of offshoots right where we write individual detailed documents providing guidance just for supply chain Open source software Etc etc But we should map that out first as to what that offshoot Will look like and then from there we can start developing those best practices The other thing I want to inject in the conversation and I don't have an answer. This is going to be trade my department of defense history But the tool I stole our wins is the forest multiplier. If you look at the cncf landscape, these are not all equal threats, right certain tools that are Highly scalable or are inter penetrated in the network, you know have differential kind of risks and threats So I know we we try to and I've seen some of this in the assessments We ask the sponsors to talk about, you know worst-case scenarios And impact of that but maybe we could do something more formal around that I think I probably could one of the one of the original intentions behind the white paper was not to Get into a lot of detail, but be able to create additional documentation Or content around the white paper for specific areas like The reason why we didn't fully expand supply chain security is because it's such a large Area with such a huge impact and potential repercussion that it really needed its own audience its own kind of outline Yeah, well put like it needs its own sub community for assessment. You mean, yeah I think it's a highly nuanced conversation Because well, even if we go like the the badging route and if you look at the batch app There's a projects that achieve the gold standard and that's the linux kernel and the update framework And that does not mean that these things cannot deploy in like sub optimal ways now if we look at solar wind and like Well, it's it's unfortunate and like The response at this point may be like hey, let's shut down every single instance of of solar winds and attacker can Actually have anticipated that and they're going to conduct the an advanced persistent threat Monitoring's been shut off organization-wide now Like they may be employing a bunch of other things like attacking like weaker points that are like Things that don't meet this gold standard or don't do this thing. So while we could Look for things that if well the in the development and maintenance of this project Maintainers are doing the right thing like this how they're signing release This is like how many keys are out there if who can sign a release All of those checks are good, but there's like That's the theory in practice. There's there's a lot of deviations that may exist. So Do we contemplate those things in scope? We don't and just say hey, we're we're making like this uh At the station of like things have been done right in the delivery But once once things are running in prod like there are all those deviations that may have occurred Are they discrepancies between like intent and implementation? That's non-zero don't know So I kind of get the sense that the conversation is steering towards establishing a set of controls and I Would caution us in going down that path We don't want to slow things down We want to be enablers of of kind of the the Innovation that's already taking place, right? And there are different projects that have different groups different sizes different levels of maturity Could we for example You know provide Code that has already been Through this process and say look here's something you can reuse as an example, right? But these are ways to help accelerate rather than You know forcing this down and saying you can't move forward unless control control control just it hasn't worked in the industry It's just it's security getting in the way essentially. Anyway, uh others are welcome to chime in. Thanks I think that's why emily calls it education. That's uh We're in another another meeting about talking about cncf as a third party and This was the professor at indiana university and he said, you know cncf has been successful when you compare it to the grid Community which you rewind the clock five or ten years They're trying to do a similar kind of thing with the grid and it failed because there were too many controls placed on the projects and Sort of the loose rains model of cncf is part of the Part of the secret sauce here. So maybe alt has is getting it You know the the need to I guess keep that secret sauce working and yet Provide projects with things that are enablers to do this I don't have all the answers either, but It's sometimes you got to make this conversation happen When it's uh, you know front of mind and this is one of those moments If there's a situation too one of the things is if they're starting out and they have that white paper as an example And they're embedding their security in there from the start. I think that's also useful, right? So I think again the six securities role and that is basically making sure that from the start if there's a new project coming in They already think about these things. So what are they? What are they doing right now? They're like I need to get my code out there So people can buy my product or you know or use my thing You know what I mean? So the end of the day those are the things are going to be the priorities for them And if we can kind of say look, it's not going to impact you. It's just just think of these things You know circulate your keys, you know, uh Make sure that your aws instances or whatever are really locked down. So, you know, you can just do those things I mean that that I think is we would be doing a service to the end users by doing it that way I just want to share my screen as we talk through this because like CII best practices does make all these recommendations and it's not it's not Putting a barrier or hindering People discouraging people from like developing and open sourcing software is like, hey Keep doing what you're doing come in here Uh, say whether you met or unmet something and like what's what's your progression towards that? as opposed to like a lot of people see the assessment as as the check box thing like oh this got us on to like incubation But a few exceptions that I I want to call out. Oh, but we've done a great job and gone back and said Hey, we actually followed up in the recommendation and they're working towards say a silver batch Some projects may do it because well, it's going to get asked again in order to graduate some others may just do it because they care about it and they want to improve this things but yeah, secure release What artifacts are performed if you could go to gold like the requirements are more stringent What are the cryptographic practices and used is it like delivery secured against man in the middle of the tax? I wonder if we should like promote this more and like incentivize it and encourage it somehow If this is the first time I'm even aware of this, right? You know what I mean like as a project that's you know, anyway part of a project, right? So like I think the awareness here again, I'm sorry for cutting you off there But like I I think an awareness of this would be amazing because I had no idea this was even available I like it. I think if we can build on this and look for ways to automate it It is yeah, it is actually not automated. There's no conformance tests off this essentially a a project person comes in and works through this list and they self-certify Hey, do we meet do we not meet this? and provide some evidence for it, but it it's like Tracks for those things and it outputs a score But no one is actually coming in like no human or machine corroborating that and the truth And how often is this updated? Are they do they have a requirement to come and keep it fresh? So I want to answer no in my personal experience. I I felt out the one for spiffy inspire When we're working to move from when we made a proposal to hey, we're ready for incubation One thing we do is we actually put this on the We have the little batch On the project read me So it says whether it's passing. I think we're like 198 percent towards the silver batch Let me get this thing out of the way and yeah, I have gone back and like we have one standing item on On one of the categories that we're working towards but it has also given the project a A guideline of hey, it actually helped us clean up and organize a lot as we were working through these things of What could we improve and where should we be doing better? And if not, well, what? Why is that what are those sharp edges? What were the trade-offs that led to that decision to be made that it it helped it helped us it helped other people But yeah, a lot of people are not not aware that this thing like exists or or projects go through it so I want to like Circle back because we we've talked about a lot of stuff and we've got 12 more minutes left. I know you guys all want to be here all day, but so I've heard that I've heard that potentially creating libraries or even Rollsets or policies for teams to either Enable some sort of automated detection that they're doing better that they're being more secure in their practices or that they're meeting CII badging criteria I've heard The retrospective on the white paper and just in this conversation alone sounds like there's a lot more work that could potentially be done there As well as more updates to the security assessments that we're currently doing the pr's that are already out there for it and the open issues Those are like the three primary things that i'm i'm hearing for how we can help educate the community better to do better and just kind of Strive for more secure projects Did I miss another like potential action? There's two things there. There's there's the areas we need to educate in Um, I think the other part which is sort of interesting which will come from that retrospective is how do we go about doing that education? um It is a 40 page white paper Targeted towards CISO is going to be the best way to to get some of those points across or is it more like that checklist? We're showing or I think that's something that's worth sort of considering in there as well I think a sandwich model is always good You start from the bottom and the top down and you meet in the middle So if it has to come from the CISOs as well as community or grassroots driven efforts, definitely We could potentially look at that from an education opportunity And uh, maybe the other one was the current topic, right Emily, which was is there anything that we can do Should we open a pr to just start collating all these great ideas, right? Which is uh from a ci i best practices Like a security best practices and maybe pull in a lot of the security assessment framework to actually make it uh, and maybe brandon is already on top of that to actually have it Rendered somewhere where we can actually look and feel and that's a great first step And then we start to see how we can potentially automate it and have projects. I love this framework I mean and to provide a security framework in the same vein would be incredibly invaluable and that satisfies education that satisfies assessment Evaluation and then we can also keep it a little bit more lively as we mature Yeah, I would agree. Um, I think I would like to see a lot of folks sign up for that retro with the white paper To kind of help drive more of this educational conversation um, I would also like to see Folks take a look at the current security assessment improvement issues as well as the prs that are associated with them I think we only have like 11 prs in the repo right now, so there's not that many to go through Just to evaluate with the current recommendations from the working group. Are they still sound? and how do we improve them and then for the other bits that we don't actually have projects in flight for Maybe create an issue to evaluate the the outfall of the solar winds event How do we do a better job of increasing awareness for the community whether or not that's through Talking to them about it Instituting checks within our own processes or even creating those libraries for teams to be able to pull in Yep makes sense It's a good like I didn't know it's called the sandwich model Well, I'm calling it the sandwich model. There's probably a much better term for it Let's go top down and bottom up at the same time. That's a great instinct. I have to I was just going to add to this. I'll put in the chat the Building scenarios is one way to get at and some simulations or You could do this with red team virtual red teaming also, but To identify these force multiplier scenarios like Compromising vm where you know the thing that fires up instances of containers these are places where if you do an inject you can scale The risk rapidly and people may not you know think of those unless they're forced into You know sort of iterating through the scenarios. I don't know where that fits into your taxonomy, but that should be on the list. I think Agreed so who's going to make the issue to Capture all the little tiny tidbits for projects we don't currently have in flight Again Emily, I'll take a stab That would be great Vinay. Thank you. Sure. I can help you if you need help Yep, I'll uh, maybe I'll let's yeah, I'll reach out. Thank you No worries I would underline marks marks point not to not to regress back at the conversation, but As scenarios are extremely important like a lot of the assessments focus on on software As is in the repository not so much as In runtime and all the possible different ways that people deploy. Thanks Agreed Jeff this is a great community for this Hey, yeah, and then how do we gamify it at all of like sandwich model or not? Like how many carrots versus how many sticks is the right combo? agreed so We've got seven minutes left. Does anybody have any final thoughts on this topic? It's job security I feel like everything in security is job security All right, are there any other topics? That somebody wanted to talk about with the last six minutes that I might have missed inadvertently One of the topics from last time wanted to just give a update there Brandon and I were discussing about how CNCF landscape and cloud native security landscape are not really Related to each other even though they sound very similar So I have been thinking about would would it be a good idea to potentially rename cloud native security landscape to something else So I have some ideas. I post it on slack. Maybe if people could vote and we could probably choose a better terminology that Clarifies that those two are not related. Maybe we would have something there So everyone's going to jump in the slack channel and vote Yes, and I'm going to see a whole lot more traffic on our issues and rprs from the community today, right? Okay, um, if there isn't anything else I will give everybody back five minutes of their day Can you pin that vote in the channel so they can find it? That was all sorry for talking about me. Thank you I agree. If I have access to pin, I'll pin it otherwise. I'll let the Maybe Emily or Brandon pin it I'm glad I'm not the only one who can't find stuff on slack All right. Well, thank you everybody for joining another awesome sick meeting. I look forward to seeing you guys. I believe are we meeting next week? Is that right? We certainly have an agenda for it All right, we'll see you all next week. Have a great day. You too. Thank you. Thanks everybody Thank you