First, I'd like to thank the organizers for inviting me to speak here today. I'll talk about the project we've been working on for predicting time to ovarian calcinoma recurrence using protein markers. In this study, we mainly focused on high-grade series ovarian cancer types. Let me start with a little background. The standard treatment of ovarian cancer is a treatment surgery followed by platinum-based chemotherapy. But the initial response rate is pretty high, but 25 to 30 percent of the patients relapse within 12 months, and too much doesn't respond to platinum therapy. For the TCJ data set, roughly 30 percent of the patients were classified as platinum resistant. So there is a need of noble predictors of a platinum resistance that can lead to change in therapeutic approach. Lately, we developed a patient classification system named classification of ovarian cancer, clover. Using TCJ data, we identified the subtypes and survival gene signatures, and then we built a prognostic model. We found that clover significantly predicted overall survival. These couple of minor curves were generated after classifying patients into three risk groups using clover, and this was obtained from the validation set. The log-rank test was performed to compare overall survival in risk groups, and the p-value is very small for overall survival. But the prediction of progression-free survival is not as accurate as the prediction of overall survival. We found a very similar pattern in several other gene signatures from other studies. Those gene signatures strongly were significantly associated with overall survival, but not with the progression-free survival or treatment response. That motivated our work. You may wonder why the sample size is different for these two plots. Well, not all the samples had progression-free survival information. That's why we have a smaller sample size for this. In this study, we aim to develop a predict of a platinum resistance using protein markers. For this, we use the reverse-phase protein arrays, IPPA technology. I think most of you are already familiar with IPPA. It is a high-throughput technology for measuring protein expression levels in a large number of samples. Using our IPPA, we measured 172 cancer-related proteins and 4-sport proteins in 412 TCGA samples with serous ovarian cancer. Of 412, 220 samples were included in the model construction. Those samples had non-missing values for progression-free survival, and they all had advanced stage digits. In this study, we developed a protein-driven index of ovarian cancer. We named it PROBAR in line with CLOBAR. Let me show you a brief flow chart of how we constructed PROBAR. First we used 222 TCGA samples as a training set. In the next step, we applied LASSO, a statistical regulation method to the TCGA set. I'll talk about this in the next slide, just in a minute. Using the LASSO, we identified the nine protein markers that are most associated with progression-free survival, and we estimate the coefficient from the COX regression. After that, we defined PROBAR as a linear combination of the nine protein markers weighted by the COX regression coefficient. A statistical challenge of high-dimensionality arises whenever you predict survival using genomics or proteomics data. For the feature selection and coefficient estimation, we used the LASSO. The LASSO estimator is obtained by maximizing the likelihood function with a L1 norm constraint. The main advantage of the LASSO is it provides a parsimonious, very simple model by shrinking unnecessary coefficients to exactly zero instead of small values. By applying the LASSO to the TCGA set, we identified these nine protein markers and estimate the coefficients. You may notice that the coefficients are very small. Those are shrunk by the LASSO, and it is known that shrinkage may help avoid the overfitting problem. These are nine protein markers we identified, and these five proteins had negative COX regression coefficients, which means these were associated with better performance, better outcome. And indeed, these were overexpressed in the low-risk group. The other way around, these were associated with poor outcome and overexpressed in the high-risk groups. These little bars indicate which samples are platinum-registant, and we see more platinum-registant patients in the high-risk group. So we identified the nine protein markers. We defined that we computed pro-bar for each patient in this way. It weighted the sum, and then we classified the patient into one of the two risk groups according to pro-bar, and we used the median as a cutoff. Here are results with pro-bar. This is based on progression-free survival, and this is overall survival. And pro-bar was constructed using progression-free survival, but it is also predictive of overall survival. And again, not all the samples had progression-free survival data available, so that's why we have a smaller sample size for this. These were obtained from the training set, TCGA training set, and we need to validate pro-bar in an independent data set. We were able to obtain 229 high-grade serious-validation samples from Japan and Philadelphia. They were treated uniformly. And using RPPA, we measured the expression levels of 144 proteins and forceful proteins for those samples. And as you see, pro-bar is significantly predictive of both time-to-tumor recurrence and overall survival independent data set. And we tried out three groups' stratification, but we saw a similar result. For the purpose of comparison with a gene-driven model, we implemented the pro-bar in our validation samples with the gene expression data available. Well, the sample size is small because it's only subset of validation samples that had a gene expression data available, so the sample size, we are just using 130. Let me first show you the result with the pro-bar. No significant difference in progression-free survival between groups. But pro-bar, obviously, it improved the prediction of progression-free survival. Finally, we tried to test the robustness of the nine protein markers that we identified using the TCJ samples. To do that, we sort of in a reverse way, we used the validation samples to identify protein markers. And then we compare those samples, those proteins, versus the original nine protein markers and see how much similar they are. Using the validation samples, we identified these five proteins. And let's compare this with the original ones. Unfortunately, AL is the only overlap. But when you clustered proteins using hierarchical clustering, we found that some proteins shared very similar expression profiles. Let me explain this. These little bars on the first row indicate the location of the nine protein markers that we identified using the TCJ samples. And this one, the second row, this one is overlap. And these are the protein markers we identified using validation samples. And overall, our protein markers were spread out over the clusters. And interestingly, nine and five protein signatures included representative proteins from each cluster. So in conclusion, we developed a protein-driven index of ovarian cancer, probar, using progression-free survival. And we showed that it is predictive of both progression-free survival and overall survival in high-gradus ovarian cancers. And many genetic signatures in other studies often contain a large number of genes. But unlike this, probar is simple, but it's still predictive, making it useful in clinical practice. I'd like to thank my collaborators, especially Koski Yoshihara and Lou Burhawk, who is my mentor. I'd also like to thank Gordon Mills, Iling Nu, and all the other collaborators. Thank you. Sure. I'm curious that one of the common prognostic protein factors was AR, the androgen receptor. And I think that kind of goes back to the question that one of our colleagues was asking about ER and lung adenocarcinoma. What's known about the role of androgens in growth of ovarian cancers or other women's cancers? You mean, ER or? I thought you- no, AR. AR. Which person? Well, I'm a statistician. You don't expect that. That kind of answer. Well- Thank you for that great answer. Okay. Well, at least it is a significant statistically, and we found in many studies, AR is a significant predictor of ovarian cancer, yes. I have a question that's hopefully more up your alley. So I was wondering, you used an L1 regularization. There's a lot of evidence that mixing L1 and L2 regularizations, such as when using elastic net, you could basically also capture redundant predictors. Did you try anything like that, A? And B, if you did not, another trick is to sort of remove those first nine predictors that you got with your L1 regularization and provar, and then see what's the next best group of predictors for basically predicting survival. Did you try any of those things? Are you talking about relaxed lasso stuff? Yes. We haven't, but probably we can do that later. I mean, I would suggest either doing elastic net or seeing what's the next group of predictors, because that would be interesting from a biological standpoint as well. Yeah, we'll look into it. Thanks. Anderson Baylor. I was wondering if you had looked at the patterns of androgen receptor co-activators or co-repressors in ovarian cancer that may correlate with your patterns of expression. Androgen receptor antagonists like flutamide have a notoriously low response rate in ovarian cancer in the range of five to eight percent, despite the fact that there are studies out there that suggest that the androgen receptor itself is expressed in about 70 percent of ovarian cancers. So I was wondering if you had looked at other factors that may be playing into your observations. We considered clinical factors, such as age, stage, grade, surgery status, and also BRCA1 and 2 mutations. And those are significant, actually. But we considered multivariate, multiple cox regression, and probar was the most significant one across the dataset. Four of your nine antibodies are phosphoantibodies. So could you please comment on the potential technical bias related to the different ischemic time? We haven't done that functional analysis. We should pass away our functions in four clusters. Probably we'll investigate further. Thanks. Okay, thanks.


Yes, sir nice teacher pretty cool. Are you alone, or do you have some friends? Excellent and that will be one minute and 43 seconds over but we'll let you do it anyway They would Weinberger from the Harvard Innovation Lab And another of those with which I have a potential conflict of interest because they're great friends and neighbors as well over to you guys It's a thrill for our entire team to be here. I'll say it. This is an historic occasion It's a lot to the Digital Public Library of America. So it's really amazing to be here to be in the presence of these other Sprinters is my head and heart are exploding with the desire to interoperate thrilling so shelf life collaborative is proposing Two projects integrated one is library cloud, which does the heavy lifting the server underneath it and shelf life Which is it's far more gregarious application that runs on top of it. So library cloud is a metadata server. It gathers metadata from institutions libraries collections And makes it available through open APIs and as linked open data The aim of library cloud is to take everything that libraries know and make it all available to the entire web Ecosystem including other sites and services like Wikipedia that can take it up But also to developers and innovators who build perhaps recommendation engines or library analytics or other portals like like shelf life or who knows what that's the whole point So library cloud currently has about 15 million items in it Which means metadata about items. We don't collect content itself including Circulation data anonymized from the partner libraries which include three public libraries and two university libraries We also have been gathering metadata about web objects And we think of library cloud as a metadata amplifier that takes metadata adds value to it puts it out into the world Applications take it up feed it back in and every turn that metadata gets more value It gets amplified in its value Which means that the impact and value of the DP LA is also amplified So let's take a look at shelf life. This by the way is a screencast that we did not our hotel over Wi-Fi last night But it's real stuff. You can come see it in our booth or of course go to our sprint project online and Spend as much time as you want poking at it So our idea is that you would get to this site either by coming to DP LA org perhaps Or maybe you go to your local library site where an embedded DP LA shelf life widget gives Users at the local site access to the content of DP LA and also to many of its services So it's a doorway to DP LA or other apps perhaps that we interoperate with We got this is a page you see when you click on something you're interested in in this case We got here because shelf life recommended the Pluto files Because it knows about what we've been doing and at library cloud has a great deal of information about the general ecology of interest So the first thing to notice about shelf life is that it is a place We think it should be a place because we firmly believe that the DP LA is going to be a major web destination But it's also a place because Shelf life is not just a place that you go to look up something You know you want or to find something that you didn't know you want which is wonderful when that happens It's also a place to spend some time to explore the browse to navigate to Contribute to enhance to learn and to share with other people because it's social all the way through So this means it has to be accessible to everyone in fact the DP LA has to be accessible to everyone So we use the single most Common understandable simple user interface around which are shelves if you prefer The now's standard opac cover browser. We can do that too when we have covers. There are a bunch of them there So if the object that you're looking at is a book the width of the book indicates its page count Actually, its length is represents its actual length But if it's online then shelf life is a fully click-and-play Application so you click in this case we're going out to the open library In fact, we're using their embedded browser. So thank you open library for everything that you do. You're you're amazing If it's another type of media if it's a video same deal You click on it click-and-play and you can watch it the same for Audio and the same for web pages. We have 60,000 Wikipedia pages about books for example that you can click on so That's a lot of context, right These shelves are in fact special they're special in three ways and the first way is that this is our attempt to integrate Five the five collections that we have all on one infinite bookshelf now You can facet these back into their original collections because browsing by collection is a particularly useful thing to do But with shelf life you can also facet them so that you see all and only the content of your local library We think this is a pretty interesting way of combining the interests of the DP LA and local libraries Second way this is special is that the depth of the color blue indicates Community relevance or what we call shelf rank which we calculate based upon user interactions with it plus using that anonymized Data circulation data that we're getting from our contributors and the third way that this is special the shelf is that all Items occur all works of culture occur in a context in fact they incur occur in multiple contexts because human brain is very contextual That's what we do and so we allow you to pivot on say the Pluto files to see it in multiple contexts So one of the contexts is that which is carefully prepared by professional librarians as they categorize so you can click on one Of the categories or any of them and see this work with all of the other items Categorized in it by professional librarians, but we also want to combine that wisdom of libraries with the wisdom of users as well So you can see it in the context of what other users attend or how they Recommended works to bring them out of a long tail. You can also Create users can create collections and our collections are very simple dead easy to do You just select some objects you say make a collection you add it to an existing one You create a new one very very simple straightforward and powerful and useful we think but if you notice down here It says Chicago starry nights. That is in fact an extra murals collection is created by them And when you click on it, we'll take you to extra murals launch it and then you can use their beautiful beautiful browser So we provide a lot of context with shelf life. That's what it's about But the context for works is not just other works the context is people as well and so shelf shelf life is social all the way through social social social So you can for example, you can rate it you can Drop in some stars. You can like it. You can follow it and following establishes a longer-term relationship with the object so that you get notified when for example, there are new reviews or the author has published a new work or Discussion you are in has been added to or maybe Neil deGrasse Tyson is going to show up at your local library and do a book chat So these are all ways in which we have emphasized the social we think that's hugely important There's one more way here though you look these balloons on the right indicate that there's been some social activity around the object You click we drop you in immediately in place where you can comment read Start up a new topic and we've been thinking this is a possibility that perhaps Comments from librarians ought to be highlighted because we want to make sure that we gather as much of what libraries know and as much as Users crowds communities know as well and make it all available So There's much much more. This is just a little taste We encourage you to come to our booth or go online to see the whole thing But we hope that it gives you a sense of where we're going with this this Shelf life library cloud are real they work they scale But this is just the beginning of figuring out what this vision is and that's figuring out that vision It's something that we have to do together and we would love to do together with sprint partners with everybody here but the real opportunity we think for the DP LA is Not simply to connect people to work. So that's hugely that's revolutionary That's amazing. That will change the world But culture doesn't consist of people interacting with works the real opportunity we think for the DP LA is to connect people to people Through works because that's how you create culture. That's how it is created. That's how it Grows that's how it's enhanced. That's how it enters our lives That's how culture comes to matter to us and we think that is fully within the mission and the hope that we've heard today Of the digital public library of America, thank you So our team is is joining us as they come down. Yes a question up here Okay, great. Hi, my name is Laura Parsons and my question is Obviously, you're gonna gather a lot of user data using this. I'm wondering what your plans would be for that Which aspect of plans what we're gonna do with it a privacy issues Yes, both of those things There are plenty of things to do with it You can we have all seen examples know of examples of sites on the web that do amazing things with Knowing what users are doing opt-in opt-in. Let me say that again opt-in From library thing even Amazon does amazing things with it as well But you know, there's lots of reader sites that do things as well Obviously Amazon is not an opt-in example In terms of the policy that we think is determined by the DP LA we tend towards Opting in we default to protecting user privacy any of the circulation data that what we take into library cloud For example is entirely anonymized no identifiers date stamp is only to a day The IDs are entirely randomized so But the policy is up to the PLA Good hard questions buried there David other questions Because library thing knows a huge Library thing which we love and who is one of our partners first of all buying is interesting proposition But leave that aside library thing which we love many of us are users here I'm sure knows a huge amount what use about what users know what their preferences are what they're saying how in fact We're getting social we have social metadata from from them already. So for example in library cloud We know how many times a work is mentioned in the library cloud discussion not who just how many times that's really interesting Library cloud for all of its wonderfulness doesn't know what libraries know it knows what readers know and it's fabulous at that but we want to make sure that none of the Wisdom or the metadata of libraries is left behind It's great Zach Great work. It's incredible. It's amazing. What very well done. Okay. Thank you I love it. I mean I'm working on something similar and I had one suggestion for you Which is consider integrating social ranking with everything? I know you have a recommendation With stars and things like that But for comments and I like your idea of giving special emphasis to librarians comments and you could do something where you You allow readers to Rank up The best comments so that you're only exposed to you know the highest quality things I mean one of the problems that the internet is so chaotic and almost too democratic and so DPLA this effort we need to figure out a way to balance democracy with a kind of curation which librarians are best at It's a wonderful question a huge set of possibilities This is a problem that's been addressed by lots of different sorts of thinking and the key thing I think is to remember that Maybe remembers the wrong word cuz they didn't say this is an open system and especially with library cloud If you want a for example a different Recommendation engine you don't like the way you like you have different preferences Wonderful that is the best thing ever Fabulous well you guys are wonderful and thank you. I know this was a huge team effort as all of them were so Please join me in thanking the whole team


Hello everyone and welcome to the September 2021 end user technology radar. I'm excited to have here with me today the technology radar team, featuring representatives from organizations such as Alliance Direct and Discover Financial Services. And today we're going to have a look through the technology radar that they've produced over the last couple of weeks. So, let me just start by showing my screen. Very quick, quick. Just a bit about myself. My name is Katie Gumanji and currently I am the ecosystem advocate at CNCF. My mission is to make end user successful, but at the same time to bridge the gap between the adopters and the projects within the ecosystem. You can find me on social media such as Twitter and LinkedIn for further questions about today's radar or anything related to the end user community. Now I've mentioned the end user community and I'd like to maybe provide a bit more information of what it actually represents. The end user community is formed of more than 155 organizations that use cloud native technologies to build and distribute their services. These organizations, we have a wide range of them across different sectors and industries featuring startups quite innovative and disruptive and big household names that have been in the industry for many years. The end user community is one of the largest end user open source community out there and extends at the center of CNCF's goal of end user driven open source. One of the main initiatives for the end user community is to provide a bit more insights into how they use cloud native. And this is the purpose of the technology radar. The tick radar is something which intends to showcase the real usage of cloud native. As such, we're going to have a radar composed of three main rings. And within these rings we're going to have adopt, trial and assess levels. Once we actually choose a theme or the technology radar team chooses a theme, we will go to the end user community and ask for their feedback. Based on their feedback and votes, we will categorize tools and frameworks in one of these levels. Adopt pretty much means that this tool is highly recommended by the end user community. They have been using this tool in their production systems and it's proven to be stable and useful. The tools that are categorized as trial is pretty much other tools that have had success with the end user community and they definitely recommend to have a closer look at those. The assess tools, these are pretty much tools that focus on maybe solving very specific problems, very specific and small problems. The end user community did POC or investigated in the past and definitely would recommend for you to look as well. If you face the same problem in your organization. Now before we actually look into the tech radar that we've produced for this quarter, I would like to introduce the tech radar team. Currently we have Sergio and Keith joining us today. Sergio, would you like to introduce yourself please? Of course. Thank you, Katie. My name is Sergio Petan, I'm the head of DevOps at Alliance Direct. I'm responsible for operating and then growing the platform for the whole company. You might know about Alliance, but Alliance Direct has a little bit of different identity. We are seeing it's ourselves as an IT company, a technology company with license for insurance. And you can see that in, I guess, in our technology stack. Awesome. Thank you for joining us today and Keith, would you like to introduce yourself as well? Yeah, thanks, Katie. I'm Keith Nielsen. I am director of cloud architecture at Discover Financial Services. My responsibilities here are sort of focusing on strategies around how we consume public cloud, the services of public cloud in relations to our private cloud. So part of those strategies are obviously the platforms, the services, how we securely deploy our applications and secure the services of public cloud assets that we consume. For those that may not be familiar, Discover Financial Services started out as a credit card company along with as a payment network. So we're around the globe as a payment company, credit card company, also banking, so typical deposits, checking products as well. So I'm happy to be here to talk about the tech radar. Thank you for being here, Keith. As well, I'd like to thank you, the technology radar team for contributing their time over the past couple of weeks to produce this tech radar, and of course contributing their expertise to the wider community. Now, the theme of this technology radar was DevSecOps, and I'd like to ask our tech radar team, why did you choose DevSecOps as the main topic for this radar? Maybe you could link it to some of the challenges you face in your organizations, or maybe some of the tools that you currently use within your organization. Who'd like to start? I could start, because it's something very, very important for us. We are at the moment at the cross check, basically we are now changing the whole CI CD pipeline and really change the other way we deliver code. And while doing that, after a few months of exploring, we finally set up on a technology stack and realized that more than half of the technologies that we're using in the CI CD were actually security oriented. And then we said, okay, let's start a little bit and let's see what we are doing. This is not just DevOps, because like just talking about the DevOps, this is definitely a big uppercase sec as the DevSecOps. And obviously, yeah, it started from there. It started curiosity, I guess, and exploration, and then realization, and then DevSecOps, it's basically everywhere. Yeah, from my perspective, when we look at, you know, public cloud consumption, trying to deploy applications, consume the services, I would say more than half of our time spent discussing designs focuses around security. And so introducing security, obviously it's first of mine for any company, but particularly a financial services company. And so trying to balance the desire to go fast and quickly from our business and their development community has to be balanced against security. So it's a constant sort of balance and discussion about how we secure these things while allowing our business to go fast. And the second part is, is that the rate of change, you know, in this space is probably one of the fastest, right? So if you look at the landscape in the ecosystem, it's, there's new products coming out every day. And so it's a very interesting space, but one that's constantly sort of forcing us to reevaluate sort of our security posture and balancing that against our desire to go fast. So this is actually very good insights because personally, I think, especially in the last year cloud native has had a major focus on security, and we've seen that from tag security as well with the release of security white paper and white paper on the supply supply chain as well. As well in the past, we had a tech radar which was solely focused on secrets management as well. So again, this is an area of hot interest for our end user community. Now, once the tech radar team chooses a topic in this case DevSecOps, what we usually do we go back to our end user community and ask for their feedback. Pretty much would like to understand what kind of tools they use currently that are related to DevSecOps, if they would recommend them to be used by our organizations or they stopped using some of the tools because they overcome the challenges. So we are actually asking for their votes. And currently we had 252 votes from our end users across 35 tools. The tech radar will only showcase a small portion of that that we can showcase within the radar. We had organizations from again from different industries and sectors, majority of them actually categorize themselves as software, which is quite generic to be honest. But if you look into the size of the organizations, most of them come from large companies, meaning that again security is at the forefront for organizations that are operating at scale. So before we look into the finalistic radar, I would like to ask the tech radar team. What were your expectations, when you thought about producing this kind of radar based on your expertise and maybe some interactions with other end users. What did you expect to be the end result of this tech radar. I can go first. I think I was what I was expecting is that we, there would be a sort of a consolidation on on certain capabilities. I think the capabilities are sort of represented but the sheer number of products is a bit staggering. So, I don't know if it was unexpected. I hoped for a little more sort of consolidation but what what we're seeing is a highly, I would say fractured strategy from from different companies and that's represented in the sheer number of tools that were articulated in the survey survey so in one one respect I think it surprised me a little bit but on the other hand it wasn't given sort of what discovers is facing and sort of trying to come up with a cohesive strategy as it pertains to DevSecOps. Sergio, I've seen you've noted a couple of times there. Do you kind of align these kind of expectations. Yeah, I'm 100% aligned with you. It's it's, I guess, I received something else and I was actually expecting. I was expecting to see a different kind of products, different kind of adoption and some of the tools and I was definitely not expecting to see so many new comers on the security table and security space. That's really amazing to see so much so much development going on in that direction. It's we are in need and great needs for better tools and then better education around security. So it's good to see that the community is providing you with so many options. It's hard to keep up with with everything that is being thrown at you right now probably, but it's always better to have options and then have to have them. That's amazing. Well, I think this is a great segue to present. Oh, I think I want to click to present our DevSecOps technology radar from the CNCF in user community. In the radar we showcase 16 tools across three levels. In the adopt, we have Istio, SonarCube, Artifactory, HashiCorpVold, Calico slash Tigera, Terraform, Argo CD and LPA. And as a reminder, the adopt level pretty much are the tools that the end user community definitely did use in the production system. They're stable and they definitely recommend for other organizations to have a look at. In the trial level, we have X-ray, so pretty much these are the tools that again the end users had some success with and would recommend to look at these tools. And within the assess again very well represented here, we have tools such as Silium, Harness, Sonar, Sonotype Nexus, HashiCorp Sentinel, GitHub Actions, LinkerD and Treve. The assess tools currently are the tools that are very promising. They are very good at solving one particular problem and the end user community would recommend to look into these tools if you're facing that problem within your organization. Now, based on the votes that we had from our end user community and the expertise from the TechRadar team, the TechRadar team as well summarized the TechRadar using in free things. So let's go through them in a bit more details. The first one is that the security is the main focus of the DevSecOps, but at the time it's at the expense of the developer experience. Now, Sergio, I know you disagree with this particular thing, but would you maybe share your thoughts why the security is compromised at the expense of developer experience? Actually why the developer experience is compromised at the expense of security? I disagree with the lights where we are actually seeing now the security. It's like you always have to compromise something to get a basic function that is such a security. I could explain the expense of developer experience through the fact that not many organizations maybe realize that they need to hire in the dev of steam, they need to hire some specialties that focus on security. And that means that daily sources are normally focused on developer experience. You take them and you focus them on security. I believe the security is now seeing like the golden age and a big amount of the tools that we're seeing and the options that are currently being developed are giving you the space to finally execute on reaching the security that you are always dreaming as a CISO or as a security expert. I'm saying it more in this life, in a more positive life and maybe a more idealistic, the sake of environment where you can balance the development experience with a security narrative. Steve, what are your thoughts on security and the ratio it has with the developer experience? Yeah, it's definitely impactful. I would say sometimes at the expense of developer experience, our developer community would probably say always, but it's about striking a balance for sure. And I think the tools are getting better. But still, there's not a sort of a cohesive sort of prescriptive way to securely develop for the cloud. And the cloud is so many things to so many people, so it's hard to come up with just one thing. And the landscape is always shifting, which also makes it difficult, right? There's a new attacks, there's, you know, new things that we always have to account for. This is pure overhead from a developer perspective, right? But we sort of are uncompromising in terms of our security posture, the public cloud. So, yes, it's definitely impactful. But I think at the end of the day everybody realizes it just takes one misstep to sort of create an exposure that can impact you for years. So, but I would agree with this for sure. And other any tools currently from the, from our desiccops decorator that you would say that are focusing more on the developer experience and maybe bridging the gap between how can we integrate security, but at the same time make it easier for the engineers or users to implement the security rules. I would say it's working progress. I guess the main focus was delivering on the security province. For example, even when you build a pipeline, then you have to realize that you have different audiences for the final feedback now. The question is, where do you go first with the feedback? Do you go to developer? Does it have the knowledge and expertise to figure security? Or you completely skip him and you go to the CSO and the CSO has a team that focuses on that. So, I guess it's the narrative is just starting to get there. I guess it's going to be a friction at the beginning. And the developers and the architects of the pipeline will have to make some tough decisions. And then maybe indeed we're going to see some sacrifice also being made in the first iteration, but I guess we're getting this one. Yeah, I would think there's not one specific, I don't think the tools themselves make it easier. I think if you look at the public cloud providers and what they're trying to do and sort of create an end to end experience with all of the tooling, I think that's sort of compelling for some companies where you're not stitching individual tools together. But at the end of the day, you have a pipeline and you have to automate all of these things, right? Now, the tools have gotten better, you know, fully rest enabled and things like that, but still you're sort of, unless you're going all in with a public cloud or provider set of tools and pipelines, you're sort of stitching these things together yourself, right? But the tools have definitely gotten better in terms of how you interact with them and sort of the information they give you back from, but there's no simple bullet here. I was actually about to say it seems like there is not one solution that's going to fit all the problems that we have out there. Let's move to our second theme. And it's this one actually mentions that the pace of changing the security space is rapid. Now you've mentioned previously that there are plethora of tools currently covering the DevSecOps space and there is barely to consolidation. Now, do you, why do you think the security space is moving so fast? I would think it's partially because if you look at the public cloud providers, right, you take AWS Google Azure, the major ones, the rate at which they're introducing new services and capabilities to the enterprise is pretty staggering, right? So not only do you have to understand how to consume those, you have to understand how to consume them securely, right? And then factor in technologies like Kubernetes, right, which is sort of an explosion of what's possible and things that are running within Kubernetes, then you just have this multiplicative factor in terms of what you're looking at and things you have to secure. So not only are you looking at end user capabilities, but now you have to secure them. And so the security tools are trying to keep up with the services and how you host them and run them. So it's sort of a never ending, you know, trying to keep up with, you know, what the public cloud providers are doing and what the Kubernetes environment is doing. So I don't see it slowing down anytime soon, right? The public cloud providers are not slowing. So I don't expect the pace of change to slow at all. So what are your thoughts on this one? Would you agree? Yes, I agree with that. I would add a different dimension and that's probably the speed of innovation currently and of digitalization, right? Because now you have most of the companies dropping their own premise, they moved to cloud, they fully go to Kubernetes and the whole Kubernetes environment is evolving so so far. So you just find yourself in a place where the old way of doing security doesn't work anymore the way it used to work. So you are looking for different security, different problems now that you have to deal with. And then of course you have a lot of small companies, a lot of niche companies that come forward and give you the solution for the problem that you're facing at the moment. And then of course a new cycle enters and then a new space is being discovered. And then of course you have to deal with that. I can give you an example is like the check off open source project, right? Once you do the infrastructure as a code and declare everything in some documentation or define it in a YAML file or an adjacent file, then security has a different connotation. You have to look at how you declare everything and you need to find the security in your declaration. So that's a different mindset of doing security. Awesome. And actually here I have a follow up question. So it seems like we have plethora again of tools to cover different small areas where we need to integrate security across different stages. Like, you know, in the stage we develop an application, we deploy it and we actually execute it within our production environments. Now, do you feel like this amount or great amount of tools within the ecosystem? Does this increase the complexity of integrating security or does it simplify? I don't think it simplifies, right? Yeah, go ahead. It's probably complicating it right now in order to simplify it in the future. I would say like that. I mean, now we are like struggling to find the best tools and then to integrate it in what we are doing. So of course we're going to see a lot of options. We are not seeing one single tool that does everything. So that means that you need to first understand the promise of the tool then you need to find an integration in your technology stack. So obviously the complexity is there. It's going to be on the shoulder of few, which ideally they're going to manage to maybe simplify for the whole organization at the end of the day. Yeah, I think it's complicated. What I would suggest is, you know, I constantly say at our company, it's almost more important what we choose not to do than what we do, right? Because there's so much distraction, new tools coming out every day. Vendors that started in one space are now emerging and trying to get into other spaces, right? Which then allows for a consolidation. So it's a constant challenge. But you need to get started and stay focused on sort of your path and get something, you know, the door otherwise you're in this perpetual sort of analysis paralysis. It's very easy in this space. Exactly. At the moment, you might end up with maybe five tools or frameworks added in your technology stack that do the same thing. And then the complexity comes like who does it best. Right. So you need to evaluate the already existing tools and then compare the results. And obviously this also evolves in time. So one that does very good today might not do as well as today in the future. So it's a struggle to find the right balance of relevant and irrelevant and then also to make the optimal decision on every point in time. So it's complex. It's a complex situation right now. It seems like in constant iteration of, you know, increasing that like optimizing the way integrate security, but at the same time, you know, make it as, you know, simple for, you know, engineers to use it for it as well. So let's move to our third thing. And this one specifies that micro segmentation capability is very important, but presents a significant challenge. Now I have a question for Keith because he mentioned micro segmentations first within the desiccants radar so would you be able maybe to explain what you mean by the micro segmentation capability and maybe going to explaining why it's such a challenge at the moment. I think, you know, the tools and, you know, we're trying, I think we're painting a bit of a dark picture. There's great tools that allow you to actually improve your security posture make no mistake about it. This is a prime example so if you look at things like service mesh, right, things represented on the radar like Istio, right, or Calico, you know, from a network micro segmentation. So these things started out years ago, right, and sort of are still trying to get full traction and adoption. They're bumping up against sort of like even you're in the Kubernetes space you're bumping up against sort of the sort of legacy sort of implementations within your own private data centers where you have, you know, these firewalls that sit in the edge. So integrating all of these capabilities within your sort of traditional data centers and being in bringing everybody along for that ride and coming to again a consolidated strategy is challenging. So sometimes it's about the tech, but sometimes it's about the enterprise and sort of what you have traditionally done in changing mindset. So it's not all about difficulty about absorbing tech.  So I think this is one area micro segmentation that we're bumping up against that whether it's, you know, API gateways versus service mesh whether it's edge firewalls versus sort of Kubernetes sort of federated firewall functions with something like Calico. I think there's there's multiple aspects to this, this theme here. What are your thoughts on this one? Nothing to add, as a point here, it's, maybe, maybe, maybe I keep, I can keep that something like definitely when you talk about micro segmentation, especially when you look at the mesh, mesh layer, it's extremely complex to have it on an already existing system because it's it's challenging everything that you have. You have to do security different. Now you have a mutual TLS, now you have a rotation of the certificates automatically. Now you have different policies that need to somehow speak the same language as the other policies that you have in your technology stack. So at one point you need to consolidate on that and go for one single way of specifying the policies. So it comes with with a lot of depth once you decide to have micro segmentation then you need to challenge everything and do it right. Amazing. Well, thank you both for your very insightful thoughts on the DevSecup themes for this technology radar. Now before we actually conclusion this webinar, I would like to ask about your experience of being part of the tech radar team and producing the technology radar. So could you be able to maybe share your thoughts around it, your experience so far? Keith, do you want to take this? Yeah, sure. No, it was fun. I mean, it's always, sometimes you kind of get wrapped up in your own eco chamber, right? You sort of listen to your own thoughts and the thoughts of those immediately around you. So it's nice to step out and see what the community as a whole is doing and hear directly from other people that are trying to solve similar problems, right? Maybe different sectors, but still we're all facing the same thing. So just see the sheer, you know, the survey results and the sheer number of tools makes you feel a little better in terms of, well, you know, maybe the challenges and sort of the overwhelming feelings that I have at times aren't that bad, right? Because everybody's facing them, right? But it was fun. It was fun to go through this exercise with you guys. Yeah, same here. I had a lot of fun and the discoveries are very, very welcome. You don't want to be too original when you do, when you choose your next technology stack. So it's always great to come forward and I'll just present the technology stack and then listen to the others and then listen to some stories about why some technologies are fine, maybe why support it's sometimes much more important than the technology behind it. So these kind of things you'll never get if you stay in your own micro-isolation, right? So you need to expand to the community and then you're definitely going to learn something. For us, the work was really good. We challenged existing technologies that we looked closer to some technologies that don't look that well. So it's a step forward for us. Amazing. Well, personally, I did enjoy creating this decorator as well and I've learned a lot from this process about DevSecOps by collaborating with both of you as well. So thank you very much for giving up your time and expertise. And just before we wrap up, I would like to mention that we have previous editions of our technology radar that you will be able to find at radar.cncf.io and some of the themes that were chosen in the past focus on multi-cluster management, secret management, database storage, observability and continuous delivery. But more importantly, I would like to invite everyone to get involved. If, for example, you have a topic that you'd like to be covered by the end user community, you'll be able to propose it by going tocncf.io for its slash decorator. This is pretty much a GitHub issue, so you'll be able to again propose a topic or you'll be able to upload some of the existing topics as well. Now, if you'd like to contribute to one of our technology radars, you pretty much have to be an end user member, so you'll be able to find all of the information of how to join thecncf end user community by going tocncf.io for its slash end user. And if you have any feedback in regards to today's decorator or any of the previous decorators, please send your feedback tocncf.io. So, again, I'd like to thank you to the decorator team, Sergio and Keith, thank you very much for being here with me today and recording this webinar on the DevSecOps technology radar. Thanks, Katie. Thank you, and see you for the next iteration. Bye. Bye bye.