Well, good morning, everybody, and welcome to a conversation with Professor Michael Sandell, the professor of philosophy who's currently working at Harvard University in the School of Government. Now, it's often said that we live in extraordinary times, and I think the fact that we have a packed room, in fact a completely full room for a conversation with Professor Sandell indicates what extraordinary times we do live in, because there was a time not so recently when the comments of philosophers weren't seen as being so relevant to markets or even government. And yet, as an anthropologist myself, I'm delighted to say that that has now changed. People have woken up and realized that you cannot predict the future or hope to understand the present just by looking at numbers, no matter how unbelievably flashy your algorithms might seem. And in fact, Professor Sandell wrote a terrific book a couple of years ago called What Money Can't Buy. Essentially, it does what it says on the tin, explains the moral limits of markets and all the reasons why building a society and a system purely on worship of markets, worship of numbers has its limits. Now, this has become doubly relevant not just because of the financial crisis, but because we are now living with, if you like, a moment of political revolution as well. So I'd like to start by asking you not so much about finance, but about the big question that everyone's asking right now. What about Trump? I managed to get through a whole two minutes without mentioning Trump, but here we go. Did you predict that Trump would win? In the weeks leading up, I thought he might. I didn't trust the polls and I didn't predict those market predictors either, which... Money can't buy good predictions either? I think not. I think what the elites missed and continue to miss are the sources of the anger and resentment that has led to the populist upheavals in the US, and in many other parts of the world. I think that the elites assume that it's just about anger against immigration and trade, and that at the heart of it is jobs, and surely that's a part of it. But it's also about even bigger things about the loss of community, about disempowerment, and about social esteem, the sense that the work that ordinary people do is no longer honoured and recognized. So let's roll back here a bit, because a lot of the conversations I've had in Washington in the last few months have been all about what it's to do with rising inequality, people bring out these charts showing what's happened to median incomes, you get into these long arguments with different political camps, some people say, well, actually, hasn't anyone noticed that under Obama in the last couple of years incomes have gone up sharply? What are they complaining about? I mean, are you arguing that that basically misses a point it's not just about economics? It's not just about economics. Inequality does matter, but for moral and civic reasons, not only for economic reasons, there are two reasons to worry about inequality. One, the familiar reason, is unfairness to those who lose out. And that can be dealt with, we debate various redistributive policies, but there is another reason to worry about inequality and it's a civic reason. When the gap between rich and poor becomes too great, we find ourselves living separate lives and the public sphere loses its capacity to gather us together and to enable us to be citizens engaged in a common project. Right. I mean, you actually say the conclusion of your book, which I, as a former author myself, I know how important it is to promote the book, so all of you with your webcams, you can see the book, what money can't buy, but you said at the end, at a time of rising inequality, the marketisation of everything means that people of affluence and people of modest means lead increasingly separate lives. We live and work and shop in plain different places, our children go to different schools, you might call it, and I think this is a great line, the skyboxification of American life. It's not good for democracy. Right. So you think that is essentially what lies at the heart of our current political upheaval, is it? Yes. Skyboxification. Do you want me to explain skyboxification? Yes, wonderful word. I just about pronounce it. I have no idea how you translate that into French or German, but anyway. I think it goes back to Aristotle or something, not really. But skyboxification, it goes, when I was a kid growing up, I was a baseball fan and loved to go to baseball games. In retrospect, what I noticed, I didn't notice it at the time, was that going to a baseball game was a class mixing experience. CEOs and mailroom clerks sat more or less side by side. There were more expensive and less expensive seats, but the difference wasn't that great. And it was a community building experience and then came in skyboxes where there were corporate luxury VIP boxes where those on the top could watch, the privilege could watch the game in isolation from the common folk in the stands. And this seems to me a metaphor for what's happened in the passage you read. Throughout our social life, the public schools are less and less a place that create a shared common citizenship where people from different walks of life encounter one another. This seems to me the most corrosive effect of the growing inequality on social life and on democracy. But you could argue that what we're doing here in Davos is the ultimate corporate box, the ultimate skyboxification, not just of a sports match but of life. Well, there is something to that and the question is what can we learn here and I think the lesson of the populist uprising has not yet really been learned and absorbed. I think that these are going to be troubled testing times for democracy these next few years. But what worries me is not just the populist protest, what worries me is that the mainstream parties and the elites and political leaders don't get it. And by that you mean the mainstream Democrats, the Labour Party, the French and German sort of centre left parties too? You think they don't get it too? I think it goes across the political spectrum. I think there is an exhaustion of political ideas and emptiness in the terms of public discourse. What passes for public discourse these days is either partisan shouting matches, ideological food fights on talk radio and cable television and on the floors of Congress, or narrow managerial technocratic talk which inspires no one. And the success of right wing populism usually reflects the failure of liberal or progressive politics and I think that's true today. It's a striking fact that the Democratic Party in the United States, the Labour Party in Britain and social Democratic parties throughout Europe have largely lost contact and credibility with the working class and middle class constituencies that traditionally were their base. Right. Well, I find that very interesting because I went to both the Democrat and the Republican conventions this summer. And the thing that really struck me was that you walked away from the Republican convention, the right wing convention, and you were crystal clear about the slogan, make America great again. Short, snappy, and it has a verb. It's about movement. Very similar to take back control which was the Brexiteers slogan. Again, it has a verb. The Democrats by contrast were a complete mess of images. They had two slogans. One was I'm with her which sounds like a boy band. The other one was stronger together which is very static. You might as well call it I'm with a status quo. If you were employed by the Democrats today, the center left, what would be your slogan? How would you revitalize the center left? Well, I'm not sure I would begin with the slogan, but I would say something about what the snappiness of make America great again that struck you. There is darkness at the edge of that slogan as there is darkness in the slogan, take back our country. Take back control. And yet, despite the darkness, there is also a glimmer of a legitimate and important aspiration in both of those slogans that center left parties need to learn and take seriously. And the aspiration, the legitimate aspiration underlying those slogans has to do with a sense of national community. The language of patriotism has been appropriated by the right for the most part. There is no reason why center left parties can't reclaim and articulate their own conception of national purpose and national community and shared identity and patriotism. So that's one place I would start. So nationalism is okay if you're on the left? Or patriotism is okay? There are versions of national... Is it okay to chant USA, USA or Robertania if you're on the left? Well, I wouldn't encourage, it depends on the settings in which those chants are intoned. And in many of the settings where we hear them, there is a darkness at more than the edge, sometimes right at the heart of the passions they stir. But I think that one of the weaknesses of center left parties has been in the face of pluralism and for the sake of toleration to insist on a kind of non-judgmental value-free politics. And that creates a moral vacuum of void that invariably will be filled by narrow, intolerant moralisms. So part of what I would call for and try to work out for the center left parties is a morally more robust kind of public discourse than the kind to which we're accustomed to technocratic impulse, the economistic faith and the market faith began with Margaret Thatcher and Ronald Reagan in the 80s. But when they passed from the political scene and were succeeded by center left parties in Britain, in the US, in Germany, the underlying premise of that market faith, that markets are the primary instruments for defining and achieving the public good, that faith was never challenged. They moderated but consolidated that market faith. And we've not really had a debate since then about the proper role and reach of markets in civic life. Right. So in a sense your slogan would be there's more to life than markets. Yes. Not very snappy. Well, more to life than markets.  That would be another. But those are critical commentaries. What remains to be done is to fill in the content. And so one element of that content has to do with a sense of national purpose, a sense of community. But I think another ingredient that center left parties need to address is the meaning in the future of work. Right. Part of the anger and frustration goes to social esteem, a sense that the work that ordinary people do isn't valued and isn't rewarded. And I think part of this is due to the, well, the financialization of the economy over the last three or four decades. When the social purpose of finance is to direct capital to socially useful purposes. But when so much of financial activity ceases to have much connection with that basic purpose in the real economy and yet is lavishly rewarded. It's very hard for ordinary people to believe politicians when they say as they constantly now reiterate people who work hard and play by the rules should be able to rise as far as their talents will take them. People listen to that. This slogan is the promise of a more perfect meritocracy. It rings hollow. And yet if there is a theme implicit in the conventions you attended among progressive parties in the US, in Britain and elsewhere, it's that. To remove barriers to a freer, more perfect meritocracy. But that's ringing hollow. You call it meritocratic hubris. You wrote a piece for the Guardian, sadly the Guardian, not the FT, but I'm sure we can change that in the future. Where you basically laid out a four part plan for what you thought were the four key issues that the centre left, if you like, need to address. Do you want to really run through them? For my mind, we need to get granular. What are the four things that they need to actually do in their new party platform when they hire you? Well, we've touched on two, a sense of national community and purpose. So, patriotism is okay? If it's directed to a shared common life, restoring public places, public institutions, class mixing of civic spaces. The second one we've touched on has to do with a serious debate about the future of work and the meaning of work and what confers dignity on work. And the relation between finance and the real economy, that's all part, I think, of the same question. Before you come to the other two, I'm very curious about that as an anthropologist because one of the things that economic anthropology has argued for a long time is that work should not be something that just economists discuss. Right. Because it's anything other than just about economics. Yes, exactly. Work is a way of making a living. We take that phrase for granted. It's about generating an income. But is that its only purpose? Or does it also confirm meaning and identity? And I think clearly the second as well. But if that's true, when people are thrown out of work and when the work they do is not really valued and rewarded, it's not only an economic issue. It's not just a jobs issue as we narrowly conceive it. It's an issue of meaning and identity and respect and social esteem. So those would be two things. So patriotism, talk about work. Well, another, you mentioned the meritocratic hubris that I've spoken about and it's connected. The more we believe that the animating purpose of our society is to enable the talented to get ahead and to remove barriers to advancement. And the more we perfect that meritocratic project, the more we're inclined to view our own success as our own doing is what we deserve. And the effect of this is insidious in subtle ways that accumulate and that I think we're witnessing now. So those on top come to inhale pretty deeply of their own success. I'm here by dint of my own talents and efforts. And that leads to a certain attitude toward those who are not at the top, a tendency to look down with disdain. And part of what we hear again and again in the close reporting of the populist anger and protest is a sense that the elites look down with disdain, the smugness. And I think that ordinary citizens are not wrong to sense this smugness and disdain. It's inscribed almost in the animating implicit ideology or public purpose. And also there's something deeply demoralizing about this picture, about our relation to our success and moral desert, if you wind up on the bottom. Because then you might say, well, the system is rigged, it's unfair. But you might also wonder, maybe I don't work quite as hard, maybe I'm not quite as deserving as those folks on the top. And when you believe both together, that's a volatile, toxic, demoralizing brew. And I think that's animated a lot of this anger. So patriotism is okay, or it is community, pride in community is okay. Well, to articulate a version of national community and patriotism. I'm looking for snappy slogans. Philosophers are not good at that. Work is good, meritocratic hubris is wrong, and the fourth one is about income inequality. It's about income inequality, and it's connected to the defect of the meritocratic ideal as an ideal of a just society in the following way. Well, first we need to talk about inequality, not only as a matter of fairness and distribution, but also as a corrosive, having a corrosive effect on the civic project. We've spoken about that. But there's also the relation between inequality and mobility. In the US especially, we've always told ourselves, this is part almost of the American dream, we don't need to worry about the inequality of income and wealth in the US as much as they do in Europe. Because here in America, we tell ourselves, you can rise, you're not stuck at the place of your birth. Those class-bound societies of Europe, they worry about inequality because they have less mobility. But this comforting story no longer fits the facts. The ability to rise from one generation to the next in the US is less, not greater than it is in many European countries. The US and Britain actually are pretty close on this. Near the bottom in terms of the ability to rise, in the US if you're born in the bottom fifth, 43% will stay there, and 70% won't even reach the middle. 4% only will reach the top. Whereas in places like Denmark and Canada and Germany, it's actually the rate of upward mobility is much greater. So today equality and mobility go together because the further apart the rungs on the ladder become, the harder it is to scramble up those rungs. So the fourth theme that center left parties need to think about and address creatively is, are we going to consider that upward mobility is an alternative to worrying about equality? Or do we need to think about both? And do we need to think not only about mobility but also about creating a more equal society where the focus is not on the scramble to the top? This connects to the meritocratic issue we were discussing. Because even if we could create a society where people rose according to their talents and effort, meritocracy is not a way, not an alternative to inequality. It's a justification for a certain kind of inequality. Even a perfect meritocracy would in some ways deepen rather than solve the problems of social esteem and community. So I think broadly, this is not a slogan, Gillian, but it's a theme. Well maybe the slogan should be, we should all be Danish. Well, you could almost say that the American dream is alive and well and living in Denmark. But here's if I could, here's the theme that I think center left parties should shift their emphasis from talking about mobility and perfecting individual opportunity and talk more about solidarity and community and what that means. So one quite interesting detail that people in the audience may not know is that you grew up in Minnesota with Tom Friedman, who is here in the audience, who recently read another terrific book, Thank You for Being Late, which essentially argues that the secret of a good society lies not so much in Denmark but in Minnesota, or at least in Minnesota 20 or 30 years ago, where you actually didn't have the skyboxification of life as you say. He uses that image in his own book. You did have a sense of community, darn cold for much of the year, but you had a sense of community. Do you think it's possible to go back to those homespun Midwestern Minnesotan values? I mean trying to run a political campaign of let's all be Minnesotans is probably not that punchy, or is punchy may not be that appealing, but is that the answer? I think there are elements, there are values, there are civic resources that are closely connected with that experience, but I think it's important not to conceive the task of rejuvenating public discourse as a project in nostalgia. It's a project that can learn from where we've come, and I think it should begin by looking critically at the last four decades when market thinking and market reasoning and the financialization of the economy, we've come to take them for granted. So looking back 40 years or for that matter over the course of the history of democratic societies, that can inform moral and civic argument and how it might be rejuvenated, not as an exercise in nostalgia, but as a way of critically reflecting on the path we've taken, and it's a way of reminding ourselves that it doesn't need to be so. The laws of the market are not given by nature. We have designed a certain version of global capitalism through political choices. It's not like the weather that we have simply to adapt to. We have made political choices, we should reflect critically on the path we've taken, and we need to listen. This, I think, is the, in a way, the most fundamental piece of advice I would have especially for progressive parties, we need, at least need to do, a better job of listening to the anger, the discontent, the frustrations, the resentments, even when they take sometimes ugly, odious form because there is something to learn. There is embedded in those frustrations. There are legitimate grievances and genuine aspirations that we have not successfully addressed for, well, quite some time, for several decades at least. Have you ever been to a Trump rally? No. Would you like to go to one? Are you inviting me? Let's go together. An anthropologist and philosopher can go off together to a Trump rally. I have, actually, and I've spent quite a lot of the last year trying to be a one-on-one anthropologist and go into different parts of America. And frankly, that journey has reminded me of how easy it is to insulate yourself. But I'm curious, if the left doesn't listen and doesn't take on board what you're saying and doesn't formulate snappy slogans, doesn't find a way to connect to listen, doesn't adopt this four-part plan, what happens? Well, first, I wouldn't describe it as a four-part plan. I know you want snappiness, Jillian. I'm sorry, I'm a journalist. I think about headlines and bullet points. I would describe it as four themes. Okay, four theme plans. That should inform the mission and purpose, the rejuvenation of public discourse by progressive parties. What will happen if we fail? Do we get dystopian demagogues for the next two decades? Well, I think what will happen is, here's how I would think about it, people want public life to be about big things, including shared purposes and meanings and sources of identity. And when for a stretch of time, mainstream parties and progressive parties don't offer that, don't speak to those larger questions, including questions of values, but instead talk a technocratic language. That creates a moral void. And what happens, and we're seeing it already, is the default way of filling the emptiness of meaning in public discourse is a kind of strident nationalism. That's the default source of meaning that will infiltrate and infect public discourse and global politics and its corrosive, ultimately, of pluralism and democracy if central left parties don't find a constructive way of moving beyond technocratic talk of addressing values, big questions about justice, equality and inequality, what it means to be a citizen, a language of the common good. In the absence of that, that void will be filled, it's already being filled in democracies and also in undemocratic countries. A void of meaning is being filled in different ways, in different parts of the world, by a strident nationalism. That's the default source of meaning and it's dangerous. So, in some ways, we are sliding back towards the 1930s. Well, there are elements of that. There are elements of that under different conditions, and the challenge is to find is not to flee from a politics that addresses common purposes, national purposes and aspirations, and moral questions even. It's to try to find our way to a morally more robust public discourse than the kind to which we're accustomed.  Well, we have about one minute left, so I'm just curious. Any leaders on the political stage today, do you see any of them doing that, capable of doing that on the mid-to-center left? No, Barack Obama in 2008 seemed to be someone who was able to touch moral and to connect moral and spiritual concerns to politics, but he wasn't able to translate that into governing, and I think that has to do with the way he responded to the 2008 financial crisis in the bailout, which generated the anger and resentment, which he placated rather than articulated, and that unaddressed anger fed on the left, it fed the Occupy movement, and then the Bernie Sanders campaign, and on the right it fed the Tea Party movement and Donald Trump. Well, I don't want to end on a down note. It's been a fascinating conversation, so instead I'm going to simply end with a quarter of arms. Who in the room would regard themselves as being centre or centre left? Yes, quite a lot of you. Well, I'd simply end by saying, we don't often turn to philosophers for political slogans. I'm not sure we've quite got one today. Themes, Gillian, not slogans. Themes or slogans, those of you who do come from the centre or centre left, you now have your marching orders. Thank you. Thank you, Gillian.


Hello, everyone. Good morning, afternoon, good evening, depending on where you are in the world. On behalf of my colleagues here at CloudSmith and ChainGuard, I want to welcome you to our discussion entitled Everything You Wanted to Know about securing the software supply chain, but we're afraid to ask. So this is going to be an open discussion, a conversation, so feel free to hop in with questions if you want to. We do have a Q&A section specifically in Zoom for that. So it helps if you throw your questions in there because it will, definitely, everybody will get the benefit of the answer if we answer it on the call. In addition to that, of course, we're all here, and I want to welcome you, but I think over the last few years, software supply chain security specifically has become top of mind as one of these concerns for almost every organization that's producing and consuming software. So we as a tech community have to try and understand this, but if we're going to wrap our head around it, we really need to have conversations like these to improve our collective understanding of where we came from and where we're going. So hopefully by the end of this talk, well, we'll leave with not just a bunch of fear, but hopefully a little hope as well about the future of supply chain security and software. So let's go through the agenda real quick here. I will first off just do a quick little speaker introduction for all of us. Everybody will do their own direction, and then we'll give you a little background on what we do as organizations with CloudSmith and ChainGuard as well. And then we'll get into the meat of the talk where we're talking about some of the things that we want to bring up today as conversations. And again, this will be open, sort of like a fireside chat. We're just discussing stuff. And so feel free to ask questions as well. And all my speakers feel free to hop in at any point. We'll be touching on points like what got us here, specifically when we talk about previous tipping point events, attacks and other things that precipitated our big concern about the software supply chain. And then we'll touch on why this is such a hard problem to solve because there are definitely some challenges there. Each attack and each different compromise or issue that comes up really has different origins and different techniques and methods. So it can be quite tricky, sort of a moving target. And then we'll talk about specifically what's being done, where we'll highlight a few open source projects in this space, specifically around efforts that Dan Lawrence has done over at ChainGuard, but also it is time at Google and sort of with open source. So thanks for being here with us Dan. And then also we'll specifically talk a little bit at the end about what are we doing as organizations like CloudSmith and ChainGuard and also how you as an organization can help and contribute to the same efforts. Sound good? Okay, well without further ado, let's introduce ourselves a little bit. So I'm Adil Lagari. I'm a solution architect manager here at CloudSmith. I'm a sysadmin turned solution architect turned manager. I spent the better part of two decades here wrangling Windows boxes and Linux boxes. So over the past five or so years, I've become an active member of the PowerShell and DevOps automation community. And I'm a speaker and author and super passionate about PowerShell DevOps and automation. Over to you, Paddy. Hey everyone, I'm Paddy Kerri. I'm an engineer here at CloudSmith. And I've been an engineer for a long time starting in ad tech and moving into e-commerce and finance and eventually found myself deep in package manager land, which is how I ended up here at CloudSmith. And I've worked on everything here from core package management functionality, securing supply chains. And now our platform team, part of the responsibility there is securing our own supply chain. So hopefully that's some relevant experience for what we're going to talk about today. Thanks, Paddy. And yes, hi everyone. Yes, I'm Dan McKinney and I work in developer relations at CloudSmith and I've had a somewhat torturous path through to developer relations. So I've worked in systems engineering and manufacturing actually. So another type of supply chain as well. And yes, it's really my role to sort of engage with the community and educate the community why this is why this is a perfect forum to do this today. I'm also an absolutely crazy retro computing and video game collector. And I work as a DJ at the weekends. So that's a regular spot every Friday, Saturday night, every Friday and Saturday night, which is very different from supply chains. But yes, it brings a little bit of color to the proceedings. So nice. Thanks, Dan. Dan. Second Dan today. My name is Dan Lawrence. And yeah, currently I'm a founder and CEO at a new startup called ChainGuard. But before that I was at Google for about nine years where I got to spend the last five or six working almost completely in open source and started worrying about open source supply chains. So I've been involved in a bunch of different open source projects there and continue to be in products like Sigstore and the Salsa projects or SLSA. So we have to talk about some of those later today. Absolutely, Dan. Thank you. And thank you for on behalf of all of us. Thank you for your efforts with open source and keeping things open for us. Awesome. Okay. Well, let's talk a little bit about each other's organizations about CloudSmith. I'm going to let Dan McKinney complete this one. Yeah. Thanks, Adele. So CloudSmith is a cloud-native package management solution. So given how essential software artifacts and packages in general are to a software supply chain, it's very high on our agenda to secure that supply chain. So we have been around a lot longer than a lot of people think if they're only hearing of CloudSmith. We've been around now. This will be our sixth year. So it's a very exciting time for us because as you said earlier, Adele, you know, secure software supply chains are really at the forefront of people's minds at the moment. And it's so close to our hearts as well at CloudSmith. So it's a really exciting time to be part of CloudSmith. So that's who we are. I love this. I can Dan again today. Yeah. So yeah, we've not been around for very long. We've been around for about five months at ChainGuard here. We got started in October of 2021 and we were trying to build a bunch of tooling based in open source to help people make secure, make their software supply chain secure by default. We've seen time and time again that if you make developers do things differently, if you make them change the way they work, and then security is always an afterthought. It's not what you're thinking about when you're in that development loop. But we believe that if we do this the right way and if we take a couple steps back and think about the problem a little bit, we can actually make secure software development a better and easier way and have it work that way by default. So that's what we're focused on doing. We're still just getting started. So thanks a lot for having us here and giving me some time to talk to everybody. Yeah, for sure, Dan. And I must say like you're being very humble.  So good for you over there. We eagerly await your announcements soon. All right, so let's hop in a little bit. And so now what I'm going to do is I'm going to take the slides off the screen because I think this is a discussion. So let's open it up a little bit and talk about what got us here. And when we specifically say what got us here, of course the scary elephant in the room is all the attacks and everything that went on. But I think there are a couple of events that will probably highlight that have helped shape the industry as far as software supply chain so much. So and I think one of the first ones, of course, is the SolarWinds attack, right? So that was December 2020. SolarWinds, if you don't know, you know, it was a very prominent active product company. They make software for network and infrastructure management. And so in December 2020, they announced that their Orion software had been compromised as a result of a software supply chain attack.  Because everybody now knows that word, that phrase specifically. So because this malicious code was inserted into the Orion software early enough in that supply chain, it remained undetected, right? So what happened was it got pushed out with the other updates to the Orion software and slowly got out to consumers. And, you know, we're not talking about small consumers here. We're talking about big folks like Microsoft and Intel, etc.  And so that's really serving the highlight part of our problem that these pipelines, we're largely up into this point as developers, we're largely ignoring these parts of the pipeline. And this is sort of a tipping point event for that sort of alertness. Now, now, Dan Lawrence, I know I've heard you talk about this before, but I think you were still in Google working on some open source projects at this point, right? And this is sort of one of those moments, those tipping points? Yeah. So this is kind of right after I switched from Google Cloud to Google's kind of core security group. So I'd finally been successful in convincing some people at Google that this was a big enough risk to take seriously as like an existential issue for the internet and for Google. And it was still kind of slow, though, until, you know, the kind of news broke about this attack and the amount of people affected and everything. So almost overnight we went from why does no one listening to us to why haven't we been doing this for years? And that was quickly followed up with some of the stuff I think you're going to chat about. And I'm in it with, you know, the U.S. government looking around too and executive orders and new regulations and new compliance and everything. So yeah, it's becoming an increasingly hot topic over the last couple of years, unfortunately, because of attacks and not because we were, as an industry, ahead of the game. But it tends to work that way, unfortunately. Yeah, it's not a problem until it's everybody's problem, right? Exactly. Yeah. So that, that, yeah, no, that definitely was one of those moments. I think as for everyone in the industry, I was working at the time, I believe at the chocolate software. And it was, you know, we kept on getting that question again and again, where suddenly folks were like, Hey, you know, we need to secure our software supply chains. And I'm like, Okay, that's great. I mean, we kind of been preaching this stuff for a little bit, but now suddenly we're getting all of this detention, right? So I think, I think a lot of folks perked up their ears here and, and sort of your, your C level staff, the higher folks who are normally not as concerned, suddenly started tuning in and asking these questions. Yeah, I've seen pictures of board decks now with this on a slide from every CISO out there. So it's something everybody's trying to rapidly come up with an answer to. I know, I know, over the past year, everyone's learned the acronym SPOM and it's kind of funny because it's like, really, we've been talking about this for a while. But yeah, no, very, very true. So, so I think, yeah, leading on to that timeline wise, right? So in February 2021, I think was one of these announcements where like, where Alex Burson, who's a security researcher, showed that he was able to compromise supply chains as well. And he was luckily a white hat hacker. So we didn't have the benefit of stress there, or sorry, the negative effects of it. But of course, they've been copied and taxed since then, right? And I want to highlight this one specifically because I think he did this basically by, you know, sort of subverting the logic in the actual process of the normal supply chain, right? So as developers, a lot of times we want to use dependence, right? Obviously, we don't want to reinvent the wheel every time. And so this is specifically referred to as dependency confusion, right? And it's similar to sort of typosplotting as another type of attack where, you know, essentially what he introduced public packages into widely used public registries that shared the name of private packages for a lot of big organizations. And we're talking 35 different companies, we're talking names like Microsoft, Apple, PayPal, Tesla, you know, like it wasn't a small attack, but basically he was able to very simply do that by creating packages on public registries with similar names. So it highlighted another thing where it's the functionality of the package registries and the package management systems by default are to trust the public registries and pull those down. But, you know, it's the whole who watches the watchmen, right? At that point, we're not sure. I mean, everybody had to overnight kind of figure out, wait a minute, we have to actually change the logic of how we do this stuff, right? Dan, I think McKinney, you were mentioning something. Yeah, yeah. I mean, I think the real problem until there was that, I mean, essentially this was a problem with like configuration, like the way people had configured their package manager clients to, you know, be able to hit public sources like in preference to their private internal repositories. But there was no red flags, like not, you know, it went ahead. It wasn't that it was, you know, in contrast to Solar Wins, which was a breach of their build system. This was normally running build systems that just were, you know, configured incorrectly, but didn't shout about it when something unexpected happened. So because pipelines are so automated now that that's going, that's what enabled that, you know, the automation kicked in packages get pulled in, things get built. And there was no point where something shouted to everybody. And why is, why was there, why was there no point when that happened? Well, it's kind of like Dan said earlier that it's not the default and it's not easy enough. It's not easy enough to have that default stance of, of the, you know, something unexpected or there's a problem or there's something we suspect may be a problem and maybe we should have eyes on this, the heavy automation sort of drove that. And look, I'm a huge fan of automation.  And that's fantastic as an industry, but it has to come with checks and balances because, you know, automation run amuck will lead to these kinds of problems. And that's, that's I think a core difference between dependency confusion and the Solar Wins attack. Both were supply chain breaches, right? And both resulted in malicious or potentially malicious software getting out into the hands of users. But there's a subtle difference there, you know, so, but yeah, the lack of visibility on that process and the lack of alerting and automatic red flags, there is what really caused the problem. Okay, so this is great because I'm going to pick on you a little bit. So I love to. So we have a Q&A question. So while you're at this talking about these specific software supply chain, Musa asks, starting from the basics, can you please define what is meant by software supply chain? So defining our terms, of course, where everybody wants to be on the same table. Okay, and I'll have a go with that then. Yes, I'll have a go with that. And well, look, I'm a big fan of doing this as well. So apologies to everybody, but I love to draw analogies, right? So and I worked for five years in manufacturing. There was a silicon wafer and fab actually. So I was very familiar with the term a bill of materials, right? And what goes into a final built product, that's very easy to see in manufacturing, right?  And at the other end comes a finished product. That's the same for software supply chains, right? You have raw materials that you ingest almost and those are the open source libraries and projects that you're building upon and you're using as components in your finished product. And unlike the manufacturing industry, the software industry, it sort of hasn't reached that level of maturity yet the manufacturing has. So for manufacturing, for raw materials and for supplies that you bring in, there are certificates of conformance. Everything goes through a quality analysis. You look at a company like Apple, they audit their suppliers heavily, right back to the mine that the the rare earth metals were mined out of and they have all of that traceability on the supply chain from raw materials right through to the finished product that you hold in your hand. And as an industry, the software industry kind of needs to get there. Now, a problem with that is that it depends so heavily on open source. Everybody uses open source and open source is fantastic. We're a great supporter of open source at CloudSmith. So we don't have those centralized industry bodies that do quality control and conformance checks like manufacturing does, which is exactly why we need what Dan said. For the software industry, we need zero trust because we don't have those centralized bodies and organizations to prove provenance of an open source library or to give that traceability all the way back to the source. So that's what a soft source supply chain is. It's everything that goes into making up your product and of course, dependency of dependencies of dependencies. It's just like the raw materials in a physical supply chain where a screw can be traced back all the way through the metal that was used to make it all the way back to the mine that the metal came from. And that's like the chain of dependencies and so on. Exactly. And we'll save some of the the problems with it for the later parts of this conversation because there are natural points that we can discuss that too. So but I mean like just summarized in a lot of what you're saying, which was awesome is the idea that, I mean, it's kind of around this bill of materials that people keep talking about software bill of materials as bomb stands for software bill of materials. And you're not you're not only trying to understand where code comes from, but it's sort of each step in that process. It's like what dependencies did I use, what image was this built on, what factory, if you want to say that, what repository was used, what commit specifically added this code and all these sorts of details. Kind of like, I know Adolfo from J&Guard says that sometimes they said for that packing slip on the side of the UPS package that details, this was checked by this person at this point went through these waypoints and came from this factory and all those details. So that's kind of where we're getting at as a software supply chain for. And a bill as well, even the environments, even the tooling that was used to build an artifact, that's part of that as well. It's the complete bill of materials, not just the raw materials themselves, but the environment that they went through, the versions of the tunes, even down to the build host that was used for a particular pipeline run. It's all relevant. Absolutely. So, okay, the last date I'm going to highlight real quick is May 12th, 2021. And the office of the president, of course, on this date issued executive order 1-4-0-2-8. There won't be a quiz later, so you don't need to know. But it was on improving the nation's cyber security specifically. And so this really highlighted that this was a real issue that the government was willing to throw its weight behind. And they really talked about removing barriers to the sharing of threat information and modernizing some of these practices in cybersecurity that had kind of fallen by the wayside. But not only that, actually establishing government run cyber safety review boards and details like that. So I think this was a big point for the industry for sure, Dan Lawrence. And I think you definitely were involved in some of this or your projects were referred to in some of this, right? Yeah, yeah, no, it was a kind of fast couple months from the attack to kind of the analysis mitigations covered from the breach earlier or later the year before out to getting this executive order out. The executive order, though, was kind of just the start of that road to the executive order, a directive on how to produce software. It was a directive to a bunch of other agencies within the government to start collecting industry feedback and use that industry feedback to shape some of recommendations that then get applied and purchasing decisions and that kind of thing. So we're still seeing kind of that process unfold. And actually two or three weeks ago, NIST, the National Institute for Standards and Biology in the US, finally published the not final version, they're going to keep iterating on it, but the, you know, a final version of the SSDF or the Secure Software Development Framework, which is really a great read, if you haven't seen it yet. But this is kind of the culmination of talking to people in the industry, people in government, people in the public sector, private sector, and soliciting what the best practices were today and then collating all of those into a checklist that you can run through and do an almost like a self-assessment for your organization. I'd highly recommend taking a look through it if you haven't yet. It's broken down into a couple different sessions. We've got something on the ChainGuard blog about it, too. But this is basically, you know, the industry set of best practices here for both consuming secure dependencies and then also operating build systems securely in your supply chain. There's some stuff in there that I think will surprise most people because, you know, there's a lot of stuff which is basics, you know, brush your teeth style hygiene that everybody knows they need to do. And then there's a couple things in there that are kind of pushing the envelope a bit. And I think we're going to catch people by surprise later this year if you don't get a handle on it now. What, do you want to mention any of those? Oh, man. Yeah, well, I mean, there's one funny one that I'm curious to see how it plays out. But it requires, you know, vendors to try to comply with the SSDF to also apply that recursively to their vendors, which makes perfect sense, because this is a supply chain. But yeah, it's going to be very slow until the first person tries to meet this. And then all of a sudden, it's going to, you know, work across the industry, because we're all in a supply chain together, most of us are somewhere in the middle, right? So it's not long before it starts creeping up on people. That's the tough part, right? Is it's provenance all the way down, right? You really need to get folks to buy into it. So I think if once large organizations start hopping on that bandwagon, I think the industry, you know, just by sheer momentum alone, it's going to be forced to to comply to a lot of this stuff. So as organizations, what you're going to see is if you're selling products and software, you're going to start seeing more and more security assessment, you're going to start seeing more and more from your customers and consumers wanting details about where, you know, the entire chain of trust all the way down to your dependency, even if they are open source, right? So I think not necessarily having to reveal the entire sausage and how it's made, but at least all of the different components that went into it. One of my other favorite parts, though, not, you know, easy or hard, just I love seeing it called out and you reminded me just talking about open source. The guidance kind of doesn't require, but it recommends reusing well tested, well developed components, including open source. Exactly. Right. We're talking about open source security. There's a lot of misconceptions that open source is worse than closed source or anything like that. And, you know, it's not open source software. It's just software. It's the same as closed source software. All software has bugs. And I think by and large, most people agree that, you know, in general, open source software is or can be more secure than proprietary software. It doesn't mean every single library on GitHub is, but I think on average, the stuff you're using, you're in a much better spot if you're using well tested, well reviewed, well maintained open source libraries. And so the business framework actually calls that out and recommends it as a best practice rather than everybody trying to rip all the open source out of their supply chain. That would be a mess and set things back even farther. So you bring up a really great point, Dan. And this is one of my little bug bears, too, is I keep hearing this in the industry. A lot of folks, and I've heard organizations even on sales calls or other places talk about you can be as secure as you want as an organization, but if you're consuming open source packages, you could be vulnerable. And I just feel like that's a little bit fear mongering, right? At the end of the day, it's really that the open source code is in every supply chain. Let's be real, right? There's pieces of it everywhere. And that's the tried and tested one. So why aren't we backing that as an industry and supporting those efforts more so? Because that's where we all, you know, collectively, as humans benefit a little bit more from the improvements and that, right, Dan? Dan and Dan? Absolutely. Absolutely. I couldn't agree more, really. In fact, I do think that frequently, you know, open source software effectively has more eyes on, you know, in the process of its development, you know, by its very nature, it's open to anyone to sort of audit and check. So, yeah, no, I totally agree with that. And I know even, well, Patty probably knows better than me, but, you know, within CloudSmith, we, of course, use open source software to build CloudSmith and we open source as much of our own stuff as we can and we contribute where we can. So, you know, I think that's something that we would definitely agree with. Patty, do you want to say anything better? I guess CloudSmith, like a lot of other companies, wouldn't exist for open source. You know, being able to trust that software, being able to trust the things we're building on top of is crucial. You know, it's also crucial for our customers. Our customers need to be able to trust that our supply chain is secure in the same way we need to be able to trust that the supply chains of our providers, whether it's cloud providers and even their hardware suppliers, it has to be secure the whole way down. And I guess we'll get there eventually, right? You know, it's going on a long, hard road to get there. But, you know, so long as we're in the right direction. That's exactly right. I mean, this is a journey, right? And we're all sort of on together and, you know, come reunified by our use of open source. And there is no, there is no like quick fix, you know, one size fits all solution here, but it's about figuring out what the bits of that solution look like, you know, and nobody can do it on their own. And I think that's where, then, Lawrence, you also mentioned, you know, this is security by default, right? There's no quick wins, but we need to make it easier and simpler for folks to be able to consume security as a part of the supply chain, right? We don't want it to be like this thing, like where it's like an afterthought or so. So for those speaking of journeys, for those who's following along with the actual agenda, we are now into the why is this a hard problem to solve, right? So I think a lot of this is, it comes from the challenge, like Kenny, you mentioned before about software supply chains and pipelines, right? We have complex pipelines now. So the more that we bring automation into it, which we want, we are also, you know, introducing probably possibly a little less eyes on some of our code with the automations and pipelines that kick off, and also some complex dependency chains, right? We depend on a software, which depends on software, depends on software all the way down. So that's where I think this can be a tricky, tricky problem, right? Yeah, absolutely can be. And you know, we've all seen this, you can pull in one dependency that maybe has a hundred dependencies of its own, maybe more sometimes it gets on pretty ridiculous at stages. And that's only one part of it really, you know, it's a hard problem on many fronts, it really is. Yeah, and I think on top of that, like, in addition to that, I mean, of course, every attack that we're about, right? SolarWinds to stay alive for sure, all of these ones that are out. So I think it was, was it the DevOps security report that mentioned that I think attacks have been up this past year by 600%. So there's definitely a lot of exploitation going out there in the software supply chain. And, you know, historically, Dan Lawrence, a lot of the stuff that's been put out there, I mean, I know I've worked on code before where I've tried to sign things, and different packages or even different pieces of code in PowerShell where it's historically very difficult to sign, get a signing key, or be able to sign anything, you know, and you either have to pay a lot of money or try and try and jump through a lot of hoops with the node servers and, you know, like signing mechanisms. So I think it's safe to say that these mechanisms used to be a little bit clunky, right? Yeah, absolutely. I think, you know, not for any real hard reason or anything, there just wasn't demand for it really, I think. And it takes, it's like a signing, it's like this, if you sign a package in the forest and nobody verified it, did you really sign it at all? So it's complicated to get started. Like, if nobody's asking you for signatures or anything like that, then you're not going to do them. And if nobody is used to asking, then nobody's going to provide them. And you kind of get stuck in the cycle. So you have to kind of break out of that somehow. And I think, yeah, the attacks we've seen have kind of proven the need for it. And so we're seeing some pretty rapid adoption in SigStore and a lot of these other projects. Just because of that now, everybody, it's aware of people are aware of it. Software producers are trying to do the right thing. And software consumers are trying to do the right thing now too. So it's kind of a great time to be in the field, trying to build tools to make that easier. Yeah, no, I agree. It's funny because I liken it to, and this is probably a poor analogy, but it was sort of like a Rube Goldberg game, sort of contraption to try and to sign things. You had to make sure a server was available to make sure you could sign things. And historically with co-signing, I know back when we were at Charlie, it's sort of you have to pay for some of these signing certificates. So I think this is good because the conversation is just naturally flowing this way. But here, let's talk about what's being done. Let's talk about some of these big open source projects. So I think a natural starting point to this is SigStore. Do you want to reference that, Dan? Yeah, sure. I can tell a quick story there. We're studying kind of co-signing and why it wasn't being adopted in different ecosystems for a while within the OpenSSF and at Google. And we saw kind of a similar thing play out across all the ecosystems where the tooling wasn't great, people proposed adding signatures to Python or NPM or Ruby or something like that. And it's just really complex when you start thinking about key distribution and who's going to verify what and what does the signature even mean. And these conversations just kind of stalled out because there wasn't much motivation, I think. But then on the flip side, there were all these kind of walled gardens and other commercial ecosystems were signing was happening all the time. If you publish an Android app or Windows drivers or something like that, you need to sign them. And so in those worlds, 100% of everything you sign is because you had to. And the way those kind of tool chains worked and everything was with kind of traditional certificate authority hierarchies, the same way TLS works in your browser. Or open source alternatives were most commonly using something like PGP and Web of Trust, which was kind of finicky and hard to get right. And so when we started looking into the commercial code signing stuff, it just kind of gave us all these flashbacks to what the internet used to be like, you know, five, six, eight years ago before Let's Encrypt came along. If you remember trying to set up like an Apache server or something and get the, you know, TLS, you'd have the green shield next to the green shield. Yeah, the green shield in your browser rather than the red X, right? You know, you'd have to find a CAA, you'd have to pay them, you know, a couple hundred dollars, they do some verification, you'd have to send them something on official letterhead for your company, right? Always kind of meeting the scoops and then email you something. Logo used to be in the corner of web pages, like the GeoTrust or Web of Trust. Yeah, encrypted. I can't just put that there. That's not how that works. But yeah. And so, yeah, you pay and they'd email you the certificate and you copied into the right directory and rename it and run these arcane commands. And if you got it all right, then you get the green shield and then you probably forgot all about it and a year later, the certificate expired and you'd have to remember how to do the whole thing again. Because everybody was complaining, right? And you forgot. Yeah. And then Let's Encrypt came along and kind of changed it completely, right? They set up a free certificate authority as a public betterment group and got accepted by the browsers and did all of this by automating everything, right? So instead of certificates being valid for a year with these kind of intrusive checks on your identity, they made a whole new standard called Acme where you could prove that you had control of a domain and, you know, the certificates were only good for a couple of weeks. So if something bad happened, it would automatically expire and they do these continuous checks. They got this going and then within a couple of years, it went from like 50% adoption of TLS out to like 99% of the web now is encrypted. And so they show that, yeah, people do really want to do the right thing if you put it in their tooling and you don't have to think about it anymore.  You don't have to worry about anything else anymore. So kind of a one line change versus what people had to do before. So we kind of tried to copy that model with Sigstore. So instead of having people buy things, manage keys, worry about losing them, we set up a free certificate authority. And this is all backed by transparency logs and some new technology we also copied from the web ecosystem. And now you don't have to worry about keys and you don't have to worry about, you know, validating each other's stuff in person. And you can just kind of get them and sign things on the fly and do it automatically inside of GitHub. And so we're seeing a huge amount of adoption in there, which is great because it actually gives us a way to kind of trace packages back to where they came from with cryptographic proofs and let people sign things with email addresses instead of having to, you know, keep things on smart cards and everything like field days. Yeah, for sure. I think Sigstore was a big turning point, right? And that's sort of one of the big projects that are coming out of this. And do you want to highlight coast sign at all? Because I think that's a big one for folks with Docker images. Sure. Yeah, we, you know, Sigstore supports all sorts of artifact types you can sign firm, where you can sign, you know, executables, drawers, all this stuff. But one of the first ones we started with was containers and the overall OCI ecosystem. Because yeah, there's all these language package managers. There's also been this movement to start using OCI registries to start storing all these other things with efforts like, you know, the artifacts project and stuff. So we said, hey, if we start here and more people start moving things into these registries, it's kind of a double win. Plus containers, you know, we're kind of critical. They're critical to most production infrastructure and they're kind of the last mile anyway. So the coast sign tool there is focused on container signing. But you could also use it to sign things like WASM and all these other ecosystems that have developed around storing other things in container registries. So coast sign is to github.com slash sigstore slash coast sign. You can download it. It's in most package managers and just kind of get started signing right away, whether it's on your laptop or in a github action pipeline or in whatever CICD system you happen to be using. Awesome. Actually, I'm sure in the next bit about what we are individually sort of doing to help this, I'm sure Pali has something to say about coast sign because it's on our agenda. So yeah, but no, I love the analogy there between like sigstore and let's encrypt because I'm old enough as well to remember the silly dance that you had to do to get a certificate and set it up. And I couldn't believe the first time I used let's encrypt, how simple it was. I mean, it was awesome. The DNS verification and the next thing everything was working. And I remember having a thought at that point that yeah, this is the way it should be. It should be this straightforward and that's where we need to get to with artifacts and packages. I mean, it absolutely is. And look, we already support on CloudSmith for artifacts that are stored in the CloudSmith repository, we already support signing them with the GPG key and things like that. But as Dan said again, that's all well and good. All the artifacts are signed, but it's when people come to consume them. Are they bothering to verify? Is anybody even looking at it? That's the other side of it too. Also, is it easy to do that? That's what I mean. It's all down to ease of use. And you make it so easy that it's automatic that people, why wouldn't you do it then is the question. Because it just, you make it difficult to not do, right? Rather than difficult to do and then people have to go out of their way to not do it. And we'll talk more about that in the next bit anyway. I mean, there's no official agenda, by the way. So for the record, we can flow back and forth. So I very much think we're now in what are we doing specifically as organizations to help and what every organization can do to support these efforts. So Patty, Matt, 30, please. I just wanted to add to the previous example that the thing that always sticks out in my head of how awkward things used to be. As a teenager, I was getting into politics, figured out how to build deviant packages, figured out how to sign them with my own DPG key, but nobody trusted them. And I remember I was 17, I had to travel 90 miles to a key signing party, show my government ID to everyone else in the group. And then they had to verify that I looked like my ID. And then maybe they would trust me, maybe enough people would trust me that suddenly my signatures would be signed or verified. And it was just this horrible, long-winded process. And I remember once, someone I knew personally for three or four years to time, refused to sign my GPG key because I didn't look like my ID. Some of these open source keys were really strict in how this stuff works. You were in a posture.  Yeah, it was just a difficult process. And by the time it was all done, I had no appetite to continue doing it anymore. Exactly. Yeah, exactly. That's a fantastic story, by the way. And you know, I've worked with Patty for a couple of years. I've never heard that story. That's awesome. That's a really good story. How do we know this is Patty now? Now I'm going to know. Have you got your ID? Can we see your ID, please? No, that's it. That's a great example. That's an awesome example. And I think this is also part of this whole thing about how not only us specifically as Cloudsmith and Shingar, but how all organizations can help these efforts. And I think a big part of this is now with SIGStore and Cosine gained popularity, Dan. And a lot of these languages now and package registries coming on board as well. We talked about Shopify's great open source efforts with RubyChance and stuff the work they're doing on there. I'd really want to give kudos to those folks because they're leading the charge. And you know, PyPy, Python folks, and then Maven Central over there. I mean, there's just a lot of like, I think there's, we've hit that, we've hit that like sort of, I keep referring to it as a tipping point, but it's sort of like momentum, right? We're pushing forward. Feels like a ball rolling down a hill and it's just going too fast to stop now. Yeah, now it's, now we've got the momentum. And the thing is that that is what we need as well. We need to have that momentum because of course, the last thing that anybody wants here is for these, and Dan mentioned it earlier with wall gardens and things. The last thing we want is for, you know, more proprietary stuff to come along. It will never get by in an adoption. We need the projects to be open, the tooling to be open. And of course, as vendors, I mean, CloudSmith is a vendor, a software vendor, we provide a service. But what we need to do is support those projects, integrate them with our offering. It's fine to be a vendor that builds value on top of those things in terms of analytics and reporting and all that's superb. But the core, you know, the core of it, Sigstore and Cosine, that needs to be widely adopted and to be widely adopted, it needs to be open, right? And everybody needs to have buy into it because we're very anti-vendor lock-in at CloudSmith anyway, you know? I mean, we are. We really are. For something as important as this, it needs to be open so that all the vendors can jump on board with it. And that's what's going to give it the adoption, right? You know, that's what's going to make it that secure by default, really, you know? This is very much my soapbox share at CloudSmith, for those who know me. I think a big part of what I see is this sort of shared ecosystem of Providence, right? And so that folks like, you know, our colleagues over at Sonotype with Made in Central or, you know, the folks at Shopify with the Ruby Jam stuff, this is the stuff I love to highlight because it's vendors, you know, that are proprietary and they're doing their own stuff, but they're working in the open source spaces, right? And they're developing sort of out loud and being open and being transparent about some of these, you know, different tooling and different organization, different projects that are coming through open source. And they're putting their weight behind that, right? And I think this is a challenge, right? I think a lot of folks, like Dan said, McKinney, you can build value on top of this stuff and I understand, you know, as an organization, you're beholden to your stakeholders, you have to make money, absolutely. But I mean, I'll be honest, I'm going to call out our industry a little bit here and say, I feel like, you know, the security by default thing isn't always adopted, right? It's often an additional feature that you have to buy on top. So I want to say that our commitment here at CloudSmith and something I'm going to continue to push for internally is that the mechanisms for security exist for the free plans, for the open source plans, you know, the communities that we're supporting, that comes in by default. You don't have to pay extra for that. You shouldn't have to. And we should support that ecosystem as vendors, right? And we should support those open source projects to continue doing it.  Yeah, that's quite the soapbox. I don't like it. But no, I think you're right. And I agree that, you know, as obviously as a vendor, as a software business, that is what you should do. You should build value on top. And that's fine. And in fact, that's exactly the right way to go about it. But the basics really need to be open, open source software needs to be available to everyone, auditable by everyone needs to be out there in the community. It's the only way to get real widespread adoption. And of course, the attacks that happen shone a light on it, but those alone wouldn't really push that forward, you know, so it has to be adopted by the community in general. And I know, look, actually, Paddy definitely knows more about this than I do, right? Because I don't, I'm not an engineer in CloudSmith and Paddy is. But so, you know, Paddy, what do you think sort of coming down the pipeline in CloudSmith internally is worth highlighting here? I know our engineering teams are currently working on, I guess, in broad terms, meeting users where they are. So secure defaults and everything are great. But we need to make sure that the packaging ecosystems that our users are working in, whether that's containers or enterprises using Maven repositories or Python dependencies, whatever they're doing, whatever open source tooling they're using in the rest of the ecosystem, we need to make sure that all the signatures they generate, the attestations they generate, we can work with those. And yeah, we don't want, we don't want to create a wall yard. We need to make sure that whatever the ecosystem is doing that we support. So that includes supporting S-POM standards, like the Cyclone DX standard, you know, supporting S-PDX in the future. But it's about supporting all of these open standards, making sure that whatever, whatever way our customers or our users are working to secure their supply chains, that we're there to meet them when we're ready to do that. Not every user is and that's why the UX of these tools and the UX of these processes is so important. If we're going to convince these people to make them move and secure their own supply chains, we need to help them to do that. It's just about working with all those tools. So when RubyGems implements six-door support, we need to be there to meet them with that. When PyPI implements their support for signatures, when Conda supports the update framework, we need to be there and sort of integrating those things as they happen. Yeah. And I mean, Paddy, I know that we have, you know, support for cosine with our OCI registries coming down the line. And I know that kind of stuff, but it's not, it's not just that we're going to provide this kind of stuff for users of CloudSmith, right? We, like I'm sure on the platform team, you guys are eyeing this stuff up keenly internally as well for our own, sort of our own build pipelines. Absolutely. It's something that came up earlier. If we want our customers to have secure supply chains, we're part of their supply chain. Absolutely. We need to be secure and our supply chain needs to be secure. It's turtles all the way down. So we need to be able to trust the software that we're deploying so that when, when we make attestations about the software that our users are shipping via CloudSmith, that we know that those are secure. We need to be able to trust our cloud supplier, our cloud provider supply chain. They need to be able to trust their providers. So, you know, we can really only do the work for ourselves and push others to do that work. But so, yeah, it's really just about utilizing all of those tools internally as well and exposing the parts of those processes that allow the users to verify that we're doing the right thing as well. It's one thing for us to say that we're secure. We need to be able to prove to the users and the people building on top of our platform that we are actually secure. Yeah, absolutely. And I think that that's that's a great way to sum it up. I think I will say, Patty, I think hopefully we're going to see ourselves on the Friends of Sick Store repo soon. So that's the idea. Yeah, that's the idea. Absolutely. For sure. And I don't know if Dan Lawrence, you'd like to say anything about what ChainGuard is doing in this space, but I know that there's stuff that you were going to announce very soon, right? Oh, no, I don't remember. I'll just say, I love the way you talked about all the different ecosystems because you're using the word when and I think that's awesome. Instead of if you're saying when Sick Store comes to Ruby and when it goes to Pi Pi and when the update framework goes to Kanda. I just love that like in the last year, I don't think you could have used that word. I think you would have been like a maybe if future, maybe someday type of thing. But now it really is kind of like inevitable. So it's awesome. Yeah, it's funny how that like a year ago, you couldn't even imagine these conversations happening the same way. Yeah. So I will just add one thing. I know we've only got 10 minutes left and there's a few questions in the Q&A that we're probably going to want to address. Yeah, absolutely. I just wanted to throw this in because this is something again, not again, this is only the second time I've made an analogy in this discussion, right? So I will make an analogy again with the manufacturing industry because the entire focus of this talk has been on threats to the supply chain, right? And when you say that, a lot of people think of malicious packages, impersonated packages, you know, a bad guy, right?  Now, it's not often that a package becomes unavailable, but it has happened, right? Everybody remember the chaos when left pad was removed from MPM, right? You know, so there's, and this is absolutely not the focus of supply chain security, but it's in our aspect. And that is that in that analogy that I used in manufacturing, what manufacturers do, they build up inventory, they have a warehouse, right? So that's something that we do think about in CloudSmith, right? CloudSmith repositories, you can pull in packages from upstreams, you can cache them so that if the upstream source disappears, you have those local cached copies of packages, your build pipelines won't fail. Now, of course- Important difference though, Dan, I will mention is private is trusted first, not public. Yes, of course, of course private is trusted first. And again, this does not absolve you from all those packages need to be verified and signed. All of that good stuff that we've been spending the time talking about. But it's just worth, it's very easy to lose sight of the fact that your availability and the reliability is just a part of a secure software supply chain as well, just the way it's the part of a secure physical supply chain. If a supplier disappears, you need to have a plan in place for that. And it's just worth thinking about. But I absolutely agree. It's secondary to the threats that are being addressed by Sexto and Cosine and vendors like us adding support for those tools. Definitely secondary to those, but it's not something I hear people talk about much or bring up in these discussions. So I just wanted to put it out there before we lost sight of it. That was all. I think in general terms though, Dan, it's a good point that there's no silver bullet here. There's no one fix. There are a hundred things here. There are a hundred different parts of the supply chain, a hundred moving parts that all need care and attention. As we work to secure our supply chain, so signing alone, great. It's not enough. No. Caching dependencies from upstream so they don't disappear. It's great. It's not enough. All of these things to work together. We need the entire community to come together and work to address all these problems or knocking them down one by one. And we'll get more secure over time, but it's going to be a journey to get there. Definitely. Couldn't agree more. Absolutely. And I think that that's part of the challenge, right? I think people look at this problem and often security folks will come up and list all the different things that you need to have in order for something to be trusted as a package or any sort of artifact that you have up in the registry. And I think the challenge there, Patty, specifically, is I feel like, yes, absolutely, there's a lot of different things and you need to secure it all the way down, but we have to start somewhere, right? And I think that that's the important thing to not lose sight of. We have to make these micro steps along the way in order to get the whole thing secure. And I think you can't let a lot of folks get a little bit overwhelmed with all the different things to do and then just throw up their hands up in the air. And I think that that's, you know, hopefully projects like Six Four Cosines and other things like that and open source efforts and getting together as a community will help us to make these things not difficult anymore. Yeah. Well, I mean, the Salsa framework is a great example of that because level one in the Salsa framework is pretty easy to achieve. It's a pretty simple step to get to that. Obviously, getting up to level four, yeah, there's a lot more you need to do, but it's giving accessibility that you can get something in place and you can feel good about it and you can feel good that you've taken the first steps on that path. So you don't jump to Salsa level four from nothing, but that would be, you know, that would be a really monumental thing to try and do. And it would put you off if you tried to do that. So you start off, as Paddy says, little steps along this journey, but we need to do it together. Nobody can run off on their own on this. We need to do it together as an industry, you know. Yeah. And so I guess this is a good, like, transition naturally into one of the questions that was up, which I think then we're starting to answer, which is, you know, Moussa heard two terms on here, cosine and salsa. And he wanted a little bit, maybe just a little bit more definition and discussion around it. We can definitely provide links in the chat if that's helpful. But if you want to spend a little bit of time touching on cosine and salsa specific. Sure. Yeah, I'm dropping the links in here right now. Excellent. There you go. The system works. Cool. Yeah. And I'll just stress that salsa is a security level. I can never remember what it stands for home and security levels for software artifacts or something like that. That's my idea yet. Yeah. And it is a framework one levels one through four today and still kind of under heavy development as we iterate and people actually try to work up their way through that ladder. Like you said, level four is pretty hard. I'm not sure if anybody's really done it yet. So as we get to those higher levels, one was designed purposefully to be easy. So people kind of get on the train and get a badge set up in the repo and show what they're doing. The way I kind of think about it is a level one is like, don't do the build on a machine under your desk or your laptop or something like that. That's basically all you have to do is like use a build system. Level two is like prove that somehow. So publish some like Providence or something like that. So people can verify that you're using a build system. Level three is like use a secure build system. Maybe not just everything will qualify today. It'd be great if all these build system operators do increase that security. But and then level four is change your whole build process to make it hermetic and not pull dependencies from the internet and you know perfectly. So you know, kind of far out there, the most secure you can possibly be. So I think there's a pretty big jump from three to four today. That's kind of my take looking at it. So we might end up, you know, adding an intermediate one or something in there shifting some things around. But yeah, it's just going to be pretty easy to understand and you get value out of each level as you go. Awesome. Yeah, no, that's a great, great summary of it. I think, and so along those lines similarly, right? It's when we talk about six store and cosine, right? Cosine specifically, I think, so six store, you can think of it kind of like your umbrella, right? The way that just being able to sign software in different ways and cosine is specific for sort of OCI container image. Right. Is that a good way to understand it? Yeah. And we can drop the links in the chat as well. So there's one last thing I'll do is there's a meeting question here, which we kind of need to get to. And I think it's kind of one of those things, which is open-ended a little bit, because I think I'd like feedback from all of you on it. And I think it definitely, this is part of the stuff that we need to work together as an organization or as a community to do, right, as all of these organizations. So Varun Brun bring this up and says, many organizations nowadays rely on distributing their software from sandbox containers or images. What could be the best practice as a customer who's consuming these kinds of distributions from the vendor? They need to contain those containers and images or sandboxes are an entry point for supply chain compromise. Any good practices for the customers to conduct the test on these binaries? So please do suggest. So this is talking along the lines of, hey, as consumers of these products for these container images, what can we be doing as a best practices when we're when we're on the consumption step of this to make sure that we're checking the boxes? I'll let Paddy and Bam take out one. Yeah. So it's more a best practice sort of conversation. But yeah, go ahead, Dan. Yeah, I can start. I mean, there's probably like 37 different threat models probably wanting to worry about here, but I break them down into like two categories for you know, two high level ones. First one is, you know, you're consuming this kind of black box thing from a vendor. You know, it might have some vulnerabilities inside of it.  So that's kind of like, you know, if you're consuming bad software, it doesn't matter if it's signed to you exactly where it came from. And you have all the metadata, right? It's still bad software, not bad, bad, like, you know, it might be vulnerable, might be old CVs might have been found that kind of thing. And so you want to do your scans on it, you want to make sure the code is running an environment of least privilege, all that stuff. So if it is compromised, right, it's not running as root with, you know, network access and access to your database and all that stuff that can get actually traded. The second one is yet, did this actually get to me from the distributor correctly without being tampered with those kind of things. And so that's checking the signatures, if they provide them checking the hashes, if they provide masking them to, if they don't, depending on your relationship with the vendor and making sure that, you know, an attacker in the middle can't change it, that you're grabbing it from the correct place and that hasn't been tampered with along the way, or somebody could insert some kind of targeted malware. So at a high level, those are kind of the two big categories I look at and you kind of need to treat them a little bit differently. Good answer. Bad day, do you want to add to that at all? I think that covers it. Really, it's about provenance on the other side is just old fashioned traditional ops stuff. It's like, you know, if you're using, exactly, yes, if you're using this sort of tooling, right, I think as a consumer even, you know, build the expectation with your vendors, ask them to be able to support that. Like if they don't currently have a mechanism for having an S-bomb alongside an image, right, then ask them for it. But in the feature request, because that's this sort of, you know, desire or demand from the industry is going to what's going to push a lot of vendors that way. We've heard it a lot on this side and believe me, we hear you and we're working on that. And we want to continue to support those efforts. So yeah, that's what you can do to be a part of that, but have that build that expectation, especially with your cloud vendors, that it should be deep by default. You should make the stuff available to us. And it doesn't matter how much I'm paying you for it. Just, you know, like I'm a consumer of yours, you should be giving me this stuff by default and available to everyone, right? Democratize it sort of security, right? It's the rising tide lifts all boats, things, right? That's it. It gets better for open source, it gets better for everyone. That's right. That's exactly it. Great phrase. Absolutely. So I think we are running up against time a little bit. So I want to thank my colleagues here, Patty, Dan, and Dan for all of your wonderful insights. So I look forward to some of the stuff we're doing over here at Cloud Smith and also eagerly watching what's going on at J&Guard to see what's happening there in this space. So folks, again, thank you so much for all your time and joining us on this talk here today. And this doesn't happen without all of you and all of our efforts together. So remember that and thanks for sharing it. Hopefully you've left with a little bit more information around making this big issue and sort of tough issue a little bit more accessible for everyone. Thank you for having us. Yeah, absolutely. Thanks. Thanks. Thanks all of you. So with that, I'm going to throw back the word of Larissa at Linux Foundation. Awesome. Thank you so much, Adele and Patty, and Dan and Dan for your time today. And thank you again, everyone for joining us. Just a quick reminder that this recording will be up on the Linux Foundation's YouTube page later today. So we hope you'll join us for future webinars. Thank you so much again and have a wonderful day.


In this video, I wanna talk about some homomorphic images of cyclic groups. And it turns out it's very easy to describe when you have a function, I have a homomorphism between cyclic groups and when you do you not, right? So imagine you have an integer N for which it has a divisor D that you see right here. So with a divisor D in hand, we can define a homomorphism from ZD to ZN. So ZD is the cyclic group of order D, ZN is the cyclic group of order N. And we'll think of this as additive notation. So we're working mod D and mod N respectively. So since D is a divisor of N, we're gonna define a function from ZD to ZN by the rule that the integer M will map to the integer MN divided by D. Well, since D of course divides N, that implies that MN divided by D is of course, well, I should say that N over D is an integer, right? So therefore M times N divided by D would be an integer right here. So that makes sense. Now we have to of course be concerned, is this map well-defined? Because after all ZD here is the set of representatives of congruence classes. So if we choose a different representative from the same congruence class, do we get the same element? That's always has to be of concern when you define a function based upon representatives of some equivalence relation. So if we were to consider that, take phi of M plus KD, right? Well, what does phi do? It just times the integer you started with by N divided by D. For instance, this is just multiplication of integers you can distribute here.  So the D that actually cancel and you're gonna get KN, which means that MN divided by D plus KN is congruent to MN divided by D mod N, right? Which is where the co-domain lives, right? And so therefore these two elements are congruence to each other. And so this would just equal phi of M. Whoops, that didn't look right. Phi of M. So if two numbers are congruent mod D, then their images will be congruent mod N. So this is in fact a well-defined map. Is it homomorphic? Well, let's take the sum and send it through the function and see what happens. So if we take two integers mod D, we add them together, A plus B. We take phi of A plus B. You're gonna get A plus B times ND, which again, this is an integer. We distribute it, because this is multiplication of integers right now. You're gonna get A N divided by D plus B times N divided by D, which this first number right here is just phi of A. And then the second number of course will just be phi of B. And so we do in fact see that this is homomorphic. So we do get the homomorphic property. In fact, this is an example of a monomorphism. That is it's a one-to-one homomorphism. How we know that this thing is one-to-one, right? So if you take phi of A and you set it equal to phi of B, then this would tell us that A times N divided by D is equal to B times N divided by D, right? For which, which admittedly this is up to congruence, right? So you're gonna have some type of KN going on over here. That's what you would see there. But another word, but that's not such a big deal really because when you have this statement right here, we can cancel, we can cancel the N plus D going on right here. But let's try to do it a little bit more properly right here. So you have this. And so the reason this is of advantage to us is we just factor this, right? Just times it by D over D. And so it's in the right-hand side, you can factor out the ND. So we're gonna get KDN times N over D. And as this is just multiplication of integers, we can cancel that. And so we're gonna get that A is equal to B plus KD times N, which tells us that A and B are congruent to each other, mod D, like we were looking for. So this is in fact a one-to-one function. And what we're gonna see, again, this is something we'll explore much more in the future, but note that the image of fee is actually equal to N divided by D, right? That's what's going on here. And that's isomorphic to ZD, which was the domain, right? Now, I'm not saying these sets are equal to each other, but they're isomorphics. It's the same group up to isomorphism. We can actually generalize the previous example. Take any group you want, any, any, any, any, any group, and then pick your favorite element inside of the group, G, what we'll call a little G. So then I can define a homomorphism from the infinite cyclic group, right? So this is the integer with respect to addition. We're gonna define a map where fee of N maps to G to the N. Where again, little G was our favorite element we picked here. Now the usual exponent rules are gonna justify that this is a homomorphism. I'll leave it up to you to verify this. Some other things I can mention here is that this map will be a monomorphism if and only if G has infinite order. Because if G has a finite order, let's say like the G has order three like that, then you're gonna get the following, like zero will map to the identity of one. One will map to G, two will map to G squared, three would map to G cubed, which of course is equal to the identity. So this maps with here right here. So it wouldn't be one to one in that case. And that's basically the argument. But we can also do the following, right? Similarly, we can define ZD to G by the rule, right? Phi of N equals GN. And this is gonna be well-defined if and only if D is a divisor of the order of G. And so again, I'm gonna leave it up to the viewer here to prove this. But if D divides the order of G, this will be a well-defined map and it's well-defined if and only if, right? Likewise, this will be injective only if D is equal to the order of G. And so I'll let you try to convince yourself so those things. But essentially, this then determines the only types of homomorphisms between groups, right? When it comes to a homomorphism from a, when it comes, well, actually we'll just say it right here. When it comes to a homomorphism from a cyclic group into a group, this is where the cyclic group is in the domain. These are the only options, right? You have to pick your favorite element in the group, but its order has to be divisible by the element, the order of the original cyclic group, whether the infant cyclic group or the finite cyclic group right here. Let me give you one more map in this video here, kind of related. If we look at the real numbers versus the circle group, right? I say this is kind of related because the circle group actually, this would be, this right here would be the group of all complex numbers of modules one, right? For which the circle group contains all of the roots, the complex roots of unity. So actually the circle group contains every finite cyclic group, up to isomorphism, it's kind of cool little thing there. And so we're gonna define a map from the real numbers to the circle group where phi, excuse me, theta maps to e to the i theta, which every complex number of modules one could be written of this form, e to the i theta, somewhat of a consequence of, you know, the exponentials of imaginary numbers and things like that. And so this map right here is, again, gonna be a homomorphism by the usual exponent rules. I'll let you convince yourself of that, but it has to do with addition of exponents and things like that. This is also surjective because like I said, every complex number of modules one has this form e to the i theta. So this is an example of a surjective homomorphism, a so-called epimorphism, right? Now this one also has sort of a fun little geometric or topological feel to it. Because after all, why do we call it the circle group? Because in the complex plane, the circle group is the real McCoy unit circle, right? It's a circle if we look at it from above. And now the real numbers, we can think of it geometrically as a line. Now, if we add a little bit of a coil to our line, what we can kind of think is the following idea, right? That if we take the real numbers and we kind of spin it around like it's this infinite long spring, and as it lives above the circle group, right? This is our real numbers right here. Oh boy, let me try that one more time. Here's our real numbers right here. And so there's a nice little like topological identification here that the real numbers essentially cover the circle group here for which this epimorphism preserves the multiplication, but also it's gonna be continuous, right? It's not a notion we really worry about in abstract algebra, but as the real numbers in the circle are topological sets, we could ask, is this map between them, does it preserve topology whatsoever? So we actually have a continuous epimorphism. And that's because both of these things are examples of so-called Lee groups, which are groups which also have some type of geometry attached to them. In particular, these are groups for which the multiplication inversion maps are both continuous and differentiable. Not something we're gonna worry so much about in this lecture, so you just wanna kind of throw out those buzzwords for us here. But this homomorphism here, it's not saying that the real numbers in the circle are the same, but basically it's saying that we can cover, because it's an epimorphism, we can cover the circle with just a line. It's kind of, it'll just wrap around, right? So it just wraps around and around and around and around and around and around, right? You're basically working mod two pi in this situation here. And so this algebra os is actually compatible with the geometry. This is a continuous epimorphism.