All right. Hello everybody and welcome to our talk on call for code with the Linux Foundation and how you can contribute to tech for good projects. Even if you're not technical, if you don't write code with me, I've got Demi Ajayi and my name is Daniel crook. Let me just pop over to the intros and Demi wants to go ahead and introduce yourself and then I'll take us to the agenda. Sure. Thanks so much, Daniel. Hi, everyone. As Daniel said, I'm Demi Ajayi. I'm the open source community manager at call for cooperation justice. So my job is to bring people into our community, both developers and non developers and get them excited and contributing to our projects. Wonderful. And I'm Daniel crook. I'm the chief technology officer for the call for code program, as well as the initiatives within that which I'll take you through in just a second. So what I want to do in this talk is introduce you to call for code and some of the activities that have been there and kind of the resulting open source projects that have come from it. We'll talk about how we support those projects, how we improve them, where you can find them through the Linux Foundation website and on GitHub. I'll talk to you a bit of those projects kind of show you the background of the technology they use. And then we'll talk about how you can contribute in general to open source. And in particular to these particular projects, where to find the resources where to find mentors, mentees, other folks in the community that you can share your knowledge with and provide your own solutions to improve these projects and therefore address some of the key societal issues that they address. So call for code as a program was launched in 2018. It was an issue between IBM, the Linux Foundation, the United Nations and David Clark cause. He's a fellow that raises awareness importance to social issues, humanitarian issues, like truth and reconciliation in South Africa, access to clean water throughout the world, veterans issues and many more topics. So we all partnered in 2018 to look at ways that we can leverage the world's 20 million developers, their skills, leverage those skills to point them at important social issues, important growing issues and how they can solve that with open source. So what we've done over the years is we've had an annual competition, we lay out a prompt to have developers go take that issue on. We recognize and award them for the work they've done, but what we do differently with call for code is that the solutions that are the top winning solutions, the global winner, potentially some of the runners up, we work with those teams and those projects to transform their raw innovation into something that can live on as a sustainable open source project. And we do that in partnership with the Linux Foundation. So throughout the years and throughout the various programs we've had almost half a million developers building new skills, learning about these important issues and seeing how they can use them to take on these great challenges. And these are issues that affect folks around the world. So we've seen folks come in from just about every continent, I think somebody registered from Antarctica this year. So I'm not sure if we get the seventh continent this year, but we've gotten folks from around the world, every region of the world, and even folks who've partnered up in the Slack community with other folks working on similar issues in different parts of the world. If you've heard of call for code before, you might have heard of some of the existing programs we've done. So when we launched in 2018, we focused on natural disasters, preparing for them, responding to them and recovering from them. And the very first winner and the very first project that we brought into the Linux Foundation was from a team called Project Owl. And what they did was create an emergency mesh network built on open standards and open source, including Arduino, the open hardware platform, MQTT, an open messaging framework, and using mesh network using Lora, Long Range Radio, another standard, to create an emergency solution for restoring network access after disasters such as Hurricane Maria, where we saw up to nine months people were disconnected from communications, whereas they were able to restore power a little bit quicker. So it's really about using cheap and quickly deployable solutions built on open standards to provide that emergency mesh network to provide just enough connectivity to get some messages across to folks who need help, from people who need help. The following year, we took the disaster theme a bit further, focusing more on the individual, the health and the well-being of individuals. So another great solution that was just brought to Linux Foundation as the Pira project is from Prometail. There are a team that was composed of a firefighter, a nurse, data scientist, a full-stack engineer and a PhD who came together to see how they can automate the manual process of tracking the chemical exposure of wildland firefighters in Spain. And so they've created a great IoT solution built on MQTT as well. It emits data back to a data science back end built with a bunch of microservices that compute averages of exposure of the chemicals collected by the devices, and saving that to some long-term averages so that nurses and folks can understand how to make better decisions from long-term health outcomes of firefighters. The third year of the Global Challenge 2020 was a very transformative year for call for code and the projects that came into the Linux Foundation. We had a dual track. There was one around climate change, so treating the cause rather than the symptom, which was the disasters. A solution that we're going to be bringing into Linux Foundation soon is around helping small-holder farmers in emerging nations understand how they can consume upcoming weather forecast together with long-term trends from NASA, bring that data together using data science and put it in a digestible format so that folks who are being affected by climate change know when to plant crops, where to do it, and what to plant using that information so that they can respond better and be more resilient to climate change. We also added the track for the pandemic last year, the social and business impact, so there's some solutions we created around social distancing, getting access to microfunding for startups that were affected by the downturn in the economy, so you'll see some programs there as well. But the biggest part of 2020 last year, 2020 was huge with the pandemic with climate change, is call for code for racial justice. So this is a program that started out within IBM with seven open source projects created in response to the big flashpoint, to the forefront racial justice issues last year, and how technology can be used to help address some of those solutions. So we have seven open source projects specifically in the call for code for racial justice area of the call for code program, and that brings us to a total of 14 projects that are available for you to learn about, contribute to, and help advance and deploy around the world. So I mentioned too, it's not just about creating new ideas to take on these challenges, what we really try to do with the call for code program, and why we rely on folks like you is we need to take this raw innovation that comes into the challenges each year, or through challenges that may emerge over the course of the year for particular issues. We need to bring them from that raw promise through an incubation framework where we can improve the hardware, the software, the data science, but also validate those solutions with community members who use them, who can validate with the developers, you know, are we solving the right problem here? Are we doing this the correct way? Are we aware of this maybe law that we weren't aware of? Have we got the documentation in the right languages? Have they been translated? And what we do through this incubation framework with the community is bring them to a point where they can live on as those sustainable projects with an ecosystem of users, and ideally have some sustainable organization backing them in partnership with others at the Linux Foundation as a neutral home. So we call this the deployment framework, and usually towards the end of the implementation is when we officially launched those projects at the Linux Foundation, where you can learn about them, participate in the community, and join and be a main container of the projects. So let's take a look at the technical pieces before we get into the non-technical. So if you're familiar already with the Linux Foundation's approach to open source, you may know that if you go to linuxfoundation.org slash projects, you'll find a whole gamut of solutions that the organization manages beyond call for code. There are lots of umbrellas around particular areas that you can explore. And so call for code itself has its own sub umbrella there, and that has information on the 14 projects. We've got them that they correspond to two GitHub organizations. There's the original one that was created for the natural disaster and climate themes, and we've added a new GitHub organization. Particularly, we can find the call for code for racial justice projects. There's also another initiative at the Linux Foundation that would be something of interest folks as well. It's a part of the CNCF, the Cloud Native Computing Foundation, called the Inclusive Naming Initiative. So that's also one where as a non-technical contributor, you might help them formulate the right terms to use as we talk about technology in an inclusive way. Just to give you a quick overview then of the particular projects, they are kind of segmented into these two umbrellas, but they all are call for code projects. We have the Pira project I mentioned came from the Prometeo team. It's around monitoring toxin exposure to firefighters. We would love to see anybody who wants to adopt this technology.  Maybe you're someone who has a background in emergency communications and potentially even an expert in carbon oxide, molten exposure. That's one where you could bring your expertise to the project. For the cluster protocol, that's the emergency mesh network one.  Maybe you have some interest in emergency communications, things like that. Those are definitely areas where we can love to see your adoption there. Isaac Cimo is a project that is focused on the in-emerging nations improving construction quality, sometimes in the absence of national standards or regional standards. What it is, a visual recognition project that assesses building quality to ensure that work done to fix or remediate a structure has been done correctly. Within that project, we need folks who may be in the construction industry or otherwise interested in architecture, whether the visual recognition models are trained on the right data to make the right decisions as to whether it's properly done or not. Lots of ways you can get involved with that one too. Maybe you bring different country expertise. It has been focused on Nepal and Colombia to date. So if you have areas that you're interested in the world where certain construction types are affected by certain disaster types like hurricanes, windstorms, things like that, you can take part in that. For OpenEW, this is a technology that came from a startup called Greo. It focuses on measuring doing earthquake early warnings. So by distributing a bunch of really highly sensitive accelerometers around the Caribbean, what it's able to do is gather information, reconcile and confirm that it is indeed a real hurricane or shaking in the ground and alert people in advance. So where we would need help here is seismologists who understand some of the algorithms or they understand some of the best places or cultural areas where populations are, how they do emergency management in particular countries with a focus on the Caribbean and in Mexico. And I won't go into the other two projects just yet, but Liquid Prep is another one focused on agriculture that you might be interested in taking part in if you're interested in how sensors work, how agriculture works, how weather works, or how you can translate this type of information into a way that can be used in the regions around the world.  Demi, why don't you give us a quick summary of each of the seven projects in the call for code for racial justice area, and we can kind of phrase those how those can be contributed to. Sure. So in call for code for racial justice, we have seven projects as Daniel mentioned. So we have fall under three different pillars. So the first one is police judicial reform and accountability. And with those projects we have fair change, which is a mobile app that allows people to people to record interactions with the police in a safe and secure manner. We also have incident accuracy reporting system, which is another platform that allows eyewitnesses to record interactions with the police as well as provide their own eyewitness accounts of that. And then what the purpose of it is to then corroborate that with official police reports. So if there are any discrepancies or inconsistencies it flags those reports so that the police department themselves their administrators can have a good line of insight into any kind of inconsistencies that are coming out of their department. And then lastly under that pillar we have the, we have public, a public defender focused projects, which is open sentencing so that's for public defenders to be able to understand and better advocate for their clients when it comes to charges that are brought against them to understand any kind of racial disparities or gender disparities that are occurring within people who are being charged for that specific crime because as we know there tends to be racial disparities in these kind of charges. And then our second pillar is under is under diverse representation so with diverse representation we have take two, which is a project that basically allows people to flag any kind of racialized content online so if there's any kind of racial bias content that's offensive they're able to flag that and it also provides them a way of being able to see potentially if content is biased that allows people to contribute to these to the database as well as to be able to see and get feedback on that as well. And then lastly our last pillar focuses on legislation and policy. So within that we have a few other solutions to fight its voter, which is a platform that allows people to along their voting journey to better understand, you know, voting deadlines, voting registration deadlines, mail in ballot deadlines, where the polling locations are in an effort to just make that process easier and fight voter suppression by providing people with information they need in an easily accessible manner. We also have legit info and truth which both seek to provide people with information about legislation policies within their community, and make it easy for them to understand how it impacts them, and then truth loop takes that it to another to a step further by allowing them to then provide feedback on on these legislation so then they can upload information and provide just feedback on how these legislations are affecting them in their day to day. Excellent. Excellent. And we do have additional projects that are going to be joined the Linux Foundation on a regular basis. So we do have some additional projects that may be of interest that fall into those three categories that we mentioned. And we do have some other ones in the other areas, the COVID area, the climate change area, the disaster areas as well. Okay, so we do have the descriptions are up on the Linux Foundation website and on GitHub, but we at IBM also have a set of tutorials that folks can use to learn about these projects. Understand it as an end user, you might want to go in there and just learn about the project see how it's trying to solve the problem. Consider how it whether it's doing the right job or if you have any suggestions for it. You can try it hands on if you do have some technical inclination to try to set up the solution on your machine or deploy in the cloud. And then there's the contribution guidelines. So each of these projects is governed through a technical steering committee and a technical charter that is part of the neutral governance provided by the Linux Foundation. You can learn about how the project handles contributions, the code of conduct, what sort of important areas that they want you to contribute to, or to focus on. And we do also have an additional area on the developer website, specifically around those particular issue areas that Demi mentioned here we're just showing the policy and legislation reform area. We've got videos, as you can see how the, what the issue is that's being solved and how the approach has come together. And we also do have complete tutorials for each of those projects, so that you can go through, learn about the project and see where your opportunity to contribute might be whatever your background. But we know that, you know, technologically, even though it's accessible to everybody right now, it seems like if you apply to any of the situations that people have the ability to build new skills. There's tons of ways specifically that you can contribute to open source in general, and perhaps one or more of these projects in particular. So, let's take a look at some of those those ways that I can approach from. So, yeah. Yeah, definitely. So as Daniel mentioned, he's also kind of alluded to it during his presentation earlier, as he talked to the projects are many ways to contribute to projects, even if you're not technical. So a lot of times people, when they come into open source communities, you know, there are two concerns they have that one they don't have the skills, and two they don't have the time, right. So what we've outlined here just a few possibilities of the types of contributions you can make, even if you're not technical. And also if you have, if you have different amounts of time available to contribute to these things. So the on the left hand side we have the hours days and weeks is not exactly precise people might have different perspectives on how long it takes to do certain things but just as a, just as a measure, just in terms of scale of what it might take to do these things. So the first one, which we find to be the most the easiest entry point is documentation because as Daniel mentioned when you're coming into a project, won't understand what it's about so you're going to read the the tutorial you're going to look at the read me. That's usually the entry point there. And that's a place where you have a great opportunity to really welcome and onboard your community. And a lot of times there might be some issues that come with that documentation. So the first thing that you can do as a non technical contributor is you can flag documentation errors things as simple as grammatical errors or spelling errors are things that you can contribute to in terms of updating the documentation. And then secondly, another thing you might want to do is document new features so as we're bringing on new features, you know, we might not be able to describe them or market them essentially you know in the way that's most understandable to users. If you have a background in writing or marketing, you can also help us and work alongside with technical writers to ensure that we're documenting new features in a in an easily to understand way, as well as maybe updating our release notes and things of that nature. And then lastly, just in terms of like something that might take a bit much a bit longer to do would be helping us in creating onboarding guide. So for instance, you know, again, as you're coming into new community, there are lots there's lots of information that's been created lots of content, lots of things to learn lots of past presentations like this or you know things describing the content. So what we'd like to see or one another opportunity for people to get involved is to then help in creating these onboarding guides, which can be PDFs, they can be websites, just many different ways to help people get on board. And also by using your own personal experience and onboarding you can help to guide the way those guides should go forward because you're able to provide your own personal experience in getting to the community as well. And then secondly, there's designing UX so whether you're officially a designer or you this is just like something that you kind of do on the side or have an interest in this is also something you can do so, for instance, you might help design a logo. I know that's something that was done previously. I believe Daniel can talk about that. I think via Hacktoberfest by another group as well. Yeah, yeah, so actually coming up on October, there's a yearly initiative. It's run by DigitalOcean, IBM kicks part in it as well as many other organizations do. So when you go into GitHub, you may see things tagged Hacktoberfest and what they're intended to be are really, for the most part really great lift contributions where you can contribute something in a non technical way. Normally the open source community, you don't like to see what we call drive by commits where you just make one issue and then you leave the community. We want to see engagement or a period of time, but what's great about Hacktoberfest is you don't have to really be afraid of that commitment to a project. You can make a small contribution by addressing maybe a longstanding issue or something where the maintainers of that project don't have skills that exist out in the broader community. Maybe it's user experience, maybe it's doing a simple translation, maybe it's doing some piece of work to automate builds or other things that community may have wanted to implement that just haven't been able to do it and that's what Hacktoberfest is all about. Great. Yeah, so during one of those sessions, we're able to get a new logo for one of our projects. So that was great. And it's also obviously just like something that happened in the matter of hours and not, you know, taking forever to get done as well. Another thing you might be able to do is conduct user research. So a lot of for our projects, these are tech for good projects, which have a direct end user, which can be across a lot of different spheres, so farmers, firefighters, public defenders, as we mentioned. So we want to be sure that these solutions are actually meeting the needs of the users. So providing user research, getting the feedback from people as they're using the product, as they're using the solution as they're developing prototypes is another way you can contribute as well. And then also on a larger scale, it can also help us in updating the UI. So a lot of times with our projects, we're looking to improve them. So another way you can get involved is to help us do that improvement, taking in the feedback from the user research. And helping us update the UI or just update the user interface or the user experience if it's like a hardware tool, for instance. And then another way to contribute is if you are a lawyer, for instance, and we do have find a lot of need for this within our call for good racial justice projects is you can answer questions if there are any drive by legal questions that we might have. For some of our solutions, we like for the legislation solutions such as Truthloop and Legit Info, we need to populate our database with that legislation. So if you're able to point us to reference legislation like maybe in a particular state or in a particular country, that's another way you can contribute. You can also help out in by helping us review licenses, whether these are like open source licenses or other things that we might be thinking about engaging in. That's another way of engaging as well. For subject matter experts and we know this varies across many different subject matter experts as Daniels mentioned, like we talked about seismologists we talked about firefighters we talked about public public defenders. Lots of different ways and areas you can be a subject matter expert in and depending on the project there are ways for you to provide help with us. So as we mentioned before, related to user research, you can actually provide user feedback. You can provide relevant data so that's data curation or manually updating our database by providing us doing some crowdsourcing of data that we might need. You might also be part of our advisory council, depending what the project is, you might have a council of people actually provide oversight, relevant to that specific area just because they have expertise there. And if you're not a subject matter expert, you can still be an advocate for our community. So a simple way to do this would be to share a project in your community. So whether that's on Reddit, or any other Slack groups that you're part of, or any other online communities or in person communities that might be interested in that. So for instance, if you're a public defender, you might be a part of a professional organization so you might talk to them about using this, some of our solutions, and things like that. You can also speak at a conference so something similar to this, maybe specific to your industry. You can talk about the projects that we have and that provides you with like a larger reach potentially to get the word out about the solutions. And you can be a project champion. So specifically this might, this is something that we have with our call for code projects here and Daniel can probably speak to a few examples of projects that have champions today. And yeah, so that's just another way you can get involved. So that's kind of a more committed way of getting the word out about the project, you know, sharing sharing it out on LinkedIn, and just being an advocate for the project in a more in a long term fashion. So Daniel, if you want to speak a little bit more about that. Yeah, yeah. And things really important too that the building on the network thing so we're speaking at a technical conference so there is a bit of an implication that you have some technical background. And so your how a very important role in terms of bridging this community to the communities in your area in your neighborhood, where you can be that advocate to talk about. Okay, I have my feet in both these areas. I know a lot of issues around legal and licensing and access to privacy and data. And yet I this technology project hasn't been introduced to them so you could do a lunch and learn. Hey listen, I know we have this long standing problem in our community. Here's a technology solution we would love to explore. Let's see if we can get a prototype or a pilot done with the technology. That's a really important part of the advocacy that's being directly around the champions. So specifically within IBM, we do have IBM champions and what they do is they have a specific responsibility. It's an honor each year to be named IBM champion, who is somebody outside the company that's an advocate for the technology that's being used, whether it's an open source project the strategic one that I've been participating in Kubernetes containers, things like that blockchain. It's a great way to be recognized for doing that.  So definitely an option there to be recognized and build your own influence, your own eminence as as somebody interested in seeing how technology can take on issues. And lastly, it's on our list today and of course there are many other ways to contribute is just being a part of the community right because community is basically the lifeblood of open source projects essentially because that's how we're able to ensure that we have contributors and also bring new people into the community who might contribute. So even if you're not necessarily going to be contributing whether technically or non technically you can still use your social networks be able to draw other people into the community as well. So a low lift way of doing this is just attending community events, you know, networking with people understanding what the needs are participating in that way. And you might graduate from that actually decide to organize a community event. So maybe just a one time event where you're maybe talking about anything that you're an expert in like recently we had somebody who came into our cute part of our community who talked about writing unit tests so providing just that kind of teaching to our community was very valuable, and just a way to enrich the overall experience for people not just in terms of contributing to code but also improving their skills as open source contributors at large right whether that's building the technical skills learning a bit about more non technical issues to you can do that that's another way you can be a part of that community just by sharing your skills, even beyond the specific scope of a particular project. And lastly another way you might want to be a community contributor would be to lead a regional chapter or lead a university chapter or some kind of university grouping that's associated with the project that you are a part of so that's definitely something that takes a bit more commitment and it's more long term but that's just another way to be involved as a community member as well. Excellent. Excellent. Yeah, and one thought just on top of that is what's also great as you can apply the same skill over and over across all 14 projects, you can deeply get into one project if you're interested in, because they all have very common needs in many ways. There's always help needed for for example translations or fixing typos approving documentation across the board making sure things are consistent. So we do really appreciate not only a deep dive into projects with your skillset but also seeing how you can spread them across the projects help us make them more accessible and ready to others. And that's really important part of being a non technical contributor. Great. So now we talked about some of the things you can do. Let's talk a little bit about how you actually can can get them get started with this getting involved. So, um, Demi, why don't you take us through these specific actions and how we can take those those needs and kind of show people how they can get started. Right. So we've identified three major ways that you can get involved they kind of progress across this these three buckets here so the first is to join the virtual community and because we are primarily virtual these days so it's really helpful for us because we're already on Slack so joining something like Slack or Discord. In our case, if you do want to join our Slack, you can do so by filling out this invitation form, which is call for code.org slash Slack to get an invitation into our Slack community which is our call for code Slack community. And then once you are in Slack, you'll be able to then find channels within our call for code and we'll talk about that a little bit later. We have many channels for our different projects and you're able to just go in there and really find the one that's the best fit for you. Based on your interest based on your, you know, time commitment abilities, and based on whatever it is that they're actively working on at that time for a call for code for racial justice within our general Slack community you're able to join the racial justice general community. And then once you join those channels, primarily, you know, we're doing things there that we can't do in GitHub right so we're planning events, you know, sending out polls or someone you should have things. So you can also join events and stay abreast of what's happening from that perspective in the community as far as events and just gatherings to talk about projects. And then the second thing you can do, once you're part of the community, you know, you've joined Slack or even in parallel is that you can join us on GitHub so our products live on GitHub so you want to have a GitHub account and you want to browse different repositories, start the repositories. So for call for code for racial justice our main repositories call for code for racial justice in GitHub and then there you'll see our sub repositories as well for each of our different projects. And you then you can read the documentation as we mentioned earlier on that's a good way to start contributing reading our documentation reading our read me's. If you find any issues there you can create issues to update those errors that there might be within that documentation. And then you can also find existing issues so within our projects where we try to do is have both technical and non technical issues that you can you can contribute to. So then you can flag those issues and you know, assign them to yourselves and begin to work on them. As you get into the community. And also you can also create issues so for instance if you're talking to other people in Slack you have a great idea, or you're you know discussing in the comments in a GitHub issue you can also have find opportunities of new things you might not have considered maybe new feature sets, maybe deeper dyes into data sets that we need to do, and you can create issues there within GitHub as well. And then lastly, the other way to get involved as we mentioned before is also just outreach right so everyone has their own network, and we are networks are valuable to contributing to our general community and the open source of advancement of these projects so the easy first step was you know just to recruit your friends or colleagues people you work with on other projects who might have similar interest already so for instance as we mentioned before. If you're part of a professional organization that might have a synergy with a project you might want to mention it to them recruit those people to join, or to learn about an event even if they don't want to join the entire community, they can still participate in coming to one of our events. And the second thing is networking so as you're going to professional organizations conferences and things of that nature. You're also have the opportunity to network and mention, as you mentioned before, and advocate for specific projects that you might be working on. And as we mentioned, you also have the opportunity to host events right around these projects or around similar issues where you can mention the projects and get people involved that way. And lastly, another thing that you might want to do, especially if you're you're starting an open source community of your own is you might want to add your open source repositories and issues to open source contribution sites. So for instance, there's code triage that you might want to add our first time first timers only is another one where you can actually have your open source issues be a part of those websites so other people can just quickly pick up new issues and get started on them as well. Excellent. Excellent. So I'm just before we get into to take you through kind of how you actually get started right away in there. As Demi mentioned, we've got the call for code community. Join that there's tons of channels I'll show you those few of those in a minute. We do have the projects you can discover on links foundation that org site and the technical resources for doing a deep dive tutorials on our developer site. We've got the organizations you can go find your open issues for those two things and and sharing with your network. So let me swap over to just kind of show you a few of these things. So to join Slack, what you'll need to do there's call code org. It'll give you you fill out this form if you already have an account you can log in there. I just got to make sure you're good with the privacy policy. You're good with abiding by the code of conduct to make sure we have a very welcoming community and inclusive community and provide your name and then basically your email address and you'll end up getting an invitation to the Slack community. Once you're in there. There is a pinned message to our general channel. Again, it covers all of the programs within call for code. There are some default channels you'll find. If you want generals the main one you can introduce yourselves. Learn about some of the goings on the community. We've got channels for each of the projects out there. And in particular for racial justice we've got a landing community. Demi is very active in there. She can find information on the events, the question the community, the kind of out sort of things that you can grab on to and contribute to. And there are listings of events as well as a help desk. So for example if you do want to find how to use a particular technology maybe have an issue understanding IOT understanding data science. You can definitely reach out to folks there. There's tons of tons of channels. We've got over 25,000 folks in here, willing to help you out willing to steer you in the right direction. So that's it. That's your first place to start. As I mentioned there's the Linux foundation website, and there is a project catalog repos pinned to the main call for code one, where you can kind of learn more about those particular projects their backgrounds and specifically drive into the repos. And one of the things that I realized we didn't mention earlier that we do have available to non technical first time open source users. We've got under our project sample repository, a link to the a new open source course that we released earlier this year. And the intent was for non technical people who wanted to take part in open source communities, how they can actually start to use the tool sets. So GitHub can be a little bit confusing for new users. It is code oriented, but people use it authors use it my sister she's a professor she uses it to manage and version her own content. So if you go into this, it's a free course, you can get a certificate for completing it after about four to six hours, and it's got really important lays down the background of what open source is why it's important. Some of the rules of the road, how to use the tools like GitHub to contribute. And if you are interested in kind of starting your own repository around a particular issue. You can do that as well. So this was really intended with you in mind, and expanding open source skills as part of digital literacy. It's becoming important as a computer and typing, we're in previous generations. So we invite you to take a look at that. Again, it's linked through our GitHub, but you find it instruction open source on cognitive class. If you dive into the, the repos, you can definitely check out the activity that's going on, you can meet the people there, kind of find out who's been working on what you'll see. What particular issues are you can check out how many forks and how many different versions of the code is being worked on by folks in the community. And again just find them and and star them and share them with others through the community. Let's see if there's anything else we wanted to do. Oh yeah, I did want to cover the developer resources quickly. So right now if you go to the developer.id.com slash call for code. You'll see there's information on the yearly competition, as well as past winners. Each of the open source project groupings we talked about today are available here. And you can check about some of the stories of the existing projects that have been deployed, the background of the communities that are there, and any sort of other information that may be of interest to you. So let me just dive into the call for code for racial justice ones, which did have those three focus areas. So you can you can kind of see how they're grouped. And you can check out the videos, things like that so lots of resources really want to make it easier for easy for anybody to contribute and and bring these projects forward and really make an impact on the world. So with that, Demi did you have any other final parting thoughts. I know by the way this is pre-recorded in advance but it will be played during the time slot during open source summit. So we'll be around to answer any questions if you have them live there and you can find the slack. If you don't find us there, we're definitely around. But we do want to make it as easy as possible for you to take part in these communities. Demi any parting thoughts. No just looking forward to your questions and we look forward to seeing you in the community. Excellent. All right, well with that, much appreciated. And we look forward to your contributions technical or otherwise. Have a great day.


Okay, hello everybody. My name is Irie Olsha. I work for Red Hat. I maintain the Perf for Reset Enterprise Linux and I also do some work upstream. I will be talking in this presentation about processor tracing that we support in the Perf. In the nutshell, it's a feature that allows you to go from functional level detail that you might note, for example, from F-trace, which is this output, and you can go all the way to the instructions. So that's what processor trace is allowing you to do. It's basically you will get a total overview about your workload. You will get all the instructions that were executed in your workload. As maybe some of you might notice, this is the x86 instructions output, and this is basically the scope of my presentation in Perf. We have actually like global framework for processor tracing, and we support other architectures. Also, we have implementation of processor tracing for S390 and ARM, the core site. However, I will be covering only x86 and namely the Intel, Intel models and features today. So it gives you the, the processor trace gives you the instructions, instructions flow. For the sake of this presentation, you don't need to be familiar with the assembly with small exceptions, and those are branch instructions. Because that's what the processor trace and other related features are based on. I will just really quickly explain what are branch instructions about. Basically, they control the flow. They whenever the CPU reach the branch instructions, it allows to see the CPU to jump to another portion in the memory. So basically they are used for conditions, for function calls. There are two types. It's really, it's either unconditional jump. This jump will happen every time that CPU reach the instruction, or there are conditional jumps and based on the type of the jump and based on some flags inside the processor which are checked before the jump, the CPU either jumps to a new location or not. So those are branch instructions and they are really crucial for processor tracing. I will start with really fast and quick example to keep you interested. First of all, you need to find out if the processor tracing is available on your system and the best way to do that is to do the perf list and look for the Intel underscore PT. Basically it's Intel underscore PT is basically the PMU itself that provides one single event, Intel PT and this is the event you want to use when you use the processor trace on the Intel. I have this sophisticated example.  So the program was running. In the perf data, you have now the trace of the program and now you can display what you actually traced and that's what we have the I trace option for. If you want to see all the instructions, you can say I0. Ah, no, it's too small. Okay, I will use the error. This output basically gives you for every instruction, you will synthesize instruction event sample. So every line now stands for instruction. You can display more details for every instruction. You can display the actual instruction that got executed and you can display it very nicely in the hex dump or you can translate the hex dump and this is basically all the instruction that got executed when you run the X command that I was showing in the beginning. So if I go and search for main function you will find out basically the routine that you wrote. This is the preparation for for Lipsy calling and all around the main function is basically the dynamic loiter and the Lipsy glue. So if you wish, you can check all that code and it always amazes me how much of the code is executed just for 12 instructions. So that was just a quick example. Before I actually go and introduce the processor trace itself I will introduce two other features that are like the predecessor of the processor trace because processor trace in Intel didn't just appear by itself. There were some development. One of those features is actually dead-end. This is the branch trace door. Last branch record still continues and people are using it. Let me start with that. Last branch record is basically the CPU feature and CPU allows you to configure the CPU in a way that it will remember the last branches that it executed. So at any given time you can ask the CPU how did you get to this point? What were the branches that you executed? And that's basically how we use it in the perf. Whenever there's the sample for event like for cycles, when you're profiling on cycles you have the sample and at the point of sample we are gathering all the information like the processor ID, thread ID, CPU number, all those information and one of those information is also the LBR record. So we asked the CPU what were the branches? How did we get here? And that's basically what we are storing for every branch. We store the source and destination address and we also store like the details about the jump which is like the usual flex that you want to know about the jump if it was predicted, mis-predicted, if it was part of the transactions and we can also get the number of cycles that were spent from the previous jump. Quite some quite of useful information to find out if you have some support on LBR on your system, which you probably have because it's kind of all this feature, you just grab the kernel log for LBR and you should see something like this. The 32D is basically the number of jumps that the CPU allows you to store and it's different for every architecture for there's the small table that gives you the idea. I included this structure. This is basically the structure that's used in the kernel to store the LBR data for the sample and you can see all the flex that we actually store for the jump. So mis-predicted, predicted, abort in the transaction and how many cycles. You can see there's the reserved field. So it's actually not a dead feature. For example, these cycles were added like for Skylake architecture, so I don't know, three years ago. So it's kind of still useful. It's still there. People are using it. How we can actually use it? PerfRecord-B will instruct the Perf to store the LBR data for the samples. PerfRecord-J is basically the filter. It does the same as B, but you can say the type of the jumps. So it's supposed to be faster because the filter is set in the hardware. So you will get faster workload. We have actually support for call chains. Also, when you say call graph LBR, the standard frame pointer call chains will be mixed with the LBRs and it will get you just better call chains. Once you have the LBR data in your profile, you can run Perf report by default. It will actually show you the most used branch. If you, there's very nice options, branch history, it will actually show you the profile and the branch, the LBR data will be shown in the call chain. So I recommend using those features. How is it with the performance of LBR? I did this really simple benchmark. So the first record is just the profiling of standard schedule pipe bench that we have in the Perf. So the baseline is like this 58 seconds. If you include the dash B, it will slow down almost four times. The call graph LBR is actually quite nicely fast because it does just the subset of the LBR. It doesn't delay the CPU that much. So that's actually really, really usable. So that's LBR, actually just instrumenting the samples with the branching course. The first feature that actually did the whole storage like that stored the whole trace of the branches is branch trace store BTS and basically it's a CPU feature. Of course, it allows you to configure the memory, debuffering the memory and once you configure the BTS, the CPU will start dumping the information about every branch that it's the execution going through. So for every branch, it's actually storing 24 bytes. Source address, destination address and eight bytes for one flag. As I said before, LBR feature is still ongoing, people are using it. BTS is actually just a dead end. It's just something, at least for Perf, something that predates the processor trace. It has some common functionality but people are now using the Intel processor trace. This is like that feature. However, how did that feature work? There are two ways how we can actually enable it and it's by using different types of events. First event is branch instructions with the period one. That's the dash C one. If you use this instruction, the kernel will notice and transparently, it will create the BTS buffer and the CPU will start dumping the data to the BTS buffer and there's a mechanism when the BTS buffer will be full, it will be dumped and that's where kernel steps in. It will parse all the BTS buffer and for every record, it will create event, branch event and store to the Perf ring buffer. Perf ring buffer will have all the branch events from the BTS and the decoder is the kernel. If it sounds expensive to you, it's because it's really expensive. It's really horrible from the performance point of view but it's working. The other way to enable the BTS is using the Intel underscore BTS event and by using this event the kernel doesn't step in. The record, the user space site that's monitoring the data that CPU is generating will actually take all the data from the memory and dump it to the data file. So you will basically end up with the Perf data with mixed data from the BTS buffer and from the Perf ring buffer. So no kernel is involved. Nobody is translating the data in the kernel. That means the decoder actually moved to user space, which is a nice thing. It's supposed to be much faster and the decoder is basically hidden behind the dash dash iTrace option. Basically, by dash dash iTrace, you're saying I have the data with the hardware trace, in this case the BTS. Synthesize from this data the thing that I will tell you to synthesize and you can synthesize all this kind of data. For BTS you can synthesize only the branches and the cores. All the other options are there because this is actually the option for the Intel processor trace. So this was like the evolution of the Intel processor trace. For BTS you can synthesize only the branches. All the rest of the options are available for Intel processor trace. I will talk about it later. So to sum it up the branch instructions with the period one works. It's not fast. The Intel BTS events is actually disabled for PTI feature, page table isolation feature, which is probably enabled on most distros now. So most likely you will not see it on your computer unless you recompile your kernel and disable it. It's not well maintained at the moment. On top of it, the BTS does just the branches, nothing else. So you will just get the overview of how the branches were executed. There's no timing information and you cannot do system-wide monitoring. You can do just attach the event to the single thread. That's all you can do. I mentioned the overhead already. So there's the Intel processor trace to save the day because it really has much lower overhead. Mostly because it's using the dump of the processor trace is basically very highly compressed data. At some point it's even like one bit for one branch. So it's really a really powerful compression. And moreover it's actually adding more information to the trace. So you get the timing information and you can get some information about the power, like the frequency of the processor at any given point. It has the similar functionality from the top-level overview like the BTS. So again, there's a buffer that you need to configure for the CPU and when you enable the feature, the CPU will start dumping the information to that buffer. As I mentioned, it's not 24 bytes for every branch now. It's actually the dump is the processor trace dumping the information in form of the packets. Many many different packets that form some sort of protocol that describe the flow. And the decoder is the guy who actually is on the other side and trying to decode that thing. How it looks like? In a nutshell, this is like the basic started sequence of the packets that decodes the counter flow. The first thing you need to know about the processor trace data is that the decoder doesn't need to know where the data begins. It gets to the data from any level, from any starting point and it search for PSB packet, which is like unique. And when it reaches the PSB the protocol is that the following data will actually set up the trace. It will set up two informations, it will set up the timing, that's the TSC packet for and it will set up the virtual address that the CPU is currently executing on. So it's of course much more difficult, but this is the idea.  When you are ready to decode, there's the most common and most important packet in processor trace. It's TNT, it stands for taken, not taken and it's decoding the branches. Basically for every conditional branch in the trace, you have one bit in this packet. So the decoder needs to have the binary, it goes through the code and whenever there is a conditional branch in the binary, in the sequence of instructions, it consults the TNT packet and based on the value, if the branch was taken or not taken, it either jumps to new location or continues executing further. So at some point you will see the dump. I will show you an example of just the TNT packets describing the flow. So that's the basic idea, that's how actually the packets look like. Again, it's much more complicated. There are many more packets about saying the state of the processor and the power state. This is just simplification to show you how it works. Back to how we actually store the data. It's similar to the BTS. BTS trace. For Intel processor trace, the buffer is called TopaBuffer and we deal with it like with the BTS. So we store all the data from the TopaBuffer to PerfData together with the events from the RingBuffer. Again, that means that you need to have the decoder on the user space side in the report and actually in the script as well. Again, the decoder that will decode the data for you is behind the itrace option and for Intel processor trace, all those options are available for you. So you basically say I have the data, I have the data with processor trace, synthesize this kind of information for me. Synthesize instructions. You can say also the period of instructions. I will show you an example. You can synthesize the branches as in BTS, transactions and power events and some other helper option when you try to debug the things. Example. Let's go to examples. So I can basically start from where I left from the previous example. So in PerfData now I have the profile of that simple X example. I have the profiling data from running that. If you actually want to check how the trace looks by itself, you use the Perf capital D, which means we will give you the raw output of the data file. And if you search for the PerfRecord AUX trace event, this is the event. This is actually the raw data that we got from the CPU. I'm not sure if it's visible enough. Perhaps not too much. Better? Yeah, you must believe me it's there. So basically there's the PSB sequence and all the stuff and now you would see all the TNT packets here. Really, you cannot see it? Never mind, it's there. Okay, another example. So the key to get the data from the processor trace is the iTrace option. That's actually the decoder of the data and we have really nice main pages. We suck at the main pages but for processor trace it's really good. So if you go for report or script and search for iTrace, much better, right? Okay. It will give you really the whole scope of what you can do with the processor trace data. So as I said, basically you tell the decoder give me, if you use the i, give me all the instructions. And the beauty of it is that you can specify also the, you can specify the period. And the period can be much slower than you could actually specify for the hardware event. So if you use Perf report and say give me all the instructions and the period I want the instructions to be at is 2 nanoseconds. You will basically get the profile which like the profiling tick is 2 nanoseconds. So this is basically you can change that and change the profile. You will get different amount of instructions events for every period. But that basically allows you to do like the profiling after the real profiling. You can do some sort of offline profiling and you can use really small periods. So for small workloads, it can show you some meaningful data. To actually see the whole flow, I was using this iTrace with the instruction of zero. So this will give me all the instructions. We have several options that we encoded to standalone options which you can use. So if you say call trace, it will configure the decoder to give you just the calls and the returns. And it will basically show you the call graph of the execution from the processor trace dump. So if I go to see the main function, you will see that it's just the main being executed over here. It goes for the F write. You will see the dynamic loader kicking here and they actually I write over here. We have some variation on those options. This one will show you the call graph together with the instructions, which is really helpful sometimes to get an idea of what was actually executed under that functions. You can also trace the kernel, of course, and you can trace it system-wide. The only thing you change on the command line I was using is that instead of... Okay, I don't do it like here. So you go perf record, Intel, PT, and instead of U, you say K. You say system-wide and you say you want this for one second. So now I have the one second processor trace of the whole system. You can run perf script saying call trace, and this will basically get you the idea of what was executed during this one second. There's a little glitch in tracing the kernel. The thing is that the kernel is not static. We have all these jump labels things that basically changing the kernel code on the fly. And the decoder is going through the debug info. So at some point, the decoder will not recognize it will not match because the executed code is actually different than we have in debug info. So basically you will get errors like trace doesn't match the instruction. We have workaround on that. In the perf RPM, we have the following script. Perf with K-core. Okay, maybe I can make it huge. Perf with K-core. What this will actually do for you is it will run the profile the same way as if you wouldn't use it. But at the end of the profile, it will store the K-core information from the kernel. So we will get actually the image of the code that was used during your profile. And it will also use it in a way that it's available for report and script. So the example is like here, I need to remove previous data. Okay, you say record the name of the data, the event for kernel, system-wide and for one second. It's recording the data and at the end it will store the K-core for you. So if you look to the data, it will create actually the directory. There's the data together with the K-core information. And when you run perf script, pt, callgraph, callgraph, script, pt, calltrace. So you will get now the correct trace without any errors because we are using the K-core to resolve all the data. One last thing I'd like to mention before the end is the snapshot mode. Basically, even though the data is highly compressed, it's still a lot of data if the CPUs are busy and you want to trace for a long time. The guys from Intel introduced snapshot mode which basically it will set up everything as for the not snapshot mode but it will not store the data. It will just wait for you. You need to send the signal to the record command and once the record gets the signal, it will dump the data to the data file. So by this you can actually run the trace and there are still some data that gets stored to the data file. It still will be populated. We have some helping events to track the scheduler, like who is running at the moment. So it will still go but it's not so much and at the point you actually want to store the data, you send the signal and you will get the amount of the data in your data file. From performance point of view, this is the comparison, it's the same benchmark I use for LBR. I didn't measure the not snapshot mode because I wouldn't have enough space for this bench. It's really a lot of data but if you do the snapshot mode, the performance is even slightly better than for the call graph LBR so in this mode it's really usable. Even let it run over your workload, it's actually not that bad. I will skip that one. Just to give the credit to the guys who make it, those are the Intel guys and again it's lower overhead, you can do the system wind and there's the timing information and that's it from me actually. There are any questions? I heard something that, it's supported in the KVM now but I don't know the details, it's not my area. Sorry, I repeat the question. If there's the support in QMU, yeah, sorry, I don't know. So the question is about the support for S390, the Z. It came just a few months ago so I believe it's there. I've never used it, I don't have the hardware and it's coming in, the patches are coming in, it's being egg so I guess it must be in some shape that is actually working so never is it. Cool. Thanks.