Okay, great. Yeah. So I'm excited to be here today to talk about actually a new open source project that we've launched at Databricks to accelerate the machine learning lifecycle. So as you saw in Professor Rodi's talk, machine learning has huge potential to transform basically everything we do from our day-to-day interactions with technology to going to space and so on. And it's obviously a very exciting time to be working on that. But at the same time, there are a lot of challenges with turning that into reality, into even working computer applications, let alone say a working spaceship. And this is what I want to talk about today. So first, just a little bit about Databricks and my background. So I've been part of this company Databricks since we started in 2013. It was founded by the original creators of Apache Spark, the team of researchers that started the project at UC Berkeley. And we provide basically a unified analytics environment in the cloud for everything from data science and engineering to machine learning. So our customers work across this whole set of technologies and problems and we get to see what the problems are. And we have customers in a wide variety of industries, including a lot of customers now in Europe. And so working across these customers, we see some of the common trends and problems they have with using machine learning and big data and production. So the main thing that we see, and I'm sure everyone else who has started doing that sees the same, is that machine learning development is actually extremely complex today. And it's more complex than traditional software development because of specific challenges that come up in machine learning because of its probabilistic nature, because of problems such as overfitting, because of the many parameters and so on. So to illustrate some of that problem, I'm just gonna show a picture of the machine learning lifecycle and explain some of the issues that come up. So here is kind of a simplified machine learning lifecycle to show the issues. So first of all, machine learning begins with data. By definition, machine learning is computer software that learns or improves or is all based on data. And of course, machine learning works better the more data you have and the more diverse and high quality it is for the problem at hand. So that's kind of the first thing you need. Second thing you need is data preparation. That's where a lot of big data technologies come in like Apache Spark to actually get the data into a shape usable by various algorithms. Then you've got, of course, the machine learning training. And finally, you have deployment where you actually have this run in production and monitor it and maintain it. And then if the machine learning application is doing anything important, you probably want to collect more data about how it's doing in deployment and feed that back in, as we saw, for example, with all the measurement systems on the spacecraft. So what are some of the challenges? So the first challenge is that there are just a lot of tools involved at every stage of this process. For example, data can be in a huge range of storage systems that are available today. And again, in machine learning, you usually do better with more data. So you want to combine all these sources into your application. For data preparation, we've got a huge range of tools and algorithms. So things like Apache Spark, or SQL, or Python, or libraries such as Psycote, Learn, and Pandas, and so on that are available for that. For training, there's an even larger array of algorithms and libraries, anything from kind of linear models to decision trees to deep learning, and many different implementations of each one. And finally, for deployment, there are also a huge range of tools and environments where you want to deploy your model. Now, normally in just traditional software development, you've got a lot of tools to choose from at each stage of the life cycle, but you can just choose one. So for example, for your web application, you choose one database, you choose one web framework, one JavaScript framework, and you're done. But the problem in machine learning is that your goal is just to produce the best result possible. So your job is actually to try out all the algorithms and keep improving the model and try to get the best result. You can't just say we'll be using linear models and that's it, and we'll be using one data preparation step. So unlike traditional software development, you have to be able to run and interoperate with all these tools. So that's kind of the first problem. Second problem is that some of these tools have an additional dimension which is tuning, so they have many configurable parameters or hyperparameters. And this basically adds a new dimension to your life cycle where you have to figure out, well, when I ran with this processing step, what did I set these tuning parameters to? And that's very difficult to keep track of and it needs to be a first class concept in your workflow. Then the scale, all of these steps need to operate at high volumes of data or they will operate at high volumes of data in production. So all of them have to be designed to scale. Then there's actual model exchange or code exchange across these. So how do you get some data preparation stuff that's written in Python and Pandas to then be deployed in production when you've got an iPhone app or something like that? That's the type of challenge you need to deal with is passing this code and models between teams. And then the final issue you have is governance. So anytime you have a large company or a company in a regulated space, you want to be able to keep track of what you did, make sure you're not using features that you're prohibited to use and so on. So these are all challenges that you have to deal with in machine learning that maybe don't come up as much in traditional software development. So just some examples of this from a couple of customers that talk to us. So this is actually the chief scientist at an ad tech firm. This is a smaller company, but this person runs sort of a data science team of a few data scientists there. And they say I built hundreds of models per day to lift revenue. The goal of this whole goal of the team is to increase the company's revenue by predicting things about what people will click better. And they'll use any machine learning library, MLlib, PyTorch, R, etc. So they're constantly trying new things that are coming out, new algorithms to see whether they help with the problem. But unfortunately there's no easy way to see what the data went into a model from a week ago and rebuild it. So they have this whole problem of keeping track of what they did as they're using this wide range of tools. And similar problems happen in much larger companies as well. So this is a large consumer electronics firm. And they say our company has 100 teams doing machine learning worldwide. And we can't share work across the teams. When a new team tries to run some code that was written by someone else, it often doesn't even give the same result. And they have lots of horror stories where you've got experts in one country trying to look at a model built in a different country and they can't even get it running. Let alone actually use their expertise to make it better. Okay, so what are people doing about this problem? So one trend that's really begun in the industry and it's happening in sort of companies of all scale is the idea of machine learning platforms. So some of the best known ones are these internal software systems at large web companies who have written about them and explain how they use them. So some examples are Facebook FB learner, Uber Michelangelo, Google CFX. But many other companies are building platforms as well. And the idea of these ML platforms is let's put some structure, some APIs around the ML development process so that we can make certain guarantees about it. Like we can make sure that we can deploy the same job in production or we know how to move the model into a new platform or something like that. So the benefit of these platforms is that they standardize the machine learning development life cycle. So data preparation, training, and deployment. Basically as long as you work within the APIs that the designer of the platform built, they make sure the engineering team that builds it makes sure that your code is deployable in production and can be monitored and can be retrained and so on in a standard way. So you don't need to reinvent that each time for every new ML application. And this is a very successful model in these companies. There are basically dozens of applications using these internally, which is great. But there are also some limitations. So one of the main limitations we see talking to users of these systems is that they're limited to the algorithms and frameworks that the platform supports. So there's often an engineering team that supports maybe 10, 20 different algorithms or maybe one specific framework and one specific software version. And if you work with that, everything is great. But if you wanna go and try something new, you're kind of on your own. And remember I mentioned at the beginning, when your job is to improve some metric in the company, like every percent improvement leads to millions of dollars of revenue, you wanna be able to try everything that's available, every new technique and see how it helps with that. And then the other challenge is that each platform is tied to each company's internal infrastructure. And so it's very hard to move things around even to change infrastructure internally. So we looked at this problem at Databricks and starting at the beginning of this year, we asked ourselves, can we provide some of the same benefits of these platforms but in an open manner, both open source and also open in that you can bring whatever technology you want into these systems and have it work by default as opposed to being limited to whatever the system decided to support. And so that's what we're doing in this new project called MLflow. It's an open source machine learning platform and it's also designed to be open in terms of the interface, in terms of what can run in it. So it's designed to be able to work with any machine learning library and language, it's not limited to a few things. And to run these the same way everywhere or in particular, we are a cloud company running in the public cloud. So we wanna make sure things are on the same in every public cloud. So you're not dependent on one provider. And it's also designed of course to scale to big data and to integrate well with Apache Spark because that's something that we know how to do well. We launched the project in June, so it's still a pretty new project. But it's actually grown quite a bit as an open source community. So we already have over 50 contributors to the project, many new features that were built both by us and by community members and at least like a few dozen distinct companies contributing code to the project. So it's exciting to see that people are interested in this idea and to see what's happening with them. So I'll talk a little bit about how MLflow is designed and then I'll go into some of the details. And we actually have a more kind of hands on talk on MLflow tomorrow at the conference as well. So there's kind of two elements of MLflow's design philosophy that also make it a little bit different from some of the company specific ML platforms. So the first is that it's an API first or what we call Open Interface Platform. So we wanna make sure you can submit runs of your code, models, experiments and so on from any machine learning libraries and language. So that by default MLflow just kind of works with whatever technology you choose as opposed to you blocking on us to make something available. So one really simple example of this is think about deploying a model. What is a model? There are a lot of formats for models like Onyx and PMML and so on that try to capture all the weights and internals, but acting the output and use that to improve your model as well. So very kind of simple concept. It's like let's log what we're doing, keep track of it in a database and then be able to see and compare the results. So that's kind of the first part is MLflow tracking is just this API. The second part is MLflow projects. So this is a way to make it so that someone else can run your code again. And this is again a very simple design. It's this little spec file you can include with your project and your Git repository that says how to run it and can declare an environment and dependencies. So if you develop your project with this type of spec, you can then push it and anyone can run it again later. And one of the great things about this is that different people can be running it later. So to run a project, you just need to type MLflow run and pass in a URL for the project and then it will just be able to execute. So what does it look like? Just very briefly, your project is just a directory with some code, maybe data files, whatever you want to put in it. It can live in a Git repository. And you have this little descriptor file that describes an environment, such as required Python libraries and also describes parameters to the project and how to call the project. So anyone can call it just by passing these parameters and it's sort of self-documenting how to run this on new data or how to change the parameters. And you can run this either from the command line or using an API to have a script that sets up a multi-step workflow. And then the final component is models. Again, it's actually a pretty simple concept is basically a way to package up models that can expose different APIs, which recall flavors for doing inference using the model. So whatever library you're using for training, you can package the result into an MLflow model and then we provide built-in tools where you can take any model and deploy it as a REST server. For example, on Docker or Kubernetes as a batch or streaming job, for example, in Apache Spark. Or just as code that you can call in a programming language. And again, the design is each model is basically, has got some arbitrary files that you saved. Each model is just a folder with some contents and it has this descriptor file that says how to load it into different formats. And in particular, one of the formats is just load this as a Python function. So any tool that understands Python can just apply this. Other formats are more library specific. So for example, this one you can load as a TensorFlow graph as well. So if you're back in TensorFlow, you can actually load it back in and modify the graph and do things with it. So this simple descriptor format also captures when the model was made and stuff like that. So what does this mean for deployment? So now, instead of the data scientist throwing lots of different frameworks over the wall at the engineer, the data scientist just say, hey, please deploy this MLflow model. And there's a standard interface for using it in various modes. So the production engineer can just plug it in and do either real time or batch scoring with it. And likewise, if you want to run your training code in production, you just say please run this MLflow project. The production engineer doesn't need to know what's inside it, even what programming language is it in or anything like that, because they just have a standard interface for running it. Okay, so that's sort of a brief overview of MLflow. So since the project started, it's been actively developed. And we've got a lot of new features since our first release. So we've got both we and different people in the community have contributed model packaging interfaces for a lot of common libraries. So you don't have to write that descriptor file yourself. You can just call a function to save your model and then all the MLflow deployment tools work with that. RStudio contributed an RAPI. We're really excited about that because they're very good at building really excellent RAPIs and is great to see that in there. We've got a Java and scholar API in addition to R and Python. And we've also got a pluggable storage backends for a wide variety of storage systems. And just a couple of days ago, we released the next version MLflow 0.8 which brings more improvements, especially around the UI visualizing multi-step workflows, compact table view, and also letting you take your model and deploy it to Azure ML serving workspace, which is a new serving product on Azure based on Kubernetes. So basically all your previous models can now be pushed to this as well. Some examples of use cases, so MLflow is open source. There are people using it through that. And we're also working with a number of Databricks customers to develop and try the system early on. So examples, European energy company is actually using MLflow tracking to build and monitor hundreds of machine learning models for entities in the energy grid. So this is a case where they want to model every power plant, every renewable energy source, every consumer of electricity as well, like a town or a factory. And so that's kind of hundreds of distinct machine learning problems. And they use MLflow to keep track of these and make sure that they have the best model and that it doesn't regress for each specific task. An online marketplace working with us is using MLflow projects to package and run reproducible deep learning experiments using Keras. So you develop the experiment on your laptop and then you can kick off runs and parallel on GPU instances in the cloud and compare the results and keep track of what went into each one. And then an online retailer is using our model packaging format to package and deploy a recommendation model with custom logic. So they found that it's not enough for them to just take an off the shelf recommendation model and serve that as a REST API. The data science team wants to write a lot of logic around that with business rules to customize the recommendations that are coming out. And they like that with MLflow. You can develop these together and package them together and ship one artifact to the production team that captures both the weights of the model and that version of the business logic that you had. So you don't have to split it up across many little applications to do this. Okay, so these are just some examples. So anyway, just to summarize machine learning platforms, we think are a very important new part of the data science and big data toolkit. And if designed well, they can simplify ML development for both data scientists and engineers. That's a key thing that we noticed is that if you've got the right abstractions, the data scientist is more productive just working by themselves and they're excited to use it. And at the same time, it makes it easier to deploy. Whereas today in a lot of organizations, these two groups at odds where the data engineers tell you, well, you can't use this, you can't use that because otherwise you won't deploy your model. MLflow is easy to install. You can just install it through PIP and start using it in Python. We also have a bunch of pretty extensive documentation and tutorials on the website. And if you're interested in seeing more about it, including a hands-on sort of walkthrough of using it with deep learning, Jules Damgee from Databricks is presenting this tomorrow as well. So thanks a lot and happy to answer any questions. Hi, it's time for the question. Yes. So if there is any question, it's time to ask Matei. If not, I have many questions. Okay, yeah. You're facing like, please, no, Jules, no questions. Yeah, I don't know, yeah. Okay, I actually can't see you guys. Unfortunately for you, you are moving the hand, you can't see. It's hard because of the lights, yeah. Yeah. Now the microphone arrived at time, yes. Thanks for the talk, Matei. Yeah, no problem. I have a question about the integration between MLLip Spark and MLflow because with some friends, we tried it and it's very nice. But what happens with these models are called in MLflow in Scala because I think that the main of the IPA is in Python. Oh, yeah, that's a good question. Yeah, so I think loading the model in Scala right now, you'd have to do it manually from the serialized code for it. So that's actually something that we're working on to just provide a Java and Scala API. So that part is just we're just not done with it yet. You have the roadmap to integrate an active way to do it, right? Yeah, we will, yes. Yeah, a lot of these integrations are still in development. So that's definitely good feedback. I'll pass it back to the team, yeah. Thank you, thank you so much. Yep. Until the next question arrived, I have one. Okay. What do you think that is going to be the future for MLflow in the next few years? Yeah, that's a great question. Yeah, so actually the pieces I showed today are kind of just the beginning of what we want to build. So we also are planning components of MLflow on monitoring models that are deployed and also a component that's basically a model registry. So it's kind of like a centralized repository where you keep track of the models inside your company. And we've got a lot of interesting ideas and feedback there. So our goal is really to build kind of a complete platform that works together. And we try to design these pieces. So these are just things you can add on to the current workflow. So even though the monitoring and so on aren't available yet, we were kind of thinking about them since the beginning so that they will work with the existing workflows. The other thing we care a lot about in general at Databricks is just making sure that the platform is very solid. So we're moving, we're going towards MLflow 1.0 version which will have basically a stable interface so you can build on it. And it keeps working the same way each time we upgrade it as well. So that's probably something you'll see basically early next year or towards the first half of next year. So we have, yeah, we're excited to keep building this out. I have more questions. Hi. Another one. No, I'm here. OK. The other. Right. Right. There you are. OK, great. Sorry, it's very hard to see. Thank you. It's really simple. OK. How many companies are using this tool at this moment? How many companies? And how big they are. Yeah, that's a good question. So the companies I talked about before, well, one of them was a smaller one, the retailer one I talked, but they're large enterprise companies. And there was a European one and a US one that I talked about before. And we have a number of other customers as well that are using this in life sciences and in finance and in other domains. In terms of the community, it's hard to tell. We actually want to run a survey of this to figure it out. But the contributors, as I said, there's around 50 total contributors to the project. Actually a little bit more now. And our team that works on it is six engineers plus myself. So we have. Yeah, exactly. So it's a lot of non-databricks contributors. And usually, if someone submits a patch, it means that they've at least tried it out to the point that they kind of want to use it and make it better. So I don't have an exact number. But we've seen, I think, if you look at the contributors, they're from at least 20 different external companies that are contributing. This is a good moment to have more contributors for you. Yeah, we love to get contributors. By the way, if you're interested in looking at it, it's very easy to contribute to. So it's mostly Python, the server, all the server bits. And then the fun end is all react. So we've had, actually, a lot of the features I showed, like the note taking and the scatter plot, were done by people outside of Databricks. They sent the patch. They actually worked with our UI designer who just talked on GitHub about how to design it. And then they put it in. And that comes from the design of the platform, too, to have these very clean APIs and data model where you can just add this stuff on top. Yeah. Great use of the audience. OK. Any further questions? OK, there's another one. Yeah. Go, go, go, go, go, go. Thank you for your poll talk. Is there any plan for .NET interface or API? Yeah, that's a great question. Yeah, we'd love to have .NET eventually. We're not working on it right now. But I think it shouldn't be too much work to add. So it's definitely something we'd love to have. Yeah. OK, thank you. That's because my company is a serial latency. So we can't use a rest API to our Kim online. So that's why I got it. Got it. Thank you. OK, yeah, it makes sense. OK. As you can see, he's really passionate because all the questions are really good, great questions. So he always says, great question. And it's cool because he loves to talk about MLflow. The last one, or you can talk with him after in the Ask the Esper if you have a question that you want to say, not with the big audience. Of course, you have the APP in the App Store and so on. You can go inside, download Big Data Spain and all the information, all the speakers, and all of that you have over there. So try to do it right now. If there is no more question, big applause for Mati. And thanks so much. It's been an honor. Thank you.


So, Terry and I thought that maybe it would be worthwhile for me to discuss means by which we've identified causal variation in HIV disease, which is what my lab works on primarily. But I have to say that just from what I've heard, I think you have all identified, you've really honed in on the allele in each case almost. And not only that, there's some really nice work to show why that specific allele is having the effect. So, I feel like I'm going to be preaching to the choir and that probably you've moved ahead of many of the areas in which I work, so I just wanted to make that clear. Now, the human major histocompatibility complex is the most rich 4-megabase region of the human genome in terms of association with different types of diseases. And some of those are shown here. There are a lot of different types, autoimmunity, cancer, viral disease, bacteria, and even some oddballs like schizophrenia and Parkinson's, which we didn't expect to be involved with HLA class one or class two. Now, I don't have drug hypersensitivity up here, but I really need to go back and do that because in most of the, in the vast majority of these cases, there is nowhere near the odds ratios that you are seeing in association with any of these diseases. So, in HIV disease, we feel excited about an odds ratio that goes above two. So, anyway, I will definitely get drug hypersensitivity up here and it's really, I think, an important point. Now, HLA variation, we think of it as being really important in antigen presentation. So this is just the peptide binding groove of a class one molecule with a peptide sitting in the groove. And we know that a lot of the effects that we see when we look at disease actually are due to the types of peptides that sit in the groove and the ability of individuals with specific alleles to respond appropriately to a given pathogen, for example. But there are other characteristics of the class one molecules, and I'm sure this is true for class two, though we haven't studied that as closely, that really should be considered when you think about the role of variation at class one in human disease. And one of those is the ability of the peptide binding groove to bind receptors on innate immune cells. So these lillers and the cure, I won't have time to talk about those there encoded on chromosome 19, the MHC is on chromosome six. These molecules that are expressed on innate immune cells such as natural killer cells and certain monocytes and dendritic cells, they actually bind, their ligands are HLA class one. It's this region of the peptide binding groove that the cure receptors recognize. So they see this region and they can send activating or inhibitory signals to NK cells, for example, and affect the innate immune response. So it's not just the acquired immune response, it's also the innate immune response. More recently, and what I will be talking about, the example I'll give today is their differential expression levels, which we think is having an effect, modifying the effect of alleles on different disease outcomes. And I won't have time to talk about this, but there's been a good bit of data recently to show that certain HLA class one and class two alleles actually serve as trans expression quantitative trait loci. For some reason, they appear to be associating with the expression of unlinked genes, other genes in the genome. And all of these things I think should be considered and probably a lot more than these in terms of when we start thinking about what is the effect of HLA on disease. So the example I'd like to discuss is HIV control, and we know there are HLAB alleles that have been shown consistently to associate with HIV control, and I've just put three of those up here. The HLAB locus is really key in terms of allelic effects on HIV disease. But recently, well, not so recently, but Jacques Faelet in David Goldstein's lab identified through a GWAS a variant between HLAC and HLAB that associated with viral load set point. And we followed up to find out whether in our cohorts we also saw an effect of that SNP, which is 35KB upstream of HLAC, but between HLAC and HLAB, those two are only 150KB apart. And indeed, we saw that there was an effect on viral load of this minus 35 SNP, what we call minus 35, where the TT associated with higher levels of viral load and CC with lower viral load. And this agreed nicely with the data from David's lab. We also showed in normal donors that TT associated the same minus 35 SNP associated with low expressions of HLAC and the CC with higher expression. So was there a connection between these two? Higher expression levels associating with CC also associated with lower viral load. What did the effect of minus 35 have to do with expression levels? And I won't, I'm going to give you a summary of what we did. Now I'll say minus 35 is upstream, 35KB upstream of HLAC. It's not in the promoter region proper. And the effect was in whites, and the effect was absent in blacks in GWAS. These things made it very unlikely that minus 35, which was identified through GWAS, to be the causal SNP. Now we started looking at promoter variants and 3-prime UTR variants, and I'm just going to give you a summary of what Smitho Cochrane identified. In the 3-prime UTR of HLAC, Sumoleos of HLAC, there is a binding site for a microRNA. MicroRNAs are known to down-regulate expression of certain messenger RNAs, either by degradation or decreasing their translation. What Smitho found was that Sumoleos have a mutation in that mere binding site, whereas other ones have the appropriate binding site for this microRNA. This variant in the 3-prime UTR in whites is in good linkage disequilibrium with minus 35. So now as I mentioned, HLAC is very close to HLAB, and one very large concern was whether the effect that we see with minus 35 or even with this variant, and I'll get to that, could be due to HLAB. So I'll come back to that in a second.  They have higher levels. Now the next part I'm going to tell you really eliminated the possibility that this was due to HLAB, linkage disequilibrium with HLAB. The microRNA gene is located on chromosome 7, unlinked to the MHC. Having an insertion at this position in the microRNA associates with low levels of microRNA, Smitho showed. Having the deletion mutation, the deletion variant in this gene, associates with high levels of the microRNA. This variant associates with HIV outcome, but only among individuals who have HLAC alleles that are inhibited by the microRNA. In these individuals, this variant has no effect. So this eliminated the possibility that this was due to HLAB because HLAB is not regulated by this microRNA. Now, so the minus 35 SNP is associated with HIV control in GWAS in whites, but not blacks. That SNP associates with C expression in whites, but not blacks, the minus 35. And we think it's unlikely to be the causal SNP. Variation in the 3-prime UTR of HLAC that's in LD with minus 35 in whites that determines microRNA regulation affects HLAC levels and associates with HIV control in whites. Among those who have C alleles that are regulated by the microRNA. A variant in the microRNA gene interacts with the 3-prime UTR of HLAC. This strengthens the causal effect of HLAC 3-prime UTR and rules out the possibility that simply marking the HLAB locus. Now, and I've got to zip through this because I know I'm running, I'm going to be running late. Richard Aps in the lab went back and looked at HLAC expression levels across individuals, each dot is a different individual dividing them by the HLAC types. And there was a significant association of expression across HLAC types where C1402, for example, is expressed significantly higher on the cell surface on average than is, for example, C0302. If we use C expression as a continuous variable in our analyses, okay, so now we know the genotype of HLAC, we can assign expression levels by imputation, and we can ask, using HLAC expression in our cohorts as a continuous variable, we can ask whether it has a significant effect on HIV control. And this is a much more direct way of doing them, for example, with the minus 35 SNP. And indeed, higher HLAC expression associated significantly with better control of HIV. And it did so in various disease outcomes that are shown here. Some of these are completely independent of one another. And as you have seen as well, you're seeing effects of specific HLA alleles across distinct cohorts, and that I think is really an important point. We're seeing this also in African Americans, so across different populations, HLA, higher HLAC expression confers protection. We see it across different outcomes of HIV infection. We see it across populations. And finally, we think it's quite important to look at the functional significance of any genetic association, and I think in general this group has done that and is beginning to do that for more of the drug hypersensitivity studies. Here we looked at the frequency of HLAC restricted responses to HIV peptides and found that it correlates with HLAC expression levels. This is a complicated slide, actually. Philip Goulder at Oxford University had data from over 1,000 individuals from South Africa where they looked at overlapping peptides across the HIV proteome, and they knew exactly what CTO responses were present in each of those 1,000 individuals. We asked then, so for example, this dot represents individuals who have C14, C1402, remember, I told you that's a high-expression allele and that's what's on the x-axis. On the y-axis is basically the odds ratio of a CTO response against C1402 restricted epitopes in this case. In this case, these are the C0102 individuals, C01, and they would restrict different epitopes because this is C01. Basically what this shows is that the higher the level of expression of the C allele, the greater the likelihood that individuals with that C allele would respond to their specific HLAC restricted epitopes. Higher expression results in a greater CTO response. It would be actually interesting to look at expression levels in the context of some of the cohorts that you've mentioned today and yesterday. What I've shown you is that measurement of HLAC expression levels across alleles allowed a direct test for association between C expression levels and HIV control where we take into account individual allelic effects rather than involving the minus 35 SNP proxy or a variant that only accounts for part of the differential expression, which is true of the microRNA. It does not account totally for differential expression of HLAC. The effect of C expression levels is consistent across black and white cohorts and across different HIV outcomes. Really important in validating any effect, any genetic effect. And then functional data explain or support the effect of HLAC expression on HIV control through enhanced CTL activity. It's possible that it's also having an effect on NK natural killer cell activity as well. So I'll stop there and what I've tried to do is just walk through an example where we've tried to really pin down what is the variant that's causing the effect. Which locus is it? What variant is it? And can we generate functional data to support that? Now be happy to take questions. So we have time for a couple of questions. Just one question. When you were doing your comparison with blacks, was that African-Americans? So I can speak louder? I'll speak louder. People don't tell me that a lot. Is that with African-Americans or was that also people from the continent? So the data that I showed you from our cohorts is with African-Americans, but the functional data was with South African blacks. And they are infected with two different clades of HIV, so the South African are primarily infected with clade C and Americans are primarily infected with clade B. So we think that this is going to be across the board. We think that if we looked in a Chinese cohort, we would see the exact same thing. Yeah. Thank you. Matt Nelson at GSK. Really interesting results. Is the effect of the different HLA alleles, B or C, is that simply, are those just a proxy for HLA C expression? Do those different allele types have an effect even after accounting for the differences in expression or are they just simply along for the ride? So it's a good question. And even given everything that I've shown you, it's not always easy to parse that out completely, but we do multivariate analyses where we include individual alleles, all individual alleles of A, B, and C in the analysis along with expression as a continuous variable. And out of those models, you get specific alleles like, for example, B57 still shows up, B81 still shows up. Alleles are well known to be protective alleles, and there's a lot of data because of the type of epitope they present. So we were happy to continue to see those, and B3501 shows up. So in this case, C1601 shows up even though we're using C as an expression of C as an allele, so we assume that 1601 has an independent effect, independent of its expression. I don't think it's so easy to totally tease all of those apart. Yeah. Thank you. Mary, you're... Thank you very much. I wanted to ask you. We talked about biomarkers for risk, and clearly the quantitation, the set of expression of the particular HLA that might be the putative HLA for, let's say, a skin reaction, the level becomes an important measure itself. So the question is, as a translational measure of risk, given that these levels are fluctuating over time in the context of infection and other things that are going on, is there a... How do you think about the idea of quantitating HLA levels? If you knew, let's say 1502 was a problem of carbamazepine, to determine whether there is a set point for those individuals who will then go on to really have a problem versus most people with the same marker, who will not if the quantitation becomes a key question and whether that can be measured as a translational measure? So when we do these measurements, it's actually... If you measure self-surface expression levels, and we did these from healthy donors, it's tricky because you need to measure them on a fax machine. Everybody that you're measuring, if you're going to compare them with one another, you need to measure them at the same time without recalibrating the fax machine. So the way we did this with 200 people is at Duke University, they brought the people in every day for a week and a post-doc just churned through them for that week so that we could compare those. Now you can look at CDNA and that way you can freeze, but for example, if you had, it would be interesting to look at people with 1502 and see whether those people who are tolerant of the drug actually express lower levels of 1502, and you can do that by CDNA. Once an individual has a reaction, there is not, I mean, that's probably massive immune activation in those individuals, so that is probably not going to measure their steady state levels of HLAB. You'd have to catch them before you have this massive disease going on, I think, yeah. Thank you. Because of time, we have to move to the next talk, sorry, 5th during lunchtime. Thank you. Thank you. So the next talk is on drug-induced liver injury and across reactivity, across drugs and organs, given by Jay Huffnagle, director of the Liver Disease Research Branch at the National Institute of Diabetes and Justice and Kidney Diseases at NIH.


What about COVID-19 and excess deaths? So I have this question from, and this is of course a big issue for RFK, but I've got this question from my listener, Jim Wilde. He says he's a long-time objectivist and he has a few questions for me. And one of them is, you know, he says, he's always been concerned about COVID-MRNA vaccines for they have never been severely tested on humans. That's just not true. And had never been tested on long-term effects, no drug is really tested on long-term effects if you think about years and decades there'd be no drugs on the market if they were tested on that. Indeed, no treatment is really tested on long-term effects. You'd have no innovation if you waited until long-term effects to rise up. So he's very suspicious of the uncanny number of people in my direct environment experienced great side effects including death within weeks. And in some cases, I was after getting the COVID-19 jab. Interesting fact, nobody in my extended environment, I know a lot of people has died following a COVID vaccine, nobody. I mean, nobody has even got really bad side effects from the COVID-19 vaccine in my, I know a lot of people. I travel a lot around the world. I meet a lot of people, nobody. Now I don't know if that's just me, that's something unique. I don't know, but we've all been vaccinated and nobody that I know of has had any serious side effects. Not saying that some people don't get serious side effects. Nobody that I know has died, nobody. But I keep meeting people who say, oh, I know a lot of people. And I wonder if you actually made a list and of the people they actually know if they named them and everything, how many they really are. Anyway, he says, never have you trusted big government or it's corporate disconies and big pharma.  That big pharma. And also of course, remember mRNA vaccines. What is the relation between mRNA vaccines and big pharma? Well, Pfizer, okay, Pfizer. But Pfizer didn't really develop the mRNA vaccine. Who developed the mRNA vaccine? Anybody know? I mean, two companies, Moderna, and it was called, I had it on the tip of my tongue a second ago. BioNTech, BioNTech. Is BioNTech big pharma? BioNTech big pharma? No, BioNTech is a small, biotech company was a small, made a lot of money, but it was a small biotech company out of Germany run by two Turkish immigrants, right? That's big pharma. What about Moderna? Moderna was a startup with really no product, no real revenue. Big pharma, wasn't big pharma. So COVID-19 vaccine was not produced by big pharma. Other than Pfizer, who teamed up with BioNTech, but the real science was BioNTechs, not big phamas. Anyway, don't trust big pharma, don't trust the CDC, considered media, censorship of Facebook and other media and all of that. Fauci and the rest of government here on the board spoken one voice at it to the famous actors, doctors, da, da, da, da. And then he gives me a million links to all the stuff, all the people who think that COVID vaccines are bad for you.  Excess deaths have increased because of COVID vaccines.  They've made gazillions of dollars for my New York Times writer, Berenson, I think, who made millions of dollars on this lie about COVID-19, about the vaccines. I don't believe a word they say. I don't trust anything they do. Again, I have watched their videos. Every time I've looked at the data, every time I've taken one of their claims and actually gone into the papers and looked at their data, they're lying or they're deceiving or they're misrepresenting, they're just wrong. Now, I can't do that for every claim they make. I can do that for every single paper. I can do that because they can arbitrarily make claims at infinitum. But after a few, after you discover somebody's wrong one, two, three times, I don't have to trust them anymore. If you discover I'm wrong and I won't admit it and so on and I keep pushing the same line but I'm blatantly wrong three times, stop listening to my show. Scott is not going, okay, I have to stop. But why do you listen if somebody's unreliable? Why listen? Why take him seriously? I can't take any of these people seriously. And I don't. Again, not randomly, but because I've tried. So that's one. First, if you have the statistical knowledge, if you can read a scientific paper, at least somewhat, I'm not pretending that I know this inside out, I don't. But I can read a paper, I can read the conclusions up to a point, I can look at the data a little bit, I can look at the stats to see if they make any sense or not. If you have that ability and you look at the data and you see that it's wrong, wrong, and then it's wrong, then you stop trusting these people. That's one. So that's one criteria. Just look if they lied or not. Look if they're trustworthy or not. Two. Most of us are not doctors. Most of us are not epidemiologists. Most of us are not biologists, pharmacy experts, and we're not going to go do the research. We're not going to go do the lab work. We don't have the knowledge to evaluate many of these claims. So how do you evaluate the claims? Well, you look for people you trust. So let's take the COVID vaccine. I know a lot of doctors. A lot of doctors. Partially because I come from a family of doctors. My father's a doctor. My father's doctors have cousins who are doctors. I have a niece who's a doctor. I'm missing doctors. Anyway, most of my father's friends are doctors. I've hung around doctors since I was a kid. I've hung around hospitals since I was a kid. I know doctors. A lot of doctors. And doctors I trust. My dad, his reputation in Israel was as one of the best doctors in the country. Not to say he couldn't make mistakes, but I trust my dad as a doctor. So I have a lot of family with doctors. Then I have acquaintances and friends who are doctors. A lot of them. People I know. People I'm friendly with. People I'm acquainted with. A lot of doctors all over the place. Then I have a lot of doctors. I just do. I have a concierge doctor in California. And before this doctor, I had a different one. Another doctor. I'm talking about medical doctors. And then I have a doctor here in Puerto Rico. Again, I have one now, but you can call it at somebody else. And all of these doctors are my doctors, because I vetted them, because I chose them, because I trust them. Because they seem to know what they're talking about. About healthcare. And then the doctors I kind of follow on, like I'm reading a book now by Dr. Peter Atia, who I recommend, Dr. Peter Atia, on Lifespan, which I'm still relatively early on in the book, but I so far highly recommend. Highly recommend. Sounds really good. A lot of my doctors, like my concierge doctors in California, and this doctor, are not Atia. They're not conventional doctors. They're more functional medicine kind of doctors. So I know a lot of doctors. Friendly a lot of friends with a lot of doctors. Friends with, and have a lot of family who's doctors. Now these people I believe I trust more than I trust some random doctor who goes online and who says COVID is going to kill you.  Because I know them. Because I vetted them. And I'm sure you know doctors, you have a doctor. You could check around other doctors, but you find people who know. People with knowledge who you can trust. It's not easy. I'm in a good situation with who got the doctors because I have a lot of connections. And if I wanted, you know, if I get sick with something really bad, there's a lot of people I can call to get advice, to figure out what the best treatment is, to do all that. Who I trust. And of all the doctors I know, of all the doctors I know, I don't know, 30, 40, 50. This, you know, something like that. Dozens of doctors I know. There's only one. Only one who is skeptical about vaccines. Nobody else's. Not a single one of these other doctors are. And about COVID vaccines, not a single one of them. I don't know. My dad has had the vaccine. He's been, I don't know. He's been boosted two, three times. I don't know how many they allow in Israel. Is my dad poisoning himself on purpose? Is he going against his own medical knowledge? Can he not read the medical papers? Does he not know mRNA? Ooh, it's never been tried on humans. Of course it's been tried on humans. Moderna and BioNTech have been trying mRNA vaccines on humans for years now. They haven't been successful, but they know they don't harm. Are the vaccines risk-free? Of course not. No medication is risk-free. You take Advol, it's not risk-free. You take Tylenol, it's not risk-free. No medication. No medication that you take is risk-free. Are there certain age groups where COVID vaccines can cause heart problems? Yes. We knew this from relatively early on. And I indeed argued early on that I don't think young people should get vaccines. Certainly not. I didn't think children should get vaccines. I think healthy young people probably didn't need the vaccine. I didn't think it was a problem if they got it, but I didn't think they needed it. Why? Because COVID was not a threat to them. Not a significant threat.  And if you look, so that's a second thing. A third thing, I'll just say this is simple. You don't need a degree in statistics for this. If you look at countries, if you look at the 10 most vaccinated states in the United States, versus the 10 least vaccinated states in the United States, just look at that. The 10 states most vaccinated, highest percentage vaccination. The ones with the least percentage vaccination. And you look at excess deaths. It's not even close during COVID and post-COVID. The states with the least number of vaccinated people have the most deaths. By a lot. I mean controlled for this size of population. By a lot. It's not even close. It's not even close. So, you know, I throw up my hands because it is shocking the extent to which people are open, receptive to conspiracy theories and with good reason we don't trust the quote political authorities. All right? But it doesn't mean if you don't trust the political authorities, I don't trust the political authorities. It doesn't immediately mean that I have to trust the kookiest, craziest nutcase who comes in the corner spouting the latest ridiculous conspiracy theory. You don't trust the political authorities. Search for better authorities. I mean, the right position has always been anti-lockdowns, anti-mandates, pro-vaccines. I mean, it's easy. Why is that so hard for people? That's anti-the political authorities, pro-science. Anti-lockdowns, anti-mandates, pro-vaccines. That's been my position from day one. From day one. I was never for mandates. Now, can private companies mandate? Sure. Private companies can mandate you wear a mask. Private companies can mandate you take a vaccine. But is the government mandate? Absolutely not. And should companies mandate vaccines? Probably not. It's none of their business. It's mostly, particularly once we discover the vaccines were not very effective at preventing transmission, then why would companies even do that other than virtue signal? So, but this is a position people can't hold. No, if you're anti-lockdowns and if you're anti-mandate, you must be anti-vaccines. No, I'm not. Frustrating, frustrating. So, Jim, no, I think the entire field of COVID vaccine, excess death, people are dying from the vaccine, is complete nonsense. It is a self-perpetuating money machine for these people. They are lying. They're making stuff up. And I talked about this years ago. I talked about when the first guy showed up on Brett Weinstein calling himself the inventor of the discovery, the inventor of mRNA vaccines. And I looked him up and he turned out to be one in a team that had originally done some of the work that would later lead to mRNA vaccines. An inventor of mRNA vaccines was a completely bogus title. And Brett Weinstein would take him seriously. He was bizarre. And then you read up his bio and you read that he was kicked out of the team and he had to leave the whole field and he had to go into medicine because he couldn't do any research anymore because he was kicked out and that he was basically a disgruntled employee or disgruntled somebody who thought he deserved credit and never got the credit. He deserved. Other people got prizes for the discovery. He did not. All of this stuff. When you discover all of that, then yeah, it's just a disgruntled thing. And now he's found a way to get back at them and he's also found a way to make a lot of money by claiming that mRNAs are killing you. mRNA will go down, mRNA vaccines because COVID use is just, I think, the tip of the iceberg of how they're going to be used. It's going to go down as one of the great biotech innovations. It's already in the process of radically altering treatments of cancer. There are a number of mRNA cancer delivery systems and vaccines that are having phenomenal results in cancer treatment. And yes, they bear risk. All drugs, all drugs, any drug that is prescribed or even over the counter drug.  On any one of those, any one of those channels. Also, if you'd like to see the Iran Book Show grow, please consider sharing our content and of course, subscribe, press that little bell button right down there on YouTube so that you get an announcement when we go live. And for those of you who are already subscribers and those of you who are already supporters of the show, thank you. I very much appreciate it.


Hey, my name is Fernando and I'm a technical marketing manager here at GitLab. Today I'm going to show you fuzz testing and how it can benefit you. Fuzz testing automates providing invalid, unexpected, or random data as inputs to your application. Individual functions of your application can also be fuzz tested in order to find vulnerabilities before the application is even complete. The application is then monitored for errors such as crashes, failing code assertions, or potential memory leaks, all which can cause security issues. Here's the project set up with fuzz testing. In order to enable fuzz testing, we add the coverage template to the GitLab CI YAML. We create a job to tell the fuzzer how to run on our application. Each fuzzer is application specific and each programming language uses a different fuzzing library. For more information on this, see the links in the description. This is the file which contains the fuzzer. Here random data is passed to the parse complex dot parse function. Since this is a Java application, it's set up using JQF for fuzz testing. Fuzz testing helps you find issues or vulnerabilities that other QA processes may have missed. By adding it to the CI CD pipeline and running it on a feature branch, you can find these issues before they make their way to production. GitLab makes it easy to add coverage guided fuzz testing to your CI CD pipeline. Once fuzz testing has been configured, the pipeline runs and gathers up the fuzz testing results. These results can be downloaded or directly viewed on your browser. Looking at the results of the fuzz testing job, we can see a few items. There is a JSON artifact which contains all the issues detected. Then there's the corpus folder which contains all the test cases from previous runs. And then there's the crashes folder which contains all the crashes from previous runs. The JSON artifact provides information on the issues detected by fuzz testing. Here we see an index out of bounds issue which calls the application to crash. This leads us to examine the code the fuzzer calls. You can see that the function checks if data dot length is greater than four, but it fails to check if data dot length is less than four, which would throw an out of bounds error. This is something the developer missed which fuzz testing detected. Thanks for watching. For more information on fuzz testing in GitLab, see the links in the description. Be sure to subscribe. Here at GitLab, everyone can contribute.


In the next 40 minutes, I'm going to show you how to build a simple remote control robot with Automotive Grade Linux. My name is Leonor Navi, I'm a Senior Software Engineer at Kansako Group. Kansako is an open-source embedded software services company which specializes in system architecture, development, building, maintaining upstream open-source projects such as Automotive Grade Linux. So today we already had two talks about Automotive Grade Linux in this very room. The beginning of the day, Walt Miner gave us an overview of Automotive Grade Linux, the new stuff that we get there. It was a great talk, Walt is over there. And after that, my colleagues from Kansako Group, Scott Murray and Matt Porter gave another talk which was more specialized. So today, now, today I'm going to present to you AGL in a little bit different angle. The agenda for this talk contains a brief overview of Automotive Grade Linux. Without that, I'll speak about the supported hardware in AGL and how we can use this hardware with some peripheral devices to build a different product, let's say a toy. Later on, I'll explain how to do integration and contribution of software to Automotive Grade Linux. Finally, we'll have some time for making conclusions and, of course, questions and answers at the end of the talk. Although the title of the talk says that we are going to build a robot, this is not the main purpose of this talk. The main purpose of this talk is to do an experiment, an experiment to find out if other industries can benefit from the features existing in Automotive Grade Linux. If you're building Internet of Things connected devices, there are certain requirements if you want to run Linux on them. You require from the Linux distribution to have a built system and a whole development tool chain. You require a certain security level. It's nice to have over-the-air updates. And of course, for certain devices, you need a graphical stack and applications if there is a user interface that is on a display for the users. Doing all this from scratch can take a lot of time. And on the market, there are a lot of open-source Linux distributions which cover these requirements. One of these Linux distributions is actually Automotive Grade Linux. So in the next few slides, we'll see how Automotive Grade Linux fits into this. How many of you are familiar with Automotive Grade Linux? Can you raise your hands if you have been involved somehow? Are you developing applications or platform developers with Automotive Grade Linux? So I hope that I'll inspire you to contribute more code and to use more Automotive Grade Linux. Automotive Grade Linux is a project of the Linux Foundation. It was created to provide in-vehicle infotainment the new Linux distribution for the Automotive industry. It's already based on top of popular open-source projects such as the Yocto project and Open Embedded. Automotive Grade Linux was founded in 2014, and in the past three years, there is a tremendous progress. A lot of companies are contributing to Automotive Grade Linux. If I'm honest, these are not all of the companies. I didn't have enough space on the slide to fill in all the companies. A lot of people are contributing to AGL. AGL is having a six-month release cycle. If you notice the dates, you can see that every six months, we have a new release. Currently the stable release is darling DAP. Now we're working on the next release, Electric EO, which is scheduled for December 2017, and actually the first release candidates are already available. This six-month release cycle is pretty much following what is happening in the Yocto project, so we try to be up-to-date with the Yocto project. We have a look at the architecture of Automotive Grade Linux, the core technologies that we have there. It's obviously a Linux distribution, so we need a Linux kernel. It's a Linux distribution that is built with security in mind since day one. So we have security based on SMAC and Sinara. This is something that you might be familiar with from other projects, such as Tizen. There is an application framework. On top of this, we have SystemD, D-Bus, and we have a mechanism for doing software-over-the-year updates. There is a sort of client called Actualizer, which is integrated with a popular open-source tool for doing binary diffs and shipping these diffs to the embedded devices on which AGL is running. It's called OS3. You might be familiar with it from the project called GNOME Continuous. On top of this, we have a graphical user stack, which is relying on the Wayland protocol and the Western compositor, which is a reference system of the Wayland protocol. The user interface, as of the moment, if you build the default target of AGL, contains a bunch of really nice-looking Qt and QML HMI applications. Furthermore, AGL supports HTML5, G-streamer, and a lot of other technologies. As I already said, Automotive-grade Linux relies on the Yocto project and open-embedded for building the whole distribution. This means that Automotive-grade Linux incorporates a lot of different layers, meta layers, from the Yocto project, such as Pocky, the reference system of the Yocto project, and a lot of specific projects that bring features that are specific for AGL. I listed a few of those layers, not all of them. Keep in mind that we have a bunch of board support package layers for all the hardware that we're supporting in AGL. Just a brief look at what is supported in AGL. We have support for Renaissance generation 3 and generation 2 boards. Keep in mind that we're speaking about an Automotive Linux distribution, so the supported hardware devices are devices that are heavily used in the Automotive industry. AGL also runs on Intel devices, on Texas instruments platforms, such as the VAU board, on Dragon board, on Raspberry Pi 2 and 3, Raspberry Pi is a community-supported board. And since a few days, it also works on IMAX 6, Sabre and Humming board. Humming board is the latest addition to the supported hardware platforms by Automotive-grade Linux. This is a screenshot of the home screen of Automotive-grade Linux. Let's have a look at the toolchain that we have for development. All projects within Automotive-grade Linux are hosted in Git. The Garrett is used for managing and reviewing code. We have Jenkins for continuous integration and furthermore testing some of the changes automatically. There is a Jira where you can file issues by reporting bugs or requesting new features. There is a wiki article and there is a new documentation site where you can find docs how to get started with the exact steps. So with this, the short overview of AGL is over and now I'll move to something more practical. And it's the experiment how to build a remote control robot. So in order to do this, and we're speaking about a hobby project that any one of you can do during the weekends, that's what I did actually. So we're speaking about low-cost hardware that you can buy easily. You need a single board computer. Obviously this has to be a single board computer from the list of hardware devices that we mentioned because we already know that AGL works on them. You need a chassis and a DC motor. If you're good with 3D printing, you can construct your own chassis. Maybe if you want, you can buy some of the existing robot chassis that you can find around the internet or in eBay. You need a motor driver to drive the DC motors, a few sensors to make things more interesting. And of course, you need batteries to run all these. So those of you who know me before the talk know my choice. I'm a Raspberry Pi fan, especially when we're speaking about hobby projects. So for this experiment, I decided to rely on Raspberry Pi 3. I believe all of you know Raspberry Pi pretty well. It's very affordable. A lot of people have it. It has a good software support in general, as well as an automotive grade Linux. There are a variety of atom boards that you can easily plug on the Raspberry Pi in order to build a robot or another hobby project. And there is a huge community. One important remark. This is not an automotive grade board, so it doesn't cover certain requirements. And it's good for hobby projects. And keep in mind that this experiment is in this category. The good thing about Raspberry Pi, actually, how many of you have Raspberry Pi? OK, all of you. That's perfect. I hope you understand my choice. So a lot of people are building remote-controlled robots with Raspberry Pi. It's a nice toy. Even a child can build it. And there are a lot of motor controller atom boards that you can just buy, plug on top of the 40 pins of the Raspberry Pi and get fixed working. I have listed some of the popular atom boards that you can buy. Here you see the exact controller that they're using. As you can see, I have nailed down just a few of them. I decided not to use the one that I can buy, but actually to make my own. Do it yourself. Motor driver board. This took me a bit of time. I'm not very good at soldering, as you can see. I had a look at the few popular half-H motor drivers that I already saw that are being used by other boards, such as the ones that I showed on the previous slide. I decided to select one from Texas Instruments. That is the easiest one for soldering, this one over here. This is how it looks. As I said, I'm a software engineer, not very good with soldering, as you can see. But it works. It controls two DC motors. You need an external battery pack to power this thing and to make it run. Now, controlling the motors, the problem with Raspberry Pi is that we don't have enough hardware PVM. There is just the single one of them, and this is kind of problematic because if you want to have a good control of the motors, you need to rely on hardware PVM. Alternatively, we can emulate with software the PVM, but the results are not that good. They're good enough for making a simple demo. But if we want to go one step further to make it more interesting, it's better to pick up a microcontroller with PVM and to have a hardware control over the PVM. There is a popular open source library called Wiring Pi, which is very convenient for integration with CNC++ projects. And it can emulate PVM from the software side. So these are super cheap, dirt cheap motors with wheels. You can buy them from eBay for one or two US dollars. That's absolutely nothing. Of course, you need to wait for quite some time for their arrival. But if you find them in a local store, just get them. They're really useful for making these hobby projects. So my idea for the robot is to have just three wheels, two of them to control, and one additional wheel. And this is a very simple code box just to show how simply with Wiring Pi, I can enable the software emulation of the PVM and to make these motors move forward or backward. So let's have a look at the sensors.  Probably the most popular that I have seen in a lot of projects, almost all projects, is the ultrasonic sensor for measuring distance towards objects. This is a cheap and effective sensor that allows you to measure the distance to walls, for example, or other obstacles and to make a simple algorithm to move around them. A lot of fun thing is that actually I haven't done this yet on my robot, but it's infrared line tracking. So basically the idea there is with this sensor, you can distinguish, for example, a black line, and your robot can follow the line. So it's a path-following robot. Other cool sensors are like the compass that you can find as an I2C sensors. There are other I2C sensors for measuring temperature, humidity, colors, lights, so you can even distinguish the ambient light. Furthermore, Raspberry Pi has official Raspberry Pi camera coming from the Raspberry Pi Foundation, which you can also integrate within the robot. The integration is pretty simple because there is a board support package layer called Meta Raspberry Pi. It's for the Yocto project in general, as well as to other Linux distributions such as Automotive Grade Linux, which are based on the Yocto project and Open Embedded. Since we're building a robot that has to be remotely controlled, we have to think about the communication. And in terms of communication, Raspberry Pi has built-in Wi-Fi, Ethernet, and Bluetooth. It's very easy to extend the capabilities of Raspberry Pi by adding radio transmission modules or infrared receiver and or transmitter. In our case, it's enough to add a receiver. And of course, we can buy some GSM modules and even control the robot this way, because Ofono is built-in in Automotive Grade Linux. This is an example of how you can add to your DIY board, actually. This is what I did. To add an infrared receiver, these are schematics that I took from another project that I'm having. It's a very popular infrared receiver, which can be easily integrated on AtomBoards for Raspberry Pi and receive commands using the Linux infrared remote control project, known as LIRC. It's a 20-year-old project that works perfectly fine on Linux. So putting all this together on a Raspberry Pi, on top of Raspberry Pi, requires making a DIY board. I'm currently working on it. I already have the prototype soldered on a breadboard prototyping. The next step would be to design this board on a kick-hat. This is my setup. I'm using these pins, general-purpose input-output pins, for attaching the DC motors. I need four pins for this. With wiring Pi, as explained in the previous slide, I'm emulating the PBM. The infrared is attached over here. This is actually the only hardware PBM pin that I have on the Raspberry Pi. I also keep the UART for easier debugging of the board and, of course, I2C sensors for attaching the sensor that I've explained on the previous slides. So, getting back to the software side of things, here is how we built Automotive Grade Linux. It's a very straightforward process. It's very easy to do it. You just have to follow the exact steps, which are explained actually in two places. One of the places is the AGL Wiki. The second one is the official documentation. So, the process has three major steps. First, you have to get the source code. As you can see from the commands over here, AGL is using the Google repo tool to keep together all the layers for building the distribution. If you want to use a specific version of Automotive Grade Linux, this is where you have to specify this version. In general, the way I have written it here, I'm going to use the AGL master. This is the current branch of AGL that is being in-developed. So, this is the cutting edge. If you want to rely on something more stable, because, you know, when we are using the master branch, there is a development going on, so something might not work as expected. But if you want to use something stable, make sure that you are using the latest version of AGL, for example, DarlingDap, as of the moment. After that, we have to set up the build environment. And here comes the interesting part, because with these magical words here, we can activate or deactivate certain features available in Automotive Grade Linux. For my remote control robot, I don't plan to add a display, and since there is no display on it, I really don't need the whole graphical stack. So, what I'm building here is the very, very simple, the very basic Automotive Grade image, which is convenient for headless devices, as the one that I'm building right now. One more thing to say, you need a Linux machine to run these procedures, or it's possible to do it in a virtual machine as well, but it will take quite a lot of time. The build of this Linux distribution will take some time, so please be passionate, grab a cup of coffee while you're waiting. Let's have a look at the three most commonly used Automotive Grade Linux images. The first one is the one that I'm actually using for this project, AGO Image Minimum, the bare minimum of Automotive Grade Linux. The other one is IVI, IVI stands for Inviso Infotainment. And finally, we have the AGO Demo Platform. The AGO Demo Platform is the image that provides the whole demo UI, shipping with Automotive Grade Linux as of today.  All these demo applications as of today are written in Qt. They're running on top of Wayland and Weston, and Weston is relying on IVI Shell to render. So how to customize the image? The way to customize it is actually straightforward. If you're familiar with the Yocto project and open embedded, you already know how to do it. There are two key files that you would like to edit while you're still working on the product. The first one is called BB Layers. It contains the list of the Yocto layers within the distribution. The second one is called conf.local.conf, where you can customize what exactly to get into the image. And in this example over here, I'm adding Lyric. This is the software that I need to scan and control the robot through infrared. Keep in mind that Lyric is provided by MetaOpen Embedded Layer, and this layer is already available in Automotive Grade Linux. Therefore, I have not described it in BB Layers because it already exists. For example, if you're working on another project that requires another software, which is provided by a layer that doesn't exist in Automotive Grade Linux as of today, you need to describe the layer in BB Layers. This is a quick and dirty way to add new packages really quickly, but if you want to build a new distribution based on Automotive Grade Linux, a new product, the better way is to do the full integration, to do your own Yocto recipes, and finally to have your image based on top of AGO. This is how you're customizing the image just quickly during development. So I would like to spend just a few slides for speaking about how to contribute to AGO upstream. So the first step, if you want to add a new project or a new feature within Automotive Grade Linux, is to report an issue in JIRA. You need an account for Linux Foundation to log in into the JIRA. It's super easy to create one. You can request a new feature, report a bug that you have experienced, and once you have this listed, the next step is you or someone else to modify the source code of AGO and to contribute it to Garrett. It's not mandatory, but it's nice to include the JIRA ID within the git commit message that will appear in Garrett. This is easier. This is a good option to continue tracking the progress of this bug within Garrett and the git commits. Have you used Garrett? How many of you are familiar with it? Okay, I would rather say half of the people around here. So Garrett is an open source web-based team called Cooperation2. First of all, it's for a number of git repositories. It's written in Java. It's available under open source license, Apache license, actually. So here is how you can create this account and get started with Garrett and JIRA actually in AGO. Here is the workflow of Garrett. So we have one major repository. For example, here we have AGO repo. This is the repository that holds the manifests for the Google repo tool that keeps all the layers needed to build automotive grade Linux. So if we want to make a change, for example, to bump the version of a certain layer, to add a new layer or to remove a layer, we have to pull this into a developer repo on our local machine. After that, we have to make some changes and to push them back to Garrett. Garrett is going to create a repository where the changes are pending for a review. There are three types of roles within Garrett. One are the developers, the people doing the push and pull from Garrett from the official Garrett repository to the one that's pending in the reviews. The second role are the reviewers. They can be again developers. They have to fetch, test your changes and give their opinion. They can vote with plus or minus. If you have enough positive votes, the maintainer, the third role, should merge your change within the official repository by doing a submit. All right. So you're welcome to Automotive Grade Linux at any time. We have a mailing list. I highly encourage you to subscribe. We have a weekly developer call that Walt is doing every Tuesday. And we have an IRC channel. We're at the end of the presentation, so I would like to do a few conclusions. So according to my experience, open source is compressing the development cycle for a faster route to the market. And I think this is a great benefit for the whole ecosystem because this way you can make your products faster and provide them with better quality because open source means that more and more people are looking into the source code and are putting efforts to get it better. AGL is based on top of already proven open source software technologies. We mentioned them during the presentation, like the Yocto project, Open Embedded, Wayland, Weston and Qtan and so on. I won't list them all now. So AGL is entirely open source project so don't hesitate to contribute to it. It offers an open source software stack which can be useful not only for the automotive industry but for certain cases if you're doing Internet of Things. I hope that my experiment with this simple remote control robot proves that it's possible to do it. You have already seen how to do integration within AGL if you're developing another project on top of it. So what's next? In terms of my own hobby project for the robot, I need to do a better board that is designed in KeyCat, a two-layer board on which I'll put all the components as I explained them in the previous slide to get it more professionally working. In terms of AGL, the next steps are releasing a stable final version of Electric EO. This is going to be the next release of AGL and this has to happen by the end of this year and hopefully to have nice presentations for the consumers electronic show next year. Starting as of January 2018, we should start developing the next release of AGL which is called Funky Founders. This is the name of a fish. The cold names of AGL are actually names of fishes. We have to say big thanks to Walt, who found in alphabetical order names of all those fishes. So the sixth release of AGL is Funky Founders. Thank you very much for joining this presentation. We have five or six minutes for doing some Q&A session and I have listed here a few interesting links. So thanks and any questions? The robot. Well, the problem is that I still don't have the whole robot built and I don't bring it over here, but as soon as I design the board within Keycat, I'll show it as a whole robot and hopefully it will be working better at the moment because it was not in that good shape and I left it at home. Okay, so if I hear well the question, is there any specific advantages of using automotive grade Linux for hobby projects such as this one instead of using Raspbian? Yeah, okay. So yes, for hobby projects you can also use Raspbian. The process is similar. The advantage of using automotive grade Linux is more for projects that are also targeting some long-term relationship and even going to the market because Raspbian is a Debian-based distribution with desktop. When you do a headless device like this robot, you don't actually need the whole graphical user interface like X11 that is shipped within Debian. While automotive grade Linux with a headless profile can be more flexible for the needs of a project that doesn't require a display, for example. If you want to do another device with graphical user interface similar to what we have in the cars, automotive grade Linux can also be useful for that. Yes, over there? Sorry, could you please repeat it a little bit louder? Okay, so the question is how large is the system image of automotive grade Linux? I'm not exactly sure in terms of megabytes how large is this. Well, do you know exactly how big is it? Yes, I also don't remember, but I believe it's below... The final image should be around 200 megabytes, but take my words with a pinch of salt. I'm not exactly sure about the size. You can optimize it if you're searching for a smaller size. Just for a reference that you can build a pocky image that is like 30 megabytes or something like that. Yes? Okay, so the question is what is the boot time of AGL on Raspberry Pi 3? I also don't have the exact number in seconds. It depends on the profile. The AGL minimal image boots pretty fast. With the graphical user interface it could take, for example, like 10 seconds or something like that. It depends on how fast and which Qt applications will be loaded as part of the demo image. Okay, one question over there. Okay, so the question is, I'll just repeat it. The question is regarding the licensing of Qt within automotive grade Linux, and more specifically, are the applications written with open source license, or if a proprietary license is required? Did I get it correct? Okay, so the applications that we have as of today within the demo version of automotive grade Linux are under the open source license of Qt. If you require to use a proprietary Qt applications and to ship them in AGL, you have to do it as part of your own layer because AGL is open source distribution, and if you want to contribute something to the upstream, it has to be under an open source license. Okay, any other questions? Okay, thank you very much for joining.