Rydw i'n meddwl, dyna'r pethau, mae'r ffordd yn ymdau. Rydw i'n meddwl, mae'r ffordd yn ffodol. Rydw i'n meddwl, mae'n Marthyn Weller. mae'r ffordd yn gogem, mae'n meddwl a'r byd. Mae'n gweithio ffodol. OER er mwyn i'r ffordd yn ymdeg. Mae'n meddwl i'r ffordd, mae'n meddwl i'r ffordd yn ymdeg. Mae'r cyfrannu o'r bydd yma yn y cwrnod, mae'n ddigon gyda'r OER. Felly yw'n hwyl i'r Hwylio, mae'n ddigon. Felly mae'n ddigon, mae'n ddigon. Mae'n gweld ymgymell Cathrin, Beth gyda'r cyfrannu Cathrin, mae'r cyfrannu cyfrannu gyfrannu gyfrannu golygu a wedi'i gilydd o'r llyfr o'r ddigon. Felly mae'n gwybod yma cathrin a'r cyfrannu ar y cyfrannu. Fe yna'r cyfrannu, mae'n cyfrannu i'r cyfrannu, having cherished these moments rather than here, which you don't know what's going to happen so so it's great, you'll be... This is your first Official recreation conference, you are really welcome to lovely community going back. So I'll pass over to my co-chairs and then introduce themselves...  You're welcome, so much. Hello everyone, it's fantastic to be here and I'm really vient to see everyone. My name is Beth Pitt and I'm part of the Gojciem team. In case ac yn cael ei wneud bod y cyfnod o'r ffordd yma, gyda'r ymddwyr Cymru, yng Nghymru, ymddwy'r 130 ac ymddwy'r 200 o'r ffordd i'r cyfrans ac ymddwy'r cyfrannu fod yn gweithio'r ei gwybod, ac yn bwysig i'r ffordd a'r ddyfodol, cyfrannu'r ddechrau'r gweithio, ac yn gweithio'r gweithio ar y cyfrannu. Felly, rwy'n cael ei gweithio ar y fan yng nghymru. I have gone through the process of being a member, alumni and part of the team. And yesterday we had the chance to have our first face-to-face seminar with I guess students around the world. So, yeah, but a great team and I'm really looking forward to staying with you on this list. Hi everyone, I'm Rob Farrow. I'm a researcher at the Open University. OER was actually one of my first, maybe the first, Open Education Conference I attended. And if you know the kind of work that we do, you'll know that we've been doing full-time Open Education research since around 2012 or so. So it's quite an honour and quite a journey to actually be one of the co-chairs now, quite exciting. But it's even more exciting to be in a position to actually congregate and meet with people again. Cos it's been a long haul, right, the last couple of years, quite isolating. So this is the kind of moment where you know, you kind of would hope to get to at certain points in the past. So it's just quite exciting to be here and it's really nice to see everyone. Well, thanks Rob. I think we've spent a couple more of time to say before we go into the keynotes. Oh, it's the other button. I see. So there's some stats. So it's taken place over three days, synchronous and asynchronous. Hybrid, online, face-to-face. We're trying to hit all the targets, all the things, all the methods. Participants from 20 different countries. 29 scholarships have been awarded to people from 10, 3, 3, 1s. Over 80 different sessions. 200 participants. So I think that's kind of really interesting, although the face-to-face is be you and select, I think. That's what we can call it. I think it's kind of really different. The hybrid event is kind of hitting the sort of numbers that we used to get. I think people are still engaged in this. So follow along on the hashtag there. I want a big thanks to our friends at Reclaim Posting for sponsoring the conference. I'll do the pitch for Jim, Jim Grill, who's my PM on the team. But if you need hosting for websites and stuff, Reclaim are the team to go to the area with you. So with that, that's the end for us, and we're going to move into our keynote, which I get to chair. So I'm really pleased. Our opening keynote is going to be Rob, but mainly Brian Mathers, who's the guy behind all the penguin drawings. And I think if you're wondering why penguins, that all will be revealed in the next bit. So I'm going to hand over to Rob and Brian. This is mainly a Brian session, not a Rob session. But I'm just going to do a little bit of an introduction. So Brian is the one who's really responsible for our kind of visual identity in GoGM, which has been very powerful in a very good way of communicating of what we're doing and connecting people. This is a logo that Brian designed for us. And you can see the inverted earth in the corner with the global south as the main projection rather than the north. And these are the really nice, interesting ideas that Brian brings to us. This is like a travel ticket from the Golden Age of travel. This is one of the concepts that Brian helped us develop. Here you can see his notes when we were just kind of going, Yeah, we want something like this and we're not really sure, but here's a load of ideas. Can you make sense of all this? We were trying to work out what we might say to people about what GoGM stands for and why people should consider joining the network and that kind of thing. And yet you can see how the process works. Brian's so good at just kind of drawing your ideas out and kind of making something coherent and visually interesting and powerful. It's time for the Penguin Explanation. So Brian can play impartial credit here. The original thing with the penguins was when we were first getting some swag done, it was like, you can have a stressed penguin if you want. And people were like, well, you have PhD students, they're kind of stressed, but maybe we should get a penguin. But then obviously the penguin kind of took off a little bit. The thing is, it was really Brian that brought the whole concept to life and brought the penguin into the heart of it as a motif and a character and a kind of avatar for people to connect with. So, yeah, we've had this golden age of travel kind of motif and this fed through some of the outputs that we do and the guides that we've created. I won't spend too long talking about these, these guides and these outputs from GoGM are the ones that Brian illustrates and does the covers for and helps us to really draw together into something quite unique, I think. And so, in a way, Brian, you're the kind of missing member of the GoGM team. It really is that much of a good close working relationship. So if you want to see some more penguins, have a look on the GoGM website and you can check out all these books that we put together. And with that, I will hand you over to Brian to do something visual and interesting. I appreciate that this is a bit of a tough gig because of what has gone before in the last couple of years.  It comes along and sort of just notches that up. A few bits and I feel like we've really got to practice being social again. I guess it's maybe a little bit different if you're an introvert or if you're an extrovert in terms of how you see that. But anyway, it's real privilege. And so I'm just saying thank you to you for coming and getting social. OK. Good. Right. We're all good. Let me see. I'm just good. A lot more visual than I used to be. If you know me, you'll know that I started my career as a software engineer. And I have built software and grown teams and sold to businesses. And by 10 years ago, I found myself inside a corporate business that I had sold a business to and slowly my soul was being drained. And it was then that someone introduced me to a stylist and an iPad. And having not really, if you'd asked me, could I draw, I would probably told you knew at that point. That was sort of about 10 years ago. There's something about a stylist and an iPad that is very forgiving in a way that a pen needs to be a pair of... isn't it? Because you can undo very easily with a stylist and iPad. And since then, the stylists and the iPads have come a lot better. In particular, the Apple Pencil is incredible. But fast-forwarding a little bit. In the work that I do now, openness is a very hard of it. So I was, thanks to what's wrong for tearing me up so nicely. And I'm maybe making it look as if I'm a magician when it comes to creating all these things. But if you've worked with me before, you know that my process starts with conversation. I was reflecting with some of the companies we have with some people in Kenya and some people in Myanmar. With some of the University projects we did. And language becomes an issue in terms of encouraging people to be open. Because I know that if I can get people talking about stuff they care about, that outcomes go away. And it's sort of gold in the way of humour, humour chiefly. I'm always on the lookout for humour. Both dark humour and light humour. Dark humour is probably even more powerful, you know. So I've actually been on the side all along. The punchline was, I'm more visual than I usually am. But it's that openness really that feeds into all of the stuff that I create with visual thinking. And every so often I forget about it. Every so often somebody says, oh Brian you're so good at drawing, can you just draw us a thing? And I'm like, oh yes of course I'm a magician, I'll just draw that thing. And then I realise, oh no, I've missed the essential part of the process which is to have the conversation. To get people talking, the people who know about the stuff. Get them talking, getting them being open about it. And in therein are the truth. So all of these things for all of the class I've been in, it all comes from those conversations. So I was just going to, a little bit to my new one, at least sit here and see if this works. So the change is what you see, isn't it just that one little speech bubble? That sort of changes what you see. And comic strips, something that again I've learnt a lot about in terms of the power of reducing something to a single black line. But the people I identify with and empathise with, and that says something. Again I'll touch briefly just on how much I've learnt from you in our community. From working with Audrey Waters I learnt the power of metaphor in terms of that. We're working with Alta, I've learnt a lot about livescribing. And really Alta gave me that opportunity to come along and try to capture and create a landscape of a conversation. I'm not again, it was a series of experiments but I learnt so much from what then people saw in those visual landscapes. Gojien, you saw the visual language. Once you have visual language, once you've settled on the golden age of travel, or England's and the golden age of travel really, you can go in all sorts of directions. It's really so powerful and sometimes finding and defining that visual language is quite a process. But once you've got it, oh my goodness, you can go fast very quickly. For me, I'm hosting, I learnt a lot about aesthetic. So how something feels, because I've realised that you process how something feels way before you cognitively process it, if you see what I mean. So if there's words involved, it's why I suppose pictures watch so well on Twitter. You absorb the picture and then you read the sentence, you know what I mean. But there's a subliminal level there that you can't do anything about. You've already had a reaction to it. So the aesthetic of how something feels becomes important. If you don't believe me, just ask Comic Sans font. If I was thinking just before this talk, I've often found contracts, like a legal contract, very sort of overpower, because there's always times when you ruin font. It's sort of saying I'm an author of a document, and you probably don't have the knowledge to be able to process this legal text. But if you change the font to Comic Sans, how would that feel? A hard document, but hey, it's private school. So how something feels is actually really key, and I get offended very quickly whenever I see things that people spend a lot of time on that really don't have any visual element at all, or somehow I'll make me switch off, I guess, because there maybe are just times in a moment, no doubt, and it didn't get me started. There's something about how we empathise with the simplicity of just two dots, and we can't help but see a face there. If you don't receive an email from me, you'll notice that I always sign off my email with a little sideways column on the bracket, and it's basically saying I'm sending me this email with a smile, because without that, emails are neutral, but of course nothing's neutral, because we paint in colours that aren't there, so you can read an email from someone. You paint in meaning that isn't there, that isn't intended, I'm the worst at this, I'm serious, I paint in all sorts of things, but I think people are thinking about me whenever I read something, right? So there's something about that human element of two dots and a line, and if you're drawing a self-portrait, drawing me as a self-portrait is pretty easy, just a circle, a triangle. There's been a few times when I've done a self-portrait drawing elements as part of a workshop, and it really is a bad idea, it's a terrible idea, because how you see yourself is not how you are seen by other people, even the strongest of people, sometimes I'm sorry, do me a little bit, and there's like really, my ears are there, and there's all these layers of complexity that we do in ourselves, that means that actually drawing humans is quite problematic, but that's why penguins are such a good idea, right? Because with penguins, you know, we can say a lot through them, we can't help but empathize with them, right? If you think about it, penguins are cute and cuddly, right? Smiling, boys, but they're also community burrows, I guess, at the picture I have in them, in terms of Antarctic. I'm reluctant to say the word colony in this context, because I know, I've never taken it in a context, communities better work, but sort of penguin communities, but also these penguins know about survival, they deal with harsh conditions, but to draw them, they're also a dream, in terms of the contrast, not the yellow, the yellow we are in, in terms of ffinting the big. So I suppose I strong-armed go GM into using penguins more and more, because they're perfect for our culture, they're perfect for, you know, to be able to communicate through, to be able to make them dance, you know, to play the tuba, I'm sure penguins have a good go of playing tuba. I should credit Rob Meller here, actually, because I'm just remembering that he does have a drawing that I did of him as his profile on Twitter, which is brief, I have to say, but maybe it's because he features his dog as well, and maybe that's the trick. Now, let's see. So, yeah, penguins are also relatable, you know. Again, this came directly from a conversation, as all of my things do. You know, some of it, I remember that Rob was sort of saying that in some way nothing's tuned, because I'm still in the room, you know, sort of getting things done. But we complain that through penguins. I suppose that's that humor, or something like that earlier. But penguins can also help us understand things that are difficult. Of course, they do them, they don't, right? It's cognitive ease, isn't it? It's that feeling that you've got a bunch of penguins looking at these big words and going, you know what I'm saying? All right. But somehow your initial default responses all like can understand this too. You okay? And it's just that little bit of cognitive ease that helps you step into something. As opposed to a word 2003, you know, diagram that sort of feels like, oh, this feels too hard for me to understand, even if it's the same diagram, but it's just your initial reaction is one of defence, rather than one of embrace. Right? You know, penguins can give us a different perspective, I think, in terms of just even looking at words from a different angle. It's a very obvious little trick, really, but you know, a penguin making a slew of angel. There's some sure they do. But also, penguins can help us to reflect, I guess, on what has been a tricky time, you know, in the last couple of years, and help us to process that. So, you know, to wrap up, what's the penguin saying on this board? Any suggestions, caption competition? I'm not just a mascot. So, I think what's started off is actually a really lovely idea in Gwgian, where if I get it right, that you were sort of sending this on to the participants of the network, and they would take a photograph of where here's the penguin in Barcelona, or here's the penguin in Beijing, or wherever the penguin got to. The idea that, yeah, a penguin is not just a mascot. It can be a companion on a journey, or it can help us understand things, or it can help us process some pretty heavy things too. Okay, so we're going to draw some penguins, okay? If you're up for that. But of course, in drawing anything, we've got to give you three golden rules, right? So rule number one, do not overthink this. This is not your PhD, okay? This is like your journal that pretty much no one has ever seen, okay? If you put it like your PhD, then you probably revert to the version of you in primary school where somebody is going, that's not right, okay? It's really easy to overthink things. I am the worst at overthink things, and that's where just a little bit of encouragement and a little bit of process really helps, okay? Rule number two, play it cool, okay? I want to mean by that ease that if you're not in the way of drawing every day, which I guess most of you probably aren't, then your hand's going to make a mark that your eye's going to go, well, that isn't right, and by playing it cool, it's just like just go with it, okay? Because I think to think of drawing something as an outcome only is missing the point. It's more of a conversation with your PhD, and some of the best stuff that I've created has been accident, right? Has been you start somewhere and actually a different idea appears or you add a different little character, or the bird that you started to create, the penguin starts to tell you what it wants to be, and that's this third rule. Follow the idea, right? Even though you start off with, oh, what I'm trying to draw is this. If it takes a little detail, follow the detail, okay? Okay? With me? Good. I should say that I also have this T-shirt I'm wearing, limited edition, but I'm going to give you the opportunity to share your creations and not that it's a competition, but it's a competition. You know what I mean? We're all those sort of things, right? And I know this session has been recorded, so there's a T-shirt for the live participants and a T-shirt for those that are never watching it, okay? So how do you draw a penguin? Have you got pens ready? Are you going to go with me? Right, okay. So how do you draw a penguin? Now this is where it gets a bit tricky, because I've got a classic music playing in the background.  Now there's no pressure, but at the minute there's pressure. Anyway, I'm going to do my best. Right? So you're sort of drawing some sort of shape like that as the penguin body. I'm going to try it.  What do you think that sort of shape is? Eggplant. Eggplant. It is, but it's like an eggplant. You know what I'm saying? It's actually really helpful to think in shapes. Quite often, usually in YouTube tutorials, they tell you this sort of thing in shapes. Anyway, and next I'm sort of drawing that little bachy, you know, big part. And if you just want a little triangle just like that, I've done it. If you just sort of do the, like a little, just the top of that bachy, the top of that bachy on the top there. When you're drawing sort of two eyes, you'll find that actually the eyes are the killer. You know, it has a very different feel if you draw them close together or far apart but experiment and just go with how it is. And then if you sort of draw sort of two circles around your eyes, like of all the penguin things I've created, most of what's doing a communication part of the eye because they're not saying stuff.  Or something like that, I don't know. It's probably different things in different people's heads. Right.  Again, don't worry about drawing over things that are just extra lines or anything like that. Just go with it. You're still with me? Good, right. Let's draw some flippers because the flippers are sort of like sort of two flat squares poking out. Yeah?  All right? Now one other trick I'm just going to show you, like I know that obviously I'm on an iPad and you're probably not on an iPad, but to make a thing lift off the page, you've got to think about where the light comes from. Okay? And I'm just going to, on this, so the app that I'm using is an app called Procreate. So I'm just going to use a layer underneath and I'm just going to use a grey, but I'm just going to show you this and you can decide whether you want to add a little bit of shading to your drawing. Okay? So let me see. So I'm thinking that the light is coming from over there. So everything on this side of the panel is going to be shaded, right? And the difference that this makes is massive. Okay? Even though you get too much strips to do with any shading at all, but obviously the pendulums that I tend to draw today, so I'm thinking, right, well, the body is probably going to cover that bit there, and there's probably a bit there, and there's a wee bit underneath that bit there, and there's probably a little bit on the beak, maybe, even as well. Maybe you can just add a little bit of a shadow that then the penguin has. And the difference that that makes in terms of it, then becoming a 3D thing that steps out is massive. Okay. Now then, we're going to have some fun.  Okay? So you can see that I've just drawn a few props right there. Most of them are properly silly. But also, if and I cheat, think of something simple, right? Don't go for something very complicated. You know, there's nothing wrong with a penguin, with an umbrella, or a penguin holding a can of soup. I hear they like soup. Fish soup. But think of a prop, whatever prop you want, and add that to your penguin. Here, might even just be a hat. A hat's actually not a bad thing. Right. If you had a good drawing of a prop, any prop, good. I know that's quite tricky, because you're inventing this yourself. That's alright.  Okay? So you think about, maybe an obvious one is a thought problem. Okay? But ask yourself, look at what you've created, and ask yourself, what is my penguin saying to me? It might capture an emotion that you're feeling, you know, or it might just be speaking back at you, complaining, I don't know. Whatever it is that your penguin's saying, see if you can capture it. And obviously, there's different ways to do that. Penguin holding a sign is quite tucked under its flipper. It's quite a good idea. You know, sandwich board. Holding a big box. Wearing a t-shirt. Holding a sign. Whatever it is. And then just write a little message on the front of that. See what I'm doing. It's lovely seeing heads down, people working your way, there's some tongues. You're getting into it. Well done. If you're not the hammer of doing this, this is getting out of your comfort zone, I applaud you. I applaud you because it is definitely good for the soul to create. But also, there's lots of cheats here. If you've drawn a penguin once, drawing it the second time is a lot easier. Right? Because I've watched the most professional cartoonists, right? And they will draw things in pencil over the ink. Getting two cracks. That's almost like cheating. But seriously, I often will draw stuff from pen, take a photograph and then draw over it. And have two goes out. Of course it's definitely better. Right. So, when does one abandon their artwork? Because artworks are only ever abandoned. They just sort of stop at some point and go, I'm going to share that. I'll move on.  The remixer machine allows you to share a photograph of what you've just done. I'm briefly just going to show you how to do that. Um, the URL is a bit of a long one. I'm sorry about that. It was remexer.visualthinkery.com slash g slash c slash oer20. Remixer.visualthinkery.com and that's a little private gallery just for oer20s. And I knew some of you have used the remix machine before. Let's see people typing. I'm going to just flip to it and then just the remixer machine. And essentially you can oh, I tell you what I need to do. I need to save my drawing. So this is my drawing here. I'm just going to save that right here.  So if I click that up and I'm going to use that and then I can sort of zoom in a little bit and drag it in the centre. And then when I'm ready I can press publish. Publish. And then you will see that my example is down at the bottom. So if you get a chance to do that then I would love to see what you have created. If that doesn't work for you for whatever reason because it is your Twitter thing or your Google thing, that's also okay. If you want to come and show me, I'd love to see it anyway. If it's doing it on the train on the way back to Edinburgh you know I'm adding it then and that's okay too. Just whenever you get two minutes, see if you can add your creation to the remix gallery. And I'll just put that and we'll just put the URL back on there just in case you need it. Now listen, I'm going to leave it there. I have used it all of my time. It's been lovely to speak with you this morning. Thank you for humoring me by actually drawing some paintings. Thank you.


Hello and welcome everyone to this Kubernetes AI day, which is running as part of the KubeCon virtual event this year. And hopefully, you know, we can all meet in person next year. It's always exciting to be part of KubeCon and giving, you know, talks, but also, you know, listening to a lot of exciting talks which are happening. So what are we here for? So myself, I'm Animesh Singh. I'm the chief architect for data and AI open source platform. And with me, I have Andrew. Andrew, do you want to introduce yourself? Yeah, sure. I work mainly on Kubeflow, integrating some electricity projects into Kubeflow and then some other like Kubeflow, KFETecton stuff as well. Great. So we are part of this team, right? And today here we are to talk about, you know, how to stand up for ethical AI. And what we mean is essentially, you know, with the proliferation of AI systems and more and more models being deployed in production, right? We want to make sure as we are building AI, right? We are bringing and building it in a trusted manner. And what it means is that, you know, we have techniques for bias detection and mitigation or being able to explain your model predictions or, you know, being able to detect adversarial attacks being generated on your models, right? So that's the focus of this talk. And then, you know, we are going to talk in the context of Kubeflow, right? And we are aware it's a very popular open source project. So both Andrew and I work for a group in IBM called CODE, Center for Open Source Data and AI Technologies. What you see is a very nice picture of our San Jose Silicon Valley lab. We have a cricket pitch as well. If you are into that, right, we can have tournaments. But it's an amazing place, right? It's surrounded by green hills and large part of our team is centered there, but definitely, you know, the team is very globally distributed across the U.S. as well as other parts of the globe. And we've contributed to open source projects, you know, across the end-to-end AI life cycle, right? So for example, you know, we are the second largest contributor to Kubeflow. We contribute heavily to TensorFlow and PyTorch, Spark. We have our own open source projects like Illyder, Data Asset Exchange, Model Asset Exchange. So essentially work on strategy open source projects which are, you know, being consumed by our products as well as take, you know, the innovations we are doing inside and move it in open source like some of the trusted AI projects which we are going to talk about today. Okay. And you might be wondering, you know, why this topic and why not, right? So I think we are all living in this, you know, unprecedented time in history, right? We are all in the midst of this global health crisis. And hopefully, you know, we are now all seen, light at the end of the channel, but still quite a long way to go, right? And as part of this within IBM, right? We have had our own brainstorming and discussions around, you know, how we can help, you know, make the situation a bit better, right? We have been also focusing heavily in terms of bringing trust and transparency into AI, right? But does it really matter? I think it matters a lot more in the times we are in, right? Even, you know, a simple Google search can show you that, you know, even at the time of this pandemic and crisis, what we have seen is, you know, the trusted AI techniques are all the more relevant, right? Because when you are actually, for example, at this point, right, we are doing vaccine distributions. How do you make sure there is no bias in that distribution system, right? Things like, you know, being able to get tested, right? How do you make sure that, you know, the people who are being approved and being approached, they are not being done and approved in a manner where we are discriminating based on sex or race or, you know, based on the nationality of someone. So how do we handle it, right? So within IBM, right, we have our own vision for trusted AI, but, you know, we are also making sure that this is something which we are aligning with in the industry. We are taking feedback, always expanding it. But when we began and at this point, right, our focus has been around four pillars of trusted AI, right? What we call robustness, fairness, explainability, and lineage, right? So what do we mean by these different things? Robustness means that can anybody tamper with your AI systems or AI models? Fairness means that is it fair? Is it not biased towards a particular race or gender or nationality? Can you explain your model predictions, right? What it is doing? If it is making these decisions which are impacting my life, can it explain its decision? Last but not the least, right? Lineage, can I trace back and make sure that, you know, if a model is giving life-changing decisions for the stakeholders, we can trace back all the way. On what data set was it trained on? What were the hyperparameters used? And, you know, what version of a particular framework was used so that, you know, you can create accountability where it is needed, right? So having, right, lineage is very, very important. Just to give you an example, right? Since 2008, right, nearly every arrestee in a Broad County in Florida, right, was being assigned a criminal sentence which was based on, you know, if they are likely to reoffend, it was based on not points compass algorithm, right? And based on research and you can trace the link here, right? Which was found that data set was found to be heavily biased towards non-caucasians, right? The other thing is explainability, right? Now, explainability is not only important for someone who is consuming the models. Obviously it's the most important for that particular person, right? If I'm being denied admission to a university or if I am being given a loan or, you know, I'm being accepted into a company or denied into acceptance into a company, the model should be able to explain to me, right? Very, very critical because these are life-changing decisions for me, right? I cannot buy a house, I cannot get into a company, I cannot get into a university. Thou shall explain why you did that. But more importantly, I'm not the only stakeholder, right? There are people in the middle, right? So there are government regulators, people who run compliance and safety. They need to be convinced that over a period of time what the model is doing, it's able to explain itself why it is doing and why it is leaning towards those certain predictions, right? And last but not the least, right? When we talk about adversarial AI, what do we mean? A very simple example could be like you are depositing a check of $151, let's say, right? But a hacker could adversarily modify your image and instead of, you know, $151, like to the machine, it appears as $757 because it's very easy to fool a machine learning system, right? And adversarily generate an image which can make a one look like a seven for a machine to understand. Now, that's undetectable to the human eye.  That's, you know, more in the financial industry, but if you look at, like, you know, the impact of this can be life-changing and more scary in examples like this. For example, you know, a lot of the self-driving vehicles are being trained over stop signs images, right? Now, either if these stop sign images are adversarily modified more explicitly or implicitly, you know, over a period of time because of the wear and tear, right? These images are not great. Or, you know, they have lost their actual accuracy. Then, you know, the way the AI model will learn to detect them will be less accurate, right? So that might mean that your vehicle, the self-driving vehicle, actually running over a stop sign, right? And you can definitely imagine the consequences of a behavior like this. So we talked about the four pillars of AI and, you know, gave you some quick examples why we think, you know, they're really, really critical in real world and to handle that, right? IBM actually created four projects and then moved it in open source, right? So projects around adversarial robustness for both, you know, being able to detect if the models are robust and being able to mitigate them against adversarial attacks, that project is called adversarial robustness 360. For fairness, we have launched a project called AI fairness 360. Again, you know, this is around bias detection and not only detection, but, you know, being able to mitigate bias in your AI models as well. AI explainability 360 is essentially, you know, being able to explain AI predictions right across for different stakeholders, right? And last but not the least, right? Project which we have not yet moved the code, right? But a lot of the research papers in the material that already being made public is around AI fact sheets, which is around, you know, creating this lineage and governance model so that it can trace back the origin of a model right from the dataset on which it started. So adversarial robustness 360, as I mentioned, right? It's a project that actually allows you to rapidly craft and launch attacks on your models.  There are quite a bit of different kinds of, you know, evasion attacks and defense and detection methods. You can assign metrics, core robustness metrics to different kind of models. So definitely try it out. It's one of the most popular projects which we have out there, right? We are also working jointly with DARPA in terms of advancing the tools and the techniques and the algorithms which are available here. The IFN is 360. A library to actually detect bias across different metrics. So at this point, we support around 70 plus fairness metrics across AI lifecycle, right? So what we call pre-processing, in-processing and post-processing phase.  Once you have, you know, trade your model, you should be able to then validate that model, right? Which is in-processing phase. And last but not the least, right? Once you have deployed your model, right? You should be able to then, you know, monitor the model predictions and say, and figure out right if they are biased or not, right? So you can work across this lifecycle, right? And not only this, right? So based on these 70 plus fairness metrics, if you are able to detect bias, there are 10 plus bias mitigation algorithms which are provided as part of this, which essentially you can take and then, you know, mitigate bias across the AI lifecycle. And last but not the least, right? AI explainability 360, as I mentioned, right? There is no one explanation which fits everyone. There is the end consumer who is directly impacted by the model predictions. But then there are, you know, regulators and people who make sure, you know, things are compliant who are in the middle. There are next generation of developers who are retraining these black box models and positioning it towards a customized use case or towards a customized industry. Plus you want explanations at different levels, right? So you want to be able to also explain datasets and the features of the datasets, what do they mean? And then when you are looking at, you know, some of them are very simple models, some of them are black box models. So in case of black box models, you might want to have a surrogate model, right? Which is learning based on the explanations and creating a parallel model, right? Which over a period of time, as it is seen the basic explanations, right? It's creating its own robust explanation model. Now, apart from moving these projects and open source, right? So as a mission of trust area, right? As I mentioned, like we created this project and we realized, right? These techniques being propriety doesn't make sense. So they need to be in open source, but not only in open source, the whole IP of these projects need to be in a neutral place under a neutral governance model, right? So move these projects to Linux Foundation AI and they are now part of the Linux Foundation AI hosted in a neutral place. So hopefully that most of you have either used or using Qflow or trying Qflow or probably have heard about Qflow, right? So it's an end to end machine learning platform, right? Which goes across the whole AI life cycle, right? So you can essentially build a model using Qflow notebooks. You can then, you know, run distributed training on those models using different libraries like TensorFlow, PyTorch, XeBoost. We have MPI based algorithms like Horoword, et cetera, inbuilt as well. You can use hyperparameter optimization engine like Catev, which also gives you capabilities around neural architectures. So if you need more auto AI capabilities, and then, you know, you can deploy models in production, right? So using Qflow serving. So end to end spectrum and tying all these things together is pipelines or machine learning pipelines, which allows you to automate this whole end to end life cycle and not only allows you to automate, but also allows you to trace the lineage, right? So I'll briefly go into them. So the two projects, right, where we focused from the perspective of trusted AI are essentially, you know, pipelines and serving, right? And the reason, you know, for focusing on these projects was twofold. One, like when we talk about pipelines, what we want to do is, you know, make sure that as we are running through this AI life cycle, there are inbuilt capabilities into these pipelines, which can then help detect things like bias or, you know, the models which are being trained are they vulnerable to adversarial attacks, et cetera. So they can help you manage the whole life cycle and ensure that, you know, they're able to detect if the models are being built in an ethical and trusted manner. And serving, which is care serving, right, can then once your models are finally rolled out and deployed in production can then help, you know, to see if your models in real world are actually adhering to these trusted AI principles and you can monitor and create metrics, right? So it has the importance of these two projects. So very quickly on pipelines, right, it's the most popular project in the pipeline, Qflow umbrella, targeted towards data scientists. So there is a Python DSL also allows you to craft pipelines in a programmatic way. Each of these steps in the pipeline is called a run, which essentially is backed by a container, which essentially has the code, right, which is doing that specific part, right? And it's very, very popular in the Qflow community, right? Because for data scientists, they can now use a Python way of running machine learning pipelines on Kubernetes and they can also leverage underlying Kubernetes constructs like volume secrets and other things. But also, you know, for operational folks, right, they can actually go, there is a very rich dashboard which shows you the streaming logs in real time, et cetera. And you can, you know, look at the real time logs and debug. And not only this, right, there is a strong lineage in governance, right? So you can track back, like, once your models are produced on what data set it was trained on, what were the hyperparameters used, what was the version of TensorFlow. So very, very robust set of capabilities in this, right? And what we have done is, right, essentially created some pipeline components, as I was mentioning, right, pipeline components are the building blocks, which essentially allows you to run. And you can take these components around adversarial detection or, you know, bias detection, plug it into your own workloads. And then, you know, identify as part of that, whether your workloads are, you know, adhering to the Trusted AI principles and then you can detect if there are bias, et cetera being found. So with that, I will pass on to Andrew to talk a bit about Kubeflow Pipelines and Trusted AI and show you, you know, how we can leverage this in practice. So, Andrew, over to you. So basically, we're going to take a look at for first Kubeflow Pipelines and see some of the components in action in that. So we have two set up a model fairness check, which works with AIF 360 and then an adversarial robustness evaluation with the ART tool that we have. And the first training step we're going to do is on a gender classification. So it takes images and tries to classify a gender based off of that. So the way you would run this through is click creating run and then specifying some parameters that you need for this. You might specify the attack epsilon for the fast gradient. Sign method parameter are favorable label, which AIF 360 needs to know what is a favorable label, what they would like to see, and then you can also specify which groups you should check the bias for. So in this case, we've chosen a race with the label zero and the unprivileged group is the race of label four. So then we just can run this through. This experiment takes a while. So we won't watch it go through the whole way, but it'll just start up like this. And then some nodes will start to show up and run through, but we already have a pretty set up one. That is executed already. So we can kind of look at what has happened. So first, let's look at the training step. See what training logs we can get. We can see all this is going on. So we can see for the failure condition is evaluated default. So it hasn't failed yet. And then the success condition is also false. So it hasn't succeeded yet either. And it will continue to check that until one shows up true. And in our case, it succeeded. So we've succeeded in training it and then it'll. Store all the training parameters so that we can. Go back into the model fairness check and the average or robustness evaluation and kind of take a look at what we've got there. So here we see some of the metrics.  We're just going to look at one for time constraints. So for the disparate impact. So the way that the disparate impact works is that it takes the probability of a favorable outcome for the. Unprivileged group and divides it by the probability of a favorable outcome for the privileged group. So values of less than 1 generally indicate bias towards the privilege group. And values close to 1, 10 to show that there's no bias. So generally we go by the forefist rule, which means anything less than 80% shows some sort of bias. And so we have a 77% here, which would indicate that likely there is some bias here and we would need to go and mitigate it in some way. And then let's take a look at the robustness evaluation. See how that went. So the model accuracy on test data in general was 85%. And then on our adversarial samples, it comes out to only 15%. So this is not very good. It indicates that we would need to do more robust training to kind of deal with adversarial. And to go a little bit deeper into that, you can also get the average perturbation on misclassified samples. So small values show that your model did not perform well at all. And in fact, very little perturbation will cause a misclassification, which is not good. And if this label is very high, it shows that it took a lot to mess with your model and get it to mispredict. So you would prefer higher numbers. So yeah, in this case, you would find that likely you need to both check on the bias and to check on robustness in your model training step. And so you could go and rerun this training step, perhaps with new data that you've created based off of the model fairness and adversarial robustness data that you created in both of these steps. So jumping back in. Thanks, Andrew. That was great, right? So I think in the interest of the time, we will probably, you know, speed up some of the rest of the slides, right? So I think I gave a quick background around K of serving, right? So it's the solution in our QFLU community for production model serving, right? So it gives you a serverless machine learning inference capability built on top of K native, right? And then, you know, there are capabilities around model explanations under which we have built our rest of the trusted AI umbrella, right? So as part of that, right? So Andrew has been working heavily in terms of integrating some of these projects into K of serving, including, you know, explainability 360 or any FNS 360. And, you know, we had the first version or done and delivered as part of the K of serving these 0.5. So if you're interested, you can definitely try it out. A lot of times it's not possible to do the analysis on a single transaction, right? So you cannot just based on a single transaction say that the model is biased or not, right? So in those kind of scenarios, right? You need robust payload logging capabilities. What that means is that, you know, you're able to capture the requests which are coming to models, the responses which are going back from the models, collect, curate, and then, you know, over a period of time, once you have that data, then you can run more advanced analysis with it. Over the course of last hundred predictions, the model is biased, right? Or, you know, 80% of these inputs which are coming there, you know, adversely modified and model is getting full, right? So to do more advanced analytics like this, you need payload logging, which we have enabled using, you know, the cloud events standardized protocol, which is a standard which is being developed by the C and CF community. Okay, so I'm going to talk a bit about these capabilities as part of the KF survey. Yeah. So what we've implemented in KF serving so far for the AX is using a method called locally interval model explainability in Lyme. And the idea is you take an image on the left like an MNIST image and the explainer will highlight pixels that it finds are highly indicative of a certain classification. So you can change certain parameters inside of Lyme to get different highlighted pixels if you want something that is like the pixels that are most indicative of a classification, then you would raise a parameter called minimum weight. And then the next project, every server bus install box is using the square attack method. So you take an image, you add some sort of perturbation or noise. And you'll be given with another image, which is very similar to the first, but makes your model misclassify or at least is the goal of the art tool. And in most cases, that'll just be small perturbations that look identical to the human eye. For AF 360, the idea is to collect a bunch of payloads, as we mentioned, and then through the payload logs, we can create bias metrics based off of a large collection of instances and outputs. And so there's a number of specifications that you need to have here. Like we saw before in the KFP example, there's the favorable label, unfavorable label, privileged groups and unfavorable groups, and this just lets you know what is a favorable outcome and what are the bias groups that we may need to look out for. And so the same deal with the spec here. The left is the same as you would deploy a normal predictor. And then the right is all the extra commands and information that you may need in the explainer to give you something valuable. And so the flow of the AF 360 is you send your prediction request to a model, then the model sends their logs to some sort of persistent storage. In our case, when we show the demo, it will be a message dumper. And then when the bias detection request comes through all of the logs and the persistent storage will be sent to the AF explainer, which will then be able to calculate metrics based off of those logs. So we'll show that here. So this is housed in the KF serving repo. You can just go to the samples and look for it. But the basic idea is to deploy the model, the bias the predictor and the explainer here and the message jumper. We've already had that set up so that it doesn't take a lot of time to spin them up because they are fairly large images, at least for someone. Yeah, so there they are. And then the idea is to connect to it and send simulation prediction requests. So I have that here. So we'll just simulate sending multiple requests just to pull some logs into the message dumpers. You can see we have about six requests and this is the responses we expect to see a few predictions and both the requests and their response will get logged. And then the idea here is that we need to create adjacent from the logs. So let's just take a look at what that looks like. It's created this data.json, which is just adjacent instances, which are all the logs that we sent to it. There's quite a few because we added some information before just for added information without taking up too much time. And then the outputs here are all the prediction responses that we've got. And so now that we have those, we can go back and we can actually query our explainer to check what bias do we see. So there we go. And that's pretty quick. So now that we have that, let's look at the disparate impact. So we can have a kind of comparison to the KFP. So again, disparate impact is measuring favorable outcomes for privileged groups and non-privileged groups the ratio. So this is much lower than the fourth fist rule. So we have a very low value of 0.5. So we would say that the model that we have deployed in camp serving is very biased. And we, what we can do since we have the payload logs for this is we can go back and we can see what the biased examples were. And then we can check if our model behaves differently when we change those examples to make them the privileged class. And if they do, then we can save new data for when we would like to retrain and keep the same examples before but say that those examples should be marked with a favorable label rather than the unfavorable label that they're giving. And this will expand our data set and make it a little bit better because we know that if it is given to the privileged class they are getting acceptable results. So now we'll hop back in. Over that, right, we are also doing a lot of great work and advancing a lot of the principles of trusted AI as part of the Linux Foundation AI trusted AI committee. We meet, you know, monthly where, you know, a lot of the partners like, you know, the folks from the companies which are listed there or, you know, folks from Microsoft and others, right, we come together and talk both about the principles. What should define the principles of trusted AI but also, you know, technical solutions like we discussed today. Right, so we are interested in this. Definitely, you know, join this. It's an exciting place where a lot of us, you know, people who are actually interested in advancing this field are getting together and bringing technologies and tools and use cases together as part of this Linux Foundation AI trusted AI. Here are, you know, some of the links in terms of, you know, some of the GitHub repos or the design document, right? This is the KF serving payload logging. Plus, you know, if you do need to reach out, like, you know, or Twitter or LinkedIn, URLs are listed here. So definitely, you know, try and reach back to us if you have any questions, comments, feedback. We would love to get your feedback and make sure that, you know, you're using these capabilities, advancing these capabilities and the ratios which you'll find or your suggestions, we would like to incorporate and advance these projects together, right? So in there, you know, I would like to round up by saying we definitely want to stand up for ethical AI. We are doing quite a bit of efforts that are both as well as through IBM, but also, you know, Linux Foundation AI has been a huge partner, the two-flow community has been a huge partner and we would like to invite all of you right into this mission in terms of bringing ethical and trusted AI to the masses and building, you know, platforms which are essentially, you know, making this core part of their DNA. Thank you. And thanks, Andrew.


your pitch count really got elevated in that first inning and throughout. How do you look at this outing. A lot of soft contact. I'm pretty pretty happy with it. Struck out seven guys just had to get a couple extra outs out there with a good lineup like that. You can't give away any free outs. So got to do better. Get more quick outs and just maneuver the lineup better. But I think I battled and kept us in the game. Frustrated by that air leading to throwing more pitches in the first in particular. I mean yeah I made two errors. So I've got to fill my position better. But like I said I had to buckle down and kind of just make pitches when I did. What do you think about the way your offense chipped away before eventually picking up the win in extras. It's great. I mean we're always we shown we can put up seven runs in an inning. So it is always a matter of time for us when we're something like that. Just who's going to be the one to break the ice almost.


You have a new traveler coming our way, a need of personalization, more and more. You need to engage one-to-one. The type of flight air booking is also changing from more long haul to more regional travel. Recognizing the trend towards digital nomads, we have entered into a memorandum of agreement to improve internet connectivity across the Philippines. We are also continuing with offering wellness and work, staycation packages. Well, in the last years there is a trend looking for sustainability in part of the visitors. We are the first destination in Spain that has created a calculator to see the impact of the tourist in our destination.