Thank you, Stuart. First of all, I'd like to say I absolutely love New Zealand. This is my first time. And one of the things that I have noticed since being here is that I cannot hold a coherent thought in my head. So I have been unable to create a thought and then actually vocalize it. I have no problem vocalizing it, but I have no idea what I'm about to say before I say it. So I hope that we both get some benefit out of our time together. So the name of the talk, Garrett and Gertie. So who knows what Garrett is? Yay, who knows what Gertie is? I read your talk submission. Awesome, awesome. Okay, great. Well, we'll get to what Gertie is in the talk. Who the heck am I? My name is Anita Kuno. My nickname on IRC and most everything else is Antea. Let's move that mouse, shall we? Go over there. I'm a member of the OpenStack infrastructure program. Who would know what that would mean? Who knows what that means? Okay, why don't I explain? OpenStack is a very large open source program to provide an open source cloud computing solution written in Python. It is comprised of a number of different programs. Right now we're, I think we're at about 21 different programs. The infrastructure program is the team that sets up and runs all of the services that are necessary for the development community. So that is any of our communication tools, all of our automated testing. If we use paste, if we use, we use a tool called EtherPad. So the infrastructure team runs and does all the system administration for that. I'm not a system administrator. I do a lot of gluing things together and running and helping out and communicating things while other people are busy fighting fires. So that's kind of my role and I like to move around a lot. So I do different things within infrastructure program, but that is my home. I spelled among wrong. I can't spell. I am among the top 50 reviewers for OpenStack. OpenStack, as of when I checked this morning, is two, I think it's 2,103 reviewers that are being tracked right now. So I'm in the top 50 of those. And I started during the grizzly release of OpenStack. We're currently working on the kilo release. And I'm an employee with HB. Things that I had to guess about you. I really enjoy doing talks, but I'm bad at predicting what people want without asking them. So I had to make some assumptions about what you want from the talk and why you're here. So I guessed that you understand the value of code review. How accurate am I on that one? Value of code review. Yes, code review is valuable. Awesome. You recognize a need to improve review throughput. Yes? Okay. So review throughput would be your ability to review the patch contributor's ability to come back, your ability to review again, and also to be able to distribute that out. So teach other people how to review so that you're not the only one. You have, I also guessed about you, that you have the ability to evaluate your behavior and implement modifications. Now those might be modifications to your behavior or those might be modifications to code that affects your behavior. Would that be an accurate assumption? Some of the people in the room? Okay. All right. Some what? Some what? That I can convince you the value of that last one, of being able to evaluate your behavior and implement modifications. You are hoping to get something from this talk. I hope to be able to give you the location of the Garrett documentation. I hope to be able to tell you what GERDI is. And I hope to be able to help you understand the anatomy of a habit. And I apologize for the line wrap in the title on that slide. And this one as well. So I have long titles for slides. Sorry about that. Why do we want to increase review throughput? Code review can be a bottleneck to development. Who has experienced that in your workflow? Who has experienced where either someone else's code review habits or your code review habits provide a bottleneck for development? Okay. So all right. So quite a few people in the room have experienced that. We can increase review throughput by distributing the load for more reviewers. So hopefully we can take what you're actually doing that works and help you communicate that, give you a tool to communicate that to others so that they can review more, start reviewing, review better and help to distribute that load so that there is more activity happening for reviews. Also, if you include more people reviewing, you improve the code because you've got a diversity of reviewers. If you've only got one or two people reviewing the code, then they're actually going to start driving the design of the code because it's their aesthetic or their values that are saying what gets merged in the code. And if you increase the diversity, you're getting different people's opinions, different people's experiences with how they want to see, and you actually have to justify what your decision-making process is rather than saying, oh, it looks good to me. So what I'm going to try to cover in our talk today is the... So for habits, the anatomy of a habit and the implementation of a habit. Garrett, a very, very thin overview since most of you already know what Garrett is and some technical suggestions. And with Gertie, what is it and how do I use it? So for habits, a definition of a habit is an acquired behavior pattern regularly followed until it has become almost involuntary. And I got that off of dictionary.com. And the two things that are important about a habit. One is that it's acquired. It's not something that you're born with. You acquire it through continual behavior, repetitive behavior, until it's almost involuntary. A habit contains no decision-making process. Once you're in the habit, it's a routine, and it does not finish until you've hit the end. So the anatomy of a habit. So we start over here on the left-hand side with a cue. So a cue can be anything that triggers the beginning of the habit. We move up to the routine. That is what someone else would observe as the habit. And then we finish with the reward. You can think of a habit as a cron job. So a cron job, you've written a cron job, and it starts quite often as cron job based on time. So the clock changes. The event, the trigger, is a certain time happens. And then the job, if it hits the filters for time and day and frequency, then it will execute a behavior. The behavior is whatever command you told it to do. Then, hopefully, the job will finish at some point. And that is the reward. So the cron job has to know when it's finished. So hopefully by executing, when it's finished executing that command, then it will stop and that would be the end of the habit. Does that make sense in terms of thinking about as a cron job, in terms of understanding the habit loop? So a habit is something that has a defined beginning, an observable middle, and a defined end. So the difference between a habit and an addiction is an addiction doesn't stop. Can you see that? Yes, unless you can't. It's a slide of goats running around and around. So they've got a series of benches and goats love to climb up hills. They want to be on top of things, a bunch of baby goats, and they're climbing up to the top and then they're jumping off and then they're running as fast as they can and knocking other little baby goats out of the way so that they can climb up to the top again. And so they just keep going around and around because there's no stop. So that would be considered an addiction because there's no end to it. So starting with a cue, what could be considered a cue?  So the trigger for this one if you were getting together with friends. So I'm sure that everybody has a group of friends and every time you get together you might have a ritualistic greeting. You do the same greeting. Maybe you meet at the same time of day or day of the week and maybe you go to the same place and maybe when you go to the same place you consume the same food and you do the same activities. So one of the triggers might be when you get together and you hang out with some friends. The routine part is whatever the behavior is. So the cue is whatever set it off and said this is the beginning. The routine is actually what the observable behavior is.  Your brain can't tell the difference.  It's a part of your brain that has to act as a routine until it's finished. So something happens you're going to execute a behavior. The reward is the feeling of completion. You have gotten to the end of the routine and the reward is how you acknowledge and say this is the end. So some sort of sense of completion. This is why using food as a reward for a behavior and lead to a behavior because the eating of the food is not linked to hunger or nourishment. It's linked to identifying the end of a routine. And so if you link a routine with a chocolate chip cookie in order for your brain to say we're done now, you have to consume the chocolate chip cookie. So using food as a reward for behavior is probably not the best choice. Pick another reward that somebody can do repetitively over and over again and it still will be helpful for them. So what? So how does understanding the anatomy of a habit increase review throughput? So if you are looking to increase your review throughput, the first thing you need to do is select your reward. Doing reviews doesn't tend to be a reward laden activity. In code submission, in development, it's submitting the code, getting the code merged. That is the point where everybody can acknowledge that, wave the flag, we all get gold stars because everybody can say we did something. We have provided value. We have lines of code with our name in them in the code base. That is not the same thing with code review. It's hard to do. It requires a lot of work and focus. You have to understand the code, sometimes to a greater extent than the people writing the code. But you don't have the same sense of reward or completion as people who contribute code and get code merged. So in order to motivate people to do reviews, you have to be able to find a tangible reward so that code review becomes desirable. People have to be motivated to want to do code review. We have a tool written by one of our contributors. It's available at that URL. If you go to git.openstack.org, that's where all of our code repositories are. Russell Bryant wrote this tool, and he contributed it back upstream. What it does is it pulls Garrett, and it takes a look at reviewer statistics. These reviewer statistics are for just the infrastructure project for the last 365 days. You can see that C. Boylan, that's Clark, one of the members of the infrastructure team, has done 2,876 reviews in the last 365 days. So I use this tool myself when I started reviewing and I still use it because I use it the same way as I would motivate myself to walk. So if I'm walking and if I did 5K one day and 10K another day and 2K another day, I like to write that down. Not because I'm in competition with anybody because it helps to motivate me to have something to measure. I can write that down and I say, I did this today. I don't have to tell anybody, I just know myself. And so when I was getting started reviewing, I would just start out and look at my statistics and then I'd say, okay, I'm going to have to do 5 more for tomorrow. Or I'm going to do 1 for tomorrow. I'm going to make sure that I've got 1 more for tomorrow. And that was my way of measuring because when you're in reviewing it can be such a big ocean you can't recognize yourself. So you have to give yourself tools to recognize what you are doing and being able to measure it. If you can measure it, then you can see where you are and you can evaluate whether or not you're improving. So if anybody would like to use that tool for their own Garrett implementation to take a look at that, you're welcome to. It's an open source tool, grab that. It will be configured for the open stacks Garrett implementation, but you can go ahead and change that and use it for yours. So the point of this one is when you're developing a habit you have to start off by selecting a tangible reward that people are motivated to be able to evaluate for themselves so they can tell when they're done. The second, for the actual routine, is filtering. Code review can be overwhelming. So people come to open stack and they say, how do I help? And we say it would be really great if you could review some code. So then they go to Garrett and they look at the open reviews and at any given time we have thousands of open reviews and it's overwhelming and they don't know where to start and then they go and they start merging code or start submitting code to be merged because code review is so overwhelming they don't know where to start and they don't feel that they have a place that can be a foundation where they can start learning. In creating a habit, habit is a pattern. So you have to be able to put yourself in a situation where you can repeat the behavior. And it's not important that you can repeat the behavior many times in one day, but that you can do it that day and you can repeat it tomorrow and you can repeat it tomorrow and you can repeat it tomorrow. When I tell people to start reviewing code they say, oh, well it's so difficult and I don't do enough reviews. Just do one a day. Do one review a day. If you can create a habit where you're doing one review a day you will get to the point where you will want to do more. But even if you do one review a day in 365 days you've done 365 reviews and that's helpful. But if you have in your mind that you're going to do one code review a day that's when you actually start developing the habit of a repeatable pattern and then you will eventually want to do more reviews. In order to develop a repeatable pattern I use this query with Garrett. And what that query did is that brought back all patches that had touched the ACLS directory. And there was such consistency in those patches because it was a new repo creation. So the Garrett ACL that's access level control so that's Garrett permissions. And that's what you need when you're creating a new repository. So there was one pattern in all of those patches. I started in just reviewing patches, changes to that directory and then when I learned I started expanding so that I could review all of those kinds of patches and now I'm a core reviewer on the repository that contains those kinds of files. So the point with this is to select a query, select a routine that you can repeat. The minimum amount that's effective that you can repeat day after day after day. People want to bite off a big chunk. You don't do that. Bite off the smallest possible that you can do every day. Identify a trigger. So you've got a reward. You've figured out how to create a repeatable behavior. Now, so you're working backwards in the habit loop and you need to identify a trigger. So the trigger can be anything that works for you. Location, time of day, your social needs and completion of another habit. So I went with completion of another habit and what I do is as soon as I log on the first thing I do is I sign in to everything. I get my recent emails and I've signed on to IRC and I read the back scroll and so on. And once I've got kind of the news, that's when I open up Garrett and that's when I bring up my custom query I'll show you in a minute and that's when I take a look at my personal, my own review queue. So with Garrett I construct a query and it looks like a big mass and we'll go through it in a minute but you construct a custom query and that's mine. Everybody I know has their own custom query but if you create your custom query that's creating your filter for your repeatable behavior. It's bigger now. It collects more than just the access level control files because I have that habit and so now I'm able to expand so that I bring in everything in a single repo. So with Garrett, the big question when I'm talking to people about reviewing there is internal documentation in Garrett about how to create a custom query, how to create a search query. So you go to documentation right at the top level navigation. When you click on the documentation the bottom navigation will change. You will have an option for searching. Click on searching and what's down here is what the result will be. It will open up in a separate page and you'll have Garrett documentation about how to do searches. If you don't know anything else about Garrett learn how to use Garrett custom queries. They are how you filter down so that you've got files in a single repository, changes to a specific repo, so that you have a query that you want and you know what you're going to expect so that your brain can fall into that habit and that you can review. So you construct a query. The top part, the top phrase, I want to take a look at patches that I didn't create. So if I'm reviewing I'm not going to review my own patches. Some people review their own patches and then we have to comment and say that's not a good idea because we know you like it. You gave it to us. So I don't want to see my own patches. That's for other people to review. I want to review patches that are open. In Garrett they can be open, abandoned, merged. So I want to see open patches. If something is merged, I don't care. It's already gone. I don't need to review it. And if something's abandoned, again, that's a dead-end trail so I'm not interested in seeing those in terms of reviewing them. Labels are helpful. I want to see the labels that I have not voted on. Plus one, minus one, plus two, minus two. So anything that comes from me that has code review zero. That also returns patches that I've commented on if I have not voted on them. But if I've already voted on them I don't want to see them again. And if that patch gets a new patch set to it that will pop up again because the new patch set my vote will have zeroed out with a new patch set. So I have to review that again because it's a brand new patch set. But if that's still in the open state but I've already reviewed it I don't want to see it again. Label workflow zero with Garrett. You can set it work in progress. That's minus one for workflow. And if it's approved that's plus one. If it's work in progress I don't need to see it because it's not ready for me to review unless somebody specifically asks me to but I don't want to return to my regular review queue and if it's plus one it's already been approved I don't need to review it. And I'm going to take a look at patches that are submitted to that specific repository which is the open stack in for project config. So I am running out of time so we will just move through the rest. So this, if I run the query in Garrett this is my review queue. So I've actually got a page and a half because it's about 20 patches returned on a page. So I've got 30 patches that meet those criteria which is better than 500. So I will go through and it doesn't mean I'll get through all of them in a day but I start with my review queue and the thing is because the query says that if I've reviewed it I don't want to see it so I review it and then I refresh the page and what I just reviewed is gone. So as I'm going through my queue is getting smaller. GERDI. What is it? GERDI is a console-based interface to the Garrett review system. So if you're using Garrett for code review and you'd rather not use a GUI you'd rather use a console. Garrett, our current Garrett which is 2.8 has a REST API and so Jim created GERDI for a number of reasons and Jim's back there. Thank you, Jim. If you would like to take a look at GERDI it's available on PiPi. Why was GERDI created? It has a local database so it's good for offline work. We have a number of our developers who fly around in airplanes and so it's difficult for them to be able to sync changes with and use the GUI so they can work offline and then when they land and sync back up again they can submit their changes and their reviews. Some folks like to use a console rather than a GUI and Jim also writes good software. For GERDI configuration you can configure GERDI use the same queries as Garrett. One of the things that I have done is I've been using GERDI more and providing Jim with lots of bugs or unusual queries. Jim is working through those bugs so that they're able to match exactly or as close as possible because Jim has his own parsing to pull it out of the local database. You can map keys to certain actions so when you're reviewing you can submit plus one, plus two. We call plus two an approval we call that plus three and we can map that to a key. If you do a review of minus one that should always include a comment of why you've submitted you've chosen to go minus one on that patch as well as some instructions as how that person can improve the patch and get to plus one. So this is how it looks like in my terminal so that's all of my open subscribed patches with GERDI it doesn't pull down everything in Garrett it's not big for local database so it just pulls in projects that you subscribe to. And this is a Josh Hesketh patch and what it looks like in GERDI so that you have all of the information and then you are able to go through that with your keyboard. If you want more about GERDI you can learn more. Jim has a talk Wednesday scheduled at 2.15 but keep an eye on the schedule in case it changes called REST APIs in the return of the console app. It will have nothing to do with how to use GERDI and it will have everything to do about why Jim was motivated to create it and why you as well should write your own console app using REST APIs. If you would like to file and fix bugs that is the storyboard link we use storyboard for the bug tracker for GERDI. So closing thoughts on habits you can never remove a habit you can only create a competing habit or a more compelling habit so if you have a habit in your life and you say I don't want to do this behavior anymore you have to have a more compelling habit that draws it away. A habit never goes away so you can't try to do nothing you have to do something that's more important to you every time that you feel that trigger you do a different behavior. A habit won't stop until it's reached the end of the file so when you do that pick ways for you to be able to acknowledge that you're at the end habits are not logical so anytime you're trying to argue with somebody about something that's a habit, habit is not logical so if you just accept that then you're able to work with it better and once a habit is created the thought isn't no longer involved so with people saying I won't do that because I'm smart, smart has nothing to do with it if the habit is there, the habit is there and it won't change unless you do a more compelling action on the same trigger. I want to thank you very much for your kind attention we've reached the end and I am out of time and I don't believe I have time for any questions do I have time for questions, Stuart? Your call. Is there anybody with a question? Either because I bored you all or I was so thorough I answered them all as I went along thank you for your kind attention.


trunk stability is so important when we canter and the reason for that is is It allows our hip to function like it's supposed to This is a super super simple exercise that is going to help you create the stability in your trunk What this exercise is is taking your hand and Pretending like you're at school and you really want to be called on alright So I'm like call on me, but what I'm doing is I'm really lengthening and stretching through the entire side of my body When when we do this sort of thing it can also cause us to be more aware of our focus And that's another piece of the puzzle that is so important in trunk stability If our focus is down we tend to collapse we collapse our rib cage on the front side of our body So we want the front side of our body to match the back side of our body So we're going to practice raising our hand above our head Creating that trunk stability and then getting our focus to come up when our focus is up We it can be aware of our surroundings our horses are very aware of their surroundings They're always looking around if you look I'm riding on a long rain He's looking over there and then as I come through here He's gonna look off in the distance here and then as I make my turn. He's going to look over in here So he's always looking around and if I am focusing just on his head I cannot be aware of what he's aware of that's going to get me in trouble because I might be not seeing the dog That's about to come running out of the trees and that dog might spook my horse or on the trail I might not see the turkeys up ahead that my horse has noticed because I'm looking down at them So by by getting our focus up and looking around and being aware We enter into the horses world a little bit more we can anticipate what's about to happen And it has the added benefit of helping us find that Front side that equals our back side. So creating that rectangular torso where we are Nice and even and solid through our middle. I Want you to watch my hip now too because I talked about that in the beginning But I want you to see it and and be aware of how this all goes together because our whole body is connected So if I'm looking at my horse's ears and if I'm slouchy in my body Can you notice how my hip doesn't function nearly as well now if I look up and I'm looking where my horse is going Can you see how my hip starts to open and close and follow my horse's movement at the canter?


The completion of Derek Jeter's March to 3,000 hits ranks among the most unforgettable events John Sterling has witnessed in over three decades on the Yankees broadcast team, many of which are in the Yankeeographies and Yankees classics that he hosts for us. Here's John with Bob Lorenz. All right, thank you. I'm here with the legendary John Sterling. I can call you that. That's okay, right? Sounds great. Thank you. Thirty, forty years ago, maybe they weren't ready to call you that, right? When you were doing a variety of sports, but now that you're locked into Yankee's baseball, you are the legend. Going into your 34th year, by the way, and I wonder about this, John, spring the season is always a renewal, right? People get excited. It's starting to get warmer and it means baseball is back. So people get excited about that. How excited do you get about it? I'm really excited this year. In many ways, I'm still emotionally a child and I've loved baseball since I was, you know, seven or eight, as well as basketball, hockey and football. Now, I think I'm very fortunate. I've had this great companion with me all my years. You can add music to that. I've had these two companions with me, so and if you say, gee, all these years, you know, don't forget the six years with the Braves, right? So it's 40 years of doing baseball and I'm as excited as ever. We're obviously airing this on yes, you've been a part of the network since it started over 20 years ago and you do Yankees classics. You do Yankeographies. When you are hosting those, does it take you back and bring those memories back to you of all the games you've seen? Very much so. Yeah, it began, well, 20 years ago when yes began and we did a whole flock of Yankeographies. In the Monson Yankeography, there's a time where Bobby Mercer is on Mike and he breaks up. Right. And so I'm sitting in the studio and I break up. Wait, we have to do this again, you know, that kind of thing. So very much live those moments. Now you obviously have a lot of great moments in terms of your radio career with the Yankees. Part of that, if we go back 30 years from now, that's when you started with Michael K. On the radio and we're all curious. Yes, what were your first impressions of Michael K and how well you might work with him? Well, here's what I did. We auditioned a few guys. They had a, I'll tell you what the game was. They had a, it was a game on I think Memorial Day and Mel Hall hit a home run on the night that ain't for the Yankees to win and that's what they used to audition. And I said, I know who it should be, but you have to decide and then I'll tell you if it's the right one and we all pick Michael K.


This is your program This is your program when it crashes and we don't like that Obviously when we're writing code because when things are on fire Our our code doesn't work So we never want our programs to crash especially when they should be operating like normal a great example of this Would be let's say for example I built a simple little function called divide takes an enumerator takes in a denominator and returns the You know quotient so the entire idea here is that well what happens if I attempt to divide? 10 By zero oh crap I tried to divide by zero and my program is going to error out and I don't want that all the time That is bad dividing by zero is bad Well in that case we can use exception handling to process our Data so that it's a little safer when we're running our code So the way to think about this is there are a number of different types of problems first one is something known as a syntax error and you were Doing syntax errors anytime you run your code and oh, I don't you know you misspelled a word or something That's actually what the typing exercises that you do for class are built to help out with But specifically we happen to have two other problems that we may need to work off of logic errors Not necessarily, but they can sometimes happen, but specifically we're looking at The runtime error if you think about it when I was working off of this code my program was running perfectly fine There are no like typographical errors here and technically this is not a logic error This is how you divide numbers So the issue specifically is that by passing in that zero parameter Again, I'm attempting to divide by zero laws of math say we are not capable of doing that And so as a result Python is not going to try and break the laws of math. It's just going to crash So with that in mind again, that's where we get into the idea of exception handling and that idea of an exception What we're thinking about here is that when our code is running there may be some Issue that causes the code to crash and that would shut down our program But maybe we don't want our code to crash And in that case just as the you know slides sort of describe it as we want to handle that exception, so how do we do that? We use some more keywords specifically we add in something called our try and our accept Now try is effectively going to say just as you can probably imagine try To do this Whatever that happens to be if it were to crash so if that crashes do this That's in a nutshell how exception handling comes into play So if we were to come in and for our sake Let's attempt to try To divide by zero again, you know, let's just try to do it Except now I'm going to start with just accept Accept is going to be a catch all whatever crash happens. Just that crash is going to happen. So print You cannot divide by zero so we've now built that out and Now if we attempt to run our you know divide by zero Instead of a crash happens. We get a print statement that says you cannot divide by zero crisis averted Now the issue is that what happens if instead of trying to divide by zero I try to divide by a String I try and divide by cat. Well, that's not a number and in fact the issue is that We're still getting the same error message We're still getting say for example explicitly the catch all of you can't divide by zero So let's take off The try catch for just a moment and now see just what error is going on there Type error. Oh specifically notice again that these are different types that Python can Present to us and there are a number of these Not every one of them is going to happen all the time some are more rare So it depends on really your code, but Specifically to our sort of example here. We are experiencing a zero division error because we're trying to divide by zero and a type error where we're using an inappropriate data type So in this case for example This is where we can use different types of Exceptions or handle different types of exceptions. So once again, we'll reintroduce the exception handling but instead of just saying Except we're going to add in a space and then denote that I want a specific type of exception to be handled this way zero Division error only a zero division error should be handled this way. So in this case Zero division Did I misspell it? No, sorry So now zero division errors will print out the you can't divide by zero However, if I try and divide by a cat, I still get the type error This is one of the things where depending on how your code operates. You're actually able to nest multiple exceptions Just like if we were dealing with Conditionals and then using L ifs. So if you think about it, there was the if statement and then L if New conditional statement L if a new conditional statement L if a new conditional statement Sort of the same things going on here Try to do the code if there's an error if that error is a zero division error Do this L if it's a in this case not a zero division, but if it is a type error Do this and so in this case we'll say I you Cannot divide strings Okay, so in this case once again, we're handling two separate exceptions We've loaded that in the memory now if we attempt to divide by a cat Instead of seeing you cannot divide by zero we Instead caught our type exception and outputted you cannot divide by strings


Freedom, today I want to talk to you about something really, wow, it's been a while since I've been in this set. Anyway, I want to talk to you about something really, really important. Freedom is working very hard and we're trying to work very quickly to bring you some very awesome features such as an ad sales team, ad buys, which means more money for you. We're also looking to bring more channel discovery. A part of doing this is we are going through every single channel in the entire Freedom family and you can help us do that. You can help us make this a reality a little bit quicker and you can get benefits for doing it. Right now, if you look behind me, oh god, I sound like George when I say that, you can see that we have a new feature within Heartbeat that allows you to look through channels, tell us what kind of channels they are, whether they're good channels, bad channels, copyright, empty, so on and so forth. By going through these, you will earn points. These points will be redeemable for places on leaderboards and other miscellaneous promotions, things like that. Going forward, more details on that later. Let's not get wrapped up into details. The most important part about this is that this is your opportunity to be a part of the Freedom family and to show a piece of emotional ownership over your part of the Freedom family. You can start with your own channel and move forward from there. Now these will be verified and re-verified and re-verified so people who maybe don't actually look at the content of the channels, those votes will be taken back and you won't get any points for them. So it's very important that you actually do this because you want to and because you're trying to help, not necessarily just to do the most, to get the most points. These points will be used for a leaderboard per se, which we will use to promote people within the Freedom family, which we will use to give you different kinds of promotion within Heartbeat and our various hubs and things like that. This will be used as a way to figure out who the most helpful and the most awesome people in the Freedom community are. Last but not least, it's been way too long since I've done one of these videos and when did we get a coffee machine? Anyway, back on task, make sure that you update your installation of Heartbeat and get the latest version because only then will you see the proper way to help review channels. Till next time, don't forget to like, subscribe, do all those things that make me love my job. Also, be awesome to yourself and amazing to each other. Bye Freedom family.


Beth yw'r unigio o gynnig teimlo'i siwell gwahanol â'r firm hon yn fwy o'r hyn gyflym yn i fi yn byw iymaeid? Mae'ch gwyn dechrau, i wneud hwn i'r byw hyn. Mae'r llunur wedi'i cael ei ddau. Yn dweud ei ni'n gwiannod i gael gor breathtaking er mwyn i gyda'r bullwll. yaredd card not to suffer stride a little bit. I thought we just relaxed a little bit too much and maybe wanted to coast through the rest of the game a little bit a little at least till a halftime. I thought they had a couple of opportunities on the counterattack because they had a lot of pace on the team. I thought we just put the game potentially at risk before halftime. felly, felly dnowid, nid oedd gyfo'r boedd, fel we wedi rhoi gydag, fel bod y bydd hyn yn fwy fwy o'r ddeithas. Ddiweddu'r ddaig i'r awyr ei gyfun, felly mae'n teimlo dugoed. A ddweud o'r sleidio'n gyfrifiadol, y ffordd dw i'r cael rhywbeth, yn y peth arddangos i'r tyfnewid. Mae'n olio'r sefydlu y ffordd yw'r hyn ychydig. Mae'r ddweud wedi'i dwylo, dyfoddu'r golygu am hynny o felly. Gallem ei fod yn eu gwneud o'r cywethaf sydd yn ymgyrchol, i nad yw i'n gwneud o'r cyfeirio, ond byddai'n gwneud o'r cyfeirio i'r cyfaint, rydyn ni'n gwneud y cael gwybod y dyma, rydyn ni'n gwneud o'r cyfaint o'r cyfaint, oherwydd i'n gwneud o'r cyfaint, a'r gwneud o'r cyfaint o'r cyfaint o'r cyfaint o'r cyfaint o'r cyfaint. Oedd e'r ffa starb gennym i'r seven o'r munud. A'r oeddwn i'r iawn i'r un gwaith yma, yn yr oed o'i cyfrifio'r gweithio'r gwaith a'r oed o'n dweud yma'r ysgol, yn sicrhau bod brush ar yr awbent, ac mae'n ddiddordeb yn cael ei gwerthio'r gwaith a'r awbent yn ei gwerthio'r gwaith, sydd yn ddod 3 o 4 o'n dwylo, a nhw'n eu cyfieithio. Felly mae'n gweithio'n cael ei gwerthio'r gwaith, mae'n gweithio, mae'n gweithio'n gwaith o'n gweithio'n gweithio. Ie, we want other players, all the positions to offer a goal threat. So you have to create more than the chances you want to score, because you're not going to score from everyone, so we encourage the players, the strikers, the midfielders, the wingbacks to get forward as often as possible and support each other, be connected. Even John Tomkinson, first half, Overlapping Brad and things like that, we want to create pressure and momentum, and we want to create opportunities to score. Because we know we have goal scorers within the squad, we understand that bit. It's good for them and their confidence that they know the work they're doing is paying off by scoring the goals, so we have to maintain that and keep working hard. On those goal scorers, another two for Tyler Smith, nine for the season now for him, another one for Andy Cook, eight for the season for him, and also for Dan Oliver coming back and scoring a goal as well to have all three of your strikers on the score sheets tonight must be really encouraged. Yeah, well, for strikers that's their oxygen, isn't it?  So what delighted me is that the stuff they're doing off the ball to help the team win the ball back and then create opportunities for others. So it's not just about being on the end of things, they're actually contributing in the build up play and our press phase and how we defend and they're getting rewards. You know, if they do that work for the team early and correctly, then it should create more opportunities for them to score. So we know when we get them into good areas, they're clinical, they can score. So yeah, it's a good night for those guys, but delighted with the work they do off the ball as well. I imagine pleasing to have had a weekend off and still maintained the level, but I would imagine again now a positive that we've only got three or four days until our next game and we're looking to keep this momentum up going into the weekend. Yeah, it's a difficult weekend because it's not having a game, but with the weather enough and the training was difficult. Basically I had a snowball fight on Friday in training, so it was good that we got on a lend me an arm and out of it for a little bit. And we looked at tonight's game, so we didn't want the weather to be an excuse for not preparing right. So I'm delighted that the players took that forward into the game. Sorry, it was Monday when it was the snowball fight, not Friday, but the day's mixed up. But what we did was it was a difficult pitch in training, but they did it right and hopefully what they saw was a little bit of tactical stuff paid off and how we pressed Liverpool tonight and got our awards from it. But yeah, we go straight into the weekend, another big league game that we want to do well in, we want to continue this form if we can. So when we come back into training, we'll assess the players where they are physically, because we're going to have to output a lot of energy on Saturday to win the game. And so we'll see what team is ready to do that on Saturday. But the players, like I said last week, they got a bit between their teeth and they want to stay in the team and they want to win.


During the break I was told why do do you use the representation greater or equal to and Not less or equal to and there was one Thing that I wanted to show you one of the reason typically suppose you have a Close with 10,000 literals Okay, the way you are going to represent it with greater or equal to one is you're the sum of the 10,000 literals Greater or equal to one Okay, now if you want to represent that close with less or equal to You are going to use the sum of the negation of the literals Is less or equal to 10,000 minus one? So you see there is no free lunch. I mean you use one representation So typically that greater or equal to is the best representation if you need to go from closes to something Different so that again the closes in that representation are really the the best thing you can have right? so Yeah, it has been so if it's the case. It's not just by chance, but it's there is really a reason why it is the case So should I go back to that example? All right, so here we are going to try We have our three cardinality of three sort of learn constraints and So this is the notation x5 has been assigned false a decision zero Here it doesn't look right because it should also no no not x not x5 So this is here then x1 has been assigned by propagates. So if you remove that one What does it mean? So we have Three plus two plus two. This is only seven So you need absolutely to propagate x1 because x5 has been removed Okay, so here you have if you remove x1 by default you see that you have three plus two Plus two plus one which is eight So by default when you read that constraints x1 is not necessary But as soon as x5 is falsified you need to propagate x1. This is what you have here Then so a decision level zero the first decision made is x1 And then you propagate is x5 not x5 and then you propagate x1 The second decision is not x4 Not x4 and you are going to propagate x1 x3 here. Thanks to this which is written here And because you are propagating x1 x3 x1 and x3 this is falsified So We are going to so x4 is falsified. So now we have to propagate these two and then you have falsified this one And these two and this is falsified now so if you look at what is What is the value here you have five plus three eight plus two Okay, so you are satisfying with an addition of two your Your threshold okay, and now if you look at here you have only three For the second one. So you are missing two to reach your threshold So now what are you so what you need to combine? C2 and C1, okay on the The last one which is x2. Okay, we have the same The same weight. So that's that's fine so what we are going to To do we are going to Weekend on x1, okay, so now we are going to have this This formula here 3x2 plus 2x3 and we reduce here The threshold and you see that just weakening doesn't change at all the what we get we still have By which we satisfy with an addition of two This formula here, and so plus 2 and minus 2 I forget to tell is equal 0 So your equal 0 is not falsified. This is our problem. Okay So because if I combine those two I will get plus 2 minus 2 equals 0 And this we need something that is strictly less than 0 to to be falsified so we are going to reduce now by x3 and We will get now that formula. So because it's a 3x2 plus x4 plus x5 And now we have that 2 here that is removed which is 1 and because we have the saturation rule It becomes x2 plus x4 plus x5. Okay, and now we go from Because x2 is falsified x4 and x5 Satisfied Now we okay, no x5 is falsified. So only x4 is satisfied. So we have we are propagating so now we move from a post Of 2 to 0 Okay, so it means that if I do 0 plus minus 2 now I have something that is negative So now I will get get something that is falsified if I combine this one and this one so But I have to adapt the the coefficient. So I have C2 and The first one that has been reduced to to that one. So I take C2 and three times That one. So C2 is 5x1 plus So plus 3x2 so I'm going to remove 3 from the combination of 3 3 times 1 so it's 3 plus 5. It's a 8 minus 3 Here I have x4. So it's 3 times x4 plus 2 times x4 x4 bar, so I'm getting rid of 2x bar. So Okay, I need to to write it but And then x5 we have 3 x5 So at the end what we get is we have different coefficients But because our threshold is 2 We are going to change the The coefficients of those ones. Okay, so we get that That's a double-edged constraints and if you you look at it We we can go back to the first decision level and In which case we are going to Propagate Because the first decision level we have x5, which is false So we if we go back And this has been also propagated at decision level zero So we are going to we we stay with with those elements and we have to propagate Necessarily x3 so typically this is the kind of So double-edged constraints you could learn if you do so if instead of using this you apply only You just take the falsified literals you would You would get typically that That close x1 bar plus x4 bar plus x 5 and This is also something that is assertive a decision zero Right, and in that case you are going to propagate x4 Because this is falsified and this is falsified so you when you look at them you are It's not really comparable. Okay, you you learn different Different constraints, but you see that here you you have that process where you have to compute typically The slack here for c1 and c2 and if you you sum up you are not negative and this happens a lot of time You this is heuristic. Why did I choose x1 and x3? I could have used Any unassigned or satisfied literal Okay, I cannot change falsified literals. I can change I can weaken by falsified or unassigned literals So there is a old combinatorial We use heuristics. So this is this is really a problem because there are many ways you can weaken something and weakening doesn't change Anything it's the fact that at some point you will be able to decrease the threshold and you will be able to Apply saturation. This is where you are going to reduce the slack So typically so it's not directly the rule you are using that is going to allow you to become falsified It's the consequence by decreasing the threshold that will allow you to do that. So it's a So this is really something that is complicated So many people try to find ways So there are there are cases in way in which you do not have to do anything So if you know that the reason is the Kani Laity constraints, you do not have to do it So there are cases for which it's simple But if you have two generic absurdity constraints one conflicting and one reason of the other then you have to do some tricks to maintain that in variant Okay, so here I try to put so I'm talking about solvers with Specific handling of Kani Laity constraints or settlement constraints inside the solvers. I'm not talking about people doing encodings and Calling a set solver, okay so typically In the 90s there was some some work on this and typically The first one. So do this one was the DPLL version This was a local search based on settlement constraints and then you had the version of grasp with be solo with Kani Laity constraints and the optimization and so on and then you see all the papers has been here typically papers because of the Shaft coming in 2001 and people saying okay We're doing to do the same thing extending to solvers and constraints with Here without changing the proof system or with changing the proof system. Okay, and Then typically nothing not much so that so for WBO it's a bit specific because WBO it's a generalization of maxat on PBO so And then we have recently some a new solver condom rounding set that appears but typically most of the work has been done Five years after shaft. Okay, typically a huge interest and Typically what happened is that the main interest is typically on maxat since a decade ago almost But typically we will see that in the real The main many of the problems that people you Expressing in maxat are actually so the burn optimization problems But it's just that maxat maxat solvers are more sexy now So they prefer to use maxat but there in reality the real problem They try to solve is typically an optimization function with Closes and this is not a maxat problem. Okay, if you have just you need soft clothes It means that you are solving I'm not saying the wrong problem. You can use both approach to do it, but The real problem is is that one And typically most of the work Since the mid 2000 has been on improving on coatings and this is the reason why it's not in here Okay, so there have been huge improvements in on coatings and we will see that I have a few results from the set computer so from the sediment and evaluation and you will see that It works fine and typically just to give you an idea the first evaluation pb evaluation in 2009 in 2005 So Niklas in and Niklas Sorenson submitted a solver based on the translation of CNF So there are three different on coatings depending of the of the pb constraints. It's cardinati and so on and They were by far the best one to solve decision problems and it was sort of they just submitted just you know to To see how it goes and it was very very efficient and typically This is and one of the reason is if you have a very efficient set solvers you can just pick the best one that at that time it was mini-sat and If you have a good on coding you can pick the most recent at solvers and you benefit from everything And this is very difficult to beat if you have to Re-implement and re-adapt for your own constraints all those things And this is also what happens for constraint programming the reason why the best solvers the last two years at the Xcsp competition are sad based is because they just reused latest set solvers new on coatings and It works well Okay, so So what about what about set 4j in that case? So set 4j typically we weren't interested in the pb shaft on galena. So So that were those were two solvers that were designed right after shaft But typically the source cause of galena was not available and the pb Shaft was not maintained and it was really the idea to change the proof system. So those solvers are generating settlement constraints instead of clauses and for efficiency reason typically they Just keep cadenetic constraints inside the solvers and typically they They were not able to work on many Problems because they didn't manage overflows So it was working on some problems, but not on some problems who are interested in and they were and the authors Decided to do other things. So we decided to have our own pb solver. So that 4j was based on The mini set specification. So end of 2003 there was a mini set specification a paper describing or was working the solver and that has been Really so there has been one revolution that has been shaft all the principles were in the solvers But most people didn't know really how to implement them and then the second revolution has been mini set because it was 600 lines of c++ code plus 300 for the libraries and Everybody could understand it master student could understand it and this is how I implement it set 4j And if until I during two years between 2001 and 2003 I was I had in mind that CDCL was just binary search On a tree and learning. This is not at all the case. Okay, if you believe it You are wrong the way it works. It's really driven by conflict Analysis and this is a completely different way of solving problem And it means that when you do for instance linear search When you do optimization here, you are not do you at all using branch and bound and so on all the things that you have learned at school I mean, this is not the way the solvers are are working And so in set 4j you have two different ways of working either You just use the natural the classical conflict analysis and then you have a Native representation of the constraints and when you ask if you falsify something it will give you a set of literals just like yesterday for cardinalities, you know, you you The cardinality or the PB constraint will give a set of literals to the solver on the solver will sort of just see Literals as if it was a CNF encoding. It just compactly represented in the solver and there is the cutting planes Engine that will produce new PB constraints So what can go wrong if you? implement those things Debollion propagation. So now you do no longer have those lazy data structure on LPB constraints So each time you are learning new new constraints your server is getting slower. This used to be the case in sat before using what's literals and So we go back in the state where when you are learning you are getting slower and slower The problem is that there is no easy way to detect That you are assertive you have to to check it's not syntactical You have to be to pay attention to that then you have that story that you need to find out to weaken and There are many ways to do it to be able to have a conflicting constraints once you apply your clashing addition and Then you have the problem that after a while so sometimes sad4j ends with with numbers with thousands of digits It's but yes, so this is just But but this is what happens if you do not any any Rule to reduce the coefficient if you just apply the rules purely as they as described After a while you have really a huge coefficients and then the solver spends a lot of time Doing manipulation of arbitrary precision at mathematics and it's so you are not doing such at all You are just you managing those very huge numbers so typically What people have done is typically to reduce So it shouldn't be closed. It's a constraint to cut it.  You you keep the PB constraint for a while and after that you remove it So typically you learn a close and a PB constraints You keep the PB constraints for a while and then you forget it It doesn't change your server is still complete Okay, the close will allow you to to finish but so this and So else you just learn that was the very first solvers so PBS which is typically like Sad4j you where you have an internal representation, but you learn closes or mini sad plus Okay, so now so that was for the generic principle So we I've heard someone of the speaker Speaking about binary search So I don't know if it's you so I wanted to give you something because when you play with those solvers Textbooks are no longer the same right And in our case we are not going to use binary search Because it doesn't work well so typically This is how you are going to solve so with so-called linear search and optimization problems so because We agree that we are only saying yes no and if yes we are going to provide a model right So typically how do you solve an optimization problem now if you have an engine able to solve a decision problem? Typically you you you are going to have your set of constraints and you will ask to your set solver Is it is the set of sol constraint satisfiable and you will get an answer? There are two possibilities if the problem is unsat you are done No answer is finished and now you are going to retrieve the model that correspond to to the answer because you know that It's satisfiable and what you are going to add a new to the set of constraints a new Constraints that is so the value of the objective function should be better than the current value and This is where I tell you that Where was that? That we have this K that is decreasing because we are going to have the objective function here And we have We are going to produce a K that is going to be smaller and smaller and smaller and and you are going to do to do that just Until your your problem is unsat and this is where we have our Researcher and seeds problem, okay, because there will be a case where you are sat and Right after that you are inside and this is how you detect that it works What is the difference between branch and bound do I have an initial upper bound? No Okay Here I can prove of teamality if I am lucky in two calls to the set solver. I can have a call Give me a solution it gives me a solution and Then I can I get a better one? No, and that's it and you tell well, yes But why would it happen? So typically set 4j when you ask for a solution It knows what is the objective function and it is going to give you some values for the phases that will Minimize directly the objective function. So in some real problems Sometimes the users were seeing how is it possible that it finds directly the solution? Say yes, because sometimes the problems are just simple You have all the weights of the objective function. You just give that to the solvers and it will Try and it will work something very close and you cannot do better So there is no upper bound. There is no lower bound. It's it's you are driven by the solver. Yes So it's the solver branching based on the objective function you have So in the case of set 4j because I Can do things with set 4j so by default when I read the the problem here The is satisfiable is not just the set of constraints because I provide also the objective function so that he can It will branch it will give the phase Depending of the value of the variable in the objective function so as to minimize So typically if the coefficient is positive and you have to minimize I would give you the negative phase if the coefficient is negative I will give you the positive phase and just doing that It allows you to find For some problems Solution that is very close to the optimal one But do you also use the magnitude of constraint like if something is large I would no I Do not do that. I mean maybe there are possibilities to To do better than what I what I do, but for the moment we we just do that. Yes You said the binary search is not good Yeah, well, I will go to that point after because you know a sat and a non-sat call well a Sat on a non-sat answer is not the same for a set solver and So, but I have a slice on this. Okay, so let me show you the thing Yeah, but in response to what could be passed you're saying that you're already doing some kind of optimization No, so, okay, I have a slice on it Let me show you because most people do not understand that when you play with solvers what is important is you need to Get the as many as much information as possible directly from the solvers is a successful application of sat base techniques works if you are in deep interaction with the solver if you are driven by the solver and And so so this is just an example to show that the same example that we show before So we have this objective function. We have those constraints. I ask my solver for a model of this one So this is sort of a dumb thing because Sat4j will give you directly the optimization The optimal thing but for the animation. It's not so I pick that model I evaluate for the objective function So it gives me five. So now it's I'm adding that constraints to To my formula. So I'm going to satisfy this now So so now I have those The original constraints plus the objective function constraints. I have a new model Which evaluates to three which is better than the five. Okay, so that's good I asked for another model and it will find another one. She's one and Now I'm going to ask is there a possibility to do better than than one. Okay, and It's not possible. So one is so the assignment I got and the Optimal value is one. Okay. So here I am I've been in the worst case because I had to ask the solver many times to get the optimal solution but It it works and so the this is my optimal solution and the value objective function is one So this is a linear search. So why now do we have a problem? Okay, so we do not need upper bounds You can take Face selection strategy that takes into account the objective functions. You can use any Settable and solver so this is a nice and you have that set set set set and set pattern And the thing is set answers are typically easier than and set once and typically There is even more if you are sat You do not have and you add that constraints. You can keep all the constraints you have learned As soon as you have unsat you need to get rid of everything So there is really a reason why in that case a sad call is not Equivalent to an unsat call. So if you do binary search, you will have sat unsat calls and That will be very different and so in practice if you do this and you use linear search or binary search In practice linear search will be more efficient because you do you keep all the clauses and you do not have to Have this problem So it's really about the the fact that you do not have the same properties When you the answer isn't that or when the answer is that Depends on the starting point right the first solution Yes So if you have a good starting point, so that's that the reason why it's important To be able to give that in some ways that information to the solver Yeah, so that the reason why I tell you there is always a limit of white box or black box. I mean So if something works in approximate and so on it's also because there was a deep connection with the underlying solvers Because if you want to be efficient sometimes you need to get the information directly so yes, so this is This is a way to Be more efficient to get the first model That's clear and typically the the the issue we have and This is what I told you when we started the lecture this morning is Typically you have the pigeon hole or the researcher seat problem Exactly for the last call that isn't that and if you are doing resolution You know that we are in trouble because you you will have a hard time to prove optimality if you stay in the case where You translate the continuity constraint without adding variables or using just resolution So then if you use sophisticated encoding which add variables. It's another story because it might just Be similar to extended resolution. So But so the theory here is is important. So The thing is ideally what I would do is I would like to use resolution my resolution base solver for the sad calls and I would like to just call the CP the cutting-phase base solver for the last call, right? That would be nice because I would get The efficiency of the solver when you do not require that proof system because the proof system is only important if you you if you want to prove unsatisfiability and The problem is you do not know when Is this the last call you don't know so how can we do this and So Olivia Ocel told me well run them in parallel say what is this is stupid, right? Say well, okay And so so since 2010 typically I have a solver called both In sad4j with which typically run those two solvers in parallel So what does it mean? It means that when you have this cold tooth is satisfiable here and here instead of feeding one solver and Feeding two solvers and I stop I take the solution given by the first one the first time sorry, okay So this is what the cutting planes? So this is for a particular problem in the PB 2010 evaluation here is the result of the cutting plane solver is in the output of the Resolution solver so you see here after one second it probably it finds a model with objective value of twenty six twenty three twenty two twenty one twenty nineteen eighteen and He cannot find Optimality can I prove to melody you see that here we have the cutting plane base solvers it's not very nice and But after 25 seconds it reached 15 which is better than 18, but he cannot prove optimality, okay So what happens if we have here in parallel? So here you have the cutting planes here you see that here I give you the The value and wish solvers give the solution Okay, so you see that the first solution is given by the cutting place solver then resolution solver then cutting plane solvers And the optimal solution is found after 300 seconds Okay, so this is the case in which the There is an interaction, so I just take the solution from the first solvers coming and it works and I have another case so typically We have here cutting planes stopping with after learning 226 assignments per second. Okay, this is a deadly slow because it should be around 100,000 normally, but Typically, it's very slow with the cutting place if you look at res parallel CP Typically the it will be 4000 which is not good, but is better than what we have Typically here, so the reason why the solver Finds something it's because you it's it doesn't learn a lot Of constraints that slowns slows down the solver. Okay, so this is really the the way it works so here you have a comparison with the parallel version here and cutting planes here You see that the parallel version works much better and here you have the resolution one Compared to the parallel one. So here you pay the price Okay, because the two solvers are running in parallel It takes two twice as much CPU time. So this is the reason why you have this Okay, but it allows you to to solve a few more problems so typically if you For for the competition typically and this is actually what people use in practice So in companies when they they want to use at 4j and they want to really prove the solver something They use the parallel version. Okay So typically it works better than any of the of the two So it's twice as slow as the resolution one So if you do not know by default you use the resolution based solver And then if you really want to prove optimality then you can use the both But what is interesting is there is that interaction between the two solvers because at any point You you pick the solution given by the first solvers and so it completely changed the search Okay, because when you have a resolution or CP, they they are not having the the same Intermediate constraints and bounds than the rest parallel CP, but it's also parallel So you when you relaunch it several times you have concurrent issues. So You have also some issues with reproductability So a few words about the PB evaluation. So the PB evaluation has been organized most for all but one case by Olivier Russel and Vasco Mancini, Olivier Russel is from my lab Vasco Mancini is from in Esk in Portugal and there they they run it from 2005 to 2012 in 2015 it was organized by Norbert Monte and they organized the last one in 2016 so there is a uniform input format the OPB files and typically the idea is to regularly like the site competition evaluate what are the What is the state of the PB solvers and you have detailed results on the website and we will see that There are various technologies, but mainly There have been a lot of people working on encodings and we there is a weighted boolean optimization Problem, which is typically max-hat for PB which exists since 2010 So I gave you the two Partial results of the competition to show you so there are there are several categories decision problems optimization with a small Small integer means you do not need arbitrary precision arithmetic to solve them and Optimization with big integer so you need arbitrary precision arithmetic to solve them So here we have that minisat plus so remember that this is typically and this is from 2005 Minisat plus didn't change anything since 2005. So this is really 2005 solvers. This is Cplex Okay, so this is ILP. This is class which is an ASP solver, but also a good Sat solver here we we have the parallel version of Sat4j Resolution and CP here we have resolution and here we have B solo which has been one of the the good optimizer since 97 and We have the solution either unsat or sat or either unsat and optimal Okay, this is proofed optimality So big integer are not very interesting because there are few solvers competing So and you see that for minisat plus is a good at proving unsatisfiable problem Well less for for optimal optimization Cplex is very good at solving optimization problem. So here There are two things Many of the problems comes from Meeplib. Meeplib is the repository of benchmarks for Cplex and all ILP solvers So there is a bit of a bias here But the point is you need to have those cutting-paints rules to be able to solve these optimization problems Okay, so if you do not have them it doesn't work. So typically they can do relaxation to LP with real Numbers if you do not have any solution. So this is what they they do typically here They will be able to answer unsat directly. So they can use a lot of things on that Minisat plus is Pluses for what what is the extra thing there? so so the minisat plus is three different on coding for cardinality and Shadowblown constraints. So typically it has three different encodings and looking at the constraints it will Translate it into cnf and then it it launched minisat So it has some heuristic to decide then yes So depending of the value of the coefficient depending of the value of the threshold It will decide which of the encoding it uses and typically this has been we will see that there is a typically an improvement on that for the after but typically This and this was designed in 2005 Okay, there have been no change Maybe just that it's instead of minisat 1.13. It's maybe minisat 2 now, but that's still the technology from 2005 and this is very this is efficient, okay And you see that in some cases we can decide unsatisfiability here of decision-prone, but those are all those theoretical Problems like Pijon also on all those counting problems. So they are not really interesting in practice and For sat it's Well, if you have a good Boolean enumerator, it's good You see that typically Cplex is not very that good on those decision problems. Okay, but Where it really shines? It's an optimization problem because it can use everything If you take so this is the last PB 16 evaluation So look at the numbers still the old minisat plus Perform very well So you have much more benchmarks. They typically took all the benchmarks available that year and You'll see here Set 4j only the pilot parallel versions here. You have CDC LCP, which is a former version of running sat So that I show you so you see that it's typically uses the same thing as in set 4j in that version So the CP is for stands for cutting planes, but this is not the real cutting plane This is general resolution C++ and you see that this is the the best thing that can work and for the minisat plus for the satisfiable problems And we it performs very well on optimization problems on and sat problems for When you have small integers And it didn't support arbitrary Precision aromatic, but it would be as good if it would And then you have naps naps is typically a nagoya pseudo Boolean solvers This is typically also on coding and then using a set solvers behind so this is just so those two solvers Do not know anything about PB They just see closes Well, yes, they do the translation from pseudo Boolean constraint to CNF But typically the solver itself doesn't have any inch about what is a PB so a PB constraint And you see that the results are pretty good Okay, let's see how it goes. So I would like just to show you Something about Carnegie detection and then the limit of why we have problems with pseudo Boolean solvers so I I Try to convince you that there are problems that are inherent to the implementation and there are also problems in the theory Okay Okay, Carnegie detection. So we have that so this is a paper we published five years ago We when I show you in the very first slide that you can retrieve Carnegie constraint from CNF. Okay, this is the What I called theoretical computer science cutting planes proof system where you do that systematic way of arranging Closes and you can retrieve those things but typically None of the PB solvers using some implementation of cutting planes can work on CNF Okay, so the idea is Can we find something to help PB solvers to work on CNF? And we want something that is general. We don't want to Use some kind of pattern matching on the on the Santa key take all form of the CNF We want something that may work In the general case. So what what can we do? So what one thing is if we can do that? we can reduce really the number of The size of the problem. So this is a case of cryptographic benchmarks by default you have 500,000 Closes, okay, if you translate it into a PB here the the size of the constraints And this is the value of the threshold which which means the less or equal to 1 less or equal to 2 less or equal to 3 less Or equal to 4 And this is the number of literals you have and typically you go from 5,000 500,000 closes to a bit more than 120,000 closes Size of the constraints is 3 means what there are three constraints So here Trifold is K and size is the number of literals on the Right hand side. So for instance if you have here it means you do you have it's x1 plus x2 plus x3 less or equal to 1 X1 plus x2 plus x3 plus x4 less or equal to 2 So it's a number of literals on the left hand side. So typically it means that those are very small Cardinality constraints But still it allows to reduce the number of constraints by a factor of 4 If you take So typically We will do that also this afternoon in the end of session if you take a pigeon also. This is with 100 pigeons so This is way too large for we could take 20 or 16 or 19 It will be already a problem, but so it doesn't work if you use General resolution on this it doesn't work, but now it's less on one second if you use the general resolution So this is really the problem if you do not have the right representation You cannot use the proof system. We have just shown so how to do this. So the thing is we We have that fact that we can replace really Huge amount of clothes by a single constraint. So that's good, but the problem is And we can use the proof system, but the problem is in practice. So it's not easy to Detect all the encodings. So one of the reasons was explained by Matti yesterday Some of the encodings are consistent or there are not if you are not a consistent There is typically no way you are going to retrieve the the thing But we are going to try to do it and So here are a few encodings so typically for from The most well-known, but they are regularly new ones and so there are many What we want is to have If we need if we do it in tactically we would need to recognize each encoding And this is something we do not want, right? So if we want to do some Semantic detection we want to use unit propagation because this is something that works well in the solver and Typically it should work on any encoding that preserve a Consistency and typically it might be the case that people wrote constraints in the for their problems and that they are hidden cardinated constraint inside things that they Just can it see that can be inferred from the the other ones. So that would be the the thing So how are we going to proceed in some ways? It will be similar to what we have done by and it means that we are going to take Close So we are trying to retrieve at most k constraint, right? And so we are we are going to translate our clothes in that Format which is some of the negative value of x1 Less or equal to n minus 1 So typically binary clothes will be less or equal to 1 Okay, and then typically what we are going to do we are going to try to extend to add literals here and Without changing at all The the threshold so the point is we will start with binary clauses because here the threshold will be one Once we are done with binary clauses will do ternary clauses and so on So we can that way it means we can try to detect at most one at most two and most and that is just Way to do it So how can we do that? typically What we we have this formula and what we want to find is to to find that x1 plus x2 plus x3 is less or equal to 1 okay So here you have we have x1 and x2 Mutually exclusive there should be By here x1 x3 that are mutually exclusive So you see you need to resolve the two to get the information it by just in tactical information I won't get it and I have x3 x2 Which is available here also if you you need if you resolve that one and that one, okay? So so it means there is really an encoding which is a bit hidden but of x1 x2 x3 That should be less or equal to one. So what do I do? I take the first? binary clause and now I'm going to propagate x1 and I'm going to Propagate x2 so did so this Constraint it's exactly that one. Okay, because here it's negated then I find Negating twice I get this So now I'm propagating by so this is unit propagation On x1, so I'm going to propagate if I propagate x1. I'm getting not x2 by not x2 I have nothing else then I have not x4 Not x4 I have not x3 Okay So this is what I get if I propagate x1 now I'm going to propagate x2 if I propagate x2. I'm going to propagate not x1 I will get not x5 and so I will get not x3 So now I'm going to take the intersection of the two and the intersection will be not x3 So what does it mean? It means that I can add here x3 the negation To this continuity constraint and this is how the the process works. So I started with this I have the intersection I could add all the elements that in the intersection to this constraint So this is how it works. So you you choose what is your typically your constraints and you you start by having your element with all the elements that are in In your continuity constraint and then you have for each subset of K and this is where it's painful. So There it is combinatoric right and then you you have to check and to do the intersection and if you take all the Intersection of all those elements and you can add them to the constraints So it's easy for one for two and when K grows it's just a nightmare Okay, okay, how does it work? so typically Lingeling has Dedicated syntactical recognition of continuity constraints because it allows it to solve more benchmarks at the set competition So it was used for that So there is another syntactical one that was present in Norbert's Not the risk solver and so in set in set 4j we We use the cutting plane session with and without preposition and there is also SB sat which used by compilation some Detection of K and T constraints So here so we are using generates resolution so These are a different representation of The constraints, okay, so we have different Encodings so pairwise is the what we we have seen this morning and then there are different ones Okay So we have Lingeling which can find the classical one very easily So here we have the syntactical Recognition of risk and then we use set 4j here We have the semantical representation that is Implemented very efficiently and risk and that we feed the set 4j here We have a SB sat or we have a set 4j without preprocessing Okay, you see that set 4j without preprocessing cannot do anything you see that so in bold You have the good results so it means that yes, you can retrieve the pairwise so for some reason here it It takes time so I don't know why but it's a bit strange, but okay. It works Typically, it works fine with set 4j cp. We will see this afternoon And you you see that we can retrieve for many of them the the encoding so this was sort of so Lingeling works very well for the pairwise encoding and SB sat has good results for the letter encoding and Set 4j doesn't work without preprocessing and then we are we are good on some of the encodings so but these are very So those were pigeonhole principles with those encodings those are very Crafted benchmarks, okay What we tried is to have so those are other crafted benchmarks from the theoretical computer science community But we we were interested to see if we can solve all of them So they are all based on Counting principle at least one at least two at least three At most one at most two at most three So you have and you see that in all those cases we could retrieve them okay, so those are typically the But because they are all based on this pairwise encoding Now what is more interesting? We wanted to check how good is the detection of Kaniyati constraints, so typically we took some pseudo code grids 9 by 9 and 16 by 16 and We could retrieve So with the same tactic we couldn't get all the kaniyati constraints But we could go all of them with the semantic Detection and there was a Challenge benchmarks by Spence and van Gelder, so this is typically a problem on which class which is very efficient Try to run during 24 hours and couldn't solve it and this is solved within a second if you know how to count So typically it shows you that there are many benchmarks from the set competition that are typically counting program just there to because it's known that the set solver cannot count right and So this is quite easy if you know how to do it and it this belongs to at most three constraints so neither I mean nor Norbert with risk can do it because it's a At most one or at most two for risk, but with our generic thing it works Okay, and so now yes So it it it would work exactly the same with pseudo-brown constraints, so if you can do the propagation. Yes. Yes You even we absorb with whatever Because you only rely on the fact that when you propagate something you just take the intersection of the propagation So you could have whatever constraints you want Then I'm not sure if it really matters to do this if you already have Cadenity constraints because typically the propagation is much more It's it's much slower on the PB and the cadenity constraints, so You need something so here you do it because it just closes and this is quick But so yeah in practice you could use any kind of constraint Okay, so now there is a reason why We have difficulties because typically So I would like so but they didn't want to show you our work Conflict analysis in the solver so I have a small one Let me see so this is this is what happens if we Do conflict analysis in in set 4j typically so We have This is falsified. This is propagated. So because this one is falsified you have to propagate this if you propagate this this is falsified and This is falsified also So here you will need to propagate X to false and Because X is falsified now you are going to falsify the X3 So what we have so so capacity is a Is the falsified so now you we need to do Clashing addition between capacity and kapatu and we're going to just trust me on that case, but We should be able To do it because it's so we have to multiply this one by three. Okay, so we have 12 plus 1 13 6 plus 1 7 6 plus nothing is 6. We have D here. We have E here and we have Here we have 15 plus 19 minus 3 so it's 16 So we have this formula and We We learn these constraints and if we remove If we falsify a what we have here is only 15 So you need to propagate a a decision level 0 Okay, so this is another case where you just Erase all the decisions you come back to decision 0 and you are going to propagate and this is you cannot see that in a sad solver This is not possible. You cannot you in what happens in a sad solver You you replace one decision by your propagation, but you you do not back jump up to the last decision level Okay, so this is this constraints, but if you look at it We have those two guys D and E They are not related so I could get rid of them because if I want to have 16 it can be a and b can be a and c But there is no other way D and E do not play any role in the fact that you satisfy them so Actually, so we call them irrelevant. They're irrelevant. This is what we have seen. So what we should have instead of this We should have this What does it mean? It means that what we learn when we use those Rules that I show you earlier. We are producing things that are not as tight As what you learn with with the closes typically so we are we so we are using we are learning things So D and E were not useless in the original constraints, but when we combine them They become useless So this is a this is a first problem okay, suppose and Yes, we could even change the year the degree. So here it's 16 I could replace the degree by 14 that would be the same constraint. Yes How do you define? Define irrelevant and how do you test for it? I understand kind of so this is an NPR problem To detect if a literal is irrelevant or not. It's NPR So the definition if if restrict if you satisfy or Falsified D or if you forget D you have exactly the unequivalent formula in term of forgetting if you know what forgetting is so typically if you if you satisfy or falsify that value you have an equivalent formula and You wonder does it happen in practice? Okay, so it doesn't happen. They do not appear in the original problems. Okay, but inside for J This is a we took many benchmarks from the 2016 Evaluation and we we check the number of irrelevant literals in the first 5000 constraints so we dump the five the first five thousand constraints we learned in sad for J and we tried to look how many irrelevant literals we have and We you can see that we have so here you have the number of instances and here you have the number of irrelevel Literals in the instances and so in some instances you can have up to 10,000 literals irrelevant literals on the 5000 constraints so it's You have a few and then you could think well, maybe that brand new Rondings at solver for me a cup not from and you know first could solve the problem. Okay, so in that case what they do is that they when they do the They use the division rules that one here Instead of saturation and so We can take another example and here in that case We falsify capacity and what they will do is they will take kappa 2 and because we We are going to weaken on e because this is where we have to To reason so he removes e and then you So it's on We have to do it on a right, so we're going to divide Here because so we weaken and he he disappeared. We have this formula now We are going to resolve on a so we divide all the the constraint by the coefficient of a So we we get a close. Okay, and if that case so typically C and D here Where irrelevant? Because here you have coefficients 3 coefficients 3 and so if you want to satisfy that constraints You should either use a or b because you remember that if you take because they have the same coefficients. They are Equal sort of equivalent. Okay, and C and D have the same coefficient one. They are also equivalent But in that case they are just irrelevant and Because now it divided by three That value they all have the same Coefficient so now C and D become relevant Okay, so what does it mean it means that They could have learned a plus big written on one if at this stage they knew that C plus D were irrelevant But they didn't know so they get a much weaker constraint And so they are also some cases much less than inside 4j But there are cases in which they have irrelevant literals so this is really the So and what is this a big issue in practice? So this is the an example that we have so typically The issue is you we could Because those are irrelevant We could get rid of those elements and we could reduce if with because we can choose by which value we Because their value doesn't Change the the satisfiability of the constraint so we can change to Satisfy them and then to reduce the degree then it means that we will we can have it here 15 So we can use saturation to remove the 17 and to have 15 here And now you see that we can divide it by five and this is preserving because we didn't use ceiling Okay, so this is equivalence Preserving and so we can instead of this we could have this and This is we have that issue with the coefficient growing if we are able to detect this We are happy, right? so typically this is one of the the reason and we have also the The fact that we we may hide cad caddinity constraints. So here You E and F are irrelevant if you know that they are irrelevant You you can divide it by you can divide the resting by three So here you will get two and that will be also much better Okay, so Well, we do not have any solution for the moment But the sink is we know that even all the rules where we know where we are aware of There is that issue with irrelevant it rolls that doesn't happen neither for closes nor for Caddinity constraints, but it only appears because of the fact that not all literals are equivalent in PB constraints And so this is something We are working on it and and so it means until we solve this There is we will have problems to Incorporate a Good proof system in our solvers because we are we are producing constraints that are two weeks for the same Okay, so So typically yeah, yeah, we can we could have used instead of PB PB representation a caddinity representation, which is more efficient, but then we do not have it So I would like to start the second lecture