Here is 1997 The computer of IBM deep blue beats Gary Kasparov on the tournament Conditions in I think a six or seven game series This is the first time ever that a computer has won against the premier human player And Gary Kasparov is considered to be one of the best players of all time in chess at around the same time And computer girl we had the tournament from 1985 to 2000 Which was called the world cup with a total price of a million dollars If someone would be able to beat 10 to 14 year old Chinese children that were go protégés in an even game So it was not for the lack of trying But in the same year 1997 There was this tournament at the winner of the tournament place against these Chinese so-called insays The program one but by a huge handicap of for those of you that know go I think 11 handicapped stones, which is pretty much the difference of Very amateur player and maybe an intermediate player so to say So let's dig a bit more into the current history in the more recent history of computer go and Also in why is go so hard to master as compared to chess. Why are we so much behind? This doesn't work. Okay. Yes 2001 an anime by the name of the current ago is launched in Japan revitalizing Go and making it more popular among the youth. It has been going downhill in Japan for some time Before that but overall it's very popular in eastern Asia So there are professional players that get paid to play you can watch it on TV and everything Here's 2009 Google invents a language called to go saying, oh, there's a game. That's like what 4,300 years old Oh, we're just gonna name steamroll right over that all board game And of course, there's also another programming language by the name of go and they're also just stream right over that because what the hell We Google we can do whatever we want Year 2014 on the right you say Remy Coulomb Here's the programmer of Program called crazy stone and it's one of the two strongest programs that we have in go and he plays against a professional That's called Yoda. He's a ninth down professional player and crazy stone one with a handicap of four stones Which is a lot less than the other handicap, but it's still a real big difference in playing strength Um 31st of October 2015 in the code centric go challenge Between the program then which is one of the two most strongest programs played in an even game against one of the strongest amateur of the world Francis of dickhood a German five-dan player The German the human prevails in a five game in a best of five series feature one This is the last move played in that series third November 2015 Facebook publishes a blog post that's like oh we're doing all this cool machine learning stuff And by the way, we wrote a go engine. It's awesome And we don't really tell you what we're doing exactly but pattern recognition and their CTO did not answer my Facebook message About what they're really doing quite to my surprise But we're gonna get a bit into that and how far how strong that engine really is for all that we know 13th November 2015 This site hits second news. It says play go against a deep neural network. We will also have a look at what's up with that What's the science behind? Just we're gonna look at a lot of stuff that is in scientific papers, but don't be afraid It's all cool. If you don't know go all cool.  So This talk is beating go things to power of randomness. Hi, I'm Toby You can find me on Twitter github and everywhere else as practice I'm out of Berlin all the way here. I organized a really good Berlin in Berlin the react user group in Berlin I'm a rail to go to coach. I do open source. I do shoes when I was shoes like that's we amazing stuff Yeah, and I work at a great company called big crowd which is an agency So we help startups build things. It's lots of fun to work there right now We're actually hiring so if you want to relocate to Berlin one of the best cities Just saying and they paid for me to go here. So pretty great so Some of you might be concerned about my t-shirt because it's like oh god It's Yoda like would Yoda allow him to wear the t-shirt of Yoda is that physically okay for him? I can tell you, you know, this is saying that Superman wears like a Chuck Norris t-shirt. So you're the obviously where's a Toby t-shirt. So We're bodies, you know, we made that out. So everything cool there So continuing with the basics of go and go as played on the board with lines in the sections And the normal sizes of 19 by 19 board Then we also have 13 by 13 and 9 by 9 normal professional games are played on the 19 by 19 board But the 9 by 9 board is also still relevant interesting. So how does go go? We start playing move black starts and we play on an intersection. So that is now a black move then white moves Black white and whoa too far. Sorry That's the basic thing. So Taking further from that a stone has so-called liver cheese and those are the adjacent Cutting points that are empty. So this stone has four liver cheese if I play another stone They form a group together and they have six liver cheese When white plays next to the black stone both of these stones have free liberties because they take one liberty away from each other So if white continues to play now, I have two liberties and in the end free liberties and now we're in a state which is called Atari and Atari's company name was really chosen by that name of that go term and go to and that term means Before capture sort of So if white plays another move that removes the last liberty from the black stone and that means the black stone is captured And taken off the board and counts as a point for white, of course like isn't stupid Like no sort of play and can stretch out. So no black has free liberties again And it's all good. So I told you right now that there is a point Which white gains from the capture? So how do you win in the game of go the game of go is basically about getting more territory than your opponent You try to encircle territory and claim it for yourself and then Territory is then one point of these intersections and then the end it is counted who made more territory So it's basically a game of dividing. I tried to get more of the cake than my enemy does So in this example, we would have encircled one point so that we were that would be one point for black If we go further white could now encircle Black and now black is at one liberty. So we know what's probably gonna happen White can capture thereby making eight captures. So eight points and of course There's no also counts as white territory. So it's gonna be eight points of territory So 16 points made here and total which is pretty much a lot on a nine-by-nine board But is this is it an endless game of capture and recapture? No, because there's also Something in the state that a group is alive So we say and this is a typical group that it's alive. It has two eyes which means we have two points of territory here and White can't simultaneously occupy both of those liberties Because white if white would play in there, it would be immediately recaptured the so-called suicide move which is forbidden under Japanese rules, but not on a Chinese rules So this gay this group cannot be captured and in the end all of them are alive So what does a game look like in the end? this is the end position of a game played between a professional player and One of the strongest go engine. I think it was then but I'm not quite sure And so how do we count that? Well, we remove the so-called that stones that have no chance to live and then count the territory and This game was drawn and if you see now It doesn't look like a draw like black has way more points. Why was the game a draw? Well, there's a concept called comey and which is a set of bonus points that white gets Because black makes the first move And so this game was a draw So but what does a real game look like on the 19 by 19 board and this is the game This is the first game between France with a ticket and then this year and You usually start to occupy the corners first and we can't go through the full game So I'll fast forward and so we can sort of see how the game starts forming here like black claims sort of the lower right corner and the upper right corner and There's also something very like for me interesting going on That you cannot see the immediate value of in the lower left corner Black starts a bit like sort of a wall that faces upwards and that is called influence. It is a very difficult Even for really good players to see like who is ahead here and who is not and if you see the game commentary There's lots of discussion about like who is in a better position at this place So you can imagine how difficult it must be for computer to assess the current situation The game moves on we see more moves played and even more moves played and at this point I want to focus your attention to the lower right corner and This looks at the first look if you don't know go it looks like okay Black and white are battling it out out there Unfortunately, that is not at all the case that black group is Practically that it is still on the board, but it only has two liberties left right down there And so it can be captured at any moment a go engine that would have to play and understand this would have to Would have to understand that this is not actually and not played our position But those stones are actually lost if we fast forward to the end of the game We can see that eventually white did capture That like group and there are lots more Many much more complicated Positions on the board that I would not be able to assess because those programs and the player played a much higher level than I do Okay So now we count this game and this game was the only game that then won and that was by a margin of one and a half points So when we talk about go versus chess What's the difference and go you start building something you start with an empty board and you build the stone stone move They might get captured, but you can't move them. You don't flip them You build something you have to plan strategically ahead like where do I want to be where do I want to stand In my game and chess you go out and destroy basically your goal is to destroy The other king and so you get less and less of what you really have When I first looked into this which was in 2010 Also an interesting concept came up the con the concept of the difference between what is complex and what is complicated Go is very simple and it's hard There is a set of formalized rules that are just ten rules that basically define the game of going I've taught you almost all of them by now and still go is very complex because there are many many possibilities Where I can play which results into many things that could happen which makes go very very complex On the other hand chess is more complicated and complex chess has many rules Which piece can move where like then there's the rojada and then at the first move like the pawns can move to And so it's much more complicated of a game and some of its complexity Stens from it being so complicated so This is a code by Edward Lasker who is a chess who was a chess grandmaster and he discovered go a bit later in his life and He said while the baroque rules of chess could only have been created by humans The rules of go are so elegant organic and rigorously logical that if intelligent life forms exist elsewhere in the universe They almost certainly play go So When we want to talk about how strong programs are I'm gonna quickly introduce you to the ranking system used into go You consider the beginner when you're a 30 Q which is basically you learn the rules and you play the first game So that's 30 to 20 Q a casual player You can become after maybe a couple of months of play of a really hardcore Then maybe in the first week or so then you're between 19 Q and 10 Q The intermediate amateur is from 9 Q to 1 Q and it's very different what people say like if you look at a go wiki They say oh you can be shit in a month. I Don't think so If you look at computer go papers, they may they mostly say oh well It takes an amateur player years to reach that level But I think they just say that to make their programs look stronger when they read when they reach the Q level And then there's also the advanced amateur and the professional Now for quick show of hands who has already played go like who at least reach the beginner state Okay, at least reach casual player Intermediate amateur Okay, advanced amateur. Oh god. There are people here that were much better to go than me like I I didn't play I don't play actively that much anymore, but I was around in a Q So a weak intermediate amateur and took me about one and a half years to go there despite like playing tournaments Online and everything so you might talk to me. Well, I said wrong about some of the positions later on that would be nice Thank you Why am I showing you this it also gives you a hint of what the worth of those handicapped stones is because in the amateur level It is considered the difference of one handicapped stones between Between the ranks so when a 9 Q plays against a 5 Q the 9 Q gets a handicap of four stones Which then looks like this the stones are already placed. So Black essentially gets four three moves So this is the handicap at which Crazy stone one in 2014 This is a handicap of nine stones that is like the highest handicap that you give in official games at tournaments as far as I know and This is the handicap at which one of the strongest go programs lost in 1998 to the strongest To the Chinese and say they're 10 to 15 year old children This is a Handicap of 13 stones I played once at that handicap at the Blitz team tournament We were around 10 Q we played against the team of done players and this handicap is just unfair So we bet the down level players although making Very very bad mistakes and we were the only team to beat them to beat the team of done players They went on to win the tournament. So that shows you how unfair that handicap is This is a handicap of 29 stones, which is just I know if you just pick up go and play it on the 99 times 19 board against me You would most certainly win most certainly In 1998 a German 5-dan player bet the strongest go a program at that handicap And there was one year after Kasparov was defeated. So that's just meant to show you how far the programs were behind So why is go so hard? What makes it that much more complicated and difficult? than chess So at first we have a much much larger board our board is 19 by 19 as opposed to just 8 by 8 for chess Then almost every move is legal like as I said, I told you almost all the woods I left out the co rule but basically you have to move inside the field you can't move at a place where somebody else already is and Then you can't play co moves and not the so-called suicide moves and that's it So that leads to a very high average branching factor What is an average branching factor if you have it's basically the average number of moves that are legal at any point in the game So for go we have 250 and chess just 55 We also have my we also have a very high state space complexity Stay space state space complexity is the number of valid board positions that exist And I think some mathematicians Calculated those so for go. It's 2 to the power of 171 and chess 10 to the there's a to 10 to the power of 171 and for chess It's 10 to the power of 47 So those are huge numbers. What does that even mean? The number of atoms in the observable universe is estimated to be around 10 to the power of 80 Which sort of means that we could take each of these atoms and make a combination with also all of the other atoms and have That as a combination list which would turn out to be 10 to the power of 160 Possible combinations if my math doesn't fail me and that would still be less Than the number of valid go positions So at this point you might get like a sense of the scale how big that problem is And also moves have a very global impact you can't just go like okay I'm just gonna play locally here where my enemy played last that doesn't work due to multiple reasons One is oftentimes. It's the best. It is the best move not to answer the move of your opponent Just play somewhere else because the move somewhere else is much much bigger And the other thing is that a move that I played down here in the bottom right I can have severe influences on a position to the top left. It might break things there and actually defeat decide the state of a group So let's finally after we go in our entering to go and why it is so hard Let's talk about artificial intelligence for a second here This is an image from Wikipedia of the alpha beta pruning or alpha beta search Which is one of the which is the algorithm basically to play chess like deep blue use it and I think Modern go pro modern chess programs Still use it. It's at that first church in that graph where you go to leave and then You have to evaluate the state of the board at that point in time because you can't play all the way to the end because even for chess That would be way too much to do and then you Always you always assume perfect play Which means that's the min max part and the first move that's our move We try to maximize the value of the evaluation function to say like okay We want to have the best position for us, but that's the first step And afterwards it's the minimize because it's our opponent's move And of course our opponent is clever and would try to minimize the overall you of the evaluation function because your opponent wants to be a good move and then We maximize and so on So what's the problem here? As I tried to show you before it is very hard to write an evaluation function for the game of go if you remember This position for instance, it's not very clear where territory will be of room in the end And how much it will be and there are other concepts involved like more you and influence that is like okay Like has very much influence there, but why it has more territory that is something that you commonly hear and game commentary And it's not easy to decide what is worth more and especially not for computers So so far we have been unable to find a good evaluation function. There's one Go program called can you go that uses alphabet of search of a better pruning? But it will place around the level of 5q 5 4 q and current strong go programs play About a strange of a 5-dan or 6-dan amateur player So we came up with this great thing which is called the Monte Carlo method so To tell you what the Monte Carlo method is I want to ask you a question first What is pie like what is the value of pie? Three point one four something cool. So how do you know that? How do you how do you determine the value of pie? Come on people no one So so it's in some old circumference blah blah blah math things there There is another way to do this you can draw a I think it's called union square on a piece of paper and Do a circle in it and then you throw random pins at it and then you count how many pins land inside the circle How many land outside of the circle multiply that relation by four and then you get an estimation of pie Which is an example of the Monte Carlo method applied and you can see it here and Right at the top I think it's cut off But you can see with the number of simulations with the number of more random dots that we throw in there Our estimation of pie gets better and better So this is the Monte Carlo method it is Pie is not its best use case, but it's usually used in spaces Where it's very hard to do a full simulation of something So we just do a random simulation and in the end we just assess what is the outcome and by running multiple Random simulations and then assessing the outcome. We can get an estimate or a guess of what the final value is And in go that is basically our evaluation function So this like the Monte Carlo method was proposed before in 1993 in the paper, but it was not made to work It was not until the year of 2006 until The Monte Carlo tree search was developed and successfully applied to go or was actually deployed developed for go and at that year The program that implement Monte Carlo tree search also won the go tournament So ever since all go programs that are successful use Monte Carlo tree searches So that was sort of a revolution and it's been rather recent of a bitter search has been around since 1957 or something So this is the basics of the Monte Carlo tree search We go through a selection phase first to select the number one to play at then we do an expansion Then we do a random simulation and afterwards we do a back propagation of our results So what does it look like in detail in this graph? We say each node Has at first like the winds that have been at that note and then last the visits how many times have I visited that note and The notes represent a board state and then afterwards. It's like we play a move and then the next note is like the board state that Results when we play that move So at first we go to the selection phase we select the note We want to play it. We want to okay. We want to see here. What happens here? We've got one again more information here Then we do an expansion that says like okay. This is the move We're gonna try here at that specific node Then we're gonna do a random play out and you might be right now You might be like, okay, what random play out like what what are we doing? Like it seems like not very smart to do a random play out But the hypothesis is this if I'm standing better at the board in general If I have a better position if I do a random play out and I win in the end Like I'm more likely to win in the end if my position was better And if you do that lots and lots of times we get a pretty clear view of is our position ultimately better or worse So and in the end we do a back propagation that says okay We have one here. So we propagate that result back to the top and here you got to be careful a bit if you remember The min max so it's also a matter of perspective. So up here The sort of darkish thing is like the black move and down here. It's the white move And of course we can't do a back propagation that when white has won then of course black has not won like has lost And so you either have to take that into account during selection Or how my go-bot does it and how I know other go-bots do it is you always alternate how you how you report the results You say okay white one then black loss white one black loss white one black loss And so you always alternate the result that you report so that the selection is sort of uniform So let's talk about a bit about the selection because it's a crucial part So we could just take like every move all the time and play each move equally all of the time But that's not very smart and since we're already doing like the random play out stuff We will at least won't be smart about the selection, you know, and This is a so-called multi-arm bender problem, which is one of the most interesting problems I've come across there and it's that scenario you're sitting at a casino and you're playing at those Bandits and you're playing at your machine and it's going good and you're like okay. I make a lot of money That's cool, but then you start wondering like maybe one of these other machines yields better results maybe they give me more money and That's sort of the thing that you want to balance also in a tree like you can always play at the game at the point Where you're winning or we think you're winning percentage is best But there might be another note that might yield better results, but you were just unlucky Unlucky sorry so far there So that is called exploitation versus exploration exploitation is like okay I'm playing at the note with the best winning percentage and exploration is okay Is there something else out here to try and balance what we're doing and for this? The standard solution that we use it's the upper confidence bound or in go It's uct the upper confidence bound applied to tree search and this is the general formula And this value is calculated for every note to then select which note we're gonna see so we take the win percentage And then we add to that some sort of exploration factor that you can set To your liking and there are multiple papers about like which is the best exploration factor of course and then you take the Square root of the logarithm the natural algorithm of the total visits divided by visits So that is sort to balance like if I haven't Visited a note a lot. Let's try and visit it again So how does that work? This is like now a real example that I took from my go engine right now at the top here The number is just number of visits because not enough space. So anyway first selection We select this note because it has the best winning percentage of all the notes that we have right now So I cut out lots of other notes to make it fit on the slide But then what's what was interesting for me when I looked again into this is on the second selection at least for the Exploration factor that I use we do not play on the free of free note that we would say that would be the exploitation It's a note that's going good We're gonna play there again, but it's like, okay This note has like a pretty good win when percent is 50% we haven't visited a lot So let's try playing there again. Let's try what's happening when I play there So at this point you might ask like what are we doing here? This is not at all like human like We're like humans don't play like that humans don't go out and be like, okay I start the game So I'm just gonna play random moves in my head and at the end I'm gonna counter perfectly and then I'm gonna decide on That's how I play Humans look for patterns they look for like how does the board look and what would I normally play and there's a lot of intuition of all involved a lot of times in human play and I chose a picture because I want to remind you of how humans Try to try to fly and I know the 16th century We try to imitate birds. We try to have feathers and feb or wings and fly and that didn't work out quite well And what we do right now is we have this huge like much bigger than birds This huge metal cages with some wings that sort of resemble them But not quite and helicopters even more different and that works out better for us Why does it work out better for us because we? Choose the strength of humans to build these mechanical things and we have our engines and everything We use that and not what birds are good at similarly and go We don't do what we are good at but we're doing what machines are good at they're good at Playing random moves and then in the end counting the result perfectly and doing that lots and lots of times per second So go engines play and do like 50,000 of those playouts per second not mine because it's a ruby, but yeah, that's you know So what are the characteristics of this algorithm or? The Monte Carlo tree search at first is a heuristic which means I've talked to you about it and we're just playing random moves We don't put or we don't have to put any expert knowledge into it so all that we need to know is How do I generate a valid random move which is basically what I teach any beginner if I teach them go? I tell him okay, this is valid. This is you can do this but not that and We have to teach it who has won we don't even really need to teach well, yeah, we have to teach him to score to figure out who has won but we only interested in who has won in go and For me that sort of blew my mind because it means that I can be like a bad Go player and I don't have to know a lot of go I don't have to encode it into the machine the machine really plays by itself It figures out what is the best move by itself by just teaching okay this is valid and then the ending can see have a one have a lost and that is really amazing because that way we're also not limited by Human knowledge because the machine might at some time surpass us and think of moves that we could never think of And otherwise we might limit it by our human knowledge or human assumptions that are encoded into the game Which means also that the Monte Carlo tree search is the go-to algorithm for general game playing Which is the discipline that I did not know much about before but basically tournaments where before you get like a rule set encoded in a file and then the Engines have to play games that I never knew before and due to that And a characteristic of the Monte Carlo tree search that does not need to know game specific knowledge It's the go-to algorithm for general game playing as well Also the algorithm is anytime with the alphabet a search It's a depth first search So I always have to search through the whole tree until I can make my decision on the best move with the Monte Carlo Tree search I can stop it at any time and say like okay. This is a good enough move like I can play this move It might be really bad But I can do that which is good if you're doing time management and Monte Carlo tree search is lazy. This is my favorite paper about About Monte Carlo and so I have to tell you this so imagine a game. This is game is called double step It's stupidly simple And double step you can make one move or two moves whoops too far whoop one move or two moves and you win when you reach the end of That thing so it's pretty easy. You know as a human you say okay two moves is always the best move, you know So yeah, we win when we're at the end so what does a Monte Carlo tree search do here now and So on the top here We have the handicapped handicapped of minus two means like I'm two steps behind Handicap of two means okay. I get a handicap of two. I'm two steps ahead on the left side We have the number of simulations of Monte Carlo simulations That I ran so eight random player out 16 random player. It's 32 64 100 Etc. And what we have in the middle here is the percentage of that black for its first move selected two steps and so I ran this 10,000 times each For each of those and we see the winning percent here So we can see at the bottom that we're doing pretty good if we do 100 playouts We're at 99.8 percent or even at 100 percent if it's an even game and we can see many characteristics here So for instance, especially with the lower play out So let's look at the first line the eight playouts We can see that when we're behind by two we only have a percentage of 86 percent to pick the optimal move because that is Monte Carlo and just get sort of desperate when they're behind by a lot They play they often play nonsense moves, especially also and go they play moves That's like okay here I played that and then my opponent would have to to not answer that move for two or three Turns in a row in order for that to yield any value because they get really desperate But what's even more interesting is the number in the bottom right? As we get more handicapped as we had more our moves get worse at least for the 100 Playouts, so we just take the best move 98 percent of the time and that's because Most Monte Carlo engines just care about if they're winning or losing they're not caring about by much help How many points they win because that yields worse results as Richard shows and you can often observe that And play when a Monte Carlo bot that you play against is ahead by a lot It will just play very very safe moves according to the motto better safe than sorry and by the end the go-bots Will win by maybe two points or something you'll be like oh, I almost bet that go but yes Not at all the go-bots as I have one either way So I'm not going to play these really safe moves because I don't care because I'm gonna win either way So a move that sort of showcases this is the last move of the game that Zen won against Francis of Decute in the commentary and Also for every other worlds of our place then let's move its mark with the X in the middle and also green highly highlighted now Most human players consider that move unnecessary because it's pretty clear that that middle is fully blacks territory There's some cutting stones, but there's not much danger coming from them Nevertheless then still played here because why then knew it was two and a half points ahead so wasting a move in the other in the Territory it was still one and a half points ahead and it would still win So it played better safe than sorry So what are some of the enhancements that we can do? To play better any Monte Carlo to search algorithm Well, my favorite enhancement is all moves as first and also has a variant which is called rave rapid action value estimation And the basic idea is this if we play random moves all the way out all the time Then does it really matter in which order we play them and no it really doesn't if the if the moves are pure random So here in the back propagation phase when we propagate the result of the game back We also look at all of our sibling nodes and look at if any of those nodes is a move that we made At any point during the game and then we also update the win stats For that move what the rave algorithm then does is it says like okay? That's cool that we do that, but it's not the most accurate thing in the end So it gives those extra moves that we update So the call of arm of wins and arm of wizards for all moves as first it gives them a diminishing value So at first they take into account a lot during the selection phase But when we reach like two thousand arm of wizards or something then it ignores those values because we fed Border and just takes a look at the real winning percentage of moves that were really played at that specific note Of course, we can also encode Expert knowledge into the multicultural search and there are two phases where we can do that First is the selection phase So where we select at which note do we want to play which note do we want to try out and Therefore we can use patterns. We can use Atari solvers and we can also use use a key. So opening libraries and The second is the play out policy. So our random simulation We can also use patterns there to try and make the play outs not purely random, but More realistic to get a better value out of them and this is very dangerous Especially if you use it with all moves as first which almost every engine does I think every engine does every good engine at least Mine doesn't do yet, but we're going to that It is dangerous because then it's not purely random anymore and you could get like false positives and false negatives when you Update the tree. So you gotta be very cautious with that and it is for improvement signs that if you do a heavy play out that biases Or a selection or a play out that biases the tree too much. Your problem actually gets worse although you add knowledge and But nevertheless all strong programs these days do the so-called heavy playouts and so they're only doing 5,000 play outs because heavy players of course take more time to calculate. Oh By the way, I forgot to say before in the match between Francis liquid and then then played I think on a cluster that had 80 cores in total and like 60 gigabytes of RAM So it's also a pretty high computing thing So this gets us to that thing that I was mentioning that probably some of you saw at Hacker news It's playing go against the deep neural network and this side was done by the author of this paper Which they took a deep convoluted neural network I don't really know much about like what that really is to be honest But they trained it with professional games And they managed a prediction rate of the professional moves on their training set that had a value of I think 44% it told us so it's very good at predicting professional moves and this paper It was also referencing this paper This paper is from 2014 and they did about the same thing with a twelfth layer convolutional neural network and they managed a prediction rate of 51 percent in there as well and Which is great like this approach has been tried before and it's only like this papers to my knowledge the first one that made it Useful because before I think was tried in 2007 2009 was always deemed not useful at the time and It's a great success. It's one of the newest things in computer go But these engines have a big problem. They only know what professional moves are and how professionals react So if you play moves like trick moves are very bad moves Sometimes they react really badly because they're not used to what you're doing there And so you can also get them into complicated fights are not like Professional fights and they were pretty very very badly and if you've seen the keynote on the first day by Karina That's also what like all their machine learning hours and they're their fuck ups We're in those algorithms then because then they fuck up badly and that is also reflected here So the strange varies badly between those but what's even what's really great about this is we can We can include this into Monte Carlo, so these these are pretty strong, but they still lose against strong Monte Carlo bots so against for ego, which is about a one-down strength but The bot from the other convolution network bot lost 86% of the time so they're not better than Monte Carlo bots, but we can incorporate the neural networks into our Monte Carlo bots and we do that at the selection phase So we don't have much computing power left because we're always using all our cores and everything to do the simulations and the tree updates And everything but we have a little nice thing in our computer. That's a graphics card So what we can do to incorporate that is to as synchronously push the computation of the recommendations Of the cure of the neural network to our graphics cards Let it compute those computations and then bring them back and incorporate them into our selection so that we try to play the moves that the network proposes to us first and For me, that's kind of crazy. Now you use all of our course You just a graphics card use tons of memory and I hope you have a good cooler in there So and this basically bumps the strength I think I implemented it and just by adding convolutional neural networks it bombed its rate It's rating on KGS by one down from like three down to four down. So it's it's pretty cool. It's a cool area of research So that Facebook thing that I made sort of a buzz around We think we found their bot on KGS playing and it plays around the level of 4q or something and in the video that they showed where they totally bet another engine They just bet can you go at the very low level? So it's not much to be excited about yet, but interesting to see what they come up with and if they at some point Gonna share their findings with computer girl and the computer girl mailing list that would be nice of them Oh, yeah, there's a if you're interested in this thing. There's a very cool computer girl mailing list where All the authors of the popular programs are there and they share and you can ask and you sometimes see me asking silly questions Or good questions depending on my mood So, of course, I wrote a go engine in Ruby which is called Rubycon you can find it at github.com practop Rubycon and you can also do gem install Rubycon And then you can invoke Rubycon to play a little game in the CLI it just implements multi-color tree search yet There's still lots of performance optimization potential for me to do The move generation and the scoring faster, but I'm on to it I also have another project which is Vekko, which of course looks beautiful which We're implemented the whole thing in coffee script and using web workers to outsource the work But I still have to do some work on that another thing that you might want to look at a much better engine is Mitchy from It's a Python engine that is in like 500 lines of Python and it is Very readable. It's all just one file, but it's like a minimalistic go engine. I take lots of inspiration from there and For those of you that aren't to rust because rust is a cool thing to be in There's also a nice rust engine Who's also is also a German So now in the end, let's see how much time to have left Damn it. Okay. What have I learned? We're quickly so there's huge difference between making X faster and doing less of X and all the time We just try to make something really fast and not go the road of doing less of it programming Ruby Doing less of things cost nothing and so it's the route. I ran lots of times Modernizing small components Monte Carlo tree search is completely independent of the game of go And so mine is as well. You can plug in any game into my Monte Carlo tree search Library and it will play it if you have a facade class to get into there and benchmark everything and there's a huge difference between solving things we always try to solve problems the human way and Not the computer way, which is what we should actually be doing most of the time as I try to Show you with the Monte Carlo tree search and then of course I recommend everyone the jar of creation like seeing an engine that you build yourself and they just go It made that move. That's cute or it made that move. That's actually pretty good. I'm pretty fighting spirit So thank you for talking to you. Thank you very much for listening to me ramble for so long This dog was beating go. Thanks to the power friend in this I'm practice. Thanks


Hi Freedom Family, we just paid you $2.3 million yesterday. Woohoo! And we helped YPP recover many of your demonetized channels, I'll show you which ones, and what we told YouTube to help convince them to reinstate monetization on those channels. And we have deals with major record labels, three of them, to get you professional music on your music and music promo channels, where you can help artists get into the top 100 billboard licenses by correctly embedding the ISRC code of the tracks you use. I'll tell you all about this. And the length of this beard is a measure of the time it's taking me to make this episode. If you remember on my last episode, I promised you I would not shave until I made another episode. Well, here it is. This is itchy. I'm eager to shave it, but let's get this episode done first. Let's start with number two. These are all the channels YouTube demonetized recently. We fought for each channel and gave YouTube additional context explaining why we think each channel should be demonetized. And I'm happy to say each one was successfully reinstated based on our arguments. Let's have a look at a few examples. True Reporting is a news reporting channel. He uses commercial and copyright footage from websites and videos to talk about news. But he does it in a way that's fair use. Let me show you how. A very interesting article from Bloomberg entitled... So he starts his videos by showing himself on a webcam. And then he goes into the meat of each episode where he does show other people's content without permission. But the key is he adds human commentary. Long blackouts in plural. My friends, we are very... And he brings himself back into the picture periodically. This is an example of fair use. As long as you add your own human commentary throughout your episodes, it's okay. If you just show clips with text subtitles or if you show gameplay from other people and you just compile it into a top list of like top 10 League of Legends plays, that's not monetizable anymore. That's called reused content. So whatever you're doing, make sure it has your human commentary. Now there are a few exceptions. And this is one of them. Gatao 2018 was reinstated because they have explicit permission to promote this movie. Let me show you an example. In this video, it's a trailer taken from the movie.  No human commentary, but it's okay. Why? Because we showed YouTube the contract between the movie production company and this channel authorizing them to upload and monetize this content. This brings us to music promo channels. We successfully reinstated Scubb Domino, which is a black MIDI music promo channel. He takes music that he arranges into piano and makes videos with a visualizer showing the key presses on his keyboard, his musical keyboard. Let me show you an example. So we argued to YouTube that this is actually original content that the channel owner is spending his time hours to arrange these commercial music pieces to play them on his digital keyboard and YouTube agreed and reinstated the channel. Now normally music promo channels get demonetized and they stay demonetized because they take people's music that's trending and popular, upload it to their channel and try to make money. The right way to do music promo and not get demonetized is to get a music from artists and record labels with their permission that they give you not what you want and upload it to your channel. And this brings us to our third point, the major record label deals. We have deals with three major record labels now and more coming that give us music every week. They pitch to us, which we then share with our music promo channels to upload with full permission to monetize as long as we correctly embed the ISRC code to help those artists get into the top 100 billboard charts and Nielsen licenses which is something you can only do if your channel isn't a music CMS. Now that's a mouthful. Let me break it down for you. Here are some music promo videos on one of our channels and I'll show you Nightcore Remember the Name Fort Minor. This is the video and if you look at the monetization tab, there is an ISRC, hi Daenerys, there is an ISRC code embedded in the music video asset type for this video. This is only possible if your channel isn't a music CMS. Normally you only have the web and TV episode and movie asset types but you gain music video which lets you embed an ISRC code. But this is not enough guys. Just adding it here will not allow those artists to get the benefit of chart reporting to get their video views counted as song plays towards top 100 billboard licenses and Nielsen licenses. In order for that to happen, the sound recording tab right here has to embed the correct sound recording based on the ISRC code to get that chart eligibility. Now this embedding is not possible through the YouTube UI. You need to use the YouTube API or a content delivery template that's only available in a music CMS. This is new information. We've never known this before. Hi Daenerys. So Daenerys is our two-year-old mother of dragons daughter who is a little excited and near me while I'm filming. So this sound recording embed is what we are now doing for all of you music promo channels that are partnered with Redum so we can use the music from the professional major record labels because that's important to them. This chart tracking is the main reason they're giving us monetization so that their developing artists have a better chance of getting into these chart licenses that they can get discovered and become popular in concerts and on radio. So music promo, sound recording embed, ISRC code, we get pitches from the labels and we can also pitch to the labels meaning if you are a music promo channel that wants a particular track we can request permission to upload it for you and if we get it you can upload it as unlisted and then the record label will release the claim from them, the sound recording claim then you can make it public and we will correctly embed this ISRC code so that everybody is happy. I hope this helps you understand that just because you've been YPP suspended it's not the end of the world, there is a good chance that you can get reinstated just email support at freedom.tm and we will be happy to help if you are in our network and we paid you that 2.3 million dollars, remember? Here's our PayPal account before sending you the money and here's our PayPal account after. We also pay you by wire, this is our Tepalti account before and after and we also pay you by Payoneer, this is our Payoneer account before and after and this is how we paid you, we have a back end earnings UI that we wrote that has this magic button that lets me pay everybody like millions of dollars with a single click there I go clicking the button and this is for, we do it in two parts this is the first 1.3 million dollar payment here and then we have the second batch which I'm about to do right here let me hide myself again and as soon as I click that another million went out the door now this only takes a few minutes to send all this money to all of you we're quite proud of our payment system and if you didn't get your payment you can always issue a catch up or just email support at freedom.tm we also pay you earlier than everybody else in case you didn't know click that I to learn how and why and you have access to instant payments which allows you to get paid up to 60 days before everybody else based on your YouTube analytics estimated earnings just check out your shiny freedom dashboard on your earnings page to take advantage of this instant payments feature in case you need money even earlier the link to all of these benefits that freedom offers to all of its partners is down there in the description below feel free to click it and read all of this in case you're curious to learn what else you have access to as a freedom partner and we have a mission statement freedom family and that is roll it and we have a two year old Daenerys who wants to enter the video hi Daenerys you want to watch so that's the new phrase she's learned I want to watch is that right with a cute face like that who can say no let me finish this video first who am I I am your Papa George your George son pie your Obi-Wan George Nobi and many other names you the beautiful freedom family has given me right Daenerys you want to watch I'm the CEO and founder of freedom freedom and the freedom family okay she's not responding to that yet so I hope you enjoyed this episode guys I hope you enjoyed getting paid early I hope you found a lot of the YPP suspension information I shared helpful I'll be sharing a lot more click that I to see the entire KYC series know your channel where I give a lot more tips about keeping your channel safe and I hope you enjoyed the cool update about ISRC codes and helping artists okay so that's saying daddy stop talking you're boring me yes okay until the next episode freedom family I will get to shave my beard now unless 10 of you at least more than 10 of you tell me I shouldn't down there in the comments below so let me know but I would like to shave it because it's itchy and until the next episode I am George and you've been watching that's right Abby would you dreaming but love you click that I to partner with freedom and join the freedom family so we can all grow together you get many perks like position music you also get epidemic sound a lot of other access to royalty-free videos sponsorships many things to help you grow just click the links down there in the description below to get involved in our community our forums our discord chat servers meet our graphics team meet our community team all of that on discord and the forums what are you waiting for get started and we will grow together as a family because this is the freedom family you are part of it we are all part of it and we're all growing together to get more George click that big F that will subscribe you to freedom central home of the George show PewDiePie gave one of you freedom family a big shout out click that video to see the shout out and to see our new 3D sets for you and click that video to see what YouTube recommends you watch next


I'm here in the National Mall to commemorate Veterans Day, standing among the war memorials which define our history and remind us of our legacy and of our heritage, from the Great White Fleet of 1907 to those operating forward today. And they're also a great reminder of how lucky we are to have generations of Americans willing to go in harm's way. So as we honor our veterans serving today and who've served us in the past, happy Veterans Day.