Hey, I'm Zach. I work for Factual, and we're going to talk about a little board game called Go. A quick show of hands. Who here has played Go? Oh, this makes it so much easier. Okay. Anyway, we're going to do a quick refresher. So Go is played on a board with intersections, typically anywhere between 9x9 to 19x19. Stones are played on the intersections, first black, and white. And each of these stones has liberties. Liberties are empty adjacent intersections. And when you play a stone in one of these liberties, this is extending the group. These contiguous blocks of stones of the same color are now one big group. And when a group loses all of its liberties, it is removed from the board. And the capturing player gets a point for each stone removed from the board, but also gets a point for the enclosed intersections. Each enclosed intersection is one point. And so if black here has enclosed the corner intersection, it has a point. And white cannot play in that intersection because it would have no liberties. This would be effectively suicide. However, if white completely encircles the black stone, for that one point, it can play in that point and thus remove the stones. Now, this may make it seem like Go is just a game of encirclement and re-encirclement and all this other sort of stuff. But if white plays here, even if it's fully encircled, black cannot actually remove this from the board because in each of these two liberties, black cannot play in either of them. It would be suicide for it to play there. So this white group is stable. It's safe. And that's because it has two eyes or two liberties that it's completely encircled. And so Go is not a game about capture so much as it is about trying to encircle territory in a stable way. And so it's less a tactical game and it's more of a strategic game. And so what you'll see is when the game develops, what it is, is it's a series of claims, right? By playing one space to the right, black here is sort of implicitly claiming the right side of the board. And white can have many responses to this. It can play to the left thus effectively sort of seeding that territory, though that may develop over the course of the game. It can play above claiming the top of the board and seeding the bottom. It could even play within the territory effectively being claimed by black, kind of stating that it can create a stable group before black can encircle it. And as the game develops, you'll see that a lot of the territory is encircled and not contested, right? This is the ending position of a professional 19 by 19 board. They haven't really gone and played out all these territories because it's not necessary, right? They kind of understand that they stand more to lose by sacrificing a bunch of pieces in those territory than trying to actually fight over all of it. And I don't recall who won here. I think it might have been white or something, but at any rate, we want to write a program that plays go, right? And I'm interested in this not because I think I have any unique contributions to make to this field, right? I enjoy Go. I think it's kind of a beautiful game, but I'm not particularly good at it. I studied this a little bit in school, but it's not something that, like, again, I have any real expertise. This is just something that no one's really done before, right? And when I say not done before, I don't mean just enclosure. I mean that people typically write this in very low-level languages, like C, C++. I think there are a couple written in D. But, like, that's as far as it goes.  You know, certainly there are some stumbling blocks here. But, you know, closure is a wonderful language for exploration, right? For experimentation. And it stands to reason that maybe this is something that, you know, would be helpful for us to have this sort of high-level language, to be able to kind of span both sort of the lower-level stuff that closure can do, but also the high-level stuff that closure kind of excels at. So, where do we start? Most AI is search, right? And so when we're talking about, you know, we're here at this particular position and we want to make a move. We want to make a smart move. The thing is that we want to search for a move that will lead to victory. And so, in Tic-Tac-Toe, we start with an empty board. And we say that this is the root of our game tree, of the game space that we're searching. And we say that the children are all of the first moves, all the initial moves that X can make. And then beneath that are all the responses that O can make. And that just kind of continues down until we get to the terminal position, so the end points of the game. And this is a space that we're searching. Now, if this were a cooperative game, if this were solid hair or if we were playing together and trying to be nice, then all we would need to do is search through for a path to victory, right? Just some sequence of moves that would lead to us winning. But we're competing, right? And so we have to assume that as much as we want to direct the game towards this ending position where we win, we have to assume that our opponent will be doing his damnedest to make sure that we don't, right? And so we have to optimize our chances for victory given this sort of adversarial quality to our search. And the approach for this is called Minimax. It was invented or discovered back in 1930 by John von Neumann. And the basic notion is that we search the entire tree. We go all the way to the bottom. And we say that each of these has a value according to its outcome. In this case, this is fairly simple, right? We say it's one at its victory, zero if it's a tie, negative one if it's a loss. And we assume that the player in the state preceding that will choose the best move for himself, right? He will direct it towards and it's there. And so we say that the value of the node just shy of the ending condition is equal to the best possible outcome for that player. And then we can go up one more level and we can say, well, the player will clearly direct it to the best possible for themselves. And from our perspective, right, we're constantly maximizing the outcome, and our opponent is constantly minimizing the outcome. And from that, we kind of go up the tree and discover what the optimal path of play is, assuming that both people are playing as best they can. And if we're able to traverse the entire game tree, this works. We can play a perfect game. So for Tic Tac Toe, we can do this pretty easily. There aren't really that many possible Tic Tac Toe games. This has been done for checkers as well. In both cases, it's a tie, by the way. But for more complicated games, this is not something that we can do. This is not something that's plausible. We can't just enumerate all the possible chess games or all the possible go games. And so we have to kind of compromise. And the way that we compromise is we say, well, you know, this is still a pretty sane way to talk about how we're searching, but we just can't get to the bottom. So instead, we go down as far as we can. And we look at this intermediate game state, and we say, how likely are we to win here? Right? And that's a difficult question to answer. And this requires not just an understanding of the mechanics of the game. This understands sort of the fundamental essence of the game. This is something where we require sort of meta information more than just the rules of this move leads to this move leads to this move. And this can be difficult for a number of reasons. One is that if we say have a chess game and we start counting pieces or something like that, then clearly we can optimize that we'll have a certain number of pieces at a certain depth of the game. But that isn't really a very accurate proxy for victory, right? There are pathological situations where that can direct us very much in the wrong direction. Similarly, we have this sort of horizon that we can't really see very far into the future. And this horizon is kind of inversely proportional to the cost of our evaluation function. We have something that's very rich, something that gives us a lot of information, has a lot of analysis that's done, but then we can only see so far into the future. So it's possible that a more naive evaluation function which lets us see further could be better. Of course, that's very difficult to quantify. And so this is sort of a trade off where we're trying to kind of twist this dial and not really perfectly understanding what the consequences of that are. And finally, you know, we as programmers, we write this code, we write the recursive search, we write the evaluation function. But understanding the local effects of this function is not the same thing as understanding the global effects of the repeated recursive application of this function, right? This thing that is pruning this tree and sending us in one direction at the expense of another. And this is a bit of a problem. I mean, this is actually something which is horribly frustrating because you'll see this movie made and you know that it's the wrong move, but you know that if you try to tweak your evaluation function to make it so that it doesn't make a wrong move in that case, it has untold consequences elsewhere, right? And this is sort of the essence of the problem. This is kind of wonderfully frustrating on some level. And this is, you know, why I am so interested in this whole problem. But, you know, all these things apply to go to a greater extent than pretty much any other major game. And the reason for this is that the tree is enormous, right? On a 19 by 19 board, which is sort of the standard, you know, size for a board, there are 361 possible moves. And even worse, the depth of play, the number of moves into the future that a strong human player can see is fairly large, right? And this is because the board remains fairly static. How many of you have ever played reverse eye or Othello? So this is sort of a variant, a simplification of go. And what this says is, if there are two stones of one color on either side of a line of stones of another color, they all flip. And what this means is that three moves out, the board can have completely changed configuration. And so it's very easy to write a strong Othello AI, not just because it's a simpler game, but because it's really hard for people to read very far ahead. Conversely, go remains largely static over time. And so it's fairly easy for people, right? Basically, we're uniquely qualified to be good go players. And so this has remained kind of a problem. And historically, a lot of people have tried to write go eyes, starting back in the 60s. There was a prize for a while that if someone could create a program that could beat a low level professional player by the year 2000, they would get $1 million. This is one of the millennium prizes. And no one won, right? A lot of people tried. A lot of people sort of threw themselves onto those rocks and kind of walked away defeated. And this is because it's a hard problem. And the computational power wasn't really there. And so what they would do is they would try to offset that by being very strong players themselves and trying to kind of encode their brains into the program. But that only works to the effect that they can perfectly capture their understanding of the game. And this is this is hard. This is harder than I think most people give it credit for. And so typically speaking, this has kind of been a bit of a backwater or sort of a failure, whereas, you know, chess is sort of taken off and kind of capture the imagination of people go has remained this sort of almost embarrassment. In 2006, though, someone tried something new. And they used something called Monte Carlo simulation. And to sort of demonstrate what this is, imagine that we have a quarter of a circle inside of a unit square. And we have a function which says, given the x and y coordinates, this is inside the circle. And we have a function which takes no arguments and simply says for a random point between 00 and 11, is it inside the circle? If we just call this a million times, and then we look at the ratio of trues to the number of samples that we had, that's pi over four, or converges on pi over four. And this may seem like sort of a trivial sort of clever math trick, but it's actually something kind of fundamental. What it is is it's saying, we understand the rules that govern a space. And now we can begin to understand its structure. And, you know, in this case, it's fairly easy. It's geometric. We can kind of say, you know, here's a space that we're trying to sample. But in real world cases, this is sort of less doable. And so it's instructive to think about what would happen if we didn't really know that it was 00 to 11. What if this was a little bit shaky, right? Depending on where we skew, this can start to change the value quite a bit, right? If we sort of skew down to the left, and all of a sudden pi starts, you know, ballooning out. And it can also start shrinking if we go in some other way. And so the thing is that it's not just that we do a bunch of simulations, we have a bunch of numbers, and it all kind of, you know, magically comes out in the end. We have to understand the space that we're sampling. We have to understand these sort of dimensions of what we're actually trying to search out. And so we can use Monte Carlo simulation as an evaluation function. We can say from this position, assuming that we simply just play naive moves, no real insight into the game, no attempt to make a good move, just a move. And we do this repeatedly, tens of thousands, hundreds of thousands of times.  Therefore, this is a stronger position. This is kind of cool because what this means is that we've taken the rules of go, no understanding of go other than just the simple mechanical rules. And we have arrived at a determination of what is a better move, right? This is something which isn't really clearly encoded in the rules, except in some sort of various sort of amorphous way. But we're not trying to talk about the space of all moves that can be done, right? You know, if we leave ourselves open to capture and our opponent ignores that for the entire game, that's not a very likely outcome. We want to talk about plausible games. We want to talk about games that are likely to be played. And so if we go outside of this range of what is a plausible game into the range of possible games, this is noise. This is actually like sort of dirtying our understanding of the space. And so this is a problem, right? This is a balance we have to kind of do. Again, this is like the evaluation function. A heavier, more computationally expensive evaluation function is at the expense of being able to see further, being able to, in our case, have more samples. But there's an additional thing, which is that, you know, for doing this hundreds of thousands of times, each new game gives us more insight into the space that we're trying to simulate. And to capture that, to take advantage of that, we use something called a multi-armed bandit algorithm. And the multi-armed bandit basically says, we have a bank of slot machines. And we don't know what their payout is, but we presume that one of them is better than all the others. And the problem here is that we can go through and we can continue to, you know, pull them all one by one and try to build up a statistical model. But once we start to have a hypothesis as to which of these is the best, every move that we do, every lever that we pull, which is not the one that we suspect is the best, is that the possible expense of our expected return, right? We have this balance between trying to explore all the possibilities and exploiting the best known possibility. And there are a number of mathematical approaches to this. I'm not going to talk about them in any detail. The one that is commonly used for go is called the upper confidence interval or upper confidence bound method. But it's important to understand that if our probabilities aren't changing over time as we're sampling, we should expect that we will converge on the best solution or the subset of best solutions. So if we put these two things together, if we use Monte Carlo playouts and the multi-arm bandit, this is called Monte Carlo tree search. And it has basically four steps. We have a tree and this tree is only partially searched, only partially expanded out. And what we do is we go and we select a path and we use the multi-arm bandit at each game node, each board node, to determine the next move. Until we arrive at a leaf, at which point we select one more move. And then from there, we do a single random play out using the Monte Carlo sort of approach. And we then take that signal of loss or victory and we propagate it back up the tree to all the moves that led there. And since a positive signal will effectively give a greater weight to the path that we took, the assumption here is that we're going to have a very asymmetric expansion of the tree, which tends towards the better moves. And so for the rest of the talk, I'm only going to talk about the simulation part of this, because this is sort of the medius part of it and also from an implementation standpoint the most troubling. So let's look at what it actually takes to play go. For each move, we have to select a move. We have to check whether it's a suicidal move, whether it's a valid move. We have to update our representation of the board. We have to check whether or not we've captured anything. And then we just repeat that over and over again. And these two, the suicide check and the capture check, are similar problems and also are kind of a little bit expensive potentially. And consider this kind of pathological case here. If we look at the corner position, black cannot move there. That will be suicide. It would make the entire group disappear from the board. But to determine that, we can't just look at the neighbors. We have to walk that chain all the way to the top, examining at each point whether or not it has any neighbors going, continuing, continuing. They'll finally find, oh no, this has a liberty. It won't be a capture. Similarly, if white moves there, it has to walk the chain all the way up that entire chain of black stones, checking at any point whether there are any liberties. And so if we have a very naive representation of this board, which is just, you know, this stone is here, that stone is here, this point is empty. This becomes very expensive. This is something which we're doing a lot of duplicated effort. Each individual move to determine something that we may have determined at the preceding move. And so the way around this is that we want some sort of incremental state. We want to have something which is derivative, right? This is not a completely like normalized representation of it. But we want to minimize the amount of computation we have to do at each individual step. And so when we put down a stone, we can actually keep track of how many liberties it has, right?  We put down another black stone. And now we can say that this group, collectively, has six liberties. And rather than the previous stone keeping track of its own stuff, it just points to the parent. It points to the owner of the group. And we place down another stone. It'll point there. And it's notable that in this case, the highlighted intersection is being counted twice. And this is because we could try to keep some sort of unique set of points in order to have an actual unique count of liberties. But that means that we have to have a whole data structure to store this. This isn't just arithmetic at this point. And so this is not a liberty. It's what's called a pseudo-liberty. This is a fairly common approach in computer go. And we will double-count any intersection that we're multiply adjacent to. But if someone plays in that position, it will be double-decremented. And so this tells us whether or not we have zero liberties, right? It doesn't matter for double-counting. Once we're completely surrounded, we will have both zero liberties and zero pseudo-liberies. We also keep track of the sum of neighbors by assigning a numerical value to each intersection. And the sum of the squares of those neighbors. And the reason for this is that if the sum squared equals pseudo-liberies times the sum of squares, then we have one unique liberty. And I'm not going to go into the math. It's not that hard, but it's, you know, I'll leave it as a proof for the reader. So let's go back to these things that we're trying to do here, right? We have these five steps. And literature varies a little bit. But, you know, people can have talked about being able to do anywhere between 5,000 and 50,000 games a second at 9 by 9. So we have one second to do 10,000 games. We'll say that there are about a hundred moves a game. There are 81 positions on a 9 by 9 board, but we get captures. And so the same intersection will be played in multiple times. This is actually a little bit low. It tends to be closer to about 120. This means that we have one microsecond in which to make a move. Here's some things that take a microsecond.  I mean, we have fast computers, right? You know, a giga Hertz processor means that it takes one nanosecond, one thousandth of a microsecond, to do a single instruction. Like why is it, I mean, yes, we have these abstractions that we've layered on top of this, but why? And here's some things from a fairly old article by Peter Norvig, which talks about how long it takes to do certain things on a CPU. It takes one nanosecond to do a typical instruction. It takes 25 nanoseconds to lock or unlock a mutex. The notable one here is the fetch from main memory. And notice that this is an article from 2001. This number hasn't changed since then. Now RAM has been getting faster, but that doesn't actually speak to its latency. What it says is that when we do a memory fetch, we pull down more memory in that contiguous block. And this means that if we're scanning over a contiguous block of memory, these reads are basically amortized. So we do less fetches, therefore it's faster, but we don't store things too contiguously in Clojure or in Java. We have a lot of references all over the place, and our caches are getting larger, and so this sort of minimizes the problem to a certain degree. But what this means is that every single time that we have to go off and search for something else, something that we haven't seen before. Every time we, you know, allocate some memory, which is, you know, something that we haven't touched in a little bit, we lose 100 nanoseconds. And so it's pretty easy to see how some of these things end up taking. They have, you know, 10 fetches from memory or something, you know, around there. And so what do we do, right? I mean, it's clear that on some level we have to take a step back from the standard approaches we might otherwise use. So I'm going to talk about some of the approaches that I used. And before I do that, I want to talk about why this is not generally applicable, right? Why you should ignore some of what I tell you here. So in my day job, I work with servers. And if I can get the response time under 10 milliseconds, that's pretty good, because that means that probably the latency is going to turn my response time into background noise, right? That's not true here. We're throughput bound. Also, we don't really have the option of using a smarter approach. We can't use a better algorithm because by virtue of the method that we're using, this Monte Carlo search, the only way we can be smarter is to do more of it. The only way that we can be smarter in the same amount of time is to be faster. And while maybe there isn't sort of diminishing return where once we get to a certain sample size, we don't like get that much smarter or anything like this, this is beyond our reach right now. No one has found a point where they're like it's okay, we can stop. Like the more computational power, the more simulations we can do, the better we will be and that will just kind of scale fairly linearly. So, we have to use mutable state. This is from the doc string for def type.  And he's right. You know, this is not something that you should use lightly. This is not something that, you know, the fact that it's available in Java so freely, this is sort of the default, isn't an indication of its safety. This isn't like we're being put it in like, you know, little waders or something like that when you have to go into the pool. This is scary stuff. And you should read this book. Because if you read this book, you'll understand how scary it is. And you'll understand, you'll gain a whole newfound respect for the ways in which we are kept on the straight and narrow enclosure. And you know, it's terrifying too. And I mean, it's kind of a very interesting book. And if you want to go down the same path, you think that you need to do this, I strongly recommend that you read this, that you internalize this and that you spend a lot of time really thinking about how you can shoot yourself in the foot and not even realize it. But here's the difference between unsynchronized and volatile. If something is only going to be read and interactively on a single thread, you can use unsynchronized. In every other case, you should use volatile. Unsynchronized is marginally faster. So here's pretty much the simplest possible mutable type, mutable object that we can create. It has a single long counter. And because it's mutable, it's not actually accessible. We can't use the dot accessors or anything like that. So we have to write a getter and we have to write a setter. And we have to use set bang inside of the def type because it's not legal to be used elsewhere. This means that if we want to create some sort of function like thing that wraps over a complicated use of set bang, we have to write a macro. It's ugly, but that's what we have to do. Furthermore, if we wanted to say add a number to our counter, this is one fairly obvious way to do it. We get the value, we add to it, we set it. In my experience, this tends not to inline. And so you get a significant performance boost by bringing the read and the set within the def type. And so what this means is that if you are as sensitive to the performance as we are here sort of artificially, you need to do this. And your mutable data type tends to get a little bit big and has a lot of functions. And again, this is just sort of the price that you pay. So I said that it took a microsecond to associate five elements into a map. The reason I was able to say that with authority is because of a really excellent library by Hugo Duncan, one that I think is like relatively unknown, sort of criminally underused. And that's criteria. Criteria is a benchmarking library. And this allows you to start to understand how pieces of your software behave. And this does all the things it needs to do to control for the JVM. This is not the, oh, I'm going to wrap a time mac around a due times and I'm going to run it two or three times until the JIT seems to stabilize and boom, I know how fast my program is. No, you don't. You need, you should use this, right? And you know, there are some caveats. It tends to exaggerate the cash coherency, which means that the memory reads are going to be kind of minimized as maybe not a completely realistic real world sort of case. And the minimum resolution seems to be around 15 nanoseconds. If things are taking 15 nanoseconds, you may be able to do them 10 times and they'll still take 15 nanoseconds.  But what this gives you is the ability to start talking about performance in the same way that you talk about anything that you use of the Ripple. You start to build it from the bottom up. You start to understand it from the bottom up. And this is a really important way to approach these sorts of problems. You need to take each individual piece of your code and you need to understand how fast it is. And then you need to take them in combination in some sort of common action. You need to understand how fast that is. And this is the only way that you're really going to be able to understand where the inefficiency, where the inefficiencies are being added, right? You need to build up an intuition for how fast things should be and you need to be very sensitive to where your intuition is wrong, right? You need to understand why you couldn't predict how fast a compound action was going to be. One of the reasons that it can be slower is because of the indirection that Clojure uses. There are lots of operators which or functions which under the covers go and do several different things depending on what it's handed. The single equals for instance goes down to, well it also goes down to dot equals but in, you know, the sort of simple case it'll go down to double equals for numbers identical for reference equality. Count goes to a whole bunch of stuff but if you're using an array as you probably will be if you're performing sensitive, the Java laying reflect array dot get length is significantly faster than count. And again I want to emphasize that you know when I say it's much faster, this is something that in any normal program would not make a difference but when we are sensitive to this when we're trying to minimize everything getting one of these things wrong, adding one of these things can double triple, you know, quintuple the amount of time it takes to do something fairly simple. And so you have to be very, very sensitive and you have to be very, very careful about each individual action that you're doing in this sort of interloop. Another form of indirection is primitive math and this is kind of a tricky one because it's really hard to know when your math is primitive. If you don't type int it, even if you use unchecked add and some other sort of stuff, if you don't properly give the compiler all the hints that it needs, it will default to the safest option and it won't tell you. And so when you are adding together two numbers and you want to know, you absolutely got to know that it's efficient, you basically have two options which is to attach a profiler like your kit or use gen class to compile it and decompile it. And neither of these are fun options, right? Neither of these are things that I want to be doing with my day. And again, this is not something I do during the day, but in sort of pursuit of this goal of let's make this as fast as you can, let's write C in closure, which may be an insane goal. This is the sort of lengths that you have to go to. So I wrote a test. It's a benchmark. It plays a random nine by nine game and I want to see how fast that is. And it's about 120 microseconds. And that means that we basically got to it, right? We're playing 10,000 games a second. We could go a little bit further. I haven't actually implemented this yet, but some things that I've been playing around with are we can emulate C style structs, heterogeneous data structures in byte buffers. This is significantly faster for the reason that we have this sort of contiguous memory and we take advantage of this architecture, which is built around the C style accesses. We can also go one step further. And one of the reasons that people don't do this in Java very often is that if we, for instance, want to get the nth element and we want to get a field and then a subfield, we have to create intermediate data structures that represent each of these intermediate steps. In C, that's not the case. In C, what it'll do is it'll look at it and it'll say, oh, you're going to this index and you're getting this field and all it'll do is it'll just do a couple multiplies, a few additions, and then it'll read from that position of memory. And this is unsafe for all the obvious reasons. But we can potentially do local analysis in closure, assuming that we have the right sort of information and everything like that. And we can do this safely as long as we have the fallback of all these intermediate data structures. And so this is actually kind of interesting for a number of reasons, one of which I'll talk about. This was alluded to in the last talk. Co-processors are coming back, right? We have GPUs. We have the PS3 cell processor. We have instructions on the x86 chips that we don't have access to, vector instructions and everything like that. And these are not things that we can really talk to very well. This is not something that's really within our purview. But what we can do is we can use something like OpenCL, which allows us to create a C like description of a highly parallel task. And it gets compiled at runtime. And it gets sent across the wire. And then we can send it data. And it will work on it. And this task, actually, in many ways, is a pure function. It takes data. It returns new data. Typically, it is, many of these are used in composition with each other. And whereas C, which is sort of what a lot of these APIs are written in, is a very good way to represent the tasks themselves, it makes for a very poor way to choreograph, to compose. And while closure is not uniquely qualified to do that sort of choreography, it's well positioned, right? This is sort of something that we do very well. We compose things. We talk about things that don't have side effects. We talk about many things being used in concert with each other. And since we're doing runtime compilation, we're not really limited by the performance per se. We're limited by our expressiveness. And so closure can be an expressive choreographer of other computing devices. And this is hypothetical. This is not something that has been realized. But I think it's an interesting sort of niche that closure could fill, which hasn't really seen a lot of attention. And so I think that this is interesting. So with that, I wrote a Go AI that's called Pushkin. It's kind of strong, not really. What it can do is it can play against GNUGO, and it can win sometimes.  It's not completely brain dead. But it doesn't play particularly deep moves either. It tends to get tripped up on particular life and death exchanges. I think that a human player realizing this vulnerability could probably never lose to it. But this is because I've only taken the sort of first step here. There are two spectrums upon which I can improve. Right now, it's playing very naive Monte Carlo play-outs. I can make it stronger. I can get it closer from possible to plausible. I could also take the multi-arm bandit mechanism and I could try to seed it with something that is my best guess as to the best move. Remember that we're converging on the best solution. So the faster that we can converge, the more we can exploit as opposed to explore. And so these are things that remain to be done. This is something that I think is a really fun sandbox for me to play in for other people to explore. And I want to emphasize that this is not a well-defined problem space or a well-explored one, rather. There have been a couple of people that have written things. Typically, they, as I did, stayed very close to the literature. And at the same time, you know, they write and see and they put all this engineering effort into it. There's not a lot of motivation for them to go wild. And there are performance constraints and, you know, on some level it is just more painful to write code this way. But I think that with the right libraries, with the right approaches, with the right sort of best practices, closure could be a really effective tool for exploring these sorts of problems. And, you know, if anyone is interested in this, I encourage you to come talk to me afterwards. And, you know, I will talk a lot more about this. I guarantee you. With that, I'll also say that, you know, as I said, I work for Factual. There are, I believe, eight or nine of us here at this conference. We do not do anything related to go. But we do handle some pretty interesting data analysis and systems engineering problems. Closure is heavily featured. And if you want to hear more about that, come talk to me. Come talk to any of us.  And with that, I'll leave the floor open to questions.


Gun Smoke, brought to you by L&M Filters. This is it. L&M is best. Stands out from all the red. A round-dog city and in the territory on west, there's just one way to handle the killers and the spoilers, and that's with the U.S. Marshal and the smell of Gun Smoke. Gun Smoke, starring William Madd. A trans-crog story of the violence that moved west with young America. And the story of a man who moved with it. I'm that man, Matt Dillon, United States Marshal. The first man they look for and the last they want to meet. It's a chancey job, and it makes a man watchful. And a little lonely. I've been on longer, Mr. Dillon, but I can't remember any no-dryer. Yeah, there's probably water in those cottonwoods over there, Justice. They'd better be. Of course, I haven't been by here in maybe a year. Oh, my, now don't say that, Mr. Dillon. Of course there's two of them. Oh, sure, they wouldn't be in there otherwise, would they? No, not likely. Those aren't range horses, either. No, sir, they sure ain't. Oh, that was the most beautiful animals I ever saw. Hey, there's a man with him. Yeah. He's gonna saddle up his awful well decked out for a cowboy, ain't he? Sure looks better than we do. Come on, I ain't shaved in a week. How have you been dodged tomorrow, Justice? Horses got blood in them, pie ever seen it. So why? And I'll fight anybody for these horses. Anybody at all. Now look, Mr., we aren't after your horses. We're only admiring them. Now tell me something.  I said, did you ever shoot a man in the back? Mr., but I never shot anybody in the back. I believe you. I believe you, didn't I?  Come on, we better hurry. No, Chester. But he'll get away. We can't stop him, not on those tired animals of ours, and him with a pair of thoroughbreds. Well, we can track him. He's a sure enough horse thief. He's headed straight for Dodge, Chester. We'll find him there.  It stands out from all the rest. No other cigarette gives you L&M's assurance. Assurance that it is best. L&M's got everything. Superior filter, superior taste, superior filtration because of L&M's superior filter. White. All white. Pure white. The purest tip that ever touched your lips. Superior taste because of L&M's superior tobaccos. Tasty. Full of flavor. And light and mild. Buy L&M Today. America's best filter tip cigarette. This is it. L&M filters. L&M's got everything. It's the best. You wrote in yesterday. Sure it's the same, too. Oh, I didn't know them anywhere. Besides that wine glass brand stand down like a pulled-gatter picnic. Okay, Chester. You keep an eye on the street here. I'll start with a long branch. See you later. You've been back. Just long enough to get cleaned up a little bit. Have you been? No, you didn't come here to ask me that, did you? Well, not entirely, no. Something going on, ma'am? Kitty, I'm looking for a man who came to Dodge yesterday with a pair of thoroughbreds. He's tall and he wears about the finest clothes that have been seen around here since Bill Hickock was last in town. Uh-huh. His name's Portis, ma'am. Jack Portis. Ah, I thought you might know him. The only man who ever came to Dodge as I didn't meet never quite got here. What? I'm sorry, I lost your trail there, Kitty. I'm thinking of that cowboy who got shot just outside of town last year. Oh, you remember? Yeah, yeah, I remember. I'm more interested in Jack Portis, so... Oh, well, I don't know much about him, except that he's a real free spender and he's already made himself a lot of friends. Nobody can pay for a drink when he's around. He got money, too, huh? Well, besides those horses, you mean. I hear they're pretty fancy animals, ma'am. Yeah, real fancy, Kitty. Well, nothing's too good for Portis. Oh, ma'am? Yeah, I see you. How are you, Kitty? Portis? I see nobody's shot you in the back yet, Portis. No. Anybody got a reason to? You know my name, but I don't know yours, mister. It's Dylan. Marshall, Dylan. Hm, I'll be darned. Why don't you say so out there? You were in too big a hurry for much visit. The way you two men looked, I wasn't taking chances.  Did you shoot somebody for them? So that's what you're thinking. Where'd you get them, Portis? I raised them. Where? I want to spread on the Wachita River, Marshall. Oklahoma Territory? That's right. That's a long way from here. Oh, I ride off every once in a while.  No harm in that. Sure.  Of course it is. Look here. Got a burn right on my hat band. See? Uh-huh. You believe me now? It could be true. You're about the most suspicious man I ever met. Well, I'm paid to be. Sure. I understand. Buy you a drink now? Some other time, huh? It'll be my pleasure. Come on, Marshall. I'll see you later, Kitty. Sure. Don't you believe in that? Yeah, I guess I do, Kitty. Oh, but you're not real sure, huh? It could be that he's just a whole lot smarter than I am. Hello, Chester. And Matt. Come on, sit down, Doc. Sit down. We just finished, but you can have our table. Oh, thank you, thank you. Well, what'd you eat? We had the chicken, but stay away from it, Doc. Oh, is that so, Chester? It's no good. It tastes like it'd been boiled with the feathers on. Oh, well, I never heard of cooking chicken that way. See, that might be worth trying. All right, you don't need me. Go right ahead. No, no, no, no, no, no, Chester. When it comes to food, you can be trusted completely. Uh-oh. Who's having the party of the day? That's Jack Porter's, as usual. Porter? Oh, that fellow. Even at noon, huh? Well, he hasn't left for company one second of it. Oh, man, he spends money the way he does, picks up friends like a dog does fleas. Well, he's bound to run out of money one of these days, Doc. Yes, I hear he hasn't gone into a saloon yet without buying drinks for the house. You know, a man like that could maybe be president someday. Oh, now, Chester, you don't seem to hold the highest office in the land in much esteem, Chester. Well, now, I like Porter's, Doc. He's a fine fellow. I was talking about how you think a man becomes president. Well, getting folks to like him is one way, ain't it? Well, I won't argue with that. No, no. Well, I can't sit here all day and listen to you two. It's not likely you'd learn anything worthwhile if you didn't. I'd better go, too, I guess. Thanks for your advice about the chicken, Chester. Oh, that's okay, Doctor. So long. So long, Doc. If you haven't got nothing for me to do, I might go on down to Depot later on. Oh, well, you can go now, if you like. Oh, it's too early. The sign if he ain't due in till mid-afternoon. Oh, I see. Well, go then. I wouldn't want a train to arrive without you being there. Great idea. Well, now, it ain't a pure waste. You never know who might get off one these days. Pretty girl, maybe, huh? Well, now, I'm serious, Mr. Dunn. Yeah, I know. Well, now, but what I mean is, hey, look, there's him thoroughbred across the street there. Oh, yeah. Who's the boy leading him? Myron Marweedle. He's a kid Portis hired to give him a run once in a while. Well, you can't say he doesn't take good care of him. No, sir. There's Portis coming now. He's going to show him off to his friends. He's got a right to be proud of him. I wished I was rich like him. I hear he's got two rooms at the Dodge House just in case he runs into somebody who needs a bed. Now, there's a real friend. Hey, look at him rare. Yeah, that's all horse. I seen him, I tell you. Don't yell about it. I believe it. Well, I want you to know it, sir. Well, you sure take up a lot of room. Oh, I'm sorry. Don't start trouble, Bert. I'm starting no trouble. He was standing in the way. The man's standing in the way. He's got a right to move him, ain't he? Is that fellow drunk or just owing me? I don't know. I've seen him around here before, Chester. No strangers to me. From the looks of them, I'd just soon keep it that way. Well, you've got your choice. What? They're getting mounted. Looks like they're moving on. Hey, those are pretty fair horses. They got two. Hey, Chester. Yes, sir? Do you see those brands? One of them's wineglass horses, too. They don't hear you. All right, let's go get our horses. We'll follow them. What about Portis? We ought to tell him. They didn't tell him. Come on. It's a camp they got there, Mr. Dillon. They ain't trying to run. No. They'll go on if I can figure it. Now, they've seen us. OK, we'll ride right up to them, Chester. Yes, sir. My name's Keller. Not that I take the strangers asking them. Now, there's no need for us to be strangers. If it's Chester Proudfoot, I'm Matt Dillon. What are you doing down here, Marshal? What'd you follow us for? I'm interested in your horses. Why? There's a man in dodge with two wineglass horses. Yeah, we know that. Keep talking, Burke. Go ahead, tell him. All right. Stole some money, too. On the Red River. Oklahoma Territory? Sure. Who's that? Uncle's. He killed him, Marshal. Yeah. He killed Burke's uncle, and he'd run off with what money he could find an M2-3rd breast. And we followed him here. A long trail, but we found him. I see. Now, what do you aim to do now? Aim to take him. What else? Now, what are you waiting for? We're figuring a way to do it. Yeah. The smart one, that Fordy. Why didn't you come to me about this? Well, ordinarily, we would, Ab, Marshal. What do you mean?  Now, Keller told you. He killed my uncle, Marshal. That makes it different. Not to me, it doesn't. Well, it ain't your kin he killed. You stay here, Burke. Both of you stay here. You can ride back and see Portis in a half an hour. He'll be at the jail. No other cigarette gives you Ellen M's assurance. Assurance that it is best. Ellen M gives you superior filtration because of its superior filter. Superior taste because of Ellen M's superior tobaccos. Yes, Ellen M tobaccos are tasty, full of flavor, and light and mild. And Ellen M's superior filter is the purest tip that ever touched your lips. It's white, all white. Truly the miracle tip because when it's added to Ellen M's superior tobacco, it actually tones up the taste. Actually improves your enjoyment of this great cigarette. No other cigarette gives you Ellen M's assurance. Assurance that it is best. Ellen M's got everything. Superior taste, superior tobacco, superior filter. That's why it's America's best filter tip cigarette. Try Ellen M today. I will go down to the long branch next to her. Sure going to be a surprise when he finds out his game is up, ain't it? Hey, what's that crab doing down there? I don't know. Uh-oh. They're spreading out. Must be a fight, Mr. Jones. Yeah, come on. Work, see him? There's Keller, too. Looks like they killed him. Yeah, right. Stay back, Justin. All right, let's leave it like that. It's Marshall. Yeah, stand back. It's yours. Good job, Marshall. First, Marshall, it was self-defense. I didn't see it. Asked anybody it dids. They may be friends of his. They're live. They weren't friends of his a minute ago. They didn't back him up, did they? Well, it's different now. They'll back him up now. You're in back... Mr. Jones. Yeah, watch this. What are you doing here, kid? Hello, Matt. She's seen the whole thing. She was standing right over there, Mr. Dillon. And she says... All right, all right, Justin. Let's hear it from her. Yes, sir. Oh, wait a minute. How do we know she ain't a friend of his, too? She'd tell the truth even if she was. All right, go ahead, kid. What happened? Well, I didn't hear what was said, Matt. But I saw what happened plain enough. It looked like he was calling Portis out. And then Portis went for his gun. Burke shot him. And it was self-defense. I told you that, Marshall. I wasn't lying. There wouldn't have been a killer if he had stayed in camp like I told you. Well, the murderer died sooner or later.  Then you stay there. I'm taking them horses with me. Take them. Come on, Burke. Let's go. Portis was a bad one after all, huh? I guess so, kiddie. If he hadn't been guilty, there'd have been no reason for him to draw up first. He didn't seem bad. Hell, you never know, kiddie. You just never know. Had Chester go through Portis' pockets. And he found he hardly had enough money left to pay for a coffin. So his big party had been about over anyway. But at least he'd had a fine time while it lasted. And everybody who'd enjoyed it with him felt even though he was a killer, it was better he got shot than hung. And that was the end of it. Except for one thing. And we didn't find out about that until a week later. Goodness, it is just plain miserable out there, Mr. Dillon. Oh, what is it, Chester? The sun, the dust, and the heat. So why don't you stay inside? I'm going to, now. Where have you been? After the mail. Here. There's not much here. Now, wait a minute, what's this? Oh, uh, well, that's for Jack Portis. There ain't no address on it where they could send it back to, so I said maybe I'd ought to bring it to you. Well, I don't know what to do with it. It might say inside where you could send it back to. I don't like opening that mail, Chester. Especially without a dead man. Well, what we're going to do with it then? And if whoever wrote this doesn't know he's dead. And I guess it's up to me to tell him. I'll open it. What's the matter, Mr. Dillon? Dear boss, I hope you've run out of money by now and are on your way back to the wine glass. Things is fine here except the wash-a-tahs near dry and a couple of horses got stole. But they was not thoroughbreds. I am keeping two pots of coffee on the stove in case you show up unexpected. Do be my gunner. Have I made a bad mistake, Chester? Yes, sir. But Portis made one too. How? Draw and first made him look guilty to us. But what happened was that he counted on his friends to back him up against those two. That Birkenkeller. They're mighty bold the way they rode in here and got by with that. We'll get a circular out on them right away. Yes, sir. All right, his man, Dillon. What are you telling, Mr. Dillon? And I'll tell him two more wine glass horses got stolen. Thoroughbreds this time. And now our star, William Conrad. Thank you, George. You know what I like about Ellen M's is they're mild and mighty easy on the draw. When you get right down to it, no filter stacks up with Ellen M's pure white miracle tip for quality or effectiveness. Gun smoke. See for yourself. Ellen M stands out from all the rest. Directed by Norman McDonald, stars William Conrad as Matt Dillon, U.S. Marshal. Our story was specially written for gun smoke by John Messon, with music composed and conducted by Rex Corey. Sound Flatterns by Bill James and Ray Kemper. Featured in the cast were John Daener, Lawrence Dobkin and Harry Bartel. Harley Bear is Chester. Doc and Georgia Ellis is Kitty. What are you doing to help your children's education? Do not wait for the other fellow. Get in there and join your local organizations fighting for better schools. Maybe a year from now, you'll have done something to help. Ask your friends and neighbors to join in the fight for the future. The smile with Chester Field. Smooth and cooler, milder Chester Field. Put a smile in your smoking just to give them a try. Chester Field's best for you. You want tomorrow's better cigarette today. Next time you buy cigarettes, stop. Remember only Chester Field is made the modern way with Accure. You'll notice how fresh and good Chester Field's made with Accure taste, how smooth they are, and how they satisfy. So buy Chester Field today. smoother, cooler, best for you. Get on your local CBS television station. Remember Gunsmoke on TV tonight, 10 p.m. eastern time. And be sure and listen to Gunsmoke again on radio next week transcribed for L&M Filters.


Welcome to another episode of brutal battle beer break. I'm carlin your host for this Um, just let you know what's going on. Just got finished a lawn mowing out there and hopefully that's the last one of 2014 Hoping but what better way to you know, give myself a little bit of a treat than to Try a three-way. Oh wait No, no Not that kind of podcast. Don't you worry brutal bottle battle is not that kind of podcast Anyway, I'm talking about three way IPA it's a special release beer It was a summer seasonal, but it was this one was canned back in July. So we're still within Good timeframe IPA is probably want to enjoy within like 90 days of their bottling or canning in this case But anyway, this one is called three way. You can also see there's three guys wailing on Right just axes as some people might say And this is by three breweries out of Oregon worked on this one together Fort George, which I don't know anything about Block 15, which I've had their sticky hands Imperial IPA, which is really nice and Boneyard, which I haven't had any other stuff, but I heard great things about like their hop venom and their RPM. So Got high hopes for this beer. This also was one. Thank you very much to Dawn bowling She mulled it back from her trip to Oregon. So I wouldn't be Enjoying hopefully enjoying this beer without her help. So thank you So let's go ahead and crack this sucker Glad it's in a can of my beers and cans It's more eco-friendly and all that jazz All right Little bit decent head to it there. So you can see three way Very orange it does say on the can that it's It's not filtered. So there is going to be some sediment in it Although it's been sitting upright for a while in my fridge. So a lot of sediments pry at the bottom Which is fine. It's also 7.2% alcohol by vine. You can see pretty orange a little hazy You can see that there is some sediment kind of floating around in it That is how I love my IPA is to smell.   I mean I get grapefruit out of the wazoo on this Tangerine grapefruit and I'm gonna be honest I don't get a whole lot of bitterness like sometimes it kind of smells like some citrus rind I'm not really getting much of any citrus rind type smell. I'm mainly just getting like grapefruit fruit That is the thing that is at the forefront of this beer. God, it smells so good Mmm, man West Coast IPA to a tee with the way it smells. Mmm.  It's so hoppy It's like it's so much hops to it. I feel like this kind of like tingling on my tongue It's like all of my my taste buds are just kind of like standing at attention as soon as the beer hits it Really nice the grapefruit But you are getting like a nice bitterness at the end That's kind of like the grapefruit fruit upfront and then it gives way to kind of like grapefruit rind type flavor, but I mean It smells amazing, but it's it's grapefruit kind of all the way through to me Mmm The mouth feel to it is pretty much what you would expect actually may be a little bit more Watery than I would expect from how you know hoppy and thick it kind of smells to me But you know totally fine. Nothing really complained about on this one. So awesome Let me see if there's any sort of extra information on this one. No It just kind of says who does it it does say make sure that you don't you know age it That there is sediment in there keep it refrigerated. Yeah, never never never age or hoppy beers and like I said within Mac, you know the best time is like within 90 days of like the canning or bottling or some breweries will have a Best buy gate, which is really nice. I know six point does that a lot. So good job for six point in this case block 15 Boneyard and Fort George. I believe it's Fort George. Yes for George. Sorry for George Great job with this beer. I look forward to having Hopefully trying some more Fort George stuff in the future I'd love to get some boneyard stuff and I actually have another sticky hands in the basement by block 15 very excited by that Don bowling. Thank you again the block 15 is from her as well. So thank you so much everyone Stay tuned for the next one who knows what beer I'm gonna have next time


Welcome back. Jeff Frick here on the ground at the Sheridan Palace in San Francisco. We're at the BrightEd Share 14 conference where they're talking all about SEO, optimization, and content marketing. I'm joined here by Lonnie Stark, Director of Product Marketing from Adobe and a partner at BrightEd. So thanks for stopping by. Oh, thanks for inviting me, Jeff. So you're doing a panel later today. So for the folks that didn't make it to the conference, it'll be at 2015. What are you talking about in the panel? We'll be specifically talking about the evolution of the customer journey and how content needs to change to support that. So as people get more devices and they're using different digital experiences, how does that really change behavior and what do companies need to do about that? So Adobe's been at the front edge of content and content generation and pretty fonts all the way back to forever. So how is that journey changing now that we've got our phones, we've got our iPads, our 7-inch pad, our 10-inch pad, our laptop, our home computer? How is that really affecting content marketing? And then the other piece of it is as there's more kind of user-generated content trying to get integrated into the program, how does this all come in together? Yeah, great question. So we've been involved with content both on the creative cloud and the marketing cloud side. And one of the things I think that's changed is brands are really struggling with how do I get relevant content out to all of these different devices. So really thinking about how you do bite-sized pieces of content, like the same message, but you might have an Instagram, you might want to share something on social channels, you may want to have deep reports. And so that's really important. So the mobile supporting that mobile piece of it. And then another I think is the speed. So we've spent a lot of time working in partnership with Bright Edge thinking about how do we really compress the time needed to go from an idea to creating it to actually delivering throughout the channels and then seeing the analytics, seeing what's really actually resonating and being able to really target that to the right audiences. Now when you break stuff up into the snackable bites as we like to call them. Yeah, I call them snackable too. You like that too? We like that. We'll break this interview into snackable bites. Yeah, they want to call like little bugs. Bugs nobody likes, but snacks everybody likes. Everybody likes snacks, especially if they're tasty ones. Yes. But is there an expectation of a different type of content on the different device? We talked to a little while ago and they said, you know, they like to use their Instagram for kind of showing behind the scenes. So they've got a very different kind of content agenda, if you will, based on the medium that they're using. I wonder if you could share some best practices or some of your insights there. Yeah, absolutely. So there are times when some of the brands we talked to wanted to do responsive design, right, for being able to just share content across different devices. So that works at least as a foundation. The second is thinking more about these snackable bites. So with Adobe, we also do a lot of content marketing ourselves. Right. So we have something called the Adobe Digital Index, which is a report we pulled out, you know, a couple pages, you got to sit down, you have to read it, right? But we also generate a lot of Instagrams or infographics of key imp sites that get shared. And so those might get shared on social channels. People see if they're interested, they'll actually go and click and find out more. And I think a great example, which is not an Adobe example, but out there is the ALS challenge. Right, right. Right.  I have not. I will donate. Do not challenge me. I will donate. But that's like the last person. I think there's like four people that haven't done it yet. Have you done it yet? Yes, of course. So, sorry to interrupt. But no, the ALS I think is a great example where it's actually very serious condition. And there's a lot of things written about it, but it was really hard to drive awareness around it. And now with something like these videos that people are posting, which are user generated, all these different channels, like I actually went on to ALS.org to find out more and actually read something much deeper that I wouldn't have otherwise read. So I think brands need to really think about how to address people that may not even be aware with something that's bite-sized, that is relevant, but quick. Right. And then also have that deep content as well. And I've seen that with publishers as well. So publishers talk about long form, which are like 20-page articles. And they're great. Like a weekend, you really get into it. The Vanity Fair version. Yeah. Right. You get into the meat, right? But sometimes you need the quick tweets and the quick things that will get people even interested in making that kind of commitment. So talk a little bit about the user generated stuff and how brands do incorporate that in, because it is interesting. And then too, I don't know if you've got any insight on virality of why some of these things just go bananas and others don't. And it's not necessarily obvious kind of what the rhyme or reason is. Yeah. I have a couple of thoughts on why things go become much more shareable. I think one is there's an element of fun, and people want things that they think are, humor goes a long way. The second, I think, is brand association. So if it's something that will make someone feel like, oh, if I share this, it reflects well of me as well, whether it is a cause they care about, whether it is something that makes them sound more intelligent, things of that sort get shared. And I think some things are useful. So most people, when you, your social community are of people who are friends, colleagues, people you know, so you don't really want to spam them, right? So you're only sharing things that are actually amusing, useful, or say something about yourself. As far as user generated content, how brands are using it, I think more and more brands, and I've watched Coca-Cola talk about this as well, right? Where a lot of brands in the past have been, hey, this is our brand, it's protected, and we are going to, you know, broadcast out to you our message. And that hasn't worked as well now, and as in needing to really incorporate user generated content to support the brand. And it's not just amplifying the brand message, but also getting commitment from your customers. Because if they feel like they're contributing to the meaning of the brand, they become a lot more committed as well. Yeah, it's almost like, as a measure, you're waiting for people to take your content and really user generate it, you know, remix it, and you always think of like logos. How hard it used to be to get a logo in company. Now they're all over the place, right? You just go on there and find it. Well, thanks for stopping by Lonnie Stark, director of product marketing from Adobe. Appreciate your insight and enjoy the rest of the show. Thank you, Jeff. So I'm Jeff Frick here on the ground at the Sheridan Palace at the Bright Edge Share 14. Be right back.


Okay, so let's do this borohydration problem, so it asks what are the products of this reaction? So this is borohydrate THF, the THF part is just a solvent, okay? So that's not going to be participating in the reaction. So you can think of, in fact I'm going to erase all of this stuff here, so you can think of the active ingredient as just this BH3 here. So BH3 remember that the boron is less electronegative than the hydrodynamic, so it's going to add anti-marcogniton, okay? So the nucleophile is actually going to be the double bond here. So the boron is going to add to the side with the more hydrogens, okay? So which side would it add to? This, the right or the left? Right, because it's got more hydrogens. These are metal groups, right? Okay, so that's all right. But you know, right? Okay, so like that, and then that's going to all at one step, okay? So it's a concerted reaction, and it gives you, so it's concerted, and it gives you syn-addition, anti. So when we do that, hydrogen up there, right? So if the hydrogen's up, that means the BH2 has to be up too, okay? So I'll just draw that extra hydrogen. So what happens, this BH2 actually becomes BR2 by adding two more of these alkenes onto it, okay? So in fact, we could write, so we could say another alkene comes, another alkene comes, right? And those guys add to that BH2 to make it. So it's going to look exactly the same, but it's going to be BR2 now, okay? So what we can do is maybe in another video show what that actual complex looks like. It's very big, though, so it's hard to show on kind of a small board, okay? So then, well, that's the main step of this problem. In our book, it doesn't want you to know how to do the second step of the reaction. So remember, this gives you anti-Markovnikov addition of alcohol, okay? So you get an alcohol. So the second step, you just need to know the conditions. Hydrogen peroxide and hydroxide ions, so strong basic conditions. And that's going to give you the final product. We can go over that second step in a different video. So that would be your product, that secondary alcohol for this reaction. And we can go over that second step, but for the test, you all need to know, okay? Questions on that one? No? It makes sense. Okay.


Perfect. Well welcome back everybody and everybody who's on Facebook. We're going into our second part of today's OpenShift Commons gathering and we're going to kick it off with the state of the container ecosystem with two of my fellow Red Hatters. Many of you have heard of them, Dan Walsh and Monroe Patel and we're really pleased to have you all back and we're going to try and stay on time today so I'm going to let them get started so thank you all. Okay. Can everybody hear me? Okay my name is Dan Walsh. I run the container team at Red Hat. I have, I now work in the OpenShift division. I used to work in the Rel division up to about a month ago. So we started a, one of the things at the low level of containers that I've been fairly depressed with is how little advancement we've made over the last few years in containers mainly because there's always, you know, there's only one, people think that there's only one way to run containers and containers, actually there's a few t-shirts running around here called containers as Linux and Linux as containers on the shirt. And containers is just a Linux concept. It doesn't have to go through one, you know, just one way of doing containers. So what we started working on about a year and a half ago was trying to break apart what it meant to run a container or how you want to run a container and really, so what do you need to do when you want to run a container on your system? So the first thing you have to have a definition of what a container is or what, you know, what the content of a container is. And this is actually the biggest contribution the Docker made to the ecosystem is basically they got everybody to standardize on this, this one way of bundling up an image or a group of software they're going to install in your environment. And luckily over the last couple of years there's been a standardization effort on that bundle to make sure that everybody agreed to continue to use that bundle. One of the things I've always feared since I started working with containers is that we'd have a bifurcation. And actually it started to happen about three years ago, Coro West decided to, they wanted to standardize on what they call the App-C spec. And they wanted to standardize on an image bundle that was different than what Docker was doing. And so we began to see sort of the RPM versus Debian thing happening all over again. So people would have to pack it software in different ways. Luckily all the major players in the container world got together and they worked on a standard and that actually went 1.0 last December and it's the OCI image bundle spec 1.0. So when I want to run a container I have to be able to identify the container, the container sits in the container registry. And everybody pretty much, the funny thing is in the competitive business of containers, container registry is really where everybody competes, right? There's hundreds of different people doing containers. Each one of the big cloud vendors does a container registry, Red Hat, we have an OpenShift registry, there's obviously Docker IO, there's Quay from Coro West, it's out there. So there's lots and lots of competitors at the container storage. But basically they all store the same thing, these image bundles. So the next thing I need to do to run the containers, I need to be able to pull the container image from the registry to my host, okay? Can everybody tell me how they can do that? And everybody in this room is going to say Docker pull. There's only one way to do that, Docker pull. In the whole world there's only one way to do it, Docker pull. That sucks, right? What is the container registry? It's a web front end, it's a web service, right? I should be able to do it with Curl, right? I mean it should be able to do it with any tool to pull off of these images off of bundles. So we started working a few years ago on a tool called Skopio, we're going to talk about at the end. But Skopio ended up evolving into this thing called Container's Image. So we actually built that Go library called Container's Image, if you go to GitHub Container's Image, you'll find it. Lots and lots of people contribute to it. And now we have ways of moving images from container registries to other container registers, from container registry into local storage, different types of things. So we needed a standard wave of implementing pulling and pushing, pulling and eventually pushing images. And that's what container images. The next thing you need to do to run a container is actually take that bundle that you pulled down using Container's Image and now you need to explode it on disk. But you have to put it on a sort of a different type of disk, it's called a copy on right file system. Okay, if you think about what when we have these image bundles, they're layers, right? We have a first layer, the second layer, the third layer. So these are layered file systems that you have to put it on top of. In these layers, eventually you'll get to a writable layer. So it's copy on right, which means I can write, I feel like I'm writing to the layer, but I'm actually writing to a different place. Copy and write file systems, the things like overlay, device mapper, or butterfs has a version. So there's lots and lots of copy and write file systems. And the only one that had a copy and write file system was inside a Docker. So the only place to store it was inside a Docker storage. So we decided to create a library called Container's Storage. Right, we took basically took the code and most of it was written by Red Hat originally, that was in containers in Docker, pulled it out and made it into a library so people could start storing copy and write file systems on the disk. So lastly, you need a standard mechanism for running. How do you define what it means to run a container? Okay, and that really has to be standardized just like the bundle. So you need to standard what it means to run a container and what it means to store the container. Luckily, OCI runtime spec. Okay, also 1.0. What that does is it has a JSON file that basically says what's going to happen in this container? What the environmental variables are? What kind of security constraints are on it? What is the entry point? What is the current working directory? All those things are written inside of a JSON file. And then you have an exploded file system next to it, which is a root of s. Okay, run C is the default implementation of the OCI image spec. Clare Linux containers is another implementation of the OCI image spec. Clare Linux happens to use KBM for isolation. Run C uses namespaces for namespaces and C groups on the local file system. Other people are building, also building OCI image specs. Dockerson's 1.11 has been exacting run C underneath the covers, and so all containers are launched with run C. So if I have these four components, right, nowhere in these components do I need a big fat demon. Okay? If I have four components, I should be able to do these things from the command line, right? I should be able to do each one of these steps from the command line without having a big demon in the way, right? And one thing that drives me crazy in the container world is people keep on putting up demons. You need a demon for everything. Everything's a client-server operation, but in a lot of ways I would just like to have it, you know, a lot simpler. Basically, you know, these are just processes on a Linux system. I should be able to just exec them. So you're at the OpenShift conference, Kubernetes, KubeCon. This week, we wanted to talk about Kubernetes. So what does Kubernetes need to do to run a container? Well, first of all, if you look back in Kubernetes, it was originally developed on top of Docker, okay? They built into Kubernetes the entire Docker API, talked to, you know, how to talk out the containers. And here again, CoreOS actually caused some problems. Good problems. CoreOS came along and said, we're going to write a huge amount of patches to Kubernetes to make Kubernetes work with Rocket. And Kubernetes basically looked at this huge amount of patches that they were going to have if that, if, you know, I'm using C code, but basically say if then, if running with Docker do it this way, if running with Rocket do it this way. And they said, well, we can't support that kind of code. So what they said, what the Kubernetes guy said at that point is we're going to define our own interface and then allow anybody to build a container runtime for that interface. And that's called container CRI or container runtime interface. So, so what they went back to Rocket and said, you guys take the rocket, build a rocket demon that implements the CRI implements our protocol, and we will gladly talk to you. At that point, Docker, they started internally to create what's called the darker shim. Darker shim is basically putting all of Docker CLI calls behind a CRI. So now Kubernetes now talks to this this protocol, the CRI protocol, and you can support multiple different container runtimes. So a year and half ago, back in September, Ronald up here, one of my best engineers came along and we said, let's do a Skunkworks project. Let's see if we could build, using those four components that we built, let's see if we could build a little tiny demon, not a big fat demon, hopefully a thin demon, that would implement those four things. Build a store in image somewhere, pull the image, store on disk and then create a run C configuration, very similar to what Docker did, and that's what we call, well actually originally we call it something different. This press got a little bit, and all of a sudden it became RedHouse walking Docker. That's not what we did. What we wanted to do was implement a workflow for Kubernetes running. So we ended up calling it cryo. So I'll let you take a So what is cryo? So from the name, you can see it's an OCI-based implementation of the Kubernetes CRI. So we took all the components that Dan talked about and created cryo. So what is the scope of cryo? Exactly what Kubernetes CRI needs. We don't add any more code than what the CRI needs. So it's like nothing more, nothing less. Just implement the CRI in each version of Kube whenever there are some changes to the CRI. We pick up those changes and we implement them in cryo. Only supported user is Kubernetes. We don't serve to support any other demons or any other orchestration tools. And we try to use standard components wherever possible to implement cryo. So in addition to the components that we already went over, these are some other components that we use for cryo. So the first of these is the OCI runtime tools. So Dan talked about RunC and how it needs a config.json. So the OCI runtime tools has a library for generating those configurations. And it's a project under open containers. So anyone who wants to use that library is free to do so. And it keeps in sync with RunC. So we use that to just generate config.json for running the containers in cryo. And then for networking, we ended up using CNI. It has kind of become the default networking solution everywhere. All companies that provide container-based networking solutions have a CNI plugin. We have tested it with a bunch of popular plugins like Flannel, Veeve, OpenShift, SDN, and Calico. And all of them just work. And finally, last but not the least, is Konmon. So Konmon is like a monitoring process. It monitors each container. It's a small, tiny binary that we wrote and see to be efficient, so it doesn't use a lot of memory or CPU. So it monitors the container for exit codes. It handles the logging. So CRI defines a format that each container runtime is expected to write out the logs in. And so Konmon is a component in cryo that does that for us. And then it also handles TTY. So whenever you want interactive terminals, Konmon is responsible for reading the master PTY from the container and copying data back and forth. It serves the attached clients. And finally, it detects and reports Oom. So whenever you do QT status, and if your container went out of memory, you'll be able to see that. So let's take a look at what a pod looks like with RunC. So you have a pod, which is the holder of the C groups, IPC net, and optionally the name spaces and newer versions of Kubernetes. And then within a pod, you have the infra container, and then the actual application containers that are specified in the pod specification. And for each one of these containers, we run Konmon. Konmon is small and efficient. And it uses C and shed library, so it doesn't have a lot of memory overhead. So this is the overall architecture when using Kubelet with cryo. So on the left, you see Kubelet. And it's talking over gRPC APIs. CRI is basically a gRPC API. And it has two different services, the image service and the runtime service. The image service is responsible for listing the images available locally and also pulling images. So whenever you, in a pod spec, you specify some image, Kubelet uses the image service to make sure that the image is present locally. If it's not, it makes a call to the pull API, which then cryo implements pull using the containers image library that we mentioned earlier. And then for the runtime service, we use the OCI Generate library for generating the config.json. We use CNI for hooking up networking for the container. And finally, we use the storage library for creating the root file system for the container. So you have cryo running on the right and then it's launching pods, depending on what the Kubelet requested to do with the CRI API. So cryo allows Kubernetes. What do we mean by that? So each pull request that goes into cryo passes all the Kubernetes tests. We don't merge any pull request if it breaks any Kubernetes tests ever. So we run more than 300 tests for each pull request before it gets merged into cryo. So again, in the theme of only supported user as Kubernetes, so never break Kubernetes. So what are the versions of cryo that are out there? So the first version was a 1.0, and the latest release there was 107. And we wanted a 1.0, and that corresponded to cube 1.7. But in keeping with the theme that we want to be tied to Kubernetes, after that we jumped our versioning to match cube versioning. So each version of cryo after that is easy to match up with cube. Like cryo 18x supports cube 18x. Cryo 19 beta was released last week. And once cube 19 is out, cryo 19 will be out as well. And what about OpenShift? So cryo was shipped as a tech preview in OpenShift 3.7 on rel. So OpenShift container platform will be moving to 3.9 afterwards, but Origin still has a 3.8 if you want to use that with cryo 18. And we'll be targeting that for OpenShift online to deploy cryo to OpenShift online as a first step. And again, OpenShift 3.9 will have full support for cryo as a runtime. And our goal is for 3.10 to fully support cryo as a default option in OpenShift. Can anybody tell me what version that will be of cryo with Kubernetes 1.10? Yeah. We're stalking Kubernetes. There's no confusion. We just picked the matching version, and it should work with cube.  And so I'm going to demo this tomorrow. Come to my talk to KubeCon, and we'll go over cryo in action. OK, so that's cryo. Pretty quick, huh? Pretty cool. It just basically, traditionally, Kubernetes has had a problem with Docker changing out from underneath it. So every version of Docker is broken Kubernetes. So what we wanted to do when we built cryo was basically to say, whatever we do, we can't break Kubernetes. Kubernetes is the thing that's important here. It's not, so basically when Docker 1.8, Docker 1.9, Docker 1.10, Kubernetes always trails behind them. As a matter of fact, Kubernetes right now only supports Docker 1.12. They're about to move up to Docker 1.13. And at that point, Kubernetes basically is sort of saying that might be the last version of Docker that they'll support going forward. And even Docker is moving away from Docker, they're moving to, I think, all container D and using cry stuff of that. So we talked about OpenShift uses Kubernetes, but OpenShift does more than just use Kubernetes for using containers. It builds containers. So the second part of this is OpenShift needs the ability to build a container image. It needs the ability to push container images around the environment. So can anybody in this room tell me a way of building a container image? Docker build. Can anybody tell me a second way? Source damage, and what's that built on top of? Docker build. Anybody else tell me a different way? Ain't that depressing? 405, you know what a Docker image is? Or an OCI image? It's a tar ball in a JSON file. I could build a shell script to build a tar ball in a JSON file. That's all it is.  OK? Last year when we were talking about this stuff back at DevConf, a fellow worker of mine decided that he would build a tool to demonstrate it. He started in the morning. And at the end of the day, he wanted to build a tool. He decided to call it something to make fun of my accent. Because I said, I told him, why don't you build me something that will build a container? Why don't we just call it a builder? And he said, OK. And that's a Boston Terrier is the dog now. Now we have to change the icon. I'm going to ruin everybody in this room when they look at that icon. He looks like he has tidy whiteies on his head. We've actually changed the icon, but I don't have the new one. OK. So what is Buildup?  You can do a builder from, pull down an image from a container registry. Guess what it uses under the covers? Containers image. What does it build on top of? Container storage. Unpacking on top of it. And you just say builder from Fedora. It creates an ID of container. At that point, you want to mount the image. I want to write to the image so I can do a builder mount. When I build a mounted, it returns where it mounted it. At which point, you can copy, use any shell script, any tool in the known universe to copy the files. If it runs on Linux, you can use it to move content into it. So you can DNF install. You can use the copy command. You can do make install. You can do anything you want to put stuff inside this image. When you're done, you can do a builder config to set those special environmental variables, entry points, things like that that they're associated with the image. And then you can do a builder push to push in anyway. So you can build standard bash scripts. Instead of having Dockerfile, it's the only way to ever build a container, which is a very bad version of bash. You can do this. You can decide when to commit, when to patch. If you want to use keys from your host, you can use that. So builder, pretty cool tool. Pretty cool little tool. And we also support build using Dockerfile. So you can actually use a Dockerfile, hit it to build it, and it will build it. But we don't like typing that in, so we call it buildabud. An eyes of bush is not responsible for that name. So you can actually build using Dockerfile. And basically, we'll build a container image. And you can push it and do anything you want, all on top of container storage. As soon as you're done building the container image, Cryo can use it. Because guess what? Unlike the Docker, unlike standard Docker, where you have big, fad container to the image controlling the storage in the image, we can shear storage in the image between multiple different processes. So builder can go and build a container and have it instantaneously available to Cryo. And we're going to talk about a couple other tools that also uses. So we can actually share storage between multiple things. This file system. Can you imagine a file system sharing between processes? What a novel concept we've come up with. OK? We're working on OpenShift. So the next version of Source Stream Image, hopefully by this summer, will actually use builder under the covers instead of just implementing Docker. So as we go out right now on OpenShift Online, we're actually using Docker for builds. And we're using Cryo underneath the covers. As we move forward, we want to basically get an alternative to doing and so using builder. The goal with builders is actually to make it less privileged, require less privileges. Right now, it still requires the same amount of privileges to build a container, but hopefully in the future, we'll be able to trim down the amount of privileges required for builder. So what else does OpenShift need? Well, one problem with Cryo is that it doesn't have everybody that goes into a Kubernetes environment right now. If something goes wrong, they get onto the box and they execute the Docker commands. They do Docker PS to see what's running in the pods.  So we needed tools to be able to do similar activity. So we're introducing something. I'll refer to it as Kpod today, because that name has been rejected, but that's what we all call it. And legal has not come back with the term that we can call it with. And we have it as part of the live pod effort. And the naming is working its way through legal. But Kpod is a tool for managing pods and containers based on the Docker CLI. So we know that you guys all understand that the way I list images, I type in Docker PS-A. I list containers. If I want to list all containers, if I want to list a short name, I do dash E. So all this knowledge has been built up on the Docker CLI. So we decided to build our thing called Kpod and use our own special CLI. So we have Kpod PS. We have Kpod Run. We have Kpod Exec. We have Kpod Images. So really, we're very creative in what we call these things. But basically, Kpod is an entire Docker CLI type environment that executes pretty much the same environment. But guess what?  So when you execute a Kpod Run, the process that's running the container is a child of the client.  So you can actually start to build smarter environments. And guess what? Kpod shares container storage with cryo. So if you want running a cryo environment, you can run Kpod PS. And it will show you all the images that are running in it. But it's in a separate process. You can launch Kpods. You can launch Builder to use these things. But basically, they all can share the same storage. What we want to do with Kpod in the long run is actually allow it to have full concepts of pods in it. So it will basically advance past just the Docker CLIs to the point where we can actually join containers to pods and start getting creative about what it means to be a pod. And this is all part of what's called the live pod project. So we want to build a library for managing pods that cryo and other tools can start taking advantage of. So here's the grandfather of them all, Scopio I have to mention, especially where we have Antonio, the creator of it. So Scopio, this is the last tool. And I think I have seven minutes left. So I'm racing through these. So Scopio, Scopio might be the most popular one of our tools out there, but nobody talks about using it. But a lot of people are. So Scopio is actually the original CLI that containers image was based off of. Scopio in Greek means remote viewing. So a little history on Scopio. A few years ago, we wanted to be able to go out to a container registry and actually look at the JSON associated with an image. The only way to look at the JSON associated with an image in the Docker world is actually to pull the image to your host and then you're allowed to look at the JSON. We had a problem with that because some of our images, frankly, are huge. So pulling down a half a gigabyte or a gigabyte of disk to your system, just to look at the JSON and say, well, that's really not what I needed, and now I'll remove it, seemed like a waste of bandwidth. As opposed, we wanted to basically go out and get the JSON and just pull that down. So we actually built a patch for Docker that basically said, instead of Docker Inspect, it said Docker Inspect dash dash remote.  They said, it's just simple web stuff. So just implement it on your own. Don't be adding new patches to the Docker CLI. So Antonio here said, okay, I'll do that. The problem is he didn't stop at that point. He said, well, if I'm going to pull down the JSON, I might as well pull down the image, too, inside of my tool. And they said, well, if I pulled the image, I might as well push the image. So he continued to develop on this thing. So he actually built the Scopio tool that does a really nice job of pulling and pushing images. With Container's Image, he can actually pull an image from one registry and push it to another registry. So now we have Windows ports of this tool that are actually moving registries around. This thing, I want to grab my phone that's telling me I have a meter. Okay, so Scopio was able to move, but Scopio didn't, Container's Image has actually got rather creative. Container's Image supports Container Storage. So I can pull an image out of a registry and push it directly into Cryo Storage or Builder Storage or K-Pod Storage. I can actually pull in and push it directly into Docker. So I can pull an image from Docker IO and stick it into Docker's database. Okay, I can push it to a directories. I can push RLCI images in Docker V1 images all with the Scopio tool. So if you're looking for a tool to manage moving these images from registries to local storage, sort of pre-loading systems, that's what a lot of people are using Scopio for. So that's a quick view of what Scopio was able to do. And I'm gonna end now. We can do questions. I got five minutes left, right? Well done. This talk usually takes an hour. And that's how much caffeine he's had. Do we have any questions in the audience here for Dan on Monroe or Mr. Scopio? Nobody is in quite, they thoroughly understand everything. All right, yes. I can hear you. Who was that? Raise your hand. Just stand up and shout it. No, make Diane run. Diane, with the best, is gonna get awfully hot. Are we gonna be able to interface with cryo as unprivileged users? He's still getting it. Can you say it one more time? Are we gonna be able to interface with cryo as unprivileged users? Are you gonna be able to implement, you will be able to, anybody that talks to a container registry as a regular user, you're talking about like doing Docker socket to allow it, well, let me tell you about exposing the Docker socket to a user, okay? Just give them pseudo window password and turn off logging, okay? Because you have basically given them full route to the system. So anytime I can talk to, anybody that can talk to a container registry, I mean a container runtime, you're giving them full route to the system. So yeah, if you believe that you should allow your users to have full route to your system, then give it to them. There's no additional security built into cryo over what's built into Docker, okay? So again, the only thing cryo does is implement what Docker wants. Whether or not we'd be, allow you to use K-part in the future to use, say use a namespace to do it, that would be something we might investigate in the future. There is a tool called Builda, kind of bubble wrap that actually implements some of that. And if you follow a flat pack project, it allows you to do some stuff. But right now, we're not doing anything special in cryo, any of this stuff that's gonna not require route. So I would prefer you to use Sudo to set up those environments. Someone else have a question? Yes? One in the back. Windows support. Do you not see the shirt? No. No. If Windows wants to come, Windows has come in, Windows engineers or somebody's come in and given us patches to make scopio to work on top of Windows and on top of Macs. So it's all open source. We've actually talked to Windows about potentially using some of this technology, Microsoft I guess I should say. So, but we're not doing the work. So I would love to have them come in and join us on. And Microsoft supports a lot of Linux, so I'm sure they'll be running it in their Linux environments. Anybody else? Kind of a detail thing. The conmon, is that a bit like an init container for a pod? So conmon isn't like an init container. It is like a small process which is apparent process of the container. And this is because the way OCI has implemented the separation of create and run, we need something to actually monitor the process. And that's the role that conmon does. So you still can have your own init inside a container, but to monitor the container itself from outside, you need conmon. Yeah, so when I run a run C container, run C actually starts pit one and then goes away. What happens is conmon launches run C and stays around running, basically listening to stand it in and stand it out from what ends up being pit one of the container. And it sits out there, if anybody wants to connect to it or attach to it, then it can give back that control of the terminal. Therefore we can, something Docker used to not be able to do was restart a Docker daemon would take down all the containers. We can restock cryo and because conmon is sitting out there basically holding the containers open, it'll continue to run. That's the way I like to think about conmon. Anybody else? If not, you're right on time. I'm right on time. Thank you very much, folks. Thank you.


It's the Everton News Daily. Carlo Anciolotti has revealed that Andre Gomez is back in full training ahead of Sunday's Merseyside Derby at Gwladyswn Park. Gomez, who took a slight knock last week and missed the 11v11 game of Gwladyswn at the weekend, is back in training, Anciolotti saying he should be fine for Sunday. He also had some other positive news and said that Yeri Mina and Fabian Delph are also having an outside chance of being in the squad for Sunday's game. He said if they missed that one they'll definitely be with us for the Norwich game on Wednesday. So a bit of good news for Everton on the injury front. Obviously last week it felt as though everyone was getting injured. Still without long-term casualties, John-Philipe Gwbaman, who will be out on next year, and Theo Walcott, who had an operation last Monday in the midst of his rehabilitation. He's expected to miss another three to four weeks as we move forward. But good news that Mina and Delph may be available inside the next ten days. The Derby referee for Sunday has been confirmed as Mike Dean. I hear Evertonians cheer. Mike Dean, who took charge of the Derby at Anfield in December, which was the rule one, fired two. Also didn't give Everton the penalty for the clear push by Virgil van Dijk on Calvert Llewyn, a two-one, which a lot of Evertonians have been tweeting about today. The VAR official is Chris Cavana, who last time was involved in an Everton game, he sent Carlo Anciolotti off at the end of the one-one draw with Manchester United, when Everton were wrongfully denied a late winner. But there you go, they are the officials. Hevin Lozano could be offered a way out of Napoli by Everton, that is according to Italian media. On loan is now being suggested, obviously Napoli value him. They paid £33 million for him last summer. Everton wouldn't be paying anywhere near that amount for him. He's had his latest fallout with Gennaro Gattuso, the Napoli boss. Gattuso was unhappy with Lozano's application in training and ordered him away from the training ground. We know Marcel Brands likes him, he took him to PSV, we know Carlo Anciolotti likes him, he bought him for Napoli. He had been linked with Everton last summer, but area Napoli and Karel Del Sport reporting that Everton cut off at Lozano a route out of Napoli this summer. For me that has to be a loan, I don't see Everton part with that kind of money.  Finally John Joe Kenny has been nominated for the rookie of the year in jail. I need a 23-year-old as being on loan at Shalka this season. He already won rookie of the month, I think it was for August. He's had a good season for Shalka, scored his second goal of the season last week. Good news for John Joe Kenny, he's got three games left before he will return to Everton. Doesn't look as though Shalka have got the money to buy Kenny and he'll be back for consideration for Carlo Anciolotti to see what he wants to do with him and whether he can stake a claim for Everton right back start and position for next season. That is it, make sure you subscribe, give the video a thumbs up and if you want more videos get over and join us on Patreon. See you later.