All right, I think we're live. Hi, everyone. Thanks for joining us today. My name is Sarah Benchaita. I'm with IBM Research Communications. I'll be your host, kicking things off before turning it over to David Shubhmell, who is our moderator and leading today's discussion with our panelists. So we're really excited to have everyone here today for what we find to be a really interesting discussion about the path to create more hybrid AI, if you will, that can demonstrate true intelligence and being able to answer questions like maybe why and how versus just what. So really excited again to have everyone here. Before we get started, I do wanna acknowledge this of course is a virtual event format for all of us, which we established in the wake of the COVID-19 pandemic. So along those lines, just wanna really quickly say we hope each of you is safe, healthy, and doing okay wherever you're joining us from. I also wanna take a minute to quickly run through a few quick reminders about how the flow is gonna go today. So throughout the event, you will be able to submit questions for either all the panelists or a specific person. Just please indicate who you're questions for. And to submit a question, you should see a little Q&A window on your screen. That's where you can submit a question. So feel free to submit questions as we go through the panel discussion. That said, we are gonna save the last 10 minutes to go through and answer the questions. So we'll of course try to get through all of them or as many as we can. For any questions or follow-up, we don't get to. Just let whoever from the IBM comm side that you've been working with know and we'll try to connect to you afterwards with the appropriate panelists. We're also gonna have a playback of this event available in the next day or so. So those of us on the IBM comm side will be sending each of you a post event recap with the playback as well as any other additional resources including the content we'll be showing you here today. All right, so all of that said, I'm now gonna turn over to today's moderators. So David Schubel who is the Worldwide Analyst for IDC's AI Software Platform Research Division. I hope I got that right, David. David, floor is yours. Feel free to take it away. Thank you, Sarah. The opportunity for machine learning and deep learning based AI application have really grown rapidly over the last decade. IDC estimates that spending on machine learning and deep learning solutions will exceed $63 billion by 2021 and growing to over $96 billion by 2023. By 2025, we expect that at least 90% of new enterprise applications will include some type of embedded AI ML functionality, recommendations or advice. However, in most cases, these models and solutions are focused on a very specific problem based on huge amounts of data that they've been trained on. In today's panel, we would like to discuss what is being done to move AI forward towards more flexible and explainable models. So I'd like to introduce our panelists, Leslie Kalbling, Professor of Computer Science and Engineering at MIT, Josh Tenenbaum, Professor of Cognitive Science and Computation at MIT, and David Cox, IBM Director of the MIT IBM Watson AI Lab. Leslie, Josh and David, could you each tell us a little bit about yourself and your research? Leslie, could you go first? Sure, so I've been studying AI and machine learning in the context of robotics for a long time. I'm really interested in the question of how to engineer general purpose intelligent robots and how we can take basically all the technical tools that we have in our toolbox and put them together to make useful and interesting artifacts. Terrific, Josh, how about you? Yeah, so like Leslie, I am very interested in general intelligence and I come at this as a cognitive scientist and also an AI researcher. So in my work, we have two goals, which synergize to reverse engineer, as we say, to understand human intelligence in engineering terms and then to use those insights to build more human-like forms of machine intelligence. And so we're very interested in where are the gaps between today's AI and what humans do. We're especially interested in the kinds of basic common sense that you can see even in very young babies, ways that like even a three-month-old baby has a common sense understanding of what's in the world and what would it take to give that kind of ability to machine intelligence that could then be the basis for all sorts of many other useful applications. Terrific, thanks, Josh. And David, last but not least, could you talk a little bit about what you're doing in your research areas? Yeah, so I'm David Cox. I'm the IBM director of the MIT IBM Watson AI Lab. If you don't know that, that's a joint research effort founded in 2017 where IBM announced it was going to invest close to a quarter of a billion dollars over 10 years to found a joint lab with MIT to sort of explore the vanguard of AI. And it really drive forward, what do we need to make AI broadly applicable to be useful for IBM's customers to really make those foundational advances? I'm a recovering academic myself, so I've only been at IBM for about a little over two years before that I was professor at Harvard and actually did my PhD at MIT. So I've actually known Josh since I was 22, which we won't say how long ago that was because I would be talking about myself how old I am, but it's a lot of, we've been thinking about these ideas together for a very long time. And I think that intersection of the brain, the mind and AI is really right for really exciting things to happen and I'm really thrilled to be collaborating with everyone here and all our other colleagues at MIT. Terrific, thank you. So everybody's really talking about artificial intelligence today and machine learning and deep learning. What in your view are the successes and what are the current limitations of AI and ML? Leslie, I'll start with you since I, Sure, well, so I mean, the striking success right now in particular in machine learning is in problems that require interpretation of signals, images and speech and language. And for years people tried to directly solve problems like detecting faces and images and directly engineering those solutions didn't work at all. But it turns out that instead we're much better at engineering algorithms that can take data and from the data derive a solution. So that's been the amazing success. The thing is it does take a lot of data and a lot of computation. And so for some problems we don't have the formulations yet that would let us learn from an amount of data that we have like available. So in my area in particular in thinking about robotics it's a lot harder to get training examples they're expensive and they grind the gears and they break things. And so we really have to focus on being able to learn from smaller amounts of data. Okay, and Josh, what are your thoughts about this? Well, yeah, I mean, I agree with everything Leslie said but also add to say that the machine learning paradigm is a certain way of framing a problem, right? Where you start with a data set basically which you think is representative of some other data that you're gonna meet in the future and you have a loss function like that defines the goal and it quantifies the goal so that you can optimize on it. And neural networks deep learning is the latest and greatest way to frame those sorts of problems and successes are many. But if you think about intelligence in the general sense it just isn't that kind of problem, right? So human intelligence if you say what's the loss function or what's the data set even? I mean, you could say, well, is the data set all the data that's ever come into your senses what about the millions of years of evolution that led up to this point? What's the loss function? There isn't just any one task, let alone a way to quantify it with a loss function. Human intelligence is about general representations of the world, concepts, abilities to plan that let us solve new problems that maybe people haven't ever worked on before until somebody presented it to you or even make up our own problems, come up with our own tasks and define our own loss function. So there are really big gaps between the machine learning paradigm which works for a certain class of problems which are very economically and sometimes sociologically socially valuable, sometimes also have their problems, right? Again, we're all familiar with those I think at this point. But when you talk about intelligence more generally you're talking about a way to understand the world and there's a huge gap there. But I think on the research side not in the way that has yet to necessarily transform our technologies that everybody in the world sees the way they see Google, Facebook, Amazon, and so on. But on the research side really exciting things are starting to happen to try to capture some steps towards more general forms of intelligence in machines. And in the work that I do, and again I've also collaborated with Leslie for a very long time on these things. You know, we're increasingly seeing ways that we can draw insights from the way humans seem to understand the world and take at least small steps towards putting those into our machine systems. Perfect. David, what are your thoughts about this? Yeah, I agree with everything that's been said so far but maybe I'll take a slightly more industry-focused point of view on it just because I'm the industry guy now. One of the things that kind of mantras that sort of keeps me up at night is automation. And to a first approximation I think when people talk about AI today that term has shifted in meaning back and forth over time. But a lot of what people mean when they say AI today is really just automation. You want to automate something. And as a result, that's why there's so many different things that people are kind of calling AI. There's been sort of like concept creep of what we call AI. But automation is incredibly labor-intensive today in a way that really just doesn't work for most of the problems we want to solve. It sounds like a joke almost to say that automation is labor-intensive. But what it means is for most of the problems to leverage the tools that we have, like deep learning, which are amazing and powerful, you need to have huge amounts of data, huge amounts of carefully curated and bias-balanced data to be able to use them well. And for the vast majority of the problems we face, actually we don't have those kind of giant rivers of data that in our business or in our government that we can kind of build a dam in front of a great expense with expensive data scientists to extract some value from that river. Most of the problems we have are these kind of more like one-off puzzles. Like either we need to solve a problem. Like why is our database now? Are we being hacked? Or did somebody push bad code into production? That's not a thing where you're going to get millions of hacking attacks first before it matters. It actually matters zero shot on the very first time. And even when companies have big data, it's rarely curated. So it ends up being this ponderous labor-intensive thing. So I would say most of the value and most of the hard problems we have in the world that we'd love to solve with automation and solve with AI, we don't really have the right tools for that. And that's even leaving aside issues like adversarial examples. So current deep learning methods are vulnerable to hacking in ways that we don't fully understand yet. We have all kinds of problems with bias with our current deep learning methods. We have problems with the interpretability. Humans have to use these tools. Ideally they should understand why they're making those decisions. It's hard for us to actually adopt them if we don't understand why they're working. And then ultimately even composing them into larger systems is very difficult. We don't have that understandability. So I think those are all barriers. And as Josh says, there's enormous opportunity in looking at the gaps of the human cognition and bringing together all these different fields to try and chart a path forward. Yeah, that sounded great. And based on that thinking, what do you think that the path forward really looks like? Josh, you kind of alluded to it in your introduction, your introduction, Dr. Yansu, could you maybe elaborate a little bit about what you think the way forward looks like? Yeah, sure. I mean, I guess some people say, well, we've had so much success in the last few years with the deep learning toolkit. Like how far can it go? Or what do we need to add to it? I guess I take maybe a broader or older perspective, which is to say, deep learning is a set of ideas that have come to us over multiple decades. They've had their ups and their downs. This is currently, we've been seeing the last few years of it up. There's a few other really good ideas that have been with us also for decades since the history of the field that have also had their ups and downs. And when you look at the real history of the field inside AI, you see, not like a desert and then a boom, not like a desert and then a boom, but rather a slow and steady thing with some key steps that really register on the outside world in the popular imagination or in technology. And so we look to that, what's that bigger picture? What's the longer history? Well, the old is a good idea in AI, the original idea of symbols. People often talk about good old fashioned AI and blame the idea that the original approach to AI was like, human programmers were gonna write a bunch of source code, like symbolic code, like in languages like Lisp or other languages that people could understand. And then that was gonna be the AI system. And people say, well, that didn't work for, and there were many reasons why it didn't work. But I think, there's a narrative that some people have told that is, oh, well, it's because symbols, they aren't flexible or powerful and we need to get rid of symbols and replace everything with end to end neural networks, for example. But I think a lot of people, certainly the people here have realized that no, that's really not the right story. Symbols are distinctively powerful. You just need, they're just not enough, right? And especially, so what's good about symbols? Well, we've learned that symbols are good for representing abstract knowledge and reasoning with it. And Leslie has a good way of putting it, I don't wanna, well, I could quote her at some point, Leslie said, symbol, it's like a big set, basically. I'll let you go into that.  Yeah, yeah. So symbols are really powerful for certain things. But for example, there's other problems which are really hard in a symbolic approach, like search. When you're working with symbols, often a problem gets formulated as like there's this very, very large space of kind of a combinatorial search space and the right answer lurks in there somewhere and you have to search through it and the algorithms that we've had, though there've actually been advances on discrete search algorithms, that's turned out to be, you know, hard to scale in some ways. But some other people have figured out, Leslie does some of this in her work, we do some of this in our work, we do some of this jointly.  Like one thing that people do is to take a problem where your basic knowledge is expressed in symbolic terms, but you actually figure out a way to train a neural network to learn, to guide your search through that space. And it basically, it's not, I wouldn't think of it as extending deep learning, but rather using deep learning, which is good at function approximation and pattern recognition and realizing that these hard search problems, in a sense, can be turned into pattern recognition and function approximation problems. You could say that was the insight of AlphaGo, right? And the way that neural nets get used in AlphaGo is there's a big discrete search and deep mind figured out effective ways and good ways to scale up this idea. And a lot of people are using this in other ways to use, I wouldn't call it as the next phase of deep learning. I would say use deep learning for what it's good at, pattern recognition and function approximation to help solve hard search problems, for instance. So, Leslie, Josh kind of teed it up for you. What are your thoughts about this? Right, so I love to think about things hierarchically. So I'm gonna agree. I think too often in the field, that pieces of technology or approaches are kind of held in opposition. You do learning or you do symbolic this, you build in knowledge or, and really I think to succeed at making intelligent agents, we need to use all the tools. So when I think about making a robot that can do complicated things, let's say in your kitchen or a warehouse or a supply depot, it will need control theoretic stuff at the lowest level to be sure that it can move stably. It will need neural network kinds of learning for the perception, but it will also need higher level and more abstract kinds of reasoning to decide what to make for dinner or to reason about whether the person is gonna be mad because it didn't do the laundry yet or to decide how to disperse the products in the supply depot. So I think the critical thing sort of technologically is to realize the sweet spot for each piece of technique that we have, what is it good at? What is it not good at? And then figure out as, I mean, for me as an engineer or maybe for Josh as a natural scientist to understand the role these different pieces play in a larger context, in the context of making a more complicated individual, but also in the context of thinking about what might have happened on the evolutionary scale versus what might happen in an actual individual. So basically, based on what Josh is saying, we've had this symbolic AI, which has kind of ebbed and increased over time. I mean, I remember writing prologue programs back in college, back in their late 70s, early 80s, everybody was going to develop an expert system. Obviously that didn't work out. Now we have deep learning, which as I mentioned, has been pretty successful in some areas, but it's limited. Based on what you're both saying, it sounds like we have a combination of these two or some form of combination of these two might really be the kind of the holy grail for what we're thinking about. Is that one way of thinking about it, Leslie? I think so, although it might be that it's not just these two, right? I mean, there are other ideas, Josh liked to talk about probabilistic programming. So that was another paradigm that was very important and it's still very important. And maybe there's probably a couple more ideas we haven't had yet. Like genetic algorithms, evolutionary search, there might be something to that too. Let me just jump in on the probabilistic programming because I think prologue, not everybody here probably knows prologue, it's kind of what's called logic programming and it's called prologue in part because you sort of, well, basically you use, you write down. Yeah, sorry, what? Didn't you just say for programming logic or something? Programming logic, yeah. Yeah, that's right. But the idea is that you write down your so-called declarative knowledge of the world, like general knowledge about the world, like a father is a parent who is male. Well, everything's more complicated these days, but for example, or a grandparent is the parent of a parent, for example. And it turns out that actually, well, it proved to be hopeless to write down all of common sense knowledge as a bunch of statements like that. In domains where you can write it down, that prologue program is still much more powerful for reasoning than any neural network, right? It does, like human reasoning is very systematic, whether we're talking about kinship or arithmetic or lots of common sense knowledge needed to read the newspaper. It turns out that if you can write it in that form, okay? Sometimes that kind of logical reasoning is the best thing you can do. But one of the things that we've learned is that, two things we're missing from logic programming. One is that often the inferences are not just true or false, but probabilistic, graded. Like you make guesses, you can't say, I can't guarantee that this is true, but my common sense reasoning says, yeah, this is probably true, whereas this probably isn't true, right? All the time in government policy and law, we care about how strong is the evidence, how confident can we be, right? When we're talking about, you know, in science and research, like how much can we bet on something? So the math of probability theory, that actually was, you know, when machine learning really became big in the 90s and late 90s, early 2000s, before people were talking about AI as the new everything, it was all about machine learning, right? And that was really a toolkit for working with probability, being able to make good guesses on a big scale. And at this point, what Leslie called probabilistic programming, that's what we've worked on this and a number of our colleagues. It's sort of the synthesis of probabilistic inference and symbolic representations. And that turns out to be a really powerful way to do common sense reasoning when you can write down the knowledge and code. But sometimes you can't write down the knowledge and code and their best way to do it is to learn it with a neural net. So modern probabilistic programming languages plug those in too, and you can combine all three powerful ideas. Or sometimes even, this is something I'm really excited about and others in our groups are too, is the knowledge can be written down in code, but it's not a human who writes it down. So there's this field called program synthesis where basically we have algorithms that write little chunks of code. Our colleague at MIT in CSAIL who also works with the MIT IBM lab, Armando Solar Lozama, for instance, is one of the world's experts in program synthesis. He's been working with all of us to combine this whole set of machine learning and probabilistic inference tools with programs that write programs. And you put all of that together and you have a much more powerful, I think broad tool set that can take the power of symbolic knowledge, make it much more scalable and usable in the real world. Perfect. Okay, I was just to follow up on something that David said, although David Cox might want to have a chance to talk here in a minute, but let me, I mean, one thing also about back in the day when we used to write prologue programs or whatever, the view, the expert system idea was that the human expert could articulate what they knew about the problem and they would write it down. And that's what it meant to put knowledge in your AI system. I think now what we've almost all understood is that humans are actually really not very good at writing down what they know about the world. I'm an expert in vision in a sense, but I can't articulate that at all. I can't say how I see you. And if I were an expert in process control, I still can't explain how I make the steamroller work well. But we are, it turns out, pretty expert at writing algorithms. And so now we design neural networks for doing computer vision that have built into them algorithmic ideas like the way you process an image or we build an algorithm that knows how to do the inference process, but we learn the detailed knowledge about the world from experience. So I think what we're finding is that a nice division of labor between kind of human engineering and maybe what nature engineered into humans is kind of algorithms and inference methods. Those are good to build in, but facts about the world often are better learned. And just to dump in on that and comment, I think one of the themes that's sort of emerging in this discussion is that there's sort of this, really helpful confluence where we can compose together all these different traditions of AI together. And at various points in the history, there's been sort of mild antagonism between these different traditions. People saying neural networks are the answer or symbolic was the answer. But Josh mentioned this earlier and I think it's actually an interesting and important point. When DeepMind developed AlphaGo, it was presented as a success for deep learning and reinforcement learning in particular. But in reality, it's a composed, neuro symbolic system. And if you look at almost any piece of software that's portrayed as AI that's employed in real world that's doing something real, it's almost always a Frankenstein of symbolic pieces, traditional blue code. Perhaps even 80 to 90% of every AI application is actually just traditional blue code. And people, I never learned prologue, I did learn less, there's probably less at there. But there is an awful lot of just imperative code and breaking things into symbols because we simply do to solve these problems. Well, because the problems demand that level of flexibility. Things like programs are incredibly powerful on express structures. And it's not clear how today with deep learning alone, you can get some of that expressivity. Sometimes it's better just to create a program and then have humans be able to understand and reason about how you can compose together these different pieces, having these sort of symbolic choke points where there's something identifiable that you can understand. And in many ways, symbols are kind of the lingua franca of our own minds. We think in terms of simple, it's very natural for us to look at a series of operations. It's very unnatural for us to look at a bunch of tensors with billions or billions of parameters and try and make sense of that. And we have to come up, kind of control ourselves to project back into a space that you understand but symbols are very natural. So I think the next evolution of very practical AI, as we do it, as we advance there is gonna be understanding what's the science of binding these things and how do we use these to complement each other's strengths. And build a system that we can actually reason about and understand, that can grow, learn, and it can be put together with something else in a way that we can reason about what's gonna happen. And I think that's gonna be when AI really hits its stride and really addresses all of other kinds of problems that we'd love to solve. Yeah, no, that's how it goes, but it's really good. So I just wanted to remind everybody that we can, I'm getting some feedback on the call now. Oh, sorry. Just wanted to remind everybody that we can put new question and answers. So if you wanna post questions in the question chat, that'd be great. Can I just add one thing to what David said? Is that okay? You guys? Sorry. Can I just add one thing to what David said? Sure. And lastly, since we've been talking about the history of symbols and neural networks, I think another thing that doesn't get talked about very much is the way that deep learning is actually a triumph of symbolic computation, right? I mean, something that I think people maybe don't know about that much, but the languages, the programming languages, which enabled the whole deep learning boom, languages like TensorFlow or PyTorch. One of the key innovations behind these from a programming languages point of view is something called automatic differentiation, which is a program that takes as input another program and automatically computes the derivatives of it, which you remember from your basic calculus class, like in calculus in high school or in college, you learn symbolic rules for taking the derivatives of polynomials, for instance, and you learn like the chain rule, right? So it turns out that the way you build an end-to-end learning system using the back propagation algorithm or gradient descent, as it's called, stochastic graded descent, is you have to calculate gradients, which are the derivatives or the slopes of the error surfaces, the surfaces that describe the loss functions. And the only way that deep learning works as a field is or what's been so powerful are software systems where you can glue together many components, and if each one of them is differentiable, then you can guarantee the whole thing is differentiable and you can compute those gradients or those derivatives. You do that using these systems for automatic differentiation, and they are symbolic programs. They're programs which manipulate other programs, taking the ideas of calculus, which were developed hundreds of years ago, but then implementing them on a scale that no human could do by hand, right? The earlier days of neural networks, when I was getting into the field in the late 80s, you had people like David Rimmelhardt writing down on his whiteboard the derivatives of functions, that was never going to scale. So it's just another example of these two points, which I think we've been talking about, that all AI systems today are hybrids of human and machine intelligence. And in this case, humans figured out really powerful symbolic algorithms that manipulate other symbol systems in order to do this kind of function approximation. And increasingly, as we see the idea of automation trying to get smarter and to automate more and more of the programmer's tasks, right? These are examples of places where we're really going to need symbolic algorithms, including algorithms that can make other new algorithms. Yeah. So Leslie, I think you have an example of this combination. You know, maybe you could kind of tee that up and talk about that a little bit before we see it. Sure. So one of the things that I've been thinking about a lot recently is how to make machine learning be incremental and compositional. So currently in the deep learning paradigm, it's typical to do end to end learning, which means basically you articulate the kind of the description of your whole system. And then you define a loss function as Josh has been talking about, say what it means to behave well in the domain. And you try to train the whole thing at once monolithically to do one job. But then if you have a new job, it's often very difficult to take anything from what you learned from the previous job and apply it to the new one. And that doesn't seem like a very effective way of scaling. I also like to think about eventually, you know, selling general purpose robots to people. And if a robot came into your kitchen, you would like it already to know quite a lot, but you'd also like to be able to teach it an additional skill that I might not have known before, maybe teach it how to beat eggs or how to solder. And it would need to take that skill and integrate it with what it already knows quite quickly. And so we've been thinking about how to do that. So I think I can probably talk over the video now if somebody wants to play it. I'm not sure if I'm gonna be able to tell if it starts. This is a robot that we have at MIT. It's named M&M for men's at Manus, Mind and Hand, which is the MIT motto. Good, and here what's going on is we just, we can put these objects anywhere on the table that we want and we can tell it a goal to achieve in symbolic terms, like it should have stuff in a bowl or have stuff stacked in a certain kind of way. And all we tell it is the goal and it infers what to do. So for instance here, it's figured out that it has to move that green box out of the way so that it can get a good grasp on that cup so that it can pour. And in this case, the robot already knew how to pick things up and put them down, but then it did machine learning to figure out in some sense, how it could do pouring reasonably reliably and how it could move things out of the way. And if we look at this next thing it's about to do, it's supposed to put stuff in the red bowl. It realized by itself that the red bowl was out of reach for pouring and so it made its own plan to push the red bowl into the space where it could reach and then pour. So what we see is that it's very flexible. We never told it what sequences of actions to take. It never learned what sequence of actions to take. It simply learned the conditions under which pouring would work effectively, how it should hold the object relative to what it was trying to pour into and then it does reasoning to decide actually how to deploy that skill that it learned. So this is a combination of machine learning and kind of old style symbolic planning and reasoning that gives us very aggressive generalization. It can learn reasonably quickly from relatively little experience in the world. So we hope this might lead eventually to robots that could come into your kitchen and you could teach them to do something new. Very cool, very cool. David, I think you have a similar type of example that you want to walk through. Yeah, sure. So Leslie gave a great example of how this sort of neuro symbolic hybrid or machine learning hybrid with symbolic ideas can give you strong generalization and fast generalization. Another area that we're very interested in is how to have systems that operate safely. So here's an example video. Maybe I'll just talk about it a little bit first before we look at what's happening. Where reinforcement learning is a very powerful new idea in machine learning and the idea being that you can have systems that learn a policy to optimize some reward. One of the problems though, is that they're a very sample and efficient, basically trial and error learning and you need to have lots and lots of errors to be able to get somewhere. And you can imagine in many different applications having those errors in the real live world is a problematic thing. So if I wanna learn to drive a car, so when I learned to drive a car I didn't crash the car even once. And as far as I know, I didn't dream of crashing the car many times. So I wasn't even necessarily simulating perhaps crashing a car, but these systems to train driving an autonomous vehicle, they need to crash a lot. And one of the things when we think about in terms of how do you deploy these things in real world applications is, well, we'd like to have some safety guarantees. And there's a whole tradition, which is fundamentally symbolic about formal verification of software. How do you impose symbolic constraints? You can write down here are the constraints that I want you to do. Leslie mentioned grinding gears earlier. You can write down equations and sort of limits on don't try to move the motor too fast or the robot will tear apart its own arm or don't get closer than this to an obstacle or during a chemical plant. Don't let the pressure in this vat get too high because we know that the vat can't pull. And one of the things we've been working on again in collaboration with folks at MIT is thinking about are there ways to use these formal symbolic software verification methods in combination with reinforcement learning, build systems that can be sort of verifiably safe. So we can have controllers that they can still make mistakes, but we can reason about some bounds of how they're gonna behave and we have much more strong guarantees. And the video which came up earlier, I'll just maybe so I can put it up again. I just encourage you to go and check out the folks who are organizing this. We made a little demo that you can play yourself where you can sort of touch and feel and play with this method. There's a little sort of hypothetical scenario of drones delivering packages, but with certain symbolic safety constraints in there. And you can sort of see how some of these symbolic hybrids can allow you to have, neuro-symbolic hybrids can allow you to have systems that are safer during learning. And it actually turns out that they learn faster, that they learn more safely but also faster. And we think this is kind of just the tip of the iceberg in terms of ways to take existing methods we have and augment them and make them more practical for real deployment of real-world models. True, thanks David. So, when do we think these kinds of hybrid applications? You talked a little bit about some of this is already hybrid, deep learning is already hybrid. How do you see this kind of research emerging into more production type applications in over the next few years? Is there a certain type of application that will be better for this than others? Anybody, pop in. Yeah, maybe I'll say just a few keywords. Leslie, go ahead. Oh, I mean, I was just going to say, in some sense, I think problems vary in difficulty along different dimensions. And so right now, things that are really very much about signal processing seem to be very well-handled actually by the current deep network work. And so if that's mostly what you need to do, classify images, understand some kinds of sound signals and so on, then those things are good. When you have problems that are combinatorial in nature like the planning example or manufacturing problems where there's really a lot of variability in the structure of the answer, right? That the kinds of answers could be rich and complex and you have to kind of put an answer together to solve the problem. Those are the kinds of problems where in particular, I think the symbolic methods even right now have a substantial amount of leverage. I think what's gonna happen, I mean, I'm terrible, I usually don't even engage in predicting the future because it's such a dangerous game and AI people have such a bad track record really of that. But it seems like these things in an actual practice, as David said, they're already happening jointly and they'll just continue to work together to do things. And what we hear about in the press might not always be exactly reflective of what's actually happening behind the scenes in order to make the systems that actually get deployed. And I think just any deployed system will generally involve both kinds of work and technique. Yeah, I'll just add to that. There's always a concern that we're having for another AI winter. So AI has had its ups and downs and its ups and close. And I think it's a legitimate concern at any given point. Are we over promising? Is this too far away to be useful? And I think in the case of these kinds of systems, I think the saving grid, I think if we're talking about, are we gonna get to human level intelligence and human level learning and reasoning, that might take a very long time. And I'm 100% with Leslie that scientists have an abysmal track record trying to predict the future. I think we tend to overestimate how long things will take in the short term and underestimate how long things will take in the long term for ambitious goals. But the same race of these things is, there's already value here. We can already do stuff with these, right? You know, the self-driving car example is an interesting one because driving a car is a very complicated thing. And many in the field, actually before I joined IBM, I co-founded a self-driving car perception startup. Actually Josh had one coming out of his lab as well, even with the different people dabbling around different parts of this. But there's a big debate in that field around, will it come in time to actually be valuable? And you know, driving is one of those mission-critical applications where you have to do it really well. It's not good enough to do kind of 80-20.  So you'd better not do it. But at the same time, there's already value. If you have a recent car, there's ADAS, you know, Advanced Driver Assist systems that are kind of giving you a little co-pilot. And that's a way that automation can be added in that gives you value, but you know, maybe you're not necessarily yet, you know, betting your life on it. And that kind of incremental game, I think is going to be the path for us going forward in AI. We're, you know, thankfully, there will be many little wins along the way where we can start deriving value from these things. And I think we're already seeing that. And to, you know, the earlier discussion, we're already seeing that actually happening with hybrid systems. It's just sometimes you don't acknowledge that they're hybrid systems. And I think one of the great and powerful things about acknowledging that they're hybrid systems is it invites us to think about what's the science of how we put these things together in a systematic way? What are the formal ways and methods and best practices and new ideas that we get at these intersections to more effectively compose these systems together? And I think that's going to be very powerful.  We're going to see, you know, IT systems that heal themselves in reason. We're going to start to see robotics, I think increasingly, you know, perhaps first in warehouses and eventually, you know, more in home settings, you know, Boston Dynamics just shipped, you know, spot one of their robots to end users for the first time. So I think that's moving faster than maybe even many would have predicted. And, you know, we're going to see in the law that kind of boring behind the scenes places, but that's actually a good thing. That means that there'll be a lot of gas in the engine for us to eventually build these things up and understand how to really confront how we compose these systems more effectively. And I think that's going to be really exciting. And that will help us solve a lot of problems that we face today. Yeah, no, that sounds great. And just a reminder to all the folks on the webinar that we're going to start our question and answer in a couple of minutes. So if you have questions that are already logged, we'll get to yours first.  Would you say that, you know, things like, you know, Burt in terms of a deep learning natural language processing, you know, it's also that kind of thing where we're using maybe ontologies and knowledge graphs and some of those other kinds of capabilities, you know, to help us. And then combining that with deep learning to get better natural language processing and natural language understanding. Do you see that as kind of the same kind of combination? Yeah, I mean, I think very broadly in natural language processing, I think we are starting to see these things. And I mean, neuro symbolic and probabilistic symbolic and neural hybrids are, I think, also become a very influential there. You know, for example, there's a long tradition in understanding natural language of what's sometimes called semantic parsing where what that means is you take a stream of text and you parse it into a representation that has a meaning that looks something like formulas in logic programs. And that's still in many ways the best way if you want to like ask questions and get answers or if you want to build a system that you can talk to and that can compute in the back end and do some reasoning and answer a question, that's still often the best way to do it. But that's again, one of these places where in the traditional methods you had these really hard discrete search problems and people have found that actually you can train neural networks to help learn guidance for that search to construct basically a program that represents the meaning of a question and then can compute its answer. So it can do something like real reasoning. And then if that program can interact with other programs the way programs have always done, then you can really start to put thoughts together and to do something that's more like reasoning. And you know, that's something that people in our group here in the MIT IBM lab have worked on. We in our own research group have worked on it. I think it's very exciting direction going forward. So I think it's, you know, you take that toolkit and it's showing up all over the place. I would even say I was gonna say in response to what Leslie was saying. It even shows up in perception. You know, I completely agree with what Leslie was saying that like the sort of lower ends of perception which is like signal processing and finding stuff in pixels, you know, up to the level of like recognizing a single object that's been solved pretty well in a lot of big data settings. But what we envision, we often talk about a scene understanding where it's not just recognizing object but it's the whole scene around you. You know, if I showed you what was really in the messy room behind me, it's a really challenging scene understanding problem as in probably all the rest of your homes, right? So there's all these objects and they're stacked on top of each other. And if you want to go get something you have to move under this. You know, if you think about like, as Leslie often likes to talk about what does it take to get a robot who could make tea in anybody's house? Or like you saw in that great demo that she was showing, right? So to perceive a whole scene, the best ways of doing it are not a single end to end neural network but one that already has symbols that represent the objects and their relations to each other. That's already a place that you're seeing this toolkit being deployed to go from perception to real thinking and action planning. And that's also true as far as I understand in, you know, self-driving cars as they currently exist. They also go through that kind of but a signal to symbolic stage. So, you know, I think we're already seeing that. And, you know, I think going forward as this toolkit evolves on the language side, on the perception side, on the planning side, that's where all these things will start to come together into something that is increasingly more like real intelligence. Yeah, no, that's great. Great, great. Thanks, Josh. So I think we're going to go to the question and answer part with Sarah again. So Sarah, did you want to pick that one? Yep. Can you guys hear me okay? Yep. All right. So first question from Craig Smith with Eye on AI. And this is open to any one of our panelists. I am particularly interested in applications for education. As the world has moved to online instruction, are there applications that you can imagine that would apply to education delivery? Absolutely.  So we currently have about half a million students taking my course. I haven't worked for Harvard for two years, but I think we were teaching when I ever did, did on campus my entire life. And I really believe that there are huge opportunities for AI in education. And Josh and Leslie are absolutely wonderful expert educators as well. And I think one thing that Edgy, you learn teaching is that it's actually a really interesting social exercise in empathy and sort of theory of mind. You want to understand what the person might think based on what you've said, and then sort of probe what they really got and what they didn't get and sort of, you know, try to purposely leave them down the wrong path and then show them they're on the wrong path and get them on the right path. So I think that back and forth, that natural back and forth has a lot of opportunity for automation. And of course, I built, we actually did try and do this. We had lectures that would fork depending on whether you've got a question right or wrong so that we could help, you know, pull you back to a better understanding. And there's some very sophisticated things happening in with code for instance, if you have a code assignment in your science class, can you look at the code and try and analyze what the person, the misconception the person must have had to have been able to make that mistake. So I think there's huge opportunities there. There are also huge economic problems with developing technologies for education. There's, you know, it's difficult to, you know, find lots of investment in that. And those are some of the barriers. But I think as we build out this toolkit of, you know, more flexible systems that can reason, I think having them reason about students and having them reason about educational opportunities, I think that's a very exciting path.  But for instance, you know, Emma Brunsgl, who is someone, she's a professor at Stanford now, but she was, I was kind of co-supervised her when she was here at MIT as a paid student, worked on formulating, you know, teaching arithmetic as a reinforcement learning problem in a sense. And you can reason about what does this mean, you know, and so on and decide which problems to show them next. And that's been easy to do in domains where not easy to do, in domains where the formulation of the knowledge is clearer, right? So teaching someone how to do long multiplication, that's an algorithm we can easily judge the misconceptions and so on. Teaching, you know, trying to understand where someone understands principles of literary criticism or something, that's pretty far out of range. So, you know, I think there's one part of it where we kind of roughly understand how it might go and then there's some other parts which are very far. Yeah, I mean, but the common sense toolkit comes there in an interesting way because there's this problem that, you know, cognitive scientists call theory of mind, right? Or mind reading where you try to figure out what is somebody's goal or what are their beliefs from how they act? And, you know, people in our lab have, and a number of others have built these tools that we sometimes call them inverse planning. You see the actions that somebody takes, what they say or how they move around in the world as the result of a plan. They made a plan in their heads, they did a planning algorithm. And you want to run that backwards to figure out what were their goals or what were their beliefs, false or true. And it turns out these kind of models, we've been building these on the cognitive side for a while, that company that David mentioned that came out of my lab was founded by a couple of postdocs, former postdocs, Yu Bao Zhao and Chris Baker. And they both worked on these computational theory of mind models and their company is called IC and that's what they're doing is they're trying to allow cars to read driver's intentions. So whether we're talking about, you know, what's been that the, you know, always considered to the big open problem in full autonomous driving is understanding the people out there, the human drivers and pedestrians or bicyclists or understanding the people who are taking your class. And it's, in some sense, it's the same problem, right? You see behavior and you have to work backwards to what's in their head, what are their beliefs and desires? And, you know, again, these are things that have already come out of the lab and are starting to be deployed in some actual use cases. Great. And Josh, this next question is actually for you. So tease it up perfectly. Data is obviously at the core of AI, but most businesses lack large amounts of data which can then lead to poor performance. How do you create systems that can work beyond this constraint? Yeah, I mean, I don't know if this is, if this is specifically for me, you know, some work that our group has done is in what we call one shot learning or few shot learning or building systems that can learn from very little data. That's certainly something that humans do. And from a very young age, right? A human child can learn a new word when they're starting to learn language from just seeing one example of somebody label something, right? Think about the first time your child saw a rainbow and it's an amazing thing. And it turns out it has a name, right?  So we, this is something, again, cognitive scientists have worked on is how do we capture one shot learning? And many AI people at this point have drawn some insight and inspiration from human cognition. But, you know, we think where we have to head for that is, you know, the root we have to take to that is again to take the ideas that we've been talking about here for the last hour, which is how do we represent abstract knowledge? The way children really solve this problem is they early on acquire very general ideas about objects and the physical world and people's goals. And that together with smart kinds of inference mechanisms, you know, we've approached this as a kind of hierarchical and Bayesian, a kind of probabilistic inference. People, we and others also try to train neural networks to learn to do this kind of one shot learning. But the key is whether you use a neural network or a Bayesian algorithm, really the key is what is the abstract knowledge that that system is able to harness to fill in the gaps where the data don't tell you where to go. And that's where symbols come in, right? So that hybrid of neural probabilistic symbols is a way that we've been building inspired by humans to build machine systems that can use abstract knowledge to learn from much less data. Great, thanks, Josh. And so this one actually is open to the entire panel. So whoever wants to take it, how does the neuro symbolic-based method address building trusted non-biased AI? Yeah, maybe I could say a few words about that. So one piece of it we've already alluded to, which is one way, one thing we need, one ingredient of trust in the system is being able to understand it. And in many ways, symbols are the language in which language is an expression of symbols, but it is an entry point for our understanding. We can see the symbols, we can see how they're interacting, we can look at a program, we can understand it. So I think that's an important ingredient of trust. The issue of bias is interesting in there as well. So one word we haven't said yet, and it's kind of about it, but it's very important is causality. So I think there's another important piece of this is really understanding not just the correlations, between different data sources, or the correlation between this pixel and that pixel, or this variable and that variable, and an economic model or whatever, but really understanding what the cause and effect relationship. And a big part of what humans do is they latch on very quickly to understanding this causes that, and understanding the network and causal relationships there. And one thing that's emerged, I mean there's a field called causal inference, which has been around for a while, but it's arguably sort of hitting its stride increasingly now in machine learning. It's really looking at how can we learn about causes and effects rather than just correlations. And in the area of bias often, there was an interesting case where the judicial system in Florida was deploying a machine learning model to decide whether to give people parole based on whether they thought they'd recommit crimes. And the idea being that, well, if an algorithm does it, surely it's unbiased because math can't be biased. Of course, nothing's farther from the truth. The data is biased. Even if you blind the system to variables like race, that's mixed up in all the demographic information and you're basically just gonna recapitulate what happened in the past. And interestingly, the solution to this, or one of the solutions to this is to really have, to use these methods of causal inference where you can say, okay, we know that a race doesn't cause recidivism. Race doesn't cause somebody to recommit a crime. So we can just break that link in a causal graph. And now we're forced to system to inference knowing this external knowledge that that cause isn't there. Of course, it's gonna be, having unbiased AI is gonna be a long journey. We're gonna have to keep doing it, keep getting it wrong, keep fixing it, getting it right. But I think these ideas are really understanding the underlying causes and facts and being able to show those and so that you can understand them are ultimately gonna be requisites for us to be able to safely and barely deploy systems. Great, thanks, David. And so we, I know we're running up on time, so to be cognizant of that, we'll wrap with one final question. Again, this goes to any one of the panelists. The COVID-19 pandemic changed many industries for good. Do you all feel it has changed how we view and what we expect from AI? I can jump in again here. I would say one thing we're hearing on the industry side of machine learning is that COVID-19 has broken a lot of models. And the reason it's broken a lot of models is the reason I just said, if the models are correlational, think about it if you have information about your customers and you know that the customers who go to fancy restaurants also shop at fancy grocery stores. Therefore, more restaurant behavior predicts going to Whole Foods or going to a fancy restaurant, fancy grocery store. Those correlations weren't true. There was a causal link there. Those people are wealthy and they could spend money on restaurants, they could spend money on fancy grocery stores, but all of a sudden COVID breaks that and now nobody's going to restaurants or they're just starting to. So you wouldn't predict now that people won't go to the fancy grocery store. In fact, their behavior shifted. If you understood the real causal model, then your system wouldn't have been so brittle. And if you could understand what causal model underlined the decisions that the model made, you might have some confidence in it. But now we're in a situation where we had put stuff in the black box and stuff comes out. We don't know whether to trust it because we don't really understand what it was doing. And in many cases, it's making an actively wrong prediction. So I think there's a renewed focus on resilience. I think there's a need to be a renewed focus on carving out the right joints, extracting the real structure of the problem. That's deeply at the heart of everything that everyone in the model looks about. Can I just jump in on that? Because I think that's a great point and a great example. Another instance of this you can see is, look at all the patterns that were broken by this singular event, right? Suddenly this niche product item on Amazon, N95 masks, suddenly started selling out everywhere, right? Breaking all standard prediction models of how likely those things sell. Another thing that started breaking was face recognition algorithms, which suddenly started seeing pictures of people wearing masks and had no idea what to do with that, right? Now we know that there's an underlying cause behind that because people are buying masks and wearing them, right? And our visual systems have no trouble adjusting to seeing somebody in a mask. I mean, it's a little bit weird sometimes, but you have to get used to it, but you can still see there's a person there and you can still often recognize them, right?  So the point is, these are all places where our common sense understanding of the real cause that there are people in the world and they're wearing masks hopefully and for good reason. And that is simultaneously breaking your product prediction and your computer vision. It really does focus people and say, okay, yeah, we need a more robust approach to understanding the world and the true causes of the data sets that we're measuring and shift from just a data focus to model building and understanding. I would say one other thing which is I've been, it's been really inspiring to see many of our AI colleagues, some of our own students and colleagues, you know, use, take this opportunity to use their tools to try to address the problem in many ways, whether that's like working on drug discovery or model building. Some of our colleagues at MIT, for example, like the Kashman Singles group, which is the probabilistic computing group that also works a lot with IBM. They build probabilistic programming, that technology we were talking about before, but then they started to work with MIT's COVID response to build new kinds of models of what's going on, you know, how can a university adapt to this new era? How can we measure and predict and imagine possible situations to inform our decision making, when to reopen, how to reopen, when to bring people back? So behind the big decisions that the university is announcing are models that are being built by AI researchers because they know it's not just about the data. There's no data. You have to build models and be able to reason with them. They have to be causal and you have to be able to deal with uncertainty. So all those same themes are being used and deployed by those researchers to try to help us make better decisions. And, you know, I guess one other way that the community is adapting as many, I know all of us, you know, Leslie, me and David, this is like conference season when the normal thing that AI researchers do is, you know, go travel around the world to hopefully fund or exotic places and give talks on our work and meet our colleagues. But we're not doing that now, but instead all the big AI conferences are going online. And in many ways, it's making the feel better because it's much easier for people around the world. I just participated in the ACL or Computational Linguistics Conference and they have 5,000 people and they're running the conference in 24 hour time zones because there's people on, you know, pretty much every continent where there are people who are working on computation language and now there's a much more diverse group and much bigger access to the AI tools. So, you know, I think that's another place where the community just won't go back. I'm sure we'll go back to having meetings in person, but I'm pretty sure we will not go away from having big global conferences online once since we've built the infrastructure because it's actually much better for the field. Terrific, terrific. Well, Leslie and Josh and David, thank you for a fascinating hour of discussion about, you know, developing flexible AI. I think we'll take the opportunity to close on that upward note from Josh. And Sarah, did you have any last statements here?  Thanks to everyone who joined us today. Again, just a reminder, those of us on the IBM communications side, we'll follow up with each of you with additional resources, including the two videos that we showed here today. And we'll also be following up with the link to the playback of this event so you can watch it onto me and after. But we'll close it out. So thank you all for joining us today. Thanks, everyone. Thank you.


All right, so I'm Matt, I'm an ATM, the co-founder and CTO of Splice. Splice is a cloud-based music platform for creators, for musicians. And we basically hook into the digital audio workstations, like Logic, or Ableton Live, or FL Studios. And every time people create music, we keep track of all their dependencies, everything they do, and we give them kind of a version control of all that work. And they can see a timeline of their work. They can actually go and see what we call the DNA of the music, so they can go and see all the different parts of the music, they can do inline comments, and they can also share the information with the rest of the world, so they can release open-source music. And we believe that we can change the way music is being created and shared and learned by open-source music at a lower level. And really something behind Splice is the concept of knowledge is power from Francis Bacon. And knowledge is power. This is a very serious quote, people. Be serious about it. So we really believe that, and we believe that we can share that with music, but it's also a hard thing to do if you don't think about security. And this is another quote, security and crypto is really hard. And we're going to talk about that, but before we get into the topic, there's another quote that I really like from my wife, who really likes Barcelona. Unfortunately, or fortunately, she's pregnant. She's actually very pregnant. She's due in just a few weeks, so she could not fly with me, and she was very, very sad.  And I figured I should start by the way every story starts, once upon a time. Oh, I forgot about that slide. So the lawyer has asked me to tell you that this is a true story, and the names have been changed, but everything else is correct. So this is the story of Alice and Bob. Alice and Bob are two young entrepreneurs, and they want to change the world, like everybody in California, and probably all around the world. But they're thinking about building a different kind of application to really change the world, and they call the application Abe. Abe is a new kind of social network. It's kind of like Tinder mid-secret, so you have this platform for pet lovers, people who love animals, and you can upload onto the website a lot of the photos of your cats, and the software will find matches for your pets, and then you can go on pet dates with the owners, and you might actually end up finding a mate for you, or for your animal. They also have this really cool feature where you can take two pets, and then it generates a photo of the pet that would get together as a baby pet. That's amazing, and it turns out the application is turning to be very successful. They hired a few developers, but, well, actually, I'm getting ahead of myself. When they wanted to build this application, they had a big question, what technology should we use? And they looked back, Facebook was the most popular social network in the world, and they looked at it and said, well, Facebook is written in PHP. Should we also use PHP? And they asked around, and people told them, now, don't use PHP. Rails is a new PHP. You should use Rails. So, they use Rails. And it turns out the application is super successful. A lot of celebrities start putting the photos online, all the pets, and we even heard, I don't know if it's really true, but I was told that Emma Watson found a new boyfriend because of Abe. So, this is a big deal. It starts bubbling up, and they get more users, but as they get more users, they're also getting into scalability issues. And, you know, especially around image processing and all these things, the bill also is getting pretty high. So, network latency, response time, image processing, all these things come together. And they don't quite know what to do. So, one night, they go to one of these nice hack karaoke nights.  I'm sure all of you do that all the time. And this is where they meet Carlos. Carlos is a well-known, respected, neck-beard programmer. She's also known as Conchita. And she's a low-level programmer who used to write assembly and C, but she recently moved to Go because Go is faster. She's not really into magic, so she's not using Rails. She basically explained to them that she can write a Go API for the image processing that would probably be about 50 times faster, and it's probably worth it. Alice and Carlos dream to become the new DHH. They want to raise cars in the 24 hours of Le Mans. They really dream about it and they're like, we need to hire Conchita. Let's just do it. And the hire Conchita, and the first thing Conchita realizes is that it's not as easy as she starts because she's getting all these web requests from the web, so people are going to the Rails app and they want to go and upload a photo, but at that point she gets the photo inside the Go API, but she has no idea where it's coming from. Like, who is this from? Is it authenticating or not? She doesn't know what to do. So, well, she calls Dave. Dave is a friend from a karaoke night and he's just back from two intensive months of bootcamp training where he learned everything about Rails and about JavaScript. He also paid $20,000, but that's a different story. He knows everything and he basically tells him or tell her, remember Francis Bacon? Knowledge is power. I will tell you the answer and I will not charge you for it. The solution is to use a web session and Conchita is like, well, yeah, I figure that much, but that doesn't help me. It's like, wait, wait a second. I'll explain to you. A web session is just like a hash. You can put whatever you want in it. You send the user back. The user does some other stuff and when they come back, you get your hash back. This is magical. It's like stateful web. This is really great. And Conchita asks, how do you get the information from the user? And Dave says, it's easy. Just call session.currentUser. And Conchita is like, wait a second. I'm not using Ruby. I'm inside Go. How does that work? And Dave answers, well, it's magical. Well, Conchita doesn't really believe in magic. So what she does is that she takes her phone. Actually, she faced time, but we'll skip that part. She calls Frank. Frank has a dark past. Frank knows a lot about the web. And Frank was one of the first developers working on Abe, but he was dating a girl called Eve at the time. And he got caught at Starbucks, hijacking Facebook sessions and stealing a lot of information. So Frank needed up going to jail. Well, he was going to go to jail. And that's not, I mean, I'll just say it. Just don't repeat it. Frank needed up working for the NSA instead. So he didn't go to jail. And nobody was able to prove that Eve was working with him. But Eve got let go. But Frank still has all this knowledge and is a good friend. He also does karaoke nights. And Frank explains, well, it's not that complicated. I'll explain to you. Basically, a web session just uses a cookie. And a cookie is a key value. And the key value is kind of a log box. And you get a log box per domain, per server. And the domain puts this log box onto the user. And the user goes away. And when it comes back, you get the log box. So the server can put whatever information it wants on the log box. And then you get it back. Now, there's just one problem. Is that you're driving around with this log box attached to your car. And you have bad people around. Actually, a lot of the bad people are you on users. If they can, they would change the username, the ID. They would do weird things like that. So to prevent that, what Rails does, and what most web servers do, is that they sign the cookie. They sign the session. So this is a way to prevent... It's a way to prevent people tempering with their data. It sounds very magical, but it's actually not. And Frank goes into explaining how that works. So everything lives inside Active Support. Active Support is a library that's inside Rails. And Active Support has this class called Message Verifier. And Message Verifier, the only job is to sign messages so they can be verified, and we know that the data was not tampered. So how does that work?  And this file has this long string of random characters. And this is the secret. This is what you use to sign everything. Now, the problem is if there's only one key, it might be really easy to find this key by looking at all the encryption. So instead, Rails does something else, and it's called key stretching. So instead of just simply getting one key and using this key everywhere, it takes one key and gets derived keys. And a derived key, it's crypto, it's a bit boring, so I'll try to make it less boring. But basically, we're using a key generator, and we pass the secret, the secret that we will not use directly. We want to derive it. And we say we want a thousand iterations of this secret, whatever that means. What it means is we're going to use a password-based key derivation function 2, not 1, 2. And remember Francis Bacon, knowledge is power. So after the talk, go and Google it. You will learn a lot. But in the meantime, I'll show you the high-level explanation of how that works. So Rails calls a key generator and generates a key by passing a salt. So salt is basically some extra information that is used to make the encryption a bit more robust so you cannot find what the original key was. That's kind of a very simplified explanation. But here's how the function works. You take a hashing function like shall1. You take the secret, that's the original secret we add. And then we take the salt and we add 1. And the result of this function becomes the salt of the second iteration. And the second result becomes the result of the third one, all the way through X amount of iterations that you want to run. And you should do about at least a thousand. Rails does a thousand. It's really the minimum recommended. So you end up with a derived key. Now you pass this derived key or derived secret into the verifier. And the verifier can take an object, like here this hash, and will generate a string that you can put in a cookie or you can send as a param or you can do different things like flash messages and things like that. Now this information is not so magical. It's basically a string split in two. You have the two dashes and you have the signature after the two dashes and you have a dump version, base 64 encoded at the beginning. So if you take this information, the server can look at that, verify that the signature matches the encoding and then can base 64 decode the string and then load it into Marshall to get the object back. And that's how it works. Now there's one thing though. You might not want to expose the session content to just everybody. But let's say you have a session and in the session you have a flag that says, you know, VIP or moron or a knowing user too. You don't want people to see this information. You might want to actually hide. It's safer to hide a lot of this information. So Rails uses also Message Encryptor. And Message Encryptor lets you encrypt and sign your sessions. So we can look at the code. It works the same way. We need to generate another secret. So you don't want to use the same secret. So we derive another one with a different salt. And then we pass both secrets into the Message Encryptor function or method. We get a new instance of this encryptor and we say encrypt and sign by seeing whatever object we want to encrypt and sign. And on the other side we can call encrypt, decrypt and verify. And we'll verify that the signature is right and the message was not tampered with and we'll extract the data back. So this is how a web session works. Conchita is really happy. There's a lot of crypto and math. It seems to be working well. She understands how that works. She can move on with her things. But all of a sudden there's something bad that happens. Big drama. She gets a phone call from Alice and Bob. And Alice and Bob are explaining that something bad happened. All the posicat photos of the celebrities got leaked on 4chan and Reddit. It's all over the web and everybody's looking at them and they're over the press. Now at that point Conchita is freaking out. Like what did I do wrong? Did I do something? Is it my fault? I didn't even write any code yet. Turns out Alice and Bob don't care too much because there's no such thing as bad publicity for them. And they get pretty cool like, hey, everybody's talking about Abe. The signups are going through the roof. Don't worry about it. Also when we fired Frank and Eve, we switched all the servers to HTTPS so we don't think the session was hijacked. Don't worry about that. It's probably something else. Maybe these people, I don't know, maybe they did like Apple and they figured out the password. Maybe the password were 1, 2, 3 or something like that. Don't worry about it. And as they're on the phone, something even worse happens. They're like, oh my God, stop everything. This is so bad. Turns out the entire database is gone. The website is down. The data is gone. Everything is gone. And at that point, this is not fun anymore. This is not like we're getting good press coverage. We got hacked. There's something really bad that happened. So Alice is freaking out. Bob is freaking out. That's not helping much. They don't know what to do. Conchita and Dave just don't even know where to start. What happened?  What's the big deal here? How did they manage to dump the database like that? On the other side, Frank is pretty impressed. It's like, man, they got pawned big time. I don't know how they did it, but it's very impressive. And this is when we meet for the first time Wendy. Wendy is Frank's new girlfriend. When Frank went to work for the NSA instead of going to prison, Eve left him. She could not handle the fact that he was working for the government. And he met Wendy. And Wendy is a very, very smart girl. And she says she knows exactly what happened. Sorry, I need to take a break now. So Wendy says Eve did it. Eve is the one who did it, and I can prove it. Eve was working on the code base. And Eve had a secret token on the machine because you committed it in the code base because that's the way it works in Rails. And because of that, she was able to generate signed cookies on her own machine because she would put this key running any Rails app on her machine and then she would create a cookie that's signed and it would change the user ID to be the user ID of Emma Watson and she would now be logged in as her and she could see everything that Emma Watson would see and that's how she leaked the photos. She had the cookies that were signed and trusted by the server. However, there's one more thing. You relied on Marshall. Do not trust Marshall. Marshall is a nice guy, but you cannot trust Marshall. Marshall is not such a good friend. It's good to have around, but do not trust him with anything serious. Instead, you should be going for Jason. So Jason is not that smart and it's very simple. It doesn't do much, but what it does, it does it very well. So let me show you, says Wendy, what happened here? She dropped the database, if did, by running a very simple code by using a very simple exploit, which is not an exploit. It's actually normal Ruby code. In Ruby, you can use these hooks into any classes, Marshall load and Marshall dump, and these are hooks that get executed automatically when you Marshall an object. So what Eve did is write this class and she had this function called Marshall load, or this method called Marshall load, and the first thing it does is connect to the active record, list all the tables and drop them one by one. Very simple code. Don't need to do anything when you dump the data, and then we're going to assign it because cool hackers put the name in the hacks so we can find them. Then she put this information in the session, all of that into her own local machine with the right key, and then she copied the session from the browser. She went on to Abe.com, she changed her cookie to use that, and now she's executing whatever code is inside the first method. Now granted, Eve was not very smart because everybody noticed now that the website is done and it's down and it got hacked. If she would have been smarter, she would have found ways of leaking the information, copying the data, like stealing credit cards. She just wanted to mess with them. But you can see that you can execute any code doing that. And this is how you can basically own any Rails server if you have this key, which is why you should not have this key and this is why things got changed in Rails 4.1. You might wonder, am I going to be attacked this way? Well first, you need to leak this key and if you trust the people you work with, it's probably won't happen, but if anybody left or if the source code got released on the web, that might be a different story. But you can verify quickly the serializer. But when he explains, you can actually, if you upgrade it to Rails 4.1, make a big change. And you can basically change the serializer from Marshall to Hybrid. And what Hybrid does is that it will take any Marshall sessions and it will cover them into JSON. Now you could switch directly to JSON, but if you do that, everybody will get logged out. You might not want to log out everybody. Now in this case, you're probably issued, but in your case, you might not want to do that. So you could switch the serializer to Hybrid, give it a month so all the active users get the sessions updated, and then you switch to JSON, everybody who doesn't have a JSON will be logged out and then we need to log in again. Now there's one thing though. Remember, Marshall is all fancy, but JSON is pretty dumb. So you cannot put any kind of complex objects inside a session in JSON. That means no Ruby objects. That means no dates, just very basic JSON structures. When she ties Relived, she understands what the problem is. She actually has a solution for the different applications now because she can rely on JSON. So the first thing she does is like, she restore a clone of the database, get it back up and running.  And lucky enough, she finds an amazing library written by an awesome and handsome developer that does all of that already in Go. And she just needs to use the code and here is how it works. This is Go code. Don't get too, too scared. A tied person structure is kind of like a Ruby class with a few attributes, ID, first name, last name, and age. The thing on the right is called tag and it basically says when you convert it to JSON, use these names instead for the keys. And then let's pretend you want to create a session with John inside the session. You would create an instance of this person, then you would get the Rails secret. So now you need to have the same secret as the one you use on the Rails app. You take the secret, you use the two cookie salts. So Rails uses two salts, one for signature and one for encryption. You can find all that in the documentation. They are not the right ones. They are not fitting on the slide. Once you have that, you create a key generator. You pass the big secret that you have and you do exactly the same thing as Ruby. You derive the keys. You get two different keys. You pass them into a message encryptor and then you can encrypt and sign. You end up with the same string that's base 64 encoded with the dash dash and the signature and you're good to go. On the other side, when you get the session, which is in the case of Kanjita, she's getting the session coming from Rails. She could open the cookie. She gets the information. She takes the string and send it into this decrypt and verify that would do the same thing that Rails does and she would get exactly the same information on the other side. Now what's exciting for Kanjita is like she listened to Evan Phoenix this morning and she knows that to a certain point she's going to have to run services but she also knows that other developers might want to use other languages. They might want to play with other things. More than likely it might not be Rails or it might not be Ruby. If it's Ruby, it's pretty simple. You can use the same code and you can use the active support outside of Rails so you can have a Sinatra API and do the same thing and do all these things. But what if I need to use a Java library? What if I want to use Rust or Elixir or Julia? That gives her a lot of options because now she understands how the session mechanism works and what needs to be implemented to get the same transfer authentication. The website gets very popular. They do an IPO, make a lot of money. Obama ends up posting photos of him and his dog. The Queen of England and her two dogs do that. Everybody makes a lot of money. Kanjita moves on to her next startup. She's building a typewriter, Bluetooth-based typewriter so she can type assembly on a typewriter connected to a Mac. She feels really good about that. Eve, well, she ends up in jail and that's a different story for a different time. Or if you have Netflix, you can watch it directly. Alice and Bob are throwing wild parties all over the world with all their friends. And Frank and Wendy end up moving to the Kingdom of Far, Far Away where the laws around cryptography and reverse engineering are different and they still consult but they have way more fun doing what they like the most. That's it. It's the way to do this with public key. Not in Rails, no. Sorry. It was already hard to actually get the patch in so the story behind it is that I was actually really trying to do like Kanjita to get my session in Go and that's when I was like, I don't want to write Marshall in Go. I started doing it and I was like, no, this is really wrong. I think Evan was like, no, what are you doing? And so I started porting and looking at the code and that's when I realized the exploit and that's what I basically documented it and a lot of people worked on it on making it into a patch and now you can choose the serializer. So technically you might be able to choose a different serializer that would use a different key and you might be able to do a public-private key system. I just didn't try it. More questions? Yeah, I'm trying everything now. Okay, one more so I feel good about myself. I had a cool experience a while ago. So one of our team leaked Amazon key pair that had pretty high privileges in our account. Did you do that during a presentation? No, that would have been cool. I did put good keys on yesterday. About two hours later I realized anyone could do anything from my Twitter account with those keys and I should probably take them down. But yeah, he put them in a public repo on GitHub by accident and he realized straight away took it down, re-based and pushed up a commit without them in. So they were gone from GitHub. About five hours later we got an email off Amazon telling us our keys had been leaked, which GitHub repo it was, and a URL to where they'd found it. So Amazon must go through and anyone from GitHub is here, they can tell us this is some sort of partnership. But Amazon must go through every repo and every push that's put up there, cash it for any sets of keys that they have which I thought was pretty awesome by both Amazon and GitHub to tell us about that. No more questions? Excellent, let's give Matt a round of applause. Thank you very much.


So, like one of the most pervasive tendencies in the American understanding of just about every social and economic phenomenon is the tendency to individualize both successes and failures. So, I mean, what I mean by that, right? So it means you look at groups of individuals and assume that their outcomes are just the combined results of individual decisions made by members of that group. So when we see that white workers consistently have more wealth at the median than black households, you know, the American mindset leads people to assume that there must be something either particularly efficient about the way white households manage their finances or something deficient about black households or perhaps black culture, right? It leads us to hyperanalyze black behavior and ask questions like, you know, what can we do to better teach black people to manage their finances, right? On the same token, when we see black Americans with higher mortality rates and case rates from COVID-19, it leads us to ask, you know, condescending questions like, you know, are they washing their hands properly, right? So the academic term for that frame of thinking is methodological individualism and it really just gets in the way of our understanding of racial inequality and especially when it comes to the economy. So what I want to try to do in answering this question is provide something like a vaccine for individuals thinking here, right? The one thing you have to understand is that the numbers don't necessarily speak for themselves. So you can look at the unemployment rate in a given month and see that the black rate is nearly double the white rate. And if you have this individualist framework, you might assume that it's simply because black workers are just less willing to work at the prevailing wage or relative or whatever low pay relative to government benefits than white workers are. But what we can do is we can think a little bit deeper and realize that one, to be counted amongst the unemployed in the statistics, you have to be searching for work, right? So that gets rid of this idea that they just don't want work, right? We can dig deeper into the historical record and learn that that two one ratio is basically been maintained for the past 50 years, right?