Oh, it's this one. Okay. Oh, it's facing the wrong way. It's like, I don't understand. Okay. Okay. We'll put it like this so we really get the sticker. Okay. Oh, one. Can you see where my cursor is? Oh, there it is. Okay, that's it. Are you ready? So, I'm going to get started. Yeah. Okay, there. We're good. Yeah, we're just starting right now. Great. And we're live from Toronto with Toronto Legal Hackers at an event about Democracy 2.0 and what legislators can learn from lean developers. Legal Hacker Amy and now comes the announcement on my computer of the brand new sticker for Legal Hackers Toronto. Right there. And so it begins. So here's the host of the evening. Christoph, we sort of put the event together and we'll get started in a moment. So, stand by for this legal hacking event. Only for the online. Yeah, so we should be on time. I'll send you a link to just done YouTube. So, you have your job, your job. And we'll be starting the event in just a couple of minutes. Stand by. Thank you. Thank you. I think there's going to be a few people coming in as as time goes on. So, we're going to jump right into it. So, I'm going to give you a little bit of the agenda after Christoph is going to introduce you to the UX and what we're set the stage for what our panelists are going to be discussing. So, we'll have 15 minutes segments. That's 10 minutes of a presentation for each use case and then we'll have a five minute Q&A period following the presentation. So, let's get into it. All right. Hello everybody. So, my name is Christoph. And before we start looking at the use cases, I'm going to very briefly summarize some of the principle of lean UX. And I have a question for you first. So, we might have some students from beat maker. So, I should know about it. Could anyone tell me what is lean UX? Please? No. All right. Well, can anyone explain what lean UX is? No, any volunteer? Yes. So, yeah, it's about iteration. Any other idea? All right. So, let me explain you quickly what lean UX is. So, lean UX is basically if you want to know about lean UX, this is the Bible about this process. It's written by Jeff Goffelf. And the most famous picture you see about lean UX is this one. It is a park. And as you can see, many times in every discipline, we can spend months designing a park, a product, or a laws with the best experts on the world. And then when we do it, when we create a park, we actually realize that the people are not doing what we were expecting. And this is really an issue that the IT industry has found many times where people were spending months designing a new tool, then realizing that nobody was using it. And this is what lean UX attempts to resolve. So, as you mentioned, it's a cycle. So, instead of planning stuff for months and then launching them, the idea is to test as quickly as possible or assumptions. So, what is the problem that people have? What do we try to resolve for people? What are the sort of outcomes we want to get out for a new product? And for each of these hypotheses, we design some prototypes or we make some drawings, we interview customers, so that we can get as quickly as possible some feedback, learn from our mistakes, and then iterate. And we do that several times with designers leading this work until we get the answers to all the assumptions we've made. And by following this situation, we hope that we will develop products that actually solve issues for people, that actually have a positive outcome. So, to give you a few examples of the principles of lean UX, this is one of the most important ones. So, shifting from outputs to outcomes. If you take an example of an e-commerce website, the outcome that they want is obviously to have people ordering more frequently on their websites. That's what really they are trying to achieve. And one way to achieve that might be to create a button so that people can more easily reorder a product. What's important to understand is that the outcome is a change in users' behavior. It's a way we want people to interact with a certain product. And an output way to do it. But if we realize that this certain output does either on the outcome, we just change a new one, but we stay focused on the outcome. A second principle is falling in love with a problem, not the solution. The problem we might encounter in IT industry is that employees are flooded by emails. And we have different solutions for that. We can develop chatbots, use existing technology. But what lean UX says is that we should not focus our attention or mind too much on one of these solutions. Because guess what? It might not work. We should focus really on the problem and then find the solution that can fix the problem. Another principle is rapid experimentation and measurement. So in lean UX, as I say, the goal is to iterate as quickly as possible to get the assumptions. But to do so, we need to be able to do these experiments quickly and measure quickly what sort of results they give. And in the IT industry, we have many ways to test. We have prototypes. So for example, if you launch a new game, you can create a prototype and test it with players. We have financial simulations for people working in finance, or we have e-commerce A-B tests. The question again is what sort of tools exist in policymaking to be able to experiment things quickly and measure the results. And finally, one last principle I would like to mention is the comprehensive documentation of the rules and elements of a design. So some of you might know Apple is famous for its design guidelines. Any new developer that wishes to submit an app, Apple has to comply to these guidelines. And these guidelines really are helpful because the developer doesn't need to reinvent the wheel all the time. And the designer don't need to invent what an app looks like. They can use existing tools, existing elements, so that they can quickly create prototypes. And the question, as for any of these sort of principles, is do we have something similar in policymaking that policymakers can use to quickly prototype new laws and new policies? So this is really a quick introduction just to make sure we all know what a Linux is about. But Amy is going now to present the panel. Thank you. First I'm going to introduce you to each of our panelists, and then they'll each have Andrew's going to keep track. We'll have a 10-minute section. We'll have to keep it going really rapidly because time will get away on us. Christian Levine is first up. He's going to explain how he used the user-centric approach to implement that in legal swipe, and he'll tell you all about that. Next, after that's use case number one, then we have Marina and Keith. They're going to be use case number two and explaining how they've advocated for best practices using UX in the city of Toronto for the city of Toronto. And come up, Fred. Yeah, yeah, sorry. And then we've got Josh Stark. He's at the back. He's from, just stay there. He's going to be explaining how blockchain and smart contracts have been implemented in ledger labs. He'll tell you more about that. And last up we have Daza who is visiting us all the way from Cambridge. He's a visiting research scientist at MIT's Media Lab, and he'll be bringing all of the former use cases together and explaining how legislation and policy has been affected by a user-centric approach as well as he's implementing that and pioneering computational law and legal science. You're up first. Awesome, awesome, awesome. How am I? Okay, good. So where to begin? So I think it's worth, if I'm going to tell you how I've implemented a user-centric design, it's worth talking about legal swipe, the app. So I'm a proud graduate of the University of Ottawa's Faculty of Law represent. We have one other person from there. And in short, we were really big on innovation there, but we're big on access to justice, right, on recognizing that there's issues that the community faces that are based in the law. And we essentially need to find efficiencies where we can and we need to do education, again, where we can. So I was in my last year at the University of Ottawa's Faculty of Law, and I was doing a lot of legal rights workshops. And I came to realize that a lot of the information we were providing was actually really difficult for the workshop attendees to understand, right? And in addition to that, the people who are coming out to our workshops weren't necessarily the people who needed the information, right? So you get, for instance, you get kids of people who are already lawyers or maybe police officers. But those youth or even those people who in fact required this information really weren't getting it because they didn't feel entitled to either come out to the spaces in which we were hosting them, or quite simply they didn't have the interest or the time. So when I graduated, I was, I was, I worked actually at a criminal law firm just down the street from here. And a patient where I was starting to recognize that a lot of people don't have the slightest idea as to what the rights are in relation to arrest and detention. And for me, it's a question of why is this, we're a smart room here, right? Well, at least it seems like a very smart room here. And if I was to throw some questions at you, I promise you, you'd have difficulty answering them. So for instance, if you're fired tomorrow, if you're terminated from your job, what are your rights? Where's the first place you go? Maybe one or two, no. I think we've got a labor lawyer in the house, right? Oh, you're going to talk to a lawyer assuming you have one in your network, right? But for the vast majority of us out there, there's no answers. We don't know what to do. Similarly, if you're stopped by a cop, what exactly are you supposed to ask them? I don't know. You just might be out of luck unless you know a lawyer or you have the ability to retain one immediately or at least get that initial consultation. You are completely out of luck. So for me, and I guess when it comes to user centric design, I said to myself, well, what's a better way for us to do public legal education? Is there a way we could give people information at the point which they require it? And I decided to tackle something which was a big, big issue last year and continues to be the issue of carding, the issue of being arrested and stopped, right? You see these videos online. I was talking to Jeremy earlier on and he's like, man, you see these videos online and it's super cool. People turn in and the cop questions them for no reason and they turn around and say, you can't do that. I've got to try to write blah, blah, blah, blah, blah, but the vast majority of us can't do that. But shouldn't this be the first, like, shouldn't this be essential education? All of you guys in here, I'm sure could tell me all about the Magna Carta, but if an officer walks in here and wants to strip search all of us, we don't know what to do. Isn't this a problem? So unfortunately, I don't have a fancy presentation for the guys, so I tried to give you guys a cool start. I'm litigator by trade, so I should be able to get the app and essentially the app is legal swipe and it's an app that gives you your basic rights as they pertain to police interactions. And taking a user-centric approach, it's like a choose your own adventure novel for any of you guys I used to read them, right? Or you go, okay, well, this is the situation. I'm arrested right now. Let me turn to the arrest page. These are what my rights are. I'm detained right now. This is what I'm supposed to do because I'm detained. And it was really great. And I know there's some city people in the room, so I'm not going to come down too hard on the civic side of stuff. But it was really great because upon creating the app, we were able to work... Could you tell me how much more time I have? I need a chance. Okay. We were able to work directly with the Community Legal Education Ontario. They actually came out right after we created the app and said, you know what, that's a great idea. We'd love to work with you. And the Ontario Justice Education Network, they came out and said, you know what, that's really great. We'd love to work with you. And the capacity in which they work with us is we actually give legal rights workshops. And the beauty of it is this actually provides us an immediate feedback loop. We go out, we do our workshops, we get direct feedback from users, just on how to improve the app itself, but on what areas in which they want the app to grow. So what do we learn from taking this really user-centric approach? Well, a police interaction is really important. If you could imagine, I developed the app out of pocket once I finished school. And I'm not like a baller or anything. I have an okay understanding of HTML and CSS. But apart from that, it was me paying like $2,000 working with a friend and saying, I'm going to make this app. And within two weeks, 20,000 downloads literally through the door. So as you can imagine, this was something that really mattered to people. And as I began to work more in the community, I got some word feedback. And I heard, you know what, we really want to know our rights as I referred to earlier. What are our rights when we're, you know, terminated from our jobs? And what are our rights when our landlord wants to evict us? That's something that's really important too. Part with working with publicly education providers is they're also able to give us insight on where people want to grow and areas in which we should develop. So if I'm going to leave you guys, I think with something here. Unfortunately, I didn't get to refer to that. I had a few quotes that I wrote down prior to and of course I never got to them. But the first is as legal or as legalists, as lawyers, I think lawyers have a duty to innovate. Trust me, just by virtue of being in this room, we're a very privileged subset of people. So we always have to be mindful of, I think, the way in which we create. And that's whether you're working for the city, how are you going to develop policy so that appeals to the public and you're not making policy for policymakers. And similarly as lawyers, so a lot of time. Okay, so similarly as lawyers, we can't simply make, I want to take laws for lawyers, but essentially we can't create in order to solely appeal to lawyers. And I think the same issue happens with developers. If you have any developers in the room, right, when you pick up software and you're like, man, this software is clearly created for the developers. We have to create with the public in mind. So the know I'm going to leave you on is I'm going to tell you about where we're hoping to develop into what the hopes are because I'm sure some people are sitting in this room like, you created this just because you're a good guy and really genuinely I did. But then I got an investor. So I had to create because I had interest in making some revenue. So where am I going with this? Well, the hope is fingers crossed. And I don't get to say it to a lot of rooms, but I'll say to you guys. We're hoping that we can not only let people know when they have legal rights issues, but the second part that's really important, which I'm sure none of you know the answer to, is you need to know what your remedies are. You need to know where is it that you could go if, for instance, you're arbitrarily detained by an officer or if you're fired. Some of you might know Employment Standards Board, whatever, maybe not. But where is it you go? So that's what our future is. Not just letting people be able to know and identify that they have an illegal issue that they're dealing with, but being able to know where they could go for remedies. Because I was speaking to someone earlier on and they said it. The institutions are there. This is Canada. This is a democracy. As a matter of fact, it's somewhere where we pride ourselves on having some of the best legal rights in the world. But the law, the legal rights, the system itself, it's only good bestable. So on that note, it's been a pleasure to speak with you guys. I genuinely always appreciate the chance to be given a platform and hopefully I get to connect with some of you later on this evening. But continue to build and when you build, continue to build with the pole of mind and continually ask yourself how could we make this a user-centric approach? How could we make sure we're not creating for ourselves or creating for others? Thank you. Thank you so much, Christian. We've got five minutes of questions if you want to think on your feet really quick. I've got one. How does your app inform policy makers? I think as we, you know, and just moving to last year when carding became a big deal, as you begin to get a more educated populace, I think they begin to hold like legislators and their politicians more accountable. And it's funny. It could be attributed, I think, to several things. You know, whether it's advocacy in the community or even certain people that are viewed. Oh, I didn't realize this was reporting. I'm sorry about that. Well, the thing was I was like walking miles away. So essentially there's half of this. Oh, really? Oh, okay. Awesome strata. So, yeah, in short, I think it, well, we had MPP Jagmeet Singh actually came out right after we created the app and he said that's a great idea. So there's a, I think there's been a lot of feedback and great feedback. We've been getting a validation from the government itself. So, yeah. Any other questions? Right. So the question was if I'm getting arrested, how do I refer to the app or how do I get to the information I need to get to? Now, our hope is that this is an application that won't be used in real time. Instead, it's going to be used to educate people ahead of time. Now, unfortunately, generally speaking, you open up an application in the moment you have to use it. So we did make something that we felt needed to be quickly navigable. But from what we've seen, the application has been used a lot actually in educational institutions, as well as in communities. I know there's a First Nations community out in Thunder Bay last year that was using the app, relying on it to assist them in doing public legal education. So, again, the hope is that the app could just be loaded with information that pertains to basic legal education, which, as I said earlier, should be, it's just essential reading. Everyone should have it, right? And once we could get to that point, then it will be something whereby people reference ahead of time. And if necessary, where's case scenario? It's like, I like comparing it to CPR. I could give you the CPR booklet and, Lord forbid, if Amy has a heart attack, we could all refer to the booklet and help her out. I don't think you're going to have a heart attack. But in an ideal world, you learn the booklet ahead of time so that, if necessary, you could utilize it. That's actually a really good question. And as I mentioned earlier, one of our most important partners is the Ontario Justice Education Network. And they work in direct collaboration with Community Legal Education Ontario. And essentially, we look at whatever they're developing. And we say, hey, could you send us over that information so that we can adapt it to the app? And we do this for a few reasons. One, being a company I won't name in the United States, they're getting sued because, essentially, I don't know if it's the law society down, I doubt it was the bar actually.  And they decided that, you know what, we think what you're giving is legal advice and we don't want to get into that territory. We want to make sure that what we're providing, but it's such a thin line, right? The legal information versus legal advice. We want to make sure we're providing basic legal information, both the rights as opposed to advice. Like, well, this is how you're going to get through the courts and sue this guy. So we basically decide based on what I won't get sued for. Yeah, that's the best path. No, because I think a lot of us assume that everyone has equal access even to lawyers. Like, you knowing a good lawyer versus a bad lawyer, it's actually a privileged subset of people who are able to do that. So in the application itself, my thought is when you have a more informed populace and a more informed prospective client, you're actually going to get better conversions and also as far as, because of course they're going to go to the law. For me, having lawyers inside the app, like right now it refers to the lawyer referral service. In Canada, we're probably not even going to do referrals even though we've gotten a few offers. We're just going to go right down to the U.S. and because everyone here is familiar with, like, CosperClick, AdWords, I'm sure. Okay, so if you look down the list of what the highest priced AdWords are, it's actually for different types of lawyers. So I think it's a good way that we can sustain. And also, I don't think we should be an application that always has to rely on a public dollar, like just as a personal thought. I think the fact that we want to go towards the private side of stuff is what's going to allow us to become a better app. Because public money is always bloated, right? Ask the City of Toronto guys. Question. Shoot. So obviously you hear a position that there's a minimum app. Yes. Because your app offers, this is where you go if you have oppressive need. If you just want to learn, go here. This is what you should learn. Right. So and it's funny because that's the same way I built out the police interaction side. So you could immediately know what to say in the moment. However, if you want further information. So there's things, if you're approached by an officer, you're immediately probably going to want to know, Whoa, am I arrested? Am I detained? What is, you know, what's my legal situation right now? And depending on what they answer, you know, you're arrested. Well, guess what? If I'm arrested, what am I arrested for? Right. So those things are immediately made available to you, but there's additional information you should reference. One last point on this UI thing. We're actually working with Humber has like a usability area that they're making now.  But the long and short of it is we're constantly mindful of how can we make a better user experience? How can we improve upon the user interface? Because of course, without users, you're nothing. So again, thank you. Thank you, Kristen, very much. Marina and Keith, you're next. Marina and Keith are UX and Open Data Evangelists. And they're going to share a little bit about their experience working with the city of Toronto, in particular with regard to user experience. Good evening, everyone. Yes, Marina and I have a collective experience where we met actually was with the city of Toronto. Both of us aren't there anymore, but we're still continuing to work together. I'm a communications public engagement guy. Marina is a UX expert. And actually Marina is just part of a startup called the Tiva, which is dealing with health data and analytics and predictive medicine. It's huge. It's going to be huge. But anyway, our collective experience started with the city of Toronto, bringing our skill sets to that project. So we were part of a larger team, but it's our common experience. And we thought it would fit in tonight as a case study. So we don't want to pick on the city of Toronto. We want to say that first and foremost. But it was rebranding the website, which is Toronto.ca. And trying to improve that, improve the user experience. Marina was, in fact, brought in as a consultant the first time that we had a user experience come in. And so on. I was there to help communicate the effort. And this is what happened to us in the initial phases of that. So quickly, I'm just going to show a video that deals with some of the things that we did through the process. I have to apologize. You can't really see that too well. I'm sorry. So those are some of the things that we did through the process. And Marina is going to talk a bit more detail about the actual user experience process. So as you can see in the video, we did a number of things on City's free brand project. We did a lot of usability testing studies on paper sketches and digital, like not even fully functional prototypes, card sorts, user interviews, communications, stream of work was happening at the same time as Keith mentioned. As a design work was happening, both to engage end users on the new website and to get feedback on the current website and also to update them on the process, on the progress of the project. For the design work specifically, we used Lean UX, meaning we didn't focus too much on producing beautiful comprehensive requirements documentation. We actually focused more on rapid prototyping, iterative design, and getting end users feedback as soon as possible. As a result, we were able to produce a large volume of quality designs, and by quality I mean evidence-based, something that users are worried about. We knew that those designs, if implemented, would work for them. We also proposed a vision for City's website, addressing things like confusing navigation, just joint user experience and content that didn't match users' expectations. As part of our proposal was also to use agile development processes, so this was an implementation strategy to make vision a reality. The proposal was to use really short sprints, like three, four weeks short sprints. Then, not much has happened. Kies, we'll talk about that. I should say this goes back to 2008, 2009, 2011. I can't remember the date that you've departed, but I retired from the city just in May. I was pulled off actually from the initiative onto an open data initiative and also a web content initiative. Some of the reasons that happened was other priority initiatives came into play. It was a small team, the resources just weren't there to continue with the same numbers on the rebranding initiative. There was division resistance. We were kind of asked to go to the moon, which was the rebranding, and then you had got to go to Mars at the same time. We were for some of the other initiatives that were there. There was also something else that happened. I'm pretty sure you're going to know the name for this. When things are added, it starts out quite simple, but then you add more layers. What is that known as? Scope creep. You got it.  But interesting enough, there was a lot of learning that came through that. We were the first through the gate in that team trying to do the process, and it certainly continued after us. They're actually pretty close now to having a new website. There's been minor changes along the way as well, but the big deal was the culture change. I think that's what impacted. I can talk a little bit about that. When I started at the city, I assumed that everybody was bought into the idea of user-centered design and UX, and they would welcome changes with open arms. From the camera? That wasn't the case. We were actually questioned a lot on everything from the methodology we were using, like people didn't trust the data from the research. Questions were about a simple size of participants and usability studies, for example, or the whole practice of UX, for some reason, became under question, which was strange to me because I didn't invent UX, but I had to defend it continuously. The other thing that happened is that we couldn't align design and development on the same timeline. For the design stream of work, we were using lean UX, producing things really fast, testing, iterating, and getting feedback loop, a shorter feedback loop. With the development, the city was still practicing waterfall approach that took forever. We were never on the same page with no design and development. We tried to implement agile development practice, and we couldn't do that. Some of the things we just realized at some point that we were trying to bring too many changes and too many drastic changes, new processes in a very conservative, risk-averse environment. That would require real culture change for us to continue with the project and actually implement our designs. Indeed. In fact, the theory that you've scoped out in your invite actually is there, the theory and the practice we felt was there, certainly. What really impacted, again, was the actual delivery. Some of the conclusions we made, really, it requires a commitment to actually follow through. Listening and learning together, it's one thing to listen, but if you're not going to learn from it and actually move forward with what you're hearing, that's going to impact. Capacity to pivot based on evidence as well, the idea of pivoting, if you're really picking up on something you need to maybe change direction than what you thought in the first place. Finally, collaboration versus dream killing. We're actually authoring a book called The Dream Killer's Office. It's not about this specific incident, by the way. It's the whole idea of in the workplace what makes collaboration successful, what brings innovation forward, or is the dream killed right with the first idea. If we had more time, I'd give you the story about the monks who decided that they wanted to do page numbers and how it took 200 years to get that implemented. The first guy, Brother Joseph, or whoever it was that said, we should just do page numbering, was, you know, no way, we can't do that. The thing is they have a vow of silence, so it was all exchange of parchment and so on. But anyway, that's what would mean my dream killing.  That was good learning. As I say, the people that followed us, were able to push a little harder and a little further. The culture can warm up sometimes, but it does take a long, long time. And that's really it for our presentation. So thank you more later with your questions. Does anybody want to start off with a question, or should I? Well, I can speak for the city now, but we were attempting to do, like, yeah, the front page, we had an idea, for instance, five and five, five changes over five days where we communicate that this is going to happen. Let us know what you think, and we take from that and keep evolving. Of course, the advantage of the internet is that you can just not look, you're printing in full color. Eight million flyers or something, and that's it. You know, like, to change it, that's going to be a problem. I mean, that's the advantage of the web.  Even that, we just couldn't get it out the door. But that has happened since, of course. And again, I think it's all a part of that learning curve. Yes, sir. I guess body language gave me away. Question is, was it done by committee? Yeah, very much so. And the interesting thing is there was a web team, and they were involved in this. But it had to be enlarged, of course, with divisions and areas within divisions, and everybody had a different opinion, different reactions. So we were hopeful that the case studies that we were doing, and even some of the videotaping that we'd done of how people reacted, would be more convincing than it ended up being. But subsequent to that, and the people that are in charge of it now, they're exercising way more control over this. They really, you know, grabbed the last sue and captured decision-making capacity. And it's a little bit more tell than ask now, but I think, again, coming out of the learning, you're too crowdsourcing, which I think is a good thing at first, but at some point you have to start to zero in at least to your question, Pat, to get it out the door and see what happens. So yeah, there was a lot of committee. Yeah. And usually too many cooks is a good thing. Certainly in the open data arena it is, but maybe not so much in a scenario like this one. Yes. Well, it won't be soon. We're actually going to crowdsource stories in the book. People will submit. It will be both a book and actually a web presence and so on. So look for it. That's it for now? All right. Thank you, Keith. Thank you, Rita. Josh Stark is head of legal and operations at Ledger Labs. He'll tell you hopefully a little bit about Ledger Labs. And if you know about Ledger Labs, you'll know that he's going to be speaking to us about smart contracts. He's going to be using blockchain technologies in certain markets with regard to UX. I think Josh is just getting started. Just a survey of the room in the meantime. Who is familiar with blockchain? Okay. So we have a pretty good understanding in the room.  Hopefully that's just going to work automatically. Okay. I hope that it works. So in the meantime, my name is Josh. I'm part of a company called Ledger Labs. We are a blockchain consultancy and development group here in Toronto. So basically what that means is we work a large variety of clients that are interested in blockchain technology in some capacity. That ranges from financial institutions like banks that want to learn more about it or build applications using blockchain tech to fintech clients that might want to build their product to the future that we expect to other things as well. So I'm going to do my fastest ever. What is a blockchain? What is a smart contract? And then try and fit it into the theme for tonight. It's working now, so I think we're good. Yeah, it's great. No problem. So what is a blockchain smart contract? That's our first section. So first of all, what is a blockchain? The easiest way I've found to explain it is to think of it in two parts. One part is it some sort of set of information. This could be like a ledger like Bitcoin or more complex like a database. And then also it's a network of computers owned by real people that jointly maintain that record of information. And there's a system of incentives that operate within that network to provide a very strong guarantee that data in the record of information is authoritative in some sense. This means that all the copies of the record amongst all the computers is largely consistent. It is in consensus. There's only ever one copy. So in the case of Bitcoin, that's why Bitcoin can have any value at all. That my record of my ownership of Bitcoin will never just change randomly. It will be kept stably into the future because of the way this network is set up. So that's very fast and simple. I hope you just believe me that these things are true. So we talked about Bitcoin already. In this case, to be sure that everyone's balance of Bitcoin is going to stay stable, this is what gives it value. But what if we did something else? What if we wanted to store more complex information? So instead of a balance of digital currency, we could use a blockchain to be sure about the state of a program. So we could have a simple script that might say if x then y unless zed or something like that and the blockchain itself would keep track of the state of that program and receive inputs and update the state depending on what was happening going into it. So the code of the program in this case is what we call a smart contract in this industry. That's that part. So what can smart contracts do? The first important thing is that smart contracts are not legal contracts. There's a distinction between smart contract code, which is just some sort of script that is stored on the blockchain and where you're using smart contract code to facilitate a legal relationship or to complement an existing legal contract in some way. So they're just tools that we can use in the practice of law in some sense but they're not legal contracts. So what kind of smart contract do? Well there's two important properties for our use case today. First that they are autonomous in some sense and secondly that they can control real digital assets. So let's look at the first one. You can have autonomous financial logic. So this is a very simple example of what a smart contract could be doing. It checks to see what the date is and then if that condition is met it does something in this case sending money to Susan. So this is autonomous in the sense that once it's stored on a blockchain, say a public blockchain like Ethereum or Bitcoin it exists independent of whoever created it. There's not a company that owns it. It's not stored on only one server somewhere. It's stored all over the world and you have a certain level of confidence or trust that it will always be there and therefore that it will execute as it was programmed. Of course it can control real digital assets. So the same example again except in this time we're using cryptocurrency that can be actually controlled by this script. So it's not like we're going to execute this program and then someone at CIBC gets a notification and they say now we have to send this money. That requires trusting the institution and the people doing that but that the code itself can have a balance of some cryptocurrency, more likely Ether if this is on Ethereum and it can natively control that and send it to an address belonging we believe to Susan. So it's autonomous and it can control real assets in some cases. So that's that. So how could be used in conjunction with the legal contract? Well the thing is that of course many elements of a contract can't be replicated easily in code. How much time do I have left by the way? I just want to check where I'm at. Hopefully that's good. So things like an indemnity clause cannot just be easily executed by code. There are many parts of contracts that exist for a judge to look at them later and decide who's right for violated in what way. Or any clause requiring human performance. If I have a contract that says you will paint my house for $500 obviously a code cannot accomplish that or monitor that without a lot of fancy hardware. So more likely what you would have is if there were a case where the smart contract code could be useful, you would have a traditional legal agreement that would be tied in some sense to the coded element and each would be doing their own function where they are more useful and more suited to that purpose. So the best way I think about it, there's another tool in law's toolkit. There's laws that contract that regulation affect behavior by doling up penalties and sanctions, but also by having some sort of more persuasive force and also as a coordination mechanism where people can look to see what the law is and therefore kind of agree and engage in some sort of activity. And now we have a new one which is that we can use coded relationships to actually control real digital assets in some sense and execute any sort of logic we want automatically with some conditions. So it's just a new way to do new interesting things in law. So that's kind of all oh yeah, and what could they be used for? So a prenuptial agreement, not likely that's complex, it's messy, it's human. Financial contracts, yes, those are mostly informational and highly regulated and highly standardized as well in many cases. Contract of the house painter probably not, but machine to machine commerce where you have to come back and forth between interesting things devices, absolutely. That could be very useful. Kind of preliminary to the topic of today, which is what are the lawmaking applications, what relationship does this have to kind of lean principles. So a couple of examples. So first talking about what's called like reg tech, broadly in financial contracts. So if we had a large share of market activity being done and coordinated through blockchain smart contracts disclosure couldn't send some to be automatic because a regulator could for example actually see the data that represents contracts between market participants. You would not have to say to companies we need you to disclose this information on a regular basis. They could actually just see the data, see the actual transactions as they are happening if that's what we want to permit them to do. And disclosure could also be selective and granular. So you wouldn't have to necessarily show all of the information to a regulator or to the public or anyone else. There are cryptographic techniques we have to prove things about a set of data without giving up the entire data itself. So these are called zero knowledge proofs. And the easiest way to think about these is that a simple analogy to explain the concept when you go to a bar and they check your ID you show them your birth date. You're giving them all of the information and they by looking at that birth date check to see that it meets a certain condition. If we had a zero knowledge proof machine you would put your driver's license inside of that and the machine would beep green if you were over the age of 19 and red if you were not.  So looking another way you could put a financial relationship into a zero knowledge proof machine that would prove some thing about that relationship, that contract, whatever and the regulator could see that it meets a condition without necessarily breaking privacy or transparency laws that we might have in a given financial context. So if you have these new disclosure capabilities you have new things. You have more data which might produce better regulation. You have more frequent data which could allow for faster responses from a regulator. And you also have a lower burden on regulated entities. Companies and financial institutions especially spend a lot of money and time on disclosure obligations. It's expensive, it's process of intensive, it's terrible. If it's happening automatically if the regulator can just actually see the data, it's a lot less work and effort on their part which might make it easier to adjust disclosure obligations because you're not requiring say banks to create entirely new different processes to capture the data and send it to a regulator of some kind. Example two instead of things with machine commerce. So a market composed mainly of machines is kind of an ideal use case for this stuff because a they need identities which can be created using cryptographic systems and B they might want to exchange in financial relationships and that's much easier to set up in a few lines of code using a smart contract than giving them bank accounts. So we can imagine autonomous delivery drones that you know can respond in real time to new regulations about airspace and that you know report information to a regulator if you want to set things up that way. It gives us a lot of flexibility in how we handle these things. There are a couple things in lean that we look for that are relevant here. One is tight feedback loops between the problem and the solution implementation. Blockchains smart contracts might let us create a tighter loop in relation of certain markets and secondly that you know we want lots of data to be able to create good solutions in the first place and similarly we might get more data about certain markets if they were more tightly integrated with this technology. So that's the whole thing and I think I did that in almost nine minutes. Great. Yes I will. So this has happened but I want to get your take on what happens when the letter of the contract doesn't reflect the spirit of the contract. So obviously code can't capture the spirit of a contract. You know whether or not so one situation where that might become real is where you have a contract that says in plain text you know something is supposed to happen and then there's a coded portion that handles say a financial transaction that happens as part of that contract and let's say that someone screws up the code and the money goes to you know Bob and it was supposed to go to Phil. Here we have a difference between what the code said and was supposed to happen and what the contract itself was as a whole intended to do. I would think that any judge asked to interpret that contract would you know be able to discern that it was supposed to go to Phil and not to Bob and there was just like you know a digit changed in the address in the coded portion of the contract itself. So I think that like in many cases are traditional interpretory tools for dealing with weird things in contracts would be sufficient. Obviously you need a mechanism to recover the money from Bob that was sent to and similarly I think you would just you know let a court deal with that and make whatever order is necessary to effectuate that remedy. Obviously it gets complicated when we're talking about other jurisdictions but I think that like our legal tools can deal with these kind of problems already. So it's a new form of a problem but it's a problem we have solutions for I think. Is that a question? So feel free to decline the question if it's not on a point but talk over the last year about using Ethereum and blockchain implementations for things like identity, worldwide identity records that are you know concrete that are that are distributed so they can be falsified. You give us a bit of information about the state of affairs on that as it is today. Like there's been a lot of proposals at NATO and different uh different levels like that. Is anything happening or is this just still talk about using blockchain technology specifically Ethereum for identity management and perhaps kind of more like global or large-scale implementations of that. I'm not aware that any kind of you know large government body or institution is planning to roll this out imminently. The government of Estonia is very interested in these things and I think it's moving forward with using citizen data or at least like making it authoritative in some sense using a blockchain but I'm not up to speed on that. More generally there are many individual projects in the space that are trying to solve the identity problem so there's one called U-Port that's being built on Ethereum that's part of the consensus organization that's basically creating I guess an identity system for individuals and machines and whatever else you want to put on it. There's one called One Name that's part of kind of the block stack organization in New York that's doing kind of a similar thing but using the Bitcoin blockchain as your underlying blockchain system. So there's lots of different kind of approaches to this right now. It's not clear which one's going to win out. It's certainly a useful application and it's certainly one that has to proceed a lot of other more complex applications before you can have phones or banking or anything else you need people to be able to associate their identity with a given on chain identity as well. So good question. So you've explained what could be the potential benefits of blockchain for the UX but it could allow for gating more data so that you can test more easily new laws and as well testing things more quickly. But do you know any concrete examples of that already applied? Do you have any instances in mind where blockchain actually helped designing a more better experience for citizens? Probably no. Using any blockchain tech is laborious and hard because we don't have a lot of the kind of good tools to make it easier right now I mean just dealing with public private key management is counter intuitive to people and there are not simple ways or not many popular simple ways for a developer to build a blockchain application and then manage their keys in a way that is both secure and convenient for them. I mean one complication here is like this is not designing a website, it's not designing like a video game or an app or something, this is designing a thing that controls real assets and it has to not only be well designed but also very secure and design considerations to make something secure and safe are often you know directly opposed to making it easy and convenient you know asking people to write down very complex passwords back up their account these sort of things slow down interaction with an application but nonetheless necessary because you don't want people to use your app and then lose money. So I think the answer is probably no, I hope the answer is yes in less than specifically for an operation for growing marijuana and what we used the chain for is they basically for compliance reasons they need to go from seed to stale to the entire process where it's been but it's something that they don't want to quote unquote fall off a truck because it's actually sellable. So the nice thing about blockchain is it's immutable meaning that we can make changes but all of the previous changes are still in the chain and then the easiest way to think about blockchain is a block of data that's stuck to each other that number which we call a hash the government is able to actually store those so what are the well they don't they can but nobody there understands it yet but the crucial point here is when they come back and they say well how do I know that the database hasn't been changed it's like I can do a lot of math but for us to change the database we have to take all the energy in the sun and then use it to change our data so there's no way of it doing it so if you're doing immutable data and you want to have people actually know that this data that's stored in the database is not completely sorry and ours is working but it's always been the same and it hasn't been changed it's actually a really good solution for that and to his point we're not dealing with a lot of keys because there's very few people who are intercepting our data so it's a really simple use case did you want to respond to that or yeah okay so so I guess when I would think of block chains and law it's kind of this idea that law is two parts it's like this mechanical rigid part that you want to always put this in this part comes out but then there's a human empathic human part where you want humans to human things and is there any part of the system the ecosystem that's focusing on like essentially creating space like markets for arbiters such as judges and lawyers to like come in and do their thinking feeling human thing and then the the contracts get to like be this rigid part that we don't want to change so yes so there's a lot of projects that have realized the need for something like that one that comes to mind is called open bizarre which is kind of like a decentralized ebay that uses bitcoin as underlying blockchain and they have a system where if a you know if I pay some bitcoin to order a t-shirt and it never comes I can complain about that and they'll involve a third party who plays a role as an arbitrator to make a decision on what actually happened and then you know either reverse the payment or you know settle it in some other way so you know similarly to like you know like something like this any sort of kind of marketplace is going to require in some cases that human element doing in a decentralized manner is tricky because how do you find those people how do you make sure that they're actually doing their job properly but definitely that's a need in lots of use cases and people are thinking about it already there's another project called damn like capital D a m n which stands for decentralized something arbitration mediation network I think by a woman Chicago name Pamela that is trying to build kind of like a mediators network that can be used by blockchain applications doing something similar so absolutely it's it's very doable and it's something that people are thinking about and it's a very important component of these applications and we're good Thank you so much Josh I think we need to meet up on blockchain this entirely on blockchain next up Dazza I'm so pleased to welcome you here Dazza I consider a mentor a very good friend he's pioneering the field of computational law and legal science I think he's very engaging and fun to listen to so without further ado Thank you and also he's having some difficulty finding where's my cursor on there you go yeah it's on the boat the top alright see it's very good to know that you're operating from camera you know to really want his camera okay thank you Amy so what I'm doing right now is hacking the law in a certain sense so part of what we like to do in legal hackers of which this Toronto chapter is a terrific example I think is in a sense hack the law right and so what that means is not well defined yet but part of what it means in Massachusetts legal hackers where I'm typically hacking the law is that we do work with legislatures and we do work with the bar and with practitioners and vendors to do agile or like lean techniques and hackathons and in like prototype jams and things like that but also we kind of take our own medicine eat our own dog food and like at meetings and when we're putting together sites we use get hub and in this case we're attempting a basically a multi-camera live hangout switching between the cameras and sound to see if there's a way that we can share information better across the chapters all around the world now by making it so we can use these super computers practically in our pockets the smartphones and iPads as programming and broadcast devices this is going to be archived directly to YouTube what we've done it's just hangout live and just Skype and other methods of doing this like we're actively exploring all the time and soon like this will be I hope another example where we're able to perfect some knowledge sharing collaboration methods document assembly display agile techniques for creating and designing testing and developing and ultimately propagating at scale innovation if we get this one right or when we get this one right maybe shortly after tonight perhaps we'll be able to go back and just hear all these amazing lectures on blockchain and on hacking contracts is a big theme now and governance there's a lot of great knowledge out there and people working on things so as I now kind of look at the theme for tonight which is what can legislators learn from lean UX developers I just have to add I feel that it's considerable so one thing is I'm going to just sort of move this back to the something different that it's like so meta of like the video package control is another that should be your next meeting package management so I feel like Kristoff did a good survey and I want to highlight two things that were within it that would be my answer to the question so one thing one I guess axiom of lean that were in your slides was that it's the word I would use is hypothesis driven I think you said kind of like there's tests you get data you calibrate that is a great technique it's really hard I don't know like how many people have ever been in a company or in a project where you really you didn't just do an MVP and kind of like a prototype and kind of flail around and maybe succeed or not but how many people have had like been on a team and maybe with a boss or someone that required you to do a so it's not easy or fun right no it is not and yet it's great that's a scientific method so MIT were very privileged to have a appointment for research I see in labs all around people applying the scientific method and you know creating new knowledge and changing the fundamental nature of how we work and what's possible so applied to software development although it takes discipline and it's not fun and it's difficult it's a great way to cause apps and services and other technical innovations that provably traceably will result in achieving the goals or the success metrics that you want or proving that they don't and then being able to move on so that bears very little resemblance to my experience as a legislative aide and you know kicking around different legislatures over time working on legislation and we're selling it to the market and things like that you know there's other types of forces that underlie the legislative process you know there's parliamentary processes and there's political processes and there's whole economics of it and there's a whole bunch of things but the scientific method I would not say is really among them and so MIT and law.mit.edu is where there's a methods context of law. One of the things I hear most from professors is why don't you start applying data driven processes to law in any way like certainly law making, law practice you know compliance on and on negotiation resolution so from a law from a legislative perspective just to start from today if you imagine something like say traffic management so you can have ordinances at the city level there's lots of transportation state and federal level these are one area where it's not unusual to see carbon emissions speed limits things that sensors easily can and do already gather accurate relevant data about and it could be possible in a legislative measure to include what's the goal of the legislation not just in rhetorical speeches from the floor but is metadata or like in a field like we are changing the speed limit because we want to reduce highway deaths by 2% within 18 months and we want to whatever increase traffic flow by some performance measure and you could actually state like the data that you'll be seeking to gather and collect and maybe point to some regulatory agency department of transportation where there'd be an end point or an interface where people could work you know whether it's like city systems, traffic systems, cars phone apps where basically people could share that data and you could start to measure performance and test the hypothesis of whether the law is working and how many laws I've been involved with no idea at all how they've worked out and kind of on to the next headline quickly sometimes and this would be better in order to have laws achieve their purpose and be accountable and also to start to cull all these laws that just build up over time and they never go away they just keep walking around like zombies it's you know I see dead people it's the walking dead if you just flip through statute folks and regulations that wouldn't last one second and if we were applying the like hypothesis driven if my hypothesis is deadliness all day and the hypothesis is that they should achieve public purposes that we've stated and when they fail to do so they should be repealed or maybe you could have like a cron job but you know to legislative sessions it just like repeals it like unless somebody hits the button that says you have to not repeal it anyway you can imagine lots of things if we use that one the second one I'd say you didn't really pick it up but I'm just going to say and I think a lot of legal hackers feel this way is we need version control we need like we need something like get hub for law or well get hub is great but you know it's an implementation of what I'm talking about it's not it's not the underlying capability and so the problem that I'm describing is I don't know what's real and I can never know what we're talking I seldom know what we're talking about the beginning of legal meetings so like we've never been on like calls with like 50 lawyers or with like hundreds of um if you have a lot of stakeholders and you're dealing with like a multilateral thing or even with like you know two or three companies and you've got like a big document and you've got five or ten people on a call for me it seems like it's always like the first 10 20 minutes is what which email was that Microsoft Word document wait which version was it I can't wait it was that you didn't have track changes well I can't understand and then you know like the calls over but you know you by time you figure out what you're talking about this is not good enough um and Microsoft office for sure however it's not what I'm talking about what I'm talking about is being able to address the law as data so laws information intensive and when we're the statutes contracts whatever you know opinions regulations like judicial everything you know petitions um it's all like words like that if with the full version control if leaving side the style and the format and I know we're all fascinated by you know margins and things but if we just had UTF-8 like encoded plain text version control system we could actually talk get to business right now and know who changed what and when and zero out a lot which is nonsense further more it's possible to have more responsive adaptive data-driven systems with version control and using this one like a get protocol so I'd say the second thing law makers and the law in general can learn from lean um you accent from technologists in general is VCS just like a version control system where you kind of hash every change every new file and you kind of date stamp it and like that's it and like um we can actually track legislation you can get listeners and data you you can engineer a ton of stuff just look at the API forget it up just kind of scroll through it and imagine if you could have like a webhook for any time like you know certain type of legislation was bubbling up that might affect you or your interest or your industry or like things you're interested in if you got integrations of things like um the parameters for um tax or um or any quantitative area of law so that you like compliance was part of the design of the law almost as though they were thinking about how they were making law not merely um the maybe rhetorical side of it and so right now legislators lawyers are good but the legal and some rhetorical is not a bad word um but now if we add the mechanisms that reflect and support the method and mode that we desire from a just and effective system of law um the technologists have got down pretty well and agile techniques we marry those two things and um you know and we have that concept of a developer lawyer like the philosopher king um this can be the mode um of um how law can be effective and something that legislators and others could learn very well and then people that need to understand the law were held to know the law like like ignorance of the law is no defense it is said well good luck finding it you know um it's sort filter search that day must come for every law and for everything that is legal so that it presents this data that we can use and and and um and therefore um not just comply with but actually maybe even get a little hacky with you know so that we can start to anticipate things and tune our systems and our reporting and tax and all the compliance stuff that we could zero out a lot of cost and let CPAs accounting financial people really do what the creative good doing um and not spend all this time wrangling we could we could actually have contracts and systems where business legal and technical requirements merged into one and that way have adaptive data-driven you know companies and um and um and transactions and in tiny areas the lights are coming on all over the um in many markets in many industries so if you look in certain areas of banking and finance it's all data-driven integrated systems and you know the if you look in some areas of like research and um like drug discovery and other some other markets they're fine they're interoperable integrated um systems that have dynamics to them and that um can operate the speed of thought um does that say one minute okay well okay give the minute back to you people um so why don't we go right to Q&A um so or statements of like um political statements if you want okay you mentioned version control um a couple of not long ago there was a murder conviction out west where within minutes Twitter identified a problem the experienced judge who gave them who gave who presided over the trial and found that the defendant had committed murder relied on a section of the criminal code that it turned out had been invalidated as unconstitutional a decade or more ago um before the sentence was uh handed down the judge called everybody up before him and said okay I screwed up called the crown and the defense said okay I screwed up what can I do about this um I intend to change it to manslaughter which is all I can convict on in light of the way the law reads right now so a pretty good example of why version control is important um but also an example of why of how the law is not codified in a single place um the the the conviction the conviction that was that was rendered was based on statutory law which is what you think about when you look at the criminal code as it is written but the provision of the criminal code was dead letter it was uh code that had been commented out by a supreme court decision um and was inapplicable and um people knew about it you know it wasn't a secret it was public knowledge it was contained in uh in the common law decision the supreme court um but had never made its way back through the legislature and there being some comments on law reform commission who's always watching the law and always watching what the courts do and reintegrating the decisions of the court back into the the code um we we don't have that here other jurisdictions do have something like that what's your name Oscar says Oscar and like he speaks the truth um I'm going to use I'm going to ask that and we're going to end because I don't have great examples I suppose and that's poignant example like murder conviction you know seriously um so this is um this really starts to point out how you could fashion a hypothesis driven reform and test in a jurisdiction so this is what I want to ask you um what if the following uh were a pilot that um say some jurisdiction that was um kind of groovy um was willing to um work on where you had a legislature and say a judicial branch maybe a bar and some hackers um where they could test the hypothesis the um they could where the thing to check would be can we create a system where um we model uh judicial I don't know let's say like um overturning would be like the classic of of a statute and then test when that would be in like federal or state rules of evidence when you if somebody like the prosecutor would have to enter into evidence the law and you'd want it when you're laying a foundation unless it's been a while since I practiced but it used to be that you have to say this is the law this is the current law goes into evidence and then um when you refer to the law on a web and look at say a where's the blockchain dude um and when you look at like the hash of that current version they enter it onto like a blockchain or into any source a website of the jurisdiction and you could automatically test when you're entering into evidence is this the same one hash that thing being entered into evidence is a law refer to the hash on the website or say the blockchain and then um if they match that's the law if there's been some extraordinary error like it in the clerk of the legislature there's some other thing like you know that's unusual but a routine um mistake um wouldn't occur and therefore the result you just described would not occur um with that kind of pilot and test um by version controlling from the authoritative source the legislature and the authoritative source of judicial um um interpretations and holdings on law the judiciary um some like uh like a test where you could um basically cross tabulate almost and and cross check um whether the authoritative sources are those that you're looking at and then therefore listeners you know with citations that could like webhooks basically that could update things with that the um problem that you said and be an application of like um what what you were saying version control could um um offer the law I think you're going to put west law out of business if you have something like that that actually works and the reason I say that is when you're looking at what the when you're trying to figure out what is the law that applies to these sets of facts you have uh the statute which may or may not be the current law that applies um you all have the case law that you have to look at um and somebody's got to make the link between the case law and and the statute um awesome thank you oscar okay so did did people get that so who else has questions I'm going to give to the um adjudicator questions it may be that the question is am I done well one more okay so we're allowed one more question um is there going to be a bidding war or do we have I really only see one hand so what christoph wants to arrange after um this one question is a discussion period where all four panelists are coming up sitting up here and we're going to kind of have a group pow wow um so there will be more time to speak um but is there one sort of personalized question right now that we want to he well he's the only one with his hand up so just in relation to A.B. testing I guess science has the scientific method where they have controls um the developers have A.B. testing and just like armchair theorizing on like what could that be for law do you have any thoughts like I guess I always had this thought about like we have sister cities culturally and like what are all the reasons that it's crazy to have legislative sister cities that are maybe matched up in certain properties that kind of this where they pass along it goes for like while in one place doesn't go in the other and you get to compare but anyway yeah I'm just curious about thoughts on how you A.B. test the law Hi Leen um so um I can think there's so many ways so let me just start with legislators um and then I'll do and then I'll wrap it up quickly with another application I think is even more it's like there for the taking um so with legislators I think one of the questions that you have when you're trying to draft and craft legislation sometimes is like will this be understandable like is this could people understand what we're saying like the health care reform in the United States some of it is so dense there's so many dependent clauses you know and tens of thousands so for like all pages all together um I think you could do A.B. testing on um for plain language like effective articulation of rules um that could be modeled and um could serve as um the basis for example for like integrations and industries that are complying in one way or another for example or for legislators that want to know how the law is doing um you could actually do some A.B. testing with um during like the markup sessions or times when you're doing amendments or just before you even like drop the bill and just get um people that like um show them like and show them be and then just see how many of them like almost like in a standardized test like do you think this law meant that you can go 65 miles kilometers an hour you think it meant that you can go 70 kilometers a mile do you think it meant like everyone should drink Coca-Cola like what do you think and then just or whatever and just kind of like you know like the WorldCat there's ways that you could do that just to see like comprehension and executability you could in contracts though I would say like just like how we do A.B. testing for like product and sales and matching recommendations the idea of being able to do adaptive systems with law I think is very very important so being able to understand and like maybe like in segments of your market when you're doing high volume um um licenses or contracts or different types of revenue schemes that are quasi-business, quasi-legal you could do some A.B. testing and then go not have the lawyers have to do final say at the end of a process that's not directly deeply integrating them like holistically connected with marketing, sales development um and the rest but being able to have business legal and technical BLT integration with A.B. testing where there's enough people on the ball with whoever the lawyers are in that company or that service or that government agency who are able to actually swiftly like um like provide the language and know what the differences are and be able to evaluate the overall rights of the parties appropriately would be awesome we could go on for months about like ripe areas for A.B. testing in a very primitive um legacy um like system law that we've inherited so the good news is for legal hackers and legal hacker curious people there's plenty of no one's ever done that before opportunities um to like um you know be the first kid on the block but also to be like a first mover advantage in markets to be real reformers um so um good question on A.B. testing okay so is it time for bring up the whole panel all right Kristoff thank you so could all the panelists come so before asking the audience if you have any questions for the panelists I just have one question I would like to ask um so I think you really did all a great job at explaining how um technology could uh change and inspire how we we we make and practice uh law and you all gave examples about how we can use a scientific methodology to test the uh measures are so Christian you mentioned how we can test if law is understandable and the gaps that might exist Keith and Marina you explained how you can test in services provided to citizens are performing well uh Josh you explained how blockchain can really help us getting more data about how things go and that's you explained about the benefits of of well all the exciting opportunities we have if we could have versions of the law um so but what is there a limit to this scientific approach applied to policy making and law practice is there a limit uh beyond which law is different than code and cannot be tested and cannot be scientifically approach uh or do you think instead that really the two are the same and you could really make law making 100% scientific um methodology yeah who wants to start yeah I wouldn't want to live on the planet where law was 100% um science is the best tool for certain things and it's like the worst for you know who you want to ask to dance or like or maybe like you're talking about prenuptials at some point or the essence of um human decisions that um are like emotional by design you know the the best computational system of sensors processors and effectors or human beings um maybe cats and some other animals too you know um to all of us but like you know there's like when we feel things are got you know the the incredible system of uh neurological processing that's occurring um is vastly in it it's not the application artificially of a scientific method um it's more the use of humans and like a sense of what is just a sense of what is appropriate sense of when something's novel or a good idea of what a new rule to apply is or maybe I didn't think that this could be a products liability case and not like a contract it's really something about the ingenuity of people when we get together we're social beings this is fundamentally what we need to do it would just be nice if we also could clean up our act we don't have to be this messy and ineffective and inefficient and um at being good at being human with a law so let's just um incorporate some of these um hacking techniques too is what I would say the only comment I wanted to make it occurred to me was something you said was that the criminal code right and code in a whole different way of thinking perhaps right um but in thinking about the session tonight I mean as a citizen if you like a person who's worked in government as well trying to communicate government civil um engagement and getting people involved and I think law has issues around that too where people sometimes just aren't aware of law or they don't like the law or they protest the law and you know there's a very common parallel there I think in crowdsourcing and reaching out to the people who should be benefitting by law but they don't understand it they can't read it we were talking about this at the start you know the long agreements that are risk protecting oriented words that that don't mean anything to the person who may you know hit that button I accept you know they just hit I accept and how many of you do that even you know terms of agreements you just on the web except so there's lots I think of possibilities of leveraging um all of these processes we're talking about and I had one personal experience that I was thinking of again as you were talking um it was in an accident was charged with passing on the left not in safety passing on the right not in safety and you know you can go to court on those kind of things and I did and prior to that I did some research went to the library and it looked like the cop had charged me for landing a plane on a highway that was the statue the library was behind in in their books so I was thinking about and so on to what you're doing that you know how current are those can you really rely on them all of that with code and being able to enter and update it immediately I mean the example that Oscar was talking about for example if you can trust that information it's way better than going to the library and looking at a statue that had changed over time and I mean I'm glad I double checked because it would have been obviously embarrassing as a citizen to say to the judge well he actually charged me with landing a plane on a highway obviously I would have been way out of line on that one so just as a citizen those are some comments you know on that same note sometimes I think sit down and I was I was listening to the story you were talking about uh I was saying the judge who essentially made the wrong decision and all of twitter lambasted lambasted and I think a lot of the issue that we have is as lawyers and especially the judiciary perhaps they have such a vested interest in making these systems inaccessible so it's not so much that we actually need to do this work whether it's creating these technologies but rather it's it really is a change in thinking and a change in which we view the systems then again maybe the system wouldn't be as strong if it was able to be changed merely because a computer program turned and said you know what this is wrong maybe maybe it is the human component that makes our laws something that can in fact be followed I don't know so the I was thinking about the scientific method question you asked and I mean part of the thing with the regular perspective is you know predicting the impact of a given regulation on society you know can you be absolutely sure about that and that's an important component of that and I agree that of course we'll never have 100% certainly certainly have more but another part of that is from the other perspective of the individual interacting with the legal system and having a high degree of certainty and more predictive tools to say if I do this thing you know what are the likely legal outcomes and today that's a lot of you know you pay lawyer a lot of money to give their opinion about something but you know conversely giving people more tools to have more predictive ability about what's like what's actually going to constrain them in what way is a very important part of this is not just regulators having you know this top-down approach but also meet the individual you know not even going to the trouble of violating the law because of a better way to understand that I will probably be doing that if I you know do such and such and such and such but that's another way to kind of turn on that side of the question okay so question we have now time time for questions from the audience so who has some questions to the panelists so from from the get go tonight the talk has been about information how we take legal information and turn it actionable for people then we spoke about how that pertains to computers and the ability to store the that legal information in its legislative format in the computer and now we're talking about how do we leverage the technology to make it more streamlined better easier for people to use and to benefit as a society so the information this legal information that people need to have for the machines to be able to do anything with it needs to be stored as data and that data needs to be actionable now we were talking about version control and classically in any institution you'll see version control for documents we'll just store a huge document but that still requires the human to read it and interpret it if we want to get to somewhere where you can have an application or software or a server make a recommendation on what's relevant and what's useful pertaining to a specific case we need that data to be properly modeled for the machines to be able to read that and I'd like to know more about what's being done to make that happen so that's like by no means my expertise but like a couple things get started I mean a lot of the work being done on that is just well just machine learning to be able to read legal ease and you know extract information from it and there's limits to that approach of course because you know contracts are complicated laws complicated but especially things like statutes that are well maintained are fairly modern you know you can relatively easily build a system that's going to be able to pull out the logical information and model that you know in math and code in a way that can be manipulated by whatever computer program you want to run through it I think like one thing you might expect to see in the future hopefully is that you know more more if that becomes a dominant way that people and lawyers interact with the logic of law you'll see legislators and contract rafters take that into account and the way that we write laws and contracts will change to be more usable by you know intelligent systems of some kind I think that's a long ways off because of course the law is extremely slow to adapt to anything and because it's not in like broad use today but that might be a way that that changes in the future we don't just have to build the tools differently but the law might change as well to accommodate the tools that are useful to us I might add that so I was talking today to people from something called Ross people heard Ross before you heard Watson probably that's IBM's big breakthrough product more or less on artificial intelligence based on machine learning and a bunch of other techniques so it's things are changing now with artificial intelligence and one of the things that when I was asked just trying to understand better you know the nature of how they've applied Watson to law practice it seems like one of the things that's possible now with these big breakthroughs in AI and with basically statistical modeling predictive analytics machine learning capabilities is that it can understand much more context so in the day like maybe you could model tax code in a tight context you know it's tax code and it's like this is a capital whatever asset and like it's supposed to depreciate over seven years okay that's something we've been able to model that type of stuff like test ban treaties in the day in the Cold War when we had a lot of like data we wanted to see who's doing nuclear tests and who wasn't like you can model compliance in ways but that's in a narrow constrained context they're able to pull back a little bit still in areas of law as I understand it so I would say I like the question and I think would be to look at opportunities for initial wins to build out a more coherent comprehensive approach to get the value of being able to statistically model to model the meaning and have like simulations and be able to query and get answers like if I was a single father of like you know 28 and I had four kids and $42,000 income would I be eligible for this like you could be able to interrogate and ask questions to get answers if you had modeled the data of the law if you could access the law because it was like data to start with I'd say like looking at constrained context where you know who the parties are it's like a welfare applicant or recipient and a certain program an agency that's providing it maybe like some other if you can model the actors model the transaction so we know this is a benefits determination for this program and you could model the data a little bit so if you kind of know there's parameters that you're setting like oh it's like there's like David Caleruso is my kind of hero he's a great legal hacker who does legal calculators so you could tell like whether you were eligible for an attorney if you're like poor enough basically be intelligent or you could do other legal calculators you could be able to take those domains and begin to model the interdependencies and rules in a way that you said I think you require a simulation require an interface where people could pose questions and get answers like would I be compliant if I built the model and it was like four stories but it had a two inch setback it was over here why like we could have those answers and so I encourage all of you like I hate to point you out but like that guy and that guy for starters to come back to these meetings at legal hackers and like roll up your sleeves and for goodness sakes looks good hacking on the law and then bring some legislators they some of them will come the young one especially in others or some of you run for it if you're known cause yes this is ready alright something called a thought vector which right now is just being applied to words and if you don't know what a vector is it's just a series of numbers that represents a a magnitude and a direction from an origin so a quick example is if you take the thought vector for king subtracts to the word a thought vector for man and then add the thought vector for woman you would get queen and you actually they've done this and it's pretty cool yeah yeah yeah but they're calling them they've named them branded them this way for whatever reason so you can come up with somewhere is probably working on it you have a series of vectors to create a vector space for a clause or something like that and the trouble is you would need a whole world view in a quantitative model and in order to ask questions around it otherwise you need the data to train it so those are the two different approaches and because machine learning typically relies on data anytime a new law or a new clause comes into play you'd either need to wait a while to get the data or to actually trust a robot to have a good enough view that you can ask it those questions alright yes one of the first computer programmers grace Hooper had this insight that she believed that the practice of computer programming was essentially a practice in the domain of studying knowledge itself and it's one of those things that in the 40s was just one of those weird things that computer programmers might say but looking at the different things that you've been saying and looking at it's really striking me how remarkable that insight was at any rate I just thought so I just want to point out there's a difference between when we talk about data and we talk about information so data is stored and consumed by computers usually information is actually experienced by humans and that's why user experience come to play and that's where we engage and users you know try and understand how that information will be digested and organized and you know labeled etc so that's just you know my perspective on clarify some things when we discuss you know big data solutions and you know involving algorithm and robots that's when we talk about data it's not really consumable by humans that's exactly why I asked yeah yes yes I just wanted to make sure yeah I think it's an open data advocate open data advocate I think it's important we talk about that because you know there's the law defined by governments and presented and then you know you defend the law so that the law may be there to read but you know defense and you know how many cases you won and arguments and stuff can that be digitized somehow and would it be accessible to other firms and other people not sort of attached to your firms or even the legal profession I mean that's the interesting thing about open data being open accessible for everyone to use and do things with and back to crowdsourcing to you might have non legal minds who could interpret things that you never thought of in your legal mindset so there's a lot of power with data to be sure but what about the open side of it and you know as a professional you prepared to release that data as an open data concept and are your firms prepared to do that you know what about other provinces and nationally and so on and one of the challenges we face with the Tiva is just that it's health data that you need if physicians have access to all of that data it's going to make a more powerful tool for them but it's limited data well it's still useful but not nearly as powerful so all of the cases that you cover and all of the other firms that cover it and suddenly you can start you know meshing up all of these different things and perhaps involving the city government too with the traffic intersections and if you know if there's certain cases that are coming up in certain intersections what is the correlations and all of that stuff I mean it's huge it's mind blowing so almost every profession I think has that a certain amount of data and how does that get out across the spectrum for citizens to use and other professionals and other professions it's huge alright so last question maybe anyone sorry well okay I'm sorry everyone for hogging the mic what Keith just said about people releasing information and making information available so that we can make better legal decisions and that our legislators can make better calls and be better adapted to our realities is very important but you keep seeing in the news and when you look up the laws themselves right now they're being passed in these huge omnibus projects that all these thick binders that nobody's ever going to sit down and process unless they have some financial incentive behind it is there anything being done to use things like better communication and the internet to break those down present them to the people in a usable consumable fashion and maybe start considering having citizens vote or be pulled directly on those questions yes there is another great legal hacker but he's got his hands full with his own profits he doesn't call himself that shame is craft who started something called open gov foundation and one of the things that they've done over several years now taking this XML specification for statutes just to mark up statutes called states decoded and they've worked for several legislatures many many cities to just help them mark up with this simple have structured data what's the title, what's the header, how many sections are there like places where you put some semantic tagging like this maybe like liability this relates to period of like the effective date is like January 1st 2018 whatever they're starting to mark it up in number one that's causing these otherwise impenetrable blobs binary large objects to data that you can begin to interrogate breaking it down number two and this is one of the really brilliant things in Baltimore I think is where they started this Baltimore city ordinances they started crowdsourcing first what's the heading partly because the headings had been provided by the vendor who had like had an exclusive contract to publish those laws so without the headings it was hard to just scan the law and it was going on but they couldn't use the headings which are intellectual property and they had like a small staff they couldn't just do all the headings for this thick volume of law they crowdsourced it and then they had people do some voting which are these headings seem to be best suited it was really good so if you know just extrapolate this process a little forward a little forward and begin to imagine processes where for example the city of Toronto could have number one apply states decoded it's not just states it's not just United States to mark up your open data set that's the law that's not open data I don't know what is and then just open up an interface where either government could host or you could go to archive.org or kidhub or anywhere you want and just let people start to tag so now you can have the environmental associations tag up like an area of law with one set of tags that has carbon footprint and this and the parties you can have another of the tax you know like defenders for justice would tag up a whole different way you can have other people that are tagging for things like compliance for like factory and you know John Deere and automation and you know having to do with like mechanisms and like reporting maybe not so much with the environment there's a infinite number of facets of the law the meaning of the law but depending on who's wondering what will begin to in the professional associations and on their own to share their tags in that way unleash the meaning and the more of a human experience of the meaning of law and have it shareable across boundaries, across industries this would be a really good thing to do especially in professional schools all these crops that people have to get their CPA and they begin their sitting for the law and they've got to look at something and start to learn and to contribute just like with captures I imagine one day when like if you can look at a number is number one we can authenticate your human number two at scale we can do we can we can crowd source on the attribution of human being making judgment about something that they're seeing at large scale and this can be a way to get to the vision you just stated for for law please do everyone hack the law legal hackers is the best place to come and start to give form to these great ideas alright thank you I'm afraid we don't have a time for me to as you as you ask conclude the event but I give back the microphone to Amy thank you Christophe Christophe organized this entire event this is the thank you so much and thank you to each of our panelists we really appreciate your sharing your time everybody if you want to join our Slack conversations just message me through the meetup site and I'll I'll I have to send you an email invite then I'll invite you to the Slack general channel we can get these conversations continuing and new conversations started we're going to have a drink downstairs at the pub we've got to get out of here but it's the taco place right Jeremy oh there's two places downstairs I think it's the downstairs place follow us on Twitter join our meetup group and Christophe has one thing thanks to BitMaker yes so you're aware they are a tech skills accelerator if you want to learn how to code this is the place to do it and the hack legal system this is the place to start if you're unfamiliar with the territory let's all continue these great conversations downstairs thank you for sure okay and so there you have it legal hackers Toronto Toronto it's on fire right now I can see why there are so many great innovations come from here thank goodness it's not too far from boss race thank everybody hack along