Many of you may know Tim O'Reilly. Tim is the head of O'Reilly Media, one of the early publishers on the internet, the whole internet guide. I actually began buying his books in the late 80s when there weren't many people writing books about computer networks. He is famous for his books, but he's even more famous for having help coin terms like open source and web 2.0. His conferences have really been places that the open source world can convene. That's been rare. There's been a lot of technical for-pay conferences, but things like Pearl and Python and Linux didn't have a place to meet. His open source convention is a well-known gathering place where people working on this kind of material can get together. Most recently, he's been doing the government 2.0 expos and summits, which have served a very similar purpose for people working on getting government online in a more cluful way. We're also very pleased to have our Secretary of State, the Honorable Deborah Bowen, who has been an advocate for putting government online since her days in the state legislature. If you do the inventory of California materials, I think you'll see the Secretary of State does a much better job than some of the other places. I think that's leadership. That's the policy makers thinking that this is important, that citizens do need access. I've asked these folks to talk with us about why this matters. We've been looking at in-depth issues of intellectual property and the inventory of materials, but why do we care about putting the law online? Tim talks about government as a platform and then we have access to government as a basic democratic kind of value. I'm really pleased that we have the two such qualified people to speak on this subject. We'll turn it over. Tim, maybe you can start. Yeah, thanks. So, I want to tell you a little bit about my thinking about open government, which some of us have been calling GOV2O because of the analogy to GOV2O. And I started obviously getting very interested in this back because I worked with Carl as far back as 1993-94 supporting his work. He really is, that may or is the father of the open government movement, the idea that we could use the Internet to give access to the workings of the government, to data produced by government. But I got really very interested most recently after the, during the election of Barack Obama, just as a technologist by looking at how effectively he was using social media, how effectively he was using technology to reach out, to harness the power of the public towards the aim of getting elected. But it struck me that if all this was a revolution in politics, that was much less interesting than if it became a revolution in governance. And so, I started to think more about what did it mean for us to apply the technologies of GOV2O, whether it's social media or cloud computing, or these new kinds of services that are really at the heart of GOV2O, data-driven services, data-driven applications that literally aren't as the collective intelligence of millions or tens of millions or hundreds of millions of people, to produce really remarkable results that we are still just trying to understand integrated into our society. It seemed like the government needed to be part of that discussion. And so, I did what I often do when I don't really know what I'm talking about, which is I organize a conference because what you do then is you find all the interesting people, you talk to them, and then you start to reflect what you hear. That was the advice I got from Eric Schmidt, the CEO of Google. He said, Tim, just kind of talk to a lot of people and then tell us what you learned. And I kind of started reflecting the story that I thought, oh, this is going to be way, way too geeky, you know, because it's sort of out of the computer industry. But it really resonated, and it was this. It was the idea that in the computer industry, leverage comes from platforms. And we have kind of a unique teachable moment around this right now because the lessons of the iPhone are so fresh. The way that smartphone applications were developed and delivered only two or three years ago looked a lot like the way government software might be procured. You know, you get together, you put out, you know, I'm sure Verizon put out an RFP of some kind, or they worked with some few chosen partners and figured out here are the features of the phone, bang, let's run with it. Apple did that, too. They did a really good job with some revolutionary new features. But the real revolution started when they opened the iPhone as a platform for developers. So they built 20 applications, they got approaching 200,000 for features that they didn't really expect. And I looked at that, and I thought, wow, that's a really wonderful metaphor for how government needs to work in society. Particularly because at this inflection point where we see that the increased percentage of GDP consumed by government cannot go on forever. People want more and more services, but they're not willing to pay for them by levying higher and higher taxes on themselves. And so we have this fundamental contradiction that is, you know, creating great schisms in our society and in our governance. And it seemed to me that the lessons of computer platforms that you can actually do less and get more are extremely potent and profound for our site. Now, government has always acted many ways like a platform provider. You know, so for example, when Eisenhower had built, or commissioned, the international and the U.S. highway system, they didn't specify what was put at the ends of the highways. They said, oh, we're going to build infrastructure that is going to lift the capabilities of our society, same thing with rural electrification. I think that there's some great stories. More recently, the internet, the GPS, President Reagan, in 1982, made a very, very important decision to open up this military asset for civilian use. And over the last 25, 30 years, we've seen just how powerful that decision was. And it led not just to use of those facilities, but a huge amount of private sector investment. And now, certainly in the last half dozen years, a real revolution in how we have access to the location and information of this huge startup culture, big entrepreneurial opportunities. And that's a great government as platform lesson. You know, create capabilities that are expensive. It was not something the private sector was going to do, put up location satellites. Even the government had real doubts early in the years. It was a big boondoggle. And yet, it was the kind of investment in a platform that took us along the way forward. Look at weather, the same thing. You know, we get our weather data, whether it's from Google or the weather channel, or from the daily newspaper, because the federal government puts up satellites, collects data from, you know, from boys, from ships, from independent weather stations, and organizes it, and then passes it out to the private sector to then commercialize that data. And I, so in thinking about this analogy, and looking at four great examples, I couldn't help but be struck by another analogy one that Carl was making. And that was that if you think of government as a platform, and the best design government acting like a platform provider, rather than like an application provider, delivering the actual end-user services. And that, of course, is the, I think, where we go wrong a lot with government. We've started to treat it like a vending machine. Donald Kell in his book, The Next Government of the United States, is the term vending machine government. We have this idea, we put in our taxes and together our candy bars in the form of, you know, roads, schools, taxes.  And then when we, you know, we don't get what we want, you know, where we think costy much, we get to shake the vending machine, and that's already a part of the situation. And you see that that's a fundamental flawed model. We want to move away from vending machine government and into a model where government is providing capabilities that allow us to do things for ourselves. So, so, but in thinking about this analogy and the lessons of, you know, what can we take from computer platforms and their success in driving a marketplace forward, you can't help but be struck by the analogies between open-source software and the law. Carl has observed that the law is the operating system, if you like, of our country. And just as we've had a revolution of technology through the use of open-source software such as Linux, more importantly through open-source software such as the World Wide Web, people forget the protocols were open. The original implementation was put into the public domain. This was this huge, generous platform creation gesture, which led to untold, uncounted societal benefits, proliferation of economic activity. So through openness, we get innovation, we get an economic engine. And that economic engine turns out in example after example to be greater than the economic invention, the economic activity that we get when we allow individual parties and comets to hold tight to some particular piece of proprietary information or software, we get this huge benefit. Same thing is true back to the early days of the IBM personal computer. IBM published the specifications for that computer. Anybody could build them. That was why we had a PC revolution. So then you start looking at the situation that we've been documenting in this workshop, you know, where access to primary legal materials is difficult, is expensive. Somehow this thing which belongs to all of us, you know, as a nation, is locked up in a variety of ways. It's made inaccessible, it's made expensive. You have to ask, what are we missing in our society because people are held back from access to effectively the source code of democracy? And with that, you know, I think, you know, I'm just thinking about this. Deborah Bowen has actually been doing things about this for the last couple of decades, and I'd love to hear your perspectives on why, you know, this idea of access to primary legal materials is important, and also just in general, what are the issues that you see from a government point of view in how government can start to act more effectively to open things up to catalyze activity in society rather than, for example, these sort of, you know, backdoor deals that end up favoring some small number of parties that's not creating that explosion that we see with open platforms. Great. Well, you know, I started into this both as a practicing lawyer in the 80s and then as a volunteer, I guess you would call them a community activist, if you're a lawyer and you're willing to do volunteer work, a seamless supply of clients, as I did. And somehow I tried to make my paying clients subsidized all the hours I used to do my free work. But the basic issues that I confronted, a lot of them were just, you know, when and where is a decision going to be made at the local level, and we were talking a lot about planning and land use decisions, by whom and then what information is available, what does the staff report say, because there may be issues that were missed, there may be factual inaccuracies. If you don't have access to that framework for decision making, you basically are just shut out of the process. And what that meant at the time, 1991, let's say, was you had to drive down to the city hall and look at postings on the bulletin board. And then, maybe if you were there when they were open, which if you had a job was no small task, you could get a copy of the staff report one day, maybe before the hearing, so that you were left scrambling all night trying to figure out how to best take your two-minute shot, which is what public participation generally is, in the city council or planning commission meeting. And the city of Santa Monica really changed things by putting up a dial-up bulletin board of planning commission agendas. And that was the one model that I was familiar with that gave people access. So at least you could stop running to Santa Monica city hall to see whether or not something was on the calendar. And it led me when I got to Sacramento in 1992 as a new legislator to realize that I had this computer that had all of the information about the legislature, all the votes, all the committee reports, everything, but you couldn't get access to it from outside the building unless you paid somebody a couple hundred dollars a month or a couple thousand dollars a year. But it was our data from that same viewpoint. Our tax dollars paid to create all this information. I think this for me, I started out with the bulletin board concept and a couple of guys from Silicon Valley and they were guys and then it said, no, you should make this a requirement that this information be available on the world's largest non-proprietary computer.  And at one point, they actually wrote a front end over the weekend because the cost estimate for doing this was eight hundred thousand dollars for taking the information we had available online. Sounds a lot like Carl's story with the SEC. Yeah. So they went home and wrote up front end over the weekend. It was kind of clunky, but we went to the Senate Rules Committee and that was the end of the eight hundred thousand dollar estimate. So it was openness in the process itself that actually moved the bill because I had a lot of resistance internally from people who wouldn't even set it for a hearing. Control over the information meant in the legislature that you, as the legislature, knew what information any single constituent wanted because they had to call your office and tell you what bill number they wanted. So you knew what everybody was interested in. If somebody had a sudden urgent two in the morning to look for something, all they could do was write a letter and put a stamp on it to ask for it. So that was, I think, that's one of the... Reducing participation in order to make it easier to control. And that certainly is true when it comes to decision making and input. It's way easier to get through a planning commission agenda if there are 10 people who are paid by their money clients to represent their interest and nobody from the public who might, no group of 70 people each wanting two minutes. Democracy is messy and it takes time and people have to make hard decisions and choose between competing interests. If you set up a system that eliminates a lot of that, it just runs a lot more easily. The most efficient form of government is a dictatorship. It's very efficient. There's one person, all the decisions get made that way. There's no participation, no appeal, but there are no processes that take time. You either okay the well in the Gulf or you don't. There's no hearings about whether or not it's set up to be safe. As it turned out, there weren't any hearings on that. Except after the fact. I was struck by the institutional barriers that there were and part of it was that Mosea can't do this. So it was all about transfer protocol and we were all saying I'm trying to configure our on X law protocols and failing as much as succeeded. So just the whole way of thinking in the way the government had grown up was that things became available on paper. But we were 600 years past food. So a lot of it was just that. It's not evil intention. It's just that's the way it's always been done. And similarly contracts when they were put out for someone to publish something had exclusive rights in them. So that California did not have any control over its code of regulations. We weren't allowed to distribute or print or put a copy of the California Code of Regulations online because there was a 10-year contract I think it was bankrupt with not sure. And we had to wait until that was over before we could put our own information online. But it's important because the whole concept of democracy is that it is complicated and as society gets more complicated the decisions and the dilemmas are more complex. But the idea is that in any corner of it there will be some person or group of people who are extremely interested in that particular thing. Whether it be a planning commission agenda or the rules for reimbursement of doctors for Medicare or whatever it is. And so that not all of us have to either be involved or be involved in a lot of things. Small groups of people involved in very particular things will actually generate. The long-tail theory. The long-tail politics is some issue that some small group really cares about. This whole system works. Now we have this greater challenge with tools that have moved us from I give you access to my information. That was the first stage of it. I give you the legislative information online. The huge battle over that bill was whether we were going to charge for it. The Senate put a hostile amendment in that bill that said we were going to charge for it. I think some bureaucrat had visions of a revenue source for the Senate. And I went home and had nightmares about Thomas Jefferson hawking the constitution on the home shopping channel. And I thought this is going to be a precedent for putting things online. It's never been done anywhere before. And if I allow this to be passed with a charging provision in it we'll never recover from it.  Unfortunately it wasn't necessary, but there's that piece of this too is that when you're doing this at the beginning you really have to be more careful about your compromises. It's everything that we do and now we're trying to figure out how to get people who are all interested in the same thing, not just to get the information and then provide feedback, but then to communicate this way too with each other and be more free flowing. It's impossible for any bureaucrat or politician to control that kind of energy. And I'm not even sure we know how to develop it in the best possible sense. It seems to me that there's a real challenge to develop institutions that work with some of this new technology. I think those of us who are technologists have seen this evolution happen again and again. And it gets misused terribly. We saw this with the World Wide Web where it became the blink tag. These web pages kind of flashed in that way. Eventually the market figured out that's not what we do. But even then we went down a certain path and it became more and more like broadcast. But then the market started to discover that there were different ways. They figured out that relevancy really mattered and they figured out how to apply relevancy to deliver better results became this priority of the company. Things changed. Now we're at that stage with social media. Big issues being raised about privacy. I think we're going to solve them. Being in a similar way with all this open government stuff, there's going to be a lot that goes wrong. There's going to be a lot that is hard. Everything from, hey, that discussion really did need to be private. There's going to be a lot of discussion about the constitution and the absence that we could never have done this to the public. There's a book called A Few Honest Man that kind of draws that out. We don't know the answers but I think we have to engage with how we move our democracy forward into a world of technology. We have hundreds of millions of people in this country. We don't have the way it was back in 1789. We have to figure out how to grow and adapt. As I told one senator who was in his 80s on that day in 1993, he said no one would use this. And I said it's not respectfully, it's not for you or even for me, it's about the kids who are in second grade now. That's long enough to go so that those kids are the kids who are part of the Obama generation. And they use it in ways that nobody can see. But we still have the problem of silos, the problem of government and bureaucrats thinking that it's their data and that they own it. Police departments thinking that the police reports are their property rather than public property. We have the challenges of figuring out how to handle error correction because there will never be a situation in which every single jot and tittle of government data that is made public is correct 100 percent. It's not in the private sector either but the consequences are different. I'm also thinking of the experience that Carl had when he was opening up various kinds of it was the congressional record so on social security record social security numbers of service members were in there. And that was always true. Transparency revealed the problem. It didn't cause the problem. And as a result of it being revealed it could be addressed. But there's a whole saying that it takes a human to error to really follow things up requires a computer. Yeah, that's fair. Because you can replicate the error at an alarming rate and speed. We have social security numbers in our UCC filings and I took down our whole UCC filings in 2007 for about three months probably one through redaction across and it's amazing what people will write it on the form where they're not supposed to. So you can't even do that with a computer. You have to have some eyeballs look at all the creative things that people did that aren't in touch. Let's come to some specific questions. So first of all I want to kind of get a sense. Is this a progressive versus a conservative issue or is this just non-political? I don't think there's anything that's non-political. But I don't think it necessarily divides right, left, or progressive conservative. And it also seems to depend so much on where you are. So when there's a struggle with this idea of particular access to primary leadership, any kind of open government. What are the fault lines? How do people come on one side or the other of that issue? What are the driving factors? We still have to get money from this or you mentioned control earlier. I can understand that the bureaucracies are drivers of control. But I think we then say the legislature. What are the sides? I think they are not much different now than they were in 1993. One of them is security. I had another legislator tell me somebody is going to get into this and change my votes on all the tax measures to look like on one of those goddamn tax and spend the liberals. And that could easily have gone the other way. So that is an issue. How do you know that the record that you're looking at is actually the record? How do you know that this? Are you looking at using digital signatures? I carried the digital signature built in California in 1995. We've had it for that long. I think the market went a different direction. We thought we would establish the infrastructure to take advantage of it. It just turns out that's not right. And you well know we have a lot of problems that we haven't addressed with terms of service being in a click through the reality of consent. It also comes back to what lessons can we really get from technology. You're right when you say that the market took a different solution. The market took the solution of error correction, not error prevention. If you look at how Wikipedia works you look at how Google doesn't give you the one true result. I'm feeling lucky it's not the default mode. It's actually hey here it gives 10 results. You don't like those? Go to the second page. I always go at least to the fifth page even if I look at it because I want something that's not obvious. Also thinking there's this great startup here in the Bay Area called CrowdFlower that works on using Amazon's mechanical work. Again it's a lot of the work that they do that drives their business is crowd sourcing error correction. Spam detection. In some sense I think algorithmic detection of forgeries is probably the way the market is going rather than over. Yeah and it is. I've made that point a lot with banking because banking people ask me about the e-signatures of competitions and you know can you compare the signature to a little scroll and what do they do with that banking thing and the answer is that is not how banks look for fraud. Banks to my knowledge never compare to scroll if you may know the grocery store with some signature they have. It's irrelevant. They're looking at patterns in the data. Absolutely. So and that's obviously less than the government needs to get with the technology. I don't know how to adapt that to things that have crucial impact to some person I mean with a banking area you can make up for it with money. We're not allowed to make up for an error in a UCC filing or a privacy breach or a privacy breach. Yeah the healthcare folks are resting under that. Yeah. I think we learned this morning that most cities sort of copyright over their laws and codes you know is that something that's just a matter of education or is there more to it you know I don't think here whether it's copyrighted or not as long as it's being accessible. I don't think that the legal argument about copyright is not that important. If it's there and it's accessible with no charge you got what you want. Yeah. And yet true it could be you know some administration in the future could change it so it's all better if you had the legal underpinnings correct. I think that follows. Once people get used to having it out and the world doesn't end and there's a lot of fear. When I first had an email address in 1995 for which I had to ask the speaker special permission because it had never been done. We were terrified. What were we going to get and how were we going to respond with it on more steps. We didn't print it on any kit for the first few months. I told the people what it was and I wrote it and they're still legislators who don't have email addresses because they don't want to deal with that. Or they have an email address that's behind this such a complicated web form that it's again a way of getting access. Yeah, so do we have any questions from the audience? Five minutes. Okay. Hi, I'm Brian Carver and maybe it's because I was on the IP panel earlier but I want to challenge the idea that free public access is all that really matters when it comes to these codes and so on. Yeah, that's really not what I meant. I don't think it's all that matters but I do think that in large measure where the law goes follows people's changing views about what it should be. That's why we're getting changes in gay marriage. That didn't proceed cultural shifts. It follows cultural shifts. Yeah, I guess the main one was if all we have is free public access and there is some assertion of copyright, then look back to Tim's vending machine model of government. The municipality has produced the code. Here it is. We've provided it to you. But that's all you can do is eat the candy bar and you're done. You can't reuse. You can't build services on top of it. It's not a platform that innovators can take and reshape or reuse in creative ways. Well, if the method in which something is made available precludes use in those ways then the statement I made is not applicable. It has to be made available in a way that is primary. We have a workshop in 2007, machine readable, machine processable but also with terms of use that allow that kind of creative reuse. And certainly people like West have lots of creative things they do with materials they get but that's because it's under their control. It's not really available for entrepreneurs to come up with new applications. Other questions from the audience? So let's kind of come back just sort of just to sort of your experience here in the state of California in so obviously a fairly trying time from a budget point of view. A lot of people saying this is fundamentally broken. How do you place open government in the context of the kind of breakdowns that we're starting to see in the accepted ways of doing things? Opportunity or threat to open government? Well, I don't think it's a threat because I think where we are in California is that the openness that we've created now cannot be taken away without a huge outcome. I don't think there's any way in which we would find access to the code or various other things that California could be taken down without a huge stink. And at this point the infrastructure is in place for those kinds of things. But budget does play an issue in at least my ability to go further than I have. And we have by comparison, California has a great financial disclosure from campaigns. It's not even near of what I would like it to be. But I'm not even going to ask right now for funding to redo that. I'm operating with 25% less budget than I had three years ago. And we're looking at core functions. But some of this openness actually is money saving. So the more I can get statements of information corporate statements of information filed online and made accessible online, the fewer people I need whose job it is to type in the contents of a piece of paper. So some of it really works together. And you can do the small pieces. What you can't do right now is a big new project for which there's not some dedicated funding. I'm more wondering if there's a point in where the operation of our government is so broken that we get an opportunity to think fresh thoughts. I had a wonderful Samuel Johnson quote where he said nothing to concentrate the mind like a prospect of being handed a fortnight. And yet it hasn't seemed to happen in California yet, despite the prospect of being handed a fortnight more than once. I think, you know, the system in California is so complicated that most people, if they start taking a look at it, throw up their hands quite quickly and say, I just don't have time for this. And I have to ask to what extent the initiative process, which is a kind of over government, is responsible for that level of complexity and whether there are lessons there for where, how you want to structure openness. And we have carved up revenue streams into this pocket, that pocket over here in such a way that I think when I left the legislature, the legislature had control of something like 11% of the general funding. So, you know, I think that is a key lesson from technology. Pretty likely you do need to do it over time. You don't kind of keep making your operating system more and more complex. It becomes more and more. It is like that. If you keep adding to the software, every suit it takes 5 minutes for the system to load and then something else comes along that people move to that. Unfortunately it's not that easy to change code. As we were all finding out. There are people like you who have made an enormous difference and I just want to thank you very much for the work that you do. I'm glad to see a large group of people who understand the value of doing this and that it's become really a part of the framework for the way we see who we are as a people. We see ourselves as having access online and we agitate if we don't have enough. It's just it's really great to see. We do have still a lot of lessons. We have a ways to go. Really harnessing the power of citizen input from multiple directions is something that I don't think I've seen a good model for use yet. But maybe someone who's listening to this can help with something like that. I certainly have seen some great examples in crisis response for example. If you look at the response in the state department very much, great example of how they harness social media how they harness interesting open source projects. They did lead the effort but they created context and supported it. Just amazing constellation of companies and tools brought together to really create a crowd sourcing platform to say there's somebody under a building right here. This guy from Patrick Meyer from Mooshahidi which is this project in Kenya that was originally designed for reporting election violence but has now turned out to be a crowd sourcing platform for originally any kind of important real time data that he found himself. The US Marine Corps is taking action for me because they didn't know that he wasn't in the country. It's astonishing in emergency situations what we've created but I noted at the Obama inauguration we had a bunch of people who were shut off into the purple tunnel where they stayed during the entire inauguration called the purple tunnel of doom they got told your line should go around this way and it never opened up. They never got back into the mainstream of the line and part of the problem there was that there was very limited signal access because it was in the tunnel. But it is the place where I want to see this develop more robustly is in the policy making context not in the response context but in the context of a discussion about for example K-12 curriculum which I think we've divided K-12 curriculum up in a way where instead of integrating civic education as part of the science curriculum so if you're doing a unit on climate change you then say well who makes these decisions and class everybody has to write a letter to somebody or we're going to do a class project on getting dump trucks in our town on something other than fossil fuel so that you link the underlying subject matter to what happens in our government political system the only way we're going to do that is if we get a lot of people who don't currently talk to each other talking to each other across a very broad spectrum of people and it's really a big policy change to change that in the civil and civic education classes so we give you all this information online but we don't tell you how any of it looks and what to do with it when you're in school so a big challenge ahead is not just access to the materials but also the hard work of actually making change happen educating the next generation of citizens of showing people what the laws and levers are that control our democracy so they can actually do something intelligent with it that's what I'm thinking about right now I'm past we have to provide access I assume that the places where it's not there that good citizens will rise up and that it will happen but then how can we take this technology and go one step further with it so that we have a healthier democracy with more people engaged and understanding what the issues are so that we don't get so many people saying well it doesn't really affect my life you know what I'm saying which I just always have to be very careful before I respond to that we all received our election packets recently from our signature audience they're black and white we removed the red and the blue ink and saved 625,000 dollars thank you very much


And welcome, my name is Shannon Kemp and I am the executive editor of Data Diversity. We would like to thank you for joining today's Data Diversity webinar, Data Exploration and Analytics for the Modern Business sponsored today by Looker. Just a couple of points to get us started due to the large number of people that attend these sessions, you will be muted during the webinar. For questions, we will be collecting them via the Q&A section in the bottom right hand corner of your screen. Or if you like to tweet, we encourage you to share highlights or questions via Twitter using hashtag Data Diversity. As always, we will send a follow-up email within two business days containing links to the slides, the recording of this session, and additional information requested throughout the webinar. Joining us today are Greg Jones and Scott Hoover. Greg is a lead developer at Smartling where he is responsible for reporting and analytics, as well as for the Smartling API that enables access to Smartling translation functionality from other applications. Prior to Smartling, Greg was lead software engineer at thelatters.com, a leading mobile career network and served in various management and consulting roles in the software development realm. Scott is a data scientist on the Looker implementation team where he helps clients build models of their business data and has authored SDKs that extend Looker's functionality through its API. Before joining Looker, Scott worked as a statistician and academia and later consulting. And with that, I will give the floor to Scott to start the presentation. Hello and welcome. Good day everyone. Again, this is Scott Hoover. I work at Looker on the analytics team. Here is a business intelligence software and more to the point, it's a data exploration solution that operates in database and it enables organizations to explore data in all of its detail. Looker has about 100 clients at the present, which one is Smartling. And then we're gonna be hearing about the, you know, the UK's Smartling has with respect to Looker. And so we're just gonna turn it over to Gray Jones, who's their lead developer at Smartling. Greg? Thank you. So I'm gonna talk about data exploration and how we're doing at Smartling. I'm one of the lead engineers at Smartling. I'm in charge of the reporting analytics as well as the API. Let's see. What about Smartling? So we're a translation management platform. We enable companies to bring their content to an international audience. Essentially, we want to speed up the time to market it takes for a client to bring content out to the rest of the world. And you do that for a number of different pieces of content, whether that is your mobile applications, your dashboard, your marketing website, any business documents that you may have. We basically have any number of ways that we can take content and trend it and get it back to the end client and eventually the users of your system. So first we have an API to get content in. So if you have internet XML files or iOS string files, we have an API to allow that content in. We also support various CMSs such as Adobe CQ, Sitecore, Drupal, and all these enable clients to bring content into Smartling where we can translate it. We focus on professional translation. We do allow machine translation, but are to provide a great timely quality translations and we get that best with human translations. And at the end, we're going to deliver the content back to our clients in its translated form. The perspective of our business is something we call the global delivery network. This is a way where we can enable clients to drastically cut down the time to market that takes to bring content to an international audience. SurveyMonkey is a customer of ours and we actually have their site in German, French, and Spanish and numerous other languages. And we do that through what is known as a proxy in which a request for say a Spanish site survey would come in. We make it to the original SurveyMonkey, we get that content in English. Parsit, look up the translations that have been provided by our human translators and return that translated content back in all about a number of milliseconds. So we can do this and drastically cut down the time to market. So instead of something taking a couple of a year or two years to actually go and redevelop your site, your application to get it to an international audience, we can do that in a matter of weeks. We do little to no development required. We have a number of data needs. So we also provide translations back to our clients within a 48 hour time period. That's the SLA that we strive for. So to do that, we need to constantly be monitoring and looking into how well is our translation workflow working. So quickly our translators translated the content, how quickly we move into the next step, how quickly are the views and the edits taking place on it before it eventually gets published. So we want to look at that across a number of different ways. We want to look with it for a customer, for a given language, for an agency who's working with us. There's many ways we have to slice and dice the data. And we want to give our business leaders control over that data. We want them to be able to look at the data and get the data out that they need. And what we want to do is, while reducing development, for the more we also want to provide a single view of the data, before embarking on this, we would be up to a developer to come up with his own query about how this data would look, or a business user writing their own SQL to get this data. And they can get different results. So we want to provide one centralized view of the definition of what this data is. So some of the practical needs challenges that we have. So we do an enormous amount of volume. We serve upwards of 3 billion pages views a month. There's 84,000 metrics for a challenge. And we're monitoring it at a given time. There's a huge amount of data, and we need to process that data and do it in a quick and efficient timeline. We also have our data spread across three different databases. Joining that data, at first, was problematic. We want a simple API to add that data, without having to go through a lot of development work to get it. So first, we used Amazon Redshift as our data warehouse. We used it as a copy or replica of our transactional data. We stored it there nightly. And what we found with Redshift is that it's extremely high format database. The query that would take hours to run in our SQL would take seconds to run in Redshift. We also decided to look for our exploration platform. We found that they were a great way to get data from the front of the business users and let them get to the data that they need. For the looker, well, first it was strongly aligned with the Smiling Principles. Continually to improve their product and deploy new features. We have a web company, so everything is done in the web. There's no console applications to install. They made things incredibly easy to do. Working with Looker, their support is excellent. It just takes a matter of seconds to get in touch with them and to be able to resolve the problems. They were extremely easy to use. A number of our developers have built out models using Looker as Scott will demonstrate later. And then our CTO, who hadn't done much development work lately, was even able to build a report in Looker. As I mentioned before, one of the things we wanted to look at was how well are translations moving through the system? We wanted to be able to analyze our workflow and our velocity, how quickly things move that workflow, identify bottlenecks. We wanted to see that because we didn't want SLA that we wanted to maintain. We wanted to make sure that if that is not occurring, that our clients are able to recoup money in such a case. So how smartly is improving the process? If we weren't using smartly, a lot of times a 40-hour turnaround would not occur. So we wanted to show that smartly is able to speed up the process. So here's a sample of a report that is taken out of Looker. This is translation velocity, how quickly strings were able to move from approved, get translated, go through a two-step review process, and eventually be published. As you can see, there's a number of languages for this particular client. We were able to meet our 48-hour SLA. Sometimes well before the 40-hours even up. There were two languages that, well, they didn't quite meet the SLA. They were just a few hours over. And then there were six languages in which the 40-hour turnaround did not occur, and even one language, Turkish, here, that far exceeded what we consider an acceptable time frame for turnaround translations. So this isn't indicated to our business users that there's some issue here. It needs to be looked at to find why are things taking too long? Is this that there's not enough context for translation, and therefore things are slower in the translation process? Are there issues being raised that are not addressed? There are things going on here that need to be evaluated. And right now we don't have the ability to go and look into the details of that, but that's something we'll be providing on the road. But this is a first step in identifying that there's a problem here. We need to do something to correct that. So another aspect of this is we want to see how well we're leveraging translation memory. I don't want to explain that a bit. When a piece of text is translated, we store that text in SmartLink to serve a translation, but we'll store it so that we can refer to it later in case a similar piece of text needs to be translated. So if I've already translated the color red, and then the color of the blue comes in, we want to show that previous translation to a translator so they can give it, they can translate it quicker. They can use same context or same tense as a previous translation, so it has a uniform look to it. We do that, and we can also charge less money, we'll pay less money to the client than charge less to the translator. We can reduce software costs. We can reduce the cost that the translator is giving to the translator. So that's a view of our translation memory. For this particular client, we want to match content 20% of the time. So 20% of the time, we had a similar upstream translation in which we aided the translator in the translation they were providing, and we're saving money to the client because of that. And so it range from anywhere from 20% to 50%, and lately there's been a decent upward trend aside from this last month. Now in a deeper dive into the data, we actually see kind of what type of things are occurring here. So for example, we have close to three million words translated. Now if there were no matches being done, we weren't doing that matching to save time and money. It would cost $421,000 for the translations. With charging a lower cost for that translation, we're going to reduce that cost to $321,000. So a savings of nearly $100,000. For this 18 month period, $5,500,000 a month that the server can save through our leverage of leveraging translation memory. The aspects we're using, Looker and Data Exploration 4, we're trying to apply at-risk customers. Those customers who have not logged into the system recently. That can be an indication that the client is not satisfied with smartly at the time, so we want to find out if there's an issue there. If they're not utilizing the full benefits of smartly, we want to know about that. This is a preemptive way to make sure that a client is satisfied. We also use different financial analysis as we show in the charging translation memory report. We want to see how is the software giving us money? How are we providing a benefit to the client? Furthermore, we've cut down on development time. We've done that. Before ad hoc queries were done, and each time a business user needed a report, it had to go to a developer to produce that report. Now that we've defined models in Looker, we've given the business users a way to access that data without needing development time. And we've provided one view of the data as part of that. Now we're still in our adoption phase, but we are proceeding with that. So we now have Looker being used by our customer support, our account managers, sales engineers, product managers, developers, our CEO who developed a report and even our CEO. We're at work with the option, but that's so far we have a pretty good amount of users who have started using Looker and exploring their own data. One of the things we've tried to do further in our weekly release notes, where we talk about the products or the new features that we've developed, we really include interest facts that we've discovered through Looker and through our data exploration. And so we include that as kind of a hint or a way for our end users to get into Looker and discover this data for themselves. And we're also starting to broadcast this data out so that it's in the office, on a wall board, you can see various metrics that we're tracking. So some future plans. So now that we've put data users in front of our business users, they're only open for more reports and more analysis at this point. So we can continue to develop our digital models in Looker to have different metrics that we can look at. Ultimately, we want to expose our data to customers. Right now, we're focused on our internal business users, but we will bring that data to customers in the end where they can view the data as they want and accept it on their own. And we basically want to do this so that we can show the value that smartly is providing them. They can see a report that would show them how we're smartly saving them money. So thanks for having us. Scott's going to give a demo of Looker. We're going to go into the context by just kind of giving a high level overview of what Looker is precisely. First of all, Looker is a web application that connects to any relational database, you know, any SQL compliant database by way of a JDBC connection. Having a connection to a database, Looker allows users, developers to build a data model using our intermediate modeling layer, which relies on a proprietary language we call LookML, the YAML-based language. This extracts and simplifies SQL generation. Basically, developers define the relations between tables in LookML. They introduce Looker to intrinsic columns or attributes within their relational tables. And then they can even build metrics or KPIs. And basically what they're doing is through LookML, they're writing LookML, they're teaching Looker how to jsql on the fly. Having this, having had the Looker analytics team and developers on the customer's end sort of build out a data model in our intermediate layer, business users and people who are interested in doing data exploration have a user-friendly web-based user interface that they can explore their data with. Three things in my demo of Looker. First, the Explorer section, and this is how users go about building queries and building queries against the database. Also, dashboard and visualizing queries. I'll also talk about LookML and embedding SQL, which Smarling makes great use of. There's actually a, we have this drive table, it's actually common in SQL kind of terminology. But basically, developers can write any sort of SQL they can think of within the mailing layer and exploit that within the Explorer section. Smarling makes great use of this and this is how the translation velocity report was created. And if time, you know, if time, I'd like to show some examples of LookML and demonstrate that it's rather expressive and lightweight, and so they turn out on introducing new KPIs or relations between tables. It doesn't take a lot of time or effort. All right, so I'm going to share my browser. So this is demo.looker.com, this is our demo instance. And for today's demo, I'm actually going to take a look at the look, which is the simulated e-commerce store that we have, but let me just give you a high overview of what you're seeing here. There are four main components of Looker, the first of which is the Explorer section, and this is where users go to do all this ad hoc querying against the database. Within the Explorer section, you can have a connection to any number of databases across dialects. So in this case, you know, we have five database connections here, and this varies across dialects. With any e-base connection, you can have any number of base views. And I'll dive into a base view and explain what is meant by this, but basically it's just a facet of your data or your business that you want to ask and answer your questions. The second component is dashboards, and this is pretty straightforward. These are just visualizations of queries that people have constructed within Looker. So after we go to our business pulse, which is just a look at what's been going on in the past 90 days for this simulated e-commerce store, you can have any sorts of, you know, visions all within one area. And our visualizations and dashboards also facilitate data exploration, as you'll see within the modeling layer, because you can actually, you know, apply filters which pass through all these tiles, and so all your visualizations update, and there's a drilling aspect to any of our visualizations or dashboards. More on that later. The third component is the looks page, and actually that's the page you're seeing now. These are just analogous to reports. So these are queries that, you know, are my coworkers have built in sync because we want to access them on a regular basis. You can also click and see, you know, what sort of reports your coworkers are building and collaborate, and the page, you can also manage the scheduling female, as well as the sharing of any look that you've built within Looker. There's the modeling layer, and I'll go into greater detail after I kind of demonstrate the explore section, so we'll get a good look at that in just a minute. But first let me go to the look and to our order items base view really quickly. So this order items base view and we have our empty or blank query canvas. And so this is the base view is this. Within the modeling layer, we started from a single table of interest. In this case, it's order items. And so we're gonna build queries to answer questions about order items. So we're gonna single table, and we find in others that relate to it. So for instance, each row represents an individual order item, but an item is associated with a parent order. So we're going to join in the orders SQL table and all the attributes that reside within that table to sort of slice and dice our queries by. Similarly, users place orders so we can join in the users table and use all the attributes about users. So if I'm interested in seeing facts about order items broken out by demographic or geographic information, this is how I'd go about answering questions like that. On the end, there are an order maps to a product as well as an inventory item. And so we've joined these tables in. Again, it's basically a denormalized view of order items where each row represents an individual order item. And we've joined all these tables and we have lots of attributes to slice our queries by. As suggested, there are two components of a query with a looker. Dimensions are basically just columns that are intrinsic to your SQL tables. So for instance, I can take an order items ID and run this. And basically, this is just the primary key of this table. Dimensions can also be calculated. So within the modeling layer, developers can write custom SQL or do arithmetic between existing dimensions or measures and create custom metrics. So for instance, this margin percentage is not an intrinsic column, but longer knows how to build this or construct this metric because a developer or a looker analyst has built this into the modeling layer. So the looker knows how to write the SQL in order to create this metric on the fly. So if any query within looker would be a measure and these are the aggregate functions applied to the above dimensions. Think of this as summing or counting or taking an average of any column within the database. Or rather, within this base view, any table that has been brought into this base view. So I'm going to start with some kind of basic principles and move on up. So I'm going to start with a rather basic query here and I'm just going to do a series of aggregates. So I'm going to take a look at orders count, order items, products, orders, users, brands and categories. So I've added these measures to my query canvas and now I need to run it. So what I'm doing is it's being told that the user wants to see these aggregate measures. It's going to construct the SQL, send it to the database and return the results that to the user. And I also noticed that when I ran this query up on the left hand side here at the top, it threw up a filter. So what we're seeing is orders or rather, aggregate results provided that the order was created in the past 30 days. So they originally set up this model thought it would be a good idea to do this to return recent and relevant results. Our date filters are also, you know, they can be changed on the fly. So what we're doing here is results that for today and the 29 days prior, come tomorrow, it will be tomorrow and 29 days prior. We can update this to reflect something else. So something like, suppose I want to see the past 90 days. I can just type in 90 days. So for business or people who don't write SQL on a regular basis, it's rather convenient to use natural language. And it's very intuitive too. So if I want to do something like 60 days ago or 30 days, I can isolate this 30 day block that began 60 days ago. And of course, this will be on a rolling basis. It's relative. And the answers have autofill. And in this case, the dates there will kind of give us some insight on what we can do, you know, what we can apply as a filter. So this query, I'm just gonna stick to this month. So this, there have been roughly 1,500 order items representing 1,400 products. This amount came from 895 orders made by 798 users. This represents 666 brands and 26 categories. I'm gonna break this out by some dimension. This is basically just group by in SQL. So suppose I want to see how this differs over some geographic attributes. Let's take a look at user state. I want to associate with the largest number of orders or order items or products. We can get any dimension or measure by just clicking on its name. So if I want to just sort this alphabetically by state, I can do so. I can also choose, you know, how about orders count and see which state is associated with the largest number of orders. I did that. The order was placed this month. We see a lot of California. So I want to isolate California and maybe build a report around this. I click on California. And what that will do is it'll actually throw up that value into a filter for me, such that any subsequent query that I run with a looker will be limited to orders created this month provided that the user who placed that order was in California. I'll see how I alter this filter at the top. I'm gonna go ahead and remove the state because we're always talking about California. And maybe now I want to break this out by time. So I have two ways of doing that because I want to break it up at the day level. I can go to filter, which is already in my query canvas and add this to the table. Alternately, I can just go to the left-hand panel where all my dimensions and measures reside and I can just search for it. So I want to see created date, order created date and I'll add this to my query run on it. I'll sort in a central order. There we go. So this bike or let me filter this and see which days is associated with the largest number of orders. I see that it was part of me. Yeah, May 20th. So suppose I actually see what's driving this. Looker is all about facilitating data exploration and this is allowing users to drill into the raw data. So any account within Looker is drillable. So I want to see the 13 categories represented by these 10 orders on the 20th of May. I can simply click on that 13 and I'm taking this user-defined drill set. So the value of the count was 13. Intuitively, when I'm taken into this new window, I'm going to see 13 rows that represent, which row represents the values captured by that count. So the categories represented here are the results that we have 13 rows with each category there. And we have a number of measures or dimensions that the analyst or developer thought would be intuitive for the user to see whenever they click on accounts. These results are not static though. I can always update them. So suppose that I wanted to add gross margin to this. I can just add gross margin and rerun this. That Looker isn't moving your data. It's actually piecing together SQL and not against the database and returning it for you. So I'll actually jump over to this query SQL tab here. To demonstrate the necessary SQL one would have to write in order to reproduce this query by hand. Pretty quickly, things can get intense. So you see that there's a lot of value added if you're in a case where analysts are using requests from business users to get to certain data. Now you've actually empowered, or rather Looker, empowers business users and analysts to actually go out and ask the questions themselves. And then you have to write the SQL, which could be rather tedious. I want to demonstrate our ability to pivot within the user interface because it's rather useful. And then I'll transition to visualizations and lastly the modeling layer. So I'm going to enter this query and start again. I want to actually ask the question, or rather answer the question, how do users behave with respect to their ordering depending on the month in which they signed up? So this would be a rather straightforward analysis. This is just a cohort analysis. To do it within Looker, it's actually fairly trivial. So I'll take a look at a couple of time attributes. I want to see order created month. I'm going to apply a filter here. I want to see the user created month. I'll match on this one as well. And then I want to see the number of users who placed orders and the corresponding number of orders. So that's past eight months. I'm going to hide the left-hand panel, and I can sort these so they're a little bit more intuitive here. The whole set is what we might get in SQL. What we have is every pairwise combination between user created month and subsequent order created month. We have the number of users who placed those orders. It might be a little bit easier to actually kind of consume this data if each cohort was represented by its own row. And to do that within Looker, all we have to do is pivot one of these time attributes. And in this case, I'll choose order created month, I'll pivot it, and then rerun this query. And that would be kind of easy to consume matrix or tabular results that, or each row represents the cohort, and the pivoted time dimension represents their subsequent behavior with respect to ordering. So that's what we have to do. For those of you who signed up in October of 2013, 18 users placed one or more orders in that same month based on this pivoted time dimension. We see that they placed 90 orders. Of that cohort who signed up in back in October of 2013, 130 users placed one or more orders in the following month based on the time dimension pivoted here. And we see they placed 148 orders in the same way. And then we have some null values here for subsequent rows indicating that cohort up in November or December could have placed orders in the month before they signed up. We can always add stuff to this. So suppose that I wanna see profit per user, I could run this, and now I've just updated my result set to have this metric, and we can look at the value of these customers that are basically our strong repeat customers. Again, we can also drill in. So if I wanna see who these customers are, I can actually click that 43, and I have this user defined drill set. So now I have my list of our most loyal customers, and perhaps I can do some further analysis and glean some insights about attributes about these users who return to the store often. Let me demonstrate creating a visualization on Looker. It's rather straightforward. All I want to do is construct a query of interest and then jump over to our Visualize tab. I'm actually just gonna build a bar chart of orders by state, something rather simple here. So I'm gonna do orders. How about a break this out by user state. So here's the result set. It's actually for the past 30 days. That's fine, I'll leave it at it as is. We can see California is the top state with respect to the number of orders, whereas Nevada is the top state with respect to average gross margin, but we're talking about two orders here. Anyway, let me sort this. So suppose I visualize this. Here's that SQL tab. So this is the query generated by Looker. And then I jump over to this Visualize tab. I can just choose the most appropriate visualization for this particular query that I've built. So we have the area charts. Bar chart might be very appropriate for this one. And this is actually kind of how the smartling team went about visualizing their translation velocity. So all the developer had to do was come up with a way to measure translation velocity, which would appear as a measure. And then they brought it up by client and then visualized it just using this visualization tab. We're gonna call them line charts in the event that you want to do a time series. Starts high end tables as well and a single value in the event that your result set is just a measure with perhaps some filters applied. So again, just a collection of these visualizations. So unless developers and business users have gone around querying, identifying result sets that they want to see on a regular basis, they can see them as looks or they can add them to their dashboard. So I'm gonna return to this business pulse, which is just a collection of these visualizations. So in the previous part of me, 90 days, we can see that in this simulated e-commerce store, there have been about 3,100 total orders with an associated average order profit of $55. And we have 1,000 first time purchasers. Again, the looker is that much like it's tabular result sets, you can drill in. So if you're in a time series and you see a spike or perhaps a big dip and you wanna see what's going on beneath the scene, you can always drill in. If I wanna see the seven sweaters that sold back on March 26th, I can click on that seven. I'm gonna use this user defined real set. Because in between queries, it's as simple as using the back button, so that's rather convenient. And we have visualizations too. So we have geographic plotting, so we can have zip represented. And again, we can drill in, see the order items or orders associated with any given zip code or a PUS, donut charts. And then there's that actually any dimension definition can include embedded HTML. So notice that from the sync board, we have a brand name. It's just the top 10 brands for the past 90 days with respect to various metrics here. Suppose you see a summary about our top brand. This little spark line to the right will actually jump me to a new dashboard and automatically apply the filter Allegra, rather within the name filter, it will pass the parameter Allegra K for me in this default through orders in the past 30 days. And basically I have a summary of that brand for the past 90 days. Again, I can demonstrate it with Calvin Klein. This is in other definitions of dimensions. So actually, if I return to this result set, where I order items, we have a user history field. And so this is not an intrinsic column in the table, rather what it does is it jumps us to the orders base view and automatically applies a filter, maps to the user that I was clicking on. So I clicked on an order item associated with Jasmine. I wanted to see her a complete order history. I click on that order history and I jump to the order space view and a filter is applied for me. We can go with the items. Actually, you can get quite creative with this. So I think this is to transition to our base view, rather our modeling layer. I'm gonna go to the model section. I'm actually gonna jump into developer mode here.  So, primary file types that we focus on when we're developing a model. There's the model.look.ml file. And this is where all the joins are made explicit, such that Looker knows how to piece together joins on the fly when users are calling from various tables. The second type is a view file and this maps directly to the SQL table. So for instance, the order.view file has all these dimensions, which basically map to the intrinsic columns in the table. But you'll notice that I have embedded SQL within this file and there's also arithmetic between existing dimensions. This is how developers and analysts can sort of embellish the default model. That is, they can use intrinsic columns and then build upon those, such that while users are exploring in the user section, they can use all these custom metrics that may be pieced together around complex SQL, but they may be unaware that it's actually doing that for them. This is the model file. And I just wanna walk through the definition of a base view. You should have an SQL example as well. So, just turn your attention to line 10 here. We're just declaring a new base view. So actually, I'm gonna go back to between the SQL and the Looker.ml. Basically, when we declare a base view inventory items, this is the table made reference to in a from clause in a select statement. We're also gonna be able to select any sort of combination of dimension and measures from the base table as well as any tables that we join in. But that's beyond the scope of this call. Or this webinar part of me. Next line is just declaring a list of joins. So, I'm gonna join in the following tables, the first of which is products. And then all I do is make reference to the foreign key in the table, in the base table. So, basically, what this implicitly is doing is it's mapping the foreign key within the inventory items table, and it's setting it equal to the product ID, or the primary key of the table that we're joining in. There are other custom join types that Looker has, but Looker really plays well with data stored in third number form, and by default, we're talking about F joins here, but there's a high level of custom, you can customize any sort of join here. So, we have SQL on arguments, where we can specify shared columns, or conditional joins in the event that you have polymorphic tables, and we also have just the ability to write custom SQL here, and so now, you can really just write any sort of SQL that comes to mind, so you can do joins, full outer joins, with all sorts of interesting conditional or join logic. And then below, this SQL is basically, or yes, this SQL is basically kind of what we'll get when we declare a base view. So, we're selecting dimensions and measures from the combination of tables that we've joined in, the base view itself is the table in the from clause, the join in are left joined, so we're going in products. The foreign key maps the foreign key in the base table to the primary key of the table that we're joining in, and then in square brackets, I have all these optional SQL clauses, so a where and a having clause map to filters within the explore section, and a group by would be just a dimension in the event that you have a combination of dimensions and measures, and an order by just, you know, click on the dimension or measure name. I'm gonna show you orders, oh, I should comment this out. I'm gonna show you a review file, and again, kind of just quickly go over what makes a view file or a base view. So, a base view has to make reference to a view file, and a view file will map directly to an intrinsic SQL table, but they could also be a derived table, which actually is beyond the scope of this webinar. But basically, it's analogous to a materialized view or some sort of SQL that you're writing on the fly, and it's transforming your data into some table and exploits, and anyway, I'll let that. Anyway, so as I've mentioned, basically maps to a common in the table. So in this case, there's a primary key of our orders table, it's the ID, and so we're gonna bring you this in, or rather make reference to that table or that column in that table by just declaring a dimension ID. It's the primary key, so we're saying primary key is true. It can specify the data type. So in this case, the primary key is integer, and there's this SQL argument, and what we're passing as a parameter is basically what I kind of say is like a handshake between looker and data. So it works, whenever a user calls the orders ID within the explore section, look to the online table, in this case it's orders, and look to the column ID, and it knows which data to kind of pull in and display it whenever they're poking around and choosing various dimensions. This created time attribute. In this case, we're looking to the orders table and the data at the date time field. That's actually interesting. What we're doing is we're creating a number of dimensions kind of in a bulk process. What we're doing is we're actually extracting various time attributes from the single time or time stamp. So we're looking at the created time, the created date, the created week, the month, the month number, year, and the date and week number. All from the single time attribute, we can kind of parse out these other aspects, these other attributes of time. There's license types of data by. So if we wanted to see the number of orders at the week or month level or by day of week, we can do that with this single time attribute, really. Also, we have embedded SQL. So for instance, total amount of orders in US dollars, this is not an attribute that resides or that is intrinsic to this orders table. This is actually calculated by doing a correlated sub-select between the orders table and the order items table. So basically, we're summing up the order items sale price, provided that the order ID in the order items table is equal to the orders die-die-die-die. So, not only does SQL dialects have this capability, and you'll notice that Looker actually plays well with all sorts of dialects. So you can write SQL that is dialect specific. So in the event that you have a PSQL dialect or something along those lines, you can utilize, or TSL, you can utilize window functions or something along those lines. If you have myQL, you can utilize correlated sub-selects and all that great stuff. And this is all specified within the admin panel. When you make a connection to a database, you specify the dialect, and it's that simple. So I'll wrap it up for the demo and then I'll transition to a Q and A. So I'm gonna stop sharing my browser. And I'll just return to my slides here. So I'm gonna ask the questions here. We're not able to share dashboards or data with people who don't have a Looker login. We do have the ability to share dashboards and queries, tabular results sets with people who do not have a Looker login. I did not demonstrate that in the demo. And this is a rather important component of Looker. Scheduled emails or scheduled looks, rather. You can email out tabular results sets. You can also send out dashpiles or entire dashboards. And we also have read-only dashboards or embedded iframes to present visualizations to people who do not have a Looker login. We also have an API. So we have a number of ways of getting Looker out, I mean, sorry, data outside of Looker and getting it into any sort of dashboarding system you have, and we also have integration with Google Docs and Excel spreadsheets. So you can actually take a public look or public save query and kind of pipe to a spreadsheet and then that spreadsheet around. And the great thing about that is that that result set is actually basically a live query against the database. So you can, it will all reflect new data as that data has changed on the database. I have a question for Greg. He covered this, but I think maybe he can speak more to it. Greg, what is your infrastructure or database? Do you have any other systems that are connecting to Looker? I don't know, it is simply Redshift. We could bring in our MySQL transactional, but pretty much we'll do a data copy one today to Redshift and use Looker off of that. Do Looker's documentation or reference guide users to refine data intake and analysis, i.e., what are noises and what are early warning? I don't think I've seen that question. If that was your, if you ask that question, what are noises and what are early warning? If you know how, if you would elaborate on that question, I'll, I could circle back to that one. We also have a question, how does Looker deal with non-relational data? So, you know, like we can talk like no SQL. Looker is first and foremost is like a, you know, a tool for relational databases. So it's basically all SQL compliant databases. However, we recognize the importance of SQL, it's actually, you know, the importance of the day and age. And so we're working on builds for Impa, Spark, or Shark. And so these will be basically HiveQL. We're looking for engines that, you know, increase query performance for HiveQL. So we build and we don't have customers on it right now. We're still in the kind of the development phase, phase in testing it, because we want to make sure that it's as performance as our relational solution. I speak to this and I think Greg will be able to speak to this one as well. What kind of training is required for this type of analytics? Based on the answer, well the answer is it depends. So, you know, users who are using, predominantly just using the explore section as well as dashboards, they just have to be data curious. They don't have to have a superb understanding or actually rather any understanding of SQL or relational databases. They just have to, you know, have questions in mind and have the curiosity to sort of answer them and poke around and build queries within the explore section. From a different standpoint, we like to identify someone who can own the model and that typically is someone who has experience with SQL.  So, you manage by people who aren't SQL experts, look at them as themselves in abstraction of SQL. And so, it's best if one has an understanding of SQL going into, you know, model development and look at her. I don't know if you can, you know, add to that, Greg, if you have any perspective on it. I'd say that we did our training much how you describe. We had a couple developers and they received the handle modeling layer and they did all of the defining of models and our end business users, they weren't involved in the model creation but we did a number of training sessions just to kind of get familiar with Looker, how to use it. Excellent. How can I incorporate Looker into existing web app? And in friends, we have ASP.net. So, Looker to have an API and we have a number of SDKs. We have Ruby, Python, Node.js and we also have an R package. But basically, you can utilize the API to sort of, you know, core existing data model and take, you know, result sets and they'll be JSON blobs and present that however you like within your application. We have a number of clients that do that. It's kind of one component of how they use Looker and we have a couple of clients that all buy entirely on the Looker data model in conjunction with the use of the API and they get a great deal of value out of it and they actually don't use the Explorer set all that much, it's actually rather interesting. So we see different kind of configurations or use cases for Looker. So, here you have the API data in SQL Server. How many Windows as you are? Yes, yes. Any SQL compliant or relational database that can, you know, by way of a JDBC driver, we can handle and actually we use, we connect to the SQL or, you know, SQL Server all the time. Let's see, other questions. So there's that first question about Looker documentation and I was not quite sure on the question but, you know, going off the first half of the sentence, do you guys have Looker documentation or guides for users? The answer is yes. Within Looker, the Looker instance itself, there's a link to the Doc section and the Doc section is an entire site on getting this users and analysts or developers up to speed, either checking data in the Explorer section and creating dashboards or the developer kind of paths, you know, how to get up to speed developing in LookML. A reference section for, you know, API operations and security, drive tables and then just a kind of a LookML or dashboards reference section. And lastly, a video library, which is actually really important in the event that you learn visually. We have a solid kind of training section for our video library. Okay, this is the last question. How long does it take to implement? That's a good question. It depends. So it depends on the complexity of the data. It depends on how much ownership did it on your end take when on the model. With that said, I actually work on the implementation team, so my job is to sort of fit out and deliver a model that is very much workable to the client. And again, that it varies on how long that takes. With that said, Looker generates a great deal of the model obviously for itself or programmatically. So I'm actually not sharing my browser anymore, but there is a section where you can actually just generate a new model. And what the generator does is it takes a view of the database that we're connected to, all the tables that are within that database, and it will generate individual view files for each one of those SQL tables. It will apply all of the intrinsic columns in that table and interest them as dimensions. So all the lines of codes where we were declaring dimensions, unless it's a calculated dimension, that is all done programmatically at the outset of a trial or implementation. And then actually Looker is smart enough to do a lot of the joins and the creative base use by itself. So prior to that, foreign and primary keys are named intuitively, and there aren't like table prefixes in the event that you have a jail or something along those lines. Our generator is smart enough to actually make these joins and declare these base use for you. So we actually just spin up an instance or spin up an instance of Looker, generate a model once the database has been connected to, and users are actually a rather workable model that they can use right out of the gate. And then Looker analytics team, as well as the developers on the client side, they just take ownership of updating the model and adding any custom metrics that users want to see when they're exploring the data. I think that's it. So I'm going to pass it back to Shannon. Thank you all for your time. Perfect. Greg and Scott, thank you so much for this webinar and the information education and the demo of the product. Always good to get some demos in for our subscribers and thanks, of course, to our attendees who always submit and are so engaged in submitting some great questions for our speakers. Hope everyone has a great day and just a reminder as everyone that we will get a copy, a follow-up email for the webinar within two business days. So for this webinar, by end of day, Monday, with links to the slides, links to the recording, and additional information so you can take a look at Looker yourselves and what they're doing. Hope everyone has a great day and thanks again for your time. Thanks, Greg, and thanks, Scott, for another great presentation.


All right, Bola Buddies, I am going through a bunch of vintage jewelry right now. I'm going to give you a sneak peek, but this is not going to be all I'm bringing, and it is going to be a fun show. So bookmark it. I will see you guys tonight at 9.15 p.m. Eastern Standard Time. Bola Buddies on what not. Okay, so I still need to go through this bag. Here are some of the beautiful vintage items that I have laid out right now. I need to go through this bag, so I will see you guys tonight. Oh yeah, and I will also have Crafter's Lodz.


Now the big decision moving forward for the Tampa Bay Buccaneers, who do you go with that quarterback? At this point if I'm in charge, I'm going with Ryan Fitzpatrick because James Winston just does not understand the value of the possession at this time in his career. He continually talks about how important it is but he continues to have games like this for interceptions today. I think they're all on his shoulders until he really proves to me that he understands the value of the possession. I'm going with Ryan Fitzpatrick.


This is a Weird Darkness bonus bite. With a little help from Hungarian scientists, the seemingly impossible just occurred. The birth of a long-nosed, spiky-finned hybrids of Russian sturgeons and American paddlefish. Though the results were accidental, about a hundred of these so-called Sturdlefish are now in captivity. According to Live Science, the Research Institute for Fisheries and Aquaculture in Hungary, scientists didn't intend to birth this entirely new species. After all, 184 million years of evolution on separate continents made it seem clear that these two kinds of fish were sexually incompatible. After placing sperm from an American paddlefish near eggs from a Russian sturgeon, however, they were proven wrong. According to Science Alert, the eggs reproduced asexually through Ganajesus, a process that requires the presence of sperm but not the actual introduction of its DNA. Published in the Jeans Journal, the findings reveal utter shock at DNA being transferred after all. The two endangered fish species now find themselves with a new cousin, despite the intentions of those overseeing the experiment. Life, as they say, binds away. According to IFL Science, the goal was to confirm whether or not introducing asexual reproduction into these fish species could help protect their dwindling populations. Both animals are endangered, with populations plummeting rapidly due to pollution, overfishing, and habitat loss. Hybridization was the last thing on their minds as scientists were well aware that the two species are further apart on the evolutionary tree than humans and mice are. The result was extremely surprising to say the least. We never wanted to play around with hybridization, said Attila Mozart, a senior research fellow at the Institute. Made up from the gametes of seven individual fish, the hybrids exhibit varying ratios of features from both species. The offspring are also aesthetically perplexing, with some partly resembling their father and others their mother. They have sturgeon fins and snouts, but paddlefish mouths and appetites. I did a double take when I saw it. Solomon David, aquatic ecologist at Nicholas State University in Louisiana, told the New York Times, I just didn't believe it. I thought hybridization between sturgeon and paddlefish? There's no way. Both the sturgeon and the paddlefish are among the largest and longest living fish species in existence, with their prehistoric divergence from the evolutionary tree marking them as living fossils. The last time they shared a common ancestor was when dinosaurs roamed the American continent. Typically inhabit the riverbeds, seafloor, and lakes in Eastern Europe and the Middle East. They don't have a presence in the United States, not even as an introduced species. American paddlefish, meanwhile, use their long snouts to dig for food in American rivers. These phenomena could lead to a higher similarity, compatibility, and flexibility among the sturgeon genomes and allow the hybridization between Russian sturgeon and American paddlefish despite the large geographical, physiological, and morphological distances, the study argued. As such, the success of this hybridization is quite the scientific achievement. Previous efforts between similarly distant families of animals have failed completely, but new turtlefish individuals being held in captivity have survival rates beyond 30 days that range from 62 to 74%. The researchers said there's currently no intention of creating any more turtlefish, nonetheless continuing to study how the two species that birthed them reproduce is certainly on the docket when they focus on finding out how to best protect them from extinction. With the International Union for the Conservation of Nature confirming earlier last year that sturgeon are more critically endangered than any other group of species, they hopefully will.