Yes. So anyone who's thinking about escaping out to the lobby because maybe the bar is open by now, I have bad news. It's not. And it's not until I say it is, because I'm the co-director. It's good to be the co-director. Come on. Come on. Come on. All right. I'm not going to use this. I have a mic. I'm wearing a mic. I know because they put it on me. So as Laura was saying, last session of the day, everyone's a little tired. I can see it in your faces. You just want to go out and have a drink. I understand. So we're going to make this a little bit more fun. We're going to do a little cybersecurity scenario improv. Okay. So y'all, has everyone been to an improv show? Everyone? Yeah. Okay. So first thing we need is like a couple of ideas to get us started, right? So just holler them out, right? So anyone, can anyone give me a name of a country that you haven't read about in the news this week? Last week? Nigeria. Okay. Great. Oh, come on. Princes. Too easy. All right. Give me a household appliance that is not currently IOT enabled in your home. Blender. Blender. That's a good one. I like that blender. Perfect. Is this Mad Libs too? It's Mad Libs. You've done this before. And then finally, somebody give me the name of, let's see, a washed up, a celebrity who will be washed up in six years. Kim Kardashian. That's good, but you're on the panel. You're allowed to do that. Come on. So that was an easy one. Anyone? That's fine too. Beaver. Beaver? Oh, Beaver. Justin Bieber. Got it. Justin Bieber. All right. Great. So, we are in just a moment going to fast forward ourselves into 2023. And for the remainder of this panel, we are going to discuss a cybersecurity incident, which I will tell you about in just a minute, as it happens in 2023. And we're going to look forward from that and we're going to look backward from that. And so, I also, I still need your participation, please don't tune out. If you guys, in the course of our conversations, have interesting twists on the scenario that you think would be fun to throw at us, or if you've got an elaboration of something that's already been said that you want to, like, you know, have us think about, please tweet it to at NewhamCyber, and these fine diligent compatriots of mine in the front row here will holler that we're going to be able to do that. Here we'll holler them at us at, I hope, in opportune moments. With that said. We're not scared at all. No. So, with that said. Okay.  And imagine, the year now is 2023. As predicted in that, what seems like far off land of 2016, 2017, the internet of things is really just the internet now.  Some of these devices are very well secured, they've been made by companies who know what they're doing, and some are not, shall we say. Same problems that led us to the Mariah Botnet of last year are still prevalent in 2023. Best practices exist, consumer reports, rates, devices based on this, but most people don't care, and they buy whatever's cheapest. Sometime in 2022, a spec device manufacturer, that is someone who makes devices on, a company that makes devices on spec, manufactures them in Nigeria, is hacked by an unknown party, and specifications for all of the IOT devices that they've made over the past few years are stolen, including some of the, you know, hot new Christmas toys and other devices that are going to be on the shelves come December. Those specifications are sold on the black market to a high bidder with unknown backing. In looking at the specs, the group that bought the device, sorry, the group that bought the specs discovers a vulnerability in the firmware code and exploits it to gain access to all of the devices made by this manufacturer, and putting it together with, already on the open source malware market, they put together a piece of malware that is released sometime in December 2022, and begins very quietly infecting everything it can find, and it doesn't do any DDoS, it doesn't send any spam, in fact, it's very, very silent, it merely tries to infect other IOT devices, and it hooks into whatever video and audio sources it can find, and starts recording snippets determined by an AI, which essentially only records when there's people talking, or images of people if it's a video camera, and these snippets are encrypted and uploaded to various cloud storage devices, and also to many peer-to-peer data networks such as BitTorrent or whatever they're going to have in 2023. It doesn't take too long, a few weeks maybe, before security researchers start realizing that there's odd traffic patterns happening, and before too long into January or February of 2023, forensic analysis has revealed the purpose of domain malware. The group that wrote the malware when they realized that someone's onto them releases the decryption key, and a search engine that basically using either voice print analysis or facial recognition allows you to essentially search the corpus for people you know. And cloud hosting providers who realize that this data is on their servers quickly take it down and delete it, but it's on peer-to-peer networks, and really you can never actually delete everything from the Internet ever, and I'm sure we'll talk about that. You know, it's mirrored by sort of 4chan-style vigilantes who think that this is the most hilarious thing to happen since 2016. So video and audio of millions of people's private lives are now online for everyone to explore. The data is tagged with date and time and IP address, and from there you can determine rough location, obviously. And so people begin skimming the data dumps, and very quickly a number of controversies as you might imagine crop up, including one involving Justin Bieber that proves conclusively that he is actually a woman in drag and that he doesn't write his own songs, and so forth, you know, embarrassing things like this. And so on and so forth, you can imagine sort of what is happening here. And there's a couple different things you can sort of insert here that I think make for interesting pathways for discussion. But the one that I sort of hooked on to was, you know, after all this has occurred, the group, you know, takes the Twitter and claims responsibility and says that they were trying to demonstrate the insecurity of the things that we all, these devices that we all use on a day-to-day basis. And now everyone knows and hopefully some good can come of it. But anyway, that is our scenario. Betsy, what happens next? The Cyber Apocalypse. I thought that's what I just described. Yeah, exactly. Well, so I think what happens next is a real change in the average day-to-day life of people on the internet. No, so I admit I get to cheat a little bit because the organization I run the Center for Long Term Cyber Security developed scenarios for the future of cybersecurity in the year 2020. And one of the scenarios we developed was about what happens if the hackers win. And in essence, that's what you've just described in a sense is a particular vector in which the hackers win. And we think that people would do three things on the internet if the hackers win. Some people would just say, okay, you know what? The threshold has been reached. No longer do I think that my interactions are going to be safe online. I'm going to assume that they're not going to be safe. And so I'm going to take everything I have and just put it on the internet so there's no question about whether or not you're going to release it. It's just out there. So that's some people. Other people or potentially people in different portions of their lives would say, I'm just getting that stuff offline. You know, I don't want anyone to be able to touch these sensitive things. So maybe online banking, no thanks, I'm going back to paper, maybe medical records, I'm not going to my health insurance anymore through the internet. So that's the second possibility. And then I think the third and potentially most difficult is that some people want to say, I still want to be online. It's so important to be online in some spaces. But I need to find a way to protect myself within that space. So we sort of use the analogy of gated communities and neighborhoods.  And people would begin to exchange information. And so the key question is, how do you get from there to here when you're surprised by this big event all at once? Because up until then, what you've described looks pretty similar to what we have today. Right. And so the interesting question about the sort of gated communities, you know, can you create that? Is that a technological gated community? Or is it a policy gated community? And can you create that through policy at all, Alan? So first with the caveat that I'm not speaking on behalf of NTIA or the Department of Commerce. I think it's actually, it's a technical question that is quite similar to something that we've already talked about in some of their cases. Because the use of private information that I have, and a foreign server has, and the assumption that no one else has this information, looks an awful lot like public key crypto. And one of the things that we've talked about is what happens when public key crypto breaks. NIST has publicly said, please stop using exponent based encryption algorithms, please switch to ECC, elliptic curve encryption, and use these standards. And we know that there's a decent chance, and in fact, at RSA, the crypto panel said, we think there's a decent chance that public key crypto, as is currently used circa 2013, will be broken in five or 10 years. But symmetric key encryption is going to stay valid. And I think we're going to see that when it comes to authentication, because it's not just the users that are at risk. At commerce, we think about those poor, defenseless, large enterprises out there.  And if now they have no ability to verify that the person at the other end of their exchange is actually you, how will they make sure that the server is going to work. And in both cases, we're going to need to return to something that looks a lot more like symmetric encryption, whether that is, we're going to give you something, you know, we're messing or everything to you at the end point, or we're going to have scenarios where we're going to have short term temporary keys. But we're going to have a certain cost of delivery. It's clearly going to raise the cost. The question is, can we still have a remote mechanism at all? Or are we going to start to use, you know, kiosk based models for every kind of interaction we want? And that's actually that raises an interesting point. And I think this is sort of you are edging this direction.  The nice thing about it being 2023 is we've gotten really good at CG. And so why not corrupt this massive data dump with false data? So if there is something that's truly sensitive out there, you can imagine those of us with resources can then poison the well. And it will take a little time to do it in a sophisticated fashion. Certainly the security researchers among us will be trying to determine the real dump from the fake dump. But to the point that it's been raised by a number of people today, that's also something that is going to be resource constrained. Right. That sounds like a business model. Exactly. And so this is going to be an area where you will have privacy and security if you have the means. Right. So that sounds a little bit to me like hackback. Because under so I'll play the government lawyer in the room, even though I am no longer a government lawyer, you're tired from that position. And by a few years now, what authority would you have? How would somebody lawfully poison that? Well, it's interesting. What would be what's the what's the law preventing you from poisoning the well? That's interesting. Well, is it is it a CFA violation if all you do is upload another video and say, Oh, I don't know. That's loading a new video. Right. Yeah. Well, so are you taking an old video or uploading? Well, so actually, I want to talk about the hackback question, too, because, you know, you can like this is prime vigilante stuff here. Like, I can imagine myself getting really pissed and wanting to go after somebody. So like, you know, what, the darker side of Ross, you have no idea. No, but yeah, like, so what what's the target? Like, who do you even try to hack back? Because, you know, are we going to see attacks on the companies that are hosting unknowingly hosting the the material or these people are supposed to be the good guys, right? Your twist was that these were not people trying to seek money out of this. They were trying to show in security. So sure, if you're hacking back, who are you? You know, you're hacking good guys, too. It's probably someone here in the room that's a, you know, started this. Yeah. It's a, you know, it's interesting question, because, well, who do you go after? Right. So if you if it's a bit torrent client that this person doesn't even know they're hosting the images, like, are you going to attack somebody? It's a private person. There's like, there's a whole range of things you could go after there. There's an attribution problem. There's attribution problems. Well, speaking of attribution problems, who do you sue? I mean, we've got a couple lawyers up here. Who do you sue in this? In this instance, if you are not a lawyer, but I assume their answer is going to be anyone we can. Everybody. It's a valid question. I think, you know, we have, in certain instances, there are limitations on liability for hosting providers. True. But as we move to the Internet of Things and corporations, companies are now connecting something that was never envisioned to be connected. Who owns the software that that device is operating? Do I have the ability to, as a consumer, as a purchaser, make changes to it to update it? Can I hack it, in theory, myself, to make it more secure? Because maybe the company that produced the product had no clue what it was doing. And it's still, if it had a password when I bought it, it's password is 1234. We're sort of in a perilous position, I think, and we really need to start thinking about what the right balance is here in terms of potential managing the risks and potentially reappropriating liability issues. Absolutely. It's not going to make some companies happy, but it's a wake-up call, I guess. Well, we'll talk about that in the end when we talk about what have we learned from this and looking backward. What should be our policies next? But before we get there. Yes. Alan, you, oh. I was just going to go back to the question of how do you authenticate anyone on the Internet? Because I think that's a big one, and we haven't really gotten there yet. And so, at Berkeley, we have some students that are trying to come up with ways that relate biometrics to things that are changeable. So, one of the problems that is posed by a scenario like yours, if everybody's voice is public on the Internet, but anything that uses voice recognition is potentially hackable. And you can make the same argument about things like fingerprints. If those have been published in your scenario, we'll take 3D printing technology in just a year or two, or maybe even close to now, you'll be able to match someone's fingerprints. And in fact, there's some Japanese researchers that took a picture like this, high-res picture, took somebody's fingerprints and were able to replicate it to a degree that they were able to actually fake out a fingerprint recognition system. So, we're probably already there or close to there now. So, our students are trying to think about what is unique to you, but changeable? The problem with hacking a fingerprint or voice recognition is you can't easily change your fingerprint or your voice unless you're in an intelligence service, and that's another story. But, so, they've been thinking about, well, what about brainwaves? So, they've done some early testing on technology that would actually take an EEG brainwave reader, put it in their ear, and you think a particular phrase. So, let's say today's phrase is, Mary had a little lamb. Okay, so if I think the phrase Mary had a little lamb, then you will see a certain pattern of brainwaves each time I think it. But if Alan over here thinks that same phrase, you will see a similar pattern for him, but not similar to mine. So, you're able to distinguish at a pretty high level the brainwave pattern of the same thought to two different people. So, the idea would be that this could be a new way to authenticate people on the internet because if you wear this device, you know it's you because it's matched to your shape, then you think a particular phrase, and if that phrase gets hacked, then you just pick a new phrase.  So, it's technologies like this that we're really gonna need to start thinking about now. Because if we're in this situation where it's too late and you've already got all of our biometrics, it's gonna be too late to change consumer behavior at that point. I can, we can go a little further into Philip K. Dick territory here. And so, one of the notions, so there's nothing new on the internet. Back in 2004, at the Kennedy School, we did an exercise on what happens to digital identity in different scenarios, and it was authoritarianism and chaos. And we delved into this, you know, same question of biometrics. And, you know, every sci-fi movie's favorite biometric is the retina, retina, highly impractical for modern biometrics. And one of the reasons is that it's susceptible to body chemistry. In fact, there's a story, and I've never chased down whether it's true or not, so don't quote me on this, but there's a story. Your live stream today. That was, yeah, that was attributed to, that was claimed by a major defense contractor that was using retinal scans in the early 90s, and they found out that one of the ways that women discovered they were pregnant was they were locked out of their offices because, you know, body chemistry changes your retina. So you can imagine a world where if your retina scan is compromised, there is a defined amount of hormone that you can take that will not affect your phenotype, but that will actually change your retinal scan. And we can imagine other types of biometrics that are mutable in a controlled environment that we're going to have to start studying now. How many times do you have to inject yourself with the... How many times do you change your retina, though? Is that my question? Like, if you get hacked four times, like you're pumping yourself up with more testosterone, is that really the right way? Well, not to mention you have to do the whole re-keying of the backend system, and that's really the part of it. Right. Cyber security jokes, man. Good old PKI. What happens in the international space here? Do countries bend together after this, or is this gonna fragment the presuming we have an international system of any kind still in 2023? What is the, is there some sort of treaty? Like, what's gonna, because this implies all kinds of jurisdictional problems, because this data is all over the place. Do you guys foresee this being good or bad? Like, do we get the federation out of this, or not? I think we, if we don't have too many countries already pursuing data localization, I think it potentially is a boon to them in arguments in favor of why data needs to say in a particular country, which I think if I read correctly last week, Mr. Bossert's comments suggesting that we are going to strongly, we as a country are going to strongly push back in trade agreements against data localization, which I think is a positive comment. Yeah, I mean, I think you could see the EU not enjoying this scenario. You know, I think when it comes to core internet values, this has made it quite interesting, right? This is the sort of thing that the press would love to talk about. There's lots of things that could be learned. At what point can this be published? At what point do international laws step in? Is this going to be, you know, a, you know, anyone can publish anything because it's the internet, or is it going to be an extension of the right to be forgotten? Or, you know, empower countries with very strong libel rules to go after anyone with exposed assets? I will go on the record as saying I don't think any kind of treaty is going to help us in this space by any stretch. I mean, Not after that. Particularly when you think about those countries among the many who care very little for their citizens, so-called privacy, and sort of the, what interests do they have in trying to pursue this? Unfortunately. So I think most of you can guess as to who I'm referring to, but I just, I think we're in a space where we have some existing international agreements that would allow us to move forward in this space. By that I'm referring specifically to the Budapest Convention. Who knows if there's, I'm not an IP lawyer. If there's some way that you could shove this under WIPO or something like that, I don't know, but I think there are existing instruments that would probably allow countries to make some progress in this space. Certainly they could be enhanced and we can get into that when we get to the next phase of our discussion. I won't jump the gun again. But I think hope for a multilateral agreement to help this is a lot. I'd expect a sort of dividing between regions of the world, so. And it's not too dissimilar to what we have now because there are different ideas of what privacy means in Europe versus the United States. And I think this sort of scenario teases up exactly those sorts of tensions. And I can't imagine an easy kumbaya or we'll just resolve this and move on. But I think it's important to focus on what might happen in the developing world too because all of a sudden with more information being public, the barrier to entry for companies that rely on data and the ability to learn more about how developed world businesses operate, how consumers behave, et cetera, et cetera. It becomes a much greater opportunity. So I would also imagine that there could be some evening out between the developed and developing world if such a scenario became true that data that right now is proprietary pretty much was no longer. Right, so our trade secrets are out the window, right? Right. It's potentially, excuse me. Sir, here's a twist. What if instead of everybody. And where's the audience twists? Come on, we're waiting for, come on internet. There won't be no cocktails unless there's audience participation. Oh yeah, no, I did say that. I don't know if anybody heard that when I started, but. Okay, so here's a twist. So let's say it's not everybody. It's just one political party. How about that? I guess this is now sort of a US centric and not global, but roll with it. You know, what if it's the Republicans? It's just Republicans and no Democrats. It's just, and only their data is sloshed all over the internet. Is there, does it change how we view this? Does it make it a part, does it make fixing it a partisan issue? Well, I think it makes it even more important and for both political parties to stand up and say that this is unacceptable. For the Democrats not to choose to take advantage in your scenario of the fact that this has made Republicans vulnerable. It's even more important for Democrats to stand up and say, this is unacceptable in the same way that impacting an election or other major US institutions is unacceptable. Because if both sides don't come on board, then you have real issues because, you know, clearly the media has been unable to help people sort out how they should feel about these sorts of things. So we need all the key players to stand up and speak strongly on such a question. I think it's a new, or it's a sort of reiterates this theory of restraint. So we need, I think as the US need to demonstrate and carry out our actions in an informed and thoughtful manner. And so that goes to our politicians as well. And so they need to potentially not take advantage of situations that could otherwise be quite public. It would be appropriate. Any chance of that happening in real life? No comment. So I'll take on one other. Oh, go ahead. Do we have? No. Just yell it. Twist. Here's a twist. Let's go by income bracket. Let's say that the data released was based on income bracket in the US, bottom 40 percent, top 10 percent, these are three different scenarios, and then middle 50 percent. So the data was tagged by income bracket? Interesting, okay. Well, one of the things we were talking about in preparing for this scenario is the equity issues that do come into this, right? So if as Alan was suggesting, there could be a business model for helping yourself get protected, then the top income bracket would be the most likely to be able to be affected by that and to take advantage. And so you could imagine a world in which certain people's data is protected and certain is not. So I think if you release the top income brackets information, you'll merge immediately towards that model, and perhaps if it's a lower income bracket, maybe there will be sort of a societal recognition that this could happen to anyone, and we should build institutions to protect all folks. That would be a positive spin on what could go. I see skepticism from Alan. No, it would be nice if by 2023, digital divide issues had been dramatically been addressed. And we are working hard on it, but hypothetically, if they have not been addressed, then I would argue that the at-risk population, if we're going this separation, is not those at the high end of the spectrum who can rely on direct personal services. And the low end, which is not as dependent on the digital infrastructure, but the sort of middle band that has come to rely on digital services for healthcare provision, for education, for really every aspect of their lives and for whom disconnecting would be a dramatic change in the quality of life. No, I would push back slightly. I agree for sure that the middle group could be very highly impacted, but I actually think that the lower income bracket also will. I think after that comment that came out in the healthcare arena about why don't people buy health insurance and not use a smartphone. Well, there was a great piece in the New York Times that showed how much people rely on their smartphones for their day-to-day information about, or even text messaging, how to get who's gonna go pick up the kids when this person has to work late at work to get overtime, or how do I find out when my services are being delivered? I mean, increasingly, government services can only be delivered if you get online, and folks may not have home internet, so their access to the internet is limited by their phones. So I do think that while that may be true today, by 2022 or 2023, you may see equal reliance by pretty much everyone on the ubiquity of the internet. Which would be victory up until the apocalypse. Right. And so I guess one question, I'll go take a step back, sort of, up a level and go back to my multilateral comment. Is the apocalypse the straw that breaks the multi-stakeholder internet governance model? Oh, that's a good question. So do countries say, those who don't appreciate the multi-stakeholder model, see it's broken and we told you it was, so forget it? Oh, yeah. I could see that happening easily.  I could see that too, yeah. What happens in the wake of the attack the inevitable, some countries are going to start passing draconian laws on, you know, the security of devices sold in their countries, you know, let's say it's the EU, right, to make it a big buying power. It would most likely be the EU in that, yeah. Assuming the EU still exists, yes. You know, what does that do to the pace of innovation? What does that do to manufacturers all around the globe who suddenly have to, you know, face huge fines or, you know, do security, right? Absent clear standards that are actionable. I think it would be devastating to the EU market, right? It is not just enough to say be secure, it's the, right, in the crypto world, we talk about nerd harder, and this is the security of the, we are going to, you know, start amputating pinkies until all of your devices are secure. Well, you're going to have a whole bunch of nine fingers being waved in the air saying, how do we do this? And so you can't just be secure by fiat. You actually need to have the tools to empower organizations to build and deploy them as well as the incentives. And the incentive side, you can help with the regulatory approach. The fingers, I think, is the incentive, right? The fingers is the incentive, it's the, but when it comes to actually making devices that are secure and interoperative and still have the features that people have come to know and love, these are things that need to be engineered carefully in open consensus-driven standards processes. Or alternatively, these are things that need to be built in exquisite vertical silos that don't interrupt at all. Right. Those are the two models we kind of have around security. And if you try to push either one of them as a shock, I don't think you're going to succeed in the short run. I would like nerd better as the new hashtag for this panel. Nerd harder. We got a twist, all right, good. Detailed US defense specs are leaked because an ally used IoT device in violation of policy. Yes, so this was a twist that I've been thinking about as well. What happens when the general, in charge of NATO, at that point has a smart TV in his home that is recording his papers on his table that he shouldn't have taken home? Does that change a lot of things for you guys? I think one of the things that's interesting about that is that it would show how difficult it is to do this well. That essentially, in day-to-day life, doing good things on the internet and being careful at all times, I think the example used earlier about updating and how many of you actually update when it tells you to? Well, pretty much no one, despite cyber security professionals in the room. And I think that would be really hard to swallow in the military setting. But I think it would also force some hard questions about default settings, for instance. Why people are able to choose whether to activate these things? Why, for instance, your camera is default set to be able to be activated rather than having to be turned on when it's being used. Same thing for updates. If the update does not require an immediate restart, why not just force it onto happening? Or let's debate those risks, at least, because there are risks on both sides. But right now, we're not having that conversation. And I think if the military were involved in such a thing, that would really force the conversation. Does anyone else like to walk into their friends' houses and say, Alexa, order 100 pounds of silly putty, confirm purchase? No. Have you done that? Is that just Alan? I saw that. You're not coming over to my house. It was stolen by Randall Monroe to cite your sources. Yes. OK, let's stop here. Let's pause here and let's turn our gaze backward. What policies do you wish that we had enacted in 2017 to avoid this horrible fate in 2023? Hallermann. I'll jump in and say I want to reiterate this point that you can't just get people to nerd better. There are some very easy things, default passwords. But security is a very complex issue. And Katie said earlier, you can't jump into having the high-end security side of things. You need to actually have an organization that is capable of doing it. And so some of the work that we've been doing at commerce is around making sure that the entire community is on the same page on issues like vulnerability disclosure, which Katie's been very helpful on as has Ross on sort of thinking about what are some expected practices. Acknowledging that there is no one-size-fits-all approach, how do we foster collaboration between the security research community and those who make and maintain the systems on which we all depend? And once we have the ability for organizations to learn about vulnerabilities, how do we make sure that they can fix them? So in the IoT world, it's not just enough to say, build it more securely. It's a little bit like when you ask someone, excuse me, do you know where I am? They said, well, you're lost. It's not wrong. It's true. It's just not actionable. And so what we've tried to do at commerce is sort of figure out what is the path forward. And IoT patching is really important. It took us 20 years to learn what patching meant in the software world. We don't have that kind of time. Not even the apocalypse is coming in 2023. You have six years. So what we need to do is at least have a shared vision of what a security update looks like, what does field-upgradability mean for a consumer-grade device, and have that be detailed enough so that manufacturers can build to it, but also have it translatable into consumer-understandable language, so that consumer reports can say, don't buy anything that doesn't have this on the box. Have people walk through CES and say, you don't have this property that everyone now believes that you should have. And I think that we can actually make progress, not by saying, everyone must do this now, because that's how you get lobbyists in the room, but saying, let's get the engineers in the room, and let's get the civil society advocates in the room, and figure out what will actually protect everyone in a way that can actually move progress forward faster. Go ahead. Actually, I just lost my train of thoughts. Well, so I guess I'd say three things. One, sort of bouncing off that, obviously, the making the consumer aware is incredibly important. And so the standards idea, sort of like the equivalent to Energy Star or the nutritional labels recommended by the White House Commission, those are all possible ways to help bring the consumer awareness as to what is secure. I think there's a bigger fundamental underlying problem, which is that right now, the consumer doesn't care. And that's because they haven't seen any negative effects. I mean, so I'll often ask, and I'll ask this audience, how many of you have actually seen real detrimental financial or other implications to any of the hacks that I'm sure you've all been subject to? So I have two hands, three hands, and one maybe. So the average consumer, and we're not the average consumer, the average consumer has seen even less. And so we're at this point where we need to educate the consumer and to make it real. And the analogy I like to use is anti-smoking campaigns.  And that made it real for people. And I think that had real tangible impacts. And we need to come up with the public awareness campaign that can do this in the cybersecurity space. And then I think finally, we've talked a lot about policies, but I always like to remind people that if anything, companies matter more than government in this space, at least they do today, that can change, but at least they do today. Companies are making decisions on a day-to-day basis about what data to collect and about how to manufacture devices. And they're often taking into account quickness to market, as we like to say in Silicon Valley, rather than security. And so we need to put policies in place that encourage companies to take their time in figuring out security and find resources when they don't have them, you know, any of these devices are coming from companies that are not traditional internet manufacturers. And so they may not have those security resources. And then we need to encourage people to consult with each other and to grow a community in which we can actually take into account corporate interests as well as government in building the space. So I think part of the challenge, though, and the consumers not understanding the cost to them is that there is not a whole lot of transparency in this space. So I think it was Intel and CSIS just had a report that was published in the past couple of weeks that talked about the fact that six years ago, cybersecurity spending was not on the top 10 lists of company boards for their knowledge and decision-making processes. Nowadays, we have the SEC potentially demanding or requiring that you disclose whether or not there's somebody on your board who has cybersecurity background, and if not, why that's satisfactory for your space. So we need to have a little bit more transparency from companies. I think one of the challenges in this space, not to say nerd harder, but we are thinking about this, and I'm not the first person to say this, I think in the wrong way in the sense that we're spending money on cybersecurity rather than spending money on, we think we need to continue spending money on that, but we need to, as Betsy was saying, and others have said, spend money on education so that we're producing products that have more secure and stable code so that we're getting companies and boards saying, rather than saying how much are we spending on cybersecurity or how are our networks secure, but how secure is our product? How are we managing and handling the information that we obtain about our customers? Are we being appropriate with it? Are we doing it in an ethical manner? Are we giving adequate security to that information? Are we enabling our customers to have enough choice? I know you can follow the FIPS framework there, but I think we're, we need to, we as consumers need to demand greater transparency from the companies from whom we buy things in order for us to make an informed decision. I think that this is where the consumer reports digital standard effort is really going to help both consumers but also boards set to say, oh, the product that you guys are manufacturing just got to, whatever it is, half of a circle, why is that and what do we need to do to improve that? Right. And the other thing I'd add regarding the cyber talent pipeline in education is that we also need to grow a workforce that is better able to handle this than exists today. And so the panel earlier with Angela and Kirsten had a number of great ideas and I think we really need to invest in those now because the reality is unless we begin to do both training for people who are going to work in this space and training for the broader community that by just living in the digital age you're gonna have to engage with this space, we're never gonna be able to catch up to the scenario that you're talking about. Yeah, I heard someone the other day referring to it as like we need basically another class in high school alongside civics. Right. That is basically how not to get owned. Right, I mean. Or whatever you want to call, you don't have to call, you can call something else. As opposed to the after school special on two kids who are about to buy a nest have a very special lesson learned. So we haven't talked about this issue. So what do we need to do now? There's been an effort, a couple of governments have been working on it for a number of years. So thinking about, so what we're talking about in part here in education is capacity building. So we have our Nigerian princes who've decided to, you know, the formal company and... Manufacturer. Manufacturer. IoT devices. Chromey devices, I think, right? Blenders is what it was. No, yeah, blenders, yeah. But I think it sort of did not very eloquently tie some of these concepts that we've been talking to together. Some of the work that I think NTIA is doing is to help those companies, but also consumers eventually hopefully talk the same language. So when we're talking about we have standards, but which standards do we think are the most important? What does it mean to build, you know, what does it mean to have a build particulars for software development? Which of the particular standards in that space is the most useful and most effective? But I think, you know, we need to translate this effort internationally because obviously we don't want to have the internet shut down and balkanized and we don't want to have the end of multi-stakeholder internet governance because we don't think that that's in the best interest of the internet. So how do we, you know, in the US, but also in mostly Western countries, think about how do we take these lessons that we are learning painstakingly in some cases and help those who are still coming online to not have to endure the wounds that we've all fought. Exactly. And I'll just bounce off that cyber insurance is another area that we haven't really talked about, but would be really important to start figuring out now in this space before all the claims come in. And so, you know, I think there's a lot of interest in that space. I mean, certainly the number of companies that are like, hey, are you doing any work on cyber insurance, or, you know, so that's definitely an area where people... You're getting this one called, it's been the next big thing since 2002. Well, maybe it'll always be the next big thing. Earthquake insurance, for instance, in the area where I live is still the next big thing that I'd love to have. But so I do think that it's time for us to start really thinking about what types of solutions we're looking at. Are we looking at private sector or are we looking at, you know, are we looking at cyber insurance or liability? And where, you know, what are the comparative areas that have struggled, and maybe earthquakes isn't a bad one to start looking at, and figure out how can we resolve this? Because the alternative is, when there's something bad that happens, taxpayers are gonna pay to help clean up the mess as like sort of an emergency response. So, one way or another, we're gonna end up paying. Let's figure out how to do it in the most efficient manner.  I'm thinking about like app developers or somebody who's just learning. The reason I'm saying this is because in my head I'm imagining, okay, take 3D printing, fast forward six years, printing circuit boards is perhaps a possibility, maybe now people are trading circuit board designs online, like how do we get that community to understand and implement? You know what, forget it, let's go drink. So, one of the garage biology is going to be one of the new big things, but I'm not really worried about someone saying, hey, there's syringe, I've been working on something really cool, try it. That's sort of, we have that intuitive risk model. Darwin's gonna take care of that one for you. And I think the path forward is going to be to make sure that the garage hobbyist is this key built-in source of innovation that we have in this country. But as we've scaled to a digital connected infrastructure, there needs to be a little bit of pressure against that. And I think that pressure isn't coming from making sure that everything that comes out of a garage goes through full on eight year government testing. But it does come from having the other parts of the ecosystem push back a little bit and saying, listen, if you're gonna connect that to my trillion dollar cloud business, it can make me look bad. And so, one of the things I love is the first question on the questionnaire that Google has of if you're going to be a part of their software chain of building products for Google is do you have a vulnerability disclosure process? And making sure that we have that spirit of connectivity, I think that's gonna spread a little further. I'll go back to the mom and pop question, because hi dad, my dad still has his password on a Post-it note, so. Well, he should, it's a good password, right? Oh yeah, really good. It's not one, two, three, four, it's okay. But so, I think that really to begin to get the consumer to take this seriously, there has to be some cost. I don't think it should be the full cost, but I do think that there needs to be some cost. Right now, if your credit card information gets stolen, even if it's your fault essentially, you get to just have your credit card company send you a new credit card and the only real cost to you in the vast majority of cases is waiting three or four days to get a new card. And so, I don't think that the consumer should pay the full cost, but maybe if they had to pay $10 to replace the card or something like that, it might lead to just enough incentive for people to take this a little bit more seriously. And I think we need to think about that in every aspect. Even when you think about training people at work and if they fail to take one of the, when they get fished at work, a fake fishing scam and they click on the wrong button, well, maybe they have to go and spend an extra two hours in cybersecurity training or something like that. So we need to think about small ways to incentivize people who don't think about this as their job and who don't really see why it matters so much right now to just understand a little bit why it matters. Back to, we have these, every year you have an annual physical or you do monthly health checks on certain things. Do we need to start doing something like that to consumers like through their app? This is your quarterly reminder that if you get an email that says you need to change your Gmail password or if God forbid, no offense to AOL, you're still on AOL and there's a new app for AOL, and a new AOL app, which I think is probably remote, that you just, this whole idea of stop, think, connect or think before you link or whatever it is, we need to, we need to. It's got to happen more often than I go to the dentist though. I think we need to, yes, well, probably. That's every, twice a year, right? Twice a year, that's right. Again, this notion of using, what we heard earlier today, using behavioral science to try and better understand how to connect with consumers. And I think it was Bobby and a few others earlier, much earlier today who talked about the need to bring many more people, experts in different areas, it's a multi-disciplinary approach to this rather than the nerds nerding harder and the, you know, you can think about some of the bar, some of the lawyers out there just sort of running after any kind of lawsuit out there. Let's try to think collectively a little bit more carefully about this. I'm now envisioning like village doctors doing making house calls and like turning on two-factor authentication for you. Anyway, you've been a wonderful audience. Thank you everybody. Please join me in thanking Betsy, Alan and Megan for giving us their thoughts this afternoon. Thank you.


I'm going to talk about Chef and Puppet and DevOps and Automation. So a couple of things. I am a little bit nervous and I'm also standing on a raised platform. I tend to walk in circles a lot and we might see some interesting acrobatics here. So be prepared to laugh. So let me, if you haven't seen by the way, they're using raspberry pies to run those slide shows. It's kind of a cool little trick. So oops, of course it's not gonna work right out of the box. There we go. So configuration management, DevOps, Puppet, Chef, all of these things are, they all kind of mean the same thing. They mean a lot of different things to some people. So what I'm gonna do is spend a few minutes with context. Context is important. So I wanna talk about what the universe seems to think that all of these things mean all the different concepts we have about what DevOps, what configuration management is and then what I think it is. Now, maybe you wanna argue opinions. If you wanna argue by the way, feel free. Except for Steven Frost, I really, I think we're argued out. So, but in all seriousness, if you do have questions, please feel free to stop me.  So if you got questions or you have comments, you've had anything to add, a lot of you probably know this stuff significantly better than I do. So please feel free to interject. Who am I? I'm Scott Mead. I'm a senior architect with OpenSCG and BigSQL working on BigSQL. I have been in the Postgres universe for about 12, 13 years now. I was one of the original employees of Enterprise DB. I was, my nickname was Configure, Make, Make, Install because I just did the builds all day long. And now I'm at OpenSCG. OpenSCG, the open source consulting group, we provide expertise to enterprises using open source. We specialize in data management. The cool thing about being the open source consulting group is I love Postgres, but it's not always the right hammer. So I get to use Cassandra and Hadoop and Bob's Backyard, whatever database I feel like at the moment. So it's kind of fun. BigSQL is the world's most complete and developer friendly distribution of Postgres. It's something new that we've launched here at the show. Check it out, it's really easy to use Sandbox. That's the end of my shameless plug. So DevOps, configuration management, rapid deployment, what are all these things? What is DevOps? Is it a development and an operations team? It's developers, it's operations. That sounds like exactly what we already do. So how is that different other than just putting a slash and making the word cooler to say? Is it developers that are operations people? Maybe. Is it operation staff that are developers? Maybe. It's always really easy to say, okay, well, dev guys are gonna be ops guys. That's, it's not realistic. I've been on both sides of that fence and when you're a developer, you're working all kinds of weird hours, you're out at Starbucks thinking about this algorithm and then you get a phone call that, you know, Prodwebo 4 happens to be down because of a bad NFS mount or something. Like, I don't, you're gonna run out of time in the day. There's only 24 hours in the day. And the same goes for ops guys. The ops guys are focused on operating the system, making sure it stays up, making sure we're hitting our metrics, making sure the system is secure, that we're patched. I don't have time to write all the code. So is it possible that developers are operations and operations are developers? Cause that's what dev ops kind of feels like, right? It's like, ah, it's one guy sitting in his dark room and he pushes a button and bam, Amazon web services happens automatically, right? It's not realistic. So what is dev ops? I think of it as kind of two specific areas. It is development and it is operations. Now, I'm gonna start with the operation side because I grew up doing operations. I was a Windows admin back in the day.  But what is it, when we're in the context of dev ops, what does it mean to be an operations person? What it really means is I want to define what I want on a system, not how to do it. So I wanna say, you know what, OpenSSL should be installed and it should be updated to the latest version. I don't wanna say I have to go download and install or I have to run the exe file, I have to do this or I have to go yum install or app get install or whatever it is. If I'm a developer, what does dev ops mean?  I should be able to do that often. And there's a lot of implicit undertones in the word often, quickly and correctly. It needs to build, it needs to pass regression, it needs to not introduce billions of extra selects against the database and all kinds of other things. So there's really dev ops, we kind of, some people smash them into one concept but it's still two. What's it really boiled down to? So let's think about it from an operations perspective for a second. If you're in operations or if you've ever worked with operations, just out of curiosity, how many people would consider themselves like mostly operations? And what about mostly dev? Uh-oh, look out, look out boys. Devs are on their way, I think they're taken over. So when all of these developers say, hey I need another node, what happens? What do we as an ops do? It's like okay, first the magic, the node appears. We called Dell, when we called them they started mining for the ore to put the thing together, they got the sand together, made the silicon and they shipped us a box. We get that thing racked, that takes who knows how long. But now this is where your standard view of operations begins, right? I gotta get the host names, the IPs, authentication. If I'm using corporate network, I'm using Active Directory or LDAP or just my Etsy password file, whatever needs to happen. I've gotta do updates. The standard packages we have, maybe I've got a monitoring tool that needs a bunch of prerequisites. The monitoring agent, maybe I've got standard mount points, security policies, SE Linux is horrible. So that took me half an hour because I didn't remember what the context I needed on slash data was. And the reality is it's half a day of work. It's a lot. It's a lot of commands and it's very error prone. This is when I was the IT intern back in the early 90s, we didn't really, didn't do much other than sit down and do this when we got a new server. And that was okay though because we only had a few servers. So then once that's ready to go, okay cool, we got a shiny new box, it's ready to go.  I've got to get any dependencies for my stack. I've got to get the database, Python and Perl, Postgres installer puts home directory and the data directory which doesn't make sense in the context of SE Linux. So I have to move that thing and then fix a bunch of configurations, install the backup script, get the backup job, get the SSH keys, do the testing, doesn't work, developers aren't happy. It's another half a day. And then, oh yeah, the firewall's got to be open too. So there's a lot of things that go into just getting a new machine, a new node ready to go. We say okay, well, automate it, right?  It's like well, if that, well, I'm actually a terrible operations person, right? So let's script it. Like we have bash, we have command prompt, we've got all these different, we have the Windows CMD, we can script it, not a problem. So okay, let's script it. So do we use Red Hat? So we do a YUM install. Did I, anybody upgrade to Fedora, like the latest versions of Fedora lately? YUM is all of a sudden something called DNF. So now all of a sudden my, all my YUM install scripts are completely replaced by a new tool that behaves differently. App to get install, if I'm using Ubuntu, right? What about that? Now all of a sudden I've got, you can just see in your head, you get one shell script, maybe it's 150 lines that does all of this stuff, right? Maybe it's 200 lines, maybe you're a better scripter than me and it's 20, it's possible. But now all of a sudden I have an if statement that says if I knew Ubuntu do this instead, or Pacman, okay. Next step, go ahead, install open SSL libraries. I dare you. All right, on Red Hat and CentOS, it's open SSLDevel, on Ubuntu and Debian distributions. Today it happens to be libSSLDev, but who knows what tomorrow brings? It's a brave new world. So, and even if we are going to kind of if def our code out, so we have multiple sections, how do I do that? I gotta sit down for, now I need to go download Ubuntu 12, Ubuntu 14, CentOS 6, 7, BrutSystemD, so there's a whole new bunch of stuff. Now I got four platforms. I've gotta figure out, how do I tell the difference in my shell script? And I just ran this last night and whoops, I just said okay, what about catting Etsy release and pulling the first line? Well on CentOS it gives me a nice capitalized, stylized, productized name. On Ubuntu I've got a bunch of extra stuff in there. So already I'm having, now am I gonna make the exception? What am I gonna do? This is a pain in the neck, right? So that, scripting it, really it's an option, but it's an expensive option. It's a painful option. So let's talk about some tools. So I'm not trying to convince any of you to use any of these tools. I happen to love them. I use them a lot. There are the core of what we're really talking about is Chef and Puppet. You notice that's at the bottom. But to get us there we need some frameworks, right? So I'm gonna use VirtualBox. You can use VMware. Doesn't really matter. It's a virtual machine. I'm gonna use Vagrant as anybody in here not, who has used Vagrant? Just so quite a few people. So I won't waste too much time on context with it, but we are gonna talk about it. Essentially for those of you that have never used it, it's a way to deal with virtual machine templates. How do I deploy quickly a VM template? Get it up and running so I can use it. And then Chef and Puppet. Chef and Puppet are gonna do a lot of the workforce and we'll get to those in just a few minutes, right? So if we were in real what now? Okay, other thing, just so you know, I could have done a whole huge presentation on both Chef and Puppet. It's gonna take forever. And I don't really like Puppet that much. But that's just my personal taste. At a high level, both of these guys behave basically the same. You've got a Chef server, you've got a Puppet server, you've got, they poll, you've got clients, the way that they work, it is distinctly different, but they're very, very similar. And they're really at a high level doing the same thing. So what do we have? If I was gonna deploy this in my server room or what we have back in the OpenSCG server world, we gotta get repository. That's holding all of our cookbooks for Chef or manifests in the Puppet world. We have a Chef server whose responsibility is to talk to network clients. And then on our nodes, we have a Chef client. Chef by default polls every 30 minutes, 1,800 seconds. So the idea is that those Chef clients are gonna talk to the server, they're gonna get their state, what the desired state of that machine should be, and they're going to do it. I don't have all of that here, and if I were running that many machines in a VM, I would screw something up. So I've loosened it up a little bit. I'm using this tool Vagrant, which I'm gonna show you guys, which is basically going to simulate my server room for me. It's gonna, I think of a Vagrant, somebody in the dark server room who put servers in the rack and then disappears back into the dark corner again. That's kind of what Vagrant's gonna do for us. And inside a virtual box, I've got my Chef cookbooks, and I have the Chef client, which we're basically gonna run in local mode. So we're not gonna be looking for a server, it's gonna directly process a cookbook file for us and start applying some things, okay? Anybody not familiar, is everybody familiar with virtual box and VMware for the most part, virtualization, okay? Again, I'm using virtual box just because it's free. It really scares me every time it comes up and the Oracle logo is there. Sorry for that. But you could use this with most hypervisors. So let's do, again, basically what Vagrant's gonna do, I personally think it's easier than using virtual machine snapshots. I could go into VMware. I can build a Red Hat machine, I could build a CentOS machine, take a snapshot and then always revert to that. But I always feel like with a snapshot, you don't actually know, what did I put in that snapshot? Did I write it down? Did I actually install this? What versions in there? There's a lot of unknowns when you're talking about virtual machines and templates and how long's I've been sitting on my file server. Cloning in virtual land can be a little wacky. If you've ever done, I did a linked clone on VMware Workstation the other day and then I realized there's all these crazy hard links floating around on the file system and it kind of makes a little bit of a mess and they become very dependent on one another. So we've got woes on woes on woes. Just I don't like them, I stay away from them. Basically, this is what we do when we copy a VM, right? We're gonna make a copy of my VMware template. Oops, I'm out of disk space. That always happens to me. I'm always out of disk space. So we go make disk space, we re-copy it, we start the VM and then VMware pops up. Did you move it or did you copy it? Well, I think I copied it but maybe I want to say moved it and then because of that, the MAC address of the network card gets screwed up. The VM gets booted, network interfaces are gone. Now all of a sudden I restart it, kind of hope a little bit. The network adapters back. I end up on Google for half an hour. How do I delete an ethernet? It's just, it's a never-ending pile of stuff when you're deploying virtual machine templates and you're copying around. So at the end of it, after three hours of doing that, I realized I actually needed Ubuntu and I started again. So what does Vagrant do? Why is it any better? So what I have here, I'm gonna make this a little bit bigger. I'm gonna try and keep this to the top of the screen. If anybody wants me to slow down, let me know. If anybody has questions about this, I can give you scripts of it after we'll try to post them. Basically, I've already done some setup work with Vagrant. You're really defining what the base image, what's the template virtual machine you're gonna use and a couple of different settings. So I have here, I wanna do some port forwarding. If I'm in virtual box, I wanna do some stuff with DNS. I want two CPUs and two gig of memory. So whatever that box that it pulls down, it's gonna get those settings in a virtual machine. So all I have to do to actually deploy this thing is I have nothing running right now. Just type Vagrant up. Vagrant is going to import. It's gonna do that whole slide of stuff that I was just whining about a couple of minutes ago. It copies the machine, fixes the MAC address. It's gonna deal, it pops up virtual box for us. It deals with any of the problems that we were having. It adds network adapters. It sets up the number of CPUs, gives us RAM. And then the really cool thing that I love about Vagrant is it actually, SSH is into the server and does some cool stuff. So you'll see in the bottom we're booting there. I'm just gonna try a couple of times. Don't fail me now. There we go. It gets in, it fixes up the VM, sets the hostname. And now I have it actually installing ChefForce as well. But basically when this is done, I will have started at a known point, which is a CentOS 6.7 server-only install. And when I'm done, I will have done all of those things and I can see what they were. They're done in the same order every single time. There's no question about what that machine is by the time this is all said and done. Now the only trouble we may have is that I'm hoping the Wi-Fi doesn't give out while it's installing, there it goes. Now what I've asked Vagrant to do, and we're gonna move into Chef a little bit, I've actually said, okay Vagrant, when you're done building that template, I want you to actually run a Chef manifest for me. So what is a Chef cookbook? So what is that and what is it doing? So basically I told you already in Chef and Puppet, and if you're a little bit older of a war horse, kind of like I am, you might remember CF Engine, the idea is to defa... Oh, that's kind of dark. Yeah. The idea is to define the state, the end state. And so what am I saying? As I'm saying, dear Chef, I would really like a file that is slash ETC slash MOTD. If you're, again, an old war horse like me, that's the message of the day file, which we don't really use much anymore. But I want you to create the Etsy MOTD file and I want that content inside of it. There's a lot that's going on here. If this was a shell script, that might be an echo. If it was on Solaris, I might have to do a print F and then a slash N at the end. There's, if I was on Windows, who knows. So the idea is that this right here will run other than the fact that I'm hard coding the Unix pass, this will run on any platform. Chef will take care of the details for me. I'm telling it what to do, not how to do it. It knows how to do all of that. So when we go back to our vagrant window here and we see when Chef fired up says, okay, I've been told to update the Etsy MOTD file and, you know, hey, welcome to my talk. And the other thing that happened here that I really, really, really enjoy is the SeLinux security context was dealt with for me. I'm sure somebody in the universe is an SeLinux expert, but I haven't met one yet. And getting the context right is always trouble. And, you know, about three to five years ago, I was the first one to jump up and say just disable that thing. Nobody uses it anyways. But now in the security landscape wherein it probably makes sense to leave it there. So you can see Chef is already taking care of me. Puppet does very similar things, right? Chef is taking good care of me there. Okay, so that's a very simple version of I did a vagrant up. I got a machine from absolute zero. Now I'm a hero. It's completely deployed. That seems simple. It was quick, right? It took, Chef took three seconds to do that. It took us 30 seconds or so to get the machine up. That's pretty good. What about if we get more complicated? Let's just, I wanna make sure I don't go completely off the rails. Let's talk about these a little bit. So Chef and Puppet, everything's ruby.  And that's what it is. In Chef, we use cookbooks and recipes. In Puppet, we use manifests. Again, the idea is you're saying this is what I want. Don't bother me with the details. Sometimes that's not a good strategy in life, but when it comes to computers, I like it. The other thing that these guys do is they provide a very consistent systems information API. So before, I was trying to figure out what is, what machine is this? Is this a CentOS or an Ubuntu machine? And I'll show you in just a second that they do that. The other thing that both of these systems have that's really nice is a nice file management system. So you've already seen the EtsyMOTD file was I could dump content into it. But we have templates. We can modify bits of a file. We can have it replace whole files, whole directories for us, and we're gonna do some of that in just a second. In Chef, which again is my personal preference. It's the one that I'm basing the majority of this demo on. You've got client server or solo architecture. This is, the next thing is the real reason why I always choose Chef over Puppet is because order matters. In systems, in deployment, in packaging, in dependencies, the order of operations is really, really critical. It looks procedural. It reads top down. Puppet, you have to define the order separate of how you put it in the file. That's why I stay away from it. Underneath the covers there is a tool called OHI that actually gives you systems information. So if you run, if you have Chef installed and you run that command OHI, it will give you JSON output of what that system looks like. Now this is a very small piece that addresses my complaint from before, my Etsy release. I've got, this is Linux. I got the kernel version. What's the Linux standard base? What platform it is? It gives me a platform family. It's all red hats, so we're all in the same family. I even know now that I'm a virtual eye. I'm a guest, virtualization. I'm a virtual box guest. So that's always kind of a hard question, is am I a VM or am I a piece of hardware? And knowing now that I'm on virtual box as a guest, that all of a sudden I can say, well install virtual box tools. I can't do, I don't even know how to get that. I actually have no idea how to get that information out of a Linux machine. So OHI does that for us. Now Puppet, by the way, if you run OHI, the JSON file that spit out of my VM with two CPUs was over 2,000 lines long. So the amount of information you have, all the disks, the memory, the brother of the guy that put the chip on the whatever, it's all there. Puppet, it's a little more terse than that. Very, very similar. It uses something called Factor instead of OHI. The reason that I don't like Puppet, the number one reason is because when I go through in Chef, and I say okay, let's look at a more complicated, let's look at the web recipe here. I say package, HTTPD. Service, HTTPD enable, and file create me an index.html, right? I'm saying do those things. Chef is going to do them in that order, in the top-down order, which is what I would expect being a procedural bash scripter. That's what I would kind of expect. In Puppet, every single time it runs, it will give a different order unless you explicitly define that order later. And you do it, the syntax would be something, and again, not necessarily a Puppet syntax expert, but it's something like package. I also am not a spelling expert, apparently. HTTPD, then followed by service, HTTPD, et cetera. So you end up all of your manifests at the bottom, you have this long thing, or inside you'll have, you could say, the service HTTPD comes after the package, HTTPD. It gets very hard to read, very hard to maintain. These manifests can be thousands of lines long, trying to parse out if the order is correct can be complicated, right? So that's why I personally, now, by the way, I did tell you guys you're allowed to argue with me. Is anybody want to argue with me? Okay, we got one, there we go. I don't actually want to argue with you. Sure. It's really an ordering, it's a dependency on the order. But there is, with Puppet, and you can do this with Chef as well, you can say the service HTTPD depends on etsy, HTTPD.conf.  Which is cool, until you accidentally make a change to HTTPD.conf and then your web servers all go down automatically, right? So we'll do that too, that's fun. I've done that by the way, just all the bad stories that you're about to hear are actual real life, Scott did this. So it's a fun exploration through life. But anyways, this is a slightly more advanced cookbook than our etsy, MOTD file. But again, I'm not saying do a yum install or do an apt get install or whatever. There is one thing buried here that I'm gonna get to which is package names. So I'll show you how we deal with actually, yeah, that's the next slide. So let's talk about it. Yeah, ah, yes. Yeah, so that's why this is important. So with OHI or with Factor, I have the ability in my manifests and in my cookbooks to actually say, what platform am I on? Am I on Windows, am I on Red Hat? Now, the nice thing about Chef is it gives me this concept of platformed family so I can say if it's Debian, if it's Debian-based, if it's a Red Hat-based, but Windows would be an option there as well. And I'll actually show you an example. One of the cookbooks that my colleague's been working on lately has where we say on Red Hat, you need free TDS-Devel on Ubuntu, it's something different. So I'll show you that. But this is basically the way around that. You get a very nice, easy to use array of, a hash basically of all the things that are going on in the system. That's basically that 2000 line JSON file exposed nicely in code. Puppet has something very similar. I don't like the syntax as much, frankly, but that's, again, just me. I'm not bashing Puppet, it's a great tool. I've seen it do amazing things and you get to feel like the master of all your child nodes, it's great. But it's, I don't like the syntax quite as much. So here, actually, this is the code example. Peter, I believe did this one. This is the OpenSSL question from before.  Now, I don't have to worry about doing this in a shell script. I can just say, when it's Red Hat or CentOS, or I could use the platform family, the package that you're gonna install is OpenSSL-Devel. Other, oops, keep smacking that remote that's in my pocket, me and my flappy arms. Or on Ubuntu, Debian-Base, it's libSSL-Dev, right? So it's now on Windows, just do nothing, right? So again, where do we come from, where are we going to? We're talking about operations people being able to define their job, do that day-long pile of stuff just to get a server to a developer in code. Maybe I take a week, maybe two weeks, maybe a month to write this thing. But when I'm done, I can do it 10,000 times. I can do it 100,000 times, and I can do it in seconds, right? So let's run that slightly more advanced example again slightly, oops, where do we go? Not at them, fire up. Okay, so again, what I wanna do here is in my base Vagrant VM that I pulled up, which is the basis of the base CentOS 6.7 server, I want to install Apache, I want to enable it, and I want to create an index page. And I really don't wanna type anything because I am that lazy sysadmin. So this is the other trick with Vagrant. You can SSH into your VM without actually knowing it's an IP address, very cool. But we'll become root. I'm gonna head to my cookbooks where I have all the ones that I set up for you guys. And this is the web. So I'm gonna say ChefClient, and this is called zero mode. That basically means don't try to talk to a server because it doesn't exist. And I want you to run the ChefWeb cookbook right now. So it's gonna fire up, and again, no config file found, we're just using command line options, fires up the client, and it found ChefWeb. It compiles the whole, it converges the whole thing. And it sees, I gave it package install, well, I can't even speak that fast. But I just told it, install HTTPD. Chef figured out we are on CentOS. We can use yum, I'm gonna do a yum install of HTTPD. It ran the service command. If we'd been on 7.1, it would've used system CTL to get the system d-side working. And it created the file for us. Now, that took me way longer to talk about than it actually happened, so it took 13 seconds. And if I pull up my browser, it's running in a VM so I had to tunnel it through, but there we go. That's my hello world. That's actually the web server that we just deployed. So I don't have to worry about any of the manual stuff, it's that easy. Let's take it to the next level, right? That's neat, but we're talking about serious stuff here. This seems really easy. What if we wanna use that, at this point we've built our server, we're the operations guy, we handed over to the developer. The developer has an application.  But we have a real app that we wanna deploy. How do we deploy quickly? My slides are a little new to me, so let me make sure I'm not too far ahead or behind them. No, I'm doing okay. Again, we've been defining what to do from an operations perspective. Now I've got a basic machine that's got the tools that the developers need to actually deploy the application. So let's go ahead and do that for a little bit. Let's deploy the core product or let's deploy our app. And how do we use Chef to do that? So what I've done is I spent a lot of time realizing that it was gonna take some time to get a big heavy app manifested out. So I picked something that we all know of, whether you love or not, which is PHPPG admin. PHPPG admin, we can run inside of HTTPD. It's not really hard to configure, not hard to deploy.  But the idea is all I have to do is, again, tell Chef or Puppet what I want it to do. Don't worry about how it's gonna do it. What I want you to do and let it go. The new thing that we see here is this remote directory. Basically what I'm saying is there is a directory of stuff that's remote to the machine, the file system. It's somewhere else, it's not in that machine. And I want you to put it at var www.html.phppgadmin. And the source is PHPPG admin. Now we talked about how Chef and Puppet all have file management systems inside of them. The way that Chef works, one of the ways that Chef works is by keeping this right inside the cookbook itself. So if we look, it's very hard to see. Let me pull up a window. We'll go into Chef app, which is my cookbook. Of course, just rebuilt the Mac. I don't have the tree command in there. Right inside the cookbook, I have this location called files. Inside of there, I've got a little bit of a directory structure, but in files, default, I have PHPPG admin. It's extracted, it's ready to go. I could take, if I'm using a much more advanced application of a J2E application or something more complicated, I'm probably using bamboo, which is gonna do a build for me, then place it in an artifact and then Chef can pull it in through here. So basically, Chef is going to say, just deploy the app for me. So when we go up, back to where the cookbooks are, and I'm gonna say, okay, I want you to run the cookbook called Chef app, and this is actually gonna deploy PHPPG admin completely for us. Now this takes a little while because PHPPG admin has a good jillion files in it. The other thing here, as it, before it started to go, we had already installed HTTPD, and Chef didn't really mind that was okay. It actually, I had defined it again in a different cookbook, said, I need HTTPD, so I know by looking at my code that this application depends on HTTPD, the Chef isn't gonna throw a huge fit just because it's already there. It might update it, I didn't specify the version, so it'll bring it up to date when it runs it. But it says, hey, HTTPD is there, it's already installed, not a problem. So now you're not doing extra if installed, don't run any of those kind of wacky commands. Yep, yes, so you can specify the version number in all of this, so with the package there is a, believe it's HTTPD equals, and then the exact version number that you want to prevent it from doing the update. Yes, so the other thing about these recipes is that I'm using very simple things, but this is just Ruby, so this is a more complicated Hullabaloo that I wrote a while back. Just use variables, so I've defined a couple of variables at the top, and then I reuse them, I do string concatenation with the plus sign and all kinds of other things you can do later. It is code, so you can let it run time, figure out some more of those things without really having to bake it all in and hard code it. There's also a concept of attributes where you can define things outside the actual recipe itself, so it's more a configuration entry, it's more the advanced chef stuff, but it's definitely possible, easy to do. Any other questions? Absolutely, so in the next one that we're about to run, we have our default recipe, which is just calling, I want you to install Postgres, and then I want you to set up the DB, so I have two different recipes, so you can include others, and you can, if you realize that the differences between Ubuntu and CentOS are too great, you can, if CentOS include the CentOS recipe, if Ubuntu include the Ubuntu recipe, because it does get messy otherwise, so you can definitely compartmentalize them. Now this is all within the same cookbook, but it's very easy to go back one level and call other cookbooks as well. But again, while we were talking there, a bunch of stuff happened in the terminal window, and now, without really having to think too hard about it, PHP, PG, Admin, well, apparently I screwed something up, I missed a dependency somewhere. Yeah, I missed a dependency, oh, I know what I missed. So this is the one question that we had before, which was what, HTTP was already installed, and then I said install PHP, right? I need to restart the web server, but as a grumpy old sys admin, I don't like services to restart themselves, so I didn't let Chef go ahead and restart HTTBD for us, so all we're gonna do is now, now in our maintenance window, we've done the Chef, everything's there, I'm gonna restart it real quick, it comes back up, I do a refresh, and boom, PHP, PG, Admin is deployed. Your app here, right? So, again, this is a simplistic view of it, you can get really complicated, and I don't wanna try and teach everybody Ruby, cause I'm not really qualified to do that, first of all. But it's actually not that bad, if you don't think about the fact that it's Ruby and you're just defining something, it's kind of a definition language, I want this to happen, make it happen, make it so, said Captain Picard. All right, so, that's an application server. Now, when I was running this last night, I was actually, the other thing that I would do is, with my Vagrant, is think about, this is a web server, right, so this is running PHP, PG, Admin, if this were cool lickety-split.com, I might have 20 of them behind an elastic load balancer or an F5 or whatever I've got going on, right? It's a web server, probably very little state on that thing. I expect it to die, I expect it to have memory issues, I expect it to eventually go away, and that's what happens when I do a Vagrant destroy, Vagrant's going to basically tell virtual boxes, shut down the VM, and it nukes all the files that we just did, so everything we just did is completely gone, right, everything. But the real aha moment for me with Vagrant and Chef and any of these tools was that, uh-oh, but I just went through, if I were to stand back and think about it, this little piece took me about an hour and a half to get set up by the time I got all the dependencies right and everything, but if I go in, I'm just going to tell Vagrant, I want you to run the Chef app cookbook after you start up, say Vagrant up, and my cute little web server that we just blew away, all that work is completely gone and nuked into the ether of RM minus RF, it's going to come back because Vagrant's going to build the machine, it's going to call Chef, which is going to deploy the entire stack for me, and when I'm done, I can go to PHP, go directly to the PHP admin site, and that's it. So as a systems administrator, where it, I mean, like I said, I'm a bad systems administrator, which is why it took me an hour and a half to do that last night. Maybe it was the party as well, that could have been some of it, but I'm able to go from zero to hero in when I did my timings last night, like two minutes and 13 seconds from absolute base centOS, and the other thing that's important is I know by looking at what Vagrant's doing and by looking at this manifest, I know, oops, by looking at this manifest, or this cookbook, this recipe, I know exactly what happened, I know exactly what order it happened in, and I know it's right, and I know that it happened exactly the same way on every single server that I've deployed. There's always the mystical things in the life of a systems administrator where I don't know why that server's misbehaving, there must be something wrong with it. It's usually because you manually edit a config file, manually edit a config file, and there's 50 of them and you forgot two. It's very normal. So that's the beauty and power of this thing is that it just does all the work for you. I didn't have to write any shell scripts and away we go. Now, this is a Postgres conference, so I'm talking about apps that we can throw away. I say I'm a systems administrator, but I learned from, actually there's a couple of you in here that have taught me this over the years. I'm looking at you, Craig, specifically, how to be an extremely grumpy old man DBA, and that I am. So it's like, okay, great, your web server, we can throw it on the floor and be done with it. That's great, congratulations. I can't do that with my database. I care about my data, right? And even if I'm doing backups, if that database server were to go away, I can't just restore it in 10 seconds, even if I'm using the hottest EMC hardware or whatever I happen to have. They're not disposable. They're not like web servers. They really are, databases are. Now there's some cases, maybe a session database could be, or maybe QA dev regression, those kind of things. It kind of makes sense to be able to iterate hard and fast. But the other thing with databases is the configurations of these things are carefully tweaked.  They bump into walls and stuff. Postgres.conf doesn't do that. So, we gotta be careful. And this is where as a DBA and as DevOps, there's a line in DevOps. I like to always think of it as the line that we sometimes have to draw because I could go really crazy with this and I will in a second and I'll show you, hey, let's just go from absolute zero to installing Postgres, deploying the schema, getting the application hooked up, wired up, and bam, we're ready to go. But you gotta be careful. You really do have to draw a line. One of the things about DBAs, and this is an ongoing, I wouldn't say fight, but a friendly battle, is in Chef, in Puppet, in CF Engine, you have, it's a highly templatized configuration, ooh, five minutes left there. It's highly templatized. So, what I'm actually doing is defining Postgres configuration options inside of Chef. And again, back to our question before, if we're linking those services together, and if I pull up the cookbook that I'm actually thinking about, which is the publicly available one from Ops Code, this is their Postgres cookbook, works beautifully, you should use it. But if I search in that source tree for restart, and then I look at the change log, all of a sudden I see, hey, look it, we added a new feature. Restart Postgres service immediately on config change. So it knows when shared buffers is changed that that needs to be restarted, it's gonna do that for you. Line, wall, line, chains, all kinds of dogs, everything. So you gotta be careful with it. And some people will not let the Chef client run automatically or they'll stage their changes carefully. But a git push shouldn't restart my database is essentially what I'm getting at. So if I was Mr. DevOps in the corner here and I do a push and everything's automated and polling, I'm gonna restart my databases and they could be down for quite a while because of that. Now, let's hope that we can do this quickly. Of course, I can't type fast enough. Chef client minus Z, minus O Chef DB. Oops, that's not gonna work, that's gonna work. What this manifest actually does is go out to PGDG, it installs the repo file and then starts installing all the Postgres components for me automatically and then, oh, there it goes. When it's done, it's gonna load up a fun little schema that I did, which is an invoicing schema. So it's actually gonna deploy the database schema for me automatically just using a Psequel on the SQL file. So we should see that. The develop package likes to take a second. There we go. It fixed up some config files, starts the database, deploys the schema and now I can SU to Postgres. And I can go into my invoicing database. I've got my tables and my seed data and everything's there for me. That was from just a couple minutes ago. We had a blank, no, nothing machine, right? Here's the thing to be careful of and I know I'm getting close to my time. The reason that I'm skipping over some of this, here's the really scary thing as a DBA. I've just deployed a schema, that's great. A few weeks later, I go through and I see, where's my schema? Here's invoicing.sql. And I get down, we'll do a search for, I see that, oh, the primary key on the invoices table was disabled by, and Craig, I know you can't see it, but yeah, it's your username. The primary key on invoices was disabled for testing by Craig, I guess. And okay, I'm gonna re-enable it. Okay, and I'm gonna save it and I'm gonna do a get push and not think about it. Now next time we do a deployment, we're okay. Well, what happens when in 30 minutes Chef Client runs again? What's actually gonna take place? Let's see. Again, this is the danger. It's gonna run, it says, okay, all that stuff is already installed, it executed it. Everything looks fine, right? What could be wrong? Well, what happened is, if we go into Postgres, we go into invoicing, and let's take a look at the table. We don't have a primary key on it, so it ran. There's no primary key there. The primary key should be this invoice ID. So let's just say, okay, select the invoice ID and the number of instances of that same, which should be the primary key, it should be unique from invoices. And we will group by one, order by one. And what do we see? Now I've got, I've duplicated all of my data. Because we had in-source control, somebody commented something out, I re-commented it, didn't think that a get push at the end of my day, went home, now all of a sudden I've duplicated all my data and I still don't have the primary key there. And the automation system just did it happily for me. As a DBA, schema management still matters, right? So I've seen people do things, my favorite was the customer that was doing some automation where they were comparing these two tables, the public that invoices table. There's one difference here, and it's actually not a difference. All that happened is this customer ID field here is in position two, is an attribute, here it's in position four, right? The automation decided that in order to resolve that difference, the right thing to do was drop table invoices and then create new table. And it did it happily. And I remember that day because they had a little Christmas tree with lights that would change color based on Nagios, and it went red. So I'm pretty sure I'm out of time. This is another one, I just ran this as one, this is the same tool, and I'm not picking on the tool APG-Diff, it's an interesting tool to use, you should use it. Basically what it can do is compare two schemas and tell you what you need to do to go from A to B. This is a massive danger right here. If you look at this, it's going to, if we just pipe this into the database, it's gonna drop a constraint, it's gonna recreate the same constraint, and then it's gonna add a primary key to my customer's table. Now if that were production, and that customer's table were huge, I just shut the whole system down while that thing's being rewritten. So if that's a hundred gig, I gotta wait for bits and bytes to do their thing. So automation is awesome, automation is fun.  We still have developers, we still have operations. Automate what you can, it lets you focus on the business. Just everything in moderation and you'll do well. Thank you everybody, appreciate it.


We will not use that method, that method is longer, we will do the other method, for an easy one less time. Now what is the energy, what is the energy, what next? What is the physical significance of which energy? Spontaneous. Spontaneous? Spontaneous. It is a non-expansion work, non-useful work, non-feeling work. What do you mean by non-useful work? Spontaneous. Yes. A derivative of the useful work, and we also call it as a non-expansion work, and which is nothing but non-TV work. The useful work means what, suppose I will take one chair from here and put it over there, right? There is no expansion in this case, but I have done some work, right? So worker in this case will be equal to work, but decreasing my history energy, that is useful work, non-expansion work. Even the pressure volume curve is not involved, right? So anyway, you do some work, you do that work on the cost of your history energy, right? So this is delta G useful work, non-expansion work and non-TV work, right? So delta G, in case of a cell, it is defined as the charge into potential, Q into V, right? In case of a cell, delta G is defined as the negative of the charge into potential V. If Q charge moves under a potential difference V, then the work done is what? Since the delta G is decreasing to like to minimize or to neglect that the negative curve, we have used this negative sign. It is decreasing, delta G will be negative if you do not put negative sign over there. Work done negative does not mean anything, right? That is why we use the negative sign over there. Q into V is the decrease in Gibbs free energy in case of an electrochemical cell, correct? Now, thing is what? You have to just find out what is this Q and what is this V? That is it, that must be question is that, okay? Q is what? Q is the charge and what is the charge we are using here? N mode of electron, right? The charge here is what? Q into mind now, what is the charge on N mode of electron? N into 1.6 into 6.02 into 10 to the power 23, right? So, this is what? For 1 mole of electron, see the 1 mole of electron the charge is what? 6.02 into 10 to the power 23 for 1 mole and the charge on 1 electron is 1.6 into 10 to the power minus 90. This is the charge on 1 mole of electron, right? Charge is important when we do the electro, light cell there also we have used this, okay? Charge on 1 mole of electron is this and when you solve this approximately in ways, 965 equals 0 to lump, which is nothing but 1 paradigm, right? So, charge on 1 mole of electron is 1 paradigm, charge on N mole of electron is N paradigm, right? Because we have N mole of electron, so Q is equals to what we can write? N paradigm is N into F, yes? Yes. Okay, now what is V? V is the potential difference, correct? But in case of the cell, what is the potential difference in the EMF of the cell? Potential difference is nothing but the EMF of the cell, because of that only the charge is going, right? So, V is nothing but the EMF of the cell. So, when we substitute these two from here, delta G becomes minus N M E. This is the formula of delta G in an electron free cell, understood, right? Now, we know this formula and we have done this in thermodynamics. The delta G is equals to delta G naught minus 2.303 RT log Q. What is Q? Q is reaction constant, everything you forgot. Kc is equilibrium constant, but Q says Q only. Kc is equilibrium constant, that is defined at equilibrium. But at any other point, the ratio of product by reaction is what? Reaction constant Q, right? So, when we apply the condition of equilibrium, right? We have a specific case, because Q is nothing but the concentration of product by concentration of reaction, right? Since we are not taking the case of what? Equipment, right? So, at equilibrium Q is equals to case. Now, when we substitute this, delta G is equals to minus N M E cell. So, delta G naught is what? Minus N M, T naught is basically the standard state. Now, these are coming to be substituted here.  I think it is plus, we have here, delta G naught plus 2.303 RT log. That is why when equilibrium is there, delta G will be 0, Q is equals to Kc and delta G naught becomes minus 2.303 RT log Kc. We have a plus sign of it, right? So, when you divide by minus N F, it becomes minus sign of it, right? So, E cell is equals to E naught cell minus 0.0591 by N log of, or we can write it as concentration of product by concentration of reactant. This equation is non-sequential. When we substitute this R value, T is 298 Kelvin and F is 965, that was 0. When you substitute all this value except this N, because N is what? N is the number of moles of electron that is exchanged, right? So, according to the reaction N will vary. So, I will take this N as it is, when you substitute all this value, you will get 0.0591, okay? So, this equation of this equation is known as non-sequential equation. Electrochemical equation is non-sequential. All other things I have to discuss only with getting on that question, okay? The numerical problem is to get one more. One more important point to write down, this delta G is an extensive property. Extensive property and extensive property are additive. Intensive property are non-arrative, like EMI of this. 1,000 questions. Write down, calculate E naught for the reaction. Calculate E naught for the reaction, Fe3 plus plus electron gives Fe2 plus. For this reaction, you have to find out E naught. The data given is E naught for Fe3 plus to Fe is minus 0.03620. And the data is E naught of Fe2 plus to Fe is minus 0.440. You have to find out E naught for this reaction. The first equation is applied to both half cell reactions and for non-sequential. Apply to both half cell reactions. Oh my God! 0.4 through 3, right? Yeah, 0.4 through 3. 0.4 through 3, right? Yeah. Okay. Okay, you see from these two data, you have to obtain this. So from this data what we can write, it means the reaction is for Fe3 plus plus 3 electron gives Fe3, right? And suppose this is E1 and E naught 1 is minus 0.03620.  Fe and suppose this E naught 2 is minus 0.4401. How do we get the first reaction from this reaction? Fe2 plus on the product side, right? So we will just reverse this reaction and add these two, right? So when you reverse this, E2 becomes positive, right? So when you reverse this reaction, the reaction becomes Fe gives Fe2 plus 2 electron. So E naught 2 becomes 0.4404. When we add these two, what happens then? Fe3 plus plus 1 electron gives Fe2 plus, right? This is what the reaction we have, okay? So for this, the E naught of this reaction will be equals to E1 naught plus E2 naught. And that will be minus 0.0362 plus 0.44. When you solve this, what is the answer you get? 0.40384. Understood? Yeah. But this is wrong. Yes, you cannot do this. Why? That's so much over here. It's not related. It's not related. EMF of the cell is what? Non-addict first class only. EMF is an intensive property. It cannot add directly or subtract. So what do you have to do here? This is correct. Till here it is correct, right? Till here it is correct. But this we cannot do. E, we cannot add directly. But delta G is an extensive property. Delta G we can add, right? So we have to find out the delta G for this reaction. Delta G1 is equals to what we can write? Minus NF E1 naught, right? For this delta G2 is equals to what? Minus NF E2 naught. Since we are adding these two, so for this reaction, if it is delta G3, then delta G3 is equals to what we can write? Delta G1 naught plus delta G2. We can add delta G directly or subtract. Understood? Yes? Now what is delta G? Minus NF E1 naught minus NF N1, this would write actually. NF N1 and N2. Minus NF N1 naught plus NF N2 F minus N2 F E2 naught. And delta G3 is what? Minus N3 F E3 naught. Okay, F gets cancelled. E3 naught we have to find out. E3 naught is equals to what? This minus gets cancelled, this minus gets cancelled. E3 naught is equals to what? N1 even naught plus N2 E2 naught by N2. Now you see, since N1 N2 and N3 is not equal, here N1 value is what? 3. N2 value is what? 2. And N3 value is what? 1. If this is equal to N1 N2 and N3, then this will get cancelled. Then the expression will be what? E3 is equals to even plus E1. So this E3 is equals to even plus E2 if you can write, when all the electron exchange and all these reactions are equal. Otherwise you cannot write. That is one special case when the electrons exchange are equal. But since it is not heritally what we cannot add EMF of the cell or subtract directly. But what we do, we find out delta G. Delta G we can add or subtract according to the reaction. We substitute delta G in terms of E of the cell and then we find out EMF of the cell. N1 value is what? 3, right? 3 into all the value you have to substitute minus 0.0362 plus N2 value is 282. 0.440. Remember this reaction I have reversed. Okay? E3 will be positively but negatively. This divided by what? N the value is? Solve this negative answer. Then what is answer negative? Answer? 0.772. 0.772. Clear 0.772. 0.772. The answer is 0.772. What is the answer? 0.53. 0.53. Shikha is getting 0.53. Is it 0.153? 0.153. 0.153 is the answer. 0.153. Minus 0.153 is the answer. 0.153.