From our studios, in the heart of Silicon Valley, Palo Alto, California, this is a CUBE Conversation. Hi, I'm Peter Burris, and welcome to another CUBE Conversation. From our beautiful studios here in Palo Alto, California, this is going to be a CUBE Conversation that focuses on marketing. It's one of the series that we have where we're talking to CMOs here in the Valley and beyond about the challenges of marketing and the role that marketing is playing in driving increasing levels of value to customers. And to have that conversation, we've got Micheline Nijme, who's the CMO of Zscaler here in the CUBE with us today. Micheline, welcome to the CUBE. Thank you, glad to be here. So, Zscaler. Yes. Tell us a little bit about Zscaler. Let's get started there. Zscaler is really transforming traditional network security. They are a security cloud platform that securely connects users to applications no matter what location, on any device, over any network. And just to kind of put it in a little more context, if you think about how cloud and mobility are changing, how organizations are transforming digitally into the cloud, network security is changing, right? Apps are moving to the cloud out of the data center, and there's no longer a perimeter to secure filled with appliances to secure their data, their applications, and the mobile workforce, while they're accessing these apps, not in your traditional method, they want secure, fast, reliable access. And Zscaler is the platform that was born in the cloud, built for the cloud, to help securely access those applications. Well, you get no argument from me about how important network security is. It's one of the fastest growing and become one of the largest areas of spending within an IT organization. But let's put that aside. Yes. And instead, talk about marketing. You mentioned digital transformation. Let's start there. What does digital transformation mean to marketing? To me as a marketer, I think of how can you run your organization with the minimal amount of manual steps you can take? Why? It's because how do I connect better with my customer? It's going to make me more efficient, effective, save money, right? It's going to cost savings there, but it's going to help me understand my customer better because I'm going to have the data. It's going to give me the data that I need, that single pane of glass, because all of these tools will allow me to have a better access to my customer, faster, more efficient, and understand them better. Well, I've been working in the marketing technology space for quite some time at various other, in various places. But one of the observations I make is that a lot of the marketing automation tools are not growing as fast as you think their tool would suggest. By that I mean, if their tool is that great, by using their own tool, they'd be growing a lot faster than they are. That always suggested to me that the real value of these tools is in how they're utilized by people. So it still is highly dependent upon how the marketing organization uses these tools, and that's even more important in the selection of the tools. What do you think about that? Absolutely. If you think about a tool, it's not moving away from the human connection, it's enabling you to have a better human connection. And if you don't have the right amount of tools, sorry, the right type of tools that allows you to engage with your customer better, then why have them? There's data that shows almost 30% of marketers have 11 to 20 tools. That's crazy. And if you don't have them integrated, if you don't have them working and talking to each other, then they're going to be a hindrance more than help to a marketer. So that also suggests, and you mentioned, we start off by talking about digital and how digital can improve efficiencies and effectiveness, but also how it can transform the role that marketing could play within a business. At Zscaler, what is the role of marketing and how do you envision that that role could change over time? Zscaler, we are so customer centric, right? Our customer is truly at the center and the heart of everything we do, from our customer success team to our CEO, to our CTO, all of our C-level execs, and everyone within those organizations. Think customer first, and marketing is another organization that has put them at the heart. And when we think about a buying journey, that's how we start our any sort of planning, any sort of strategy. We look at how do we engage and drive interest before they buy, of course. How do we engage with them through this purchasing experience? And then how does marketing impact them post-sale? And it takes sales, customer success team, marketing, all working together as a true machine, and that's how we've evolved marketing at Zscaler. So is marketing a cog in the function, or, which of course it is in certain respects, but are you seeing that marketing is expanding its role within Zscaler, becoming more of a steward of customer-related data, becoming more of a advocate for customer experience? Is that happening at Zscaler as well? Zscaler, from the start, to be honest with you, has always valued marketing. So our CEO is a number one advocate and fan of marketing. You cannot have any more happiness as a marketer to have that. So he's been a huge advocate. So I can't say it's been more. I think what we're seeing more is optimizing it. How are we understanding the buyer a little bit better? How do we understand their challenges, their needs, and then what are we doing to help solve that problem? So that's kind of how we're evolving. It's less about having more fans because we've been so lucky to have such huge advocates from sales, customer success, all the way to the exact team. It's been really wonderful. Well, I'll tell you a quick story. This is a CEO at one of the Valley's juggernauts once told me a number of years ago when I asked him what was the value of marketing to this particular company.  Sounds like that's not the role that marketing plays at Zscaler. You mentioned being customer centric and having marketing sustaining a focus, helping Zscaler sustain and reify that focus in the customer. We see a new challenge emerging in the industry. How do we use data to personalize what we offer to customers and sustain that engagement versus violating the privacy? What's some of the challenges that you've encountered as a CMO at Zscaler as you try to find that balance, especially given new regulations like GDPR? Right, GDPR, when you first hear about it, when it first came out, it was a beast. And as a marketer, you start thinking, am I doing the right thing? Am I following? Am I being compliant? But at the end of the day, marketers have had bad behaviors. We have not been looking at our database and understanding how to engage, how to deliver that value to our customer all the time, consistently. You say it, but sometimes you realize in your best practices, it doesn't always come true. So with GDPR and other standards that are coming in, it is enabling us to make sure and look back, are we following what we need to do to be compliant? But are we delivering the value to the customer? And for us, it's been, to be honest with you, great. As a consumer, I love it. As a marketer, as challenging it has been, it has really helped us reset, make sure that we are engaging properly. We are delivering that story that really the persona cares about and not just hitting them with communications over and over without really thinking through of what your needs are, what your challenges are, and then how our solution helps. I've always liked to say, and I want to see if you agree with us, that if marketing focuses on delivering value with every touch, they rarely get into problems about abusing privacy. Does that resonate? Completely agree.  So it is huge in terms of delivering value. So data is essential to marketing. Customer data sounds like it's essential to Zscaler. Correct.  Let me see if you can help me out here. Yes. Connect Zscaler secure networking to making the CMO's job better. So when you think of Zscaler, we have so much data, understanding kind of the usage, the patterns. And so by delivering, by having the CXOs, the CIOs understanding what data is in front of them. We're giving them this visibility that they've never had before. So they know what apps are out there, how they're being used, where there's security risks. Marketing has similar data, different tools, but we have that access to data, understanding how the consumer, our customer, is engaging. Where do they go on the web? I mean, it's a little scary, but we do understand how they're engaging with us, and then we make better marketers. Just as we have better security networks in the cloud built because of the accessibility and the visibility, marketing is getting those similar tools as well. I got to believe that as a CMO who's worried about the customer experience, especially in a world where there seems to be an ongoing onslaught against trust, that marketers also need to feel that they can trust their data, that marketing is becoming increasingly distributed and part of a community to get the work done, and that it's an essential feature of all of this so that a marketer can say, I trust my data and I want my customers to be able to trust their digital interactions with us. Have I got that right? Absolutely, you have to trust your data, but you also have to have trust in how you use your data. If you're not being accountable on how the data is used, you're not being a good marketer. You don't want to take advantage of the data that you do have. You should be able to understand the data, and then how do you turn that around to engage with your audience? It sounds like Zscaler, a trusted, secure, networking, cloud-based service is potentially a real tool that a marketer can use, both internally as well as externally with customers. Absolutely. All right, so one last set of questions. There's been a lot of conversation about how diversity improves a business to shareholders, to employees, and also to customers. So you're often, tell us a little bit about what it means to be a woman in the CMO job in a technology company. Peter, I've been a woman in the technology company. My entire life, and what is important to me is that I'm looked at just as equal as everyone else, but really I feel responsible and accountable and almost a role model to every woman out there. So for me, I hold myself at a higher standard, but because I need to make sure I'm delivering and being my authentic self, I'm able to voice what my feelings are at any time. I don't see myself as a woman amongst a C-level exec. I see myself as a CMO amongst a C-level exec. So for me, it's being accountable, being responsible as a role model for these women who are growing, it's mentoring them. But also, again, I don't differentiate as a woman versus a man. I think for me, I'm responsible as a marketer, as a leader in how do I lead the entire marketing organization, men or women? How do I lead them no matter what? I am their leader, I am responsible for them and I consider them as kind of my family that I'm trying to nurture and grow as well. All right, let's go out three years. Where is Zscaler in the industry and what role is marketing playing to get Zscaler there? We are changing and disrupting how network security is being managed. We are the largest cloud security platform built for the cloud, born in the cloud. And I don't see us, anyone else, kind of coming to us in three years from now. Fantastic. Michelin, Nijme, CMO, Zscaler, thanks very much for being on the team. Thank you for having me. And once again, I'm Peter Burris. Thank you very much for watching this important CUBE conversation. Until next time.


Alright, good afternoon ladies and gentlemen. I am Captain Nathan Flack, current student at the Air Force Institute of Technology. I'm doing research on multi-domain operations education. So I'm here as a research student collecting data on ECSC but also have the chance to present some of the research that I'm doing and to get feedback from you all. So this will be a pretty informal presentation and I invite you to throw out questions or comments as we go here. And really what I want to get at is how are we thinking about teaching multi-domain operations? What does that look like? And what ideas are out there for really engaging people on this topic? So who here knows about Ender's Game? Yeah, got lots of hands. Okay, so basic story, Ender's Game, he uses the military at the time, they use games to teach their students almost exclusively. They're in a classroom environment, in a military training environment, but they're using games. So what does that mean for the military? It was interesting, this article if you go out and look it up, Chip's article, it talks about the military strategy of Ender's Game, and back in the 90s they would have Marine lieutenants and captains read Ender's Game to pull out some of those military strategy type of things. And this is actually something the military has been doing for a long time. We've been using games to train people. So here's, if you can see, here's some airplane spotter cards. So this is not the World War II edition, but they would distribute these cards in World War II. They were just basic plane cards, but on them were the outlines of airplanes. So you would, as you were playing, you would learn what the Allied and Axis airplanes looked like from the front, from the side, from the back. So a way to use cards, a game, to train military folks. Also ran across this one recently, this is Art of War, so that book turned into a card game. It's also just a basic deck of cards with your different suits, but it has some military strategies on there. So interesting, we've been using cards to train people. So this talk is about interactive learning tools in Air Force education and training. So I'll kind of explain some of my research, and also talk about a product that we've put together at Hazard. So here's some background. So one thing that we're working on right now at the Air Force Institute of Technology when I got there last August was this thing called Cyber Education Hub. And essentially, this is a cloud developed platform where people at the lowest levels of our Air Force and DOT can contribute to the content that's out there that we're training on. So it's essentially, it's kind of like a YouTube or a Netflix where anybody can contribute and anybody can consume content and creates social networks around what we're learning and what we're training. So that was something that they were focused on a lot when I got there. But one of the things they were looking for is we're going to have this great platform where people are going to be, but what kind of interactive hands-on, as much as you can be hands-on with a computer, but what kind of interactive training and education tools are we going to be able to place on this and integrate into this platform. But at the same time, you have what's going on over at ATC with the Continuum of Learning initiative and they're not going to remember the name right now, but essentially they're learning ecosystem that they're putting together. And they're trying to make this ecosystem that has all of these apps plugged in. As I dug more, there's not a whole lot of apps that they have right now. They're kind of laying the foundations and looking forward to that. But what are we going to put in that? And then at the same time there's a lot of this talk about MDO. One of the big things obviously conferences like this and the Chief of Staff of Air Force and then also on the Army side talking about multi-domain battle, multi-domain operations and multi-domain command and control. So try to build a research project that took all of these things into account. So first of all, when you talk about educating and training on multi-domain operations, we have to define what it is. So I went to a conference in Europe the European Cyber Warfare and Security Conference. These are some of the same slides that I shared there, but I had to get really basic with the research students there is an academic conference about what is multi-domain operations. That's kind of where some of this comes together. But the goal really is to, that the Air Force is looking at, is mastering that multi-domain command and control and have a few quotes that's going to get at that. But kind of, this might be something we look at and say, okay, it's all of these domains connected together. But in fact it really turns into all of these domains connected to every other domain. And that's where you get a lot of the complexity. So when we think about training and education, how do we simplify this for students so that they can start to grasp these concepts? I'm going to watch that video. Alright, so when Chief of Staff, General Goldfein came into office he sent out a focus area, he sent out three focus areas. The third one was multi-domain command control. And these were some of the quotes that he gave. So the ones involved there. So he's talking about military capabilities working in concert across all domains. And he really padded home, we need a new way of thinking and a new way of training. Then he followed that up with the MDC2 implementation plan that was signed by three top leaders. And it really focused on interoperability but also hands-on training and education. Giving everybody, giving as many people in the Air Force hands-on learning in multi-domain operations, multi-domain command control. So trying to bring all that together, what are some of the challenges for multi-domain operations in educating an entire force in it? So three, these six things, I'm sure you all could probably come up with more or come up with this exact same list. There's a lot of complexity to it and then things are constantly changing. You're trying to take cultures and knowledge across different domains as well as different services and try to integrate that together. And obviously General Goldfein's been talking about this a lot but he really sees it as we don't need a new technology to make this happen. We need a new app. It's going to be the applications that connect all of these shooters and sensors across those domains that are going to make this happen. So thinking about this, where are the gaps today? So I see kind of three gaps that I put into this. So kind of lack of understanding of MDO. At least it's hard to come to a definition of MDO that kind of everybody can agree upon and gather around. Availability of collaborative interactive trading tools is kind of a gap where we're moving towards filling those holes in more areas than just this but that's definitely still a gap. And then personnel are overloaded with training. You know, we've heard death by power point right? So how do you cut through all of the training that's out there to really engage people and get them to think about this topic? So what's the need? We need more learning tools to engage our personnel. We need experiential learning that's going to make people put these ideas and concepts in their own words so that they can understand it and they can express that to others. And then resources that are ready to be integrated. So the big thing about the MDC2 implementation plan was that a tag ATC said, hey, you need to start integrating MDO education into all of your current courses. Now, it didn't say all of them, but it's starting at what we already have out there today, integrate MDO there. So how do you do that?  They have to go out and do the learning. They have to read for themselves. So how can we build a tool that somebody can take their classroom in an efficient way? So here's a proposed solution. So to build a serious game to bring these concepts together. So both an experiential learning in the classroom and something that can be integrated by instructors at various levels. Serious games, there's a lot of research out there in the commercial world about them. It's been a growing area of research in the last about 40 years. So they provide hands-on learning. They engage personnel and the build long-term competency is what we've been seeing from the research. So we wanted to take that and build a serious game that would use all the different morphine domains and integrate them together in a way that a student can take and to use in a classroom environment. We wanted to paint the big picture so people start seeing how these little things connect together. Relatively easy to learn in a classroom. As you'll see, probably get the idea that the game we ended up building ended up being very complicated. That's one of the big pushbacks or critiques we get about the game. But I would say I'm very much open to suggestions about how to get after integrating six games together in a game that's not a complicated. If you have any ideas, please let me know. You see the Venn diagram on the right. We're going to spend some time in that. Where do serious games fit in our world today? You'll see the Venn diagram learning, games, and simulations. Games, think of Starcraft 2 or Settlers of Catan or throw out your favorite video game or board game. As I'm going, maybe start to fill in your head examples of what you see out here. Obviously learning is going to be very much classroom instruction. There's a lot of different types of learning, but that's a big one. Simulation and getting into the simulation games a little bit would be like a flight simulator. So you have those that very much are on the game side of that spectrum. Very much on the training real-to-life side of that spectrum. You bring in VR and some of those other capabilities. Here's one that's more towards the training and exercises. Command, modern, air-enable operations, if anyone's heard of that. Very much tries to integrate in a computer game, a real-world military capabilities. They're in the middle between learning and simulation would be more games. BogSat, who's heard of BogSat? I got one bunch of guys sitting on the table. You have a bunch of people in a room thinking of your tabletop exercises that are playing out in a scenario and trying to think of the different actions an enemy or your friendly force could take. Over on the other side between learning and games you have an edutainment. How to do newware that came out in the last 10 years. So I think that we're in the world that's coming to San Diego. You're thinking of edutainment. It's entertaining and also has some real-world learning built into it. Serious games kind of sits there in the middle. I wanted to share Foldit. If you've heard about that, it's a game that's real-world protein folding. Foldit came out in 2008. Three years after its release, players of the online puzzle game Foldit helped decipher the crystal structure of the Mason-Pfizer monkey virus and AIDS-causing virus. Although this solution had troubled medical science for the preceding 15 years, the combined effort of thousands of players produced an accurate model of the enzyme in only 10 days. So they turned this problem that they had of, hey we need to figure out how these proteins and enzymes fold together. I'm not a microbiologist or anything like that. But they built it into a game so they could spread out that the brain power that it took to do this and made it in a fun way and engaging way that people wanted to spend time figuring this out. So we built a collectible card game. And some of the things that kind of got us to the collectible card game was it has the ability, you have different cards in the game we built. One version has 57 cards. This version that's up here has 40 or 54 cards. But you could take one military capability or one capability to distill it down to a card. We've got a picture and some things and I'm going to pass around the cards here in a minute so you can kind of see what they look like. And then you can put some traits, you can put some ideas about how that card interacts with the environment around it. And then you can bring in another card that does the same thing. Distills a capability down into its essential form and kind of outlines a way that interacts with other cards. And then when you keep on adding more that you're building this environment that now you have to think about okay, there's this one capability. How does adding this or how does using this in a way work itself out and affect that environment? And that's why we focused in on this collectible card game. And they're actually very popular. So Earthstone might be one that you've heard of. It's an app or a computer game. Probably can't see the numbers in the app. But 1.5 million reviews and 4.5 stars. UGO is another one that's very popular. It's got 1.7 million reviews and it's almost got at 5 stars. So a lot of popularity. Pokemon might be another one last year. So it would have been 2018. There were 2.1 billion Pokemon cards sold worldwide. So this is a very popular type of game and so we wanted to leverage that and match it up with the military capabilities. So we built the game called Battle Space Next. So I'm going to pass a few of the decks of cards around. So the game was actually the game was built by Colonel Allen Lin who's an instructor at AFFIT. He's since moved on doing some other things. And he was actually gone before I got to AFFIT. But I picked up his game that he developed and tweaked his ruleset and tried to engineer it to be able to work in a classroom environment. And that's where Battle Space Next came from. So it seeks to integrate cards, capabilities for all these different domains. So we included some electronic warfare capabilities as you'll see. Also information operations. And then we focused on the cyber side of the game. And really that cyber integration piece is where I think we get a lot of value out of this game. So this game is coming out of the Center for Cyber Space Research at the Air Force Institute of Technology. So we were really interested in that cyber education side of the game. So what we did is we took a cyber kill chain that's pretty commonly known out there. Trying to remember the Lockheed Martin I believe was the first one to come out with it. But distilled that down into some basic steps. You've got your scanning and reconnaissance that you've got your gaining access and then you have your exploits and attacks. And so we tried to come up with real world capabilities like NMAP and like distributed denial of service, firewalls, rootkits, and try to put those in that construct to help students learn that. Here's some of the lesson objectives that we've put together for this game. And you can kind of read through those. This would be something that an educator as somebody who's doing a class right now is going to be very interested. They're looking at different training resources out there. How can I use that to meet my requirements of what I need to teach my students? I think a big one, and it goes back to game theory, is the anticipating, adapting, and responding to an enemy or to your environment. I think that's a takeaway from the game that we're already seeing in some of the preliminary data we've been able to gather. So really the game is about integrating hands-on MDO training and education into whatever military environment would have a need for that. So this kind of gets into the research side of it. I'll go through these pretty quick just because it's not too relevant to you, relevant to me right now and my advisors and things like that. So trying to get at the efficiency, that's the integration piece into a classroom, and the effectiveness. That's how the students are perceiving it. So if they're engaged, if they're having fun, if they're learning, that's good. That's really good over here. But for the instructor side, does this meet my requirements? Does it fulfill my learning objectives? And how much time is it in energy and effort and money is it going to take to get this thing, whatever it is, might not be a game, and get it into my classroom. So I'm doing a human subjects research experiment. You'll see on the next slide we've run it through several different organizations already. Like I said, I'm here at Air Command and Staff College with their multi-domain operational strategist program, their ENDAS program over there with Dr. Jeff Riley who presented at the conference yesterday really getting his students feedback on the game to improve it and also to see how it's performing in the classroom. So here's some of the different engagements that we've had with the game. Getting some good feedback. I think I've put together probably about four different versions of the game now. I'm guessing the version that's being passed around is probably not the final one. But different versions of the game have gone through different organizations. An exciting one just towards the bottom. The newest career field in the Air Force, at least at the time, was the 13 Oscars, which is the multi-domain command control career field. So these got to go down in their first month of their very first course to their initial skills training and run them through the game and collect some data. They had some fun with it. They had some good critiques. They found some mistakes on the cards that only a 13 Oscar would find. So here's some of the environments that have already been targeted with the game and some other potential ones. So part of this presentation might be, hey, there's this research going on to AFFIT and maybe you want to leverage that to take a picture of your environment. So thinking you might not be in a military classroom but what about unit training? Is there a need for the people that you work with, people that you lead to have MDO experience maybe you bring in a game like this and you can actually it's kind of a fun thing. Just kind of have people go and do it but then in the end bring out the discussion points that are really important to you. So I'm a poor researcher so I'm looking for other environments around this experiment to collect data so if that interests you I'd love to talk afterwards. And here's some of the resources that we're building putting together at AFFIT to make it integratable into other environments. So Instructors Guide is going to give my reading list essentially for MDO and what I've been able to find and put together and leverage from other people along with videos of how to integrate this into a classroom maybe some example debrief discussion questions to have afterwards things like that that's going to help somebody take this and plug and play into their classroom. Here's some of the research efforts that are ongoing. So short term right now we're working on a digital game so taking a card game and putting it into a digital environment that has slowed down over the past couple months unfortunately with changing priorities and things like that but I think there's a lot of potential there for taking this because you're able to reach a lot more people especially as we move more towards the continuum of learning environment. You're going to have people that are sitting at a computer away from other students and this would be a way for them to engage with the game. And then long term I think there's a lot of potential for a game like this and you know I think training on the electromagnetic spectrum and how that influences military operations I think there's a lot there that we haven't explored yet. We get to the cyber piece of that but as General Holmes pointed out there's this other thing over here called EMS that we really need to start figuring out and might it be more the umbrella term for cyber operations so more exploration of that I think a game could be a good way to attack that. I was hoping so maybe it might be you know a year or two year and a half it might depend on the next student class at AFFIT if anybody picks it up and carries the baton from what I've been able to put together I'll show you in the next couple slides the digital game has kind of redesigned it we're putting it together now. We have very rudimentary cards moving around on a screen but that's about it. Also when you're looking at a game you can play an AI engine so you have an actual computer player that you can play against but then also you can have AI on AI there's a really interesting video out there about Starcraft 2 and an AI that they built to play Starcraft 2 and that kind of opens up a world of possibilities when you're looking at military training and how are we helping decision makers and strategists and planners to do this the best way. So here's an initial design of what the digital game could look like so you kind of have your hand down there at the bottom and then you have your playing area where you're playing different cards than other labels of things. It's a little hard to there's a lot going on on the screen for the crowd that doesn't know about the game and how to play. Alright so in conclusion we looked at some of the background, foundation of the game gap analysis and our proposed solution and then we got into how we're going to answer these questions about efficiency and effectiveness and then we launched into future research so I appreciate you all coming out. I'm not going to keep anybody around too much longer because I know this is the last presentation of the day so I'm sure there's some things that you're all looking forward to get into tonight but I've got my email up there, I welcome any feedback now or later on through email and I know some people have been asking about how do we get this game, things like that so I would love to keep a discussion going with you. So I'll open up now for questions. Yes sir? So the card game, the cards are they available to anybody who would like to use them to get them to use content? You do that, how do we? Yeah so we've got like 30 sets of the cards right now that I'm kind of taking around and doing research with so that could be one avenue, we could get interaction with the cards but then also they're PA approved and so we can actually take them to commercial printers and we can work on that so we could work out something where I can get you the card files and then you can take them to whatever printer you would want in order to print those off. We ordered 30 decks of cards, each one has 54 cards and it came out to about $10 a deck, so pretty reasonable if you were looking at ordering a lot first squadron. I guess they alluded with the cards there were some kind of instructions or something to make sure the whole process. Exactly, yeah so it's definitely still in flux because as I'm doing my experiments I'm learning more about actually how to do it but we do, there's like a very complete instruction guide for how to play the game. I can provide you with today if you wanted it. How long did you work with Dr. Riley? So I got to meet Dr. Riley at the MDO Working Group that we had at PathFit and then we kind of had a back and forth about using the game in his classroom. So it's only been probably about five or six months now. Actually before I met him I did some of his articles on Over the Horizon which is a great resource for MDO education. Yes ma'am? Did you do any research on the No, I'm not aware of that. I'm assuming it's modeling and simulation through like the Air Warfare Center or something like that. Good, thank you. It'd be great. That's one thing I'm exploring as I go to now write all this down for a thesis is I think there's a lot of evaluation that has been going on in the war games and simulation and trying to pull the goodness out of it. Anything else? Does the game currently only have American assets? Yeah, good question. So it's all US military assets not because we couldn't do other things. There's kind of a requirement but we'd like to keep that at the young class level and publicly releasable level because that gives us a lot more ways to get in, especially when you think about recruiting, ROTC, some of those environments that could really benefit from a game like this, we want to keep it at that level. But if we're taking it to a different level I think having it publicly releasable you could have some of the assets have a card deck that's actually modeling the enemy. But actually from our senior leaders it's kind of been more like that's kind of an aggressor squadron type of role. We actually want the common person out there to be more thinking about our capabilities and how to leverage those. So that was interesting to hear. Good question. You could take this to the TS level if you want. I build cards with a PowerPoint file. So you take the PowerPoint file, it's got the templates in there, you type up the different things, put the card parameters in there, and you've got a card that's ready to go to the printers. So there would be a lot of possibilities there. You're thinking who's going to print it? I actually, early on in my development I print out paper copies, cut them up, do my little arts and crafts thing. And then they make the gaming world is great. They make these little cards and you just tuck them in there and you can play with the card sleeves. So there's nothing commercial printing that doesn't stop us from using this in an environment that might not be publicly releasable. As far as ideal audience that you would do some research with your recommended group? Yeah, so I've done RTC cadets, so sophomores and juniors in college all the way up to multi-domain strategies and the 13 Oscar career fielding. So that's pretty wide range. I think as we're seeing the game part of the research is to, okay, where's it be the best place? I think RTC, OTS, and maybe SOS the junior captain's kind of level because you start, you want people to kind of think of the big picture. And anybody could take the cards and kind of look through the cards and it would be a learning experience for them but who would actually be able to engage with the strategy part of the game, which is that higher level learning on the blooms pyramid. Yes, sir? Have you ever thought about considering coming down to the 39 O.S. schoolhouse where we teach IQT for all of the systems in Syro? Yeah, so haven't made a specific connection with that, but there's actually been some connections with Keesley Air Force Base and some of the training squadrons there. So later on in the next quarter, hopefully going down to undergraduate cyber training and then some of the other cyber tech schools for some of their training squadrons to run and experiment with them. So that's on the radar. But I'm willing to work with anybody to try to hit that. Training environments are going to be a little different, right? So I would say that the tool is more engaged towards an education type of thing where you're talking about the big ideas, the big picture. When you're thinking about training, you want very tight restrictions on whatever tool. So I think there could be possibility of taking the template of the card games and things like that in the rule set and applying it to a training environment but with different cards in a way that's going to focus in on one or two learning objectives. So there's definitely more discussion there. And whenever you're thinking about trying to fit the whole world of video into a card game, there's going to be some complexities that just get lost that aren't going to, you know, a trainer's going to look at that and might not be that valuable to them. How long does the game last? Yeah, so the ideal would be about 30 minutes. So I had some play testers that were looking at it a couple weeks ago and seriously these guys played for three and a half hours going on. Like guys, it's not really what it's meant to do. So, you know, that was a concern. So the experiments, I say, four hours of interaction with the card game to really get a handle on what's going on. And that's from very first opening the box, reading the instructions all the way through to playing a few games and then doing the debrief and discussion part of that because that's such an important part of these serious games. You know, just play it and then everybody goes home. You know, talk about the experiences that the different players had and try to draw out the real world implications of those things. So that debrief discussion on the back end is a very important part, especially when thinking about a classroom. So that, yeah, that four hours. Is it only 1v1 or can you make a multiplayer? It is 1v1. The way I do is when people are first learning it, they have a strategy partner that they're going to look at the cards together with them, think through it together, bounce different strategies off of one another and then play a demo game where they play each other and they, you know, see how their strategy fares kind of thing. And then they go and play the opponent, which is just another person in the classroom. So probably for future research, but in the mid 90s, I got to go and check out the Virginia Tech ROTC unit and they were, the colonel that was ahead of that unit that created a fight simulator, you know, aggressor versus Air Force in your red blue. And then built an Excel based POM process so you had to POM all your war fighting assets for that you were going to take into the fight.  Yeah, that's good. Something for future researcher for the next thesis. That sounds very similar to something that a 13 Oscar instructors pointed out. It's like, hey, we could use, like really use this for planning and like even authorization of different cyber, you know, weapon systems and things like that. Where now you're having to go and make a case to whoever used the card in your games. So that's a little more long term, which could be a good application of this. You give a deck to all the students on day one and then you use it for this learning objective, you know, the second week and this learning objective, the fourth week. And then, you know, in the end you build up to a complete strategy. It's very similar to what Dr. Riley is doing. He says, okay, we're just going to focus on ground. We're just going to focus on maritime and then they're building the students up to bring it all together. Anything else? Alright.


A man sets out to be a soldier, he hopes to accomplish many things. He wants to serve his country, he wants to carve out a career for himself, he wants to see something of the world, he wants excitement, a challenge, he wants to prove himself a man. I think I've accomplished all of this. I've got my campaign ribbons and my sergeant strikes, I've served on three continents and been in 15 foreign countries. About interesting assignments, I've worked with top-notch people, I've been given more responsibility than I'd have dreamed possible. I've had a lot of satisfaction out of being a soldier, and I think I've been a good one. One thing I'm positive of, in my 14 years in the Army, I've never had a more satisfying job than I have now, as a drill sergeant. A year ago, I didn't know what a drill sergeant was. I was sitting in a PX in Kaiserslautern, Germany, wondering where the Army was going to send me next. It was then that the Army's new drill sergeant program was just getting underway. The drill sergeant concept came about following a directive from the Secretary of the Army to U.S. Connarch, Continental Army Command. His orders were to revitalize Army training, and in particular, basic combat training. There's no doubt that our analysis has proved that the basic combat training program can best be revitalized by the assignment of more and better sergeant instructors at the lowest possible level. This new basic combat training program shows much promise. As you know, through the Office of Director of Individual Training, Connarch re-evaluated its own methods. Good, well-motivated combat soldiers. The assignment of additional or more sergeant instructors with their hour-to-hour and day-to-day contact with the basic combat trainees will produce more effective soldiers who have a better attitude towards the Army. Connarch's research revealed that the early attitudes formed by recruits influenced their entire Army career. Soldiers who had a positive attitude toward the Army generally looked back favorably on basic training. So a very simple hypothesis suggested itself. If you could improve the basic training environment, you could improve the performance of the entire Army. For the recruit of today is the frontline soldier of tomorrow. The decision was to change from a system of centralized training committees to unit-level instruction, which is considered the prime recruit training level. And to provide the personal contact needed, Connarch created the role of drill sergeant. The man underneath the broad-brimmed campaign hat became the heart of the new system. The best non-coms in the Army were needed for this program, and Connarch set out to attract them. When my CO in Germany said he was recommending me for drill sergeant school, I didn't know what to think. Met him all, and I'm sure you'll find the standards very high. As you know, the higher the standards, the better the reward. And the Department of Army doesn't expect you to get out there and put out all of this effort without some rewards, some of which will be free laundry, issue of extra uniforms, and it is a stabilized tour of which you'll mean something to you as a family man. Sounds real good, sir. And another advantage is the promotions, there's additional promotions looking out for the drill sergeants, and this will probably offer one of the biggest challenges to your personal leadership that you've ever experienced. When I returned to the States, I was assigned to drill sergeant school, one of just six in the whole country. Good morning men, welcome to drill sergeant school. You're all top soldiers or you wouldn't be here. A lot of you have been in the Army and in charge of troops for a long time. Many of you have been away from the basic fundamentals for a long time, and are rusty on the finer points of training recruits. Is that correct? As far as the Army is concerned, there is only one correct method to conduct dismounted drills. To give PT, to mount guards. At the drill sergeant school, they review every subject taught in basic combat training. They study concepts of leadership and motivation, and most important of all, every student is taught to be an instructor. The complete nomenclature of this weapon, the M-14 rifle, is shown on this chart. Later on, you will learn to disassemble and assemble this weapon. The M-14 is magazine-fed air-cooled. It is also a requirement of the new program that all officer personnel assigned to basic training regiments be similarly indoctrinated. At each of the six drill sergeant schools, there is a short course for company grade officers. In order to indicate to you the magnitude of your responsibilities and the need for superior training, I want you to note, Lieutenant-Colonels and above, who are assigned to Army training centers, are required to attend a three-day senior officers orientation at Fort Monroe, Virginia. Members of the reserve enlisted program. After four months in one of your training centers and receiving basic combat or advanced individual training, approximately 212,000 of these young soldiers enter the replacement stream to supply Army commands worldwide. The remaining 145,000 return to their parent reserve or National Guard units. Truly, your training centers represent the heart that pumps the lifeblood to the Army's ready forces. Upon completion of the required six-week course, new drill sergeants are authorized to don their distinctive campaign hats. Georgia first-class voice, Georgia first-class stout. Another milestone in their professional military careers and another moment of pride to be shared by their families and friends. The new drill sergeants are assigned to a basic training company at one of the Army's basic combat training centers. Gentlemen, you know that the degree of proficiency that each trainee attains during his eight weeks of basic training here depends primarily on how well each of you do your job as drill instructors. Now, we have been discussing the training program for a second week of basic training, and I'm sure that each of you know your individual responsibilities. Now, do you have any questions? Sir, in reference to Thursday's noon meal, we eat on the range or we truck back to the company and have dinner in the mess hall. Thursday's meal? Yes, sir. We will eat the Thursday's noon meal at the range and then we will march back. I plan on departing the range approximately $1,300. Are there any other questions? Mr. Olsen, you're dismissed. I was as nervous as a recruit myself those first few weeks with my new company, but I remembered what they had taught me in drill sergeant school. And as I began to learn more about the recruits in my platoon, I began to see more ways in which I could help them get off to a better start on their Army careers. For the more than 350,000 young men who enter the Army each year, basic combat training is a period of transition from civilian to military life, a difficult period for many. Inductees or reservists in for just six months, the Army is genuinely concerned with each of them as individuals about their health, about the cut of their uniforms, about the type of Army career they are best suited for. Although they are younger, recruits today are bigger, healthier, better educated, usually with high school certificates and more carefully selected than ever before. Do you have a driver's license? Yes. Your date of birth is 20 December 42? Yes.  Yes. No defendants? No. Do you wear glasses? No. What did you do in civilian life? What was your main civilian occupation? Uh, farm. You were a farmer? Mm-hmm. Did you operate any machinery, elevators? Yes. Large trucks? No large trucks, but large tractors. Did you operate small vehicles such as a pickup trucker? Yes. Would you like to be a light vehicle driver? Yes, I would. You would? Beginning at the reception station and continuing to the time they complete basic combat training, eight weeks later, they are under the constant guidance and supervision of their drill sergeant. His job is to teach soldiering. In fact, he gives the recruit the majority of instruction received in basic combat training. He teaches them the new method of hand-to-hand combat, a combination of judo, karate, and old-fashioned street fighting. It's more deadly because it utilizes a man's natural reaction to kick or strike when under attack. There are 12 drill sergeants assigned to each company, three to a platoon. We get to know these soldiers pretty well, each one. We ought to. We're with them from dawn to dusk every hour of the training day. After hours, we're available for consultation, too. This isn't the kind of a job you can walk away from when retreat sounds. I guess you might say that we're a combination instructor, advisor, friend, big brother, training specialist, and even father-confessor. We have a responsibility to see that each and every one of these recruits gets through, no matter what his problems are. We feel that if a recruit has difficulty, it's not that he failed to learn. It's that we fail to teach. That's why we keep a close watch on our platoon and help each man all we can. Yes, Sergeant. All right, good. We'll have a little bit of trouble with this written test. I want you to go over a few of these questions and answer them for me here. I'd like you to read those questions now and the answers to me. But sometimes a recruit just needs more instruction. If this is the case, we recommend that he be recycled. That is, put back a week or two. That's correct. That's good. All right, read the second one now. Time is the crucial factor in training today. D, if you are now or at any time, it was awarded. That's correct. All right, read the third one now. There are still some people who have this mental picture of Army basic training as it was in World War I, the combat training. To accommodate shorter periods of military service, the Army has streamlined the course to eight weeks. That leaves only 44 training days to turn a raw recruit into a fighting man. Just 352 hours to make a soldier. There are just 12 hours for bayonet. 4 hours for grenade. 8 hours for first aid. And 3 hours for military courtesy and customers. He must also be made familiar with such things as night firing, individual camouflage, and new methods of protection against chemical, biological, radiological warfare. The objective of basic combat training is to develop a disciplined, highly motivated soldier who is qualified in his basic weaponry. Physically conditioned and drilled in the fundamentals of soldiery. Today's recruit needs incentives, just as their incentives for us to become drill sergeants. This is one. Our honor board. We use it to stimulate rivalry between the different companies in our battalion. Competition gets pretty spirited, such as in pugil stick training. This is one place a man gets a chance to use his full strength against an opponent. Other incentives are the rifle marksmanship awards, especially the sharpshooter and expert medals. Every recruit wants to be a hotshot with a rifle. On the 25 meter range, the recruit learns the fundamentals of the M14 rifle. How to aim it. How to zero it in. How to fire it from the basic shooting positions. The second week of range work is devoted to basic rifle marksmanship. The new system substitutes realistic silhouette targets for the old conventional bullseyes. Finally, the recruit is ready to fire for record. Targets pop up at ranges of from 50 to 350 meters to qualify as a marksman and pass basic training. He must hit a total of 30 or more targets out of 84. 45 hits make him a sharpshooter, 60 an expert. In our company of 235 men, we had 108 sharpshooters and 12 experts. Only three men failed to qualify. And after a little more help, they made it two. That's pretty good shooting. We like to think that our personal coaching helps the men score better. Our boys did well on their PCPT too. The PCPT, or physical combat proficiency test, is part of a standardized series of tests given at the end of each training cycle. And combat boots is a pretty fast mile. Along with physical training, the recruit is tested on every military subject taught during basic combat training. However, if he fails to achieve the minimum score, he is dropped back for additional training and retesting. In the Army, this is called recycling. If he does extra well, he may be one of the top 20% in each company, eligible for immediate promotion and an increase in pay. Another challenge to the recruit is the week he spends on bibwag and on the combat courses. Under safe, controlled conditions, he gets a first-hand feel of what combat is like. Or, as the former Chief of Staff said, the Army no longer has time for combat shakedowns. When we go today, we could go shooting. Week cycle came to an end. It felt sort of lonely. We had a good bunch in our platoon, and I remembered all of them. I remembered the ones who were always first in line for Chow who ran into personal problems. I remembered the few who got sick. I remembered the one who didn't make it. I also remembered the fun my gang had whenever there was a few minutes for relaxation. I remembered how young they looked that first day. I remembered how proud I was of them yesterday at the graduation review. And how proud they were of themselves. Ready to go on to advanced training. My job was done. But tomorrow, a new job begins. The job is a job being well done, according to this paragraph taken from a letter written and signed by an entire platoon of former recruits.


We'll get started in about two minutes. I have my time, the official time from AT&T, is 4.30. But our clocks in the hallway say 4.28. So I imagine most people are looking up in the hallway. So we'll wait to give people a few minutes to find us back in the deep, dark corners of the conference. So I want to thank everybody for taking some time this afternoon to spend it with me. My name's Greg, and we'll go through a little bit more of a formal introduction. Here's this is making too much noise. So I will take off my official garb that they made me wear. Hopefully people have had a chance to make it down to the booths and see what's going on down there. There are some good demonstrations, some good conversations happening I've seen. And I would like this to be interactive. So as we have discussions, feel free to ask. I will do my best. There are very bright lights, so I sort of see bodies out there. So wave your hand if you have a question, or better yet, walk up to one of the mics. The reason we ask you to walk up to one of the mics is they're streaming this. And so we want everybody around the world to be able to hear your question, because it's probably the same question they have. So if you do that, we'll be able to share that knowledge with more people. Now the clock's going backwards. It says 4.27. I'm not sure how that's working. Well, I'm going to go ahead and get started, because I really dislike people who are late. So we're 90 seconds in, and we'll start over. Thank you for coming today. My name is Greg Prisbee, and I'm here to talk to you about you must be 12 factors high to ride. Who here is a developer? There we go. That's good. Who here is operations? You have pretty much the same hands. Anybody operations and not a developer? Well, everybody here plays that wonderful DevOps title. And what's funny when I found out about DevOps, there was a talk about it, and people were like, oh, DevOps. And they described it to me, and I said, well, what is this DevOps? And I said, well, you know, you're right code that's in production that does stuff.  I mean, that's just how I grew up. That's how things work. And to that point, so that you have a little bit of understanding of where I come from, because everybody's going to have a slightly different view, a different background, different experience. So I started writing code when I was 11 years old. Yes, they had computers back then. Yes, they were available to people. They were time shares, and they were in the school. But you still had the ability to write code. Anybody remember the Star Trek game? The original Scott Adams adventure game? Oh, come on. I can't be the only person that old in the room. No? So that's how I started. And there was a book called The Big Book of Basic. And it was this giant book. And you sat there and typed in all the stuff. And that's how I learned how to program. So from there, I programmed, I did war games for the United States government for a while. And then I got tired and decided to get out and do consulting. So now I was no longer coding. I was also starting to architect. And I was starting to design. And then sometimes I would do the coding or sometimes push it off. I stayed away from QA as much as possible. Next thing I did was someone said, hey, we need a database. So I really dislike databases. But there was a person named CJ Date who wrote SQL. So he's the one who invented that language. And so I got that book. And I actually had to implement an SQL engine. So I learned a lot about that. And then when I went into production, I was a guy supporting it. So again, already doing sort of that DevOps. Moved into operations, true operations, supported over 45,000 servers in a curve roast infrastructure with a team of nine when I left. So understand that whenever you're doing something, you want to automate it. You don't want to repeat. And so that's a theme that's going to be coming back during this. To let you know I've been doing open source since 92. I was doing free software back before it was cool in the 80s. When the 90s when open source came around, I got into that and started with Linux. So I worked for VA Research, VA Linux. I worked for SUSE Linux, or Zuza if you're German. And I worked for Red Hat. Now I'm at Mirantis. So I've gone through a number of companies and seen a number of different things happening. And I want to share those experiences with you today. Pothole. This is DevOps to me. This is 12-factor right there. And how is it? Well, I'm driving to work one day, and I see this. And I say, you know what? Wouldn't it be great to have an application available to me that I could say there's a pothole at this location? So I have an idea. I have a problem that I'd like to solve because this people can blow out your tire or whatever so you get a flat, throw alignment out, cause an accident. All sorts of bad things can happen from this. So from a public safety point of view, it'd be very important to be able to solve this problem. So I have this idea. I want an app. Or I want to have a website where I can enter this information. So now I have an idea. Now in traditional, so this is one idea, in the traditional world, you're going to have two modes of IT. And what are those two modes? And if you were at the keynotes on Monday, you heard them, and you heard our CMO Boris explain his thoughts on Gartner and pieces. We'll talk about that in a little bit. But you have the traditional IT, the enterprise IT, the legacy IT. And in that environment, if I had this idea and I wanted to develop it, it would probably take a little bit of a while. Not writing the code. That's the easy part. But getting a place that I can do the development, well, maybe I could do it on my laptop or my company computer. But then I need to give it to QA. And I need to explain to QA, here's the operating system. Here's the tools that I need. Here are the libraries that I need. And let's put all those pieces together and put my code on top of it. And that can take some time, especially if I don't have a lot of servers lying around. Or the servers there are running a different version of the operating system. And they don't have the tooling I need. So that could be days, months. I have customers, it takes weeks, I'm sorry, it takes months for them to get those servers. It takes a while. And then production. So again, my idea in writing the code, I could do that in an afternoon. But to get it into production may take months. And that's what we mean when we talk about the bimodal IT, mode one, the enterprise, the legacy, some of the parts that are slowed down. That's from the infrastructure side. There's also coding pieces. And we're going to talk about different ways to code when you move into mode two or the cloud apps. And so that's what you'll hear people refer to as cattle. The idea is that I can get as many of them as I need. And when I don't need them, I get rid of them. However, I want to get rid of that cattle and only keep the ones that I need. So I just pull them, use them, and throw them away when I'm done. And that's really what mode two is there. So I have the idea of an app. And now I need to try to get it out to the market quickly. And we're going to do that with following some principles around 12-factor app. So one, I have an idea. Two, I have modes of IT. Three, I have parts. So this was actually not part of my slide deck until Monday night. To me, this was a known thing. The technology is easy. OpenStack has gotten much easier over time. That's not the problem. The problem is how do I get people to change the way that they work to be able to consume? How do I get them to fail faster? How do I make life easy for them so with the push of a button they get everything that they need to be able to do their work? Then I'm going to talk a little bit about the process. So how do I take that code? And then through process, make it available for production. And I want to automate that process. And so you'll hear terms CICD, continuous integration, continuous deployment, and possibly continuous delivery. Most people are into continuous deployment. Some actually go all the way through to continuous delivery, which is installing it into production. So what we're going to talk about is how do we get those people to change their methodology and how they like to work? What are the processes that they need to define to be able to do the work? So here are the people. The people, as I said, each one up there has a skill. They have a way that they like to work. And what we want to do is to bring them all together under a common goal and make their life easy to be able to deliver these Mode 2 type applications, these Cloudy applications, these 12-factor apps. So that's what I was talking about. One of the things I talk about is you'll hear people talk about we're going on a journey. So we'll see if through these few slides you'll understand the journey. So here's my process. How do I get from point A to point B? How do I take the code? How do I build it and deliver it into QA so that QA can do their testing? And then how do I go to production? I need to find my roadmap. I need to find the process to get there. And so this is the process part. And then finally, we have part three, which is the technology. And the technology, as I mentioned earlier, is not the hard part. We can get OpenStack. Heck, you can buy an appliance that's drop-shipped with two power cords, right? So I can plug it into different PDU so I can withstand losing a PDU into the data center. I can get a couple of drops up top so I can get routed into the proper networks. And I'm ready to go. So being able to consume OpenStack, or be able, I'm sorry, to deliver OpenStack, that part's easy. But how do I actually use this thing?  We'll talk about the app that you're building, the CI CD pipeline that you have in there. Maybe you have some sort of application or orchestrator that's part of it also. So putting all those tools together and understanding the process and educating the people allow us to solve that one idea of how do I build an app to identify pallet holes so they can get fixed? Any questions so far? Are you by following me? Yeah, making sense? My whole goal, so this is the beginners, is to introduce ideas and they start to get people to think a little bit differently. So what do we have here? We talked about the one idea, two modes of IT, the three parts, and now we'll talk about the four words that you need to keep in mind. Performance. So people don't think a lot about performance, right? Hardware has become cheap. Anybody who's been developing for 30 plus years remembers when hardware was very expensive to run in a timeshare or to purchase, right? So you had to write efficient code. That's no longer the case. People don't worry about memory leaks anymore, they don't worry about those things. It's like I'll slap something together, I'll just crab this piece of code, that piece of code, put it together and let it go. In a web app, right, in these 12-factor apps, performance is important because you're scaling out to what's referred to web scale. I'm not gonna have one copy of this code, or 10, or 100, I may have 10,000 copies of this little piece of code running out there. And if I'm having problems with the performance, maybe it's too slow because it's going out the disk. So hey, let's do something in memory instead, right? Maybe I'm spending too much traffic across the network. So how can I solve that problem, right? Maybe I'll write to a backing store that I can then pass a pointer to, right? An object pointer to be able to pull that data back. So you need to think about that. There are plenty of tools, there are plenty of books, there are plenty of tutorials on how to do performance. Don't underestimate it, spend some time there. Elasticity, right? That's why I just talked about this thing is gonna grow.  And there are a lot of tools out there that you can use. There's a tool, I believe, Chaos Monkey, that you can throw and to see how your app is going to behave, right? When I throw in bad data, but also you can do scale up to see what will happen. Resilience, you want to loosely couple your pieces. So what I do is I think about the UNIX model and I will come back to that. But I will simply say, if you think about UNIX, you have LS, right? You have grep, you have find, they do one thing. So think about when you're building your app, you're gonna have these different components that do one thing. And I'm gonna have a defined input and defined output. I'm gonna pass the data or pointers if I store it somewhere else on how to move around. So by giving that, it's loosely coupled, which means if one of those components dies, there's not a whole lot in there, right? It does something very small, very quick. And so if another one springs up, it can handle it, so I need to have some resilience to say, well, if I didn't get a response, and that component I was talking to is gone, go ahead and fire up a new one and resend the data. So think about how you're gonna be building these things. And there are plenty of books on how UNIX was built. They're a little dry, but they're interesting and they may give you some ideas on how to do it. And the last one's probably the hardest one, security. So who here works in security? Okay, who here has friends and works in security? So you're in security and people are friends with you still? Is that what you're saying? What I have found is security is the most maligned group outside of networking. People never like security and they don't like network engineers because they say it's always a network that's broken. So what I have found is people don't think about security up front, but if you spend just a few minutes thinking about it, you can put it into your app and have this wonderful tool. If you try to go after and you build your app and then you say, well, now I need to roll it out. And the security group looks at it and says, no, no, no, no, no, you can't because you did this, this, this wrong. So engage your security groups early, talk to them. Remember, these are loosely coupled components. So I have data moving. All that data should be encrypted, period. Everything that goes on the pipe should be encrypted whenever possible. And the important thing to know, so this has been a fact for a while.  So it's not that they can't get to the network, they're already on the network because they're an employee. So the thing is you're protecting against that. So be sure you encrypt. If you encrypt data at rest or not, different discussion. One of the axioms that someone used to tell me is if there's no meat, it's not secure. Has anybody heard that before? There's no meat, it's not secure. So this was when I worked for a three letter organization that was not a government agency, but was the way that most people got to the internet in the 80s and 90s. So that may give people an idea of that company. And we had our security curmudgeon. And Rob was excellent. And he would say, well, we have to have these certificates and these certificates have passwords on them. So we need to type in the password so that the certificate can be used to encrypt the data to pass along. Well, that means someone needs to type it in. Yes, when the system reboots, someone needs to type that in. Well, what if I put it on the file system? Well, then it's not secure because someone can find it on the file system because it has to be in plain text for me to be able to be read by the application to decrypt or to be able to use the key for the certificate. Well, what if I encrypt that file? Okay, what are you gonna encrypt it with? Well, something else, I'm gonna have that password in clear text over here, but you're not secure, right? If I don't have a person involved, at that point, I don't have security. So keep this in mind, right? Ask the hard questions, because it's better if you ask yourself than have the security people afterwards come and tell you this and you have to start over. Trust me, we're gonna get to the 12 factors. I'm just trying to set the mode here. So five, right? So we have one idea, two modes of IT, three parts, four words, and five design principles. So the first one should look similar. This is my Unix thing, right? So for instance, if I wanted to go to a subdirectory and find every file that I could and look for my name in that file, you know, the easy way, to me, off top of my head, my fingers know it, I don't have to think about it. It's find, space, dot, space, minus, type, space, F, space, pipe, space, X-args, pipe, grep, space, minus, I, minus, L, space, double quote, my name, double quote, right? So that will go, and if you look, I've taken X-args, I've taken grep, and I've taken find, and I've put them all together. So find, get me a list of files. X-args, take a list of files, and each file passed to the next string that's there. Search files, right, for a pattern. So again, these are components, and when you're building your web apps, your 12-factor apps, you wanna think about this. How do I break things in the components that do one thing and do one thing well? Decouple the data. The important thing here, and we're gonna talk about config files, I don't want any of the data inside my app. So this was a big deal with web stuff 20 years ago, maybe 30 years ago when the web came, trying to remember now, that you would have all your data in the HTML. And then someone said, well, there's this thing called CSS. So let me have my design guys build stuff, and then all the data that gets pumped into it somewhere else. So WordPress, perfect example, right? I can have a theme that I can load into WordPress, but all the data that's getting put up there, the pictures that's getting put up there, it's all different. I can give a different feel, even though the theme is the same. And again, that's the same thing, we wanna decouple the data, and we'll talk about why that's important. Communication we talked about in performance, right? I don't wanna be passing stuff along. If that component dies, I don't want all the data and all its knowledge to die with it, right? I need to make sure that I can get the data in and get the data out. So by using a backing store and storing that information and passing the object around makes it a lot easier to do that. And I'm also passing less data around the system. So keep your communication to a minimum, and that will help with your performance.  When we've talked about both of those a little bit at length. So with that, here are the 12 factors. So these are the 12 things we're gonna talk a little bit about. But before we drive into that, I just wanna ask, does anybody have a question or anything off hand that they would like to ask at this point? No, excellent. I'm also glad to see everybody's heads up, and I don't see anybody, as they say, praying at the iPhone or your device. So thank you very much for your attention. This comes from a website, 12, the number 1212factor.net. You can go out. It has wonderful text. It does a pretty good job explaining them. You type in 12 factor, space, discussion or detailed description in your favorite search engine, and it'll bring up a lot of sites. So I was able to use a lot of those sites to get information in different ways to think about it that I thought would have helped me when I started down this journey. So the very first one we'll talk about is code base. One code base is in some sort of version control, okay? So Git, RCS, CO, CVS, Subversion, Bitbucket, I'm sure there are plenty. Git being the popular one that people are using today, and sort of the de facto standard when it comes to this. What we're talking about here is, remember we separate the code and the data. So my code is in there, and I'll probably have a file for my data, because my dev data is gonna look different from my QA data, which will look different from my production data. And we'll talk about and config what that means. But I want to be able to pull out that code, not have to touch it, and when I installed it in an environment, it's gonna run in that environment, okay? Because I worked for a company in 95. They hired me in operations to help them deploy web. Excuse me. And what they were doing at that time was the developers would do this nice little website for a company that does student loans in the United States. And then they'd say, okay, we have the code, now we're gonna go to QA. I said, okay, great. So when they went to QA, they would have HTTP or colon slash slash dev.website.com slash whatever, right? And they'd have all these hard-coded URLs in it. Okay, what happens when you go to dev? Oh, we go edit every file, and we place the word dev with QA. Okay, what happens when you go to production? Oh, we changed QA to prod or a dub, dub, dub. I said, so you're testing code, and then you're editing it before you put it in production? Oh, yeah, yeah, yeah, how else would you do this? I said, well, what you're testing isn't what's going into production. How do you know what you're testing's right? People make errors.  So you don't wanna do that. So that's why we have that code base that is that single code base that I'm gonna pull out and I'm gonna deploy in all the different environments that I'm not gonna edit, not gonna touch it. So important, keep that in mind when you're doing it. If you're thinking that something is different from dev than it is in QA, don't put it in the code. Dependencies. So how many times have you installed or tried to install something, only to find out it had the wrong version of G-Lib C, or it had the wrong version of Python, right? Or one of the Python libraries you needed. In the code, declare what you need. Specifically say, I need in GenX version blah, right? I need Apache web server version X. I need JVM 7.2.142, right? Say what you need and declare it. I can have as part of when we talk about how we build, release, and run, I can have files that are part of my code that make sure I check out or grab the right pieces from wherever they are to make sure that all my dependencies are set. There are things like CfEngine for anybody out there, Puppet, Ansible, Chef, Saltstack, and I'm sure there are plenty more. But all of those can be used to help build this dependency. Depending on the language, right? Perlin, Python, very easy to say use this library, right, and specify the version. So you get the right version because that's what you want to test. Again, my dev, my QA, and my prod, or prepod or staging and so on have to be identical for this thing to actually work because when I scale for web, I can't have any room for error. So config, this is how I handle separating that data. So for instance, the database that I'm gonna point to, the URI, is gonna be different in dev than QA and prod. I may actually have my own little database that I've installed and that I'm working with. And then a QA may have theirs and in prod it actually may be shared. But I can read that information from this config file at startup time. So startup time, we're gonna come back to that. So think about how you break out. Whenever I say, well, I need to hard code something in, no, put in a config file and then I have a dev config file, a QA config file, a prod config, whatever environments I need, right? I may have 12 developers and each developer can have their own config file and keep that under some sort of revision control. The other thing that I think here is when you think about logging, if I'm doing dev, I'm probably want a lot of logs and may want to actually go to console, right? When I go to QA, I may want to have less logging and when I go to prod, I may want to have the minimal amount of logging until I run into a problem, then I want to kick it up. So imagine setting a variable called log level that I could change in the config file. So again, change in the config file, all of a sudden the app can read it, get it and understand what's going on. So separate your data or your configuration information from the code base, which is important when you do that deployment. Backing service, we talked a little bit about this. The idea is treated as an attached resource. Imagine it's right there right next to you. It may be on a different server. It may actually be in a different network. It may be in a secure network because maybe that database has some sort of confidential information that security requires to be in its own zone, right? So think about how you're going to be able to get to it and in the config files where it's defined where it is, but you want to store stuff out there or be able to reach those backing services separately. Build, release, run. So this is where it gets into the Garrett Jenkins for just two tools that can be part of CI CD. You want to strictly separate the build and run stages. So I'm going to have a way, right? For anybody old enough to know what I make files were, which were replaced or they didn't last very long. It was an MIT X Windows thing, but be able to build that piece of software. And so maybe that's making sure that it has the right libraries, right? All those dependencies are met. If the code needs to be compiled, compiling the code, put it in some sort of thing that can be released then and that release is what's installed in the environment and then be able to run it. So think about it, right? As a developer, I want to be able to develop something that is a nice tightly deliverable object thing that I can give the ops. Ops then can put it in that environment and run it. And ideally, I will never hear from ops again, right? They're not going to call me in the middle of the night because, well, you're using Perl 3.6 and we have Perl 4.5 installed here and that was deprecated and so on and so forth because you've defined those dependencies. You've made it a well-defined solution.  So what you're trying to do here is to make sure that stuff is stored in a backing store somewhere, right? So I can have input, output, and it does some work but there's nothing in there so as I scale out and I add more resources or more components at that level, I'm able to do what I need to do, okay? So again, you need to think about, I don't want to have state that I need to pass along. Portbinding is interesting. It's, people are back and forth on how important this is. So if you think I have a web app, right, so I have some sort of web server, I'm already gonna get port 443 because remember we're doing everything secure, maybe 8443 depending on my environment so I'm gonna have that encrypted traffic but that can just redirect to my app that's listening on another port. What that allows me to do is when I go to debug, I can say, well, you know, I'm coming through the web server and my app's not behaving properly. What happens if I go to the app directly, right? So I go, the app's running on port 5000 so what if I go to that port directly and interface it with that way, right? There may also be a need during an admin process, right, or a cleanup process that you want to talk directly to the app and bypass what other tooling you have in there and that's okay and by exposing it and having that app listening on a port will allow that to happen. So concurrency, right, scaling out. We need to make sure that this thing scales out and you've noticed that there are just a couple of themes, we just have 12 ways to differently talk about those. So I want to be able to handle all sorts of crazy traffic when I do this. So if you think about that one idea that I had, in my app, I thought about it actually this morning I was trying to think, I need to have an app I can talk about and I thought about all the flooding that had been happening in Texas and I said, well, there are probably a lot of potholes from the flooding. So what if I was driving along and I had an app on my phone and on that app I clicked a button and it opened up and I maybe have a login, maybe I want to be a good Samaritan and I want to pat on the back from a Department of Transportation for letting them know about this or maybe not. But I want to be able to talk to my phone to be able to get the GPS coordinates because I want to be able to put those coordinates in or maybe I want to type in the coordinates. So I'm going to gather some information and that's going to be sent to an app somewhere or a service somewhere on the web and that's going to listen. So when it comes in, in my mind, and again, this was in the shower this morning so the design's a little shaky still, I would say, well, if I see what looks like GPS coordinates, which are well-defined so I can just do a pattern match, then I want to take that and send it to a database to convert that to an address. It's a service that's out there, maybe I need to write that service and so that'd be another component. So as people are sending this, I may need to scale out how many ways I'm looking this up depending on how fast it's responding. So being able to scale up is very important here. If I get an address, address may be good enough, but I want to verify that address against a database to make sure that they didn't type it in wrong and to add a zip code if they didn't put a zip code. So that would be the second component that would be part of it. The next component would be to figure out which agency is responsible for that road. So I need to look up that road against the database to figure out or some sort of service that may be a web service provided by the county that says, oh, that's in downtown Austin. So that goes to the downtown Austin Department of Transportation. So now I have that information. Now how do I take that and write that into a form that I can send to DOT on my behalf to open up a ticket that says, look, there's a pothole, get out there and fix it. So again, I've broken it into as many little small pieces that I can think of and look for services out there that I may need to write or that I can consume, but I need to make sure that those scale up because right after heavy rain, I'm gonna need a lot of these services to handle all those requests. A month later, the road should be fixed and no one's doing it, right? So again, there is a different scale depending on time, depending on an event. So I need to make sure my app can handle that by scaling up and down appropriately. Disposability. So remember I said I had come back to the config files and the startup. It's very important. These apps need to be startup as quickly as possible. Subsecond, ideal. In reality, maybe a second or two. I want these things to start up very fast because I'm under load and I need to respond to that load. Likewise, I want them to be able to shut down nicely. So an old timer land, I send it a sigterm. It says, oh, I got that. I have a queue of requests coming in. So I'm gonna turn off the queue. I'm not gonna take any new requests, process what's in here, gracefully shut down. So you need to think about how you handle the startup and the tear down well. The one that's hard is how do I handle a crash? But remember if this thing is stateless, then that makes it a little bit easier because I don't have to worry about that. So the best I can do so that I'm not having anything in there that I need to remember over time, let's get ran out to the store, make it so that when I restart, I don't have to go through a cleanup process. So you need to think about how I handle the crash, which is a little bit harder, but that performance and breaking out the config so I can get that sub-second start is something that's important that you should be considering and thinking about. Dev and prod should be the same, right? And this is where it gets into that continuous integration and those different config files. So we've talked about this before. The idea is my developer, he writes the code, make sure that it's all bundled together, right? So he's gonna build it, he makes a release, pops, then it's gonna release it to where it needs to go and they can then run. Making sure those environments are the same as important so that goes back to being dependencies and being declarative. This is what I want. Assume nothing, very important, okay? Now almost, I think I'm gonna be in a hostile environment. You know, I am a UNIX guy from day one. I've never run Windows or done Windows development. I do run OSX, which is UNIX under the hood, but I just think I'm going into a dot net world and they're not gonna like me or understand what I'm doing, so I'm gonna make it as declarative and explain everything step by step so that a person could push a button and get a release that they drop in place and it works, okay? So again, use that mentality when you're doing your development. Good development anyway, but again, because of the scalability, right? Because of being able to start up very quickly are all important things that you need to keep in mind. So we talked a little bit about logs. Remember, maybe you want to go to a console, maybe you want to go to a file, maybe you have a centralized logging location. So my app is only gonna write an event stream. It's just gonna stream out some way. And then based on configuration data, which is separate, I can say, oh, well, in this case, I wanted to go to a console, right, dev console, or I wanted to go to, you know, slash var slash log slash my app dot log, right? And then over a period of time, doing log rotations through another system. Or maybe I'm using Splunk, or I'm doing some sort of log stash, or HECA, or some other sort of tooling that I'm using for that. Let's figure out that, but that's handled through the configuration. It's not the app. The app just writes the logs. He may check and see what the log level is that's set in the config file, but he's just gonna write the logs out to a stream, whatever level that should be. And then finally, admin process, it's number 12. Your admin tools ship with the product, okay? Because, remember we talked about being declarative, having those dependencies? Well, if I'm not running and testing my admin tools, when I'm testing my code, maybe something changed. Maybe I deprecated something, right? So I need to include those tools when I ship. It may be a feature that's built into the app, right? But it's only accessible if a certain flag is set. Maybe it's a separate app or a separate tool that is available via the command line. But for instance, we talked about the backing store, never, ever, ever, ever, ever, ever go to that database and start mucking with the database. Use your tooling that you built with the app to be able to go and to check the database and do database cleanup and to do that maintenance, right? If you wanna do log maintenance or maybe you have some, you're writing some sort of binary string or some sort of interesting information that I need to be able to decode. I should have an app that ships, I'm sorry, a tool that ships with my app that can do that, okay? So make sure that if I'm an admin, that doesn't mean I just log in as root with all privileges and do what I want. I have to use the tools that ship with it that should have the same privileges that everybody else is using. So don't make it a special case, okay? So with that, we have run through what the 12 apps are. We've talked about the one idea and two modes of IT, the three parts, the four words and the five principles and then the 12 apps. There any questions so far about this? It's a quiet crowd. Yes, right here, thank you. Thanks for the presentation. So see this 12 factor is basically for cloud native applications, right? Yes. This can be applicable for when we write new applications, like we can consider all these 12 factors. What about the applications that are returned like 10 years before or 15 years before but still running in the on-premise data centers? How do, what is your point of view in addressing? So is refactoring is the only solution or do you have any other ideas on how we can make the legacy applications to be 12 factor complaints? Great question. So I know there's a second person so if you wanna come up to get in line, that would be awesome. So what I heard is I get it, I understand these 12 factors, but I got legacy apps and I wanna move them and run them in the cloud. Is that correct? So my Greg's answer, I wouldn't run them in the cloud, okay? You're gonna open yourself up to a world of hurt if what I wanna do is just take my legacy app and move it to the cloud. There's no value in that. You can do it, it will work, right? That's where you have that pet, right? So it's that special thing that's out there running I need to care and feed but I'm not getting that scalability that I want, right? I'm not getting a lot of the things, the flexibility that I want with my cloud app. Now, the easy thing to do is say, well just rewrite it from scratch, right? Refactor it and figure out the question become. So if I have a customer who came to me and they said, we're really interested in cloud. I said, okay, tell me what you plan on doing. Well, I wanna run SAP in the cloud, on-prem. Why do you wanna do that? Well, you know, it's an app that we use and I was thinking maybe that would be a good first step. And I said, so did the CIO fly over to Europe recently or on a very long plane ride where there was a magazine that said, you need a cloud in your organization? He said, well, there is an MBO tied to that. I said, well, there you go. It's not a good idea. You need to find the right use case for it. So you need to think about your business and what new apps. So ideally, it's Brownfield. The other thing that people will ask me is, well, can I replace VMware? You can. I wouldn't recommend it. That's not what it's here for. It's here to change the way that you think and to be able to deliver software faster. If you're delivering software faster and you need to be, you know, someone is saying to me and I will say it, the uberization of technology, right? Will you be uberized or will you be uber? And I don't like those words, but really, that's what becomes, are you going to be a disruptive force or will you be disrupted? So really, I wouldn't do it. I would take a look and figure out, is there a different way to solve the problem? And if so, I would build a 12-factor app to solve that problem, okay? You're most welcome. Next. Could you just give a practical example for number six? I didn't quite sync it. Oh, do you remember what number six was? The processes. So let me, you know what? Let me go back here. Yep. So when I think about it, right, I am going to pass, for instance, let's take the address, right? So I want to do an address lookup. So I'm going to have a process where I'm going to pass it an address and then it's going to pass an address back to me, okay? So when I pass that address, right, 123 Main Street, Anytown USA 12345-6789, I need to validate it. So I'm going to go out and I'm going to validate and then it's going to send back a correct address and that correct address could be the same or not. So there's no state inside that process, right? That process is going to talk to another service and then it's going to return something back. So it's not remembering something so that when someone else comes and hits it, that it has to say, oh yeah, this was Bob who just asked me about something else a moment ago. Does that help? It scales up faster because I don't have to be sticky, which means anybody who comes in can grab anyone off the shelf and he can do the job. He or she can do the job and return the answer. Thanks. Thank you for asking because obviously I wasn't clear enough and I'm going to update that example in here to talk about that. Yes, another question over here. So since this is an infra conference and a lot of the folks are infra people, right? As opposed to app developers. Does the 12 factor thing take into account anticipating running on two different clouds or data center regions for region level redundancy, right? Because you shouldn't be assuming that this is this ever available cloud, right? It is, it goes down for upgrades. So should app developers be thinking about multi? So if they followed those 12 things, right? So I have a URI that I need to go to. So by putting it behind a load balancer or putting it behind some server that's going to a VIP or something so they can have it in multiple locations, solves that problem. I know from my configuration when I go to my backing store, it's going to be HA1.myservice.com, whatever, right? And then how that gets handled across multi-cloud and the rest is outside the web app. The web app doesn't worry about that. My 12 factor app doesn't worry about that. My infrastructure handles how I'm going to scale across the cloud, multiple clouds. Does that help or it sounds like it looks like you're not okay with that? I think that's, I mean, it seems like there needs to be a little bit more of a collaboration between app and infra for that because if you have a VIP and you just assume that the infra will sort of take care of that across different data centers. I mean, maybe it's in the app, right? So I'm thinking easy way to do it. My app could be cloud aware, right? And with zones and the rest and there are probably some other things you could do down at that level. Not completely, but at least with the notion of having two regions, two sites at a minimum that abstraction. I don't know. No, that's a good point. I need to think a little bit about that. So it appears that I don't know if we're out of time now or not, but people are getting up. So I want to thank everybody who came. I am absolutely happy to stay and answer questions because I don't know. Keith, do you know what time we end? It's over? All right, everybody out. Anybody wants to come, ask questions, please. Come up and ask questions. I want to thank everybody for your time today.