Hi, everybody, thank you so much for having me. Amazing hospitality in Doha. Everybody's been very nice, and I was tweeting at how insecure the amount of breakfast at this conference makes me feel. Because I run a conference in the US, and I go to lots of events, and this is like the biggest conference breakfast I've ever seen. For omelet stations and enough croissants to fatten the entire country of Qatar. So I had a few, so I apologize for looking a little chubby. But today's topic is gamification. And I'm sure many of you have heard about it or have an opinion about it. And I want to talk to you about how it applies to mobile and how it substantially changes all of our lives. In order to put it into context, though, we have to ask ourselves, we need to talk about a context in which gamification is useful. And one of the core concepts that we think about that affects all of us is this notion that change is hard. Change is a hard thing to accomplish. And I put up an image of US President Barack Obama for a particular reason. As many of you know, his platform for election was based on a concept of change. And of course, you can just ask him how hard it is to change a large system like the United States after four years have elapsed. And of course, change is very, very hard to accomplish. And we heard many panelists talk about needing to support feature phones in a world where that seems increasingly crazy. But is change really that hard? Isn't this weird? Here's a graph of social media adoption in the world. So in 2005, nobody tweeted or posted status updates or took pictures of their food or their cats or wrote a blog about it and made little collages of everything that was on their mind at any time or shared their music online. 2005. And in six short years, over 60% of people in the West do it. Wow, 60%. Have we ever seen a behavior change like that in the history of humanity? Can we point to any examples where a behavior goes from no action to 30 to 40 times a day per person in six years since the beginning? I don't think so. So is change really hard or is change really easy? It's hard to tell. In fact, it points out an important dichotomy, which is that change is happening on two parallel tracks at all times. Changing people's behavior collectively can happen exponentially, as in the social media example that I just gave you. Change from 10,000 meters is exponential. But on an individual basis, change is linear. That is to say, each one of us changes our behavior like it's the first day of our lives every single time.  My weight loss process is independent of yours, no matter how supportive you are. I start from the beginning. I do it myself every time. And so while we get very excited about these big changes, like mobile commerce, like games, like social media, we must recognize that each and every one of us comes to this process of change alone and individually. And so there are three key recurring themes about every single thing that's ever changed human behavior in a substantial way, in a substantial lasting and durable way. These three things keep coming up over and over again. And they're really simple. It's disarmingly simple, but they're always there. And they are feedback, friends, and fun. It turns out if an activity has feedback, friends, and fun baked into it, people love it. Surprise. Feedback, friends, and fun. Now, many of you may already be thinking to yourself, huh, I know some things that have feedback, friends, and fun in them. And you know I'm getting up here, I'm talking about games, so obviously that's gonna be one of the core ideas. But this notion of gamification has really taken off in the context of this idea that we want to change behavior, that we want to engage people. And one of my favorite examples of how powerful the notion of feedback, friends, and fun can be is embodied in a core question about what is fun.  I'm gonna describe an experience. You're given a small plot of land. It's not necessarily fertile. You have to till this land and get some seeds and plant them, and then hopefully the rain will come or you have to water the grass. And then when your plants grow, your crops grow, you harvest those crops, and you take them to a market in town where they give you some money for them, and then you do that again. And hopefully try to get some of your friends to come and start a farm with you. Does that sound like fun? Does that sound like a fun game? Everyone, you wanna drop what we're doing right now and go out into the fields and farm? Right, sounds so cool. Of course, I'm describing Farmville. That's the world's most popular game, the most popular game of all time, measured by number of people playing, or by amount of time, or by dollar spent per user. Far and away, the most popular game of all time is based on a medieval ritual of farming, which all of us, our forefathers, gave up so that we could be here in this lovely hotel in Doha today to talk about mobile and gamification. Why does so many people wanna play this game? It doesn't make any sense. And it's because it's not the theme of something that makes it fun, right? It's the mechanics of something that make it fun. The theme is allure. It's like what's in the window? The clothes or the thing in the window. But it's the mechanics that get somebody to stay and keep staying and get engaged and keep engaging, interact with other people. Now, it turns out that there is a big reason why games and gamified concepts are so powerful. Human beings are wired to enjoy the challenges of gaming.  Any kind of challenge, any kind of achievement, dopamine. And the dopamine release in your brain makes your whole entire body go, that's fantastic, right?  Because that was really good, that was pleasurable, that was enjoyable, I'd like to do that again. This applies to all kinds of experiences in our lives, whether that's trying to run a marathon, or jump over something, or learn a new task, or play a video game, we are wired as people to continuously challenge ourselves and then to try to exceed those challenges each and every time. It's no wonder in that context that games are so powerful at changing behavior. It turns out they also affect our brains. In research in Germany, on the effect of learning to play a game, in this case the research was about learning to juggle, researchers gave a group of people a juggling challenge, 12 weeks, and they gave them MRIs, so scans of their brain throughout the 12 weeks and they learned to juggle. And in 12 weeks, they noted a market change in gray matter in the brain just from the activity of learning to play, learning to juggle. And they redid the research five years later, 2004, 2008. They redid the research about five years later to find out that it doesn't matter how good you are at playing the game, it's simply the act of trying it that changes your brain, just attempting changes your brain. So it's no wonder that we love games so much. There's this concept that reinforces it called fluid intelligence. And some of you may be familiar with this notion, but basically we have two kinds of intelligence at any one time. We have what's called crystalline intelligence, which is basically knowing the things we already know and being able to recall them and work with them. And we have this concept called fluid intelligence, which is our ability to problem solve in real time and come up with cool new ideas. This kind of intelligence has frequently been the one cited as the important future construct for people. And in lecturing at Harvard on how to increase the fluid intelligence of people, Andrea Kusevsky identified five core things that people can do to become more intelligent. And she said, the research suggests that if you seek novelty, you challenge yourself, you think creatively, you do things the hard way, and you network, you will increase your intelligence. Do these five things resemble any activity that you may have recently done? These are some of the core design concepts of games. In fact, most of the games that we play have those that we're really attracted to, have those elements in them. Thinking creatively, doing things the hard way, right? Games are not about doing things the easy way, they're about doing things the hard way. Imagine if the game of Pac-Man was just you came in and you went a straight line down the middle, ate up all the coins, bing, congrats, and you did that again and over and over again. Would that be fun? If I took out all the challenges, it wouldn't be. It also turns out that there's a profound shift happening as a result of games in the world. And it's happening to our children in a really dramatic way. You know, I am 37 years old, so I was among the first generation of people who had games in the home. The Atari 2600 was the first game console that I played with. And in my age, the only thing I had to be able to do to play a game was do this, right? Move a joystick and press a button. That's all I had to do to play. Today's kids, they don't just do this anymore. They have to do five, six, seven things simultaneously to be able to play a game like World of Warcraft. They have to be able to operate their character, chat with other people in text and voice, deal with short and long-term challenges, and with the constant interruption of their parents. This makes today's children extraordinarily different from people my age. They are from the very youngest, from the very, very beginning of their childhood, trained to multitask at a level that no human has ever had to before, principally because of games. And this has a very specific impact on our world, which you can take both positively and negatively, which is that the world that most of us inhabit seems painfully slow to our children. Painfully, painfully slow, extraordinarily boring. Sitting through my lecture would be inconceivable for most of our children and they will likely never sit through a lecture like this in their lives because the world is changing. I'm cool with that, by the way. The other way the games have affected our minds, our brains, our culture is the notion of economics. They have both introduced billions of people to some of the core concepts of economics in a way that no textbook and no real life experience ever has. If you think about the complexity of the economic system in most games, it rivals our national economic systems, but also it changes people's perception of how economics work in an interesting way. In most games, in social games like Farmville, one of the main things that they're capable of doing is removing an important barrier to customer satisfaction. I'll explain what I mean. So most places that want you to pay for something, like let's say an online subscription site to a newspaper, they say, okay, we have some free content for you, but when you want to subscribe, like the New York Times, which is a big newspaper in New York City where I live, like the New York Times says, okay, there's some free content, you can read three free articles a week, but if you want more than that, it's gonna be $4. It's gonna be $4 a week to access the newspaper. That's how it works, it's gonna be $4 a week or you can have it for free. So here's how Farmville works or most social games are brilliant. So comes to a moment in the game and says, oh my gosh, you are awesome. You are so good at what you've done that your farm has just gotten bigger. Congratulations. Now you're gonna need a much bigger tractor to keep farming your farm. And that tractor costs 20,000 coins. Oh, you don't have 20,000 coins? Okay, no problem. Well, you can do one of two different things. You can keep farming and selling your stuff at the market and eventually you'll get 20,000 coins and that's up to you, you can do that. Or, you know what, you're kind of busy. You're busy, you've got other things to do. Do you have $4? Give me $4. I'll give you 20,000 coins, it's no problem, you're busy. You've got kids, you've got a life, $4? Okay, so it sets up a very, very interesting choice model for consumers. You can either spend your time or your money. So consumers are naturally happier about that choice than they are for any other purchase option they're given in the rest of the world because they had a choice.  And so lesson number one of social game economics is give people the choice of time or friends or money. And lesson number two, which is embodied in this interesting little stat, the 92% of people who play social games don't pay for them. So this whole industry is based on 8% of people paying and 92% of people never, ever paying for anything. But that 8% has funded this entire humongous business. How is it possible? Until you know that top players of the game Farmville spend upwards of $10,000 a month playing the game. Yes, how do you think Zynga gets to be like a $10 billion business? They spend upwards of $10,000 a month playing this game that everybody else plays for free. What, that's crazy, right? Lesson number two of social games, never limit the upper amount of money somebody will pay you for a product or service. And this is part of the big excitement, part of the excitement enthusiasm about gamification is challenging some of these core ideas. So it's no wonder when you put all of that into context that games have become so important and so powerful, they're changing our brains, they're changing our economies, our youngest generation today are brought up on them, they see the world differently because of them, the world is changing. So this concept of gamification, let me give you a quick working definition. It's the process of using game thinking and game mechanics to engage users. The process of using game thinking and game mechanics to engage users. That means necessarily that it's not just throwing some badges on your website. For those of you who know many of the examples of gamification, the ones that people think of first are all about using badges to engage audiences. There's nothing wrong with badges, badges are awesome, but they're not the be all end all. And the second thing that gamification is not, and this is incredibly, incredibly important, it is not about using game ideas where they do not belong. It's not about introducing game concepts where they're not appropriate. It's not about making everything in the world into a game, that is simplistic, that doesn't work. It doesn't work. We have to take the best ideas from games, mechanistic, thematic, all these different elements and use them in the ways that are most appropriate for the situation, for the customer, for the context. We do not seek in gamification to make everything about shooting zombies. That doesn't work. And most of the examples of the stuff that failed in the past was that kind of thing. Your soup company wants to do gamification, so we fly through space and shoot soup out of the sky.  And then we land on Planet Alpha 17 and we build a space station out of delicious soup. And then we buy the soup on the web store, right? That was the ridiculousness of old gamification. Now, we take the ideas and the concepts from games to build unprecedented engagement. That's what we do. So me, I'm the chairman of Gamification Summit. I've written two books about it. I help a whole bunch of companies, big and small, apply gamification to their lives. This is my next event since San Francisco in June. It's pretty cool. Here are some of the companies that build a gamification technologies. Lots of the things I'm gonna show you today were homegrown, but lots of the things that you might choose to do with gamification can leverage existing technologies from various vendors to make it possible for you to do this much faster and at greater scale than was possible before. And I highly encourage you to find out more about it. Okay, I wanna give you some examples so that we can contextualize gamification and you can start to think creatively about the ways that it will affect your business, whether you're a mobile operator or thinking about mobile or whatever it is that you do. So, are you guys all familiar with speed cameras? They take your, they're awful, right? Everywhere in the world they have these speed cameras. They take your picture, you're going too fast, they send you a ticket in the mail. So, in Scandinavia, the ticket that you receive is based not on how fast you're going, but on how much money you make per year. So, the wealthier you are, the bigger the ticket you get for speeding. So, it's pretty interesting stuff actually. Sweden last year gave out a $250,000 ticket to somebody for speeding, which is kind of amazing, right? Okay, so, as part of a contest, as part of an interesting design contest, guy named Kevin Richardson re-imagined the speeding camera. So, instead of when you drive by a speeding camera, you're going too fast, it takes a picture of your license plate and sends you a ticket in the mail. The way that speed camera lottery works is anybody who drives by the speeding camera at or below the speed limit is entered into a lottery to win the proceeds of the people who speed. Right? Right? Such a brilliant small change in the design.  The punishment is still there. Nothing's changed about the punishment, but little positive reinforcement. Guess which one works better? 20% reduction in speeding in Sweden from this technique. 20%. Nobody has ever been able to get speeding to drop by 20% and everything has been tried. An astonishing achievement by any measure. Sweden is rolling this out across the country as a result. Of course, many of you are familiar with Foursquare. It is often the barometer for the concept of gamification and highlights how important mobile is to the concept of gamification. They really have grown together in tandem by empowering people to be able to do all kinds of things. You'll see most of the examples I'm going to use today happen outside using either the mobile device or mobile in the broader sense where you're on the move, like speed camera lottery. Of course, Foursquare, what's really interesting about Foursquare is that despite being the first successful mobile location based social network, and in fact, Kevin earlier kind of touched on it, which I was giggling about, Dennis and Avine who founded the company, they used to have a mobile location based social network called Dodgeball, which did not succeed and was bought by Google as a talent acquisition and promptly shut down by Google. And when their lockup ended, Dennis and Avine relaunched their company Dodgeball on the iPhone with game mechanics. After 25 or more companies had attempted to build mobile location based social networks and failed, what did Dennis and Avine understand about mobile location based social networking that nobody had understood before? Really simple. The process in the original mobile location based social networking concept, it was all about finding your friends nearby. This is what Kevin said. I want to find out where my friends are nearby. So I'm going to check in to let them know. But that doesn't work. And it doesn't work for two important reasons. The first one is only a very narrow slice of the population cares about a problem like that. Anybody under 20 knows where their friends are all the time. And anybody over 40 doesn't care where their friends are. So this idea of where are my friends are, I can find out, is really only about urban dwellers between the ages of 20 and 40. It's already a pretty small market. And number two, it takes nine steps to turn a mobile location based social networking check-in into a cup of coffee or tea with a friend. Nine steps. It's not, I check in and magically, I'm having coffee with my friend who's around the corner. It takes nine steps. Think about it. I check in here at the Shark Hotel. Now, one of my friends needs to be nearby. That's step number one. Step number two, they need to be watching my social media traffic so they see my check-in. In time, while I'm still here, they need to have the time to meet me. They need to respond to my thing in time. I need to see their response while I'm still here, right? Think about this. Now, I've set up an already, it's an impossible task. We need to negotiate where we're going, choose a place to meet and actually have that cup of coffee. Wow. When was the last time you did nine steps on any discreet activity? Nine steps in a discreet activity. When was the last time? Anyone? No, because we never do that because nine steps is impossible because nobody does something that complex. Certainly not an optional activity. So what happened with all of the mobile location-based social networks? What happened? You checked into an empty room. You said, I'm here and nobody responded. Or you said, I'm here and once in a while someone would say three days later, oh my God, that's so cool. I was near the next door, right? That doesn't give you positive reinforcement. So Dennis and Naveen reimagined it and they made the multiplayer game mobile location-based social networking into the single-player game mobile location-based social networking where you check in whether or not anybody is nearby you. You check in to earn badges and become the mayor of a place. And it doesn't matter if no one else is here. It doesn't matter if your friends were here four days ago. The game is not necessarily, the gamification is not necessarily always about the core activity. Sometimes it's about something around the core activity that gets people to behave a certain way. Another super cool example of gamification on mobile comes from reimagining work, right? Most people's jobs that, you know, I don't know if you guys work in big companies or not, but most people's jobs don't have a lot of positive feedback in them. So the process, you know, I used to work in a corporate job and the way that we received a review was once a year my manager would begrudgingly come down from his office and say, okay, I have to give you your review. So you've been really good. Here's your official raise. And then disappear for a year. So we didn't have to do that again. And so these companies, Ripple, a couple of examples, Ripple and Duprox are reimagining that whole idea to make the workplace a more positive and constructive environment. So the way that they work, it's very simple. Thank you so much for helping me out with the reports this weekend, this week. I send you on my mobile phone a little virtual item thanking you for doing that. Thanks for grabbing the coffee, right? It records our actions. It sends you virtual goods, karma, excitement, right? Positive reinforcement for your activity. It makes feedback a 360 degree instantaneous activity between subordinates and superiors, between peers across an organization, transforming the landscape of feedback, transforming it inside companies completely. Major companies like Facebook have standardized on concepts like Ripple. Ripple was just purchased by Salesforce a few weeks ago. This concept of changing feedback using mobile to make it instantaneous inside organizations, really compelling. Another example comes from the automotive business. So if you buy a hybrid or battery electric vehicle, it has become standard that your car will have a game in the dashboard, which has happened kind of spontaneously in the industry over the last few years. It started with the Toyota Prius, which put a little feedback flower in the car that showed you how you were performing and had all these stats if you've ever owned a Prius that showed you how your fuel economy was over time and has advanced to the point where Nissan's Leaf, which is a battery electric car, today has a Facebook connected social game in the dashboard that allows you to compare your eco driving with your friends so you can see how ecologically you drive. Or the Ford Fusion and Focus hybrid vehicle, which has this little green plant in the dashboard on the side that you guys can see I'm highlighting. And the way that that works is as you drive more ecologically, the plant grows lush and rich and thick. And as you don't, it withers and dies. It is an example of what we call a Tamagotchi or virtual pet game where you need to constantly feed a virtual pet in order for it to be sustainable. So tell me this, everyone. This is $150 million worth of engineering on the board right in front of you right now. Seriously, it costs a lot of money to put games into the dashboards of cars and make them work in real time. And it's a product designed for people who've already purchased the item. Because by the time you get this thing, you just bought a car. And you're probably not gonna buy another car for another five to seven years.  Well, there's a little bit of a social factor, right? The person sitting in the passenger seat can be excited about something to do in the car other than fight over the music. But also, it's to positively reinforce the reason why you bought the car in the first place every single day. Because otherwise, how does your car tell you that you're such a good person for buying that hybrid? How do you know? You spent all this extra money to buy this hybrid or battery electric car. How do you know how well you're doing? How do you get that feedback that you need? This is how. And it's been incredibly compelling. Another example from the US, a company called Zamzi, which I advise and help design. Zamzi is a little mobile device that you clip to your pocket. Kevin had his Fitbit. This one is targeted at younger children. And so this product is basically a socially connected game attached to a little device that helps children become more active and move more. And they've had astonishing results. In the first version, first release of the product, they got low socioeconomic status children, so poorer children who don't generally exercise very much, to move the equivalent of an extra marathon every month just by playing this socially connected game. An extra marathon every month. An amazing, amazing accomplishment using mobile gamification. Introduce you to another guy who has an interesting story to tell, a guy named Charlie Kim. And Charlie runs a company called Next Jump, which is a very successful $100 million New York-based company that provides employee incentives. And Charlie is also really into fitness. He loves working out. And he believes that all of his employees should work out regularly. And there's good science to back up why you want your employees to work out. It turns out it helps the brain function in your more cognitively sharp if you work out on a regular basis. So Charlie wanted all of his employees to work out. So he did what most companies do when they want that, which is he built a gym. He put a gym in each one of his offices and he says, here, here's a free gym. And he found that not very many people were working out. Not as many people as he would have liked if he would have worked out. So he built a mobile application, simple little mobile application that allowed people to check in. And he said, you know what? I'm going to put up a prize, $20,000 prize. The top five employees who work out the most every year will split this amazing $20,000 prize. And Charlie Kim got 12% of Next Jump employees to work out on a regular basis, which is an amazing stat. 12% is a lot. That's a lot of health. And he thought, I think we could do better. So he retooled the game, the contest, in two critical ways. The first one was, he allowed people to play in teams instead of individually. So he said, okay, instead of counting your individual score, now we're going to make teams by location and those teams will compete to see who works out the most. Same prize, now split among more people, so expected value goes down, but anyway. Same prize, $20,000 now playing teams. And number two, he introduced a leaderboard so the teams could compare their scores against each other. Today, nearly 80% of Next Jump employees work out on a regular basis, 8-0. I'm telling you, your gym doesn't have that many people working out on a regular basis. You've never been in an environment like that in your whole life. Every time you go for a meeting and Next Jump, it's like, hi, welcome to Next Jump. I'm really excited you're here because everyone's been working out all day. It can be a little stressful, a little bit stressful to be there, but it's fun to see it. So they've had an extraordinary accomplishment by using some gamification techniques, including teams and leaderboards. And it turns out the team play is really powerful for engendering amazing changes in behavior. Share with you a couple more fun examples. Here's a guy named Ananth Pai. He's a teacher in a class outside of Minneapolis in Minnesota in the U.S. And so Mr. Pai was a business executive who decided to become a teacher, a middle school teacher, or elementary school teacher. And so the school district gave him a failing class to teach, a third grade class, that was performing below their level. And he said, okay, I know you want me to fail, but I'm gonna take this challenge from you if you let me redo the curriculum and do the curriculum my way. And they said, sure, your students are already failing, so what harm can you do? Redo the curriculum. So Mr. Pai went out and redid the math and reading curriculum for his third grade students to be entirely based on games. So he replaced the books with video games, many of which are played on the Nintendo DS, right? Including games like Brain Age. And he supplements it with lots of real hands-on teaching. So he's constantly working with the students, but instead of reading textbooks, they're playing games. In the space of 18 weeks, Mr. Pai's failing third grade class went from below third grade level reading and math scores to mid fourth grade level in reading and math. In 18 weeks, two years in a row. 18 weeks, one whole grade plus, amazing, right? And when you talk to the students about what they like about Mr. Pai's class, you say, what do you think of Mr. Pai's class? The thing they say over and over and over again is Mr. Pai's class is fun. And Mr. Pai's class is social, right? It's no surprise that making something fun with feedback, with friends, will work well. And then Nintendo DS is an interesting, if powerful tool for that. One last example for you. You guys all familiar with the hip-hop artist Jay-Z? So last year Jay-Z put out an autobiography. And the book was called Decoded. And let's say many of Jay-Z's friends are not, many of Jay-Z's fans are not necessarily the world's biggest readers. So there was some concern about whether or not this biography of Jay-Z would be a popular book. So he decided that what they should do is they should make a game out of the book. And so a very, very inventive idea with the group called Area Code, they went out and built this game for Decoded and it was so cool. Every single page from the book was hidden somewhere in New York City based on whatever Jay-Z was talking about on that particular page. So for example, if he was talking about his favorite pizza restaurant in Brooklyn where he grew up, at that pizza restaurant they printed 50 boxes with that page from the book inside the cover. So that was spread out all over New York, 300 spreads. And the contest was the first person to capture all 300 images hidden around New York City, photograph them and upload them to a site powered by Bing, won an amazing prize, which is two tickets for life to any Jay-Z concert for the rest of your life. Cool, right? Amazing, amazing prize if you like Jay-Z. If you don't like him, not such a good prize. But if you like him, it's great. And the cool thing about that prize of course is that that really doesn't have anything to do any cost to Jay-Z because he gets 100 free tickets at every concert. But to the average person, it seems like an amazing prize. It's really interesting and we'll talk about why that is in a minute. So it took 30 days for somebody to get all 300 spreads up online and win the prize. And what was really hilarious about it was the winner had actually hired people to help her find all of the locations. She went on a site called Craigslist, which we have in the USA classified site, and she said, I need help, I'll pay you $10 an hour. Your job is to find all this stuff. It was worth it and she won the prize. So you can imagine being able to do this concept kind of for any context. If you can do it for a book, I assure you you can do it for just about anything. So at the heart of why this worked, for Decoded, is an important insight about how to make gamification work for your business and your projects or whatever it is that you do. And this model that I put up on the board is called SAPS. And this is a list of the rewards that your customers want from you. The list of the rewards that consumers want. Status, access, power and stuff. In order of stickiest to least sticky and also cheapest to most expensive. Isn't that interesting? Isn't that interesting? Like I'm a marketer, I was trained in marketing, that's what I did for my MBA. I was trained in marketing and I was taught that what consumers want is they want free stuff. They want discounts, right? Isn't that what you want? A great deal? Isn't a deal what you want? Don't deals drive the world? Doesn't money move everything in society? Isn't that what the entire planet is based on, is trying to get more stuff? Isn't it? I don't think so. And I think the evidence is all around you. Status, access and power, which are the main things that games trade in because when was the last time your Xbox gave you $5 for successfully beating a game? Status, access and power are significantly more powerful, emotional and effective at creating long-term engagement and behavior change with consumers than stuff. Because what happens when you start giving people free things for their actions? Eventually they just price it in. Oh, it's expected, of course. I get a mile for every dollar I spend. That's not exciting, that's just what I get. And then they start demanding it. And then they start expecting more of it. And the next thing you know, you're on a race to the bottom with all of your competitors to give away as much as possible to consumers. But status, access and power, well, they scale much better. I'll give you an example, thought process. And I give this example everywhere. It's kind of funny, but. So I will offer you as a reward for your compliance in today's amazing lecture that your children will never sit through. Two possible rewards. The first one is a free cup of coffee at the cafe of your choice, at the lovely cafe around the corner here in the hotel. Or I'll give you a magic superpower. And your magic superpower is, whenever you even so much as think about wanting a cup of coffee and you go near a cafe, the barista inside the cafe will immediately make your coffee exactly the way that you want it. Exactly the way you want it. And it will be ready and hot and fresh for you on the counter the minute you walk in. It doesn't matter how many people are in line. It doesn't matter what time of day or night it is. It doesn't matter where you are, right? The transit lounge at Dubai. It doesn't matter.  You bypass all the other people in the line. You walk right up to the counter. You pick up that cup of coffee. You look at everybody, laugh a little bit and walk out of the store. But you have to pay for the coffee. Which one do you choose? Which one? Now I ask this question all over the world. People who live in New York where the stereotype is that we're very tense all the time and like rushing everywhere and places like Portland where everybody's high all the time and like not really in a rush to do anything. And the answer is always the same. And this is because in your mind and maybe many of you were thinking about this when I said the example, you thought yeah, you know what? I really want that superpower because I'm busy. I'm a busy person. I have things to do. My time is really valuable. If I didn't have to wait in line for that coffee, I think of all the things that I could do. My time is valuable. Now if you're an economist, you will know that the right answer to that question is A. It's always A. Because in A, where I give you the free cup of coffee, I'm saving you $3, which is money you would have spent anyway. And in B, you're still paying for the coffee. It's still costing you the $3. That time you're saving isn't worth $3. Because no matter how much money you make, you can't take that time and actually monetize it because it only comes in small intervals throughout the week. You can't aggregate that time and do something meaningful. The only thing you'll do with that time is send out a tweet. And I guarantee that that's not worth $3. The right economic decision is A. But we never choose that one. Huh, why is that? Why don't we ever choose, why don't we often choose the rational economic decision? Because people are not rational. And there's increasing science to back up this idea that we make decisions emotionally. We make decisions instantaneously. If you've read the book's blink or nudge, you will have seen some of that work. We are programmed to instantly decide, I like that guy or I don't like that guy. I like that thing or I don't like that thing. That looks good to me. Oh, that looks terrible. Instant decisions. And then we backfill that with our rational minds. So we make the decision fast and then we rationalize it. The beauty of games and gamification are that they speak directly to our emotional core. They act on our brains in the most basic and powerful and meaningful of ways. They cause us to be inspired and to try to do things that are more meaningful and directional. When we bring feedback, friends and fun, when we understand people and create an environment that allows them to experiment and multitask and see the world in a new way and rethink their engagement, we can produce amazing, amazing, amazing things. So with whatever time it is that we have left for questions, I do wanna say that if we don't get a chance to talk, I'll be around for the next couple of days so we can chat, but this is my Twitter handle at Jesus Kerm and I love for you to ask me crazy questions on Twitter or in person. I love it. I love questions like, Gabe, well, this is all fun, but how do we gamify something serious? Like cancer, I love stuff like that. So if you wanna talk about how to gamify anything at all, you can reach me on Twitter or...


Hello, fan. Thank you so much for watching this video. Today I have the honor of sitting down with Joni Evans, one of the co-presidents of the Federation of Gay Games. Joni, welcome. Thank you so much for sitting with me. My pleasure. My pleasure. I know you have an insanely busy schedule, so I appreciate you taking the time out really to sit down. And for probably a lot of my followers who aren't even aware that gay games existed, I didn't know it was a thing until a year and a half ago. OK.  OK. Well, I grew up in Birmingham, which is like the second city in the UK. So and growing up, there wasn't a lot of opportunities for me to play the sports I wanted to play. Football was my sports soccer on the other side. So in the UK for girls, there wasn't the opportunity to play after a certain age because it was football was banned for women for about 100 years because after the war, women used to play football. When the men went off the war, the women were playing football. And then when the war ended, the FA, the Football Association, banned women saying it wasn't the sport for women. And it sort of taken like 100 years, like the recent euros that we had in the UK was like, there was a lot of information came out about the banning and whatever because there was a lot of women who continued to play, you know, in secret, in secret. How bizarre to not be able to play a sport. Yeah, because the first England women's team, they played without the permission of the FA and they only recently got their original cap. They got the caps for their game. So in terms of that, it's like women, women have never been stopped by playing whatever sports they have, but it's actually getting a recognition for playing the game, which is another thing. So in terms of that, I never felt that I had the opportunity to play football as I was growing up. So when I came out, I came out when I was 25 and it was like, well, I wanted to be doing something. And then I knew I wanted to do football and but was how I was going to do it. Because at the time I was playing netball, which was more acceptable. So and netball, it's a it's a bit like basketball, but you can't keep bouncing the ball. And it's quite a national sport in the UK and Australia as well. So it's now become more professional. But that's always been the go to female sport in the UK when the men had their go to sports football. So and so now it was like, so I came out at 25. I wanted to do something. I wanted something recreational that I did. So I found a team of women that I assumed were all lesbian because somebody told me it was a lesbian team. And so in playing, but in the times that we were playing in our in our leagues and stuff because we were relatively out anyway.  You were not afraid? No, no, we weren't at that time. Because the moment I came out, it's like, yes, I found myself. And was there danger in that though? Not not on the women's side, really, because it was more that that gay men had more because people didn't recognize lesbians. You know, you're talking about the late 80s, early 90s. There was no laws against being a lesbian, but there was laws for men and stuff like that. So in terms of that, it wasn't. But it was really just thinking that for my team, we just wanted to be out because we were getting so much abuse like homophobia, racism from other teams because we just, it didn't matter if we won or lost, we just wanted to play the game. And so we thought, well, if we're going to get a lot of stick because people saying, I'm making fun of us being lesbian, we might as well own that and say, right, we're lesbian, so what? Yeah, the less you have to lose, the easier it is to just say, f it. So we did get a lot of flack from the league and stuff around stuff that we did. And, but, you know, we played in like East, East London in Hackney, on Hackney Marshes, and, you know, so we were very much a grassroots team. And in terms of that, we became the, once we decided we were going to come out, we then had to change our constitution, we had the code of conduct. And in terms of that, it's really, and we couldn't say we were totally out, we had to say we were predominantly lesbian because we didn't realize at the time that at least one or two women were straight. Oh, yeah. So, and it's more about our team being accessible for everybody. So it's like, it's as long as there's a woman that wanted to play sport, they had to come with the understanding that the team was predominantly lesbian. So everybody was welcome. So, yeah. And so am I correct in stating that that was like the first of its kind in the UK? It was. Yeah, we always thought that we were first in the world. And I was telling everybody we were the first to out team in the world because we didn't know of any other teams until we actually went to the gay games. And then it was really then we found out there was a team in Australia called the Flying Bats. And then we're saying, well, when did you come out? And it turns out there's only like a month's difference between both teams coming out in the same year. So I said to them, well, you can have the world and we'll have Europe because teams from Europe don't really travel too far. So we could live with that and that was fine. But it was a good time for us to do that as well. A significant accomplishment regardless. And even today, because when they had the women's euros, I was called upon. I've done like a video for the FA talking about my experience of playing grass roots, football and where it's led me to, which has led me to be the co-president of the gay games. So it's actually having that is showing like where women has taken their sport over the years. And it was really great to be part of that because it meant that the history of my team is now in the annuals of the FA when they would give us such a hard time in the beginning. So yeah. Okay, so while you're playing on this team, you also happened to be the team was part of a documentary called Running Gay. Yes. Can you tell me a little bit about that? Yeah. The woman that actually did it, she'd already had two people that were in the film already. There was a male athlete and a female athlete, both were white. And it was a view that they wanted somebody grass roots, I mean. So, and for something else that was, you know, because it was a team sport that was showing a different aspect. So they had the guy that was there, he was like, I think a triathlete. So he did several sports and he was gonna be attending the Vancouver Gay Games. And the woman, she was a long distance runner. So she did long distance running. This was in the 90s? Yes. Okay. So it was then, so basically she came along, filmed my team, she came and filmed us. I had to do a special coaching session so that they could see that they came and filmed it at Gay.  I didn't even realize what it was gonna do. Cause it got shown at the first thing, the time I saw the whole thing together, it was shown at the lesbian and gay film festival that was on that year in London. And then it was shown on national television cause they used to have a gay streaming program. And at the time, cause I lived in Hackney and it's a predominantly black area. You're not really sure how it is. If you're not street wise and stuff.  Yeah. And then he went, he went cool and he fist bumped me. And then I knew it was like, it was just like, the thing is, is that if you, the most that you fear and it doesn't happen like that, it just really changes your, how perspective that people think. And I think if somebody is a perfect stranger that somebody that I thought it might have a different, a negative reaction to get a positive reaction from that just meant that there was no fear then about it. But what it did, it really highlighted our team and what we were doing.  Like we were one team and then we like have three teams. And you're talking about 30 odd years ago, my team is still going. They still followed the same ethos. At the moment, I wish they did more because I sort of carried them and I'd want them to carry themselves. But, you know, the thing about having exposure is to make it less for anybody else coming up behind you. Because at the time, we weren't the only lesbian team in London. There were other lesbian teams, but none of them was as vocal as my team. So we would sing everywhere where we went.    And then when we, because my first games was New York. So when we arrived in New York, we found out there was another women's team from England that we knew and there was a men's team, their Stonewall men's team were there. And then we found there was other athletes. So there was a quite a small group of a contingent from the UK that none of us had met each other before that we knew we were there in New York. And it was the best experience. So in a way, gay games kind of brought the visibility within the community to know that you're out there. Yeah. I think one of the things about the gay games is that you don't, when you're in your own bubble in your own town and city and you don't know that there's this big event that happens every four years, that is part of what you do on an everyday basis at home and you can go and showcase what you've been doing all that time. It's just a huge, I don't know, I can't explain how it is but it is such a, it made such a difference to my life knowing that whatever I wanted to do, there it was. And it was how can I get involved in that and do more and what would it mean to be part of that? And that led you to becoming the co-president. Well, you're talking there's been many years in between that from 1994 to eight years ago. But I think because I was never part of any city teams or whatever and in the UK, we never did a lot of things. So for me personally, it was very much an individual thing, my involvement with the federation. I used to go to meetings as observers, pay my own way just to be there to hear what was going on. I'd be do committee work. That's what I did this time. And that was my way in to see how it was. And then there was a time when I took some time out and I wasn't involved in the federation at all. But I never was there as a delegate for any particular, the only time I was as a delegate was in 2006 and it was because somebody had asked me to go in their place. It was a guy called Ivan Bussins and he was, he had terminal cancer and we had connected in Chicago and he was saying, look, there's loads of things that we do and I want you to be involved in, you know, Team UK for us to sort of like do bits and pieces as a yes, I'm on it. And then he couldn't go to the meeting, the Arga that was in Leon. And then he asked me if I'd go in his place. And it was at that time that the federation introduced the role for a specific female role for the female vice president of diversity. And so with people that were there, I had some guys from Germany, one of the German teams are saying, go for it, go for it. I was going, no, and I was really shy thinking, I can't do that. I don't know enough to put myself in that position. And they encouraged me. And then I phoned Ivan who was in hospital and I said, what do I do? And he goes, go for it, go for it. And the thing is, is having somebody that believes you're new enough that they want you to go for something that would make such a difference, not just to the organization, but to yourself. But at the time it was difficult to, the post was for a year and then it was gonna start on a two year term, but I only did it for a year because I didn't feel confident enough to take it on. It was at a time I didn't have a home computer and all of that. And it's like, it just seemed really daunting at that time to take on something on such a vast scale on an international level and be responsible for people's diversity. It was a bit daunting. So. And now here you are? Yeah. But being co-president came about because London did the bid for 2018 and weren't successful. And at a time there was a vacancy for the co-president. And they asked me if I'd do it and I was thinking, why me? I did say to them is it was because I was black because there wasn't any diversity on the board at the time and I made them feel uncomfortable for about a few seconds. But it's not really that, it's people within the Federation would look on the commitment that you give over the years and whether or not you do big things or little things, people do watch you and they do listen and they do make comments everywhere else. So I know that I'm in this role because of the work that I had put in before because people don't take people on the board lightly, especially a co-presidency position. And in the beginning I didn't feel like I could do it but for me it's all of what I've believed in all of the years and now I've got the platform to say that I want to be there to be the voice for the person who like me didn't know about the gay games, like me didn't know that I could play any sport that I wanted. You know, opportunities are out there not just to do a sport but all different manners, coaching, managing. You know, trying to get people involved you don't have to be a sports person because there's all background stuff that still needs to be done and it's important. Well, I find that oftentimes the leadership determines the culture of an organization and being an outsider and coming to the aga for the first time in any gay games related event at all. I find that between you and Sean, yes, you have your wealth of experience and your knowledge but your energy, you also have a very generous spirit. You're open, kind, big hearted and so coming to the aga and witnessing that is it makes me want to be here so I think that's great. I think it's that, you know, we want to be an open organization because my main focus is that our board reflects the membership we serve because if we can do that we know what our membership needs and we can continue to give them a good service. I think one of the things is that I love our board. We all get on, we all have the same, the makeup of the board is that we all have the same feeling, the same idea about what we want and we are very much here for the participants. We're here so that whatever we're producing every four years is going to be acceptable for whoever comes, they're going to have the best time. Nobody knows what goes on in the background, it's like we're like gray-haired and pulling teeth and you know what I mean, all of those happen but then we know that nobody, that would never be reflected in the games because somebody would have the best time. We have a new officer at large and he was in the last games and I'm telling, no, Marquis, Marquis, yeah. He was in Paris and then it was like, I'm selling him all the horror stories of Paris and they're going, I never saw any of that. I said exactly because that's the experience. We don't take any negativity that happens in trying to get things done. That is never, we never get that reflected to our participants because it's about them having the greatest experience that they can have. Okay, so to sidestep for a moment, why a gay games and why not just the Olympics? The Olympics is there, there's other world competitions.  Well, the thing is, is that I think Tom Wendall had the right idea because he was an Olympian and he was in Mexico and he felt that he wanted to bring the same sort of thing to the community. I mean, in the beginning it was called the Gay Olympics but the USIOC wouldn't let us keep the name and they bought Tom Wendall to court for the name to be removed.  As if they have the legal right to own something that's been around for thousands of years. But if you're thinking about 1982 and what the world was like around HIV, AIDS, the gay community, so in terms of that, the USIOC didn't want to be associated with that because at the time, you know, there was like the Petal Olympic, the Petal Olympics, there was all different Olympics and they never got anything but the Gay Olympics, that was it. I mean, now it's stigma. But now the relationship, we could use it, do you know what I mean? But it's gone so far beyond just like what would be the benefit. We run exactly the same type of event.  It's a multi-sport event. The Olympics is a multi-sport event. Although we probably do more because we also have arts and culture because we are reflecting the whole of our community. And then it's like, if you look at the number of participants, the Gay Games has more participants than the Olympics. But obviously we don't have the same risk. In the spirit of inclusion. Yeah, yeah. But we don't have the same prestige, money, you know, or whatever. So it's because we're a voluntary run organization, our board is fully voluntary. And we're gonna change that. Yeah, of course, of course, yes. We're looking for paid stuff. Any sponsors out there, please. We're gonna work on that. Yeah. But yeah, my thing is that if we can generate our own money, we can do a lot more because it's not just about producing the games at the end of four years. It's the work that we do in the run-up to that and how we promote ourselves, how we engage the community. How do we engage more women? How do you get more young people?  The only qualification you need to be is 18. But we also recognize that there's people under 18 maybe 16 to 18 that we can get involved because it's at those crucial ages before they reach adulthood that there's nothing for young people. We're very much aware of young suicides and stuff like that because there's no outlet. And it's how can we change people's attitudes? Because if you start going at people under 18, it becomes, people make it sorted when it's relation to the gay community. And whereas we're trying to say we've got young people who don't know of confusion about their sexuality, their identity and all of that.   You know, you can be your true authentic self and that's the main focus of the Gay Games. It's a sense of normalcy without it necessarily being focused on being gay.  That is exactly it. And that's the thing. It's just about being your true authentic self. And people can't be that where they are in certain instances. But you know, I make a point of being my true authentic self in everything I do because I know that people look to say, I want to be that person. People say that they can be.  And it's like, I think it was Gandhi saying, you know, we'd be the change. Well, I can't remember the word in his, but you know what, that sort of thing. You know, if people don't see it themselves, then they don't know what possibilities are out there. And I think over the years, I don't even know half of the things that I've done and what I've done, but I keep hearing things and, you know, people say things to me, but I'm just doing what I feel is a natural thing in my life because I would never want anybody to go through what I went through about not having any knowledge. I had no knowledge. I grew up in Birmingham, the second city is like, it might as well been in the boondocks or whatever because the city wasn't aware of anything. I didn't know of lesbians till I was 25.   And that I wasn't going to let anybody be in a position where they were in silence and didn't have an outlet. So. Do you have to be on a team to compete? No. No, when I played in New York, I played with Hackney and for me, I went to my first, I went to Chicago Games as an individual player. I put my name in to say I'm looking for a team. I got picked up by some women that played in Dallas, Texas. I had a fabulous time. I met new people. Individually, if you play a team sport and you're on your own, you put your name in a hat and somebody will pick you up from another team because not everybody can have a full team. Some people may. And those sports were, it's not even a team. Yeah. No, some teams are made up because if there's, I think in Paris, they had people and because there was enough people that had said they were playing soccer, they made up a soccer team of all those spare people.  I've got friends all over the world. It's just incredible at the experience that you get that it's, yeah, it's more than I can imagine that it could be in any instance, yeah. Okay. So in that same vein, what words of encouragement do you have for someone who is watching this, never heard of the gay games until now, is interested, but it's still a little hesitant, reticent to either participate or attend. Well, obviously, I mean, you've got not very good website, but you could probably look on our website. But I think it's trying to find out where the teams or if there's a team in your area, because say it's more easy in America because Americans have been a bit more aware of the gay games than most of the world. But in terms of that, it's trying to find where your local groups are. That might be a start, but it's also for us at the gay games, people can contact us if they wanna get in contact with local teams or we can be a first protocol. We've got contact at gaygames.net if people have any questions. We can get people in touch with, if there's somebody in their area, we can get them in touch there. People wanna help us. We've got loads of work that we're doing. People don't have to be part of any sort of team or group because we've got loads of work that we're doing on a year-by-year basis. That's great because for me, even my first question, because the physique bodybuilding is not gonna be available for 2023. However, there's powerlifting. I've never done powerlifting. So my initial reaction is, okay, what are the logistics? What do I need to wear? What does the process look like? So knowing that- We could get you in contact with whoever runs the powerlifting thing. And that's through the website. Yeah. I mean, I can tell you now because Adrian's here. Yeah, I'm saying for the folks at home watching and thinking the same thing. Yeah, it means that we can be the first protocol for them to get in touch with whoever's part of their, whatever sport they're doing. In terms of that, if we know individuals, because if we know an individual, that individual we know will give that person anything that they want to get them involved because we're just always looking for people to help.  And it's really to get people involved in that way. So you don't have to do it as a city team. I don't have a city team that I follow. We have out for sport in London, but I'm not really part of them. I'm just an entity on my own. But there are ways of getting involved. And if we don't know it yet, unless we know what people want, we can't know how we're gonna give it to them. So first protocol, you can go through our contacts at gaygames.net. And feel free to questions in the comments, wherever this is posted, feel free and I'll do my best to link you as well. Yeah, cause we can answer anything. And part of it is that we want to know people are asking us questions cause we don't know what is wanted cause then we can do better or do more or go look at a different area that we could. I mean, we're really nice to know how we can engage younger people into our organization, especially if they don't know about it. It'd been really nice to know what young people are looking for in an agency like ours. I mean, I know it's called the gay games, but people need to look at our mission because our mission statement says a lot more about what we do. But it is that thing, is that we welcome anybody to assist us. Okay. Yeah. Is there anything else that we haven't discussed that you'd like to share? No, it's just that we've got two coasts, hostly, personal and registered. If people want registrations are open for both the games and in Hong Kong and Guadalajara. And then we've got our next gay games in Valencia in 2026. So there's lots of opportunities to participate. Is there a hard deadline for registrations? I think, no. I think for registrations now they're open. We're more than likely to take registrations up to day of events because it's really about making people, it's about inclusion. Yeah. So that's great. Everything that we do is either that, it's really around trying to make sure that we do everything so that people can participate, do everything that people can be included and we do everything to make people be able to do their personal best. Okay, great. Please come, say hi. Well, thank you so much for sitting down and having this conversation with me. I know this is so valuable for everybody watching. It's my pleasure. I mean, the thing is, we want people to know that we are a welcoming organization and that people can come to us for everything and we want people to know about us anyway. So anytime, if you need me to come and speak again, I'm quite happy. Yes, and I was going to say that I would love to follow up with you throughout the year. If questions come up for my followers, we can maybe deep dive a little bit. I'll be happy to do that. Okay, fantastic. Thank you. All right, thank you. All right. See you. It's very important for me to be out as a lesbian because that's part of me. And I think because I enjoy football so much, if I can't give myself to football, the whole of myself to football, then I wouldn't feel comfortable and feel comfortable at all about me being a lesbian. So I have to be out as such.  But it's like, you know, people ask me, I'd say yes, I am.


Hello and welcome to OpenTelemetry Collector Deployment Patterns. My name is Judas Yipashyan-Kroling. I'm a software engineer at Red Hat and I work on the distributed tracing team. I'm a maintainer on the Yeager project and a collaborator on OpenTelemetry. Now for this conversation here today we're talking about patterns. Before we go into that we're going to invest a couple of minutes talking about OpenTelemetry and the OpenTelemetry Collector. Now those are the patterns that we're going to cover here today. The first one is the very basic pattern and if you followed a quick start of OpenTelemetry Collector you know them already. The second pattern is the normalizer pattern and followed by a couple of variants of a pattern for deploying OpenTelemetry Collector on Kubernetes. We talk about load balancing, multi-cluster and multi-tenant scenarios as well. All right, so all of the patterns that we talk about here today they are available in this repository here. This repository contains images and configuration file examples and a deeper explanation on those patterns. Now you can download this slide deck here either from the conference's website or from the spot from here and I'm also sharing this slide deck right now on Twitter. All right, so let's get started. OpenTelemetry is a project that was created with a fusion of open tracing and open sensors. It is actually composed of two big parts. The first one is the specification and conventions part. So it is where the community gets together to determine what are the cementing conventions that we should all be following when instrumenting our applications and it also defines the specifications for telemetry data types, for traces, for metrics, for logs, and so on and so forth. We have a group taking care of the client APIs or instrumentation APIs as the case. We have a group making a definition about OTLP and OTLP stands for OpenTelemetry Line Protocol and it is in concrete terms it is a proto-birth basically, but it is a specification of how we can transmit data, telemetry data from one service to another. So it specifies both the message and how the services should look like on the client and on the server side. And then the fourth big part of OpenTelemetry is the collector. The collector is where we are focusing on here today. And the collector, if you go to OpenTelemetry's website, the documentation, and open the collector documentation, you see that the definition of the collector is this one here. It's a vendor agnostic way to receive, process, and export telemetry data. It kind of hints at the internal architecture of the collector and the collector is composed of the following components, right? It has receivers, it has processors, it has exporters, and it has extensions. So there is a special type of component that is not part of the pipeline. And a collector is viewed as the pipeline for the telemetry data. So data is either received or pulled by receivers. And the receivers put data at the beginning of the pipeline. Now once they flow through the pipeline, they get to the processors, and the processors have the ability to look into this data and do some action with them. Or just observe this data and create perhaps a new data point out of it. Now a processor, most of all the processors have finished doing their jobs with that specific data point with a batch of data, data reaches the exporters. And exporters can, again, be passive or active so they can be sending data actively to a final destination. Or they might be making data available for other systems to extract from the collector. Now when we talk about receivers, we are talking about things that emulate a Yeager server or that emulates the behavior of Prometheus or that implement called TLP on the server side. Or they might emulate a Zipkin server as well. Now processors, they might be doing sampling, they might be changing attributes or adding or moving, updating some attributes from the data points themselves. They might be doing some batching, they might be doing some routing and so on and so forth. And when it comes to exporters, we have a bunch of them for pretty much all the relevant systems and vendors out there. So we have exporters for Yeager, Zipkin and TLP. We have exporters that emulate or that expose a Prometheus compatible endpoint so that other Prometheus compatible systems can scrap data out of the process. And we have exporters for pretty much all of the commercial commercial vendors out there. All right, and besides the collector proper, besides the open time to collector itself, we have a couple of repositories or a couple of projects that are part of the same ecosystem. And the first one and the biggest one is the contrib. The contrib is where all the non-core components live, including the vendor-specific ones. So if you want to use a vendor-specific component for open time to collector, you'd probably download the contrib distribution first. Now, you might notice that the contrib distribution, the contrib binary is actually quite big. So you might want to consider using the open time to collector builder to pick and choose which components you want as part of your own distribution of the collector. And then for that binary, you have only the components that you need. So it's very slimmed down. It's very lightweight. All right, this is how a configuration file looks like for the collector. So we have sections for the extensions, for the receivers, processors, exporters. And we tie them all together under the service node. So we specify the extensions that we want there for this process. And we specify the pipelines. Now, the pipelines can be traces, metrics, and logs pipelines. And we might even have multiple pipelines for the same data type. So we can have multiple traces pipelines here, for instance. In our example, we have only one receiver, one processor, and one exporter. All right, so let's get started with the patterns then. The very first one is a basic pattern. And in this case, we have our application instrumented using the open telemetry SDK, exporting data with OTLP to an open telemetry collector located somewhere. Now, that open telemetry collector then exports data to a final destination. In this case here, Yeager. Note that throughout this presentation, I'm using Yeager as an example here of the final destination. And that is that with pretty much everything that you want, right? So it can be a different tracing solution or it might be a specific vendor or it might be even something not tracing specific. So it can be, in most of the cases, it can also be like Prometheus in here. And so this is our first pattern. I hope it was enough to warm up. And a second pattern that we have is or it is a variant of the first pattern and it is a fan out pattern. Going back to our original image, we have the same application instrumented with open telemetry SDK, exporting data with OTLP to open telemetry collector and the open telemetry collector exports data to Yeager and additionally to an external vendor. Now, the point here is we still have the ability to own our data, right? So we can still have access to our raw data within our realm, within our infra. At the same time, we can send the same data to an external vendor and have a different view of this data. Now, the second pattern that we have today is the normalizer pattern. And in this case here, we might probably have a Prometheus client instrumenting our application for metrics and we might have our application instrument using open tracing for the traces with the Yeager client as the actual tracer. Now, data is then either made available to Prometheus or sent to Yeager and in this pattern here, we are using the collector as a drop in replacement for Yeager and Prometheus when it comes to the contact with our application. So Prometheus is now scraping the open telemetry collector that we have, so not our application anymore. At the same time, the application is sending the Yeager data or the Yeager client is making a connection to the open tracer collector thinking that it's a Yeager server, Yeager collector. Now, the point here is that our open tracer collector that is sitting in the middle between those systems, it has the ability of looking into all the data points that are flowing through this pipeline here or through this communication channel and ensuring that they all have the same set of basic labels. So let's say that we want all data points to contain the cluster name they originated at. So we can ensure that the collector has a couple of processors adding a collector label to metrics and to traces. Now, our third pattern here is a couple of patterns actually to deploy on Kubernetes. The first one is using a sidecar. So a sidecar on Kubernetes is basically a second container as part of your pod. So you have your application pod with one container in it and you add a second container with the open telemetry collector. Now, that open telemetry collector acts as an agent that is possibly deployed on another namespace, observability here in this case and from that collector we export data to Yeager. Now, there are a few advantages to this kind of approach here. The first one is if we decide to change the way that we send data to Yeager or if we change our Yeager location or if we decide to not use Yeager anymore or if we decide to use another exporter in addition to Yeager then we can just change one deployment here in our observability namespace. We don't have to change any of the sidecars. Now, the second advantage is we're not talking about only one application here. We're talking about multiple applications in multiple namespaces. So the advantage here is we get better client-side load balancing when we have multiple instances of the clients making a connection to one server. So when we scale up the number of instances or when we scale up our open-touch collector on the observability namespace then all the new collectors or all the new instances of the collectors on the workload namespaces they would then be using those new instances. So it's very likely that load balancing would work better when you have more clients than fewer clients. Now, another advantage of having sidecars on a per application basis on a hard basis or per deployment is that we can fine-tune the configuration for that collector to the necessities of that application of that deployment. So if we have a critical application here if we have a critical deployment we can have a very specific configuration for the collector perhaps with a more resilient more retry mechanisms and perhaps even more memory more CPU allocation for that specific process. Whereas for lower criticality services we can have a lightweighter configuration for that agent, for that sidecar. Now of course managing hundreds of sidecars might be headache and that's why we might want to use an open-to-lan tree operator here to manage the sidecars for us. So the open-to-lan tree operator can inject and manage sidecars on our behalf. Alright, the second variant of this pattern is not using a sidecar, but using a demo set. So the advantages are very similar. So we have a collector that is very close to our application, to our workload making it easy for the application to offload data very quickly to the agent in this case here in the demo set. But we don't have many of the advantages of the sidecars, for instance it is harder to do multi-tenancy in this kind of scenario because we have multiple namespaces running on the same node and we have only one collector running on that node. It means that that collector is going to see data from all the tenants in there. So it's harder for us to manage a tenancy at that level. Not impossible of course but harder. It is also harder to have multiple instances of the collector on that same node. So if we need to scale up for some reason then it's harder to do. Again, not impossible, but harder especially when it comes to service discovery. Now the biggest advantage of demo sets over sidecars is, as you can imagine, the overhead. So OpenTime 3 collector itself is not big or it doesn't have to be that big in here. So as a sidecar I would say that the memory consumption of the collector itself should be around 5 to 10 megabytes. But it just adds up when you have hundreds of those instances. When we have as a demo set we only have one collector per node. So our overhead is lower. Now the idea is that each one of those collectors would then collect data from the applications running on those nodes. So it's local to the application and resiliently, securely and safely send data to a central collector. Now our next pattern is about load balancing. To explain a little bit about why you need load balancing at the collector level and instead of just a regular GRPC or HTTP load balancer we have to go back a little bit and understand how tracing actually works. So if we have a user doing a transaction on service A it's very likely on a microservice architecture it's very likely that service A would make a connection to service B to service C to service D and each one of them would then make downstream connections to whatever services they need to get information from. Now the way that tracing works is not that we are going to wait for all the services to complete and then service A sends data to the tracing backend. It's not like that. The way that it works is service A is responsible for collecting and sending data for related to its own operations. So only the spans belonging to service A are going to be sent from the service A to a collector somewhere. Now the idea that we have here is that each collector should have a complete view of the trace so that it can make a decision based on that trace. So for instance, if we are doing tail-based sampling, we want to take a look at the whole trace and make a decision whether we want to send or not, but perhaps we are doing some analytics on the tracing data. So we want to take a look at the whole trace perhaps compress it in some way perhaps just extract some metrics and discard the trace itself and things like that. Now to do that we need to ensure that all the traces all the spans for the same trace are at the same collector. What we do not want to have is spans for the same trace at different collectors. Now we ensure that by having two layers of open telemetry collectors.  Now this load balancing exporter will split the batches that it receives from the clients. It will look at each single span in it it will take them, extract the trace ID, hash it and determine which collector should be receiving this data point. And it sends data to that data point. So you can have a HA like deployment of the load balancer with three replicas for instance where and at the same time having hundreds of instances of the collector backing that load balancer. So as a final destination or a major destination for this data. Those collectors might then be doing the same thing and sending data to the final destination like eager or your other tracing system. Alright, so the next pattern that we have is a multi-cluster pattern and in this case here it is very similar to one that we've seen before for the Kubernetes either sidecar or demo site. So the idea is that we have our application very close to our application. We have a sidecar or an agent that receives this information from that application and sends information to a central collector local to the cluster. Now that collector then makes decisions about data from the cluster itself. So perhaps it is adding cluster specific information to the data points. Perhaps it is doing tail-based sampling at the cluster level but the point here is that we centralize all the data to that collector and that collector then makes a very secure connection to a collector on a cluster on a control plane cluster. Now that communication might have different resiliency, different reliability requirements. They may have different security requirements. So it makes sense to have to extract that knowledge or that logic into one collector that works at the boundary. And then on the other side, on the control plane cluster we don't have this collector that is receiving data from other workloads. So of course we're not talking about one cluster only, we're talking about multiple clusters sending data to a central control plane cluster. Now that one is receiving data processing the way that it should be processed and sending data to the final destination here. So this is the multi-cluster and finally we have the multi-tenant pattern. And in this case here we have multiple data coming from different tenants. So we might have one application that is multi-tenant or we might have multiple tenants as clients of our application. So I named it here open time to collector as a service, but it's in the real world it's very likely to be like IT department owns your observability stack and each department is then a tenant that is then charged back based on the resources that are consumed. So we have one central location for data to arrive at. So our open time to collector that is then taking care of some logic that applies to all the tenants. For instance in doing security, in doing data cleanup perhaps we are removing some personally identifiable information from these pens. Perhaps we are adding some other information to those pens and sending data to the final destination. So in this case here two different ergers, one for each tenant and then we can charge one of those tenants based on their user usage and the open time to collector is central here it could then be owned by IT itself. All right we have a bonus pattern here and that is a per signal deployment of open telemetry collector. In this case here we we have one collector taking care of the metrics part and one collector here taking care of the tracing parts. Now there are some reasons why we would want to do that. And the first one is the way that we scale an open time to collector doing scraping is different than the way that we scale as in a push based model. So when we have a push based model we can just scale the number, we scale up the number of replicas that we have and the new client to then just find new back ends and send data directly to them. Now when we are talking about pulling data out or in a typical scraping mechanism from permissives we cannot just increase the number of replicas for the collector. Because the more replicas that we have the more scrapers we have and they're all going to the same targets. So we might want to have a different instance of the collector taking care of metrics than the one taking care of the tracing data points. Another reason to have it displayed per signal is that right now the maturity of each one of those signals are not the same. So tracing components are very mature, whereas the metrics components are not that mature. So we might want to mitigate a risk, we might want to get the traces on a highly available scenario for the production quality deployment for the tracing pipeline and for the metrics pipeline we have something that is more fragile than the tracing one. So those are the patterns that I had for today and I think there are key takeaways here. The first one is the open telemetry collector is very versatile. It's incredible how many things we can accomplish with the collector and I think the key here is to understand and getting to know the existing components. So once you know which components you have, you can start planning, you can start drawing and architecting how you're going to deploy an open telemetry collector. Now a key thing as well is to understand that collectors can be changed together. So if you go back to our patterns here, most of them they are about chaining open telemetry collectors together in different levels. So we have sidecars talking to other collectors as a local to the cluster talking to collectors on a control plane cluster and so on but they are all the same minories they are just configured differently and finally mix and match components and collector instances. So plan your deployment and mix and match and if you don't actually have to use the same binary you can build your own binaries depending on the needs that you have at each of those levels. Alright so those are some of the resources that you can use to continue from this conversation here. So some information about open telemetry the collector, the location here for the contrib repository, the computer and the patterns from this presentation here. If you've used open telemetry collector before I'm quite sure that you have your own patterns. So here's my action item to you. Go to this repository fork it right now and include your own patterns. Thank you very much.


Hello, welcome to Toffy TV, this is the FIFA Career Mode episode 8. Today we take on Burnley away at Surfmore in the Premier League. Yeah, are you feeling confident? Yeah, you've got to be confident having a big win last time out, 7-1 against Crou. Still unbeaten. Still unbeaten in other than pre-season, which we'll talk about. And obviously we've got that big potential job on the horizon with actual real evidence in real life. Got to impress. That's it. You've got to be confident. Look, Burnley had a very difficult team that's obviously to go on and beat, but I'm confident in us, yeah. Get into it. Right, so I've had a message from Josh King. He said, hi boss, good to see you. It's great things are going so well for me and the team at the moment. I don't think I've ever enjoyed my football as much as I am right now. I'm happy at the club, and so I was wondering if it might be a good time for us to talk about renewing my contract. Obviously a couple of goals, you know, a win against Crou last time out. It's been a play at one game. This isn't the best time. No. We'll see in January, son. There you go. I know you want to get things sorted out, but there's so much going on at the club right now. That made him happy. Our man management is... Perfect. Perfectamundo. I was going to say in... Do you hear that farad? Perfect. I love Nedwina. We're going to pick our team for Burnley away at Turfmore. Dom, big striker. Up top. Can cause them problems. Charlotton on the left, would you say? Yeah. Give Daniel Hamers a rest. He was brilliant against Crelix, I've got to say. Sar going straight back in. Hamers. Go on lad. Why am I going to because I've got to make the bench as well? Right, here we go. Big game. Turfmore. Let's do this. Right, here we go. Burnley away. Second away given of the season. We've already beat Old Trafford away once. I've gone, I've gone. Oh, it's Sar down the right. Look at yourself. Second chance. Oh, lucky Sar. Oh, you're just a bit off there, Hamers. I was trying to switch it to Richie. I was trying to get back in Columbia. Oh, it's a great ball to Sar. It's on Sar's side. That's outside. Oh. Oh. Oh, my God, he's still going. Oh, oh. Unlucky Sar. Oh, no. Yours at JP. Get it. JP's, I'm in a great season so far. And again, and again, inside. Oh, what a ball. Oh, yes. Oh, good save Nick Pope. I should have took it earlier, Dom. Oh, what a great tackle. What an absolute great tackle. Oh, what a run. Come on, Sar. You're faster than them. Yes, Sar. Yes, Sar. Oh, that was a terrible pass, Sar. No, Sar. Oh, then absolutely it's atrocious pass. Yes, Sar. Can I have your pass? That's a Peter's. Brownhill. Chris Ward. Oh! How do you get away with that? I wanted, but it's fine. He plays Dom through. Got to me. He's one of them. The keeper. Yes! It's 1-0 to the F. What a goal.  What a goal and what a passage of play. That was 1-2s left, right and centre. That's the furthest goal he's ever scored. It is, yeah. Look at this. Look at that 1-2 left, 1-2. Great run. Managed it on side. Perfect timing. Bang, back to the keeper. Absolutely ex-semeccional. Oh, is that meaner? That great tackle. Well in yellowed. Great tackle. Cut that out Godfrey. Cut it out Godfrey. Nice JP. Got to be. Look at this. Get to that with Charlotte. Hold it up, hold it up. Get out to him. Oh no. Oh no. Yours at JP. Oh what a great save. What a save. He's keeping us in the lead. It's 1-0. Here we go, second half. Good first half I'd say. Didn't take all our chances but JP kept us in the lead at the end there. Nice hammers. Yours at Saar. All the way lad. Oh well played. Oh, the skill is beautiful. Oh, not the greatest touch. See the core, I keep chasing. That's it. Playing. Shoot. Oh, oh yes. Getting. Didn't have a clue what was going on but I clicked a random button and we scored. Hammers, that is beautiful. What a celebration as well. God, I hope nobody comes in for him in the game. Falling apart. Sean Daish is in this. That's it, typical barely. What are we doing? Easy to play, it's well played. Nice and we want it back again. I'ma throw and goal again. Dom. Back in, back in. Oh, should have scored it shouldn't have. On a ball. Dom's in the box. It's Dominic Cavett-Lewin. Get it. Oh what a good save by Nick Pope. The aim of the touch. The aim of the cross. Good effort from Hammers Rodríguez. Oh no. What was that JP? Easy. Oh, what a ball. What was that Dom?  Ben Godfrey. Overlap. Oh, it's nice. Look at Dean. Switch of play. Sa. Sa. Oh, he smashed it over the bar. Let's give Hammers a rest for Wmangatuka. Get him on.  He's on the ball. Plenty of space running too. Wmangatuka makes the run. He's going all the way. Wmangatuka back post. Not as easy as it looked. Back in science, Wmangatuka. Oh, nice bit of play. Dean on the overlap. He's still going. Shoot. Wmangatuka. I thought Wmangatuka was going to score on his debut. He's not going to lie. Drop that Nina. Good tackle.  How jammy was that then? Boring.  Go on Richie lad. Take the pass one. Take the pass two. Sackle would come to the core eight. Wais it into Calvin Lewis. Oh, what a goal. Beautiful footwork from the core eight. Laser off the dunk. And he smashes it in the back of the net. 3-0. This evident team is so good. It's special. I wonder who's that coming down to. Wmangatuka. Got to be Wmangatuka on your Premier League debut. What was that? Was that a pass? Was that a shot? That was dreadful. It's Siar on the run. Someone is back post. It's Wmangatuka's debut. Yes! Wmangatuka on his Premier League debut. Makes no mistake that time. It's 4-0. We were never going to beat a turf more. Get it. It's a great ball. Oh, it's going to get to it. Oh! That's got to be game set and match. There you go. Full time, 4-0. Brilliant match. Yes! Shall we get into a press conference? Yeah, let's get straight into it. Can you explain the disappearance of Fabian Delph? After that win, do you think you can win the league? I think we've shown a real great start. I don't want to be too optimistic and say we can go in the win league. But we can go on and win the league. I think we can go on and win everything. Yes! Come on! Do you think Dominic Calvert-Lewin is the best striker in the world? I think he's definitely the best striker at Everton. In the world. Yes! He's the best striker in the world. Come on up the dump. Up the dump. There you go. 4-0 at Surfmore. Had a deadline day as well in the next episode. Tune in to find out whether he makes it or not. Tune in to find out whether he makes any more signings or shifts any players. Probably going off of Henry Gormister. I don't know if I've got cables or something. There you go. You've got more videos and good and live exclusive videos. Have a good patron. Check out Cam's channel on YouTube. The Mighty Blues. He's just it 4k subscribers. Get in. Thanks for watching. See you later.