In the documentary, Supersize Me, Morgan Spurlock eats exclusively of McDonald's for a month, and predictably his weight, blood pressure, and cholesterol go up. But so do his liver enzymes, a sign his liver cells are dying and spilling their contents into the bloodstream. His one-man experiment was actually formally replicated. A group of men and women agreed to eat two fast-food meals a day for a month, and most of their liver values started out normal, under 30 here for men. But within just one week, most were out of whack, a profound pathological elevation in liver damage. What's happening is NAFLD, non-alcoholic fatty liver disease, the next global epidemic. Fatty deposits in the liver can result in a disease spectrum from asymptomatic fat buildup to NASH, non-alcoholic steatohepatitis, which can lead to liver scarring and cirrhosis, which can result in liver cancer, liver failure, and death. It's now the most common cause of chronic liver disease in the US, affecting 70 million Americans. That's like one in three adults. And fast food is a great way to bring it on, since it's associated with the intake of soft drinks and meat. One can of soda a day may raise the odds of fatty liver 45%, and those eating the equivalent of 14 chicken nuggets worth of meat a day have nearly tripled the rates of fatty liver, compared to like seven nuggets or less. It's been characterized as a tale of fat and sugar, but evidently not all types of fat. Those with fatty hepatitis ate more animal fat and cholesterol, and less plant fat, fiber, and antioxidants, which may explain why adherence to a Mediterranean-style diet, characterized by high consumption of foods, such as fruits, vegetables, whole grains, and beans, is associated with less severe non-alcoholic fatty liver disease, perhaps because of the anti-inflammatory and antioxidant effects. Maybe because of specific phytonutrients, like the purple-red-blue anthocyanin pigments in berries and grapes and plums, red cabbage, red onions, radicchio. These anthocyanin-rich foods may be promising for the prevention of fatty liver, but that's mostly based on petri dish experiments. There was one clinical trial that did find drinking a purple sweet potato beverage seemed to successfully dampen liver inflammation, though. A more plant-based diet may also improve our microbiome— the good bacteria in our gut, the old adage, we are what we eat may be changing to we are what our bacteria eat, and when we eat fat, we may facilitate the growth of bad bacteria, which can release inflammatory molecules that increase the leakiness of our gut and contribute to fatty liver disease. Fatty liver disease can also be caused by cholesterol overload. The thought is that dietary cholesterol found in eggs, meat, and dairy oxidizes, and then upregulates liver X receptor alpha, which can upregulate something else called SREBP, which can increase the level of fat in the liver.  That's what may be triggering the progression of just plain fatty liver into serious hepatitis. The accumulation of sufficient concentrations of free cholesterol within fatty liver cells to cause crystallization of the cholesterol, one of several recent lines of evidence suggesting that dietary cholesterol plays an important role in the development of fatty hepatitis, fatty liver inflammation. In a study of 9,000 American adults, followed for 13 years, they found a strong association between dietary cholesterol intake and hospitalization and death from cirrhosis and liver cancer, as dietary cholesterol can oxidize and cause toxic and carcinogenic effects. To limit the toxicity of excess cholesterol derived from the diet, the liver tries to rid itself of cholesterol by dumping it into the bloodstream, and so by measuring the non-HDL cholesterol in the blood, one can predict the onset of fatty liver disease. If you subtract HDL from total cholesterol, none of the hundreds of people they follow with a value under 130 develop the disease. Drug companies view non-alcoholic fatty liver disease as a bonanza, as is the case of any disease of affluence, considering it's already high in rising prevalence, needing continuous pharmacologic treatment. But maybe it's as easy as changing our diet, avoiding sugary and cholesterol-laden foods. The unpalatable truth is that non-alcoholic fatty liver disease could almost be considered the human equivalent of foie gras, since we force-feed ourselves foods that can result in serious health implications. However, having such a buttery texture in human livers is not a delicacy to be enjoyed by liver doctors in clinical practice, as it can have such serious consequences.


I would like to introduce Prasad Bhattakey, Chief of Experience Design at YUJ. Prasad is responsible for anchoring new engagements, building strategies and team. Making a difference is what Prasad brings to the table, as he drives user research, interaction design, visual design with equal zest and excellence. There is a gap the design community needs to bridge, and that's what he's going to talk about. Welcome Prasad. When we were talking about impact, I was, when the conference team came up, I was thinking, okay, what are the projects that made impacts to the customers and so on and so forth. But I realized that at the end of the day, the talent that is going to work on a project matters the most. And so, hence this topic. So, the quality of the talent equals to the quality of output, and hence the impact. And the talent would have, especially when you're working in an industry, the technical skills, maybe design thinking, wire framing, visual design, what not, equally matters as matters as the soft skills. So, it's kind of a mix of both, which creates a good career for you. One of them is missing, kind of tends to become a challenge in the industry, at least that's what happens in the industry. So, good career would mean satisfying work, genuine output, you're a happy life, you have profits for your company, and you have happy customers, profit, good salary, everything works very good, that's an ideal situation. But today we have a problem. Today, because a lot more companies are wanting a lot of UX to be done, design to be done, empathy for the users, there's a certain surge of requirements for user experience designers. So, the demand has really gone up, and we are really struggling with supply. So, that's the reality, and this gap is not good for our UX community. So, let me just quickly talk about why am I saying so. Right, so when there is a lot of demand, there is going to be scarcity in the actual real talent that is required, because suddenly you can't have millions of UX designers in the industry and everyone is doing really good. So, there would be always a gap and then there will be scarcity. The cost will go up, because to do the same good amount of work, you will be scrambling for designers. So, you'll end up paying good designers good salary, and so the cost kind of goes up, but there is a burn, because now you're going to work late nights and try to finish the projects. Some other projects which want to really do good UX work are going to do a lot more iterations than required, and so the burn will be extremely high. Now, what does this mean? This means instability in the industry. On the other hand, the reliability goes down, because now the quality is going to shrink. I have seen a lot more CTOs and CEOs saying, okay, let's hire a designer quickly so that our design or our product becomes more competent in the market. It's still UX, looks beautiful, but the quality actually goes down, because they're going to have a suboptimal designer designing it. Why am I saying so? Because I think all as a design community, we need to raise to the occasion and really see how we can bridge this gap. Knowledge is going to be a big challenge, because not everyone is going to have the knowledge that it needs to be a true UX designer. They may be put into a situation where they're expected to design something that they don't have knowledge for. So overall, our UX brand kind of starts diminishing. So this gap is increasing as I see. So let's look at some of the reasons why this gap is increasing. It's no more designing wireframe. Still about four, five years back when UX was critical in India, people who you can do wireframe, think before we start coding, you're a UX designer, fine, but that's no more the case. And people still talk about IA and UT as some of these jargons, but that's again not something that is expected. There's a lot more that is expected from a designer. Lot more times you would have noticed that some of your bosses or some of your seniors would say, I think this screen looks good. Why don't we just replicate and submit it off? So all of this is gone. This is what the businesses are now looking for. This is not only because what I'm saying, I am also seeing this, what my customers are expecting from me. They're really wanting something which is new, which completely overrides their competition. Business impact conversion. We have seen this more in time and again where business conversion is really expected. That designers are moving away from wireframe but are really seeing how the designs that they create really have a return on investment for the businesses. Design strategy. Are designers talking strategy for the long haul? Not really short term. Why don't I make this icon look beautiful? Or why don't I make this button look green or blue to match the color scheme? That's not what design is. Design is about coming up with a strategy which matches with the business strategy of the company and making an impact there. I think designers have to start getting there. Demonstrate leadership. Today, I guess, with this whole new UX generation of aware people, the CEOs and CTOs are really more aware of what UX is, what UX methodology is. So when they are going to talk to you, you need to talk their language of what UX is. And demonstrate that through your deliverables. I feel the principles of human cognition, that is something I see serious lack of people really talking that language. When you talk human cognition, I think people get it that that's where designers are making that impact. And I feel the gap is going to increase much more than what I just said. And here is why. So trend is that technology is really moving fast. We have seen bot. Bot technology is coming, virtual reality coming, augmented reality coming. And people are expecting personalized experiences, customized experiences. There is apps. Today, we see that there are a lot of discussions that are happening, that the apps are a little passe. People want inline experiences. Why should I even open LinkedIn? If I just talk to my app and say, OK, open LinkedIn and I want to give an update, the app should open automatically. So those kind of interfaces and interactions with machines is something that we should be ready for. And that's where our knowledge of human cognition will start mattering a lot versus can I design a file in a Photoshop, or can I create a wireframe in an Axiom. There are also things like voice and touch. And all of this needs to start talking at multi-channel or multi-devices. Are we ready as designers there yet? And I think we are not because we are not even thinking about the overall end-to-end user experience that we need to build. So as I said earlier, the computers came in the 90s. The IT boom happened little in the 90s when a lot of engineering teams started doing engineering, but almost no design for 15 years. In the last four or five years, we have seen that the design has really taken an active participation in product design. So the problem is that there is a dearth of UX talent in the industry. And because there's a dearth of talent in the UX industry, there are no people going into the academics, and so vice versa. Because there are no one in the academics, nothing is coming out into the industry. So it's a big, big dangerous sign for UX as a brand in India. Less opportunity to get trained on the job because there are less talented people in the top when more and more people, like what Ajay said, I want to learn and I want to be mentored, what should I do? If there are no people in the top going to do the mentoring, how is the next generation of designers really going to do meaningful user experience design? And then the top management have seen a lot of this happening where they would say, OK, we want this product to look beautiful in compared to our competition. Get some designer to do it. And the moment a designer says, hey, yeah, I do a lot of UX because anything that I do is users see it, and that's why it's user centered. They get hired. They get paid a lot more, and that is kind of dangerous for the true designers who are sitting in this conference. So studios, UX departments in different organizations and startups, they are looking for right candidate at the right price. On the other hand, there are experts. There are mid-levels and freshers in the industry, and they want to really build strong careers with a good mentor so that together they can really grow. So this is what happens at huge designs. I just thought I should kind of share this story with you. In the last about two to three months, I've gone through 300 resumes. I was able to hire only one person. Running a design boutique is extremely important when companies like Honeywell, eBay, PayPal, Cisco come to us saying that, OK, why don't you? We have a design problem. We have a problem with our users. Why don't you solve them? If I have to put a designer in front of this customer, it is an extremely tedious job for me to select the talent. So this is a type of a quick cut at resumes that I receive. There are two resumes here. Which one do you think I should be worried most about? Any guesses? Which one? Wrong. The left one is what I need to be worried about. Because these are the resumes that I get. I do UI UX. I think users. I do IA persona because I can Google up this and put it in my resume. And that's where my problem starts. Because this is easy for me to quickly take it out. But this one is tough. This is what happens, right? OK, let's call in for the resumes. Give them some initial questionnaire. I give them a questionnaire so that I can see their design thinking what their passion really is. The design test, so maybe if the resume is OK, I say, OK, let me give him a design test. The design test goes well, and then the interview comes in. People feel the moment my resume is accepted, I think I'm going to get the job. But that's not what happens in the interview. The interview becomes much, much more difficult to get in. Some of the reasons why there is a higher rejection. Claim to be user experts and are overpaid. Individual contributors. Today in the industry, when you are a UX experience designer, you are expected to not only drive some of the interaction designers and visual designers, but drive some of the client team members, where you are going to anchor their design vision. It's not that, OK, a project manager will come to you and say, OK, why don't you design the screen? Gone are the days. The days are where the designer is expected to drive the entire engagement at the client location. Not much to show. They just show portfolios with a lot of static screens. They look beautiful because half of the thing is wide hero images with lovely photographs. It doesn't tell me anything about what is the design thinking that you had employed to solve the customer's problem. Both of these things are extremely important. Next time when you send resumes to UG designs, please make sure that any graphic that you put there is saying that please tell what was the business challenge that you had at hand, and how did you solve it? Misfit to the culture. Running a design boutique, it's important that all the designers, as we all said, it needs to be a team playing activity. They need to fit into the culture. How? We also spoke about, Satish mentioned, about ability to emotionally handle yourself. Most of the designers are very passionate. That's a great news, but the moment they are extremely emotional about their design, they fall in love with their design. They do not want to open it for debate. They do not want to open it for critique, and that's where the problem begins. Because the moment the same design goes to the customer and the customer kind of critiques it, you have no way to handle that situation. Our culture, as I said here, drive customer engagement and design leaders. We have a round of internal design reviews, and that the designer has to be able to handle it. And if he's not ready for it, it becomes a problem to our culture. So this is what I've made a comparison, and I'm going to quickly move and open all the slides and the bulleted points here. So this is what the industry is looking for. Someone who tells a story. I'm telling you this because I'm expecting when the people kind of apply to design agencies are able to deliver what is on the left-hand side and completely ignore what you have on the right-hand side. So candidates usually come and offer saying that, OK, wire frames. I wear the user's shoes, and they deliver wire frames. OK. They have skill in gathering requirements. That's not probably what we're looking for because there are business analysts, project managers doing that job. Designers, the skill is not about gathering requirements. Mostly single-handed and don't get into brainstorming sessions, they are really passionate of getting into the UX, but they do not know how. On the other hand, this is what the industry is looking for. Someone who tells a success story. This is what I did for my customer, and this is what the business impact it had as an ROI. I'm going to skip some of this. The third bulleted point. This is how it was before, and this is how I had kind of made as an impact, as a new design. Able to talk design, not just meaning, but the application of terminologies. One of the person came in and said, he's an FT UX. I was not sure what FT UX means. And I asked him, what was it? He said, first time user experience designer. So what happens is we are diluting our own community, and I think we should not do that. People said, OK, when we say, OK, you have done information architecture. Tell me more about what is information architecture. And they say, this is page level navigation. We are diluting our own value. The moment people do Google, quickly search for keywords and paste it in their resumes, they don't know how to handle that in the last session of the interviews. What really matters is the work quality, and not the number of years that the designer has spent in the industry. This is the number one bullet that I really, really miss when people come and talk. They don't listen. So one important thing about any good designers to really listen. Listen to the customer, because the customer is telling his problems, and he really, genuinely wants you to solve his problems. So do we have listening skills? Once you listen, you also think you don't have to respond immediately. Sorry. And then ask probing questions. Tell me more about it. Can be curious about what the customer really is going through, so deeper probes. Then listen again. Think, and then respond. This is one skill I see kind of missing, because designers are very passionate about their work, good. But the moment the customer opens his mouth, starts to tell the problem, the designer is ready with the solution already. That's not the thing, because we are solving design problems that are much more beyond one screen or a button or a two. I spoke about open the designs for review. Thinkers first. I think this is another thing. Designers are creators who they create using different tools as they sketch. So they get into a doing mode very, very quickly. They get a problem, and they start scribbling a wireframe without having to think about, OK, what is the problem that they are going to solve. So I'm urging all of you guys to really say, OK, whenever there is a problem that comes to you, take a minute or two, maybe more. A minute or two is just a statement. But take some time to really think about what the problem that you're solving. Then bullet number one, where you think, contemplate, ask probing questions, get back, and then respond with a sketch. Design management, time and quality is equally important. We are under tremendous pressure as an industry to deliver. Most of the time you would have noticed that the management, the project manager or a product manager would say, hey, this screen should be done in five minutes. This is where we have to take a stand and say, hold on. This is not what design is all about. And you need to put him on a tight spot, saying that, OK, I have questions for you. I'm going to interview you. You tell me, answer my questions. In that case, I will be able to design and deliver. We usually get into a doing mode. We'll say, OK, let me create a wireframe quickly so that he's impressed and he'll be happy. You're actually increasing the problem because you have not understood the problem and you've created a wireframe for something that is not a problem. Asking a lot of questions, especially, because if you ask these questions, the CEO will love you.  The CEO is a human. All of us are trying to solve some problems. The CEO is not the answer to the problem. They do not know what the solution is. They know what the problem is. Talk to him, talk to her, of course, both of them. And try to understand what the problem is. And they are seeking solutions from you guys and not vice-versa. Typically, our culture is, someone has told me to solve a problem. That means he knows the answer. He is just going to validate it. So I'll create a sketch or a design and go back and validate it. And he'll say, oh, yes, that's a good design. That's not a good approach. He is coming to you. He or she is coming to you to solve the problem. One thing I've noticed, I've done so many interviews over the last six, seven years. No one asks a question. When you're attending an interview, please make sure that you have your questions ready. You have gone to Use Designs website. Find out flaws. Ask those questions. We will be more than happy if you kind of question the interviewer. So last point is extremely important. There are several occasions where people have come in. They've written all these jargons in their resume. They come extremely confident to the interview. And when we ask them questions, they are not able to ask questions back. I would expect all the candidates to come and ask questions. So that's what happens. Assume interview is not a UX expert. Most of the time, probably, they would have read on Google. They would have studied in their education institutes. And they may not really study the company whom they're going to interview with thoroughly well. They directly land on to the interview. Maybe they don't have time for preparation. But they come there, assuming the other person may not know so much of UX, because I know UX. I've read a lot on Google. They go there and start talking about it. And when they start, they get questions from the interviewer. They have no answers. And that's where there is a chances that you've lost that battle. 50% of the battle is lost there. I've spoken about the visual portfolio. They're mostly responsive. I asked you a question, you responded. It's not participatory. I think designers need to start participating in a kind of dialogue, not a monologue, but a dialogue with the interviewer, saying that, OK, this is what it is. What do you think it is? If I say, OK, that's a wrong definition of information architecture, then it's important that this guy says, instead of that, he could say, OK, maybe I think my knowledge is limited. I don't know. Tell me more. That way I know that you're open to learning more, and possibly then there is an opportunity to mentor. I've mentioned that, and I've mentioned that too. Seeking a job, as we discovered in the morning, most of them come because UX is a paying job. They do a lot of HTML. They want to really grow. I understand one of the participants right now also asked me that there's a visual designer, and he wants to get into the UX. How should he do the transition? It's not about going from visual design, stopping to do visual design, and getting into user experience design. It's not a department ship. It's not a silo. The visual designer could really show value as to how his visual design has an impact on the way user experience gets influenced. Today in our organizations, visual designers are able to influence in the way the wireframes are created. Usually the scene in the last four, five years has been that the wireframes are created by the interaction designer, passed on to the visual designer, and the visual designers are at the receiving end. Today there's a major shortage. More than what user experience designers we need, or the interaction designers that we need, there's a more dearth of visual designers in the market. So visual designers need to now think how visual aesthetics can also have an impact on user experience. It's not just about interaction design. Both go hand in hand. So ideas to reduce this gap. So all of these slides that you have seen are basically my thoughts that have contemplated and thought of jotting them down. None of these are googled up. These are my thoughts based on my experience so far. And this is something that I see as a community that we need to build. So some of these ideas are just my ideas. Feel free to add to these ideas at the end of the session or write to me in an email. So head of design UX practitioners to push UX process. I think if the bosses there, the UX process heads, are going to not take a stand and push the envelope, nothing is going to change for their middle managers or the people who are designers in the team. And if you see that your bosses are not pushing enough, you know what to do. Don't let sloppy practices come in. If you're going to skip a process or two in user experience design, that means you're sure that the output is going to be suboptimal. That means your product is going to be suboptimal in the market. And so the market share of the product would be suboptimal. And so would be your careers. So it's an interesting dynamic that is running under. And every day, every month goes by. And if you continue to do sloppy work, the output is going to be sloppy. Start reading, start talking about it. I think we may read on Google, but we don't talk. We assume, oh, now I've got the knowledge. But the more you talk, you will be able to prove your vocabulary about these terminologies and put it into practice. Invite folks to critique your designs. Very critical. It's an amazing experience when you invite designers to critique your own work. Because one, it breaks the ice that you're not going to fall in love with your own design. You're going to get a different perspective. And that perspective actually makes your design much more stronger. Extremely important, able to justify your designs. Your design, because you felt so, you read somewhere and that has got reflected into your designs. But if you're not able to justify it, why did you make the layout? Why did you make the design the way it is? Without any rationale, it's going to be subjective. It would be perceived as a subjective design. You have designed it yourself. So read on cognitive psychology, cognitive science. There are a lot of principles that come in extremely handy and they're not long paragraphs and complex text of research. They're one or two liners, usability laws that come in extremely handy for just while designing and then justifying your designs. A lot of us have thoughts here as well. And so why not take an occasion and put all these thoughts together and present it within your organization? It could be, again, your CEO or the CTO or your product manager, team up with the product managers and come up with some of the thoughts on your product and take a stand and present it to the management. They might see you completely different rather than an individual contributor. Teach at a design school, I think we spoke about that or help a startup. Maybe you can charge them a small fee. That will give you a feeling of being an entrepreneur. And good work always needs to get paid. Submit works for international competition. We again barely do that. We should start doing that. And overtake the product manager in some of the design decisions that will drive the product vision. Because designers do that design thinking. What Satish did. Satish mentioned about, OK, I wanted to push the envelope and I did. That's what we could do. We have a lot more thoughts. Why don't we do that? We always perceive ourselves as, OK, I'm a designer. I'll sit in front of Photoshop or XOR and create wireframes in Photoshop. And my job is done. Who am I to question? But start doing that and you will start seeing the difference in about 15 days. If you're working late hours, that means something is really wrong. In the last seven years that we have been running huge designs, we have been extremely meticulous about our methodology. People don't go home. Sorry. People go home by 5.36 maximum. And it tells us a story that if there is a methodology to it, you will be able to predict your work hours. You will be able to predict your deliverables to the customer. You will be able to manage the product project in a way that it needs to be.  And by the time you eye-trade, by the time you do the design reviews, it's 9 o'clock in the night. This is the last slide. So all the slides that you saw earlier was something that you can do at an individual level. But as a system together, these are my thoughts that we should do for a collective skill development in this country because there is a gap. The demand is going high and the supply is real. Really, really low. I want to really trigger this initiative where the students could come over to the industry. Knowledge session with customers. Irrespective, I'm sure you would have noticed this. And I have noticed it a lot. When you have sessions on user experience designed with your customers, they say, this is really good. This is really good. Now, can you deliver me your first wireframe? Can I see something tomorrow? Invariably, even customers are not educated. You need to still educate them. Irrespective, they come to a design studio to do good UX work. They will still not have as much respect for the time and the pain that it takes to create designs. They're invariably asked for immediate deliverables. So I think we need to do a lot more knowledge sessions with the customers so that they understand what the reality is because no one understands this whole giant called UX. Other than that, OK, that makes the life of the user easy. But how it is done, no one knows it. Sharpen this word. I think you need to sharpen us. We need to sharpen our swords for high-end skill development. As I said in the previous slides, there's virtual reality coming, augmented reality, bots, touch-free interfaces, voice interfaces. I think we are not even ready for that. And our knowledge of cognition is really low. Academics should definitely strengthen the initiatives of giving practical knowledge. There are internships which are month-long, three-months-long, but I think they are not enough. And the goal that is still given to them is deliverable-based. At the end of the project, why don't you submit wireframes and why don't you submit visual design? I think that needs a big shake-up. I feel we have a long way to go. But if you have any suggestions for me, please go ahead and write to me on that email. Thank you so much. As you talked about, I just want to know, I'm in a dilemma. I'm very new to this industry. What does it take on a UI UX designation which designers hold altogether? Yeah, that's a great point. I think there's nothing called as UI and then UX. It's just user experience, whether you design a user interface or design something else. I think we should, I would encourage everyone to just stop using this UI slash UX. UI usually came because a lot more people did HTML design. So they were doing user interfaces than design didn't even exist in the 90s and the early 2000s. So they started using UI and then they started Googling up and realizing that, okay, there's something called as user experience design. So they said, okay, now I do UI as well as UX. If you want to truly get recognized as UX expert, stop using, even if you were doing, stop using the words like HTML, JX, UI in your resumes, but mostly portrait, even if it is a small one month experience of UX, please elaborate what did you do in that UX experience? Okay, thanks a lot. Earlier, Satish talked about taking risk while hiring. And say in an interview, you realize that there is someone who does not know half the stuff which you talked about, but he is high on learnability. He's ready to learn and he accepts that, you know, I do know that I don't know half of the stuff, but hire me, I'll learn on board and I'll do my best. So what would be your... I think it's a great question. I see this at least once or twice a week where the resumes are really not reflecting the true essence of UX and that's not his or her fault at all because in the last 20 years, they have grown, they have taken their careers from let's say visual design to doing bit of UX or HTML to doing bit of VD and then realize UX is also paying up, but the salaries have gone extremely high and they can't take a salary cut and then join at a little more starting point for UX. And that becomes a big challenge for the, even if I want to help, I can't help. There were several occasions when the candidate said, I forget the salary, I want to join as an intern. We have done that, where then the passion really comes out, but it really depends. If you have kids, you have family, you can't really do that, but in that case, if someone is really able to absorb you in at that kind of a salary, I had examples where people were drawing for in lax and couldn't, the same thing, right? Couldn't answer the question, what is information architecture? It was a pity and that's why this slide deck that I've put together, because I really felt we are at a juncture where UX is going to crash if we don't do critical thinking as a mass. It's not just about me or you individually. We have to collectively think about how do we avoid this UX crash? More salary is going up, but not doing quality work. Invariably would be affecting the others in the market and overall UX brand will collapse. Invariably none of us will be doing quality work in the industry. Hi Prasad, my name is Ajay. I'm an UX designer and it was a really insightful presentation. It's a question and also sort of the dilemma that I also faced. I feel that your hiring criteria is a bit stricter. What I take away from the presentation is that it's your view on the other side of the consulting, like huge design. I just have one question. If you have faced it or you may face in the future, like you showed a number where you have 300 resumes and you end up selecting one, how would you respond in a situation where one of these 300s or few of these 300s are hired by the big ears that you work for and you eventually end up sending your one best designer to work for those people whom you have rejected?  I think that's a realistic situation. And it doesn't matter to me. At the end of the day, see, I think huge design started in 2009. We are still 30 designers. If I wanted, I could have become 100 to 100 designers in one single group by hiring everyone who comes to my agency and say, okay, let's do UX. So it's at the end of the day, my call to say, okay, what is the threshold of quality that I want to deliver? And especially when Fortune 100 companies like Honeywell comes to us and survives us for three years, invariably having 30 designers in their studio in Bangalore, I think it talks about my quality deliverable. So in fact, I have a question for you. So in terms of the timeline part or whatever, so I mean, most of the time, we really don't get to do up the basic logic even what to our expectations criteria, we don't get to do because of the budgets, even changing a small feature in a product will not be possible because it kind of accepts a testing phase. There is an issue in the main cost, everything, just resource utilization, et cetera, et cetera. So what is your message to the biggies in the company? Because we kind of really don't embark on UX, which I've kind of really seen in my career in the last 12 years. So kind of I really see there is a big gap which is kind of there where the biggies really don't understand our logics. So I think it's our failure that we are unable to convince them. As design leaders, if you have the right rationale, see if I can tell, and that's why I keep saying that, okay, the understanding of cognitive science, if we understand that, let's say if you have a flow of three, four screens on the phone or the tablet app that you're designing, and you know that this flow is suboptimal and it is going to affect the client's business, let's stop this, they will listen to you.  And that's what happens. I mean I'm just saying in a very, not like one or two projects when a big corporate's like that, so there are a huge number of projects which come in and it's like a big mode which will kind of work more like a factory mode or something like that. Our times to BRD and the SRS documents are the thing we just need to kind of get started. And we just get a very little minimum time to even to kind of do all the stuffs that we really want to do. In fact, your presentation was really fabulous. It's kind of really, I mean kind of spoke what we internally went through like all these days. So like that, so I'm just trying to kind of understand so do we need to do something about in a big way? Get a good UX champion, question your boss. If the boss is not right, that's why it is happening. Sorry about that. I mean, no, I mean just generally saying, I mean, this is happening on a day to day basis. I just want to kind of get your insight on that. I have seen when I'm consulting with other big firms and really big firms and I've met the UX designers there, so-called UX designers, they will talk, they'll talk, but they are unable to deliver. I had this director of user experience design for one of these top companies, I can't name them. They said, if these guys could do it from Pune in Bangalore, what are you guys doing? And these are actual quoted words except for that I can't take the name of the person. So I think if we leave it in, I think we will be able to deliver one day at a time. One year from now, I think we'll have more faces here saying that, oh, I think we started feeling for our own leadership versus being a football. That's why I think I had a football image somewhere because we are always football to what the product manager and the project manager ends up telling us to do and we do it with a feeling that whatever I'm going to create will change the world out there, but we lack the sword. We don't have the science. We don't have the rationale that we need to fight with. The moment you start fighting, they'll know that, oh, this is a bigger enemy to fight, enemy in a positive sense, and they will go on the back foot to kind of start respecting you that you are more powerful than what they're thinking is and that will start changing the scenario. Thank you, Prasad. Thank you. That was a great session. Please give a big round of applause.


All right, great. What's a good email? Is this your cell phone number? I'll stay in touch. Thanks for the conversation. Boom. That's the script. Okay. This is what I believe. I'm sharing that with you for nothing.  Where do you guys get your leads from? What leads you got? Well, now we're using Reminder. We have Remind over there, but Remind is a platform that gives us high mid self-sports. And we're just throwing that in the system. And we're trying to work with that. But other than that, what we also do is just get zip codes. And we just take the data and just call on the zip code. Hey, you know, I know you have a house on the market, but have you guys considered selling it at all? OK, well, for the right price. Perfect. Perfect. All right, so throw those scripts away, like put them in a trash can and light them on fire. Like, yeah, that would be the first step.  Would you consider selling it all that? OK, in my opinion, look, look, man, I'm I'm just telling you what I think. Is it right? Is it wrong? Do you like it? Whatever. I don't know the way that I approach it. And this is from trial and error of 100,000 calls. And all I did was call around zip codes. That's all I did. That's how I built my entire business. I didn't do expires. I would if I had to go back and do it all over again, I would have really hit expired.  When I call them, I'm going to say, Hey, this is Ricky Crew through Max of Orange Beach and all these scripts are on my site. You just go there and download them. But it's like Ricky Crew through Max of Orange Beach. How are you doing? I'm doing good, too. I'm enjoying the day. Isn't it gorgeous? Cool. Well, look, I don't want to take up too much of your time, but a house around the corner just sold from you. Didn't know if there's anything in the world I could do for you today. No. Cool. Was there an agent you would work with if you were to do something? No. Cool. Well, look, I'm sure at some point in the future, you're going to do something, right? Cool. I would love the opportunity to work with you in that day comes. Would it be OK if I stayed in touch with you? All right, great. What's a good email? Is this your cell phone number? I'll stay in touch. Thanks for the conversation. Boom. That's the script. OK, now from that script, that script is designed to create a natural organic conversation. You don't know which way it's going to go. And so you want to you want to really connect with these people. They start talking about the weather. Let's start talking about the weather. The weather part could be are you getting ready for Thanksgiving? How was your New Year's?  What's going on? So, you know, it's just something non-cellsy to make them feel comfortable and loosen them up. And then we don't want to take up too much of their time, a house around the corner sold. So we're not calling to try to get them to sell. And so what that does is they say, man, this is a different sounding real estate agent and they open up a little bit to you. And then when they open up to you now, you know, you have an opportunity to really connect with them. Now, if they're not interested in doing anything right now, we're going to get their contact information to build that personal brand with them through weekly emails and text messages and social media or whatever you do. But we're going to get that and then we're going to build that database.  And it's like today, if you eat a hamburger, you don't look any different, right? But if you eat a hamburger every day for a year, you look completely different, right?  But if you eat a salad every day for a year, you look completely different. It's the same thing with this system I have. The first day you do it, nothing changes in your business really. No real huge monumental breakthroughs or anything. But if you do it every day for a year, your business looks completely different. You're doing deals left and right. You got all kinds of people in your database. You're building personal brand with. Those people reemerge six months later and say, hey, I've been getting your stuff. I'm interested in doing this. I want to buy, I want to sell, whatever. You're going to run into people that want to do stuff. People start to recognize you as a low-pressure agent that really cares about them and they like that. And they say, if I ever do anything, I'm going to deal with that guy. He's not trying to sell me anything. He's just trying to help. So like I said, you may love that, hate that. I don't know. But I believe it's the way of the future is just helping people, building brand. And then they'll come to you. That's a big part of it. But also we're looking for deals now. And through this process, we're going to find deals now. And that's a two again, a reason why I might, if I had to go back in time, I would add expires to my repertoire and really hit them as how can I help you? Why do you think it didn't sell? What, the debt, the debt, and get in there and really see why they're selling and try to help them. Cause people are selling for a reason, not just to sell something. They're selling because something happened in their life. We got to find out what that reason is. Every time we got to find out what the reason, why are they buying, why are they selling? And then from there, we can really help them. You know what I mean? So it sounds like I like what you're doing. I would just need to know more about what you're doing.


Hello, it's Waylon Chow, and this is Contractual Terms and Defects, Module 2B, Part A. In this part of this module, we will look at the differences between a PUF, a representation, and a term of a contract, and then explain what is an express term of a contract versus an implied term of a contract, and look at how we go about interpreting express terms of a contract when there is a dispute as to the meaning of what is something that is written in a contract. In many contract negotiations, there is a lot of back and forth communications between the two parties involved, and those communications have certain various legal status. So the three different types are a PUF, a representation, and a term. We'll go back to our friends Daria and Jamal to help us understand what is a PUF versus a representation versus a term. So Daria asked Jamal again, do you want to buy my Apple laptop for $500? This time Jamal says maybe, tell me more about it. And then Daria says one of the following three statements. The first statement is, it's a great computer, you'll love it. The second statement is, it's got the LISMAC operating system, and the third statement is, I'll throw in my laser printer. With the first statement, that's considered to be a PUF. It's only mere sales words that Daria is using to try to convince Jamal to buy the computer. As a PUF, if that statement turns out to be false, if Jamal does not love the computer or it's not a great computer, there are no legal consequences. The contract is still a valid contract. The second statement, it's got the LISMAC operating system, would be considered to be a representation. It is a statement of fact, which induces the other party to enter into a contract. We talked about representations in the previous module. And we'd also mentioned in that module that if a representation or a statement of fact turns out to be false, it is a misrepresentation and the other party can sue for misrepresentation to say that the contract is voidable and can be canceled. In other words, that's called rescission. And the person can also sue for any damages that are suffered because of that misrepresentation. The third statement, I'll throw in my laser printer, is considered to be a term of the contract. It is an actual contractual promise. By Jamal promising to pay $500, Daria is promising to deliver the Apple laptop and the printer. It is a part of the agreement. Now, if that promise is not kept by Daria, then that is considered to be a breach of contract and Jamal could sue her for that breach of contract. There are two types of contractual terms, express terms and implied terms. Express terms are the ones that we know best. They're the ones that are either written on paper or they're stated verbally out loud by one of the parties. And both are considered to be terms when a reasonable person would believe that the statement was intended to create an enforceable obligation. In other words, they are terms, they're not representations and they're not puffs. Now with implied terms, those are terms of the contract. They're still part of the contract, but they're not written or even stated verbally, but instead they are somehow implied by law. Now, in what situations does the law imply terms? One or a couple of situations is where it's necessary to imply a term where it is an obvious consequence of the parties agreement, where it's something that is essential, that was missing in the agreement, but it's essential and an obvious consequence of the agreement we need to just automatically imply it, or it is required for the purpose of business efficacy. Another situation where the law would imply a term is to reflect standard industry practice and also based on common law rules for certain types of contracts, terms will be implied. One obvious example is with an employment contract. There is a common law implied term that to terminate an employee, they need to be given reasonable notice. And also terms can be implied by government legislation that identifies specific contracts and requires specific terms to be automatically implied for those contracts. One example is under the Sale of Goods Act, which requires terms regarding the quality of the items or the delivery of the items to be automatically implied into sales contracts. To get a better idea of when the law will imply a contractual term, let's have a look at this quick quiz question. At this time, please pause the video so that you have a chance to read the question and consider your answer. Okay, so the question says, even if a lease does not expressly say that the item leased must come back in the same condition in which it went out, subject to reasonable wear and tear, a court will imply such a term. Courts will do so on the basis that such a term, D is in accordance with the usual industry practice and the industry the parties are engaged in. Now, let's go through the other answers before we explain why D is the right answer. A says it was clearly intended by both parties. Since it was not expressly stated that the items would come back in the same condition, we can't say that it was clearly intended by both parties. Now, B was clearly desirable. Just because it's desirable is not a basis for implying a term into a contract. And C would improve the contract is also not enough to justify the implication of a term into the contract. We would need more than just mere improvement. The term would be the kind that's required to make the contract effective. And we skip to E is found in some leases just because it's found in some leases, but not many or all leases does not make it does not justify that such a term be implied. Now, D is the right answer, which is basically saying that this term is a part of industry practice in the leasing industry. So whenever items are leased out such as a car, it's normal in the industry to expect that that car be returned in the same condition that it was when it was originally sent out. But subject to reasonable wear and tear of that car, of course. Sometimes we need to interpret the meaning of express terms used in a contract. Remember that express terms are contractual terms that are either actually stated verbally or put into writing in a contract. And there are instances where if a word or a phrase or a sentence in a contract is written in a way that's not altogether clear and the two parties involved have a dispute as to the proper interpretation of that term. We need some legal rules to help us determine what is the appropriate interpretation of the express term. Now the general rule that the courts apply is how would a reasonable business person in the party's position have interpreted the express term. And that's a nice legal statement, but whenever we use the word reasonable, it's not a very precise term. So we need some more guidance to help us determine what precisely is reasonable in a particular situation. So the courts give us a bit more guidance. So there's something called the golden rule. The golden rule says that we should start with the plain meaning of the words. In other words, it's called the literal approach. We look at the literal plain meaning of the actual words used in the contract. And to determine that plain meaning, the most obvious place to look usually is the dictionary. We look up the common meaning of a word that's defined in the dictionary. So golden rule says that we start with that plain ordinary meaning. But if by applying that plain ordinary meaning, it leads to an absurd result, then we need to look at some other different interpretive approaches beyond the literal approach. Now these other interpretive approaches are the first is called the contextual approach. That's where we're looking beyond just what the definition is in the dictionary. We're looking at the actual parties' intentions and the surrounding circumstances to determine what did the parties intend these words to mean. A third approach is called the contra-proferendum rule. So that rule says that we apply the meaning of the word or the phrase that is least favorable to the party who drafted the contract. Or in other words, we interpret the words in favor of the party who did not draft the contract. So we look at those two other approaches, the contextual and contra-proferendum and also the literal approaches, which is what we did in the first step. And we go back to the question, how would a reasonable business person in the party's position have interpreted the express term?


A phenomenal and touching response all across Tamil Nadu, we are here at Tiruchinapalli. Over seventeen hundred kilometers we have traveled in less than three days. And in every village, every little town, people are waiting with great enthusiasm all by themselves. The wonderful thing is every individual trying to do their own thing to campaign for Rally for Rivers. And here we've been at Tiruchinapalli where Kaveri flows at its widest in Tamil Nadu. But unfortunately this year, this 2017, after Monsoons, it is running in a small trickle. There is no mile wide Kaveri that we used to see near Srirangam temple. The reflections of the temple cannot fall on the sands. We only see the shadows of the temple. Not the great lively reflection that we used to see in the mile wide Kaveri. There is just a trickle left. It is time to act. And there is tremendous enthusiasm in Tamil Nadu. In the Tamil Nadu government, he's also very enthusiastic as every other state is. And it is time that every Indian citizen, every Indian citizen who consumes water must rally for rivers. Now we are leaving Tiruchinapalli just enough time to take a bite of food and then on to Puducherry or Ponducherry as it's been known for a long time. It has been a French territory, a unique little niche in India. We will be in Puducherry for tomorrow's events and then we go to Mysore, more towards the source of Kaveri.


For the introduction, it's great to be here. Actually, Forstem was the very first conference I spoke at in 2013, back then in the GraphDef room, and that was the second time I really had to be back. It's always an event I really enjoy going and it's very nice to be here. So latency SLOs done right. To get you in the mood, I have a question for you all. You all have APIs that you manage or care about in some ways. So if I ask you the following question or your manager asks you the following question, like how many requests in January were served within 100 milliseconds? Of all the requests you got in January, it seems like a fairly basic question, right? You're monitoring it. So how would you actually do it? Can you answer this question already? What about 150 milliseconds? What about maybe 180 milliseconds? What about if you had a problem in June 16th, 2018 between 9.12 and 9.35? How many requests were served within 100 milliseconds? I guess like, who can actually answer that question for any of their APIs? Okay, that's great. That's like, can I have like someone tell me how they do it? Splunk, yeah, excellent. So you keep slunk data for more than half a year. Okay, I admire this. Okay, actually this kind of questions are examples of latency SLOs. And we have at Zirconos been able to answer these kinds of questions very convincingly. And at the latency SLO workshop at SRECON last year, I realized that it's actually for many people next to impossible to answer these questions. And I was like, okay, this was like really shocking to me like how hard that really is. So I thought, well, this is maybe a good topic for a talk. So what I wanna do here is basically give you a few ways how you can do that with the tools that you have. Like everybody uses for monitoring and log analytics. And then some ways, some pitfalls that are often arise and some misunderstandings and clear them up and also show some of our tooling that is also a largely open source, which might give you more insight into your APIs and allows you to answer questions like that very convincingly. So some thing more about me. My name is Heinrich. I'm data scientist at Zirconos. I originally come from mathematics and recently I talked a lot about statistics for engineers. So kind of being a mathematician, entering the IT operations, monitoring domain. This is what I was talking about like what percentiles are and this kind of things and how you apply them correctly. I moved to the countryside in Germany so that I can shop more firewood myself. And yeah, I'm Heinrich Hauptmann on Twitter if you wanna follow me around. Okay, here's the little plan. First, I wanna just talk a little bit about why you want to monitor latency. Then the second thing, if you look at the title, probably I should explain what an SLO actually is. And then I will give you three methods to calculate latency SLOs effectively. And we'll talk about each of those methods and in the end we will have a confusion. Isn't that wonderful? So yeah, without further ado, latency is important. I will just say so much. Forgot some props. Like this is the book that everybody who does SRE or deaf ops stuff reads. This is the Sightler-Ridpy engineering book from Google authors, Niall Murphy and others. And they had this four golden signals in them. They are page 60. Here they are, four golden signals. Latency is actually the first one. And it has latency, traffic, errors, and saturation. And those four metrics were later rearranged into the red monitoring methodology. So if you wanna monitor APIs, then recommended best practices actually do red, which is rate, errors, and duration. And duration is latency. So they took the four signals, then got rid of one of them, then rearranged them in a different way and made an acronym out of this. But it's still like the best thing I recommend everyone to do. And some, many people agree with me.  So I will actually not talk much more about it. So the second part I should explain is what is an SLO? Well, actually, if you're monitoring something, then you wanna have an idea of how should this value actually look like? So you have some metrics and what is the expectation? And SLOs are kind of a qualification of the service quality that you expect the service to have. And there's methodology also proposed in this book, which we have also seen in Richard Hartman's talk before me. This is this SLO, SLI, and SLA. Three acronyms which are used in this context. You start on your site, your service quality measurement by specifying certain service level indicator which very concisely measure the reliability or the quality of your service in a specific way. And then you have service level objective, which are basically a target value or a corridor of values you expect that SLIs to have throughout the duration of the, yeah, throughout the time. And the SLO is something that you might communicate to your users or internally. So you say something like I have a 99.9% uptime goal that would be an example of an SLO. But it's important to realize that service level objectives are something that take place in longer time spans. It's something that you manage stuff on. So you wanna do management decisions like am I, should I push out more features or should I be more conservative, spend more time in investing in reliability of my service? This kind of trade-offs are managed within SLOs and it's real in art and how to specify good SLOs. And then also SLAs, this is basically what happens if we don't meet our SLO and this is more a legal question. So I'm actually not going to talk much about it. Here's the first example, it's an availability SLO. So the SLI here is SSH into the target host and I spit out a one if it's working and I spit out a zero if it's not. And I do that every minute, put that into a metric. So that's a very clear service level indicator. And I have an SLO of a 99.9% uptime over the last month. So I take a month of data, I look how many ones did I have in that month and how many zeros did I measure. And if I have one more than 99.9% of the time then the SLO is met. And if I don't meet the SLO then you will get exactly one cake. That might be an SLA you follow. You can just put other incentives here but yeah, it's up to you what you're doing there. So here's an example of a latency SLO which was used in precisely this way in this SREcon workshop I attended last year. The SLI was the proportion of wallet requests that were served within a second. So this is a metric. You do that every minute. And then the SLO was 90% of the wallet requests in the past 28 days. So over 28 days, a long period of time were served within one second. And yeah, so you will recognize that kind of questions from the very start. This is basically the same thing. I ask you how many requests and we are now asking for a percentage. Yeah, an SLA was skipped here. And interestingly afterwards they showed you this data about the API. They showed you percentiles. And I guess like most of you will actually do percentile based monitoring. It's very common and it's actually what you're recommended. So the story with latency monitoring or let me just back up a little bit. The problem with latency monitoring is that the thing you are monitoring, the API, is inherently a thing that is event based. You just have tons of events which come in like maybe 10,000. And then you're trying to store them in a metric which is something you have just a single value which you're storing for that thing. So you basically have to compress a whole lot of information of 10,000 events and latencies in that period of time into a single number. The first thing everyone did was averages. Let's just do the average latency that we have. And then Doug Ulugang from OptimizeD wrote a very nice blog article which nailed the problem with this. And he said, well, measuring the average latency is like measuring the average temperature of the patients in a hospital. You don't really care about that. You care about the sick patients most and so you wanna focus on this.  So we do 90th percentile, 99th percentile, 99.9 percentile. And so this is basically very currently R. So everybody is doing this. And they had it on the slide and the question was, was the SLO met? So you have 28 days of service here, 90% of the valid requests in the past 28 days were served within one second. And they asked the audience, well, was the SLO met? And there was, looks right. There's, so there's 1K milliseconds, it's here. And, oh yeah, I should, sorry, this is actually, this should be 99%. But never mind. So you look at the 99th percentile, so you know 99% of all the requests were below the 99th percentile. And the question is where 99% of all the requests passed, served within one second. And they're attempting to look at the 1K millisecond mark here, this is, and the 99th percentile and say, well, we were below the one second mark 90% of the time. So, should be okay. You should be fine. But is that really true? So what if I told you that actually 99.9% of all the requests occurred there? We don't actually have the request count here, we don't know. And by the way, this is not how a real percentile metric looks. This is how a percentile metric looks. This is at least how our percentile metric looks. I don't know if you have stuff like this. This is a very, very well behaved, constantly loaded service. This is something, I have periods at night here where I don't have any requests, like nobody is using my service. So it's a poor service. And if I take the 99th percentile of no requests, it's missing data. And down here I have maybe five requests and the 90th percentile is just shit, it doesn't do anything. So if I look at this, I can actually, not at all plausibly tell you if 90% of my data was below that.  So that's the first realization, which is quite important. And it is actually not so easy to communicate. And I tried to make that clear, but I constantly have the discussion in which is basically it's a percentile aggregation problem, right? So SLO asks you, compute a percentile over one month of data or week of data, and then also across the service. It doesn't ask for is dub dub dub one doing so many requests. So I dub dub dub five. It asks to all web servers do or that serve the APIs do that. So you need to aggregate across multiple nodes, you need to aggregate across multiple weeks, and you just cannot do that with percentiles. And then people say, well, kinda you can maybe. And this is a reaction to my 2016 monitorama talk and John Rouser is a data scientist at Snapchat. And he said, well, if people like Hartman say that you can meaning can't really meaningfully aggregate sample percentiles, then yeah, I'm annoyed. Well, sometimes you actually can. And it went back and forth. I wrote him a letter. I said, John blah, blah, blah. And then he said, well, actually, I wrote a 10 page blog post in R showing you that you can actually aggregate percentiles. And yeah, so I have the link here. You can look all of this up. And it's actually a very beautiful post. I mean, he's really great in these planes. Okay, if you have multiple distributions and you sample from the same distribution and just average percentiles, then you will get true percentiles. This is kind of a statistical phenomenon. If you're sampling data from the same distribution, you will converge. Your statistics will converge. If you average them. However, in practice, you are not sampling data from a same distribution. So I don't really go into the details about what's on this chart. I will do later. But I took up some production data and I compared a one hour average percentile to a true 90th percentile over one hour. So one thing, I took all the data, computed the 90th percentile over the full data for one hour, or I took one minute percentiles and averaged them. And the results were 300% error. And the worst thing is usually it's fine. Like if you have five nodes which already do kind of the same work and you're averaging the P90th, you won't be not much off from the true P90. But you have two nodes, one blue and a red node. So the blue is doing a lot of work so you have a really nice distribution. It's very typical latency distribution here. And the second one doesn't really do a lot of work and it's P95 is down here. So the true P95 of the total will be pretty close to here because the red one isn't actually doing that much. This might be a failed service or something that just started up or you have a problem with a load balancer or something. So you have an in, like in not a good, your service isn't in a good state. And if you're doing the average, then okay, you have a 30% error or something. So it's substantial. And the real problem is that it works most of the time, but in the situation that you really care when something goes wonky, when a disk starts to fail, your aggregated percentiles will be terribly off. So yeah, I hope I convinced you that its percentiles are not really it. And to be completely fair, the Google guys didn't actually take a percentile metric as an SLI, but it's something else. They did a proportion of the valid requests which were served in one second. And this is precisely the first method on how you can do it right. So I will talk about three ways to do it right. The first one is log data, the second one is counter metrics, and the third one is histogram metrics. So if you have log data, if you have Splunk for half a year, then you can actually answer this kind of questions very easily. You can say, well, select everything from logs where the time is in some time box and the latency is below a threshold. And you would say, okay, you don't have Nesquil query interface for your logs. You might have another query language, but this is essentially what that is. You just put all your logs in a data store. You have a field for the latency and then you query that. And you can do that with log tools. And in this way are the original question. I will go back to that. How many requests were served in January within 180 milliseconds? Just go and count them. You have everything. So it's great if you can do that. It's correct, it's clean it easy. Problem is that you need to keep all your log data for a month, which can be very expensive. And every Splunk customer knows this. And the problem is not really that Splunk is ripping you off. That might be also the case, but it's not the main reason. It's that it's just a terrible lot of data for every request to have a log line, which is maybe 80 or 100 bytes. And you have to store all of this for every request. So if you have meaningful volume, like 1,000 requests a minute or 10,000 requests per minute, this is gigabytes and gigabytes a day. So it's just by design very expensive and very few people can actually afford to keep log data for very long. So the second thing is, this is called counter metrics. So it's also very simple. So you are interested now in the one second thresholds. I wanna know how many requests were served within one second. So I make a new metric and the common name for this is LT lower dash one second here for less than and just count how many requests were served faster than one second. Pretty easy. So I add a new metric, do that for each node. And the beautiful thing is now it has become mergeable. It has become aggregatable. So the percentiles, I couldn't do the aggregation. With the counters, I can just sum them and I can integrate them over time and this is how it will look. So I have here the in black, the total request count in red. I have the slow requests and then I can select the timeframe and I can here integrate the graph. So I will just sum up every line I have and I arrive at some numbers which are the, like I don't have a laser pointer with me but it's the two line graphs which come up high at the very end. So 8.9% slow requests. So for this API, my SLA was met. So 90% should be a fast. So latency SLOs by a counter metric, they're easy, they're correct, they're cost effective because you're not storing a lot of log data. They give you full flexibility in choosing aggregation intervals. Note that you can hear freely select time ranges and you can freely select like the number of nodes you're aggregating over. But you need to choose your latency thresholds upfront. You have the hard coded the one second there, you think here. And many people have that do that seriously actually do precisely this. They just hard code a bunch of latency thresholds and they do metrics for them. Cloudflare has examples where they do 1,000 thresholds for each API, not for all the APIs but for those that are monitored like this. So afterwards they can actually just select the right latency threshold they might be interested in. We have a technology called HDR histograms which allow you to do better. And the basic idea is that instead of storing the individual durations like here in a log form or something, we store the histogram representation of them. So we put them into bins and then we just count how many samples were in a bin. And then we apply one more trick. This would be very, very similar to what Cloudflare did with their 10,000 or 1,000 metrics. We just don't store stuff that doesn't have any samples in it. So here's a bin from 2,100 to 2,200 and we don't store that. So we use a sparse encoding for that. And this way we can get away with very, very low storage requirements and very broad range of data coverage. So we can basically cover the whole float range from 10 to the plus minus 128 with 46,000 bins and still just do 300 bytes per histogram. So it's a metric, but it stores much more than just a single value, it stores the full distribution. And so this is how it looks like. So for each point in time, we not only have one percentile or one average, we have the full histogram. And then we can go and aggregate it freely. And this is just I selected a timeframe and then I can just view the complete distribution. And then at the very end, I can do my latency as a low, very easy. I just go select something here and then I have 43 of 40 million total requests in the time span of a month. And I hover over this and I see 89.3% of my requests per faster. And this is pretty eye-opening if you have that kind of technology available. So here I have a demo. I select a date range, like say two weeks here. I do set and here's the updated thing. And I just can check, well, okay, here my latency below 27 millisecond was 69%. And I can do that on high volume APIs. So this are actually disk IO, block IO latencies over a month. So it's not a highly loaded system, but you see three different disks. They each do IO, block IO requests and I can see the latency distribution of all of them. So with this kind of technology, you can only monitor web APIs. You can do a low-level system APIs like SUS calls or block IO latencies and still do aggregate ASLOs on them. So it's not hard. So we actually have two commercial products, Suconos and R&DB. R&DB is a time series database which is fully histogram capable. And Suconos is a SAS monitoring tool. Both are as a full system, not open source, but they have a very generous free tier. So you're welcome to try. And the core technologies behind histograms are open sourced. So there are two open source histogram libraries, one developed by Giltini from ASL Systems, which is an early proponent of how to measure latency. Has an excellent talk on this. So if you're more into that problem of how to do actually subtle benchmarking of latency, you should definitely watch that. And he came up, which is HDR histogram name. And we have a very, very similar thing called lip circle hist, which we use in our product. So yeah, I just want to tell you, right, percentile metrics are not suitable for SLOs. If you want to take that away from here, if you just have percentiles, then it's not good enough to do your SLOs. You can actually, if you want, HDR histograms are just the ultimate answer that I know. And you should try it out if you want. If you don't have HDR histograms, the following thing is effective and can be done. It's not bad practice. So you store locked data usually for three to five days. Have that available. And there you can do the first method. You can freely choose your thresholds. You can experiment with this, determine what are sensible service levels for you. So is it one second, is it 100 milliseconds? What is my typical performance? This you can do in the first three to five days of data. And then you are adding the instrumentation for the counter metrics.  And then aggregate those metrics over weeks and months as needed with the tools you have.  But yeah, as I said, this is how it works. And this is actually how Prometheus histograms work. Many vendors have started to use the term histogram.  So with this HDR histogram technology, you don't need this upfront choice. You have 40,000 pins and you are just basically covering the whole range you ever want without any upfront choices. So yeah. So in my opinion, this is the best way to do arbitrary latency SLOs for high volume APIs or what have you. And here are some other correct ways to do it. But please be careful with the percentile metrics. They give you a good impression of how your API is doing. But actually doing math on them is very, very hard. They are not just a very kind statistics in a way that it's not easy to aggregate them. So with this, I'm going to close. Thank you very much for your attention. And I think we will have two minutes for questions or something, right? Please remain seated. Please remain seated during questions. Do we have a sound? Hello? Yeah. If you don't have full historiologs or full histograms, can you but adjust the usual volume counter and a regular sampling of the latency? Can you approximate something correct by multiplying the latency samples by the volume of request during the period? I don't know if I understand the question correctly. So the first question was about subsampling, right? If you have locked data and just take a portion of that. You have the usual metrology system that measures the number of requests. And then you have some kind of probe samplers that measures the latency everyone's in the wild. Okay, an external probe. Yeah. With external probes, so the question was, can I cook something up with having the total request count and the external probe latency? So the first thing I'm going to say, it's good to have external probe latency because the internally measured latency and the externally measured latency can be quite different. It's good to also have external probes. But the main problem with external probes is that they are usually not representative. You see, you have a multi-modal latency distribution. You're just picking one out. And usually you're not picking out a random one, which should be required to this kind of up sampling. For example, usually you just hit the homepage of your thing. This will always lie in cache. So this will be a very fast request. So you have to be very careful to properly probe, externally probe your API for this up sampling methods to become effective. So I have not seen that. And usually the external probing is drastically under sampling. So you do one sample per minute. And at least in this example as I showed, we are at least talking a hundred or a thousand samples a minute. So the errors will be pretty drastic in this cases. Yeah, I hope that answers the question. I feel like I haven't quite understood the difference between number two and number three, because in both cases you have a histogram with counters. And you can't really, if your threshold falls in the middle of a bucket, you still can't understand precisely how many requests fall. And HDR histograms predefined those buckets for you. But conceptually it seems similar to how histograms are implemented in Prometheus. So from a theoretical standpoint, they are very similar. We count how many things are in certain buckets or less than a certain value. The question is the catenality and the cost economics behind it. So for example, doing 1,000 bins is not a lot. It will only cover a little range. And the costs you are paying for it are actually already very high. If 1,000 metrics per API you are caring about and probably even multiply this by the number of nodes or endpoints you're monitoring. So it gets expensive very fast. With the HDR histograms you do sparse encoding. So if the bin is not hit, then we don't store this. And usually your latencies are bounded within a certain range. They are not taking one time a microsecond and then five minutes. So usually you have one or two orders of magnitude which allow you to dynamically choose the buckets you really wanna record. So it's a cost and a performance differential. So with the HDR histograms you don't have any choices. It's pretty cheap and it's as powerful as if you would do 46,000 metrics of the Prometheus style. But yes, in theory they're exactly the same. Yeah, you can view it as a clever compression which gives you a factor of 10,000 more efficiency. Any other questions? Theo's my boss, so I'll take that with a grain of salt. So for people who are using arbitrary bins, so in Prometheus you have to choose your bins, right, and everyone chooses wrong. Every time I choose I choose wrong later. How do you choose better? So if you don't have HDR histograms and you are using Prometheus, which most people are, how do you go about choosing a better set of bins to protect yourself going forward? I think that's a complicated question, but there are two hints. One, you have to think about where you want to aggregate your metrics. So one error that Google is known to make with their choices is that different surfaces choose different bins and in the end they want to aggregate across them and they can no longer do that. So you can only aggregate the counts if they have the same thresholds. So you have to first get an idea in the organization across which teams and which instrumentation, like service instrumentation, race, you are actually in the end wanting to aggregate. So once you have this, the other question is like, where should we all agree to put our bins? And there the only method that I can come up with is actually use log data for that. So if you have log data, you can just say select latency from logs where in the last three days or whatever and you can draw the histogram yourself. You can see all the modes with any Python tooling. It's very easy to build. You just do a select and you just draw a histogram of all the values to get from there. And then you will have a picture like this produced from log data and you can say, okay, where is my 90th percentile here? Where is my 95th percentile? And you can just take this latency thresholds and start working with them. And then if your API is evolving, you might want to change it later on, but yeah, then you have to change it and you have to wait a certain amount until you can again compute those SLOs. But this is actually from a practical standpoint, the only methodology I can think of which makes sense. More questions? I think that's not the case. And I thank you again for your attention and have a great conference.


Hello and Happy Halloween to everyone here on the channel today is that spooky day Halloween and today is a special Halloween episode here on torque drift big. Thank you for the game Des for hooking me up with some gold because we're gonna do a big Halloween crate opening Why not? It's Halloween. We're gonna open the spooky crates. So we're gonna buy as many as we can I believe we can get at least 20 to begin with but we're gonna be buying more because I'm sure we'll get gold Throughout and we'll buy some more to finish this off totally make sure you guys follow me on social media Let's get to buying some crates and see what we can get. Yep. We're gonna buy 10 We're gonna buy 10 more. So now we're a hundred Gold short of buying 10 more crates, but we should be able to get that if not We're gonna buy a pack of five.  Well, we got two two sponsored crates to open up first. We are opening them up for the Z So got some money and some tires. So let's Go with that. We are trying to pull some of those special Halloween items here for today So we're looking to get that pumpkin alloys Maybe the bonehead alloy or spider alloys because why not? We got a bunch of decals too that we keep getting because we're trying to get as many of the Halloween items as we can So let's start opening up the first of our original 20 We did we didn't have that before so I think we're gonna get some money off that. No, we didn't have that web We must have had a different size web spider and an eye. So we got some graphics some Special graphics that we can be using to make some custom deliveries even out of the Halloween time So let's open up Number two some more brush sets. I Do like all the different Halloween themed brush sets that giant jack-o-lantern. It's fragile though. So don't drop it so Trick or treat that's actually sweet.  So we're getting a lot of brush sets. We haven't pulled the wheels yet I really want to pull some of the wheels because it would be sweet to actually have those. I Believe we already have that brush set. So we'll take some take some money So I don't actually think you get gold from these Halloween crates. So Unfortunately, we won't probably live by five more we'll be able to do 25 create openings for this episode. Oh We just pulled a pro LS for The 350 so now we have pro LS for 350 the Halloween that Halloween brush set is sweet So we can go LS powered on the 350 if we wanted to Got some more candy brushes some candy stuff We're just taking as many of these things we can we have not pulled a rim set yet Which I'm kind of upset about because I really want to pull one of these three alloy sets And hopefully we can pull these sets for today. That is I like that I like the vomiting pumpkin the vomiting pumpkin is pretty awesome. So we got 10 more in our first original 20 that we bought some more Some more spider some pumpkins and all the letters are sweet to get so If you guys have gotten the custom Custom wheels and stuff like that. Let me know down in the comments section down below that is They have a lot of really good graphics in these crates a lot of really cool things to make some really dope liveries and You know, it's awesome that they're doing these Halloween and Christmas themed stuff and things of that nature Just pulled some brakes I believe we have all these so a little more a little more money no more gold though Unfortunately, no more gold another vomiting pumpkin and another thing so we're getting a lot of duplicate Brush sets, but that's given because there's a percentage of things of that nature We just pulled a stroked RB 26. I believe we already have that So that's gonna be a huge money increase for sure Because we already had that in the car. So we got four more in the original Pull we got some suspension and things of that nature Three more to go then we're gonna buy some more obviously because we still have some gold left and There we go. There is one of the custom alloys of bonehead alloy we did finally pull one of the but one of the you know custom alloys the 2.91% the bonehead alloy that's pretty sweet. I want the pumpkin alloy is the one I would really like to get Actually, no I stand corrected that spider alloy that looks sweet the fore spoke with the webbing Yeah, we're probably gonna be using that on the Z. So now we pulled the two of the three I would love to pull the pumpkin because it's a custom alloy of course or a What a carbon alloy whatever you want to call it, but unfortunately we've run out of crates so far so Let's go take a look at those parts real quick and take a look at the wheels Because I want to take a look those actually look Kind of really good 21s. Oh, they have a lot of adjustment they have a lot of Profile adjustment to maybe size alloy with having little issues with that so I Do like how the spider alloys look? reset that and then the cusp the Bone alloys I do like them, but I kind of like the spider alloys a little bit better We're gonna throw our gold ad vans back on for right now in the back We rock in the gold and oh we get up see we get up the you know up the profile on these because Gonna keep these up to a 12 kick them up to a 30. There we go So now I figured out how to actually do it on the spider alloys, so we'll do that after we open a few more crates because We can buy We can get some more gold do that first And then I'm gonna open some get some more of that so we can buy five more Actually, maybe maybe a couple more so by Yeah, only five more Halloween crates because we're five short of buying an extra crate So we got five more crates to open so hopefully you can maybe pull the pumpkin alloy I'm being optimistic to pull the the carbon rim set The pumpkin alloy, but you know, I'm definitely gonna try it. No, no, no luck thus far Unfortunately, we have not been able to pull that we got three more to go Getting kisses from the puppy over here We're pulling some more parts. Hello dog. Oh, so I Hope you guys are having a safe and fun Halloween tonight If you're going out and doing whatever make sure you guys are staying safe and having a good time We got one final crates open. We might get lucky Unfortunately no Whammy on the pumpkin alloys. Oh, we did pull two of them. So we are short, but Do we have something? That we don't need That we could sell and get gold Apparently I got parts coming in for that. So we do have some custom Underbodies we could sell from the Z Could sell the multi-pulse for a lot and be able to buy some More crates if we really want to be that greedy on that, but we've so be short a little more I don't think we have anything on the Z that we don't need That we can Sell for gold because that's really The things that you can sell for that are obviously neons and custom wheels and things of that nature nothing super crazy So what is obviously we don't have any more? So I think that will do it here for the special Spooky episode once again a big thank you to the game devs for giving me the gold to unbox a bunch of Halloween crates For this Halloween special could be doing a lot with that stuff gonna be making liveries as well And we're probably gonna probably gonna rock the spider wheels on the back now that I think about it. I feel like we should Rock those at least on the back Now that I figured out how to actually okay, so we're gonna put those there And then we're gonna go to and then we're gonna knock them down to Knock them down to a 19 Why can't I increase profile For some reason I won't let me do what I want it to do on that so Unfortunately, I will have to figure that out and adjust it I may have to bust out the controller to do it because I don't really want to rock some 20 ones in the back with You know Although that does look pretty sweet We could rock some big 20 ones We'll have to ponder this idea Let me know what you guys think of the spider wheels and like I said if you guys got those custom pumpkin wheels Let me know down in the comments down below So as always I think you guys coming back and watching I'm evil rabbit Hope you guys have a safe and happy Halloween and as always I'll see you guys on the track