Hello, everyone. I'm Carmen Dicuto, the Senior and Product Growth Marketing Manager at Amplitude. And I'm really excited to be with y'all today, discuss a topic that I am passionate about, how product adoption is a learning experience and what we can learn as PLG practitioners from the learning design field. Before I dig into why I am so passionate about this topic, though, I do want to give it an overview of what we will cover today, plus a little housekeeping. I'm going to set the stage for today's discussion by digging into the framework challenge. I'm sure most of us has faced as we have taken on newer roles like growth. I'll then transition into why learning design is a great fit for product-led growth user design and introduce the behavioral engineering model, walk you through how to do a BM analysis, and then end our time together by sharing resources. While there is not a Q&A scheduled at the end of this session, I will do my best to answer any questions in the chat. So please feel free to ask away there during this session. All right, let's dig in and start on why I'm passionate about this topic and why I'm even qualified to be speaking on this topic. There are a few things that you should know about me and why, again, I feel particularly qualified. And then first and foremost, I have spent nearly six years in the latest and greatest emerging go-to-market roles as startups. So from enablement to product marketing to growth marketing, and more than once, these roles were brand new to me and brand new to the organization that I was a part of. And so as someone who loves order and process and having proven frameworks, I spent a lot of my time in the early days of these roles trying to learn and find frameworks that existed that I could adopt as my own and the roles that I was fulfilling. And oftentimes I couldn't find what I needed either because it was potentially behind a paywall or it didn't exist yet. And so I often find myself kind of making up my own, which can be a little time intensive, especially since it's not proven yet. You don't have a lot of guidelines. You kind of have to figure out a lot of things as you go. And as somebody who has a degree in sustainable development, and I try to be as strategic as possible with my time, oftentimes I found that making these frameworks and the time that went into them wasn't necessarily a sustainable practice for me. And so I really reached the point and the challenge that I was facing that I'm sure at times many of you face as practice trainers and emerging roles is that we were expected to have a really fast impact and so much so that we can often overwork ourselves trying to deliver on outcomes that are expected of us or that we promised. And if we're having to find frameworks or build frameworks at the same time, we end up working a lot harder than we do end up working smarter. And so even though this is a very cliche saying, I do think over the last couple of years if we've learned anything and the way our professional and personal lives have merged and the way we manage our time is that we really do need to be working smarter and not harder. And one way to do this is really to not try to start with scratch with new frameworks or processes but rather in some form or the other recognize that all of the frameworks we need already exist. I actually had a colleague say this to me at one point and it was this major light bulb moment for me. You know, some of the frameworks that already exist have actually been built for the very specific problem that we're trying to solve for and will be a great fit. But others may exist to actually solve a different problem but they could actually overlap with the problem we're trying to solve for and work just as well for us. We just need to find the right one. Now, again, if you can't find an existing one for your specific needs, you can begin to look at other disciplines to see what frameworks you may be able to borrow or adopt. And we can start by asking questions like what disciplines share similar objectives to ours. And you can ask this question of yourself or you can do what I did and talk to someone well respected within your field. For me, that was John Cutler, a colleague of mine who's also a well-known thought leader in project management and product development. He helped point out to me something that was so glaringly obvious from my past work as an enablement manager that I couldn't believe I hadn't thought of it on my own. Cutler pointed me towards learning design. As an owner of the full customer lifecycle who has a lot of happy emphasis on activation, adoption and monetization, it was this major light bulb moment of of course this can work for me because when I think about the journey our users take and the things they need to learn and do in order to reach their aha moment and develop a sustainable habit with the products that I work with, I realized that learning design is actually a gold mine. So why is learning design a gold mine? Why is this something that we should be looking at as PLG product practitioners? Well, in case it's not as obvious to you as it is to me with my background in training and enablement, I want to quickly break down the why behind this and why there's so much alignment between product led user adoption and learning and design. In product led user design, we are thinking and designing for our end user and we're really trying to deliver values to them before we capture value in return. And we do this by creating experiences that are gonna motivate them and help them change their existing behaviors in order to drive adoption of our products and services. In learning design, you're more or less doing the exact same thing. As a learning design practitioner, we're thinking about ways they can deliver value for their end users, who are your learners and motivate them to change their habits and behaviors to adopt the new learning. So in this context, we have this common objective and the simple fact that users are learners of our products and learning design has frameworks available that are gonna save us a lot of time and money if we adopt them to figure out how we can view our users journey as a learner journey because adoption is truly learning. Now, this can be true, especially if you have a high tech or very technical product. So for those of you in B2B SaaS in particular, this may very well resonate with you. So now that I've outlined why, what the framework challenge we face is and why learning design is a great area for us to look at as product practitioners for designing for our end users, I actually wanna dig into one area in particular and that is the behavior engineering model. Now, the first iteration of formal introduction of the behavior engineering model, which I also call BEM was founded by Thomas Gilbert in 1978. Gilbert is known as the grandfather of human performance technology and the work he created was really the foundation for the learning design industry as we know it today. And he focused on six areas in particular that can create behavior gaps for learners and in our language for users. And of the six, there's two in particular that often get a lot of time and attention, as a learning design practitioner, oftentimes you get very focused on the knowledge and skills aspect and what you have to teach somebody around knowledge and skills in order to make them successful. But while there is no record of this statistic, there is this well-known, highly held belief that knowledge and skills only make up about 15 to maybe 20% of behavioral change while environmental factors can make up for more than or can make up to 85%. So for learning design, this meant that creating educational content with the sole goal of dissensing knowledge is insufficient. The entire experience of a learner must be taken into consideration as is with the users of our products. Now, as I mentioned, Gilbert builds the foundation, the foundational framework of behavior engineering model or BEM and there have been many iterations and improvements made since then. And there are definitely multiple options out there that you guys can all go look at and I'll share a couple additional resources at the end here, but there's one in particular that really stood out to me that I thought could be applicable for everybody who's working within users. And this book is written by Julie Dirksen and the book is called Design for How People Learn. In particular, I love how she positions the aspects of behavioral change as potential gaps and the questions she poses to uncover these gaps can be easily modified to better understand our end users. So let's go over those behavioral gaps. Again, she took the six that were made by Gilbert and modified them for today's age. So in her mind, the six behavioral gaps are knowledge, skills, motivation, habits, environment and communication. So let's start with the first modified behavioral gap and uncover the questions that we can ask around each of these gaps. So knowledge is the first one and you can think of the questions very simply as what do your users need to know to use your product and when do they need to know it and what format does it need to be delivered in? These are important foundational questions to answer, especially if your product is more technical or there's great variance in your user base. For example, with an analytics tool like the one Amplitude provides, there are knowledge requirements in order to set up amplitude correctly so that other users then can run accurate queries. If data is not properly instrumented, queries will not provide accurate results and important product decisions could be made with bad data or the wrong data and with low confidence. When we design our onboarding and overall end user experience, we have to think about who will be implementing our product, how they'll be implementing it and what do they need to do in order to do so correctly. Without this level of thought, you may find you have a lot of new users who never even complete the activation process of your product. The second is around skills. One way to think of skills as knowledge and action. What knowledge must your users put into practice in order to have the skill set to use your product correctly? And are those skills required similar to other tools or products they've used before or are there things unique to your product that they were going to need to practice potentially multiple times in order to be proficient? These questions as well as questions around knowledge can play a big role in how you design onboarding for new users as I previously mentioned. You know, you can think of things like, do you now send a new user to a demo environment first so they can begin to practice the skills that are needed in order to gain a deeper understanding of how to best utilize your product before they implement and adopt it themselves? Or can you provide quick end product tutorials to give people a deeper understanding of what is expected of them? The right way to ensure your end users have the skills they need will depend largely on who your users are and what your product is. Now again, it's important to remember that we put a lot of emphasis on knowledge and skills, but they may only affect up to 20% of the behavioral change required for true product adoption. As product practitioners focused on the end user, we then really need to dig into the next four to make sure we're addressing 100% of the behavioral gaps that hinder or promote it. So the third is gonna be around motivation. So motivation can vary greatly across your segment of users, which often means it overlooks because it seems so much harder to control for. However, you need to identify areas of resistance and get a sense of the user's attitude and get a sense of your user's attitude as a learner. While the first user product may be very motivated to adopt your product and very excited, the next user may be far more reluctant to do so, even if they're working for the same organization and on the same team. It wasn't perhaps that next user's decision to adopt the tool, but perhaps now it's a requirement for them to do so. As a result, their motivation level can be significantly lower. So much so that the only motivation they feel about adopting your product is now it's an actual job requirement and it can have negative impacts on their role if they don't. And when you add in other considerations like varying skill sets and the different roles somebody may play within their organization, the why behind adoption can vary just as much as their motivation. Companies that are attentive to the different motivational factors of their end users find creative solutions to drive adoption. Gamification has become a particularly popular tool to drive both knowledge and skill building, but also overcome the motivational gaps that may hinder somebody from actually getting on board with you. The fourth thing is about habits. One of the biggest blockers to learning something new is to unlearn what you already know. In every training or change management projects I have led, I have learned that if this gap is not addressed, it will sink the entire learning opportunity. In addition to identifying what they need to unlearn, you also need to identify what new habits they need to develop to truly adopt your product. For example, prior to creating amplitude analytics account, many product managers haven't had a dedicated product analytics tool and have been heavily reliant on their data teams to answer their product questions. But now with the self-service product analytics with amplitude, there are two components who want to take into consideration about our end user. One, they are unlearning their reliance upon the data team and two, they are learning how to use this self-service tool to answer our own product questions. We need to create an end user experience that doesn't just make it easy for them to run their own queries, but also that they must feel as confident that they can answer the question themselves as they were on the reliance upon the data team. The fifth gap is environmental. This category in particular can seem incredibly broad and intentionally is so because there's so many environmental factors at play that it can affect your learners across products, market segments and industries. The easiest way to think about this is what is needed to support success and what could be blocking success. This can look like materials, resources, technology or incentives depending on your users. For example, my primary product led focus has been on amplitude's analytic product. We have wide adoption of our tool across segments and industries, which means that with each customer needs to achieve, success can vary greatly. And at the most simplistic level, one team may have all the technical resources they need to drive successful adoption of amplitude while another team does not. We have to think about how to make both of these teams successful by building better in-product guides, guiding them to invite an engineer or another technical team member, providing easy to find documentation and behavior-based in-product notifications. We can think about the environment as everything's surrounding their product use and designed for a 360 degree experience. The last behavioral gap is communication. For some, this may seem like a stretch when applied to uncovering behavioral gaps for their end-user, but if your product is very technical or B2B sass, this is a gap that can be easily overlooked or one you can place entirely on your marketing or your CS teams. However, your end-users need to understand what the goals for using your product are. I was recently part of a discussion with our sales leaders around adopting a new tool for PQLs. As a growth leader, I can easily see the value and the why behind this new tool, but our sales leaders weren't as convinced about why the product helped them achieve their goals. The demos showed them the basics, but not why the tool was a better fit for meeting their needs than what they currently have. Now, as a growth leader, I could still just choose to move forward with a decision like this and buy this new tool, but until it's become clearly communicated what the goals of the new tool are, I will never see strong adoption. And I think a mistake that is commonly made by those designing new products and experiences is that they put the onus on the internal buyer of the new tool and your own CS team to help continuously communicate why the new tool has been purchased and how it's going to reach your goals. But if we really want to be disruptive designers, we should be understanding how we communicate and use your goals within the product itself. So now that we have an understanding of the behavioral gaps and the associated questions that you can use to dig into each of these, let's talk about an actual BEM analysis. We want to again begin by evaluating the user journey as a learner journey. We do this by first starting with a behavioral analysis, utilizing quantitative data. For some, this may be harder to come by depending on the available resources and tools you have, but ideally you will have a product analytics tool like amplitude that supports behavioral analysis. You'll want to understand who your customers are, what actions they're taking and what product journeys lead to retention and monetization and which ones don't. A good behavioral analysis doesn't just focus on what your users are doing, but also provides context behind why they're doing it. Once you have that foundation, then you can begin by doing more of the qualitative analysis. Even if you have great behavioral data that shows the why behind in user actions, there's also more context that can be uncovered. Customer interviews are ideal because you can dig into areas that can be hard to measure like motivation or other environmental elements. Some of these can also be uncovered in customer surveys or reviews or in community forums, whether it's one hosted by your own company or it's an outside community group. And the next area I dig into after completing and exhausting the options from the previous two is recorded customer interactions. I review customer support logs including calls, emails and support tickets. You can also uncover a great deal in sales calls as well, particularly if you're also paying careful attention to your competitors. Once you've evaluated the learner journey and gathered all the information you need, it's time to begin identifying the gaps in each of the behavioral gap categories. Once they're identified, you can begin designing and testing those designs against these gaps. How and what you design will be completely different for each product and user base. And so every solution is going to be unique. Every way you layer these different pieces and actions together is going to be unique for you. But regardless of what you're designing and testing, there are tools out there like amplitudes, experimentation and personalization tools that can help you deliver and iterate these designs so you can confidently close the learning gaps for your end users. Now, kind of wrapping everything up, I thank you so much for your time. If the concept of learning design frameworks as a way to drive user adoption has inspired you, here are some resources that I think are a great starting point. Again, I highly recommend Julie Dirksen's book as I have shared with you here today. But there are two additional resources you can dig into depending on your capacity and your interest. Agile performance improvements by Bob Winter was another one recommended to me by my colleague, John Cutler. There was more technical and dense read than Dirkens. And I apologize, I said two additional resources. I forgot I took one of them off of this slide here. These two books are your main and primary starting point and I highly recommend starting with these. Now I'm also available to you as a resource feel free to connect with me on LinkedIn. You can send me a LinkedIn message if you have questions or you can send me an email as well. I love engaging with people and I would love to share my passion for using learning design frameworks to support product and user design. Thank you so much for your time.


So, my name is Ryan Finley. I've been primarily a Ruby developer for a long time now, and I'm here to talk to you about the Elixir programming language today, but first a bit about my perspective so you know where I'm coming from. At work, we work on projects that last for many years, not many months, so I'm hesitant to adopt any tools that might be short-lived. We've got more active projects than developers, so I need strong standards and norms from the communities of the tools that we pick. You could say that I'm conservative about what tools I'll recommend our customers use. So Erlang. I've been dabbling in Erlang for almost as long as I've been using Ruby, but it never really felt like it fit my problems as nicely as Ruby did for me. I never really got any momentum with it, even though I had a few friends that really fell in love with it. OTP is a collection of middleware and libraries in Erlang, and using it is pretty much how you get things done in Erlang. Erlang was originally developed at the Ericsson Computer Science Laboratory and used to build telephone switches in the 80s. OTP stands for Open Telecom Platform, which isn't really relevant anymore, and it's a collection, I already said, middleware and libraries. Elixir is to Erlang what Scala or Clojure are to Java, if Java was already a functional language to begin with. It's a Ruby-like syntax that compiles down to Erlang's VM. You can freely use your Erlang code in your Elixir code, and from my perspective, inheriting the stability of the Erlang platform provides a solid foundation, and it's what caught my attention. So as a Rubyist in the Elixir community, it kind of feels like being at an expat bar. Not everybody came from exactly the same place as you, but everyone's basically speaking the same language you're used to, even though they might sound a little funny. They have their own mantras that will maybe scratch an itch for you if you've been in the Ruby and Rails world for a while. For example, instead of don't repeat yourself, they say no magic, and in my mind, those two kind of balance each other. Phoenix is the dominant web application framework, and they like to say Phoenix is not your app because Elixir has a standard folder layout, and Phoenix conforms to it. If you want to get started with Elixir, I highly recommend the little Elixir and OTP guidebook. It took me about four or five times for someone recommending it to finally buy it, and it's been great. My advice to you, if you want to get started with Elixir, and if it's your first functional programming language, would be that it might feel a little weird at first, but to try the new ways of thinking and to do things the way the community does them before you try to make yourself feel comfortable. There's some videos up here linked. I guess I'll find a way to share these slides. The first one, it really addresses that point that it's called giving up the object-oriented ghost, and she covers a lot of the initial reactions you may have to being in a functional environment. The others are really good, too, if you want to do a deeper dive. I do a lot with active records, so I thought leveling up with Ecto was really fun, and then the abstractions one was just sort of just deep thoughts. I thought that was good. So the title of this talk is object-oriented thinking, so enough about the background. On the left side, you'll see the definition of object-oriented according to Al and Kay, according to a wiki. On the right side, you'll see the definition of the Erlang worldview according to Joe Armstrong, according to Wikipedia. This is not all of its relevance to this short talk, so I removed a bunch of things to help us focus. You'll see two things are pretty much the same for our purposes. And once you do that, you see in the Ruby world, everything is an object. In the Erlang world, everything is a process. In the Ruby world, objects communicate by sending and receiving messages, and message passing is the only way for Erlang processes to interact. And objects have their own memory. Processes are strongly isolated and share no resources. So that's what I mean by object-oriented thinking. It's about message passing and private internal state. In Ruby, message passing means calling a method. But in Erlang, they have this thing called the actor model, where every process is uniquely identified by a name or a PID and has a mailbox. So you can communicate with any process by sending messages to its mailbox and you receive responses in your mailbox. Erlang added symmetric multi-processing support in 2006. So that means if you've got multiple cores, it automatically uses them all and you get that performance benefit like Apple more recently did with Grand Central Dispatch. But there's an added benefit in the Erlang world that there's something called distributed Erlang, where you can run Erlang on a bunch of boxes, and if it fits your model, you can scale that way. So all of your actors can live on different machines and you can just scale up pretty indefinitely for some use cases. So I'm going to show you a really contrived Ruby example of a chat application, it's the beginnings of a chat application. So all you can do is post the message. You can get your full history, you can reset your history, and then we have a private method that records the message with a timestamp. So this should make sense to, I think, most of the people in the room. And here's a version of it that you would probably get upset if you found in a code base, where I'm using method missing to look a bit more like the elixir code that's coming up and to put the emphasis on the messages that we're receiving.  So a problem for object-oriented programmers in a functional world is how do you preserve your state? And this one simple trick is how you do that. You'll see that the receive block here looks a lot like the method missing we had, there's three separate cases here that we handle. We handle the processing and response, but it's all wrapped in a loop that passes, so each call, each post message, get history, reset, invokes loop again recursively and passes in the modified history. In the Ruby world, if you did this, you'd get a stack overflow, but this is a completely valid approach in elixir because of the magic of tail recursion, and I've got a relevant XKCD here. So now that we've got this state in our infinite loop, how do we communicate with it to find out from elsewhere in our code what's been going on? The way we do that is we spawn a process. So I've got some example code here. Can you guys see that? It's tough. All right, I'm not sure what I can do now. Sorry, I guess I'll read what's going on. So this is IEX, it's just like IRB, but for elixir. On line number one, I'm spawning the process and receiving the PID into a variable. Lines two and three use the PID to send a message. Line four asks for the complete history, and then this is all asynchronous. So when you use send, it'll send something back, but that goes into your inbox, which in our case belongs to IEX. So flush is gonna read our inbox, dump it to the screen. So this is the contrived elixir example again. You see when you call get history, it calls send using the PID of the person that made the request. So that's how we got all that information. But then we had to call flush in order to be able to see that in this example. So this example, so with our previous example, you've got processes that can send or receive messages. But that alone is a little bit difficult to work with. We've got to keep track of our PIDs, everything's asynchronous, and we haven't handled errors or any other failures yet. So this is an example, probably also hard to read, sorry. Rewritten using an OTP template. So on line one, we call start link, which spawns the process. But also links it to the parent process, which I'll talk about more in a second. Two and three sends in the chat messages the same as before, but this time it looks a lot more like calling a method on an object or a class. And line four, we call get history again. But you'll notice there's no line five because everything's synchronous now. Because we can define it that way pretty simply, which I'll get to in a second. This, you'll notice, is much longer than the Ruby code. I think no magic sometimes means much more verbose. I'm gonna break this into sections and step through each. So the first here, we're saying use gen server. Gen server is generic OTP behavior. The advantage of using it over just spawning a process is that it'll include functionality for tracing and error reporting. And it'll also fit it into a supervision tree, which is how you build resilient code. It comes with restart strategies and a bunch of other stuff that is really interesting to read about if you're so inclined. The gen server design pattern also communicates our intentions with other developers reading the code or our future selves. Makes our code much more maintainable. And then on line four, you notice we have a name parameter here that looks like a Ruby instance variable. This is, lets us do sort of a singleton pattern where we don't need to keep track of the pit anymore. So the next section is the client API. We've got the start link, which we use to start the process and link to the parent process. Then we have our post message, get history and reset history just like before. You'll notice that lines 13 and 17 use call and lines 21 and 25 use cast. Your call is for synchronous methods that you want to return instantly and cast is for asynchronous that you don't care when it's done. So those, these API methods call our server callbacks that we also have to define. So this looks a lot like our receive block that looked a lot like our method missing block where we've got post message, get history and reset history. But they can, they're using the OTP sort of framework to either reply or not reply because they're synchronous or asynchronous. And this lets us, instead of one big receive block, we have more manageable small functions. You'll notice that these methods pass the state around using a variable called history the same way we did in our loop. So this is sort of housekeeping stuff at the bottom. GenServer has an init and a terminate that's a lot like set up and tear down in an object. And handleInfo is a genServer thing to help you track things down, debug. That concludes my tour of contrived OTP code. If you have any questions, I'll be happy to talk to you about it around the conference. Thanks.


You also want to get up your calculator, preferably your graphing calculator, but A calculator, get up your calculator, do that right now, please. And what we're going to be looking for today is a pattern, a unique but very, very important pattern. We're also going to be waiting for my calculator to actually turn on, there we go. First thing, make sure you're in degrees, press the mode button and make very certain you're in degrees. Today is the last day we're going to be in degrees. Next class, we're going to enter the wonderful world, so Friday we're going to enter the wonderful world of radians. Who's in physics? For the rest of the year, you're going to have to get into the habit of physics degrees, math 12, radians. Do not write physics 12 tests in radians, you'll cry, you'll be able to find any of your answers. Call that the Jordan-Sakilomal, right? And don't write this down.  I would like you if you would be so kind, find the sign of 40 degrees and give it to me to four decimal places, Amanda, that's my way of saying go to your calculator, get it out and type it in. What'd you get to four decimal places? Point 6427, and could you also find the sign of 140 degrees, please, and give that to me to four decimal places? Is it also point 6427? Really? Could you also find for me the sign of 220 degrees and give that to me to four decimal places, please? Call me silly, do you get the same decimal, point 6427, but a negative in front of it? And how about the sign of 320 degrees? I'm going to bet that you get negative point 6427, do you? Really quickly, find the cosine of 40, I'm not going to write it out, just type it in. Nicole, what do you get to four decimal places, cosine of 40, point 766 to four decimal places? Okay, so point 7666, find the cosine of 140, I'll bet you get negative point 7666, yes? Find the cosine of 220, I bet you get negative point 7666, yes? And I bet you, Holly, if you find the cosine of 320, get positive point 7666, Amanda, do you think I have all these memorized, or do you think there's some kind of a shortcut pattern that I know that I'm just quickly doing almost effortlessly in my head? These four angles all have something in common, let's try another one. How about, are you going to make it, Andrew? You sure? Not looking good. Okay. How about, oh, the sign of 50 degrees, Andrew, can you give that to me, the four decimal places, please, point 7, 50 degrees, point 7666. Can you find for me the sign of 130 degrees, point 7666, really? How about the sign of 230 degrees, negative the same?  Where would 40 degrees be, if I started going, it would be right about there, this would be 40 degrees. What about 140 degrees, well, 140 degrees would be, how far have I just gone, right there, 90? How much further do I want to go? There's 140 degrees, how big is this angle here, how much further did you say I had to go, so how big is this angle here, 220 degrees would be, 220 degrees would be in here. Think about it, how big is this angle here, 40, are we spotting a pattern, and Trevor, what's all the way around, how many degrees? All the way around, how many degrees, 360, so 320 is right about there, can you tell me how big this angle right here is, 40 degrees. We call that a reference angle and we say that 40 degrees, 140 degrees, 220 degrees and 320 degrees, they have the same reference angle, if they have the same reference angle, their trig values are the same decimals. Some negative and some positive and Ashley, I'll teach you a shortcut to figure that out in your head too, but for what it's worth, they'll have the same decimal values. The tangent of 40 will be the same as the tangent of 320, but they might be positive or negative, but the decimal will be the same, here's 50, that's 50 degrees, 130 degrees is in here somewhere, you know what, if I think about it, I think that's 50 degrees because it's 180 minus 130 would give me 50 left over, 230 degrees is right there and you know what, that's 50 degrees, 310 is right there and you know what, that's 50 degrees. The reference angle, by definition, the reference angle is always measured to the closest x-axis, it's never measured to the y-axis, in other words, the reference angle is either this one here or this one here, think about holding your arms up and if you have the same reference angle, Kirsten, if you have the same reference angle, then I guarantee your sine, cosine and tangent values will be the same decimals, it wants us to walk through that, we just did page 232, hugely important concept, the concept of a reference angle. In order to investigate pairs of angles with identical trig ratios, same decimals, we're going to introduce the concept of the reference angle and the reference angle is the acute angle formed between the terminal arm of the rotation angle and the x-axis, now Nathan, it's easier to draw, if you draw a graph and you put an angle there or there or there, the reference angle is going to be this one or this one or this one or this one, it's always measured to the x-axis, it's always measured to the x-axis, it's always measured to the closest x-axis, that's how it's defined. That's really the image I have in my head, when you saw me pausing for a half second, I was visualizing the graph, I was visualizing the angle and then I was dropping down to the closest x-axis and it was either going 180 minus or 180 plus or 0 minus or 360 minus, I was doing some basic math in my head. So there's the fancy schmancy definition, I'm not going to ask you that, I'm just going to ask you do you know what a reference angle is and find it. In each case it says sketch the rotation angle and then state the reference angle, so 120 degrees, well 120 degrees, Carson how far? How far have I gone right there? 90, how much further do I want to go? Now that's not the reference angle, 30 is not the reference angle, the reference angle is always closest x-axis, this is my reference angle, how big? 60, now I need, I don't want to write out reference angle is, so I have an abbreviation, this is my abbreviation, this is Amanda not an official one, so don't think everyone, you know if you're props next year in university you're going to use this, but I do, I call my reference angle capital R of theta, to me that stands for reference angle. Would you say it was, Carson? I agree, 243 degrees, B, ready Katie, here we go, how far? How far? Too far, so 243 somewhere in here and the reference angle is that one, how big? How can you calculate it? But there's no one rule, the rule is different in each quadrant, which is why I haven't given you a rule for calculating a reference angle, you got to think, how can you figure it out? How far? How far? 243, how can you figure it out? Can't you just do that? Yes? So how big? Katie, use your calculator if you have, but give me an answer, I'm going 63, yeah, 63 degrees, Roxanne 337, how far have I gone? How far? 180, 270, 360 would be too far, 337 would be right there, now the reference angle is this one, Roxanne how big? Because it's always measured to the closest x-axis, how can you calculate it? Can you tell me out loud? I think not quite, ready Roxanne? Look up, look up, look up, look up, look up, how far? No, no, look where I'm going, how far? Okay, how far is the angle they gave me? 337, what's left to get to 360? Because that's your reference angle, it's 360 minus 337, yes? In your head or on a calculator, what is it? I'm getting 23, yes? By the way, you want to be able to do most of these, or it's going to take you a long time. Ready Nicole? What angle did they give me? Okay, how far? 70, the reference angle is measured to the closest x-axis, what's the reference angle? Also, 70, in fact, the reason you never dealt with reference angles in earlier Soka Toa Trig was because all your angles are between 0 and 90, and by definition, if it's between 0 and 90, it's also its own reference angle, so you were using reference angles without realizing it. Okay, the only reason we have to introduce reference angles now is we're going beyond 90 degrees and we're going to combine this in a few minutes with x and y and r, and get some neat stuff. Jessica, what angle did they give me in E? Do you say negative? That means go this way. How far? Negative, how far? Negative 90, how far? Negative 180, negative 270 would be too far, so I'm right there. Reference angle is always measured to the closest x-axis. How big, and I'm going to tell you right now, reference angle is defined as always positive. How big is that angle there? 23? No, yes, no, yes, yeah, 20, yep, right, 180, negative 180, negative 203, how big is this space? The difference between them, 23 degrees. Yes, last one, ready Ashley? 537, you know what, I'm going to take a shortcut. How far? 360, and remember I said it was your nine times table is the each axis, so for 50, 45, right to here would be 540, I want to go 537, so 537 is going to be like right there, and the reference angle is this big. Oh, what did I say? How far to there was 540, and we're at 537? How big? Yes, what? Three degrees. Not only if I give you the original angle can I say, hey, tell me the reference angle, I can go backwards, I can tell you the reference angle and the quadrant and say, hey, what was our original rotation angle? Oh, example two. Reference angle is 25 degrees, we're in quadrant two. First it says do a sketch, and even if it didn't say do a sketch, I would do a sketch. Quadrant two means I'm here. Holly, how big is the reference angle that they gave me? This right here is 25 degrees, so if that's 25 degrees, here's my rotation angle. I think 155, isn't it? I think you got to carry a one, don't you? Yes? Matias, second one, what reference angle would they give me? What reference angle would they give me? Quadrant four is here, and the reference angle is that one, 60 degrees. So the question is, what's the rotation angle? How far all the way around? And what's my reference angle? Yes, okay, figuring this out. I can't give you a generic way to sketch it each time, look at what quadrant you're in each time, and you're going to be sometimes subtracting from 360, sometimes subtracting from 180, you're adding to 180, do a bit of arithmetic. You had 300, yes? Good, good, good. Eight degrees and quadrant three, so here's quadrant three, mute. Eight degrees is the reference angle, and Nicole, that means this is eight degrees, and they want the rotation angle, which is this whole thing here. How big? 188, 180, and eight more. 180 and eight more would be 188, Mr. Good, there we go. Andrew, reference angle of 39, oh, quadrant one, quadrant one, and 39 degrees, so quadrant one, I'll put my angle there, says the reference angle is 39 degrees, and oh jeez, I want the rotation angle. Ah, it's its own reference angle, which is nice, and quadrant one, that will always happen. Ooh, what about a reference angle of 90 degrees, and we're in between quadrant three and four. Now if we're in between quadrant three and four, I think what we're talking about, there's quadrant three, there's quadrant, oh, we're talking about that line right there, are we not? Shannon, how big is the angle? 270? Yes? Yes? reference angles. If you're having trouble with this, make sure you do the homework, make sure you do the homework, fundamentally key. Now, I told you when I did this little trick here, that reference angles have the same decimal value, which is how I was able to do my little predictions, but you'll notice Nathaniel, I was also able to tell you sometimes whether it was negative or positive in my head without having to think about it too much, and there is a great trick for that as well. Okay? Nicole, sine is what over what? Y over r. Cosine is what over what? X over r, and tangent is what over what? Y over x. What we're going to look at is how those values change from quadrant to quadrant. Now, r, we said was the radius. Remember, r was when we rotated the arm around, sorry for you guys, it's this way. When we rotated the arm around the origin, the radius was the length of that arm. That means, Sabrina, by definition, the radius is positive. It's a length. X and Y are not necessarily positive. It depends on which quadrant you're in. I'll show you what I mean. It says this, in quadrant one, sine, which is Y over r, in this quadrant, Y is positive because you're above zero, and r is positive. You have a positive divided by a positive, which is a positive. Over here, sine is what over what? Sine is what over what? Trevor? Y over r. Now, in quadrant two, which is right here, is your Y coordinate positive or negative, positive, and r is positive. You also have a positive divided by a positive. What is a positive divided by a positive always, Trevor? Yes. Ah, but in quadrant three, Katie, sine is what over what? Now, in quadrant three, you're down here. Is your Y coordinate positive or negative if you're down here? Ah, and r is positive. In fact, in quadrant three, it's going to be a negative number divided by a positive number. And do you know what a negative divided by a positive always ends up being? Negative. Sine is negative in quadrant three. Look up. Sine is negative in quadrant three. Ah. Oh, positive in quadrant two and quadrant one. Ah. What about in quadrant four? Well, here, Y is negative, Y over r. I also have negative divided by positive. Sine is negative in quadrant four, which it is. What about cosine? You said cosine was X over r, Nicole. Let's write that in the first quadrant, X over r. And in the first quadrant, that's a positive number divided by a positive number. Cosine will be positive between zero and 90. But in quadrant two, over here, I'm to the left of the Y axis, Nathaniel. Am I not? If I'm over here, you know what my X coordinate is going to be? Negative. In fact, cosine is going to be negative divided by a positive. What's a negative divided by a positive? Hey, got your calculator? Find the cosine of, let's see, an angle between 90 and 100. Find the cosine of 110. I guarantee it's negative. Yes? So is the cosine of 170? So is the cosine of 153.2? Anything between 90 and 180, cosine is going to be negative. What about in quadrant three? Well, in quadrant three, I'm still to the left of the X axis. Cosine is still going to be a negative divided by a positive. It's going to be negative. If you find the cosine of, oh heck, 222, it'll be negative. I don't know what the decimal is, but I know it'll be negative. What about in quadrant four? Well, in quadrant four, although my Ys are negative, I am to the right of the Y axis. My Xs aren't negative. What are my Xs? They're not negative. What are my Xs? Positive, which means I have a positive number divided by and the radius is always positive. Cosine is positive. Find the cosine of, oh 300 right now. Guarantee it's going to be a positive number. Is it Cassandra? Did you try it? You did? I don't think you did. Now, just because of that, I'm tempted to start lying and see if you actually will catch me rather than nodding. We'll find out. Nicole, what did you say tangent was? Tangent over here is Y over X. Now, this is going to change. In the first quadrant, both X and Y are positive. It's positive divided by positive. It's positive. In quadrant two, my Y coordinate is positive because I am above the X axis. What can you tell me about my X coordinate? It's negative. You know what? The tangent between 180 and 90, it's going to be negative. Between 90 and 180, it's going to be a negative tangent. In quadrant three, my Y coordinate because I'm below the X axis is negative. But my X coordinate is also negative. What's a negative divided by a negative? The tangent between 180 and 270, thank you Trevor for testing it, is positive. Try the tangent of 220 for example. It's going to be positive. What about in quadrant four? Well, here I'll have negative Y value and a positive X value. It's going to be negative. You have to memorize this. Well, no. I'm going to give you an easy cheat. You will memorize this, but we don't memorize this. We'll memorize a shortcut. Which one is positive in quadrant two? I'm going to put a big capital letter S right there. S4, sign. Oh, and cosecant as well because cosecant goes with sign. Which one is positive in quadrant three? I'm going to put a big capital letter T there. Which one is positive in quadrant four? Which one is positive in quadrant one? Did you say all of them? We use the cast rule. That's what I was doing at the beginning of class when I was rattling off whether they were negative or positive. I was simply envisioning the word cast and it was pretty easy to do in my head. Or if you want to start in quadrant one, there's all sorts of acronyms. All add sugar to coffee. All students take calculus. Arnold Schwarzenegger takes cocaine. Well, whatever. Whatever acronym you want to come up with. Sign values are positive in quadrants one and two. Cosine is positive in quadrant one and four. Tangent is positive where? What quadrants? One and three. If you know where they're positive, can you figure out what quadrants they're negative in? I'm going to skip that. But I'll certainly ask that. This can be memorized by the cast rule or add sugar to coffee or whatever acronym you want to. The reciprocal trig functions follow the same pattern. So where is cotangent positive? The same place tangent is positive. Where is secant positive? What does secant go with? Same place cosine is positive. I don't really memorize those. I think about those for one second. I match up the pairs and I go back to the cast rule. I'm not going to come up with a second rule. I don't need it. This was hugely useful 50 years ago. Still useful now, but even more useful 50 years ago because 50 years ago you didn't have calculators to do trig values for you. You had a table and you had a table of trig values from zero to 90 degrees and that was it. You had a table that listed the sine, the cosine and the tangent of zero degrees of one degrees of two degrees of three degrees of four degrees of five degrees all the way up to 90. And if they said to you find the sine of 140, here's what you did. By the way, it says rewrite as the same trig function of a positive acute angle. What that's saying is rewrite it as a reference angle. We always would start out with a little sketch. Kirsten, what angle did they give me? 140, which I think Kirsten is about there. Kirsten, can you tell me will the sine be positive or negative? How do we know? So what does that mean? Will the sine be positive or negative? So what did the cast rule tell us? The cast rule told us what would be positive in that quadrant. So what does the S stand for? What does the S stand for? Sine. So ready? Here we go. Will the sine be positive or negative? I already know that this answer is going to be positive. What's the reference angle? Kirsten, the reference angle is that angle right there. How big? 50? 180, 140. Let's go with carry the one once again. Yes, Amanda? 40. The sine of 140 is the same as the positive sine of 40. Try it on your calculator.  They are exactly the same and the sines match positive and negative b. Tangent of 323 degrees. Sketch, C, A, S, T. 323 degrees. Let's see. Shannon, how far? Shannon, how far? No, how far have I gone right now? 90. How far? Keep going. 180. How far? How far? 362. 323 in here. So first of all, can you tell me is my answer going to be negative or positive? Got that. Next, I want the reference angle. How big is the reference angle? Yes, that's the whole package now. Maybe I was going too slow. The tangent of 323 is the same as negative the tangent of 37. Check it. Try it. Yes, Andrew, you too. Yes? This is how we did it in the old days. Now, Mr. Duk, if our calculators will do this for us, what do we need? Reference angles are hugely useful for other reasons as well. This is only one of the areas that they were useful. Cosine of 235 degrees. You know what I'm going to do first? Ashley, I'm going to sketch. Oh, and I'm going to write the cast rule. C-A-S-T. By the way, you're going to be drawing this now probably on about 80% of the questions for the rest of this unit. So get good at it and quick at it. Ready, Ashley? Here we go. Here we go. Ashley, how far? How far? How far right to there? 270. Too far? 235. First thing, Ashley, can you tell me in this quadrant, will cosine be negative or positive? Did you say negative? Because if you did, you're right. If you didn't, say you did anyways and no one will know. The cosine of 235 degrees is the same as the negative cosine of, oh, oh, reference angle, this guy here. I think 235 minus 180. Huh? 235 minus 180? 55? Yes, I meant, Gary, use a calculator if you have to. You did? It's 180, right? How much past 180 did we go? So what was it, 55? Cosine of 235 degrees is the same as negative cosine of 55 degrees. This is what I was doing here, by the way. A reference angle of 50 degrees, I went 180 minus 50. 130, and I'm in this quadrant, sine is positive. I went 180 plus 50. Hey, that's 220. Oh, and in this quadrant, sine is negative. Sorry, 230. Hey, and this quadrant, sine is negative. I went 360 minus 50. Ah, 310. And this quadrant, sine is negative. That's all I was doing. And for what it's worth, you can do these in your head in theory. D. Ooh, negative. Okay, no problem, no problem, no problem. Amanda, you know what I'm going to do first here? For D? Is this sketch? Yep. In physics, my kids would say dull, right? Draw a little picture. Oh, and the cast rule. Now it's negative 105. Negative means go in the other direction. So here we go, here we go. Ready, Amanda? Ready, here we go. Here we go. How far? No, gotta be fussy. No, no, no, gotta be fussy. Negative 90. Yes, yes, yes, yes, yes, yes. Okay. Now, ready? Ready, ready, ready, ready? How far all, if I went all the way to there? Too far, because they want me to go to negative 105, which I'm going to argue is right about there. Now the reference angle is this one. Oh, first of all, can you tell me, is the sine going to be negative or positive in this quadrant? Negative.  What do you got? Yes. Now you can sit up straight and crowded. Secant of 358 degrees. I'm not going to panic yet. You don't know who first? Carson? Darn right. And right in the cast rule. Carson, how many degrees did they give me? You know what? That's going to be right there, right? Because what's all the way around? Oh, so can you tell me what the reference angle is? Okay, so that's two degrees. Now, this is going to be the same as, secant goes with which trig function? Secant goes with cos. Is cosine negative or positive in this quadrant? Positive. Is secant negative or positive in this quadrant? Positive. So positive, secant of two degrees. How would I find the secant of two degrees? One divided by the cosine of two degrees, because secant is the reciprocal of cosine. That's, by the way, that's why again, you don't have secant, cosecant, and cotangent buttons. You just go one divided by the trig function. Last one. Cassandra, what am I going to do first? Darn right. Now, maybe you're getting good enough. I think Shannon was, where you can start to say, hey, 107, there. Just past 90. How big is this angle? 73 is correct. Is it, right? Yeah, that is. Yeah. Oh, and you know what I forgot to put? My bad. C, A, S, T. Now, Cassandra, cot, that's for cotangent. What does cotangent go with? Which trig function? Okay, tangent, positive or negative in this quadrant. Ah, so it's going to be negative cotangent of 73 degrees. Yes? Carry the one. Woo-hoo. Skip number one. Skip number two. Three all. You want to reach the point where this becomes boring in a waste of your time. You have to be that good. So I'm going to be giving you a fair chunk, but these, if you get the hang of this, they go quick. So three all, four all, six all, seven. Yeah, skip eight.


Water professionals across the globe share a vision of a world in which water is wisely managed in an equitable and sustainable way. IWA Connect is the place where you can find other professionals, share information and work together to develop your ideas by creating your own IWA experience online. IWA Connect will help you easily find and engage with your peers worldwide. Join groups to collaborate and contribute to specialist discussions based on your expertise area, location and projects. Connect to share your insights, profile your work, learn about industry developments and innovate solutions. Grow your knowledge and network and this is only the beginning. IWA Connect inspiring change by connecting you to the global network of water professionals.


Jangan beritahu saya. Jangan beritahu saya? Jangan beritahu saya. Okey, satu langkah. Mari kita periksa dengan saya. Saya ada beberapa penyakit di sini. Banyak penyakit. Saya akan mempunyai... Okey, saya akan tinggalkan untuk orang lain. Jadi, pertanyaan. Ini bukan pertanyaan penyakit. Sep videos kita membolehnya, kita akan mem metebutkan penyakitinations. Jangan beritahu saya siapa pun Ya, ya, ya, of course, of course. So, so, so, when, when I speak about data engineers, I also include data, data ops. So that's like a sub branch of DevOps that, that does, does data stuff. So, okay. So, right. So, one sticker question. Who, can, can anyone tell me what is Kubeflow? Okay. Yes, that's right. Kind of. Okay. Sorry. Sorry. I'll just press it down. So, so, 2019 is going to be, the prediction will be that, it'll be the time when there'll be a large uptake from the AI side, the machine learning side on Kubernetes because we are facing a lot of problem with our structure and deploying apps. So, to answer the question what Kubeflow is, Kubeflow is actually the combination of Kubernetes and TensorFlow, right? So, it's basically a framework for doing machine learning in the cloud. So, but, but, we are not, Kubeflow is pretty new is at 0.4, I think 0.4 just released. So, what I'm talking about today are basically the basic components of Kubeflow. So, Kubeflow can think of it as a massive help chart with many sub help charts. So, so, so that's basically what Kubeflow is. And actually, Kubeflow is not using help, it's actually using Kesson app. So, some information about me, I'm, I'm data scientist at Onusby and before that I was doing computational biology in the domain of genomics. So, any researchers here? Oh, okay. Okay. So, so this is the setup I had when I was doing my postgraduate, right? So, we had a simple SGE cluster. So, there's a hate note for submission of jobs and basically you write scripts and they send it into the cluster to do work.  So, this is very different from me going into Onusby. So, I don't have that type of resource at my disposal. In fact, this is like my home setup, right? Just simple desktop. And at work, I was actually just, I was given actually something even worse. I was given like a very low-powered laptop to do machine learning. I was thinking, how am I going to do this? Right? Ya, no, this is my home one. That's the i9. So, so. The 10 call. Thankfully, Nvidia actually sponsored the titans because you can actually apply for it for, for using an academic license and actually only one of it belongs to me. The other titan actually belongs to another researcher that we just shared. Basically, I'm sharing this desktop. Ya. So, how do I get from this back to something like this? So, when I came in, I sort of knew that Kubernetes is going to be the solution. Because I have seniors that are in other companies that are using this. Specifically, people from human longevity. I'm not sure if anybody have heard of them. But that's how they do their training to you. So, actually, this is Tommy here. Ya. So, he's like evangelist for Kubernetes in Onusby. So, he's like telling everyone, okay, please use it. So, that's how I got to know about that. Okay. So, similarly, like in the picture of the SGE cluster, you have nodes, right? So, instead of compute nodes, these nodes do different things. They more or less serve web applications and maybe run some databases. But from the data scientist's point of view, we need to crunch numbers. We need to do compute. So, I'm looking at this as a compute resource. Okay. So, I knew nothing about Kubernetes. So, I basically went with Tommy for the TripCon at Shanghai. That was a pretty interesting affair. And I was actually pleasantly surprised that there's actually a whole track just for ML and data. So, I was actually shocked that there is a whole track. So, you can see for two days, it's back to back talks of using Kubernetes for machine learning. So, most of these is basically talking about how to actually bring from a desktop to production. So, this slide keeps coming up. No matter where I went, which haven't talked that happened, I see more of the machine learning talks happening. So, this paper is actually the hidden technical depth in machine learning. So, many people think that as a data scientist, I just have to write training code. But really, I have to do this whole set of things. And it's crazy amount of stuff. And the yellow dot is just that. So, this was actually an animation. So, it will be this and then many other things come up. And this is what I want to do. I want to be doing machine learning. I want to train, I want to run, and I get data and come back. So, this pre-standard. So, credits to Yorg, not honestly Yorg, but that there was another speaker which did this slide. So, I basically pillaged it. So, we would do this, but there's also this other slide that is a huge problem which is at inference time. How do you deploy it? So, this comes back to this image. So, there's a lot of depth of help here. But we also need Kubernetes here as well to train. So, I'm sharing with you what I've learnt from KubeCon. So, basically that same image now is has been one of the speakers actually broke this up into the three rows. So, you have the data scientist, you have the data engineer and the data ops. So, this is slightly different from DevOps. This guy basically has to understand enough data science to bring it into production. And that's actually DevOps. So, ML and we are supposed to be according to the speaker that basically Yorg. Species Traction Analysis and a bit of Model Monitoring is basically what data scientist should be focusing on. And the data ops are basically helping us with process management tools, monitoring data verification and collection.  So, you have configuration and basically resource management. And there's also this chit-chang which is serving. So, you can see how Kubernetes is everywhere in the chart. And you will see that Calum actually sits here. So, besides the division of labor, there comes this new term called data ops. You can call it AI ops or something like that. Not sure, whatever name you want to call it. So, this guy basically needs to know these two things. If you work long enough as data scientist, you will also pick up some of these. And whoever that has with you will pick up on the data scientist. So, I'll share with you. Q-Flow, from the very beginning when I mentioned, is actually a framework to try to cover this whole piece by each component. So, I would not recommend that you use this in Q-Flow in production yet, but we can take inspiration from its individual part. So, our style is the first, which is actually Jupyter Hub. So, who here knows what are notebooks? Okay, so, everybody is familiar with notebooks. Okay, there's different types of notebooks. There's Zeppelin notebooks, there's Jupyter notebooks. Basically, I think everybody just loves Jupyter notebooks and nobody really likes Zeppelin notebooks. So, we have to get back. But the thing with Jupyter notebook is that it's not really supported by most of the paid-for solutions. You have to run it on your own. And this is basically how we came up with this, right? So, Jupyter Hub came, when Jupyter Hub was first conceived, it's supposed to be running on just a single server. But now, there's this new help chart that's called Kubernetes Jupyter. So, basically, it's just Jupyter Hub on Kubernetes cluster. So, when you stop this chart, if you list the ports, you'll get basically a hard port and a proxy port. And each user will get its own container that's running its own Jupyter notebook. And there's something called Kube Spawner. So, you can actually specify the image. Honestly, we have two teams in data science, we have an e-commerce team and we have a logistic team. So, you imagine the types of software that we're using are very different. We can have a base image for data science but each logistic team will be using something entirely different from what we're using. So, maybe something like Cplex, but I know that's not... The last five, something similar. And we can actually store this in a registry and you can pull this and then you can spin up the containers. So, the good thing here is that this lets you customise your data science image. Who plays Kaggle here? No. So, if you've been using Kaggle recently, you'll see that actually, their kernels are getting more interesting. You can actually load save images as well. So, I'm guessing this actually runs on top of something that's like Kube Spawner. So, basically, when the pot is spun up, there's a persistent volume so that if it dies, at least my data will survive. And this is linked up to this. So, this is the base image and then I'm happily here doing all my data science stuff and I don't have to care about the server anymore. So, having a very good base image is important because not everything can be done in the notebook. You might need to access, I mean, not access actually. Nowadays, you pop forward into the container. You use catty. So, you will just go inside the container. So, you will treat it like a base. You will install all the software there. And this will also help you, this base image will help you get to a working state in the same day you join a company. It used to be like, you take maybe a week to set up your environment. But once you have a base image, one hour, that's all you need. So, something which I did not mention much is that there are GPU images. So, if you have a Mac, these images will not work. So, in the Qt spawner, you can actually specify a GPU resource. So, you can say on the machine where I have two titans, I can actually have two users spun up. So, that's the beauty of the thing. So, just like a step by step, you will pop forward to the proxy and then you will be greeted and you will sign up basically. And then here is like spawner options. You can fully customise this. This will go into the help chart basically when it's being deployed. So, you can specify the image and then maybe how many CPU what's the memory constraints and the GPU constraints. And then you can start your Jupyter notebook. Here, Jupyter notebook. So, if you want to switch, you just need to change the URL. Okay. So, we'll move on for analysis. So, we spend, as data scientist, we spend a lot of time here and there's actually some time that is spent here. Actually, a lot of time is spent here. So, feature extraction. So, we have to do some scrubbing of the data okay. So, once you've got this feature extraction you want to do some ETL transformation and that takes time. And you need to repeat this. It needs to be reproducible science. So, what's the best way to reproduce this? You need to write some sort of execution plan. You do the extraction, you do the transformation and then you do the load. There's nine from really long goal and even in the SGE classes you have the ability to link jobs together. So, if one job's finished, you're done. And then, there's also something like airflow. So, who heard about airflow? I'll give the sticker. Okay, pass it. So, what do we use airflow for mostly? You have to put your hands up. Quick, come on. Okay. It's airflow for automatic age running the jobs. Ya, ya. Awesome. Say again. Yes, scheduling the jobs.  But, when we are doing okay, feature extraction yes, you might want to do this over and over again if you have retraining tomorrow. But, there's something else which is hyperparameters search. So, if you've got a lot of parameters that you need to search through, is airflow the best tool that you have? Not really, right? And, only recently did airflow have support for Kubernetes. So, only in the newest release in 1.10 was there port operators. So, before that there was only docker operators. So, the only option before that to run airflow in a multi node machine I guess mostly on mesos marathon. No? No? Okay, sorry. Okay. So, I'm focusing here on the hyperparameters search part. So, you have a standard DAG. Okay, you've got feature extraction, you've got feature processing and then you've got training. And, you want to try the different parameters here. So, your DAG changes and this space that you're going to be exploring the combinations that you'll be exploring will be very huge. Right? You can do a random search but it will still be huge. You can do train and then you have to do the evaluation. So, each step can be containerized. Right? This helps with the DevOps as well because they don't want to know how your data science algorithm works. Right? And the data scientists are not the best. We are not trained usually as engineers. We are trained to get the job done. So, our code base is not really the best. Right? So, once you containerize, it's a black box. Right? It really truly becomes ML black box to further the stereotype but this is how it works. Okay? So, I'll talk more about job submission now. So, like, in the SGD cluster you do something like this. Right? You can write a very long batch script to do the submission but this is as light as it goes. If it queues up, you specify the amount of calls. You give it some variables and then you execute the script. Right? I did not include here that there will be positional argument that you can pass. Right? So, routinely I will do like 5,000 jobs when I was doing my PhD. Like, easily. Then I'll go I'll usually submit it on Friday and I'll come back on Monday a bit later. Right? Hopefully nothing goes wrong. Right? But it's pretty smooth. This is a very stable structure. Right? And you can if anything does go bad, you can you can actually check on the status of the job. What happened? Like, how did it feel? And things like that. So, so how am I going to replicate this in a in a Kubernetes environment? Right? So, so, I was desperately searching and then I found QtFlow but when I tried, I mean, so I was very excited when I went to QtFlow the founder actually said please don't use it for production yet because the founder himself is not ready for production. Right? But the QtFlow has this component called QtFlow Pipelines and it was just released but before that they were actually using Argo and in fact QtFlow Pipeline was based on Argo is still based on Argo. So, Argo is a workflow management. I think you can use it for CICD as well. So, there's GitOps. Like, if you want to do GitOps this is like the perfect thing. Right? So, so like I said workflow is just one component. There's actually four different components to it but I'll just be talking about the workflow. So, like Alex mentioned on CRD so Argo also has a customized resource definition, right? So, it's a workflow definition. So, so when you install Argo, you will have basically two ports running and that will be the UI port. So, there's actually a interface um, better than actual. And, and there's a workflow controller. So, this is basically managing all the workflows that you'll be giving it. Okay. So, so, like, like QSAP, you will have Argo submit. So, Argo is actually a wrapper around cube control. So, cube counter, cube, whatever you want to call it. So, so, it will still follow cube control. That's why I'm on YouTube. So, so, um, you will have the name so you have to define the name, right? And, and basically here, there will be a job ID that's attached to that. Right? So, you can try different parameters. They will all have a different job ID but the prefix will remain the same. Right? And, and you can actually pass the argument. Right? So, okay, I'll show you how to do it later. Okay? And once the script is done, you can, you can actually get details about it using Argo Gap. And, you can actually look and visit the outputs of these jobs using Argo logs. Right? So, this is really, really useful. Um, okay. So, this is basically how a definition of Argo workflow is. So, um, you can actually write templates so in, in case you have nested, uh, what do you call that? nested workflows where we use certain components. Uh, and you see here there's actually a container image so you have to build the image. So, so previously when, uh, you can just submit a script into your SG cluster now you have to put the script inside the container. Right? The image. Sorry. And then you run the image with an entry point and then you have arguments. Here the arguments can be actually template and basically you can run these templates with different values. Right? These can be, the values can be written into the flow or it can be given to Argo at runtime. So, so you can like just Argo submit minus p for parameter and then give it the, the argument name and the value basically. So, so here actually I, I didn't know that there was no videos allowed in the videos or rather the presentations. So, I'm not sure how it works, but this basically how the DAG looks like. So, you start with the DAG name and the first that and these two actually carry out in parallel. So, you can actually spin two containers in this workflow. Right? So, how you do this is actually have a master structure. So, you have a sequential one so this happens first and these two happens sequentially. So, you can do this for any type to all parts of this. But, but personally when I first transition to on the speed from academia to industry, the worst component was actually serving and like, Vincent say I need a lot of support from that false site to get my machine learning models up. Okay? And it's like a double whammy like, okay, there's lots of potential and there's not enough manpower and basically I was at mercy of all the developers. Right? So, but going forward I think all developers not just devops should know how to write their own help charts. Right? So, with help tree if if the dream is true you can start writing in a scripting language it will be less it will be easier to sum up. Because lots of demo files don't look very nice. So, but I the scripting language will be in Luar. So, they actually announced it at keep calling us like, what? So, but the the person who said who announced it says that there's no stopping open source for building something like a python scripting layer over this it's just that Luar was chosen because it has a good integration will go. Okay? So, I like this chart this picture because like, okay, so this is the typical flow of a data scientist. So, you have algorithm design maybe if you are just pillaging from an existing paper you don't have to really do this. Then you have exploration, proof of concept and then you use some small data to validate this and then you validate this with production. Right? Maybe this will this will be a small set and this will be a big set. Right? And then here it says promise your DBA you will write select and no install update. Right? So, you need to basically ask for a lot of permissions to do all this stuff. Right? Because you need to draw data and everyone is very willing to provide you with this. I know people have been stuck in this cycle for a long time. Especially the bigger the company you get the more time you get stuck here. And you have to integrate with staging and then you have to prove that it works. I'm not sure how you prove your machine learning algorithm works in staging and I'm still trying to figure that out. Then you have but then you have to go through QA. Right? Provider QA or data science.  Right? And here be dragons in between. Right? And then finally you have money. Hopefully prove that your algorithm works and you belong in the company. Right? Else it's off plus. Okay? So, the infrastructure for serving these microservices will be managed by DevOps. But really we need to pick up the I mean we need to pick up the slack developers to write our own help charts. Right? And whether there will be a repository is another question to be said because if you are deploying a lot of microservices going through personally I don't feel the need for a CICD as long as the chart is installed. It's working. I don't want to see that again but different ideas.  Okay. So data scientist we have to do the help and then this is basically how we connect the help charts we need to connect with the feature part and I basically drew like there's a duo right? So we need two people to party. Right? The data scientist Maybe the data scientist is this one and the data office is this one. Right? We will change rules. Okay. So this is what I've gone through so far. There's another component here which is in model monitoring. So I'm using a lot of these traditional monitoring tools like data doc and hopefully graphana. Not graphana yet but not only we normally these are used to measure like CPU usage, memory usage, you know, connections but I'm using this to look at more business metrics like dashboard which is life. Right? And stream data. Okay. So while doing this I I learned that Alpine is not my best friend. As long as it doesn't lead to the C library. Ya. So I I found out that it's not good at dissolving the DNS. So I spent a lot of time building a deployment image but I found that it's not the best thing. And then Ya. I don't know. Okay. So extra slides. Actually I'm working on the recommendation systems. So this is a bit off-topic. So basically you have products and then collaborative models which are run on like batch. So this will be your workflows and then there will be ranking. Right? So you basically generate a series of recommended products and this is where the microservices will have to do the processing life. Right? Based on the features that come in life. So this will sit in the currently structure. Okay. Thank you. Ya. So do you use TensorFlow serving and microservice or GRPC? Yes. Using GRPC. So so you... Okay. So in the help charts there is the liveliness probe and the what's the other one? The readiness probe. Right? So TensorFlow serving doesn't have a rest endpoint to check that. So you have to use the GRPC TCP to check whether it's like. So but it cannot serve the model directly because the input needs to be transformed first. So it's usually a pack. So you have one flash and a TensorFlow serving. So the amazing thing that I found was that actually TensorFlow serving I didn't have to scale the pots. It's actually really fast. But that Okay. So back to this chart, right? So in Qflow the solution for serving is actually one more layer sitting on top of on top of charts. It's actually something called Seldar. So the input is kind of similar to TensorFlow serving. So, yeah. Is TensorFlow serving is that K-native serving? Like serverless? No. It has to be running. Right. That's actually GKE has something which is kind of serverless as well. So you can upload your models directly. Yeah, there's K-native. Yeah. Does that under the impression? I was just wondering. Okay. You said serving, I just thought it was K-native serving. Ah, okay. It's a different animal. Yeah. But the thing with TensorFlow serving was that if you have big models that have a lot of weights and needs to be initialised into memory, you need, like, I think Brem Koon suggested like a sidecar to hit it first to warm it up. I was the first user that uses, I worked more often and countered this because I'm testing it out so the first user will fail because it's still warming up with all the weights. So that's one problem with TensorFlow serving. Thank you, Masli. Very good.


Hello, and welcome to this screencast. In this screencast, we will calculate the center of mass for an example of a discrete point mass system. We want to find the center of mass of a system that has four point masses. Let's draw a picture of this to see what's happening. Our first mass, M1, weighs 10 pounds and is located at x1 equals zero. All of the weights of the point masses are multiples of 5 pounds, so we'll use two boxes to represent 10 pounds. Next, we have a mass 2, which is equal to 15 pounds, at x2 equals 2, so we'll draw three boxes to represent the 15 pounds. Much further to the right, at x3 equals 7, we have a 5 pound point mass. And then finally, at x equals 8, we have two boxes to represent a 10 pound point mass. We would like to find the center of mass of this system, so if we imagine that these boxes are placed on a teeter-totter like this that will tilt to one side or the other, then finding the center of mass means finding the balancing point of this system. So if the weight were distributed evenly, then the center of mass would be right in the middle of our teeter-totter at x equals 4. However, we have more weight on the left side of this system, so it doesn't seem like it would balance right now. Instead, it looks like the teeter-totter would tilt down on the left side, so even before we make any calculations, we can predict that x-bar, the center of mass where the teeter-totter will balance, is going to have to be less than 4. On the top right of the screen over here, we see the formula for calculating the center of mass. This formula is essentially calculating the weighted average of the point masses, and that weighted average is that balancing point, that center of mass that we're looking for. So first, we will evaluate the sum that we see in the numerator of this formula. For each point mass, m sub i, we multiply its weight by the location x sub i, then we add the results for the four point masses. So our first term is x1 equals 0 times m1 equals 10, then next we have 2 times 15, and then x equals 7 and 5 pounds, and then the last term is our x equals 8 times 10 pounds. And when we simplify this, we get 0 plus 30 plus 35 plus 80, and that numerator then is equal to 145. Next we'll evaluate the sum in the denominator of the formula. This sum is easier because it's just the adding the weights of the four point masses, and so the result is 10 plus 15 plus 5 plus 10, which simplifies to 40. To calculate the center of mass, we divide the 145 by 40, and then we're going to simplify that to get a decimal approximation, and the result is 3.625. Now if we go back to our sketch, we see that x bar equals 3.625 is just a little bit less than 4, and that matches our earlier prediction that we would have to move that balancing point to the left of x equals 4 for our teeter-tartar to balance. And so that gives us our final result, that our center of mass is at x equals 3.625. Thanks for watching.