Hi, I'm Zor. Welcome to Unisor Education. So let me continue with my favorite subject, construction problems in geometry, especially those related to circles, tangents, angles, whatever. They are very, very useful and I very much hope that you have already solved all these problems yourself before listening to this lecture. If you didn't, just stop listening to me and try to solve all of them yourself. They are easy and it's just the process of solving these problems really what's needed, much more than just listening to my solutions to this. Because the purpose is for you to develop your creativity, your ability to put different ends together and come up with a solution, not to listen to whatever the radio solution I provide. Well, at the same time if you have already solved, then by all means please listen to whatever I'm saying and just compare, I mean, there are different solutions to the same problems without any questions asked. Now, so let me just continue. This is construction number four lecture, and it has 10 problems as usually, my standard. All right, let's start. Construct a circle that contains a given point and is tangent to a given straight line at a given point. Okay. So you have line and a point and another point. And you have to have a circle which is tangent to the line at that particular point and also going through this line, through this point.  You have to build this circle which is tangential at point A to a line and going through B. Okay, this is one of the examples of using the locus, two locuses actually to locate one particular point because one quality which this particular point, the center of a circle which we're looking for, possesses is that if you draw a perpendicular, if you connect it to the point A, it will be perpendicular to the line. The radius to the point of tangency is perpendicular to the tangent itself which means that this center is supposed to lie on the perpendicular to a given line in a given point. So this perpendicular is something which we have to build first as the locus of all points from which we can use as a center of a tangential circle. At the same time, this same center is supposed to have another property. It's equidistant from A and B because these are both radiuses, right? Which means where is the locus of points equidistant from two given points? Well, it's a perpendicular bisector between them. So this is the sector, this is the segment and this particular line is a perpendicular bisector of the segment AB. So one locus and another locus. So this point is supposed to belong to both of them. So that's why it's on their intersection. So obviously, now when we have finished analysis of this problem, the construction itself is simple. Number one, we draw a perpendicular to our given line to the given point. Number two, we have this segment AB because B is also a given point and do the perpendicular bisecting. Now the intersection of these two locuses is the center of a circle which we are looking for. And the distance from A and from B is the radius. Knowing the center and the radius, we know the circle. Next, number two. Construct the circle tangential to both legs of the given angle if one point of tangency on one of the legs is given. So you have to draw a circle which is tangential to a given circle, not a given angle to both legs if one of the points of tangency is given. So you have an angle and you have this point. Now, let's think about this point, the center is every distance from two different legs of an angle. Now, why? Well, for all this reason, because it's tangent to this, so the radius to the point of tangency is perpendicular. So this is the radius, the distance from the point to, from a center to a leg. Now here, the same thing. From the radius to the tangent, it's perpendicular, so that's the distance. Now, so where is the locus of points equidistant from two different legs of an angle? Well, obviously it's angle bisector because any point here will be equidistant from both legs. Why? Because this is the hypotenuse. These are right angles. And since this is the bisector of the angle, these two angles are congruent to each other. So triangles are congruent, and that's why these casualties are congruent to each other. Okay, so this center is supposed to be on the bisector of this angle, which angle is given so we can draw a bisector. That's one locus. Another is, let's think about it, this point is located on, well, this point is supposed to be located on the perpendicular to this leg at this point. We have this point as a given, and this is a perpendicular, which means that this is another line, the perpendicular to this leg at this point where the center is supposed to lie. So we have two lines, the intersection of these two lines is a circle, a center of the circle we would like to construct. So we found the center, and we have the radius actually, so we have the whole circle. They are more easy as you see. Given two straight lines parallel to each other and point between them, lines and point between them. Construct the circle tangential to both lines and containing the given point. So we have to build a circle which is tangent to both lines and going through this point. Again, analysis. This center, since it's the center which is on equidistant from these two parallel lines, and again, why is it equidistant? Because this is, this radius to the point of tangency is perpendicular. This radius to the point of tangency is also perpendicular to tangent. Now, these are two parallel lines, which means this is basically a diameter. And no matter where this particular circle is located, this length is fixed and the center is in the point of this segment. So what we can do is we can say that this particular center is always in between these two lines on equal distance from it. So we have a mutual perpendicular to these parallel lines divided in half and draw a perpendicular bisector to this segment. So this is the line which is also parallel to these two and lies right in the middle between them. Everywhere on this line, we can put a center and a circle will be tangent to both parallel lines. Now, out of all these lines, we need only one, out of all these centers, we need the one whose circle actually going through this point. Now, how can we do that? Well, let's think about it. We basically know the radius right now, right? Since this is half of the distance between parallel lines. So we know this distance. So the center is supposed to be on this distance which we have determined on this distance from the given point. Now, where are all the points which are in the given distance from a given point? Obviously, this is a circle around this point. So wherever this circle of this radius, which is half of this distance between the parallel lines, wherever this circle intersects our midline between these two parallel lines, which is this and this two points, that's where the center of the circle which we are looking for is supposed to be located. So this is one of them. And this is another solution when it actually going through this line but on another side of a circle. So two solutions exist in this particular case. Analysis actually gave us one locus and another locus. One is a straight line, another is a circle around the given point of the radius which we have already determined. Intersection between these locuses give us the solution. Construct a tangent to a given circle such that it forms a given angle with a given straight line. Okay. So you have a circle, given circle. Now, we have to construct a tangent which has a given line, a given angle, this is an angle, okay, with a given line. So we have a circle, we have a line, and we have an angle. So we have to draw such a tangent which makes this particular angle we put to this one with the given line. And by the way, there may be more than one solution in this case, but let's talk about one particular solution. Okay. So we know that this particular angle is given, right? We also know, obviously, that this is perpendicular to the point of tangency, all right? Now, what can we say? Let's just choose any point on this line and using this angle, draw a line at this particular angle. It will not be tangent, obviously. But, however, what we can say is that the real tangent which we are looking for and this line which has exactly the same angle, these are parallel, right? Because these are equal congruent angles. So these are parallel, and this is transversal. If these angles are the same, lines are parallel. Now, what does it mean that the lines are parallel? It means that the perpendicular to one of them is perpendicular to another. So, since we have already built this line, we can always, from the center of the circle, draw a perpendicular to this constructed line. And wherever this perpendicular intersects the circle, is our tangency point for the tangent which we are looking for. So, as soon as we got this point, we draw a perpendicular in this point to the radius which exactly is the tangent which we are looking for. Well, now, why there are multiple solutions? Well, number one, this actually, this line, this radius which is perpendicular to this line, is intersecting circle in two points which means this is another tangent which also makes the same angle alpha with our given line and is also a tangent. On another side, we can have the line which makes an angle on another side. This would be alpha. And then we can draw a perpendicular to this and it also crosses our circle in two points which means we can again put one line and another line. Both are tangents and both are making angle alpha equal to this one with the given line. So, it looks like there are four solutions, at least. I don't think maybe there are more. It looks like only four. All right, so let's enter this. Let me make one quick point. In all these situations, I usually draw a circle or a line or a tangent or whatever else as already kind of ready for me to analyze. So, then I'm doing analysis and I'm saying, okay, this particular, let's say it's a center of a circle which we're looking for, is supposed to satisfy these properties. And then, when I have found that through the analysis that there are certain properties that this particular center is supposed to satisfy which means it's supposed to be on this local center that locals, then the real construction actually starts. Okay, I draw a locals, I draw another and then the crossing of these two lines or crossing of the circle in the line or whatever else I'm getting my answer. So, the whole problem solving actually is done in like two phases which I'm bunched together actually as one solution. First stage is analysis and then second stage is real construction. So, let me try maybe next problem to differentiate actually these two stages in more clarity, I would say. Okay, given a circle and a point outside it, construct a second from a point to a circle such that the cord formed by it was a congruent to a given segment. Okay, circle, point, second, and we want this particular segment to be congruent to some given segment.  So, how can we do this? Well, let's think about it. If we know the circle and we know AB because this is our A, obviously as you know, if you have a cord most likely you will need to put the radius perpendicular to the cord. And let's think about triangle KBO. What do you know about this triangle? Obviously you know the radius which is R and you know KB, which is one-half of A where A is our segment. So, you know the catatose and hypotenuse of this triangle. So, you know the triangle. You can construct it by hypotenuse and a catatose. This is one-half A and this is R. This gives you this particular segment, OK. The segment OK you built basically. You construct by constructing this particular triangle, right? Now, is it sufficient? Basically, yes. Because what you have to do really after that all you need is draw a circle concentric with the given one of a radius B. And now you see that MA is tangent to this small circle, right? So, this is analysis. I can find out the OK. I can then circle around O with the radius equal to B and I can draw a tangent to this smaller circle. All right, so this is analysis. Now, how the construction actually is doing? OK, here is the construction. Now, you don't know this line. We have to draw it, right? All we have is a circle, a point and a segment. So, what do I do first? First of all, using the radius, I know the radius of the circle since circle is given and half of this segment, which is half of A, I built a right triangle. Now, we all know how to do it, right? For instance, you can build the right angle. You cut half of A here. Then you have in the compass the R using this as a center. You basically cut the R and this is your B. OK, so you build this triangle so you know this segment. You take the compass, you take this segment, use it as a radius of a new circle here. Now, what's next? Next, I said we have to draw a tangent from a point to a circle. Now, you do remember how to do it. It's one of the previous tasks. You basically connect them together and divide it by half and use this as a circle. So, wherever it crosses, since this is the right angle, right angle is inscribed into this bigger circle and supported by half a circle, that's why it's right. And that's why we use OM as a diameter. We found the center of this diameter and drew a new circle and that's what makes the angle OKM the right angle. So, wherever this circle intersects this smaller circle, this is a point K. And Mk actually gives you the second we are looking for. That's the construction. So, first I did some kind of analysis, then using the analysis I construct the second which we are looking for. Construct a circle of a given radius tangential to a given straight line and containing a given point. A circle of a given radius tangential to a given straight line. OK, so you have a straight line. Now, let's consider that we have built this circle of a given radius and we also have a point. So, we have a line, we have a radius and we have a point. All right, so let's just think about where exactly this particular center of a circle which we are looking for is located. On one hand, since this distance is known, this is the radius, it means this center which we are looking for is supposed to be on the line parallel to the given line and on the distance equal to the radius, right? So, every point on this line can be a center of a circle of this radius which is tangential to the line. At the same time, this center is on a distance r from a given point which means it's supposed to be on a circle around that point of a radius, same radius r. So, wherever this line dotted line and this dotted circle cross each other, that's where our center is supposed to be located. So, this is an analysis. Now, the construction itself, all right, so this is the line, this is the point which is given and this is the radius. So, first I construct the perpendicular, I cut the radius, the perpendicular again or parallel to this line. And then, the same radius from here, I'm circling around my given point. So, these are two centers which can, two points which can serve as centers. So, from here, this distance is r and this distance is r which means that this is a tangential to the line and containing this point. Same thing with this, this distance is r and this distance is r. So, this is another circle which is also tangential to the line and going to the point given. That's it. Okay, given a circle and a straight line outside it, a circle and a straight line outside it, find a point on the line that the tangent from it to a circle has a given length. So, if you find this point, this is a tangent, it should have a given length. Well, obviously, you have to draw a radius to the point of the tangent, of the tangency and what you know is this is an r and this is the a. Okay, let's just wait until my phone stops ringing. So, this is the right triangle because this is a tangent. This is the radius to the point of tangency so they are perpendicular. The a, b, we know that's the given and the radius is given. So, the triangle is given and o, b is given, well, not given but it's basically constructable as well. So, you construct this triangle. Now, I'm going to a construction stage. You construct this triangle by two categories, r and a. This is a. This is hypotenuse. Now, having the hypotenuse, you basically draw a circle around o using this hypotenuse and the radius and wherever it crosses the line which is given that's where the point from which we should really draw a tangent to a circle is located. And from this point, the tangent, this one and this one, by the way, would have exactly the same lengths as well as this one and this one. So, we have two points and four different tangents which basically make up a solution to this problem. Instruct a triangle by an angle an altitude from the vertex of this angle. Instruct a triangle by the angle an altitude from this angle and another altitude from another. Now, what do we have here right now? So, we know the angle and we know two altitudes, vd and ce. We have to construct a triangle. By the way, I don't know, this is probably just my attempt to make some kind of a repetition of whatever we did before because this is just triangles there are no circles here. However, I found this problem and it might actually have a certain connection to circles. But anyway, let's just think about how to solve the problem. So, we know that there is an ec and vd two altitudes and we know the angle. Alright, so, basically, if you will take a look at the triangle e, b, c, it's the right triangle and we know two elements of this right triangle. The acute angle b and we know the characters e, c. So, in theory, we can always build triangle e, b, c knowing whatever we know. Angle and this altitude. Alright, so, let's just start building. So, for instance, we have built this particular triangle using this angle and the characters e, c. Now, what's next? Now, we have this altitude which means that... what can we say? So, we have to do this. Okay, we have the point c. We don't have point d. We have a distance between b and this line which means this line, if I will draw a circle, okay, here is the circle connection. Now, I understand why this is here. If you will draw a circle of the radius bd, center b, it will be tangential to ac, right? So, as far as our construction going on, so we can do is, from the b, as from the center, we built a circle of the radius equal to our second altitude. And, since cg is a tangent, now, and we already know how to do it, we built a tangent from c to this circle and that would be our point g. And that would be our point a. So, that's how we built it. First, we built this triangle by an angle and a catheter. Then, a circle around the vertex of the radius equal to the second altitude and then the tangent from this point c, which we have built in the very beginning as part of the ebc triangle, the tangent points g and continue to point a intersecting the continuation of ebc. So, that's how we get the ebc. I was wondering why we have triangles. Now, I understand why I put it there. Not just a repetition, okay? Given two points, construct a straight line such that perpendicular to this line from given equal to two given segments. Okay, let's say you're analyzing this problem. So, you have to find a line with these two distances given. So, you have this distance and you have this distance. So, the distance from a point to a line is along the perpendicular. So, these are two perpendiculars. They're parallel to each other, by the way. But in any case, it's also very useful here to have circles arranged because if I will have a circle from this is the center and this is the radius, it will be tangential to the line. Same as here, also tangential to the line. So, if we know these two radiuses basically these two circles, we can draw the circles and what do we have to do now? We have to do a tangent which is common to these two circles. So, we have to be able to draw a tangent to two circles. Okay, let's consider this problem separately. Now, let me forget about these circles and let's concentrate on this problem again. What if I will draw a line parallel to this line? Again, we're back to analysis. What we know right now is this is, if this is A and this is B, this is B minus A, right? So, what we know right now is that if I will draw a circle here of a radius B minus A, then my problem is much simpler. Now, I don't have to really draw a line tangential to two circles. I have to draw a line which is tangential to this smaller circle and originating in this point, at this point. Now, this is the problem which we have already solved many times, how to draw a tangent from a point to a circle. We know how to do this, right? You use this as a diameter, draw a circle around it and wherever it crosses, this is the right angle which means this is a tangent. Okay, so we know how to draw a tangent from a point to a circle. So, now we know how to approach this problem. First, you don't have this line yet. Now we're in the construction stage. Okay, so first, you calculate B minus A, the bigger minus smaller, and use this as a center. That's number one. Number two, so this is B minus A. Number two, you draw a tangent from this point to a circle. Okay? You know how to do that. What's now? Now, we can draw a perpendicular to the point of tangency. Now, this is B minus A. Then we cut A and we draw a parallel line here. So, since these are both perpendicular to the same line, these are parallel. This is also A. And this distance is A. This distance is B minus A plus A, which is also B. And that's what actually is needed. So, we draw a line which is on a certain distance A from the point A and B from the point B. And the way we do it is by drawing a circle, drawing a tangent and shifting the tangent down by certain lengths. Okay? And the last problem in this lecture construct a circle tangent to a given circle at a given point and containing a given point. So, there is a point on a circle. What we have to do is we have to draw another circle which is tangential to this circle and going through a point. Now, two circles are tangential to each other if they have only one common point. And we know that the perpendicular to this point where they touch each other is basically a tangent which is tangent to both of them. We solved that problem before. All right. So, what we do know is we know construct a tangent. Okay. So, basically having this point we know this line. All right. The center is supposed to be in continuation of the line which connects the center of a given circle and the point of tangency. At the same time the center is equidistant from these two points. Both are given points which means it's supposed to be in the perpendicular bisector of this segment. All right. So, one of the analysis the construction is so given a point on the circle and another point. So, what do we do? First, we draw this. Secondly, we do a perpendicular bisector to this and this is the center and this is the radius. That's it. That was my last problem Well, again, I hope it was educational. Don't forget that Unizor is the place where you can find these and many other very interesting problems. Solve problems yourself. That's very, very important before you listen to these lectures. And also parents and supervisors and the teachers who would like to control the educational process of their students are welcome at Unizor. It provides you the ability to enroll your students into particular programs check their score on exams which they should take and basically mark the particular lectures or programs whatever as completed or not completed depending on the results of the exam and try to make sure that your students are really going through the maximum possible score on every exam. It's very important. All right, good luck and thanks for your attention. See you next time.


I'm Steve Nunn, president and CEO of the Open Group. Welcome to Toolkit Tuesday, where we highlight the various components and leading experts of the Architects Toolkit, a collated portfolio of the most pertinent technology standards for enterprise architects. During the series, I'll be calling on a number of recognized experts who will bring their particular insights on how to most effectively use the various tools in the Architects Toolkit. We'll have a mix of interviews, panel sessions and pre-recorded presentations along the way. While all standards of the Open Group are designed so they can be adopted independently of one another, the greatest value for an organization can be derived when they're used in unison, that some of the parts should be greater than the whole. In the Architects Toolkit, we have collated a portfolio of the most pertinent ones for architects, together, all in one place. For most of these tools, certification from the Open Group is also available, so practitioners can demonstrate that they have the skills required and recruiters can take the guesswork out of the recruitment process, all backed up by our Open Badges program. AI Chat Bots An interesting idea I've come across recently is the development of AI bots to help us and our customers ranging from sales assistant bots, customer service bots to product inquiry bots. You know the sort of thing that you see on the bottom of a website that offers a Chat Now function and takes you to an automated service. These are all developed and trained on their own body of knowledge or AI corpus, and many of them have now added machine learning techniques to obtain feedback on the usefulness of their answers and to improve or continue learning, as us humans would say. But actually, are we creating lots of mini people, mini silos of knowledge, each based upon their own set of data? Don't we need to share that knowledge across the enterprise? Don't we need to share our learnings and improvements, whether that's human learnt or machine learnt? That join up clearly requires joined up thinking and potentially joined up data and processes. In other words, an architected approach for the integration of these AI entities. A new use case for us in EA potentially. Hello everyone and welcome to Toolkit Tuesday. I hope wherever you are in the world you're keeping safe and well and it's great to have you with us today, whatever time of day or night you're joining us. We appreciate your time. Do use the Chat Channel please to communicate with other attendees. We love to know where you're joining us from in particular. So please use that in any other comments and things you want to share with the attendees. On this session, we're going to be doing something a little different but there won't be an opportunity for live questions today. But if you do have a question that you'd like me to answer as I'm our main speaker today then please do submit it anyway in the Q&A channel. And to do that click on the three dots in the bottom right hand corner of your screen and you will see the opportunity to click on Q&A. If you submit something in there then we will do everything we can as long as you leave us your details. We'll do everything you can to get you an answer to your question. So remiss of me not to actually start today by thanking Paul Holman. Another great EA Minute Paul and I really want to thank you for all of those that you've contributed over the course of Season 1 and Season 2 so far. Look forward to more in the future. They're a great way to start our sessions and I know that they're very much appreciated by the attendees. So thank you Paul. So as I say today is a little different. This is our last one before we take our end of year break and more on that a bit later on. But I thought I'd take the opportunity to just do a little recap on some of the things that have been going on in the Open Group Enterprise Architects framework over the last few months. We've had some quite significant announcements over the course of 2022 many of which have been covered by a separate episode on Toolkit Tuesday but just to do a little recap I'll do a short version of some of the main points. So bear with me as I move from host to presenter mode here and let's see what we can do here. And away we go. So obviously like I said I'm not going to have an opportunity to go into great detail on any of these topics and many of them have been covered but it would be remiss of me of course not to start with our announcement in April of this year of the Togov Standard 10th edition representing a lot of work from members of the Open Group and input from Enterprise Architecture practitioners who are non-members as well in various ways we have of doing that. But we announced that in April this year and a reminder what is the Togov Standard unlikely that you're attending one of these and you haven't heard of the Togov Standard but just in case it's an Enterprise Architecture methodology and framework it's developed by practitioners for practitioners and is based on tried and trusted best practices and it's used by many of the world's leading organisations for their EA work and particularly we've seen a really significant use and stories of how beneficial it is for use in digital transformation and improving operational efficiency generally. So it's comprehensive, modular and configurable. So what do I mean by that? Well you'll hear more about the modular structure in particular in just a few minutes but it's configurable, you don't have to follow it slavishly it's a lengthy piece of work but what you can do and what we encourage is that you tailor it to the use in your organisation. So the focus is on the 10th edition, what's new? Well basically we've got updated modular structure and some extended content and what we've done is we focus more on what the market told us what you told us you wanted which was more better and topical guidance on how to deliver the best enterprise architecture that supports stakeholders and their organisation. So a little more on that, the modular structure. We've broken it down into fundamental content and Togaf series guides. So the fundamental content I'll say a little bit more in a moment but you'll be very familiar with that and the Togaf series guides are really where the how-to stuff comes in the guides on how to operate or how to use the Togaf standard in a particular environment for example if you're operating in an agile environment how to do that if you want to focus more on the business architecture side then we've got actually six new Togaf series guides on that topic and all of this is contained in the Togaf library along with a lot of enterprise architecture content that isn't actually part of the Togaf standard the white papers guides and other tools that we have and the Togaf library is part of the broader the Open Group library that you'll find on the homepage of our website www.opengroup.org So what do we have in the new edition? We have a number of different formats which is also a great step forward we have the HTML format, we have a PDF and we have the published books in groupings and you can buy everything or you can buy groupings but so those of you who like the books and there are a lot of those they sell very well, we still have those but there's also a lot of you who have said to us over the years it would be great to have this more digital format, more readily approachable format so that's what we focused on in the HTML version and when it comes to what's in there you'll see a lot of familiar things what you know and love about Togaf in previous iterations so the ADM, there of course, catalogs, diagrams, matrices all the things that you know and love from the previous versions of the Togaf standard but you'll also see how we've reorganised the fundamental content and basically there you have the ADM right in the centre there but we have the starting with of course the concepts the introduction and then we have the ADM in the middle and we have applying the ADM and ADM techniques and architecture content and enterprise architecture capability and governance now these are essentially the six parts of the fundamental content as we've reorganised it and as I said that's the fundamental content but also part of the 10th edition is this expanded domain specific guidance and we've got 20 new series guides general guidance, establishing the EA team agile EA, digital enterprise, business architecture, security architecture data and information architecture and reference models and methods and there's more information on all of these of course on our website but this is to give you an idea of the type of content that we've got in the new version if you're wondering how to get started this really is a great way this would be our recommended approach you'll find it on our website, in fact the link is right here on the slide and it's an introduction to the TAGF standard 10th edition and it really is a great place to start include some recommendations on how to use it but it also it's just it will be it's the go-to place to start if you're not one of the people who's been infamately intimately familiar with building the 10th edition so do look out for that white paper freely available on our website the digital edition I said something about that earlier so to say a little bit more about it you can click on any of these things on the screen depending on what you're interested in so you know get certified on the right hand side if that's your interest, practitioner guidance you know if you want to establish a team that's something we hear a lot about how do I actually build an EA team EA practice inside my organization how do I go about starting that wherever you want to start you can click on any of these in the digital edition and it'll take you to the appropriate place in the 10th edition we've also added an ability for you to provide feedback directly through the HTML version on the digital edition so look for that little icon on the side there and you can submit feedback and that will get taken into account along with the other suggestions that we get from surveys and from members of the forum that will get taken into consideration as we evolve the 10th edition and specifically the series guides going forward as I said the TOGAF standard 10th edition is part of a much larger EA content in the open group library and you'll find all sorts of things in there Archimate modelling language for example and the exchange file format that allows you to exchange models that are created in any tool that meets the specification for the exchange file format you can exchange those models there's really great stuff in there and guides, white papers if you're not familiar with it go and take a look and if you haven't looked for a while you might be surprised at how much new content there is in there looking ahead to some of the content we'll be obviously improving things as we go on we're not stopping here and the most of the activity is happening around the development of more series guides because that is as I say the guidance to how to that's what people seem to want to hear the fundamental content largely stable doesn't mean it will never change of course but it's largely stable and it's the series guides where most of the activity will happen going ahead so we were then able to announce so it was back in April we announced the availability of the standard and we said that certification would follow changes to the certification program would follow in the coming months and then in October we were able to announce the details of that program so we now have something called the TOGAF certification portfolio that includes certifications and credentials that are built both on the TOGAF standard 9.2 and the TOGAF standard 10th edition so it really is a blend here it's a portfolio of certification standard so TOGAF 10th edition certification in itself it doesn't necessarily exist and I'll explain what I mean by that in just a few seconds so this is what we had up to October this year we had TOGAF 9 certification TOGAF 9 foundation certification and TOGAF 9 certified foundation with a number of credentials business architecture, TOGAF essentials up to date knowledge and integrating risk and security this is what we now have having added and created our TOGAF certification portfolio so we've created three new certifications there down the bottom the business architecture foundation the enterprise architecture foundation and the enterprise architecture practitioner and we also have a number of other credentials that are that are kind of domain or interest specific we have ones for agile specialist digital specialist, apply practitioner and leader and that really depends there's much more information on our website but that really depends on what your interest is what your career path has been or is likely to be as to which of those are most relevant for you we've also introduced something here called the applied practitioner badge which is for those individuals who are certified and you have completed your learning studies with through an accredited training provider at the open group and these are basically these learning studies are basically practical exercises that allow you to demonstrate that you've understood the material and you can apply the material and it's a mandatory element in fact of all the accredited training for these new certifications so it gives an extra way of saying this is an applied practitioner credential an extra way of saying that you actually know how to apply the standard not just know it's content so that's something new and we may well add to that later big question we always get asked is I'm Tokyo 9 certified how do I get to the newer versions of the certification well the answer is we do have a bridge exam so it does provide an easier more straightforward route to certification under the new certifications if you are Tokyo 9 certified so if you are Tokyo 9 certified do look at the bridge approach you could also just take the new certifications from scratch that's obviously an option for anyone but if you are part way through Tokyo 9 certification don't worry carry on go through the rest of that certification and training and get certified and then look to the bridge to update to the latest versions if you so choose but you know Tokyo 9 certification will not be stopping we'll be continuing that for as long as there is any demand for it and there is considerable demand right now so something else that was new and announced at the end of October an update, minor update to the Archimate specification so we're now at Archimate 3.2 and really the change that was made there the changes that were made were in these four areas notations definitions the metamodel structure and some relationship derivations now I'm not going to go into detail on this the last episode of Toolkit Tuesday that we had two weeks ago was focused on what's new in this standard so if you're interested in that then do go to the recording of that and you can find out more you can also see more on the website but I'm flagging it up as something that those of you who do use Archimate and that's a growing number of you I know out there then you may be interested to know that there have been some some improvements made and minor modification to the standard the last thing I want to talk about today is something that was also announced at the end of October we had a big month then but very very exciting because it represents a lot of work over many years but we have been able to announce the IT5 standard version 3 delighted to do so it really is a great piece of work and makes a fundamental shift in the IT for IT standard and where it is focused so the IT for IT standard has traditionally been what we call the standard for running the business of IT well in today's world where digital is so important this has really been turned around to focus very much on the modern automated digital delivery model the core of the standard we have the product backbone now it's all about data driven approach it's still organised around the value streams with some terminology changing but basically we've really really upped the focus of the standard to be on digital so that you can really use it to manage digital products and services and it's a great piece of work as I say pretty much brand new at this point and you'll be hearing more about it including I hope a specific episode of this on Toolkit Tuesday early in 2023 but for now just a few ideas a few points about it as I say you'll still see the value streams and therefore the end-to-end value chain that we've had as part of IT for IT in the past but it's very much focused on optimising for digital we've still got the capability model and we still have model information and data and very very much data driven and automation, integration flow all of these things will be helped by following version 3 of the standard in fact when we announced it our current chair of the open group governing board Lars Rossen who has been really the architect of the IT for IT standard from day one said with version 3 we finally got to something that delivers on the vision that he and the early founders of the group had way back when and it really is a great step forward and I encourage you to look at it and a great thing is it's all documented in the ARCUMATE specification so those of you who are familiar with the ARCUMATE modeling language this will suit you down to the ground and it represents another example of how the open group we're pulling together are standards so that they are more obviously usable together which was obviously the topic of last times Toolkit Tuesday the portfolio of digital open standards so if you want to know more about that look for that too that's also another significant development this year but I haven't covered it in detail today because we've just covered it in the last episode but sticking with IT for IT for just a moment what's coming next is something called the IT for IT standard magnificent 7 use cases so the forum has identified 7 different use cases 7 different personas and what they might be interested in and how using the IT for IT standard can be of use to people in that situation they're very useful they're a bit of fun too and they will be coming very soon so watch out for those so complete with the new version of the standard the magnificent 7 use cases when they come and an update to the IT for IT certification program to cover version 3.0 of the standard there's a lot going on in the IT for IT standard world and encourage you to look for it it will absolutely be of great value to you no question so that is IT for my whistle stop tour of what's happened with the enterprise architects toolkit updates in the in the coming well over the last few months so I will stop sharing my screen and I will revert from presenter to host mode again so thank you for sticking with that it's a whistle stop whistle stop tour as I say some of those things have been covered you'll find episodes covering nearly all of those things on toolkit Tuesday you'll also find more information on the open group website so do feel free to go there all that remains for me to do today is to tell you that we are taking a end of year break for toolkit Tuesday so this episode December 13th will be the last for this year for 2022 but look for something new and fresh in on Tuesdays in January I won't say more than that for now but look for something different over the course of January and then toolkit Tuesday as you know it next episode like this we'll pick up again on February 14th so easy date to remember Valentine's Day February 14th 2023 will be the next toolkit Tuesday but we'll have some interesting stuff for you on the Tuesdays in January going forward one last thing to also put on your radar is our next the open groups next quarterly event will be a virtual event an entirely virtual event this time from January 30th through February 1st so a 3 day event and the main one of the continuing themes for next year is EA and sustainability and that will be part of the theme for our virtual event at the end of January so watch out for more on that from the open group that you haven't opted out of receiving the mails if you want to have more information on that event and others and we look forward to you joining us on that hopefully but also on the next toolkit Tuesday and in the meantime thank you to all our presenters that we've had so far in season 2 and again to Paul Herman for his EA minutes and a big thank you again to you most especially to attendees who take the time out of your day to join us we hope you find them valuable please leave some comments in the chat before you go today and I look forward to joining you joining us at a future event meanwhile have a great rest of the year and I hope that the end of 2022 and the start of 2023 are all that you would wish for yourself and your families and thank you for joining us I'm Steve Nunn this has been Toolkit Tuesday


All right. Well, I would like to begin this session and the goal here is to do a deep dive into the open telemetry metric system. I was asked to give this talk by the organizers and they suggested that it would be a good idea to have a conversation. A year ago we did this earlier on in our open telemetry project and had a similar presentation here where Liz from Honeycomb spoke with me for an hour about the Metrics API and it was really helpful to have outsiders and insiders talking with me about it as we went. So we're going to do that again and I have with me Sheldie and Justin. Sheldie is at Honeycomb and has agreed to be an eager participant in this session and then Justin is at New Relic and has been one of the contributors, especially working on our semantic conventions for the open telemetry system. Okay, I probably should introduce myself. I'm an engineer at LightStep. I've been working in observability for I think about 15 years before LightStep I was at Google and have been involved in like tracing metrics and logging for a long time. Okay, so the outline is there are three parts here. So we're going to start with what we were trying to achieve and what we're after in this project because we're not done. It's underway. And we'll talk about the sort of timeline we think we have. The bulk of this sort of slide deck is talking about data model and the things we discovered when we tried to combine a Prometheus system with a Stasti system with a tracing system, what happened. I'm going to talk about some of the ways you can configure this thing that we've built the open telemetry metrics, which includes both SDKs and the collector and all the various protocols that we have at our disposal. Okay. So this is for anybody visually impaired I'm better at drawing slides and I ended making them on the computer so I've scanned a bunch of hand drawings for this deck and hopefully you can follow along and read my writing. I'm trying to sort of start by saying who this is for. I want to attract like an audience to talk about metrics now so I think we can basically agree that there are many different participants in the in the in this type of community. We have people who are trying to configure telemetry and diagnostics for their platforms for their for their for their companies or where they work. There are engineers who are actually writing instrumentation and software at those companies trying to make more observability and better diagnostics for themselves. And then there are actually users who are trying to understand why is my system broken why can't I, you know, log into my, my machine or whatever. So these three different groups have kind of different perspectives on the problem space. And we end up thinking about what each of them wants as we talk through this problem space. Okay. Ultimately, the goal is to get some dashboards or some other form of alerting or monitoring on your data and that looks something like this drawing that I made here on the right. This next diagram comes straight out of the open telemetry library guidelines this was put together in the very early days of the project basically saying what we're after as far as the project as a whole. One of the things that we wanted to do is build a vendor neutral system so that you could as a as a developer decide to use open telemetry without locking yourself into somebody's SDK or some some vendors system. So what that means is that we've created an API separation from the SDK. That means our interfaces are decoupled from our implementations and this means that you can swap in another SDK, or some alternate implementation later. So the diagram has here your your application code running sorry. And then it goes into this sort of green box which combines both the API. That's the spec that we've put together for how you interact with metrics, as well as the SDK which is the default implementation that all the open telemetry libraries are going to include. And together this should provide you a high performance pipeline and you will then configure an exporter for the protocol that you want so that you can expose your data in such to the system that you want to. So how much is how much how much more effort is it to use something to use the telemetry setup versus some of the vendor neutral integration or the vendor specific integrations that people may be used to. I'm not exactly sure which vendor integrations you might be thinking of. So some say SDKs come in with a bunch of built in metrics for say your platform so you start using this library and you get host metrics by out of the box you get Kubernetes metrics out of the box. This is something that the open telemetry system will include. So automatic metrics as much as possible are going to be included for you. So we really structured this talk to leave sort of the details that you as a programmer might want to know to write your own metrics. It's really not the most important part of this topic, this talk because what we're really trying to do is help you set up an ecosystem help you set up a collector and an export pipeline, very few engineers actually write custom metrics and so that's sort of the least important part of the top talk here. Gotcha, thank you. Alright, so what's next here. One of the major requirements here is that we are an open source project and this lease, at least from my perspective was one of the biggest challenges of joining an open source project is now you don't work for a company anymore you kind of have your own. We have the community to think about first and so this has been a project that that moves at the pace of the community because we're getting what the community wants so I've drawn a picture here of a kind of a system that basically is a collection of a pipeline for both tracing and metrics data and we're trying to show that this collection infrastructure is going to work with all the open source systems that you're already using because that was one of our priorities. Okay, one of the sources of our sort of most sophisticated requirements came from open census the open census system is really what gave us the open telemetry project, combined with open tracing. And the open tracing side of that combination gave us that requirement about SDK separation from the API. What open census gave us was the requirement for a very high performance SDK, and the requirement to have that SDK be configurable. And in metrics what that meant was the ability to choose which metrics are going to be exported to choose which dimensions are going to be exported, and they called that the views API. So all those requirements were given to us sort of at the starting point we need to have a better way to configure which what happens when you use a metrics API. Next is not just about how do you record numbers. We need to make sure that the entire problem is being solved so in addition to you creating your own application metrics, you have plugins for your host metrics you have plugins for your these metrics, you have instrumentation for tracing is shared has shared resource attributes for your metrics and so we want to have specifications that tell us exactly how you should label data in a standard way. So that users will be able to find it and understand what they're looking at. So this is really a task that bridges technical stuff with kind of language stuff and understanding and questions about how we talk and what names we use. So this is an interesting sort of corner of the specification which is really about language and terminology and and understanding. Before I talk about data model and what you can do with this I want to be honest with you about the current timeline we are right now the open telemetry project is trying to finish the tracing spec is and freeze that and make sure that we have a stable tracing environment soon. Because of that attention being given to trace the metrics project has been slowed down a little bit. So we're kind of like putting our attention and tracing while we are waiting to finish metrics, but there is also just a large number of moving parts in this problem space, we have specifications which we've, we've definitely finished the, for the most part we finished our API spec, but the SDK specification is probably going to be the last thing to finish because we have to actually implement this SDK and a bunch of different languages and figure out what else needs to be answered. But as far as sort of the collector support that's been sort of ahead of schedule because we had the open census collector already and a number of the integrations and the receivers and exporters that you've heard about today, we're already in place for metrics so that's that's a little bit ahead, and then there's a few kind of lingering questions about data model and the protocol that are things that we may want to solve in the distant future we may not need to at all those are sort of open questions, things, questions about sort of the obscure corners of the world where you might want to use the protocol for something different. But for the most part we're getting close and we think that by the second first half of next year you'll be able to use this for real. Okay, I'm going to start talking about data model now. Before you do, we have a couple questions in the chat that I wanted to share so Raj has a question. Does the SDK capture raw data at a time stamp or is there data aggregation processing in the SDK. Right, so I do plan to talk a little bit about that later there is definitely that's one of those performance requirements that we got from open census. We need to have a high performance metrics library and we know that the sort of gold standard at the point when we started was Prometheus libraries. So Prometheus libraries are organized in such a way that your updates are very fast, mainly because you're actually pinning memory to keep, you know, a variable in place that you can update quickly. So we're going to talk later on this talk about something called aggregation temporality which lets us move that memory in our system, but we're still going to be able to aggregate in the process over short windows of time and output data that's been aggregated before it leaves the process. And yeah, we'll go into that later in your presentation. And then the other question that's been coming up a lot today. I'm not sure if you're planning to talk about this a little bit more but the relationship between open metrics and hotel or Yes, that is absolutely going to be part of this presentation. It's one of the bigger questions that I have to answer and I hope that I will be able to answer that by the end of this presentation. Great, then I will get out of your way. Well, alright, so I just want to start with data model like this is a, like what are we doing this is about metrics so I kind of want to bring us all back to the top level which is, we're trying to look at some numbers probably in a visual We may also be alerting and monitoring on them but the classic application for metrics is to create some charts and put them on a dashboard. So I want to talk through the different kinds of charts and dashboarding facilities that are commonly available through metrics, because there are different data types here. So the next few slides are going to talk about your sort of typical charts that you might get out of a metric system. That's what we're after. So first year as a count time series what I'm calling a count time series is just that there's some counter and it's, it's cumulative in the sense that I've begin this counter at the start of my process, and I may add to it, I may subtract to it, we haven't talked about monotonicity yet, but over time that total is kept as a sum, and whatever value I'm looking at as a function of time is the total cumulative count for that metric. This is going to come up a lot throughout this talk I use the, we use the terms cumulative and delta. I'm going to use the Greek letter sigma to refer to cumulative since it's a mathematical association we all have. And then you can imagine looking at the same data as a rate. Now this is one of the big deals here in metrics is that you can often represent count or some data is either a rate or a total and that's this comes. This is one of the complications that we're going to have to deal with, because they have different properties, but they're roughly speaking equivalent. If we talk about showing a rate, it is going to be associated with what we call deltas your reporting the change in some quantity and then you're going to plot the change. So this change could be can is a number that can rise and fall it and if the number, the count is not monotonic this change could actually be negative. So, this is another way to talk about reporting some data. We also have this notion of a gauge in the kind of traditional metrics interfaces and the data model here is a little different from sort of. What I'm trying to show with this visualization diagram here is that you may set a gauge many times during interval, but what we commonly report when we're talking about gauges is the last value that was set. So you have many points visualized in this in this graph, only the blue colored dots are the ones that are going to be actually reported. So this is called last value reporting we call this a gauge. And it's interesting because we don't actually evaluate or keep information about every point in some sense the number of points is irrelevant here. All we want to know is that there's a signal, we can evaluate it at a point in time, and we often record just one time. This is a relatively inexpensive type of aggregation. Now we also have this thing we call histograms or sometimes we call them distributions. The idea is that here you have these individual measurements, instead of just recording say the last value of one of those measurements. We're going to somehow capture more, we're going to capture both account and the value so that we can speak independently about the count and about the values. And here now is a distribution. On the top I've got red, purple and blue showing you quantiles so this might be P99, P50, P10 telling you where the distribution of latency or some other value in your system is. And then on the bottom, I have a rate plot which is showing for the same data set how many points were there per unit of time. So a histogram is sort of the most expensive type of metric data that we report because it includes two independent pieces of information. And we often use it to summarize a whole distribution. So these tend to be a little bit more expensive. The goal of those last four pages there was to show you that there's something familiar and common that we are after even though this data model may feel confusing after we start talking about it for a bit. Now, as far as what's different in open metrics and open telemetry, this is the, this is where it starts. There's two pieces of the data model that are not present in an open telemetry system. And I just want to say it's not because they're missing, it's because when you're pulling data in a metric system, you don't need these features. So the first thing is resources. Resources are a concept for these key value attributes that we attach to our diagnostics or instrument or observability data. It's not just metrics. This is also for scans and blogs. So I use the term attributes for this key value association. And the difference between what we've gotten open telemetry and what you have and say open metrics is that this concept or resource is in the protocol so that when you report a batch of metrics data, you're going to have one section for resources, and then you're going to have a lot of metrics data inside that encapsulation. And so this allows us to begin creating common coordinates and telemetry between traces and metrics because we have standard resources, and we can configure these export pipelines to attach standard resources to both our traces and our metrics as they pass through the export pipeline. So having resources is a first class concept. And then of course we have resource conventions for how you should name your resources. I've shown one example here, which is for the service attribute namespace. You've got service name, service namespace, service instance ID and service version. We are suggesting that these sorts of attributes may be included with metric data. And then some point later you may have to export them into a system that doesn't have this concept of resources and we're going to have to talk about whether those resources are or are not attached as metric attributes or sometimes these are called labels and tags, we have a terminology problem there and try to avoid it for this talk. So, so the first thing that we're adding that's different from open metrics is the concept of resources. And they say that you don't need these in a pull based system, because if you think about how Prometheus works, you scrape the target, the target gives you all of its own metric data, including its own metric labels, but it doesn't know its own resources. The whole point of scraping and a pull based system is that you discover the targets, you scrape them, and you know what their resources are so that's the person that pulls that data, who's responsible for attaching resources, and that's why you didn't need it in open metrics. The other thing that's different in open telemetry compared with open metrics is this notion that we have support for temporality. The word temporality is one that we sort of made up it is a real word, but we're using it to refer to this distinction between delta and cumulative reporting. So, and I want to make a distinction between because we use this word in two senses here it's pretty important to the design of the whole system. So bear with me. I think that there's something called temporality is trying to describe a relationship with time. And when we report metrics data, they may be cumulative, or they may be delta. Generally speaking, deltas mean that you're reporting differences since the last report, and every report is independent and does not build on the previous we need to you. So we're reporting changes one after the other in cumulative reporting we're going to repeat report some total since the beginning of time, and we're going to keep re reporting that total since the beginning of time. Okay, why are we talking about this. Well, there's something here called instrument temporality which is what type of numbers does this instrument deal with, and that's an API question, and there's something called aggregation temporality which is what is in the protocol that I'm putting through my pipeline. Okay, this is confusing. I have no slide on it. So why do we care. Well, the, this choices entirely about keeping state and stateful interactions between out of the SDK and the API for open telemetry. So, if you think about reporting say a total, your current memory usage, you don't want to report your current memory usage as a delta because that would require you to remember the last value that you reported for your current memory usage. So we find it's better if you're reporting something like a current memory usage to report a current total. And then the system that we're going to have if you're reporting information from a callback, you're generally going to report totals. If you're reporting something in a mainline code as part of a transaction or request, you're going to report deltas. From the start we have this built into our instruments, and it's because we think it is more convenient to talk about deltas in some contexts, and more convenient to talk about totals in other contexts, it keeps state out. Now, the point of creating this concept in aggregation this instrument, sorry, the point of creating instrument temporality is that we can adjust aggregation temporality. So what goes in is not necessarily what comes out. It turns out that if we have deltas coming in, and we want cumulative coming out, we're going to put memory. So this is why we have these questions that there's a trade off trade off is between reliability and memory costs. There's going to be more on this topic we will keep talking about this. So, I need to. So we're sort of, we're still talking about data model but we're trying now to integrate the concepts from Prometheus and the concepts from stats D, because one of those systems uses cumulative and one of those systems uses deltas. So, however, in the API for both of those systems counters use deltas so both Prometheus and stats the expect when you're using a counter that you will tell it a delta. Or when Prometheus writes data to its right ahead log or exports data through its remote right is sending you cumulative. And so, this is why Prometheus has trouble with cardinality requires you to keep memory inside the client library for every counter you've ever used because it has to track the cumulative value. In open geometry, we are giving you these instruments that have delta instrument temporality counters, and this not this instrument that we call up down counter are for for inputting changes or deltas to a sum. So these are just like the Prometheus concept these are just like the stats D concept, except that we've given you two ones for monotonic counters, and ones for non monotonic counters. And this is actually the simple case. These are deltas. Now when we talk about gauges. I mentioned earlier in some of this breakout sessions that there's a little bit of a terminology problem. And the, that I don't want to go too close to the details here because I lose you and you'll get bored but basically Prometheus and stats to use the term gauge in slightly different ways. And I've copied a definition out of Wikipedia for the word gauge itself, turns out that the actual word has two meanings in our actual engineering practice. This is a confusing term to begin with. And it turns out because there's it's used differently in Prometheus and stats D. It's even more confusing. We're trying not to use it. So the thing that you were using a gauge for, however, is a real application in Prometheus and stats D gauge got used whenever we wanted an individual measurement, but not the cost of a histogram. So open geometry created a new distinction to cover this gauge use case. And it's when you're recording an individual measurement. So I'm going to talk more about what an individual measurement means, but the point is that if you take a measurement, and there's something. So you say suppose you measure a latency, you would never add that measurement of latency to another measurement of latency, just for the sake of adding those two numbers together. So that is an individual measurement and you're interested in knowing individual latencies. So there's some. So, when you when you use this value recorder instrument, you're going to be computing a distribution as opposed to computing a sum which is a much simpler operation and much cheaper operation. I mean, why gauge doesn't exist and I think I'm getting a little confused or I'm not sure that people are following you at me at this time, because gauges are very confusing concept. But what I'm trying to say here is that we've replaced the concept of a gauge instrument with two new instruments. One is going to be called value recorder, and one is called value observer. We use value recorder to record individual measurements. We use value observer to witness an individual measurement such as a callback observing a total. I'm afraid this slide's not going off, but I'm going to keep going. The problem I'm trying to explain with gauge is that sometimes they got used to record sums that were cumulative or sums that were not monotonic. And sometimes they were used to record sort of other measurements such as a temperature or a latency which is not something you ordinarily add together. So the point is gauges got used for several different things in each of these systems and we are trying to provide you instruments that get used for exactly one thing. If you're observing a sum or you're observing a non-monotonic sum, instead of using a gauge, you're going to use one of these new instruments called sum observer or up-down sum observer. And these are instruments that have so-called cumulative instrument temporality. We're going to keep talking about this. Can I pause you? Yes. Please, Shelby. Can you go back one slide? So when you say monotonic, just because I might have missed that meaning at one point, that's like something that's unrelated to the previous value, right? I may have confused you and I would love to just completely reiterate this. So I've used term deltas and cumulatives like you just described. This is about the relationship between a prior interval and the current interval as far as reporting some values. The question about monotonicity is really about whether the function is always rising or always falling, for example. So the distinction is between sums that rise and fall versus sums that only rise. And when we have a sum that only rises, more often we're interested in showing that as a rate. But when we have a sum that rises and falls, more often we are interested in showing that as a count. And so you're probably familiar with metrics interfaces having the ability to choose whether you're talking about a rate or a count. This is something that we're actually encoding in these instruments for open telemetry. So I can imagine like for in a very lay example, right? Age is always in relation to the previous year. You can always just say like age plus one, right? And that's my birthday. It's related to the previous year. But for like temperature, it's completely unrelated to the past yesterday's temperature.  But usually you just want the individual like today's temperature. Yeah, that's a good observation. You don't ordinarily monitor changes in temperature. You monitor absolute temperatures, but you might monitor something like that is being counted as a rate. And then you might monitor that as a change. Gotcha. Yeah. I like using the very human examples, but it's also helpful to have them. I let your other examples of things like CPU usage, right? Where you always want that at the particular moment. You don't care how you don't usually care how much that relates to like five minutes ago. Yeah, I like your example of using age, though it's a great one. So you could imagine two ways of monitoring you an age of a person. If you're if you use a counter, then every year you increment it by one. If you're using a sun observer, then every year you output your current age, and that's always one more than the last year. So and we could talk more about the differences. We will talk more about the differences is one of the key aspects of this design. Gotcha. That's super helpful. Justin, please. Would you explain a little bit about why an instrumentation author shouldn't just always use the up down some observer. When you could use it to record age as well right is there a good reason to only use the sum observer in that case. Yes, and I am, I promise I tried to put a slide about this actually it's the next one. So, there has been this debate and I promise you we're going to spend more time updating our specs to try and clarify this issue. So, I am now regretting that I didn't include more detail already on the instruments that we've designed we've got six instruments and some of this is what we're talking about right now. But when we talk about semantic kind there really are two to my knowledge and in the technical committee we are in the SIG we've been talking about the words adding versus grouping, what we're trying to distinguish is between things that you add or things that you average. And things that you add are interesting. Let's suppose you're sampling. So if I have a bunch of observations of things that I add. Well, the larger numbers are more interesting because they contribute more to a sum. So when I'm sampling numbers and I know that there are things I care about the sum. Well, then I put weight on those, the higher numbers get more weight. When I'm sampling or down or, or any kind of aggregation that I'm doing, where where some is the property I'm after, then I want to know that it's a sum. Whereas if I'm doing something like down sampling or reducing dimensionality of one of these other types like a, where traditionally we have used a gauge or where you use now a value recorder or a value observer, then there's there's no just there's no difference in importance between a small number and a big number. You have a zero latency that's a significant measurement is no more or less significant than 100 second latency. So when you're talking about these individual values or individual measurements, you want to think of that as a different semantic type than it is then say an increment where we contribute to a sum because bigger increments matter more. This is a little, we're sort of straying into the theory of measurement and I should have put I could have put some slides in here about that but we do in sort of in statistics or in math we talk about scale for the numbers. So you can talk about ratio scale and interval scale and logarithm scale. I answered the question in those terms as well but I'd rather do it with the way I just did. So, I just discussed roughly what was on this slide is to say that you, it, we are interested in keeping the semantic type when we have when we have that information, and this choice Justin's question the choice between value recorder and or sorry value observer and and up down some observer. So they're very close but conceptually we want them to be different. And I promise you will keep writing to try and clarify this point. Okay. Well, I finished my section on data model. So that's good, we made it through and you asked the same question, right, as I was reaching my last slide. Okay, so for the rest of this talk we're going to go through some of the ways that you can configure export for your metrics data, especially talking about how we control costs. Okay, so in not particularly great order here we have a few features that are sort of new and interesting for open telemetry. One of the things that's been, I think, holding back the industry. Generally speaking is that we need variable bounds histograms or we need histograms that can support high resolution and be relatively compressed various ways of saying the same thing sometimes we call these sketches histograms by however you formulate them are approximate representations of a distribution and we want to get better at this. So, I've named some algorithms that are that you that are used for this. We are currently negotiating and working out some of the protocol decisions that we want here. And we are looking to standardize on one of these algorithms as a sort of recommended approach. Open telemetry is going to offer you essentially better histograms then then you've been getting out of the box for some of the metric systems that you're interested that you've been using. DD sketches the leading contender. A lot of the discussion that we've had about temporality in the past 15 minutes, ultimately comes down to cost and cardinality. So, what we have a situation where we expect that users or developers are going to be writing instrumentation and putting metric labels together that they're sort of that they may not actually know what are the useful labels or what the cost tolerances of the people running that code are going to be that you may end up in a situation where you're generating more labels than you actually need. So what do you do with that situation. Two things that we know how to do that we're including in part as as part of the open telemetry metric system. One is that we have the ability to do in process aggregation so these these SDKs are relatively sophisticated they include the ability to co less events that happen over a short interval of time. And then if we want to do label label reduction we can actually erase those labels and just aggregate those values together so that we get fewer time series with more aggregation happening. So this is a facility that the SDKs include and it's something that we have to configure so later on we'll talk about how to configure that but for now the SDKs include this mechanism. The other thing that we can do to control cost is the reason why I've been talking about temporality so much. So in a stats the system you may have been used to cardinality being okay because every at every flush interval, you can completely forget what you've been reporting and just begin accumulating a new state, whereas in a Prometheus system. If you use high cardinality those labels are stuck in memory for a very long time. So a stateless export pipeline is one where you force the use of delta export of delta aggregation temporality in order to allow yourself to flush out memory. So, look at some diagrams. Here's a diagram of a standalone SDK running open telemetry metrics. So you've got your application code running. You have a runtime metrics instrumentation package running that's telling you garbage question statistics maybe you have a host metrics instrumentation package that's running that maybe telling you CPU usage and memory usage and so on. You have the open telemetry API. Beneath that you have the SDK. The SDK has a frontline component we call the accumulator which builds up short term state. So over one interval whether that's a second or a minute or 10 minutes could be anything. You build up a record of everything that's happened during that period of time. And then every once in a while the SDK flushes its state. So this orange box here with a diagram saying here is where we do delta to cumulative conversion. So this orange box represents a long term memory commitment. In order to implement cumulative export for your metrics, you have to put memory somewhere and in this configuration, this standalone configuration, you just put that directly in your SDK. This is a configuration where you don't want to use a lot of cardinality because it's going to sit in memory for a long time. But this is a this is a configuration that's very compatible with downstream systems. You can blindly send this to an endpoint say a collector and it can export to Prometheus because you put the data into cumulative form before you sent it. Now the problem is you have to put a lot of memory in your application for this and you have to hold on to that memory for a long time, and it prevents you from using high cardinality metrics. So we have this other option to configure an export pipeline that is stateless. And the way we do this is by making sure that the aggregation temporality matches instrument temporality. If you're putting deltas in, you give deltas out. If you're putting cumulus in, you put cumulus out and this way you have no memory requirements. So in this diagram, it's the same as the diagram before, but now there's no long term memory commitment and there's no orange box. There's nothing happening in that box. It's just passing straight through. So this is going to cost you less. And if you have an OTP endpoint that supports deltas natively, this is actually a good configuration. And I want to say if anyone's listening here, if you're the author of an open source backend for metrics data, this is an opportunity, a big opportunity to accept native OTP data because it will allow the clients to begin configuring themselves statelessly. And this is going to be open the door to high cardinality metrics. So that was a sort of complicated configuration here because it is standalone. You've got several things running inside your process and all those things running inside your process add up to risk and cost. So there's another way you can configure this. And this is using the collector as an agent. So I might want to export cumulative metrics because that makes the downstream system happy. I can, for example, write to Prometheus, but somewhere in my pipeline, I have to configure this point where I convert deltas to cumulative, which leads to a long term memory requirement. So in this configuration, I have an application running inside of a host, but it is a stateless exporter. So it is, it is outputting OTP with aggregation temporality to require no memory. The hotel collector is running on the same host. It is able to implement that delta to cumulative conversion. And then you can output OTP converted to cumulative from your host, but your application is still running statelessly. So this is a way that we can reduce risk on the application itself. If you were accidentally to put high cardinality metrics in this configuration, it'll crash the collector, not your application. Actually, the collector is probably written to handle that better than your application is. So that's one configuration. Of course, there is still a long term state requirement here. We've just moved it from the application to the collector. So there's another configuration. This is where we talk about it just a standalone Prometheus exporter. Suppose you have an existing Prometheus setup, you just want to add one new target. So in this situation, it's roughly the same as that configuration that I showed you before. I'm running my runtime metrics plugin that tells me GC stats. I'm running my host metrics, which means I don't need to run some other Prometheus tool to export node statistics. I've got my API and my SDK. I'm still doing that delta to cumulative conversion because I must for Prometheus. And then I can either pull that data using open metrics, or I can push that data using a Prometheus remote write exporter. In both of these cases, I'm writing to Prometheus. In both of these cases, I have to do a delta to cumulative conversion because, again, Prometheus requires it. But I could also do that statelessly. Again, almost the same diagram as before. I've got a stateless application. I'm sending through a stateless export pipeline. I'm receiving that at the collector agent. I then can do delta to cumulative conversion. But now I'm going to use the Prometheus remote write because there's some impedance mismatch using open metrics to pull from a collector. We're not going to do that right away. So this is a way you can export to Prometheus through a stateless in process exporter and a local agent. Not the only way you can configure export for open telemetry metrics. I want to do a little review on some of the stuff that we just talked about. There's sort of two ways that you might, two ends of a spectrum that you might configure your metrics export. You can do either push and pull or you can do stateless or cumulative. But we generally pair the pull model with cumulative, which is the Prometheus model, and that's the default for open telemetry. And that has properties are good for Prometheus. You're automatically going to work with Prometheus, even if you have a collector in the loop. You get easier reliability because when you drop a record or a data point in Prometheus, when you drop a cumulative data point, it gets smoothed out by the next data point that arise so loss loss of data is pretty okay in a Prometheus system. But high cardinality is pretty much not allowed and it will blow you up if you do it by accident. So, in this other side here we've got this Prometheus this push model and this stateless export model so one thing that's going for is it's exactly the same as we're doing already for traces if you're going to set up an export pipeline for your span data. You might want to set up the same exact topology for your metrics data. So the idea that you're going to attach resources as you collect data through your infrastructure. It fits very well it matches trace collection. It means you're using delta can delta aggregation temporality. It means that you have to be a little bit, you have to work a lot harder for reliability. So you have to avoid drop packets means loss of data and replayed packets means double counted data so we have to be careful to avoid dropping data but we also have to avoid replay. But the upside of this configuration is that that high cardinality is no longer a cost inside the process and high cardinality if the downstream system supports it is going to be okay and actually could be good for the user. I didn't cover this combat this sort of worst and best of both cases which would be a push cumulative model that is definitely a valid configuration and it's exactly what you want if you're going to send Prometheus data through a collector. So, these are choices. Currently, we're, we're the default for open telemetry is cumulative because it's the most compatible but it does mean that basically we're giving you Prometheus status quo you're going to be keeping memory for your cardinality until we reconfigure your system to use this push stateless export strategy. Yeah, question came in in our chat about what things exactly do we need to configure to get that stateless model you're describing. Um, good question. So we need some more collector development there is not an actual cumulative delta to cumulative processing stage today. It's imaginary. The reference implementation for the hotel metrics SDK which is the one that I've been developing and go does have this ability to select the stateless versus cumulative mode today so the SDK support has been specced out the reference implementation does exist it works, but as far as a collector support it's not there yet. If you had a vendor that was supporting both forms of otlp. Today, you could get that from police from the reference implementation. I hope that answers the question. I'm very excited about this because as a user from a stats D background, I was used to having a little bit more cardinality than I'm able to get from a Prometheus configuration so I would like to push more high more cardinality on the metrics world until it's a real problem and I'm hoping I think there's a good opportunity for data scientists to come help us with actual cardinality control mechanisms like we can down sample to control cardinality. For example, I think that's actually really exciting topic, but I won't say no more. I think we're almost at the end here of my slide deck. I have a few more diagrams just to kind of give you a greater picture of all the ways you might configure open telemetry metrics. This is an example of a sort of Kubernetes deployment where you've got a node, you've got a daemon set collector running on every node. You've configured your receivers for open metrics for stats D for otlp. You've got the Kubernetes receiver built in so you're getting the Kubernetes state metrics, you've got a host metrics running on the collector so you don't have to run those host metrics on every machine running on the pod, or on the node for example. So these are just a number of sort of this is the scale of these installations is now going we're going to be able to talk about one collector per node and a bunch of different targets per node and so on. And this is just part of this the plan for open telemetry collector. And part of that resource model is that we're going to implement hierarchical collection so that you have a node collector that's collecting all the resources, all the metrics locally and it's going to pass that to a regional collector that's going to attach all of your regional resource attributes, and it might pass to a global or other levels of hierarchy that you can use to organize your metrics data. So that you might be able to bypass certain Prometheus features with if you're using Prometheus you might be using recording rules to get some more functionality, but we can just talk about an expert pipeline that aggregates everything together into one place without that type of recording rule functionality, which is like a right time aggregation. All right, there's one more stated for last, I've discovered that there are a lot of Prometheus users out there who have invested a tremendous amount in their configuration so Prometheus does a lot of things and it's going to be difficult to replace all of that functionality. So we were looking for ways to help Prometheus users get on board with open telemetry and begin using otlp. So I found this thing that the Stackdriver group had done a couple years ago. It is a Stackdriver Prometheus sidecar. I haven't talked about this much, but today it's open source now we opened up the repository publicly. This is a sidecar for Prometheus, basically lets you read Prometheus data and send otlp. This I see as a short term solution because ultimately the Prometheus project will add metadata to its remote right protocol and then it will be able to replace this sidecar. So today you cannot send Prometheus data directly out of Prometheus into otlp without a tool like this, but hopefully in the future that restriction will go away. So I've got a few links to this code base as well as a couple of the Prometheus issues that are currently trying to address this shortcoming, so that hopefully we can retire this code in the future. But meanwhile, Prometheus users should be able to try their open telemetry metrics system, and this should also help us migrate from Prometheus onto open telemetry in a gradual way. Now I think I've actually reached the end of the talk and I'd love to have questions and discussion from the group. Are you expecting questions on the Zoom chat or we can ask them just like. Since I've been presenting I wasn't reading Zoom chat but I'm going to un-present. And then we can all talk and I'll read the chat. Have a quick one please. So this morning we were talking about traces and context and context propagation and this afternoon we are talking about metrics and labels and tags. So do we lose the context of traces when we handle open telemetry metrics or is there a way when we can link or correlate the traces context with the metrics labels? First I just want to point out there's been a bit of terminology debate and we haven't actually settled it. So the term attribute, the term label and term tag are almost synonymous and used interchangeably here. I hope that's not the question you're actually asking. I believe you're asking about how we talk about distributed context and getting those attributes that come from, say, distributed actors in your system onto your metrics. This is something that I mentioned in one of the breakout sessions earlier today was the sort of driving, I think a driving motivator for the census design and part of the sort of the heritage of open telemetry at this point is that we were based off of open census metrics. So open census metrics did include a way to, open census did include a way to get your distributed context attributes into your metrics. It's something that you have to configure. We're talking about the terminology for this. We've got something called an enhancer at this point, which is a hook that you can use to pull attributes out of your context and put them into your metric system. I hope that answers the question. We're still talking terminology. Okay. I think maybe there are a few pieces there. One of them is that we can pull key values from your baggage from that correlation context and put them onto metrics. Another one I think is just built into the semantic conventions of our metrics. We are trying to design the semantic conventions so that you can make correlations between those metrics and the traces that might be representing the same timed operations. And then the final thing that is maybe the most exciting in my opinion is the concept of exemplars to go on metrics. And what this is is like each metric will include just like a list of a few data points a few spans that are examples of that metric, which then there's like a very strong correlation between those spans and then that specific metric. Yeah, so to make that clear, we have ways, at least planned out or specced out in protocol to get data from your context, including both those attributes as well as your span ID and your trace ID, and to get that associated with metrics. Fantastic. Cool. Thank you very much. So we were planning to take a five minute break before the next session. But if there are any other questions, you can take them into the hotel community day Slack channel. Thank you so much, Josh, for presenting and sharing your expertise. I shared my notes in the chat. I'll share them in the Slack channel as well. I got a lot out of this. Thank you, Shelby. I'd be happy to stay on for a little bit. I do see the questions now when I was presenting I couldn't see them. I'm not sure how I could have fixed that but there's one here, for example, I can see about hotel policy on discovery when using open metrics approach. That is really one of the greatest questions and it's not one that we're really answering here open metrics or the Prometheus system has a great big piece of code that does service discovery. And right now there is an ongoing discussion about how to either emulate or simplify or replace such functionality in the hotel collector world. And the current state of affairs is actually not so good. We've linked in a huge dependency on the Prometheus system and are actually using that service discovery configuration, which has caused some friction and some dependency bloat. We are working on that. Bowdoin, would you like to speak? Anybody want to speak on that? Something that needs to happen. Yeah, I actually have a dream here that somebody will take the Prometheus service discovery code, factor it out of Prometheus and create a first class service called service discovery service. And then we can have an hotel collector plug in which will reach out to the service discovery service with some sort of sharding information and get a list of sharded targets for it to scrape so that we can then rather than linking in huge dependency on Prometheus just actually call out to a service that does the same stuff. And then the Prometheus service discovery will become its own thing. Awesome answer, thank you. Also, it was mentioned on the slack that this is a break of five minutes before the next session. So everyone who is willing to chat more, feel free to but otherwise, it's considered a break. Service discovery service discovery service. Oh geez. All right, everyone. Thank you. I'll be in the next session. I look forward to it.