It's beginning time. Oh boy, okay, I think the doors are closing here. I will go ahead and start once a few more people get a chance to sit down. Hello everyone. Hello to my coworkers sitting here in the front row to make sure that I'm extremely embarrassed if I crash and burn in this talk. And hello to everyone else. This mic keeps on slipping down but I think it's doing okay. Hi, my name's Phillip. I am an open telemetry maintainer and I also work for a company called Honeycomb. We do observability and tracing observability. So to the surprise of nobody, I'm giving a talk about how you can trace with open telemetry. Specifically for monoliths though, which I think is something that doesn't come up very often in the context of distributed tracing. Usually people talk about it when they're referring to microservices and big old distributed systems and stuff like that. But I don't think that's really the reality that a lot of people live in. And in particular, I think I wanna begin by saying that I think monoliths versus microservices is a pretty meaningless distinction. I think the messy reality that we all live in is that we're kinda living in some sort of in-between world between microservices and monoliths and distributed monoliths and all that. And in particular over the past couple of years that I've been working for Honeycomb and helping people adopt tracing and observability for their systems, I've certainly noticed a few patterns. A common monolith application like yes, of course it's running in a single process, but it's typically running behind a load balancer. Maybe there's a database server too that it's talking to. Maybe there's a message queue involved. Maybe it's calling an authentication server somewhere. There's some third party vendors. Maybe you've actually split out some of that monolith into separate services and created them as microservices. At that point, you have a distributed system already. In fact, I would wager that a true monolith that just talks to a database when that database is on a separate system, that is already a distributed system. It may not be the most distributed of systems, but it is still one. And by definition, it can benefit from distributed tracing. Now, it may not necessarily benefit as much as the largest spread of like thousands of services or something like that. There's quite a bit of benefit to extract from that. Separately, which I've observed plenty of times in my own career, is that microservices grow into monoliths themselves. It's pretty easy to just keep on adding code to stuff. And eventually you go, oh geez, this code's kind of a mess. We should modularize this thing. And before you know it, you end up with a bunch of distributed monoliths because it's a lot easier to just modularize code that's inside of an existing process than it is to factor it out into its own separate system and so on. You may actually go and do that, but you may not necessarily do that in some other cases. And it's perfectly okay. This is just the natural evolution of a lot of systems. But I would say that regardless of where you are, if you started in like a pure microservices environment and kind of ended up with some form of distributed monoliths, or you had a monolith that you kind of partially broke out into services but never really got around to fully microservicing it, you already need distributed tracing if you're not adding it already. And that's to say that there's a lot of problems that occur in the context of systems that are outside the actual app code. Somebody may have had the bright idea to add GRPC to your system and now your load balancing is totally screwed and nobody knows what's going on. Or maybe somebody got a little too fancy with a SQL query and now it's taken a really long time and you're not really sure if it's your ORM or if it's the database itself. Maybe there's, sometimes there's bugs in databases, maybe it's something to do with your network transport layer. These things are also really, really hard to diagnose. And very often you don't even really know where to look for. Sometimes if somebody's really experienced, they kind of have a clue as to where something could be going but your average developer is gonna be like, okay, this is a unique issue, I've not seen this thing before, what do I even do? And so you can, even with the most basic of distributed tracing in just a monolith talking to a database, you can ask all kinds of interesting questions. Like how many database calls did this request take? Which database calls are the slowest? Which SQL statements are the ones that are not doing too hot right now? What's the average latency of my requests? Is this external API that I'm calling the thing that's actually causing a slowdown or not? And you can start slicing and dicing that by a lot of very interesting characteristics. And so this is all just at the edges of your system, I've not even really gotten into adding tracing to the monolith itself. But when you can start to answer these questions, you can unlock a lot of interesting fun behavior and realize that your application is doing a lot of things that it shouldn't be doing and diagnose a lot of problems that your users are already probably experiencing today. So you can rein in the edges of your systems with this is in particular, this is automatic instrumentation. You can also do a whole lot more. And so one example I wanna bring about, bring in about the early, the introduction of this journey is with a company called Intercom, who's one of our customers. There's probably some of you who are using Intercom, there may be somebody in the audience who works for Intercom. Intercom is sort of like a support chat system and like there's sort of a thing that you can embed as a part of your product and have a good support system and build like product tours and that kind of stuff. Their app was a big old monolith and a handful of smaller services that they kinda broke out here and there but they hadn't really fully gone the whole microservices route and they didn't really have a desire to do that either. They had a lot of structured blogs but they didn't have good tracing and so they decided to adopt it. And so they started just by instrumenting API calls, database calls and eventually getting into instrumenting their service themselves and they wrote a wonderful blog post that you can Google about where it really transformed their internal engineering. Where in fact they didn't even necessarily use tracing to be like, okay, this is how we're gonna diagnose the problems that our users are reporting but even onboard new users and understand just how this thing is working in the first place. When a user comes in, what's the journey that it actually takes like this request? Which other services does it hit? Which database does it hit? What are the paths within the monolith itself that it actually goes through? And distributed tracing actually ended up becoming a way to just simply better understand their systems in the first place so they can improve them more effectively rather than just sort of trying to be like, well, I know what this function does and I kinda know what this function does but I don't really know if one calls the other in this big old soup of code that we got going on here. And in fact, their journey here took about a year. They iterated a whole lot and so like it wasn't one day they had a whole bunch of amazing instrumentation. It was sort of a little bit more and more every single week and they eventually got to the point where now they have really, really rich tracing instrumentation. And so I wanna talk about that a little bit. You can call it tracing for a monolith but it's really just tracing for a single service. In fact, the principles that apply to tracing a distributed service are the same that you would apply to pretty much any other service that you're doing whether it's a small service inside of what you're doing because you wanna onboard tracing for the first time or if you're fully bought into the idea of tracing and you wanna distribute, you wanna trace your very large monolith that you have going on there. I'll call out a few examples along the way. So if there's one thing that I want you to take away from this talk, if you've not done tracing before is that traces are just better structured logs. In fact, a trace is actually a collection of these things that are called spans. And a span is just a structured log that has some IDs on it. And the collection of those IDs allow for you to define an order of operations and a parent-child hierarchy in your system so you can understand that, okay, these are the operations that occur within a given request and this is the order in which they occur. And so there's a name because everything needs to have a name. There's a span ID which represents the operation that is actually being performed. That span may represent the entirety of a request or it just may represent a single function call. There's a parent span ID so it knows what came before it so you can define that order of operations. Key value pairs of data, this is the log part, right? Basically there's a name and then a value and a name and a value and you wanna add as many of those things as you want. And then there's a list of what are called span events and span links. These are not really necessary per se but they can often be really helpful in modeling certain parts of your system. So especially with span events they sort of represent a moment in time that something occurred within the context of an operation and you want that time stamp and maybe some other metadata associated with that operation that happened. This is a really easy thing that you can just attach to a span. So it's like structured logs but it's better. It's more powerful. It's got a lot of goodies baked in and open telemetry defines all of these things across 11 different languages that are fully supported. So it's something that you can frankly get started with today. I like to show people this visual representation which again actually came from Intercom Engineering. This is from the blog post that I mentioned earlier. On the left hand side you can see a big incomprehensible soup of text. This is actually their structured logs before they added traces. And so there's a lot of actually really useful information in there. I'm not gonna try to zoom in so you can actually see what's in there. It's pretty incomprehensible. In fact, you might notice there's some colors in there. They tried to color code one thing and another thing so they can kind of visually distinguish hey there was this operation and this other operation and stuff like that. That's a losing game. Like that's really, really hard for you to parse. Imagine a user is reporting something wrong and like you have to just stare it into that thing for minutes, hours maybe and try to understand what the heck is happening. Now, so when they transitioned over to tracing the right hand side, this is a trace within Honeycomb but you can get the same visualization in pretty much any tool that supports tracing. You can see there's a hierarchy there and actually operations that are nested. And it's this order of operations and hierarchy that tracing builds in for you that really helps with understandability of the system. So it's not just that function A calls function B rather function A and function B are operating as a part of a hierarchy that's actually contained within this other thing that's happening. And so there's a whole bunch of nice little visual tricks that you can do with a tracing tool like understand okay, this like sub hierarchy is uninteresting to me so I can just collapse it and move on to the next thing. And you can't do that when you have this flat list of structured logs that you're looking at. So this is kind of conceptual. How to trace a single service is what I would like to call it, it's simple but not necessarily easy. So it's simple in that there's very few steps that you actually need to take but a lot of those steps need to be repeated in the context of your application and the information that you end up capturing in your traces is gonna vary from parts of your application to other parts of your application from system to system and from team to team. So this is probably one of the easiest things to actually Google if you go to open telemetry, let's say you have a Java service, right? Get started open telemetry Java. We have a really thorough doc site that tells you how to add automatic instrumentation. And this is kind of the first step that you wanna take whenever you wanna start instrumenting a single service is there's a couple different languages support actually modifying a, well adding that instrumentation without modifying a service itself. So in the case of Java, there's a Java agent that you can load as a sidecar that will instrument all incoming requests and outgoing responses, things like calling a Kafka cluster, pretty much anything where there's a library that you use to interact with another part of a system, it will instrument that and create spans for you automatically. This is true for Java, this is true for Python, this is true for .NET, this is true for Node, this is true now also for Go services. And if you don't want to use an agent or if you can't use an agent for whatever reason, maybe the language doesn't support it right now, there are instrumentation libraries that accomplish the same thing. So it's a little bit more work, but basically you just initialize open telemetry with these libraries, you sort of register like, hey, we're doing a Spring Boot application. So we're just gonna instrument string boot requests and that's it. And you don't have to go and figure out how you're gonna like parse this header data and that kind of stuff. It just sort of takes care of that for you. And so critically, what this auto instrumentation and instrumentation library stuff does is it creates the structure of your entire application or the skeleton, if you will. So you can now start answering those questions from earlier, like which requests have the slow database calls or if I'm calling this external service, is that external service the thing that's slowing down? It'll tell you on a service by service basis which things are slow. And it'll allow you with an observability tool to actually calculate a lot of useful information, like average request latency and average error rates and that kind of stuff and things that you would normally try to get from a metrics-based solution you can actually derive from these traces directly, which I think is really powerful because that allows you to stick with tracing as your primary way to do instrumentation. But then once you do that, you're gonna wanna get a lot more useful information out. So it's not just slow requests that you wanna care about, but there's particularly users that are involved with those requests or let's say you have an IoT based system, there's particular device IDs that might be experiencing, if you can imagine a IoT device experiencing something, but experiencing like slow database requests or bad behavior or something like that. And you wanna be able to start slicing your data based off of that application context rather than just what's going on at the edges of your system. So I like to think of it as sort of like layer one and layer two problems. So like layer one problems are like, at the edges of my system is everything doing okay, but then now I wanna dig into like my actual application. And the way you do that is by using the OpenTelemetry API, which again is very simple to add or at least start using, right? Once you add automatic instrumentation, what that will do is it will initialize OpenTelemetry in that process. And so then you can use the OpenTelemetry SDK to access those APIs and you can start creating spans. And the mental model that you wanna follow is whenever you think, oh yeah, we should log this thing. That's sort of a sign that you should either create a span or add data to an existing span. It's kind of up to you if you want to represent an operation as a span itself or just capture key value pairs in existing span. This is where it's simple but not necessarily easy because this is gonna be getting into the domain of your application, right? You wanna create a span at the start of a function or a method to sort of track. You can think of it as like tracking the operation itself. And within that operation, you can attach as much arbitrary metadata as you like. In fact, you can use span events to say, oh crap, this thing happened right now and we're just gonna add that data to the span and then end it and then ta-da, we're good to go. That's gonna be more challenging for a large service because you may not necessarily own the part of the code that you wanna instrument. And so that's why I'm continuing to say the word simple but not necessarily easy, especially when you want to instrument deeply such that you can use that instrumentation to understand your whole system. That's when you're gonna need to get a lot more team members involved instead of just willy-nilly throwing spans around and hoping things work out. And so there's sort of a secondary way to do this or rather additional way, I guess, it kind of depends on the size of the service. So hopefully if you have a big monolith, it is at least organized into different modules or if you're following like the Martin Fowler kind of spiel about bounded context, sort of like defining organizational boundaries within your application. Those organization may map to like, they may map to specific teams or they just might be domain organization. But regardless, there's a way that you can also represent that in your telemetry when you're creating spans within that single service. The easiest way to do that is to add an attribute to a span that you create. This is metadata that basically defines that name and that name can come from configuration. It can be defined programmatically such as if you're using .NET, you can just say the current name of the assembly and if you organize your modules into assemblies which a lot of .NET applications do, this is a great way to do that. Or you can even just define this as a required parameter whenever somebody wants to create a span in a trace, you can just say, okay, what is the bounded context that we're in or what's the name of the module that we're in? And the reason why you wanna do this is because it allows you to slice this data based off of that application context that you're in. So this is very analogous to services and distributed tracing. So with the full distributed tracing assuming you're like in a microservices world every span has what's called a service name on it. It's actually defined as what's called a resource, but I won't get into that. Basically whenever you create spans inside of a process it has a thing called service.name. And so that allows you to say, okay, well your which of my services are the slow ones and you can sort of like stack rank by services like latency and errors and that kind of stuff. That's great if you have lots of different services but when you're inside of a single service that big chunky boy that you got there is a lot of different modules inside of it. Well, organizing by service isn't necessarily the most meaningful thing in the world because well, it's all in the same service. So how do you do that? Well, this is sort of recreating that thing. And what's great is the open telemetry API makes this extremely easy to do and you can then use certain language constructs such as assemblynamefor.net or if you can get the package name of Java or if it's even just a file that gets loaded at that point in time that gets configured externally. Regardless, you then have an organizational attribute on every single span that you can then start to slice and dice your data by. And so this is really helpful when your monolith has its boundaries defined by different teams and you can actually even have that organizational context be like a team name or a team ID. And so then you can start to use sort of like the organization of your own company to your advantage when you start looking at things. So you can say, oh, well, is my team the one that owns the service or is it another team that owns the service? We see that quite a bit actually with some of our customers where they just attach literally like the name of the team to the instrumentation that they're capturing and then they're able to very quickly hunt down. Okay, this thing that's slow right now that I'm observing is this team I'm gonna go and talk to them. If you don't have that ability, it can be really hard to hunt down who the right person is for something. And so encoding that into your instrumentation is actually fairly easy to accomplish but it really, it really, really pays off in the long run. And so I wanna show you a visualization of an in-process span. This is actually from Honeycomb itself. This is a feature that we have called, it's called Query Assistant. It's just, so we have this big service, it's called Poodle. It's named after a dog, I don't know why, but it's named after a dog. This Poodle service does a bunch of stuff. It basically is the thing that powers our entire UI. There's all kinds of different traces that are shot off from all kinds of different user interactions. And so it basically is a monolith. So one specific piece of that is this feature that I worked on where you can type a little thing into a box, hit a button, and then it goes and it executes a query for someone. So that itself, maybe somebody news coming in, they're like, oh, there was a bug in this feature. How do I know how this thing works? How many operations does it actually do? Well, the answer is it does 48. And any of those 48 steps could potentially go wrong or something could be slow. In fact, within those 48 steps, we make another network call. We call into another subsystem. We get a bunch of schema information from our database and stuff like that. There's a lot of ways that this, this is actually kind of a complex feature. There's a lot of ways that it could be subtly wrong or we could introduce a new bug or something like that. And God helps somebody who's new on the team if they did not have this instrumentation in the first place to understand where something is slow or where something might be erroring out. Or even if the wrong decision is being made somewhere within that soup of function calls that we're doing. And so this is extremely helpful for onboarding but then also really understanding sort of where your latency and your errors lie. And so in this case, we had an operation that actually took what, eight seconds. That's actually quite a bit and about four of those seconds lie from the middle of this thing. And so that already gives me a sort of an indicator of like, okay, half the latency is coming from this one spot. Maybe we're not making the right API call here or maybe we could actually improve the way that we're making this API call. And this is sort of just immediately visually evident from this sort of thing. And all of this is from, frankly, not a whole lot of effort to instrument manually with open telemetry APIs. So I've talked a lot about instrumenting with spans and traces but chances are you probably got a bunch of logs in your monolith and you shouldn't have to throw them away. This is, we worked with one customer in particular who really wanted to stress this where they had a big monolith. The company's name is Loan Market. They're based in Australia. They were completely sold on the idea of distributed tracing, adding traces. They're like, yeah, we get it. We get it. This is what we're gonna do. But we have a big legacy of logs that have been useful for us in the past. And if you tell us that we can't use that, then you're not gonna get our business. I mean, that's a, so like open telemetry has recognized this fact. Okay, like this has been the case since the inception of open telemetry but now I think it's a really, really, really good time to start adopting because of a concept called the open telemetry logs bridge. So it used to be called the logs API. It got renamed to the logs bridge to better represent what this thing does. So the idea is that for a given logging framework, let's say, again, you've got a big old .NET application. It's using the Microsoft Net Logging Framework which is kind of a standard one that most people do to construct structured logs. What open telemetry will do is you initialize a trace within that same process. So that can come from automatic instrumentation or you can just do that manually on your own. It's kind of up to you. But then in memory, there's this object called trace context. And so trace context is the thing that contains that trace ID and that span ID that sort of tracks like what the operation is at this moment in time. That same object is then serialized and deserialized at process boundaries. And that's what sort of creates that trace for you as you start creating these structured logs that we call spans within a process. And so what it does is it actually intercepts that log as you call it and it wraps the body of that log with the span ID and the trace ID. And so it's like kind of creating a new span almost. It's not quite a span because the duration part is missing. There's a couple of other things that are sort of missing. But critically, the body of that structured log that you may have created is now directly correlated with the trace that you have. And so that means that you may not have like the ideal world of really rich tracing instrumentation going through your application, but you're kind of halfway there. And that automatic correlation is what's really, really important because then you can start saying, okay, I know I have this useful application context in my logs, I just need to know what was the request that came in or what was this other information that's existing inside of my spans. And like maybe I can even correlate that with things that are happening downstream like database calls that are going on. And so this logs bridge, I think is gonna be an extremely important concept for the majority of legacy applications to start onboarding onto open telemetry. I'm here to say that like now's the time to start looking to see if your logging framework is supported because this is stable. And this is stable on the spec, this is stable in the transport layer. And that means that framework support is starting to onboard across all the different languages right now. And so what you do with that, and this is what our customer loan market did, is they use this as a basis for slowly and gradually migrating stuff over to traces or sorry, spans I should say, but on their own time and when they felt like it when they had time. The value proposition of observability was not to throw everything away and start all over again, but this time with distributed tracing because everybody says it's better. No, it was to keep sending your logs where you're sending them. In fact, in this case, they would eventually send them over to Honeycomb or another tool alongside their traces, but they didn't have to interrupt that workflow. They didn't have to stop what their devs were doing already, the diagnosed problems. And they could figure out their own migration story.  What they would do is one of their leads started to look at all the different places where they were logging things and said, okay, well, this one's actually really important. Like these other things maybe not so important. So I'm going to focus on the important area and I'm going to turn that structured log into a span. And then I'm just going to repeat that process again and again. And so where they are right now is that they have migrated a decent number of their structured logs over to spans. And so their traces are getting richer and richer, but critically, there's really no loss of information throughout that entire process. Now, what that has actually meant for them, and this would actually mean for you if you were to do this too, is that your workflow for looking at a trace and then looking at like a correlated log might be a little bit disjoint sometimes. This kind of depends on the observability tool that you're using. Honeycomb supports the ability to look at this kind of stuff, but when you're looking at a log versus part of a trace, you have to use our UI in a particular way that's sometimes not the most optimal. And frankly, I think most observability tools are kind of like this too. Some will support a way to sort of like go to like the logs database, but because it's correlated, it'll actually take you to the right thing so you don't have to go and search for it. So it's not all unicorns and rainbows, right? You're gonna be migrating, migrations are kind of messy, but you can actually do it and you don't need to like have this question of like, oh, well, if I add tracing, is that gonna be compatible with like everything else that we're doing? That's not a question that you have to ask. And so to be honest, in the hotel community, I don't think we've really figured out what a good migration story should look like for organizations. And this is where I think a lot of us would like some of your help. If you're in the process of migrating a monolithic service over to tracing with open telemetry, we would absolutely love to hear from you and try to understand all the problems that you're having. If you're motivated by this talk to go and do that, yes, absolutely please. I would personally love to hear like what your issues are. And in particular, I wanna try to get a lot of these common things documented in the open telemetry website so that the next people who go and start to add better instrumentation to their monoliths don't have to run into a variety of stumbling blocks that might be common across lots of different orgs. So I wanna end with just a few takeaways that you can take back to your team. So traces are for monoliths, right? If you see the phrase distributed traces and somebody says, yeah, that's what you do for microservices. That's not necessarily wrong, but it's not really inclusive of the fact that there's really nothing about tracing that is inherently for microservices or against monoliths. They're sort of one and the same. And in fact, the practices and the idioms around tracing apply just as much to a monolithic service as they do to a widely distributed set of microservices. You don't need a ton of services to get value. You can ask even really basic questions with tracing that you couldn't ask before, at least not without a lot of effort with your logs. You can start with one service, get value out of that and incrementally add more. In fact, there's technologies where now in, like if you're running everything in Kubernetes, you can use the Kubernetes operator that will auto instrument every single service inside of a cluster. So that's really kind of a great option, but that's sometimes not the reality that a lot of people live in where part of stuff is in Kubernetes, some stuff is not necessarily in Kubernetes. But regardless, you have a lot of options available to you. And I would say that like if you can adopt the mindset that structured logs are, or rather I should say traces are a superior, like better form of structured logs because they have an order of operations and the hierarchy associated with them, then it can actually be fairly easy to get the team to onboard with this kind of stuff, especially when they can see that trace waterfall for the first time and really sort of see, okay, this is what my service is really doing. And lastly, you bring your logs with you with the LogsBridge API. This was something that was not really true about two years ago, maybe even a year ago, arguably. There was a little bit of support a year ago, but it was kind of janky and not that widespread. Now is the time to start looking into if it's actually supporting your logging frameworks or not. Now, it may be the case that your particular logging framework is not supported by the LogsBridge standard in Hotel, and that's fine. Check back in a couple different months or just go ahead and proceed and go ahead and add tracing anyways and look at your logs in one place, look at your traces in another place. It's kind of a messy world, but it might be better than not adding that tracing in the first place. So that's what I got. I have a little bit of extra time for questions, so please feel free to ask them. Thanks for the overview, it was really helpful. One question I have is around more like legacy monoliths. My experience and maybe others is older monoliths tend to adopt a polling architecture where they're like polling the DB or polling some API or something and creating a lot of noise. And so it creates a barrier to entry for like auto instrumentation because then you just end up with like a mess of useless traces that you have to sample out or it raises the barrier to entry. Do you have any suggestions on how you can onboard to tracing still in kind of an auto instrumentation way without necessarily having to tackle architectural constraints and all of that up front? That's a good question. So for anyone who didn't necessarily hear the question was that sometimes a lot of monoliths have architectural patterns like polling a database all the time instead of just like calling it when it needs to call it. And that can potentially create issues when you add tracing. So yeah, I mean, I think there's a couple ways to think about that. First is that most of the auto instrumentation components in hotel allow you to filter things out without needing to sample entire traces. So if something is proving like really problematic like basically like imagine you add the Java auto instrumentation agent and some of the instrumentation is super noisy because of this architectural pattern that you just can't unwind right now but then some of it is actually really useful. Well, you do have the ability to turn certain things on or off through configuration. And so that can be like a really good stop gap in the interim. Now it's not like a complete solution but you can sort of get something instead of nothing. The other thing is the sampling of that data. This is sampling gets into a bit more of a complicated topic but there is a concept and sampling known as dynamic sampling. And so the idea is that it's able to understand statistically based off of the frequency of certain kinds of traces and data that they contain and sort of up sample or down sample based off of the uniqueness of keys that you sort of established. So like if you have a particular key representing some operation that's going on and there's a lot of noise it can actually very aggressively sample those traces that are experiencing a lot of the same values in that key. So it's like that concept is kind of like that's a whole other talk. In fact, Kent Quirk here is he gave a talk on this topic as well. I would certainly advise chatting with someone like Kent about this concept. This is something that does exist today. It's not in hotel but it's likely going to be added to open telemetry pretty soon. So that's kind of what I would say for now. Hi. My question is will be there any performance overhead if I start tracing all the requests that comes to my web app? Yes, there will be performance overhead. The answer to that though is there's a really, really big old it depends for that though. It kind of depends on the shape of those requests, how much data is involved in each one and the provisioning of your infrastructure. So like I've seen somebody come in and say this added 20% CPU and memory overhead to my nodes and that was unacceptable. Usually the case was that they had one like really, really noisy piece of instrumentation that was kind of low value for them and then they turned that it off and it's like sitting at like maybe 1% to 2% overhead. There was another one with Python where we had a customer that was specifically like a thing that they were doing, they needed a custom what's called a span processor. It's a way that you can customize the way that open telemetry processes data and exports it while it's actually generating that data and they wrote something custom because they needed to add some specific contextual information and they felt that that was the best way to do it. But then that added overhead of about 30% or so to the nodes and so we worked with them to figure out, okay, there's actually a better way to sort of add this data and then again it dropped it back down to about 2% so I like you may very well experience some significant overhead, but I think there's a couple knobs available that you can sort of dial things back on and sort of get to an acceptable level. But that being said, because instrumentation creates overhead, there's, you know, in the open telemetry space we try to do as much as we can, especially by batching data up and flushing it asynchronously. But you're not going to be able to avoid some percentage overhead. Thank you. Great doc. Just wanted to ask you a quick question about if all my services are running on Kubernetes, is there any reason I would want to instrument my app for hotel if I could just drop an EBPF exporter or something like that? I would say yes. So I think EBPF is actually pretty radical. At some point EBPF might be the solution that we all get, but right now it's unable to extract any context from the actual application itself and it's unable to do context propagation throughout a given process. And so what it'll do is these EBPF agents will typically capture networking data or like if a database call was made and so some of the value of hotel auto instrumentation you could argue goes away if you could drop in a network agent. But if then you want to start being like, okay, well what if I want to attach a user ID to this stuff? Or if I want to attach a device ID or something like that, kind of don't really have that many options with these agents. The hotel operator accomplishes a lot of what that does by what it does is it adds an open telemetry collector into your cluster and then it configures all of your pods to send hotel data to that collector that's dropped into your cluster and it auto instruments each of them with whatever language you have. And so you'll get not only roughly the same shape data that you would get from an EBPF agent but often you can get actually much richer data by default especially if you're using Java. The Java auto instrumentation is frankly an incredible piece of technology and you can even get like some like method level instrumentation is automatically that are a very tall order for an EBPF agent to get in any kind of coherent manner. But that being said, I think this is a space to watch like things are gonna be evolving quite a bit. I would also say that like EBPF and hotel are not necessarily at odds or like necessarily different technologies. Both communities are like, we're certainly aware that we're trying to accomplish a lot of the same problems and we wanna try to make it so that like maybe someday the thing that you add to your cluster whether it uses EBPF or not is just kind of like an implementation detail. So you mentioned that span events were a way to tag that a certain thing happened at a certain time given that logs typically act the same way given event happened at a given time. Is there a way to turn existing logging into span events? And would that be something that should be at least should be evaluated? It depends. So yeah, so in particular one of our, one of the customers that I mentioned on market this is actually what they did is they basically wrote a wrapper that basically every time they're logging something with their logging interface. It actually under the covers just takes that same log, it just turns it into key value pairs on a span event. And so that same timestamp is, that's just the timestamp on the span event. That was something they had to configure in their code but it was a relatively light lift. It may vary from language to language if you have the ability to intercept method calls like that pretty easily. I think the answer is like kind of but there's not really a way to sort of like a configuration file that says like hey turn the logs into span events. And I don't know if that would necessarily be possible but I think also a thing that's worth looking into is this answer can also depend on the back end that you're dealing with. So this is kind of Honeycomb specific but what Honeycomb does is when you have an open what's considered an open telemetry log. So like it's the same log but it's been wrapped with like the spanity and the trace ID is we actually annotate it internally the same way that we do a span event so that it gets loaded up as a span event in our UI. So that if you've been creating logs as span events and you have these existing logs like it's kind of one in the same when you're actually investigating things. Now not every tracing tool is going to do that. So your experience might vary from tool to tool but that's kind of I said it depends but like it kind of depends on like two different ends of the end of the spectrum. So yeah. Thank you.


Good evening everyone. This is, I would call because I'm the moderator, interesting panel but it is an interesting panel because we are discussing live sports and marketing. We were discussing ice breaking session, we were watching that last over, right, the match, the super over, yes. So, marketeers are finding sporting platforms all the more attractive, you know. And with the coming of premier leagues and sporting leagues, it has become the playground for brands. So, I want to start straight away in the interest of time, Masai, that what explains this interest in sporting platforms from policy bazaar, bazaar perspective. What is that thing that creates that interest for marketeers? So, to be honest, it's not a new trend that marketeers find, cricketer or sports, a very lucrative and interesting platform to advertise on. It's been since time without beginning, I guess.  It's not a trend in India, it's an international trend. People have used multiple, be it NBA, be it Super Bowl. Everyone uses live sports as a platform to advertise and build a brand. But the question here is that why are people, why are marketeers continuing to invest? Like we were discussing outside, you know, we wish there was a cricket match every single day. Cricket match every single day, yes. Because nothing else gives you the kind of eyeballs that you would ever want to achieve. So, for keeping, to be honest, keeping our brand in mind, it's a difficult category, fintech and inshore tech. It's a very, very difficult category for someone to explain or to build a trust to invest in a lending product or an insurance product. So it's a very difficult category to explain, unlike a category which is bought on a daily, no one wants to buy an insurance product. No one thinks about, wakes up in the morning thinking about a lending product or an insurance product. So it takes so much more task to communicate the right message and also put it in the right platform.  And also, also fact that amongst all sports, cricket is the only sport which is very lucrative and conducive for advertising. I think it's made for advertising, somewhere I feel that, you know, you have like 50 overs or 20 overs exactly after six balls. There are breaks and then there is an also that, also the worry of attention span that, you know, a viewer might move out. That's also not there. You know, if there is an ad break, he has to stay because the next tour is going to be played. And then you have crunch matches like two tiebreakers back to back. And then, you know, it just takes the attention, so attention span is not a problem at all. Also, because the length of the matches are so big that you have a critical mass of audience, you know, it's actually coming back on the matches again and again. So it's kind of very conducive and single largest. So two things which work, which is television, which is single largest, reach medium in the country and also cricket, which is the single most watched sporting event in the country.  That's what you want. Build trust, build recall, and then have a great product when you're pretty much home. Aditya, your thoughts on this. What explains that unshakable faith in this live sporting platform? So see what Sai mentioned completely agree. It is not about live sports or cricket. If you look at other options, the ad breaks are anything between 8 minutes to 10 minutes. We all make a 30-second or 40-second, we eventually end up airing a 20-second. Right? So if it's an 8-minute break, you're talking about 24-25 sports.  You might be sandwiched between multiple sports. You might be somewhere, 14th, sometimes 27th. Unlike in cricket, it's either an exclusive break or a top ad or a bottom ad. There are two or three ads. So the ebb and flow of audiences is actually practically zero. That works. Two, the content which is available, which is of entertaining nature, is becoming very regressive. Yeah? While this is unpredictable content, people love to see what's going to happen. Exactly what happened today, back-to-back, two super overs. I was just cursing, hope this should have been on a weekend. I would have got more downloads. So the beauty of sports is unpredictability, ebb and flow is very less, and advertising becomes sort of a solo break or a one or two brands, which will then give you the attention span, which will give you the reach and which will give you that credibility if you're doing it over a long period of time. So the cliffhanger moment in a cricketing life spotting is far more interesting than a soap, you say. You want to add something? Yeah, so exactly what it was saying. I know I wish it was on a weekend, because by the virtue of being an online brand, and I think you'll also agree to it, that the measurability becomes extremely effective. So I've seen that whenever we advertise the product, which is a free credit score, I'm sure all of you would have seen, I've seen people coming on the site, and I've seen in our span around 10,000, 15,000 people who have actually taken credit score in the span of 15 minutes, and then in the span of one hour, around 20, 25,000. So in the span of one hour, when there were around three spots or two spots. So that is the kind of impact and measurability that by the virtue of being online when you're able to do it. And also there is so much data that is available to us that even if we have limited spends, we can go back and analyze which series to put more money on, which series to put less money on, which series might give us the right bank for the buck, not from the view of the TVRs or GRPs, but purely our own data. So that brings back the trust more on the platform. Tell me if I ask you that there's an unfair advantage to Cricket and Bollywood always, and is that also working in favor of sporting platform, especially Cricket I mean to say, or is it also the fact that you can have far more elements to play with in a live scenario or sporting scenario that works for the game? I think there's nothing much apart from, so news is also as good as entertainment these days. So if you leave that apart, there's nothing much apart in television to advertise beyond Cricket and Bollywood. Other than that, I would rate Cricket in sporting events slightly more higher because you get much more stickiness to this medium than any other. Sorry, what was the last second point? The number of elements you can play with in a sporting game. Also of course, live sport means someone is actually glued to you and some of the work that brands can do, which is integration within the television, within the live sporting event, it's unparalleled. It's beyond just spot-wise. So that kind of really helps immerse the consumer and the brand together. And something which I think we haven't done successfully, but I think Baijus have done it quite well. So how do you see the scope of having a lot more elements in this genre of advertising as compared to other platforms? I think we are forgetting one big thing over here, which is the broadcaster. Today's task force, the way they manage and the way they cover Cricket Match also adds to a lot of incremental opportunity for brands. He's not showing the Cricket Match on one singular channel. He's showing it across 12 to 14 channels. Earlier we used to have dub content. Now he's having live feeds going, commentaries happening in the respective language. So if you're looking at a Bengali feed, because I don't understand Hindi or I don't understand English, I can watch that Bengali feed and the commentators are your ex-Bengali cricketers. The amount of packaging which happens around Cricket, the pre-shows and the post-shows, the amount of analysis which happens, now all this thing adds to the experience which is just not going to happen when you're talking about Bollywood or movies because it's just a three and a half hours of movie with one and a half hour of advertising and if you have the money, you will get the first spot. If you don't have the money or incremental money, you will get the 25th spot. So it's also about the opportunity which has been created by the broadcaster and all the measurability which goes behind that. Someone was asking me that, is TV or television a dying medium? What's your view on that? My response was till the time there is live sports, television won't die. So that is the power of live sports. If you have to sum up the impact, the measurability power of it, after your association with the live sporting platform, can you take a little bit through what has been the impact and how policy bazaar has benefited from it? A little bit of a back story. So a quick back story which is always, if you have to invest in Cricket or any sporting event, budgets are always a concern. You have limited budgets and while we love to have a Cricket match every day, but there are limited budgets. So it's extremely important that how does one invest wisely when you don't have unlimited budgets. So what we've seen is that we started investing very small. I remember 2014 is when I joined and that's when we started. The focus was on starting building the brand and that's when the first 2014 was, 2015 was the World Cup in Australia. And at that point of time, you know, limited budgets and we said, you know, we want to be on Cricket, we want to build the trust and how do we do that? And we took the baby steps and we became a broadcast sponsor on Cricket World Cup but we did only high definition because the budgets did not allow us to be on standard definition. But at that point of time, the high definition, the base was around not even 5 million. It was less than that, like 2.5 to 5 million households. That was the base of high definition at that point of time. But it gave great results. Also because at that point of time, our base was low but we also saw that we could easily measure. Like for example, and then we started constantly investing in Cricket because we could see the impact. Now we've gone to the level of data analytics that I can tell you, not looking at the TVR numbers that how will ICC Seven Nation Tournament perform in terms of impact for me and what should be the right price for it. Within that, the cuts of, you know, how would a high definition perform? How would a standard definition perform? How will combination perform? How will bilateral series between India-South Africa and India-Australia perform? And how their price and what will be the impact on my site and should I pay that price or not? And all this decisioning is based on pure data and the power of investing in all the sporting events for so long. So how would a Asia-Cup perform versus a Seven Nation Tournament perform? So all this has done two things. One is help, increase of faith in Cricket, invest wisely and also definitely increase the market share. I'm sure if it was not for live sports, the recall of Policy Bazaar and Paisa Bazaar would never be so high. And we clock what, around 40 million monthly users for Paisa Bazaar and 30 million for Policy Bazaar. And it's a testimony and we were what, a six-year-old company for Paisa and an eight to ten-year-old company for Policy Bazaar. So it just builds trust and traffic and helps you invest wisely. So Ateetha, your impact story through sporting association. Can you tell us a little bit about the impact it helped you create and the numbers, a little bit of metric side of it. So as I mentioned, nobody knew us as we launched the product in India and then eventually slowly and gradually people started accepting e-learning as a concept. But today after four years of consistently investing behind live sports, what we see is that the friction of selling a e-learning concept month on month is coming down. Earlier, about two years ago, for example, we used to take about 90 minutes to convince a parent, explain them about the product and then eventually sell it. 90 minutes in a day with traveling time and you know how all the cities have been dug up. We are the fourth congested city. Bangalore is the number one congested city. Time taken to reach from one place to the other place is also 90 minutes. Yeah, finalists mostly that's what we're waiting for. Today if I look at it, that same 90 minutes has come down to about 70 minutes and now if I just extrapolate this across 3,000 salespeople, I'm getting more number of man hours to be on the ground and sell. Overall sales have gone up because time taken to convert the sales have come down and that's primarily happened because of the overall coverage and the visibility of the brand. So that's the number one impact and apart from that it's very clearly demonstrated in our data points which is our own data exactly what every tech company has. On match days, total number of download goes up anything between 2x to 10x depending upon the match. World Cup matches, the big World Cup matches 8x, 10x, 12x jump on the baseline of that particular day. Bilateral series depending upon which teams are playing, the uplift is different. India-Australia was a 5x, India-Bangladesh was 1.2x but it's still better than regular advertising. So we're primarily focusing on within sports cricket. But what about the other leagues that are coming up and getting good viewership? Are you also seeing an opportunity halves that also helped you consolidate the brand? I think cricket pretty much exhausts about it but having said that we have looked at other sports as well. But largely PKL. My only concern and PKL has been growing year on year and I think it's the second largest sporting event in India at the moment. But other sports according to me are not rightly priced for the kind of visibility they're giving right now. I'm very very excited about the women's World Cup which is going to come. I think if it's rightly priced it's going to be very very interesting and hoping India does well then. The last World Cup we were late into it but we took the semifinal and the way it was priced it did really well for us. But I think the pricing is a slight challenge for other sports but again you have premium brands for it. It doesn't work much for us but there are premium brands who look at Austrian Open or French Open or they look at badminton or other sports or golfing events where we are a very mass brand and we look at we want maximum eyeballs at the lowest cost possible. So if that is the objective then we kind of want to like focus on the platforms and the sporting events which will deliver that for us. So for Baijus how important are non cricket leagues I mean how do you see an opportunity in that? There is an immense opportunity it's only a question about how well we understand those sports. So now for example this year there is Olympics that is there is also Euro 2020. Most of us don't know that the Euro Cup is happening this year. Last year also was a big sporting year year before last was also a big sporting year with FIFA World Cup in Russia. So the opportunities are multiple it's also about how well you know the sport. As marketers as Indians we probably don't know how to use other sports. Having said that slowly and gradually football is taking shape. If you look at the ratings of what the domestic football league does in Kerala. Domestic football league does in West Bengal it is higher or at par with the World Cup cricket ratings in Kerala and West Bengal. It is much much higher than any of the bilateral series. So there is a great opportunity and then you have the opportunity to only take that market or look at those teams. So again knowing the sport and how to use that sport for your benefit as a brand is also equally important. Between agencies and the marketers none of us have much idea. And then channels also the broadcasters are also not too sure in terms of how to market it because it's always jammed between two cricket series. So then it's a leftover conversation if there is some savings available of a cricket match has got washed out take that money and put it over there. So it's a piecemeal conversation it's a very tactical conversation it's not strategic in nature the way brands are looking at cricket is much more strategic. But it's a matter of time slowly and gradually things will take shape. And again I would completely give the credit to star sports for the way they have started marketing even non cricket. There's another concern I mean I may be wrong but the rise of sporting leagues and all is there is over exposure now. You know there are too many opportunities and platforms. Is that a concern also now in this which was so called newer domain. Will that become a concern is this a concern already. What is it like. No I don't think so I think the more such platform because if you look at today's overall television industry scenario apart from sports and news and some few movie channels. It's not a very healthy it's not delivering the right numbers. So the more focus is there are more leagues and they are if they package it well the way he's been saying that they package it well and they promote it well and they strategically put it out there. I think this is only going to help television as a medium and also help marketers. I don't think it will. There'll be overexposure. Not a fair of. No it's not a question of overexposure. There are 30 days in a month on an average. Yeah there are about 700 or 750 channels which are showing content 24 by 7. Cricket is happening what four days five days in a month which is India cricket and 4550 days now 55 odd days which is three hours in the evening. So where is the overexposure. It is not about overexposure for sports to become big for every brand to leverage it. It's also about the players. We need to have superstars emerging in every sport. Today most of the teams are not a great buy because if you look at Sri Lanka as a cricket team there are no superstars five years of five years ago you had two of the biggest names in Sri Lanka. Bangladesh you don't have big guys. Pakistan let's not talk about it. South Africa is in shambles. So there are only two or three cricketing nations which has superstars. So as marketeers every morning I wake up and say please make more superstars superstars will drive eyeballs and eyeballs will then drive business. And you say one day one match every day right. That's what you wanted right. Because we have like what Manoj 30 news channels 50 news channels as Bach will report whatever it is you have some like 30 40 GC channels everybody's showing the same content. This is one channel showing three hours of entertaining content. So absolutely no overexposure and we should have more such content for businesses to grow. So as we move on from here also in the interest of time. What do you think in the marketing mix how important would the sporting pie become say few years down the line. How important would it remain also or where would it go. It will remain the largest. I told you it will remain the largest. I don't know any other answer. It is our percentage of the money spent on cricket is only increased year on year. So it will remain this way because the other donors are not helping themselves. No I agree because see again the ecosystem needs to work. You have a great broadcaster who's doing everything sports. As I said you need to have superstars in football in tennis in cricket. Yeah today we don't we will all watch Olympics. When PV Sidhu is playing a quarterfinal or semifinal or final match all of us will tune in. Twitter will go bananas wishing or all the best go India women go India all that thing will happen. But if we have such six or seven or 10 such superstars we will start following that sport. Hockey is not been doing well because there aren't any superstars. So it's the channel or the broadcaster is the super or the key players who drive the eyeballs and then the overall infrastructure. If all these three things go well hand in hand this is just going to grow exponentially. Exactly look at the way they built PKL like there are superstars in within PKL. I'm sure people who are following it remember the names of people who are coming from the teams. So it's really how they how a broadcaster also backs the content. This is my final question. Since you practice and evangelize post-marketing is there science to win the best practices according to you. What are the best practices for sports market here according to you if you have to just sum up one or two points. I think one thing extremely important is consistency. If you want to be on sports want to build your brand on sports consistency is extremely important. You can't take one or two series and then kind of you know think that you know it will do wonder for the brand. People who watch sports they kind of are passionate about that particular sport and also it gives you incremental audience. So one is you have to be consistent. We present on sports to be able to achieve a certain goal that you have in mind. And also do not worry about you.  At times you feel that no you're done putting a lot of money. The series does not do well. It's not in your control. So you one should not fear that you know you burn so much money and the result the series is not series tanked or in there not qualify. So therefore it is a waste money waste of money but one should not fear. If you have if you're investing money like that then you will not be able to build a brand with the help of sports. So it is high risk high return. So what is the takeaway so far been you know for your association with the sporting. What would you say what is the thing you want to share here. No no just have courage and patience. It takes a lot of courage to invest in a sporting event or a live event. And once you have invested the money just have the patience and the perseverance. It some will some things will not work but most of the things will work. And if you are then doing it consistently you will see that it's like a sip.  Same way with cricket do it for four or five years at the end of five years you will double everything. Some matches will do well some matches will not do well. I'm a good on time I have I can take liberty of asking two questions. Yes please yes raise your hands you can ask. Yes one yeah we can hear you. T20 how many number of spots are good enough for a one day match. Okay so let's let's look at T20 so there are 40 overs so 40 breaks on an average 15 wickets will fall. Yeah so there are 55 breaks. Now 55 breaks it all depends upon what is your exposure's frequency are looking at are you looking at a 6 plus 7 plus. So on an average most consistent advertisers they will look at a spot every eighth occasion. So there are 55 60 opportunities in a T20 match assuming 40 overs and 15 16 wickets. You're talking about 60 break 55 60 breaks divide by eight so eight to 10 spots eight to nine spots would do the trick. And similar way you need to look at it for ODIs but if you move to a test match then plan for four days because fifth day any which ways is a wash out. Yes one one sorry sorry sorry we will give more yes please ask yes. Like star sports is doing a great job in terms of packaging its sports and especially IPL coming down the line. How do you think with hotstar also doing that job for the major cricket series being covered there and a lot of people now they claiming a reach of 400 million plus and TV doing a reach of around 650 million types in the overall audiences the difference seems to be going down by their own quoted numbers. How do you think TV to digital in this case especially sports will happen for you guys and how will you take it. So I think what what hotstar has done is that the audience that were being missed by television is being captured by hotstar. So it's an incremental audience that is I'm sure there will be overlap audience as well but it's an incremental audience that is coming to you. If and if your question is on packaging or on my question is if you have those many money and you have distributed between TV and hotstar. Yeah so I think if there is a choice between TV and hotstar I would personally take TV but I think one should think about if you think about incremental audience because that is a similar audience that comes it is it should always be TV plus hotstar ideally because it is it is like right now we were watching on hotstar. I'm an audience so but if I was on if I was at home I would have been watching on TV. So it's always incremental audience so television got missed out say one individual. So it's always incremental audience that is coming to you. It is if budgets allow it should definitely be TV plus hotstar but then you do then you need to be smart that how much impression you need to buy on hotstar. If you're doing a broadcast sponsorship already on television then should I should you also do a broadcast like a complete sponsorship there or buy some impressions so that you're just present there. Then it becomes a how smartly you buy but I think put together TV and hotstar reaches to the maximum possible eyeballs. Last words from your side. But any like like he said you know every eight spot not just in this question but every eight spot can be should be the right way of looking at how what percentage how should you divide the money. How do you divide like what percentage should you put on TV versus hotstar. If you have bases of data if you have any science on it again it depends. So let me give a live example if when we were on World Cup with the broadcast sponsor on the World Cup on television. So our strategy was pretty simple that the we will take all in will take India and key non India matches which are on the weekdays and will not take weekends because weekends pretty much my audience is at home. So I would say the percentage wise it will be like this 20 percent on hotstar but it was a smart buy because already burning a lot of money on on on the being a broadcast sponsor on TV. So there is no specific thumb rule of 60 40 or 50 50 or 70 30. It depends on the budget you have and very how committed more. All right. Thank you. I have the queue that is over. Thank you so much for this lovely discussion. Thanks.


You know what stood out to me, you know from the moment that we got them in the beginning of July Throughout the summer is just how much these guys love the game, you know, they're obviously Extremely talented players. There's a lot that comes with them You know a lot of people talk about a lot of the attention But all in all when it boils down to it these guys are just hoopers and they just love the game They love the process to work. Kyrie's been in here grinding out every day Katie obviously can't play going through his rehab and all that stuff But they're still been a part of everything, you know team related which um, you know guys of their stature They don't even have to do that, you know, they could show up to training camp tomorrow That could be their first day reporting No, but they've been bought in from the moment that they become a part of the Nets organization And they've been wanting to get better compete making sure they're doing all the right stuff and Like I said end of the day these guys just love the hoop. They love the game Which is the biggest takeaway for me, which I find is awesome Ryan said it. You've had a busy summer Just take us through your experience with Team USA and the things I mean considering what you accomplished last season How would this summer will help take you to even another level as you enter this next season? Yeah, it was unbelievable experience You know long long summer We were part of the USA experience for 50 plus days So you get to know know all the guys pretty well know all the coaches well But you know all in all from individual sort of development standpoint You can't help but get better in that environment, you know, you're on some of the best coaches in the world best players in the world competing day in and day out You know all in all the experience was was incredible, you know didn't didn't finish necessarily the way that we had wanted it to But you know one of those situations where I'm gonna be bonded with these guys for life You know you go through something very unique and then just from a basketball standpoint you improve a lot You find out a lot about yourself as a player especially going through adverse situations like that But I think it's not gonna do nothing, but help out everybody that was a part of that experience Joe, what do you do physically to make sure you know your body gets the replenishment and needs after going through? What's a grinding summer like you did? I mean do you have to kind of take different? Precautions or tactics to get ready them for the NBA season. Yeah, so since I've been back. It's sort of I Guess a slower sort of tapered transition into training camp, you know A lot of the guys have been in here sort of ramping it up to get ready for training camp Playing five on five three four times a week I got back as more sort of recovery a little bit for that first week And then slowly getting more acclimated into playing playing, you know a couple games here and there But not the full go but as soon as training camp starts tomorrow I'll be full board everything will be going is just sort of that initial two weeks of just sort of you know Letting my body kind of recover from all the travel the games all that


So Breast Friends is about a group of female relay runners. So the head of the team is called Ash, she's confident, she's the best runner and a couple of days before a big race, one of the girls gets injured on the team and they have to call on this outsider called Adele to come in on to the team and Ash, the head of the team, she takes an act of disliking over Adele for no really reason, only that she's real confident and it kind of threatens her kind of status. So as the film goes on, the girls, all the other girls in the relay team really like Adele and they try to bring her into their group but Ash kind of doesn't really want it to happen. So one day in the change, when they're getting changed after practice, one of the girls invites Adele to sleep over and at the sleep over Ash's world kind of turns upside down and she begins to realise that she might have feelings for Adele. How would you categorise it, like would it fall into any kind of style or genre or was that similar to the United Team when you were riding it? Yeah I think the most important, the one in which I was kind of in for was coming of age. When I was actually writing I was going more, now it kind of shows how it kind of things change over time when you're when it goes into post-production but at the start it was like a comedy with bits of drama but then it kind of ended up a drama with bits of comedy and I guess yeah I wanted an LGBT. When I was writing it at first I kind of was more like when I was looking back over the first drafts of the script it's it's more only hinting at LGBT things kind of along the lines of handsome devil that kind of things where it doesn't go kind of full into things but I think I kind of in when I was riding it I kind of was scared to go like fully there but then I had a lecture tell me just like go for it you can't like think about what everyone watching is thinking when you're writing when you're writing so then I kind of went into LGBT genre. Yeah I kind of seen a lot of kind of coming of age things where say I feel I would watch a lot of coming of age with guys and then and it'd be they'd be very like nervous they'd be you know yeah they'd be naive and they'd be very kind of like finding themselves along the way but I kind of had I was kind of interested in people who were kind of dominant and high status people because I find that I think what high status people they kind of I don't know people kind of think that they don't have insecurities and they don't have proper problems because they are so confident on the outside and I think I wanted to explore the idea that like a real sporty person can be real sporty confident person can have like struggles exactly like you know like insecure and naive people yeah yeah and I kind of not in the actual with the actors who I had so basically when I was kind of doing script and so when we're in college we kind of go off and we go into some acting places that we went into the Meissner and people in town and so I'd bring my script and I so I brought the scene like the touching of the boobs scene I brought that one to them and I kind of went in really naively and with these types of things you have to be very you know almost you have to kind of like talk like a doctor to people you have to be confident in yourself to say okay now you're gonna touch this person's breast you know you have to be if you're kind of like oh am I now you're going to touch someone's poop and if you're like that then they'll get anxious so I kind of gone into this thing and I was really anxious and then it just went completely right I just went like wrong when I was doing this rehearsal and then I kind of was like oh this is gonna be this is gonna be really hard so I kind of went off and then I kind of was trying to figure out why I was kind of uncomfortable in myself talking that way but then I'd gotten advice by someone and to say like talk to people like as if you know how like when like a doc if you like a doctor's telling you about something about yourself they're very like matter of fact and I kind of took that upon myself to be if I'm confident in myself what I'm saying they'll be confident so when I actually went to go film those like real sexual that them scenes it literally it was just so easy that I I was kind of stepping around stone I was stepping stepping around being like okay so now you guys are going to they're like oh okay this is grand and they all were so confident in it and I I'm like it was amazing to see like the journey of it because I think if I didn't have that time where I was kind of struggling with it it might end up like helped you know with the end product and but yeah it was tough it was tough at the start and but I feel like it was more tough on my end I feel like it wasn't tough for the girl like well in my hopefully wasn't tough with their girls but yeah it was really fun that was like my favorite scenes really to all them scenes for my favorite to do really yeah yeah yeah I think so and I think it's kind of changing now even in the last like like like six months kind of going on year and and yeah so I kind of felt that sometimes I don't sometimes a female sexuality to be very like won't show things I'll be very off-screen and and what I kind of wanted to do at restaurants was that I wanted to really like even when I've seen it like being screened the minute when they're all like touching each other's boo boo's it's like people are kind of like oh she went that people weren't think might not think I was gonna do like it's gonna come up you know and I think sometimes it can make people feel a bit awkward and you know bit like you can kind of hear everyone and I love that I don't I kind of like making people feel a bit awkward because I feel like it kind of it's like no it's it like I feel like in film everyone always thinks that like oh there's happy and then there's crying but like you know there's other kind of emotions that I feel like it's great for an audience to feel and I think like anxiety and kind of feeling anxious is really kind of yeah yeah yeah yeah that's my yeah no so and so we premiered at Gaze so that's like the LGBT Q plus festival and I'm actually so glad it went there first because I was great seeing like reaction among your peers and stuff and I know yeah it was it's kind of mad then we went into Cork and that was kind of it was like sometimes you have like good kind of audiences and you know maybe you might not have a lot of people in the theater and some of that sometimes you can be like oh but then you have a good one again then you have you know so it can kind of like change each time but it was like it's amazing scene and then we went to I went to London for Underwire which that was like do you know sometimes like when you go sometimes I feel like you go to things you just assume that people won't be there but then like it like had sold out and it was in BFI South Bank and it was incredible just like oh got it like a full like you know full theater I don't know I kind of stuck a shock to those things I know that's kind of yeah yeah yeah yeah yeah yeah it's amazing yeah yes Australia yeah that was on a Gaze on tour so and Gaze do like this program where they like travel with the program that they had the Irish shorts and so then I went to Australia which is my and I got like messages like obviously I can't go to Australia but so it got like I got like messages over Instagram people like seen it as well which is kind of crazy so it's that's where like kind of online stuff is unreal because like you know you can kind of see how it goes down and where you can't get to go you know yeah no it's fantastic


A recent economic impact study on the Minnesota State System revealed that Bemidji State University and Northwest Technical College have combined for more than $300 million for Minnesota. Reporter Cheryl Moore breaks down the numbers and tells us what went into the study. When you're thinking about the economic impact and you think about the dollars that are invested into higher education, into Minnesota State, for every one dollar that's invested, the return is $12. Bemidji State University and Northwest Technical College impact the region in numerous ways, but thanks to Minnesota State, we can now equate that impact to a number, $321 million plus 2,700 jobs. We meet as presidents on a regular basis, on a monthly basis, and one of the things that we talked about was making sure that we were telling the story about Minnesota State and what the impact of all of the college's universities across the entire system was. The $321 million combined impact is broken down by three factors, direct effect, indirect effect, and induced effect. When you think about the partnerships BSU and NTC has with the local businesses surrounding tribal colleges and Bemidji High School, the impact is crystal clear. We have students who are continually volunteering. They might be out raking leaves, they might be out helping with a project in town, so there are plenty of opportunities for students to give back and contribute to the community. The goal of the whole Minnesota State study was to provide a full, incredible assessment of the total economic and employment impact. That way contributors know exactly how their investment helps Minnesota. I think that's a really important point when you're speaking to legislators and lawmakers to really say, here's what we're contributing and now how might that impact what you feel you can invest in us. Reporting of Bemidji, Shawmour, Lakeland News. The last economic impact study was done in 2013. It's possible another study could be done in three to five years. If you enjoyed this segment of Lakeland News, please consider making a tax-deductible contribution to Lakeland PBS.