Let's start our session about troubleshooting.net application. Welcome, everybody. My name is Sergey Mityukhevich. I'm a cloud engineer from Alturas. I'm sure you have probably already seen this slide. But for those who didn't, please review it, because it contains important fire announcement. But while you're reading, let me very quickly introduce myself. So I started as a .NET developer 70 years ago, but then I moved to DevOps stuff completely. And I can claim that I have seen the platform from both points of view as a developer and as a DevOps. And my current topic is closely related to both of those experiences. So today, we are going to talk about efficient ways to troubleshoot .NET application deployed to Cloud Foundry. I can split the topic of troubleshooting .NET application into three broad areas. So first of all, we're going to talk about how to utilize logs. We will see how we can use structured log into efficiently log messages and help platform to aggregate those logs and present them for us. We will talk a little bit about monitoring as well, how we can use monitoring to notify us if something goes wrong with the platform. And finally, we will talk about debugging and remote debugging of .NET application deployed to Cloud Foundry platform. A few important things that you need to know about logging in Cloud Foundry and this is related to most Cloud platforms. So first of all, your application is not supposed to log anything into log files. So instead, you should print output of your logs in the standard R standard out of your process. And also, it is important to know that a platform is responsible for aggregating and handing logs for you. So later, you can consume those logs and utilize them and use some log aggregation frameworks that I'm going to show you a little bit later in my presentation. Another important concept that is related to logging in Cloud application is called structured log. So what does it mean is that usually you don't just print some string message as your log message. You attach some important information to each of your message. And this allows later to very efficiently sort your messages, filter your messages, and drill down into an error if you had one. And later, I will show you how we can utilize facilities provided for us by .NET Framework to implement such kind of logging. Here is the code that does this for a sample ISP.NET core application. So the top code snippet is responsible for configuring logger. So here, I'm using SeriLog library to implement structured logging. But that's not as important because almost all log libraries, like nLogs, for example, or log for net, implement similar kind of features. So here, we configure default log level for standard libraries because we don't want to be overflown with messages from Microsoft namespace and system namespace. But the most important line here is the one where we configure that we are going to write our message to console. So that's essential for cloud application because, as I said previously, we don't want to log into files. We just output everything to console. And also, we are using JSON formatter. So if you use JSON formatter, this allows you to wrap your messages in JSON object and attach fields or text to each message that later can be used to search for logs. And second code snippet is configuring the application itself. And the most important line here is the one that says use SeriLog. This is extension method that actually applies previously configured logger to current application and allows a speed.net core app to use this logger. Next, after we have our logger configured, we can try to utilize our logger. But what we really want to achieve is to transparently add those kind of texts that I was telling you previously to all our messages. We don't want to attach request ID and user ID each time we log in something. And that's where middleware comes into play. This is a very useful method to implement some functionality before and after your request is processed. So here you can see that we are writing some string to log before the request, after the request. And most importantly, we use beginScope method of isp.net core logger. And this method allows us to create custom scope. What does it mean is that it creates a new field that is going to be attached to each subsequent message in this scope? By default, isp.net core creates scope for each request and it attach such fields as request ID. We will talk how to very efficiently use request ID later. But nobody prevents you from creating custom scopes. And it's not necessarily should be the case that you should wrap only request into custom scopes. You can wrap any other functionality into custom scopes as well. So if we are talking about custom scopes, a few important things that I want to highlight. So first of all, scopes allows you to make your logging transparent for the code. So you don't attach user ID and scope ID each time you log in something. Instead, you use such facilities as custom middleware or custom filters to do this job for you. And secondly, it ensures consistent logging format so you know that request ID is going to be attached to each message. And then you can utilize this when filtering for your messages. And as I said previously, you can wrap any other kind of functionality, for example, request to database transaction. Database transactions can be wrapped in a custom logger scope if a lot of work is going to be done during the processing of this transaction. Let me also show you a few fields that I usually consider to be important when talking about log. So first of all, if we are talking about custom scopes, so usually we attach ADs that later can be used to filter our messages. The most important one is request ID, but also it's important to attach user ID, maybe thread ID if you are doing some multi-threaded processing during your action method execution. But also, those fields are used to filter your messages. But also it is important to trace everything that goes inside your application and goes outside it. So what do I mean by that? So it's a good practice to log the content of your request because later you can reproduce this request. And if you rely on custom headers, for example, you can log completely the whole request. Also it is important to log everything that you send to external system, and you can consider your database as external system. So you can log your SQL queries, and later you can analyze performance and find bottlenecks when analyzing those queries. And if you talk to some external system, it's also useful to log this. And the main idea why I pay so much attention to this is because this login can be implemented in a consistent way using such techniques as I showed you previously with custom middleware. For example, you can use your ORM to attach some login information before each request and after each request. OK, at this point of time, we have our login configured for our simple application, and it brings something to standard output and call finder is responsible for collecting those messages. Now we need to talk a little bit how we can utilize this and how we can filter our messages and search them. So in my presentation, I'm showing you screenshots from applications that is called Kibana. And here I'm using a very popular stack that is used very commonly with Call of Foundry that is called ELK, Elastic Search Log Stash Kibana. So this stack is responsible for taking messages from Call of Foundry, storing them. And later, Kibana, this is the screenshot from this application, provides you a way to see your log messages. And here you can see that we have three messages, something that two messages from our custom middleware. And the important thing here is that our custom field is attached to all of the messages. And we also see that request ID is attached here. Very briefly, I want to explain how ELK stack works. So Call of Foundry by itself doesn't provide you any possibility to efficiently search for logs. Instead, Call of Foundry just provides you a way to grab all logs from the system. And it should be a responsibility of some other system to collect those logs, store them, and provide search and filter possibilities. So the application that collects logs from Call of Foundry is called Firehose Nozzle. Then logs are sent to intermediate buffer. Usually we use radius for this. Then there is a component that is called Elastic Log Stash. So Log Stash takes JSON strings from an input. The ones that we have been logged previously and converts them to an object. And those objects are stored in Elastic Search. And later, Kibana can talk directly to Elastic Search and take log messages out of there. This is very briefly how it works. So now I want to show you how we can use these to search for logs in case if something happens. So for example, let's consider a very common scenario. We have an error in our application. So first of all, we find our error message with stacked rays in Kibana. And then we can use request ID field attached to this message. We can use this request ID to filter all messages that belongs to this particular request. Then we can analyze what parameters have been sent to us. We know which user was in context of which user our request have been executed. So we have basically all necessary data to fully reproduce the request if we need. And usually this should be enough to troubleshoot any kind of error. And from my point of view, a good way to say that, OK, I'm doing login right for my platform is that if you can troubleshoot any issue just by using logs, then you are really logging enough information. And you are really logging information that is necessary for you so I'm not overflowing with some unnecessary information and that doesn't help you during debugging your application. Next, I'm going to talk a little bit about metrics and how we can use them in Cloud Foundry and utilize metrics from .NET applications. So if logs are mostly used to troubleshoot some issue, like in the examples that I discussed previously, we have an error and then we use logs to troubleshoot the issue. Metrics usually are used to send a notification that something is wrong with your system. Like disk utilization is too high or memory, we are going out of memory or we're having too many requests and our application cannot keep up with them. By default, Cloud Foundry provides you a possibility to access basic metrics such as, for example, disk utilization, memory utilization, number of requests, average time of processing the request. And those metrics are very useful. And if we are using PCF, there are recent applications that is called PCF metrics. And those metrics are available by default. And you can just open the dashboard and see how disk utilization changes over time. But that's actually very easy to use. But what I really want to focus on is how you can create your custom metrics. Creating custom metrics can be very useful because obviously, Cloud Foundry doesn't know anything about your application business logic. It doesn't know when your customer logs in intersystem or when some important event happened, when an error happened, and so on. So it's usually useful when designing your application to came up with a few important metrics. And here I'm showing you how you can very simply send value for some metric. And I'm using Graphite for this. So Graphite is a time-serious database that can store values for some metrics. And later, we can use other software to visualize those metrics. So let's see how it works, for example. If we, this is a diagram taken from one of products that is called Harbit that is doing exactly this. It hosts time-serious database and connects to Cloud Foundry and grabs all default metrics out of the system. By default metrics, I mean metrics that came from virtual machines where our Cloud Foundry is hosted. I also mean metrics came from the platform itself and from all components of the platform. But also, there is a way for your application, like I showed you previously, to send metrics directly to this database. This is very useful if you want to compare two metrics. For example, you want to see how disk utilization changes when a number of login attempts grows over time, for example. Or you want to correlate two metrics, one of which is application-specific and other is platform-specific. And such kind of solution provides you a way to do this. So all of them are stored in time-serious database. And we use an application that is called Grafana to visualize those metrics. So actually, on my previous slides, there's a sample of graph that can be built in Grafana to visualize some metric. And the last functionality that is very important is alerting. So if you have some metric, usually you want to be notified when the value of this metric goes beyond some threshold. For example, if CPU utilization goes beyond 80%. And usually, monitoring tools allows you to use logic to say, OK, when this event happened, I want to email to be sent to system administrators. And this interface shows you an example how we can create such queries and how we can use value of particular metric to notify user. So same as for logs, I want to talk a little bit about what kind of metrics I consider to what kind of metrics are important to be monitored. So first of all, default metrics that we talked about previously definitely are very important, container metrics, CPU memory, disk number of requests, request latency. If we are talking about application-specific metrics, from my previous experience, I can say that the most important one is database metrics. Because I have seen a lot of times situation when application works well in development, but as soon as we push it into production and put it under a load, the processing time goes exponentially. And that's where monitoring can help you because you can see, OK, request time goes up and we see the number of database requests. We can relate those two metrics. And if we see that a number of database requests also grows exponentially, this might indicate an error in your application. For example, one very popular error is N plus 1 request. When your application doesn't use SQL to get all information it needed in one round trip to database, but instead talk to database iterate over all items in some database. This is very common and it's very easy to see when you compare a number of database requests with a number of items in a particular table. And finally, the last topic for today is how we can use remote debugger in .NET application. So I want to tell about this on using ISP.NET core application as a sample deployed to a Linux container. And I will talk about Windows in the end of my presentation as well. So this diagram illustrates how this process works. So inside Cloud Foundry, we have a .NET process running. And in order for .NET debugger to be able to attach to your application, you need to use special software that can help you. So in case of Linux containers, we use VSDBG application that should be installed in the application container. And VSDBG is responsible for monitoring your process. And then VSDBG talks to debugger and actually is responsible for executing all debugger commands. In order for this to work, we need a special file that is called LungeJSON. And this file can be used to configure debug in VS code. It is possible to use LungeJSON in Visual Studio. But in Visual Studio, there is no native support for LungeJSON. So you need to type a few commands from command window if you want to do this. But if somebody is interested, you can connect to me after presentation, I can show you how it works. But from VS code, you just create this file. There is a native support for this. And what's important here? Important thing here is how we connect to remote debugger. Do you see that we use CFCLI? There is a command. So it's called pipe program. It's called CF. We use CF and then pipe arguments. So we use CF SSH to go inside container. Then we specify name of our application. And dash C means commands that we want to execute inside container. And VS code attach actual command that launch VSDBG and attach to running .NET process. In order for this to work, obviously, we need VSDBG to be installed alongside our .NET app. And here is a sample of a Docker file that can do this. So what is done here is that we are using two containers. The first one is used to build our app. We execute publish command here. And the second container uses output of the first one. And it runs our application. This is done in the last line. Here you see that we dynamically create URL to run our application. And we also install VSDBG alongside with some other packages that are required for VSDBG to run inside container. So after you've done something like this, this is rather simple Docker file, what you can do is to just use normal CF push to push your .NET application into a Linux container. And here you specify name of your container. You should put it in Docker Hub or your private Docker registry. And after you've done this, VS2Diacoat will show you a dialog similar to this. Here you can see all processes running on inside remote container. So you can just pick the .NET process, the ones that you are interested in. And basically you are done. You connect it to the application that is running inside container. So on Windows, when doing application dodges previously, most of the times what we did is to use some hacks to get inside VMs that runs on Windows. And usually we deploy a separate VM, debugger VM, in the same network. And then I install Visual Studio on this VM and use RDP to connect to this VM and just use native Visual Studio debugger to debug our application. So Cloud Foundry was not involved. And actually, this is not a cool solution because you need to access your operation team to help you with this. Just as a developer, you can just deploy VM in the same network to the same network where your Diego cells are host. But I advise you to attend tomorrow's keynote because tomorrow Piotl is going to announce support for native debugger on Windows. And that's a very cool feature. And you can use Visual Studio to attach to any remote process that runs inside Cloud Foundry. So I'm very excited about this announcement. I definitely want to check out how it works. So I hope they'll show us a live demo on this. And a lot of things that I want to tell you is that actually, sometimes it's forced to use debugger because you don't have any other way to troubleshoot an issue. But if you have to rely heavily on debug, especially in production, this might indicate that you have problems with your architecture and problems with your login infrastructure. Probably you are not logging enough information and you can troubleshoot it. The problem with debug is that it's a spent state. So actually, you cannot debug application, like live applications that works under loads that because as soon as you attach to a process, the process is stopped. And another problem is that we cannot use debugger for post issues. If somebody complains about your application and you cannot reproduce the issue, you basically don't have a lot of opportunities to troubleshoot it using debugger. So in such situation, logs are much more helpful for troubleshooting. And if you want to see how all this work in practice, you can check out my repository that implements the sample application that I have just shown. It's very simple default.net core application that has all those features that I have just shown you. So you can clone it and play around with it if you want. So basically that's it that I want to share with you. And if you have any questions, you are most welcome to ask them. I will try to answer all of them. You mean debug during development? So not log as much as possible, but instead I recommend to when logging something, just have clear picture in mind why you are logging this, in what case you want to log this. So I recommend logging such scenes that can't be logged transparently for your application, such as I mentioned previously, all request data, all database data. It ensures consistent logging format. But I cannot say recommend log as much as possible because I see a lot of times code like this, like one line of code and okay, we are calculating parameter x, then we are talking to database. And this makes your code less readable. Instead it's much more useful to have a wrapper that will log everything that you log to database and get rid of this login code from your business logic. So that's how I see a good debugger implement, good login implementation in the application. Dynamically turn what? Yeah, that's a perfect question. Yeah, so most of the loggers support doing this and one common way of doing this is provide you a logger configuration that externalize from your application. Instead of doing the things that I have been doing like hard code, the logger level, you can use JSON file to configure your logger levels for different namespaces. And also some logger libraries support configuration based on environment variables. So you can set environment variable, restart your application. If you have a lot of instances, then they will be restarted one by one. So there will be no downtime and you can dynamically adjust login level. But important thing here is that, okay, if we are logging a lot, how we can avoid impacting the performance of our application. So first of all, in your log aggregator, in such tools like ELK, you can set up a special process that will clean up all logs and you can set up policy. For example, all logs, older than one month should be deleted. Or for example, we keep error logs forever and we delete debug logs as soon as one week, for example. And this can be very useful. But in terms of slowing down application itself, well, I don't think it has very like extremely large impact on application because a lot of logger libraries introduce intermediate buffers. So instead of just writing each time to log file or to standard art, they log everything into file and then flash it periodically. So I never seen problems with application performance because we are logging too much. All right, anything else? Questions there, yeah? All right, thanks everybody.


Ellen, we're here at Mechanics Hall in the ballroom with some brand new banners you've created. Tell us about them. Well, I have here a baker's dozen of banners, 12 plus one, that have been created in various ways. Most of them were done in collaboration with some of the worker groups here in Portland, both formal associations such as labor unions and informal groupings of workers. USM students were also involved in making many of these banners. This has been over about a year and it started because my father wrote about the original 1841 banners in the early 1990s, I believe it was. And he described to me the way these early banners were paraded on Congress Street and the visual image of that was just so striking to me that I have kept it in my mind for many years and this opportunity to recreate or create a new series of banners that are inspired by the historic ones seemed to be the right time to do that. The historic banners represented different groups of makers here that were part of the mechanical associations, is that right? Yes, they were the cobblers, shoemakers, coopers, shipwrights, printers, those were the original banners. These banners are in some cases of those same groups, the coopers for example, there's a coopers banner, there's a luthier's banner, there wasn't an original luthier's banner but we do have a luthier's banner this time, there's a baker's banner, but some of these groups are not some of the original banner groupings but they just happen to be the groups of people that found me and wanted to work with me to do this. And what do these banners tell you, what kind of story are you trying to convey today and perhaps going forward? I'm wanting to create a thread with the past, I'm honoring the Mechanics Hall and the association, wanting to make sure that Portland reserves a place for the diversity of workers, different kinds of people working very different kinds of jobs, I'm hoping that that continues as part of Portland's future and I'm also hoping that Labor Day could possibly incorporate in the future a celebration that brings out all different kinds of working people in Portland to the streets and provides an occasion for them to display pride in what they do. And we're here on Labor Day weekend, the Friday before Labor Day weekend, what's your plan for this evening? To march these banners down the street. Between here and Monument Square, we're actually going on the sidewalk, but we have a nice wide sidewalk between here and Monument Square, so we may not make it all the way to City Hall, it depends, we haven't had much of an opportunity to rehearse actually carrying these out in the elements, so this year, this time, it's going to be a short parade from here to Monument Square and back. And maybe in the future something bigger? Maybe, hope so.


This video is brought to you by my super hot patreon supporters I was really into Goosebumps as a kid for my 10th birthday I received a replica of the haunted mask from the book cover. I threw together my entire costume to accommodate it I was going for a suburban Halloween special vibe, you know There was a lot of nuance going into it coming from an 11 year old I took to the streets with my little brother and our parents and I found it that none of the other kids in my Neighborhood knew who I was they didn't read the Goosebumps books like I did It was almost a complete bus until one of the parents answering the door freaked out. They're like, oh my god I used to read those books as a kid and I was like, yes It made the sweaty uncomfortable stinky expensive mask worth it Just one single comment made my entire night another year I went trick-or-treating with a girl I had a crush on and we went to the rich neighborhood one house had the entire front porch tarped off There was a giant crowd of kids huddled around it.  Everyone was too scared to approach them and get candy I wanted to impress the girl I was with so I decided to just march right on in kids held my hand as I slipped Deeper into the porch and I reached for the candy bowl then the figure slumped over It was a scarecrow and no one had any real reason to be spooked. Hey guys. Look. It's not real. It's just a doll Every year we'd stop at my uncle's house and give us some weird stuff One year he gave me and my little brother some toilet paper another he gave us an ear of corn One infamous year he handed us skin deer legs from his hunting trip The year that I took that girl out. He gave us waffles. I gave her half of my agos She was having like this contest with their older brother to see who could get better loot The Ago waffles served as a tiebreaker. So she won that year My freshman year of high school was the last year I still felt like a kid And I went trick-or-treating with the girl I had the biggest crush on ever We went through her neighborhood, which was pretty much in the middle of nowhere There weren't a whole lot of houses and many of them just sat alone on weed over properties with the lights out The moon peeked out behind the trees bare branches. There was a lot of walking along dirt roads The streets were mostly empty that year because it was like on a friday And everyone was at some party or something like that orange street lights straight cats Just hanging out and talking and stuff We walked down a super isolated backwards path to get into another neighborhood And we could have sworn that we heard footsteps behind us when we panicked Although to be honest, it was probably nothing, you know Not many houses were open that year actually We just went over to her house and watched the conjuring and I remember thinking it was remarkably Unscary based on what I heard about it all things considered. I think that was probably the best halloween I'd ever had the following year I was just too cool for trick-or-treating. I went to a concert in Boise I saw one ok rock or one oak rock or however you pronounce it. I don't care neck deep We're actually doing a once-in-lifetime performance as their alter ego slip neck sleeping with sirens in all-time low After the concert was over while everybody was leaving they played the Marilyn Manson version of this is halloween over the speakers and the entire crowd Stopped what they were doing and they started singing halloween 2016 I hung around downtown Boise with a new friend of mine at the time who walked through some record stores We made our way through the spooky old neighborhoods in town He had a costume on he was dressed up as 11 from stranger things. I did not have a costume I didn't get the memo. He wanted to go trick-or-treating, but I was like I'm in my street clothes. I can't trick-or-treat I'm 17 like no one likes 17 year olds trick-or-treating. I don't want to add insult to injury by not dressing up So we found a compromise. I just walked around. I played ukulele at people's doorstep and they gave me candy It was actually a lot of fun and an amazing memory I guess this video is a little bit weird because I don't have one great story of trick-or-treating and halloween and stuff For me halloween is just a collection of images Scenes events and people sneaking around the backwoods of small town Idaho in the middle of the night with my Crush under a giant moon thinking that a real-life serial killer is hiding in the bushes besides me That's halloween to me exploring neighborhoods making friends. That's halloween too old family traditions halloween I'm a legal adult now. So I can't go trick-or-treating. I can't sneak around neighborhoods in middle of the night on halloween Because that would kind of worry some parents I can't do a lot of the things I used to do and that kind of sucks I used to be bummed out about it, but now I kind of had a bit of an epiphany You see I'm not on the receiving end of halloween anymore. So that means that it's my responsibility to be on the giving side It's now my job to take everything I loved about my halloween memories And help the next batch of kids create their own experiences too I am free to set up tarps on my front porch in a strobe light and jump out last minute I can give any future nieces or nephews rolls of toilet paper for halloween. The possibilities are endless, you know There's so much to do if you're watching this and you're young enough to participate in halloween and trick-or-treating and stuff Enjoy it spend some time with friends going on adventures Take a dark and spooky path to get home if you're my age and you're too old to finally do that stuff Use your imagination and see what you can do to make every halloween memorable for somebody else Even if you have to scare them have to death and make their parents mad They'll look back on it and smile when they're older. Have a happy halloween. Be safe. Be smart and stay spicy Also, I got some patreon shoutouts to do Otto Zaguala wants to listen to rock with your grandma Harry Phillips who's using their shoutouts for self-promotion wants you to go listen to their awful cover of the middle by jimmy eat world Link in the description by the way. Also special. Thanks to daddy cthulhu ollie a frankie blue good gent frankie Gabby taylor. Hmm and spencer persil these kids are the best and i'm super fortunate to have the good vibes on my patreon You can join too by clicking the link this this one right here. Happy spooky season


Jennifer lives with her dad and her dad kind of runs his pet store. Make sure my supper's all ready!   She's on a mission to make Jennifer's life a living hell.  Her dad is really into quoting scripture. You have this power, you must learn to harness. She was in sort of this snake handling church. They start to torment Jennifer more and more as the school year goes on. Does start to harness this power that she's got. Get a big payoff at the end with some effects driven scenes. Supernatural revenge type story, which is very kind of Carrie. But prefer this over Carrie. And that's saying a lot.


Tezal London Take your Tezal course in London, England, and experience the excitement, culture and history of England's capital and most famous city. London is an ancient city whose history greets you at every turn. To gain a sense of its continuity, stand on Waterloo Bridge at sunset. To the east, the great globe of St. Paul's Cathedral glows golden in the fading sunlight as it has since the 17th century, still majestic amid the modern towers of glass and steel that hemmed in. To the west stand the mock medieval ramparts of Westminster, home to the mother of parliaments, which has met here or here about since the 1250s. Past them both snakes the swift, dark temps, following the same course as when it flowed past the Roman settlement of Londonium nearly 2,000 years ago. If the city contained only its famous landmarks, the Tower of London, Big Ben, Westminster Abbey, Buckingham Palace, it would still rank as one of the world's top cities. But London is so much more. It is one of the world's three largest financial centres, alongside New York and Tokyo, with a dominant role in several international financial markets. The city is also famous for its west end entertainment and shopping district, with locations such as Oxford Street, Leicester Square, Covent Garden and Piccadilly Circus attracting tourists.


Create a placeholder, easy and quick. Type equal sign, la ram, close parentheses, then press enter. The program will create by default, five LaRam-Ibsen paragraphs. If you want to specify the number of paragraphs, type the number you want inside the parentheses, a 2, or 3, or whatever you need, then press enter and see. Also try equal sign, ran, close parentheses, and press enter. Now you can see five random paragraphs. If you want to establish the number of paragraphs, then type the number inside the parentheses. Wow, that was fast! Use these tricks in your next Word or PowerPoint project and see how fast it is.