We all know how expensive medicines are, but how costly can it really be to make a pill? To answer this question, we need to understand the path of a medicine from the bench to the bedside. You may have heard the numbers. It takes an average of 12 years and 1 to 2 billion dollars to get a medicine to patients. Well, let's break these numbers down by following the journey of a molecule to becoming a medicine. Today we will not talk about the distribution of a drug, but instead we will focus on the cost of developing a medicine. It all begins in the basic science laboratory where scientists study the underlying biology of a disease. What changes to a cell or tissue causes this disease? And can we design a medicine to stop or reverse the effects of the disease? This is called the discovery stage. At this stage, thousands of compounds are screened, often in a petri dish or test tube, to determine if they have the desirable effect. One in a thousand medicines tested in the discovery step proceed to the next stage, development. If the medicine looks promising, the company conducts experiments in animals to determine the safety of the medicine and proof that it works. In the development stage, scientists also study how the medicine is absorbed, broken down, what is the best dose to give, and how to administer it. At this point, an important decision is made. Is there enough evidence of safety and efficacy to test this medicine in humans? If so, the company talks to FDA and gathers all of the information known about the drug to file an investigational new drug, IND application. Of course, a human will respond differently from a mouse, so clinical research requires lots of thought and planning. This is when things get complicated if they haven't already. Researchers at companies design and conduct clinical trials each with a specific purpose. Phase 1 clinical trials take about a year, test the safety of the medicine in healthy volunteers at low and high doses, and cost about $1-5 million. Phase 2 clinical trials take 2-3 years, test the safety and potential efficacy of the medicine in a small group of patients, and cost about $10-20 million. Phase 3 clinical trials are the most time-consuming and expensive, costing over hundreds of millions of dollars. They take 3-5 years and assess the safety and efficacy of the medicine in hundreds or thousands of patients. Keep in mind that sometimes multiple trials are needed to test a single compound. One in ten medicines tested at the clinical trial stage successfully completes all three clinical trial phases. Throughout this time, the company communicates with FDA about the safety results and the design and progress of clinical trials. At this point, the researchers can file a new drug application, NDA, which FDA will review. This application consists of everything we know about the medicine, every single piece of data from the discovery stage to phase 3 clinical studies. The company has to pay a large fee of a few million dollars to FDA to make this review possible. The review process takes up to one year, at which point FDA decides whether to accept or reject the application. If the application is accepted, the medicine can be marketed and sold in the United States. Now 12 years and 1-2 billion dollars later, one compound has been FDA approved because it was shown to be safe and effective for the treatment of a disease. Only one in every 5,000-10,000 medicines tested makes it to this point. The others failed because they were not safe or effective or didn't have the required properties. The cost to study each of them can only be recovered through the sale of the medicine that is approved and marketed. In order to fund the development of more treatments, the approved medicine needs to generate the revenue that can be used to support research on thousands of other compounds for the treatment of diabetes, colon cancer or cystic fibrosis. As you can see, the cost of developing a treatment can be very high. It includes millions of dollars spent on chemistry in animal studies, hundreds of millions spent on clinical trials, and the amounts not recovered from the 9 out of 10 drugs that are tested in humans but fail in development and never earn any revenue for the company. Now our journey takes us to the bedside, where a cancer patient is receiving an FDA approved life-saving therapy that they know is safe and effective to treat their disease. Looking back, this process is complex but worthwhile. A thorough and trustworthy investigation and review of the drug's properties ensures that the benefits of taking the medicine outweigh the risks. This rigorous process keeps patients safe. And the safety review continues as more patients take the medicine and more is known about its risks and benefits.


What is up, you guys? Welcome back to another one. If you're new to the channel, I am Goldpony, a new car truck SUV reviews on YouTube. And today we're in the brand new 2023 Nissan Frontier SV, courtesy of younger Nissan in Frederick, Maryland, for more information on their inventory. Please feel free to check out the link in the description box below. So today we are in this one because it was recently redesigned last year for the 2022 model year.  So ultimately in this video, we will be testing out and going over everything about this one from acceleration to braking, steering feel, ride quality, sound system, exhaust clip, all that fun stuff. So having said all of that, what do you guys say? Let's just go ahead and jump right into it. And as always, let's start with pricing. And so since we're in the SV trim level today, there are a few different configurations for our particular trim level. You got the King cab four by two, starting at $31,890, crew cab four by two, starting at $33,290. And lastly, the crew cab long bed four by four, starting at $39,120. And by the way, those first two configurations, the four by twos, you can get that with four wheel drive, of course, just simply add approximately $3,000 to either of those prices. But regardless of the configuration that you go with, the power plant on the frontier SV is going to be the same. Powering the beast is a 3.8 liter direct injected V6, putting out 310 horsepower at 6,400 RPM, 281 pound feet of torque, coming in at 4,400 RPM, power sent to the rear wheels or all wheels through a nine speed automatic, zero to 60 time coming in at approximately 7.5 seconds, which actually is not too bad for a truck. I will say that, but MPG numbers then coming in at 18 in the city, 24 in the highway for the rear wheel drive, 17 city, 22 then on the highway for the four wheel drive, taking regular unleaded fuel. And so now having got all of that out of the way, what do you guys say? Let's go ahead and find it straight away. Let's put the frontier here to the test and let's see how quickly we can get our new 2023 frontier SV here up to speed. All right, it looks like we got her straight away here. Let's hit it. Huh? There's nothing wrong with that. That was plenty of an acceleration from merging on to the highway. Like I said, zero to 60 in 7.5 seconds, it's pretty darn good for a truck. And you can feel that there is definitely plenty of pickup, no slipping whatsoever, even with all that power being said to the rear wheels right now. So excellent acceleration for the frontier without a doubt. But anyways, to go along with that acceleration, as always, braking is equally important. So as expected, you will find four wheel ventilated disc brakes coming standard. And actually when it comes to the 60 to zero stopping distance on the frontier, get ready you guys, 115 feet. So since there's no one behind us, I love it. Love the braking feel on the frontier. So definitely not a soft braking feel and surprisingly as heavy as trucks in general are, instantly brings you to a stop at the frontier. So 100% great braking feel here in the frontier without a doubt. The touching on suspension and handling up front, you're gonna get an independent double wishbone type front suspension. In the back, over slung, multi-leaf rear suspension, front stabilizer bar comes standard. But if you were to go with the crew cab configurations, you will also get a rear stabilizer bar to go along with that. So overall as far as ride quality goes, it's been perfectly fine actually in my short test drive here today. And honestly, Frederick doesn't have the best roads either. So yeah, ride quality's been perfectly fine. As far as steering feel goes, that's probably my favorite part of the frontier. Such a heavy steering feel on this thing. I absolutely love it, especially at low speeds. It's a very heavy steering feel, but even at higher speeds, it just feels right for a truck. I'll just put it that way. I love the steering feel on this thing because a lot of times, usually in SUVs, but sometimes in trucks, you will get kind of a loosey-goosey steering feel. So I don't know. I just love the steering feel on this thing. It instantly points you in the direction that you wanna go and I can't talk today. So anyways, as far as cabin noise goes, we're going 35 miles per hour right now. Not a whole lot of exterior wind noise or even road noise coming into the cabin. So that's perfectly fine as well. And touching on visibility, let me adjust my mirror. It looks like every other truck in the world, you can see perfectly fine out that rear view mirror. So definitely a big fan of that. But that pretty much rounds out the performance segment of this review, you guys. Let's now go ahead and take a look at the exterior of our brand new 2023 Nissan Frontier SV. All right, so here she is, you guys. The new 2023 Nissan Frontier finished in tactical green. In case you were curious of our exterior color name that we have on this one here today. So let's go ahead and get started with where this one is made. So taking a look at the van, the first character is the number one, indicating that the Frontier is built and assembled in the US. So that is where this one is made. So that's pretty cool. But anyways, let's go ahead and start up front, body colored front grille with a gloss metallic intersection. That's of course what you guys are looking at. And that's what comes standard on the Frontier SV. You do actually have some steel front skid plates down below and that's gonna come standard as well. Front tow hooks can be had with the crew cab long bed configuration and they are available otherwise as well. They're still wrapped in plastic. This thing is so new right now. Anyways, to the sides, halogen headlights do come standard. Of course, the automatic feature as well, meaning when it starts to get dark in the night, those headlights will turn on automatically for you there. Automatic high beams are gonna be optional. So essential with that is, is if you have your high beams on at night and the sense of the vehicle coming in the opposite direction, it's gonna automatically dim them back to low beams and when that vehicle is gone, it's gonna automatically then bounce them back up to high beams. So pretty cool feature there. And then LED daytime running lights are going to be optional on the Frontier SV then. But taking a look down below those headlights, you guys are gonna see, we actually do have fog lights as well. So again, coming with the crew cab, you're gonna find some fog lights down there and then LED fog lights are gonna be optional. We got the halogens of course, so we do not have that option, but didn't wanna point that out. And then one of my favorite parts about the front end of this one, you actually have Frontier lettering, kind of the front portion of that hood there. So I definitely like the look of that as well. But anyways, that pretty much rounds out the front end of the Frontier here. So now go ahead and make our way to the side. All right, so now, since we are around to the side of the Frontier, black windows surrounds, do you come standard rear privacy glass to go along with that? Of course, take a look at the side mirrors. They are body colored power adjustable side mirrors. You can get heated side mirrors. They are an option though for the Frontier SV. Of course, you can get the king cab or the crew cab configuration. In case you didn't notice already, we do have the crew cab setup. So we'll be testing out that rear leg or a little bit later in the video here. But then take a look down at the wheel setup, 17 inch six spoke aluminum alloys do come standard. And then running boards are gonna be optional for the crew cab. We don't have them. Quite honestly, with the ride height of the Frontier, I didn't personally have any issues getting in and out of this one. So you really don't need them. But anyway, I still want to mention it. But anyways, that pretty much rounds out the side profile here. So now go ahead and make our way to the back. All right, so now since we are around to the back of this one, first thing I kind of want to mention is it's kind of an integrated tailgate spoiler. You guys can kind of see there's a little lip there. So thought that looks pretty cool. Wanted to mention that. Of course, Frontier lettering spelled out horizontally etched into the tailgate itself, yet again, just like the hood. So it's pretty cool. LED tail lights actually do come standard across the board. So a little added illumination at night there, like that as well. Then take a look at the tow hitch down here. Of course, that's available. And when it comes to the towing capacity, the max towing capacity for the Frontier SV is gonna come in at 6,270 pounds in case you were curious. But then of course tucked away down below underneath the passenger side there, you will find a single exhaust outlet. So having said that, I do believe you guys know what we have to do next. This always here is that exhaust clip. All right, so now since you are around to the back of the Frontier, when it comes to opening that rear tailgate, it actually is a dampened tailgate. So it's not gonna come flying down all in one second on you. It is going to lower slowly. So I was a big fan of that. Once that tailgate is down, the overall bed length is gonna come in at 73.3 inches. The bed width, 61.4 inches. And then between the wheel wells, that's gonna come in at 44.5 inches if you wanted the specs. As far as payload capacity goes, that's gonna come in at 1,600 pounds then. But factory spray on bed liner is going to be optional. We do have that, and I love that. It's back there. 120 volt power outlet. Again, gonna be optional. We have that as well. So always like that so you could charge up a circular saw or really anything back there. So that's pretty cool. Overhead cargo bed lighting is actually gonna come standard. And then there is some LED bed lighting that is going to be optional if you wanted that. We also have that as well. So that is pretty darn cool. So then making our way up to the rear legroom, of course, is going to differ between the king cab and the crew cab. King cab comes in at 26.2 inches. So almost unusable. You might be able to put some small children back there, but that's actually less than my old Ford Mustang GT. Crew cab is gonna come in then at 33.2 inches, which isn't horrible. So for reference, I mean even six feet tall, this is how much space I had in our crew cab that we have today. And of course, when it comes to those rear seats, they are 60, 40 split flip up bench seating. So they actually do fold up if you wanted to put your massive back there or whatever else, it really doesn't matter. Rear center armrest with cup holders coming with the crew cab. So I can show that to you guys. Then there's also going to be a USB A and USB C charging port and actually 120 volt power outlet back there yet again. So that's pretty darn cool. And of course, if you go with the crew cab, not the king, but the crew cab, you will get a sliding rear window back there as well. But now it's going to make our way up to the front seats, manually adjustable seating coming with the king cab, six way power driver seat with two way power lumbar coming standard with the crew cab. Claw seating is going to come standard leather seating is going to be optional for the crew cab. We don't have that with us today, but heated seats then are also going to be optional for the crew cab, but we do have that option. So that's pretty cool bet. Actually, overall seat comfort hasn't been that bad. So I will say that. And perhaps that's because they are power adjustable maybe so I don't know. I just haven't had any issues with seat comfort so far. But now let's go ahead and take a look at the steering wheel. It is not telescoping. It is tilt. So we did want to emphasize that, but I will say for my six foot self, I was still able to find my perfect driving position probably because of the power adjustable seats, but still Nissan, if you're watching this, I would definitely recommend making this a telescoping steering wheel because at some point some six foot four, six foot seven adult is going to hop in this thing. They're going to have longer legs. They're going to have to put their seat back more and they're probably going to want that steering wheel to telescope out a little bit further. I'm just saying. Anyways, leather wrapped for the crew cab heated. It's going to be optional for the crew cab. We do have that option, but then make our way to the startup. Let me start by showing you guys the key here. All of your buttons are located on one side of the key. Got your Nissan logo at the top. The remote start by the way goes for $375. And then you got lock and unlock, but it is all keyless entry with a push button start. So all I'm going to do here is simply put my front of the brake and press that black engine start button located just by the driver's right knee. And so once started up, tachometers all the way to your left speedometers on your right. Relatively large digital display front and center to control what is on that digital display. There are steering wheel mounted controls found on the left side of the steering wheel, giving you things like a digital speedometer. How many miles you have left until you hit empty. There's your outside temperature, of course. There's actually some off-road statistics, which I thought was pretty cool. So it tells you what kind of elevation you're at and what kind of angle the front tires are turned at stuff like that. So that's pretty cool. When you need your next oil change, trip A, trip B, average MPGs, the list goes on. So pretty much everything could possibly one of the digital portion of the gauges there. But then let's go ahead and make our way now to overall interior quality. There is a power moonroof available. It's going to be optional for the crew cab only. Home light controls are available for $110 if you wanted those. Dual zoom climate control is going to come standard on the crew cab only. So both driver and passenger can set their own temperatures there. Wireless phone charger is going to be optional on the crew cab and that more than likely is going to be located in front of the shifter. Although in our case, we just have some rubberized storage probably to put yourself in there still nonetheless. So that's pretty cool. But anyways, to the right of the shifter, you got a couple of cup holders, more rubberized storage behind that within the center armrest. You actually have some little cutouts for pens and stuff and actually decent amount of storage in there. So didn't mind that. But overall everything is finished pretty much on the basic side of things. It's to be expected. I do like this kind of rubberized storage though just above the infotainment screen up top here. That's pretty cool. I didn't expect to see that. There actually is an overhead sunglass holder up there as well. So it's very functional. I'll put it that way. But now let's go ahead and make our way to that infotainment screen I just mentioned. Eight inch color touch screen display does come standard Bluetooth and audio streaming as well. Android Auto Apple CarPlay. Factory navigation system is going to be optional. And of course you can check out your radio information up there as well. So when it comes to the sound system, six speakers is going to come standard. But there is a premium package that will give you a 10 speaker fender sound system that we do not have with us here today. So having said that, what do you guys say? Let's go ahead and turn on the radio, see what we got playing today and let's test out our six speaker sound system that we have with us here today. Oh, don't you know we took a big step forward. All right, so little bit more bass than I expected. Clarity, it sounded like a six speaker sound system. I'll just put it that way. Not the best when it comes to clarity, but more bass than I expected. I'll put it that way. It's six speakers. If you wanted a better sound system, I like that Nissan allows that to be available with the fender sound system. But anyways, lastly, I wanted to mention to you guys on the infotainment screen is when you do put the frontier in reverse, you will find a rear view camera coming standard across the board, which is always is going to lead us into safety. And so to start, front side side curtain airbags do come standard driver and passenger knee airbags up front as well. In the back, you're going to have latch, AKA lower anchors to tethers to children for the rear car seats, rear child door locks, tire pressure monitoring system. But then I do want to mention a package option called the technology package. It goes for $990 that is an option that gives you all the advanced safety, essentially, including lane departure warning, blind spot warning, rear cross traffic alert, a rear sonar system, rear automatic braking, high beam assist, and adaptive cruise control then as well. So overall, when it comes to my final thoughts on the frontier here, nice design. Like I said last year, I do like the redesign. It was well overdue for a redesign. It looks a heck of a lot better now. Love this steering field. It's probably my favorite part, my most memorable part of the frontier. It's such a heavy steering field, which I personally prefer. So also brilliant braking for a truck. I'm just gonna say that usually when I ramble off the 60 zero numbers, if I say 115 feet, I'm usually driving a sports sedan or something like that. So to find that on a truck is pretty ridiculous. So big fan of the very firm braking feel that instantly brings you to a stop here. As far as constructive criticism goes, I'll give it two things. So telescoping steering wheel is definitely a necessity in my personal opinion. I think just most vehicles out there, if 99% of them have a telescoping steering wheel, unless it's like, I don't know, a budget car or something like that. So I definitely recommend that, especially with trucks because a lot of times you're gonna have taller individuals that are gonna need that telescoping steering wheel. But also interior quality is definitely on the basic side of things. Wouldn't have minded if they made it, I don't know, just a little bit more special. So just round the shifter in the cup holders, for example, it's just kind of this matte black plastic. Even if they put a design to that and still kept it plastic, but just put a design, I think that would make so much of a difference in my personal opinion. But anyways, let me know what you guys think of the fronts here in the comments section below and that is about it for this one, you guys. Thank you so much for watching. Feel free to follow me on social media at the bottom of the screen if you want to see what's coming next on the channel before it gets to YouTube. Be sure to hit the subscribe and the bell notification button if you're in the new car reviews. That is what we do here on this channel after all. Do appreciate you guys watching more than you know, but I will see you guys all in the next video. Stay Gold.


The story you're about to hear is true. Only the names have been changed to protect the innocent.  You're assigned a missing person's detail. A wealthy businessman in your city drops from sight. There's no trace of him. There's no apparent reason for his disappearance. Your job? Find him. The documented drama of an actual crime. For the next 30 minutes in cooperation with the Los Angeles Police Department, you will travel step by step on the side of the law through an actual case transcribed from official police files. From beginning to end, from crime to punishment, Dragnet is the story of your police force in action. It was Saturday, February 6th. It was foggy in Los Angeles. We're working the day watch out of missing persons. My partner's Ben Romero. The boss is that brown chief of detectives. My name's Friday. I was on the way back from communications and it was 8.15 a.m. when I got to Room 67A. Missing persons. Hello, Joe. Oh, hi, Vance. How are you coming on a Jarrett case? Oh, nothing. It's a dead end so far. Yeah, it's a tough one. Joe? Yeah, Ben. Just got a call from Evans Harbor Division. They found Jarrett's car. Where? Parked at the end of Pier 6, abandoned. Found about 7 o'clock this morning. You know, they checked the car over. Mm-hmm. They found Jarrett's coat in the front seat, letter in the pocket. Yeah. Suicide note. Jarrett Keith Jarrett was a prominent and prosperous man. He was a well-known civic figure in social life. As a young man, he'd inherited a moderately successful business from his father and over a period of 20 years built it into a million-dollar concern. He was apparently happily married to an attractive woman and he was the father of two grown children, a son, Keith, age 26 and a daughter, Evelyn, age 23. According to his friends, business associates, employees and advisors, there was no logical reason for Jarrett taking his own life. The letter found in the abandoned car on Pier 6 was brought downtown on the handwriting check. It matched perfectly with the other specimens of Jarrett's writing. There was no sign of the gun which Jarrett was supposed to have carried. 9.35 a.m. Ben and I drove down to San Pedro, boarded police boat number one and joined with officers Brett, Hanby and Tredder in a two-hour search of the area around Pier 6. We found nothing. 11.40 a.m., the launch headed back for the dock. I can't make it out, not a trace. Well, it might take a little time. Pretty fair chance that the tide swept the body out of the area. It still turned up. This fog's not helping much. Yeah. Maybe I'm just on me this morning, Joe. I sure can't buy this thing the way it stands. I'll put in with you there. Jarrett disappears Thursday night after work. His car doesn't turn up till this morning. He made up his mind to jump off the pier while he's stalling around an extra day and a half. I don't know. How about Ms. Jarrett? When's she due back? She's flying in late tonight from Sacramento. We can check her in the morning. Looks like we're just about in. Yeah. Well, a crime lab ought to be true checking the car, huh? Yeah, just about, yeah. These boys on PD-1 sure can handle this thing, can't they? Again. Well, I'm going to call in from the phone and the dock. You want to stand by here a minute? Okay. Watch your step, Joe. It's Leverett. All right. Michigan 5211, please. Highly Joe Friday. If you look over that car, yeah, we'll check with you. Right, thanks. Joe, anything? Jarrett's gun. The crime lab boys finally found it. Where? In his car. It was hidden up under the dashboard. Yeah? Gun's been fired recently. Two empty shells. How the recovered gun might tie in with Jarrett's disappearance and the suicide note. We had no idea. When the car was found abandoned at the end of Pier 6, the obvious guess was that Jarrett had taken his life by drowning. Well, that was the case, what accounted for the two gels in his gun. Why was it carefully hidden under the dashboard of the car after it was used? Why were the only fingerprints on it? Those of Jarrett. Together with Sergeant John Epperson, Ben and I spent the rest of the day interviewing the friends and business associates of the missing man. The answers were the same. Not one of them could think of a good reason why Jarrett would take his life. More than half of them thought that he'd met with some kind of foul play. Late that night, Mrs. Jarrett arrived back in the city. The next morning, Sunday, Ben and I drove out to the mansion in Val Air to talk to her.  I was fixing these flowers. Not very good to keep them out of water very long, you know. Yes, ma'am, you go right ahead with it. Do you do for some brandy or something? No, ma'am. No, ma'am. I think I might stand a little brandy. It helps sometimes. Good morning. Sergeant, I'd like to have you tell me the truth. What do you think has happened to my husband? Well, that's just what we're trying to find out, Mrs. Jarrett. All we have to go on is that letter that we found in his car. Ah. I can't believe there's anything in that. Jack and I have been married 31 years. He'd never take his own life. I know him that well. The letter was in his handwriting, ma'am. We've confirmed that. That's why I say it's so confusing. Look, not a gorgeous rose. Will Scarlett they call it? That's gorgeous. Yes, ma'am. Would it be possible for us to talk to your son and daughter, Mrs. Jarrett, just a routine investigation? Well, I'd rather not have Keith or Evelyn brought into this if you don't mind. Are you sure you wouldn't care for some brandy? No, thank you. Thank you. You sound arbitrary, ma'am, but this case isn't exactly trivial. And I'm afraid that your son and daughter will have to be brought into it sooner or later. Well, can't be helped, I guess. If you like, I'll give you their addresses. Keith lives in Hollywood, and my daughter, Evelyn, has an apartment near Malibu. Either one, Mary? We're both single now. Evelyn's been divorced. Any trouble between you and your husband? I mean, any recent arguments? Yeah. Not recent? No, we can get along too well, that's all. When you don't think any family trouble could account for Mr. Jarrett's disappearance, huh? I don't think so. Jack and I rarely argued. We didn't get along, that's all. It was no worse last week than it was 10 years ago. Oh, he has his work. I have my interests worked out as well as a lot of marriages, I guess. I'd better get on with these flowers. No offense intended, ma'am, but you don't seem too disturbed with what's happened. Suicide note and all? No. Yes, I don't. You don't think this marriage trouble that you mentioned a little while ago could lead to your husband's suicide? I didn't mean that much to Jack. The good or bad his family never meant that much to him. There's always the company business making money. We never worked together. Jack didn't care about having a real family. He didn't mean that much to him. He wrecked everything I ever wanted. Boy, you were the story. Do you know of any business troubles it might have upset him? No. Jack never discussed business with me. He had his own interests. The children had their own interests. I guess I have mine. Ma'am. I drink, Sergeant. It's something to do. I drink every day in the week. I drink quite a bit. It's only been since the children left. No, blame them. Jack didn't care. I guess I don't care myself anymore. We never were a family. Do you know if your husband's seen your son or daughter recently? He never visits them. He never had anything in common with them. You still don't think that this trouble has any connection with what might have happened to Mr. Jarrett? Maybe. I don't think so. It doesn't seem to matter that much to me. How do you mean? It doesn't matter, Sergeant.  I don't love him. I just don't care, that's all. Before we left the Bel Air mansion, we got the addresses of the son and daughter in Silver Mr. Jarrett's room. No leads. Ben and I drove down to Malibu to interview the daughter, Evelyn Jarrett. She lived in a small modernistic cottage bordering on the beach. She was attractive and well-dressed. At the age of 23, she'd been twice married and twice divorced. She sat on the sofa filing her fingernails while we talked to her. I don't mean to be rude, Sergeant. I have a date at 2.30. I don't want to be late. We'll try to make it as quick as possible, Mr. Jarrett.   I'm sorry. None of us were ever really close to him. When's the last time that you saw your father, Mr. Jarrett? Well, up until last Thursday night, I hadn't seen him for about five months, I guess. Well, you saw your father last Thursday night, was it? Yes, he called me that afternoon and said he wanted to have dinner with me and my brother, Keith. Did you have dinner with him? Yes, a little Spanish place out of Milrose. Keith was late as usual. Wasn't that pretty unusual for your father, having your brother get together with him for dinner? It didn't happen very often. Mother probably told you about that. Well, how did he seem that night? Did you notice anything unusual about him? No, he was pretty much the same as always. Asked Keith and me how we were getting along. If we needed any money. You didn't mention anything about leaving the city. You going on a trip, maybe? No, I didn't. We had dinner, a few drinks, then I left. What time was that, Mr. Jarrett? A little after midnight, I think. Keith and my father stayed on. They said they were going to have a few more drinks. Care for a cigarette? No, thank you very much. Here you are. Thank you. How was your father when you left him, Mr. Jarrett? Pretty good mood. He and Keith were arguing as usual. After a few drinks, they always argued. Anything serious? I don't know. Keith had some debts. He wanted Dad to pay them off. There's one thing I'd like to ask you, Sergeant. I read the story in the paper, that gun you found in my father's car. The father of 32? Yes, that's right. There were two shells fired. Funny. Dad hasn't had that gun for a year. The reason I know is because after my divorce, I moved into this place alone, I thought it would be a good idea to have a gun around, so I asked my father for his. Yeah? He told me he'd given it away already. He didn't have any more use for it. Who'd he give it to, you know? He told me he'd give it to my brother Keith. That was almost a year ago. Must be a mistake. What car? Maybe Keith can tell us. As soon as we left Evelyn Jarrett, we drove to a public phone booth and Ben put in a call to the Green Castle apartments in Beverly Hills, where Keith Jarrett was living. There was no answer. Ben called the manager of the apartment. Yes, sir. How's that? When did you say? I see. All right. Thank you. Bye. We've got something. What's that? Jarrett's son doesn't live there anymore. The manager says he moved in a hurry. Sunday, February 7th, 4 p.m., Ben and I drove to the apartment vacated by Keith Jarrett and checked it over. No leads. We called Mrs. Jarrett and her daughter Evelyn. They had no idea of Keith's whereabouts. Mrs. Jarrett suggested that we call a fashionable tennis club out in Hollywood to which her son belonged. There they told us that Keith Jarrett was scheduled to play in a tournament match on their courts the following morning. Monday, February 8th, 10.30 a.m., we checked in at the tennis club. Joe, I talked to the manager of the desk. He said Keith Jarrett's around here someplace. He's having a boy paging to send him over to her. Okay. I wonder if this is where he moved after he left the apartment. No, I talked to the clerk of the desk. They don't have living quarters here. And that's sure a beautiful stretch of lawn in there. Yeah. I wonder if they have any trouble with gopher. Nice job. My name's Jarrett. You're a gentleman looking for me? Oh, yeah. The police officer's Jarrett. I'd like to talk to you a few minutes if you don't mind. There's a match coming up. Yeah, I got time. What can I do for you? It's about your father, Jarrett. We've been assigned to the case. Thought you might be able to help us out. Oh, yeah. I'll tell you anything I know. Big a suicide. Kind of unusual. We haven't spent much time together the last couple of years. My father and I didn't get along too well. How do you seem to be Thursday night all right? Yeah, pretty much the same as always. We ate, had a few drinks, and it was about the size of it. What time did you leave the restaurant? Do you remember? Oh, one, 1.30, I guess. Yeah, that's right. We drove to my place for nightcap. Talked a little, and then he left. What'd you talk about? Do you recall? Same old thing. Try to argue me. They're taking a job with a company. He's been trying to sell me out for years. I never could see it. What kind of work do you do now, Jarrett? Tennis. How's that? I play tennis, all of the tournaments. I like it pretty well. My father never could see it. He calls his tennis funds. Let's see. You don't have a regular job that you work at? No, that's what we always argue about. I like tennis, though. I figure if you've got the money and you don't have to work, you might as well be doing something you like. Hey, you fellas care for a stick of gum? No, thank you. Excuse me. You know, I've been doing pretty well since I've shanked in my backhand. I'm seated fourth in this tournament. That's it. It's a pretty good life, all in all. A lot of fresh air. I've seen girls around, good-class people. You still live in town, Jarrett? A friend and I readily thought they would just move there. When'd you move? Sometime last week. Why? Thursday night? Yeah, that's right. Thursday night. Just supposed to mean something. You left pretty much in a hurry, didn't you? I guess so, yeah. Mind telling us why? Look, let's go back in the lounge. Yeah, okay. A little more privacy and no use of advertising. What's that? I got quite a few bills around town that need paying. I tried to get my father to cover them. When I saw him Thursday night, he turned me down, so I figured the best thing to do was move. I'd lose some of the bill collectors for a while anyway.  Well, you don't let them know. I haven't gotten around to it yet. Yeah, I don't think I get it, all these questions. Just routines, Jarrett. We understand that your father gave you a gun about a year ago, a .32 revolver nickel plate. Yeah, that's right. Well, if you had the gun, your coming was fine in your father's car. Because he asked me for it back. Well, it must have been a month, six weeks ago. You say why he wanted it back? No, I didn't ask him. I dropped it off one day when I was going by his office. You can check with his secretary there, Mr. Butler. Say, fellas, if you don't mind having a run, I got that double-smash coming. Just one more question, Jarrett. Can you think of a good reason why your father'd want to take his own life? Just one, yeah. You met my mother? During the next three weeks, besides checking on dozens of other missing persons' calls, we continued pushing the search for John Keith Jarrett. One by one, we eliminated the possibilities. Jarrett's personal physician told us that after a recent examination, he considered his patient to be in sound physical and mental health. We rechecked Jarrett's business rating. His company was in excellent financial condition. His bank statements were gone over. Nothing showed there. We talked to Jarrett's lawyer. He could think of nothing to account for his class disappearance or possible suicide. We went back and retraced our steps. Outside of the gun found in Jarrett's car, which tied in with nothing, there was no hint of f***ing involved. Two months went by. We kept pressing the search. We got nowhere. Thursday, April 11th, I had lunch at the Federal Cafe, and then I took our car, 80K, to the police garage for servicing. 1.15 p.m. I checked back in at the office. John. Hi. Got something. Yeah? A Jarrett case. This take-back just came through from watching. Let me see. And look here. Here you go. Walter John Mitchell. Port Santa Tomas. Santa Tomas. Yeah, we've been there. Little fishing town north of San Diego. Oh, yeah, sure, sure. Walter Mitchell, small boat operator, applied for a Coast Guard pass, March 18th. Let me see. Checked the fingerprints on Mitchell's application, found to be the same as fingerprints on missing persons bulletin. Well, how about that? Yes, we made it. John Jarrett, Walter Mitchell. Same man, huh? 2 p.m. Then a night checked out of the office, got in the car, and drove down the coast to Port Santa Tomas, a small fishing town with a population of 250. On his application for a Coast Guard pass, the man who signed himself, Walter John Mitchell, listed his address as Pier 2, birth number 11. When we checked there, the birth was empty. We talked to one of the villagers fishing from the pier. He pointed to a fishing craft lying halfway out in the bay, and identified it as the Blanche S. Mitchell's boat. Said he was having engine trouble.  We took him up on it. It's your hot down here. I think I'll get rid of this coat. Yeah, it's a good idea. Yeah. I'll give you a hand there. Thanks. Yeah. Well, it's your long pull. Want me to take over for a while? No, we're almost there now. Yeah, easy now. We're coming alongside you. You all right? That ought to do it. Mr. Mitchell? Walter Mitchell? Yeah. Police officer, sir. I'd like to talk to you if you have a minute. Hold this study there. You grab hold of my hand. Yeah. This is my partner. Sergeant Friday. How do you do? There you go, sir. Here we are. You sit on that hat cover if you like. Clean. Scrub it down myself this morning. Oh, that's fine. Thank you, sir. Beautiful day, isn't it? Nice warm sun. Look at that. Water's clear as crystal. Good life out here, obviously. Yes, sir. I guess you have pretty good hunch why we're here. Well, I don't know. I'd like to have you tell me. Well, your real name's Jared, isn't it? John Keith, Jared. Yeah, I thought so. I've been halfway expecting it. Tell me how'd you find it? Your application for a Coast Guard pass. Check the fingerprint on it. Well, yeah, but you didn't have anything to compare them with, did you? Well, when you applied for that permit to carry a gun, we got a set of your prints. They went out with the missing person's books and on it. Oh, yeah, I forgot. Washington matched up the prints and sent the information along that. Yeah, that's all right. Sure handed to you, fellas. I thought I was pretty smart covering up. Well, how about it, Mr. Jared?  Yeah, imagine I do. Smoke. Oh, thank you. Yes, sir, I'd like one. Here, I've got a match right here. Well, no need to tell you. You've had a lot of people worrying over you. I don't think there are too many. My secretary, maybe, right? Well, yes, sir. Look, I realize you men know your business. I know I don't have to remind you, but I'd just like to make it clear. What's that? I know a little bit of the law, and I want you to understand this. If you should tell anyone where I am, I'd consider it an invasion of my personal rights, OK? Any way you want it. Maybe you can't figure out my disappearing the way I did, but I think that you know that legally I've done nothing wrong. I respected the rights of the law, and now I want the law to respect my right. Yes, sir. It's fine with us, Mr. Jart. It's just one thing. Yes, sir. We've chased down quite a few blind allies on this thing, that false suicide at your stage, the gun, the letter. Do you think that maybe that earns us an explanation? I'm sorry about the legwork, believe me. You can hear the story if you want to, short version along. Which one? Any way you want to tell it. You met my family, have you? Yeah, we have. My wife? Yes, sir. My son, Keith? Yeah. My daughter? Yes, sir. And you know what they're like. My wife's been an alcoholic for eight years. Why be polite? She's a drummer. My son's a tennis bum. There's no other word for that. My daughter? No. You know it as well as I do. Two divorces, different boyfriend every week. He might as well be walking the street. I worked hard for my family, Sergeant. I was proud of them once. What would you do if they turned out the way they had? I don't think I could say, Mr. Jart, I don't know your family that well. There's a good reason you don't, Sergeant. I paid a lot of money to keep them off the front page, one jam after another, scandal, ten years of it, covering up for them. I got tired of it, dead tired. Maybe I'm just getting old. You men care for a cold bottle of beer? No, I don't think so. Well, there's nothing more to tell, officers. I worked most of my life for my family. I've had three vacations in 20 years. I tried to do everything I could for them. They're rotting apart, Sergeant, all three of them. I didn't want to stand around and watch them rot. That's why I left. I see. They each have their own trust fund.  Yes, well, I guess it's none of my business, Mr. Jart. But do you think that you might have been to blame, maybe? Your children, I mean? I don't know, maybe. If I made the mistakes, I guess I'll pay for them. Not much from chaining your life when you're 52 years old. It's lonely sometimes. Pretty lonely. Well, what are you going to do now, do you know? You sail around, I guess. Take the blanchets down to Mexico, all around, do some fishing. It's lonely sometimes. I'll get used to it. Say, you men wouldn't like to stay on board for dinner, would you? I'm even learning to cook. No, thank you, Mr. Jart. Guess we better start getting back, then, Joe? Yeah. Yes, we have. Well, thanks very much for dropping by, officers. Sorry about that extra legwork I caused you. Oh, here. What's this? Two dollars for the rent of the road. Glad to meet you, Jart. Good to hear. Same to you Friday. Goodbye. Goodbye, Mr. Jart. Watch your step there. Yes, yes, we got it, all right. Okay, Joe, come ahead. All right.  Okay, thanks, Mr. Mitchell. Goodbye. Goodbye. Good luck, gentlemen. Good luck to you. Nice fellow that, Mr. Mitchell. Yeah. Strange one, huh, Joe? Oh, I don't know. He seems pretty happy. Must have been pretty rough on him. Yeah. What's it prove? It's no crime to get lost. Just heard was true. Only the names were changed to protect the innocent. On April 19th, a final report was filed by officers investigating the disappearance of John Keith Jarrett in a moment the results of that report. For a complete investigation by officers into the case, it was found that John Keith Jarrett had acted within his rights. Legally for the officers to disclose his whereabouts would have constituted an invasion of privacy. Three years ago, unknown to his family or friends, John Jarrett died and was buried anonymously. You have just heard Dragnet, a series of authentic cases from official files. Technical advice for Dragnet comes from the office of police W.H. Parker, Los Angeles Police Department.


Morning, you tube. Pike's Peak is set morning. Seth's awake. He's helping me put shop blocks in my shorts for the race up the hill, the mountain. Ready, buddy, to go to Pike's Peak? Yeah. A tall dark roast. Grande decaf. And a grande decaf vanilla latte. And a large water. My chance. Thanks. Yeah, a man of two springs. Yep. For better or worse. For better or worse. Pike's Peak Ascent, 2017. Almost 8,000 feet of vertical. Over 13 miles. Yep. So it's a little warm up. Watch last year's vlog from the Pike's Peak Ascent. My brother took ninth, crushed it. I'm racing this year. I did not race last year. So basically, this year's vlog is going to be a lot simpler because I'm running. And I can't carry the phone to the top. I can't carry the Canon 5D to the top. So anyway, bear with me there. And away we go in about 15 minutes. I'm just trying to stay off my feet a little bit. Save those legs. Feeling good. Feeling good. Feeling good. We get to decide what's on it. You guys might be having a hard run. It's like Yosemite Sam. But we're having a hard time putting our chairs together. Whoa, whoa, whoa. Don't tell us you had a hard day when you had to put a chair together, that was that hard. This is what the start is like while we're waiting for you guys. Isn't that exciting? Wind is all the way down there, rock climbing. Just got to get as close to you guys as possible. Peak Ascent, 33. Past winner of the Pike's Peak Ascent. Tough race. We'll see what happened. Probably just a tough day. Just a tough day. We had a good day. That is a beautiful one. That is nice. Wow. You want to see it, Seth? I like it. Was it on this phone? Yeah. I'm going to see what's here. Yeah. I want the wind blows. Pretty cool. Whoa.


Okay, so I don't I can't see my next slide. Okay, so I know it's not a big deal You probably you know if you've done presentation before you might know what's coming next, but I Average between 250 and 300 slides So knowing the next one is kind of handy and I won't be able to do that today. So Please bear in mind with me So I'm going to start Hopefully it's it's all of us So my name is Daniela. I work for a company that's called learn kubernetes We are based in primarily in Europe. So London and Milan and just recently in Singapore But I'm not here to talk about what we do. I mean we do training for kubernetes That's basically what we do But I'm here to talk about a little bit of story that started in a year ago so a year ago I was here in Singapore actually and there was a really interesting tweet on on Twitter and it was from a guy in Japan Manabu and I can't read Japanese and I don't know if you can Good for you good for you really good for you but it basically says what if I take a kubernetes cluster and I tamper with the networking Will kubernetes still work? Can I still use my cluster or you know, it's gonna go and fall apart So we we basically took that as a challenge as we said, can we actually do that? Can we actually take whatever Manabu has done and replicate the findings that? If I'm on that time Now before we dive into what we actually did a little bit of a recap of what kubernetes is and and how it works So usually what happens is we have a collection of servers like this one so those can be on-prem service those could be virtual machines and As engineers or DevOps then the challenge is how do we actually manage a lot of them? And then how do we manage them efficiently? so One way it's not the only way But one way is to actually use kubernetes and and the way it works is we have a master node Which is going to receive all the commands and then we actually have the other nodes joining the master and then when that happens We call that a cluster Okay, and then knows I mean in this pictures. They are all the same But it actually could be of different sizes the only things that we actually care is the memory and CPU of those nodes So as soon as you add nodes into the master into the cluster Then you're going to add memory and CPU to the overall memory and CPU of your cluster So that's all kubernetes is it's basically just merging all of your servers into a single machine Okay, you can imagine having kubernetes as your single VM in the data center But why would you do that? Well, the reason is pretty simple now You don't need to deploy to a particular server anymore when you deploy you can just Deploy directly into a single machine So I just say kubernetes, please deploy me this is application four times. This is going to create Four application inside my infrastructure Then because kubernetes has got you know this layer of abstraction between your serve and the data center They can take a smart decision. You can look at the nodes and say actually, you know, I'm going to place the application like this and That's basically the beauty of of what we see with kubernetes And then this kind of design leads to some interesting results The first one being well if kubernetes can actually analyze my infrastructure and take care of it on my behalf Then when a node goes away kubernetes is smart enough to actually move that application to a node that is available Okay, so we said up kubernetes, please give me four applications It realizes that one was gone The node was lost and just rescheduled that application into a new node And that's great. So we've got reliability is built for full tolerance and everything works Now if you are deploying so this is this is a simple example if you're deploying inside a Cloud provider you probably have a load balancer and On top of your nodes and then what happens usually the node goes down The load the node is detached from the load balancer and then you would reschedule that application somewhere else That's basically how kubernetes works and and why why we find it so useful But um the other interesting things for kubernetes is that you know, it's designed to scale So instead of having like three nodes we hadn't we might have more nodes As an example, this is basically how we structure application is kubernetes We usually have the application underneath Then we have an internal load balancer, which we like to call service Because it's not an overloaded overloaded term at at all in confused science and then at the top We have like what we call an ingress which is like an external load balancer But we talked about scales so what if I'm really really keen to have a really really large cluster But I don't have enough application to actually feel that cluster like in this case, okay? I was a little bit too keen to scale my cluster and then there are only two application and three nodes So if I access the first node, would you expect a response from the application? Yeah, yeah Second one, you know what's coming If I access the third one No Anyone else saying yes? Yeah Yes, it is a part of the cluster Then yes, no I'm actually going to the nodes trying to access the service. What would you expect? Yes Five or two anyone else Okay, let's Is the traffic lost? Yes, is It's a timeout Yeah, all of them. Is it a four of four? Which one is it? Actually none of them it works You get a response back even if there is no application on that node Okay, so why is that? How does that work? Right? Is it is it magic? Um So the first you know when when I look at this at the first time was Is the load balancer actually doing the smart routing? So if you look at the application and so I've got two application deployed in my cluster and there is like an application load balancer for for AWS or you know an application load balancer in Azure then That load balancer actually knows where the application is so you can just route the traffic directly, right? Which is great like this Does that sound fair? It does right? Unfortunately, I don't want the logic to stay in the club in in in the load balancer, right? There will be cloud specific load balancer So it's not really something that I can take away and it would be really really hard to implement if I've got an un-prem cluster It's also a little bit of a single point of failure and and leave it hard to scale depending on what you do You can imagine that if you're doing an un-prem data center Then scaling a component like this could be quite problematic plus you need to sing those rules somehow So this is not what Kubernetes does. This is not how we route trapping inside the cluster The other option right could be what if we actually route all the traffic to the master node first So if the master knows knows everything about a cluster So it makes sense to go through the master node the master node is going to tell me where the application is deployed and I'm going to route the traffic to that node Fair enough Like this Does this work Well, it's great, right? I've got someone say no It does work right So It's got a vendor agnostic. Yes, it's going through the master that doesn't belong to the cloud provider We solved the problem, but it's a single point of failure and it's really really hard to scale right? How would you do that? How would you scale those master nodes and you need a lot of master nodes and Imagine so we're talking about if you if you're deploying on GKE, which is the Google cloud platform We are talking about 5000 nodes. How many master nodes do you actually need to support 5000 nodes? There should be exactly but then we start having all this sort of conversation. It's really complicated This is not actually what happens in Kubernetes The first idea we had wasn't too bad, right? So this load balancer that knows everything and can route the traffic That's actually a really clever idea. The only problem we had it was that it was outside of a cluster and not inside So what if we actually move then load balancer and we break it apart and we have one in each node Okay, if we do that That's quite a clever idea because if we go and we have traffic going directly to the node We can go through we can go through again The next time we go to the load balancer. Well, if it knows all the routes and just say well This is not mine go somewhere else Okay, this is actually what happens in Kubernetes, right? We have on each node. We have a component It's it's actually what I have in Kubernetes. So you can see it's got Cloud vendor and Gnostic redundancy built team because the more node we have we add the more load balancer We're gonna have and in scales with the nodes as well So the component that does that in Kubernetes is called cube proxy And it's a binary which is installed in each of every node in your cluster and that component is in charge of setting these rules in the node itself and But but the question is how does this cube proxy know the roots, right? I thought the master would know the roots. So is what is your props doing? So it turns out that when you ask for a deployment to keep an anchors. This is this is what happens So we ask for deployment for an application then the request goes inside Kubernetes The first component to receive the request is the API server the API server stores the request Into into the database the database will store that request for you And then there are a series of components inside Kubernetes that will analyze the status of a database and create the application For you and there is a third component Which is the scheduler which will look at at the or at your application will make sure that Any of the pending elements are going to go and be scheduled At that point what happens is we've got your we've got another component inside Kubernetes So next to cube proxy we've got the cubelet, which is like a glorified agent The agent goes and asked the master node if there is any update So any application that should be deployed on that particular node if the funds one then is going to delegate The creation of the application inside a node so when that process happens then The application is created an IP address is assigned and that IP address assigned is then returned back to the master node So you can imagine that the master node Has got all the information so when he created the part when he created the applications what it did We had like a table like this We knew where each part was because we have signed that with the scheduler to the node Right, but we didn't know the IP address at that point in time now because the cubelet created the application And and send back the message to the master node now. We also know the IP address assigned To that particular node. Okay, so we have all the information inside a master node So the list is always up to date So what that means is that when you add an application then the master node will be updated And when you remove one when you are another one is going to be updated as well when you remove one It's going to be removed as well So that happens every time you create or delete one of your application inside your cluster So that's not the only list though. There is another list Which is to do with what we call in what we call service, which is basically an internal load balancer So the internal load balancers got a list of IP addresses as well So in reality, we've got two long lists inside a master the first one belongs to the pile to the applications and the second one belongs to Lot balancer basically. So what kind of IP addresses am I routing that traffic to So those two lists are quite handy, right? But they leave inside a master node. So qproxy doesn't know anything about it. So what happens is Is that qproxy will ask for this list to the master node and set up Routing tables on each and every node inside your cluster. And that's basically the magic behind Kubernetes. So what happens when so we could finally answer the question What happens when I hit the node and there is no application? And the first thing which is going to happen Qproxy will read the routing table and say hey, there is nothing here. Go somewhere else and somewhere else is actually Looking at a table looking that there is an internal load balancer The internal load balancer points to either port one or port two. We look at port one and port two We take we take the IP address we find every route request Okay, so that's what happens in Kubernetes every time you hit a node now That was only the intro for my talk. Okay, so now, you know a little bit of our Kubernetes works how the traffic is reading inside your cluster and and you know when We're ready to be ready to actually break things right so now that we know how it works What happens when when things go wrong? So the first scenario is That let's say we've got Somehow we are a little bit unlucky and then one of the nodes the one on the left Cannot connect to the master anymore What's going to happen? Yeah, so because the The service on the left is not able to connect to the master It won't be able to update the routing table Okay, so if any request comes in they're still going to use the old routing table Which you know Could be good could be bad You know depending on how many pods how many application you created and destroyed in the in that time It might still go to an existing application or it could might be to fail Whereas the other nodes will work as usual right because they can contact the master They can download the list and they can continue working So that's What we see when when there is a something like a spread brain It almost works kind of good And the the disadvantage is that it could be a routing a stale routing table And eventually when when when the when the network recovers and we have a single network then that Node could contact back the master node and just get back the new list So the good news is that eventually kubernetes will converge and then fix everything by itself Good news I guess So the second one We start breaking things. So the second one is actually what if I kill That process inside a node, right? So we talked about cube proxy is a is an agent that it's inside a node But what if I actually go and stop that process? What is going to happen? Well done So if a new pod comes in and we don't have cube proxy That's the routing table if a new so sorry if a pod crushes Well, there is no one to update those tables, right? So it's going to stay the same So we get similar scenario to what we had before So for the existing pod is going to work for some others it won. So it's basically Not great, but it still works And but like you said, it's it's a demon set. So in kubernetes We've got something called the demon set which is basically just a fancy way for saying that when it goes down When it crashes it's going to be restarted Okay, so that's how kubernetes is going to solve the problem for us And yeah, it almost works. So we've got the stale routing table and We can kind of make it work. So that that's basically number two So now let's go even more Deep into this what if you have not necessarily good intentions Okay, and let's say that The Routing table is lost As in you go inside and you tamper with the routing table. Okay So you change the values or you remove the values without anyone noticing it And that's basically what? Manubu from from japan did so that's what we found out when so when we published this article last year And he basically described this situation where You go inside a node You actually tamper the routing table and then you see what happens So this is the scenario that we're going to analyze today So we've got one single node one single load balancer And then the plan is very very simple We get into the node We drop the routing table And we see what happens Okay, so before we move forward What's your guess? What is going to happen? Spread brain scenario. Let me see. Oh I want I want to do spoilers What is going to happen? Okay, okay, so is it going to work It's going to recover We've got a lot of faith. I like that anyone else continue work just because you said it Anyone else saying that it's not going to work Okay, we'll see so what I did what we did is So we took that back to the team and then we just replayed everything that manuba did and Well, the story is a little bit more complicated. I'm sure there is Romans somewhere as well. They're going to make a movie. But in essence, this is what happened So we basically set up a very simple loop In bash that what it does is every second Is going to print a date And the value that we get back from the application Okay, and the application is quite simple. It always replies with hello world. Okay, so this is what happens Okay, that's quite easy. Okay every second we see a response from From the application And then what we do is we are very very nasty. We get inside a node. We drop everything Okay, and then we observe what happens So we expect nothing happens nothing nothing Oops Nice, I mean There is a little bit of a gap, but I can sort of deal with that But it works Okay, and then but if you look at numbers So it's 47 up there. It's 14 down here. That's probably 27 27 seconds What happened is 27 seconds And I don't know why why 27 seconds? Why not, you know, five or 10? So It's about 30 seconds. Okay, so we can leave it that So maybe something is Going on with I don't know maybe load balancer, right? So we have two many things to You know analyze so the easiest things we can do at this point is to actually remove things from the equation So we have a load balancer at the top. We just remove the load balancer and we repeat the same scenario, but this time We got direct to the node Okay, so same loop this time.  So I see hello world We drop the routing table Nothing. Whoops Time out time out Back Okay So why this time we see the time out and then before we didn't Okay, when we go directly to the node and we use curl then we see the timeouts twice And and then it goes back up A couple of interesting things here first is that the curl times out the 10 seconds So this is making me think basically maybe the load balancer took a little bit longer to time out But but at the time it took you know to time out the the class of some reason recovered and then we didn't see that time out happening And so that load balancer must be having a timeout, which is greater than 10 seconds Then the other weird thing is that someone is fixing the routing table for me And then the last one is Why for a second? Why not 20? Why not 10? What's going on? So you might have guessed that This is uh, this is sort of cube proxies Fault, okay, or or well I guess fault. I guess it's probably the right or is the right word, but what what is happening? Well It turns out that these rules That are set up on each and every node. They're actually synchronized on a on a regular interval So there are two flags as you can set to actually make sure that Your routing tables are refreshed frequently enough So if something happens to these routing tables like me going inside and dropping the routing tables Then it can be fixed in in in a proper manner. Okay, so these are for example two flags. So the first one being Are often so this is 30 seconds and then what's the minimum time after every refresh? So this is to make sure we don't do too many refresh at once and we overload the master node Okay, so these are two two mechanism that we have to control this routing table on each and every node um so If if you well lost a little bit why we're doing uh the the curl This is what happened basically just there's a visual representation. This was our setup This was the routing table And what we did was we kill the routing table So remove the routing table from the node and we made a request that request Couldn't like it would go through the load balancer the load balancer will forward that request in the node But because there is nothing listening on the node It would just You know wait until until the time out by that time qproxy will recover Will refresh the routing table that will come back up and then finally You know because there is something that can accept incoming connection And it will go through And then finally will reach the application That was what we did with With this simple test Okay, so these two flights you can tweak them. They are part of a cube proxy and That's basically everything I wanted to show you today I've got a couple of things that I learned in the process I think I really you know something that I want to share with you today So the first one is um, so and this is perhaps the At least the most interesting for me is that We originally look at this problem a year ago and By that time we wrote a blog post I took this same Talks to another conference and now I'm doing it again And every time I give it again I basically find an error of what I explained and I need to go back and rethink about how kubernetes works Now this is just to give you an idea that things inside kubernetes are quite complicated Okay, uh, they're quite complicated for a good reason because we want to build the product which is reliable But at the same time, uh, I wish we could do better with You know documentation and explaining how things work now. I showed you a very simple example of networking in kubernetes But this is what it looks like With the nitty gritty details of ip addresses and routing tables So the same as before so it might be a little bit overkill, but this is what happens So you go inside a node you're actually requesting for an ip address which doesn't exist That's another thing weird things about kubernetes kubernetes will kick in We'll read that request we'll realize that the ap address doesn't exist We'll look at the table and say it doesn't exist replace that ip address for a part inside your network Then depending on the type of network you've got It will go through a routing table and then go back to your part Okay, it's a little bit more complicated than what you see And what you saw earlier today, but that's basically just to give you an understanding of you know How complex are things if if you start digging inside the clustering site inside kubernetes works The other things that I find really useful is article like this So this is a pretty old article, but julia evans issues She's amazing. All right, so this this kind of article really go down deep down into what Networking is in kubernetes. Give you more idea how things are actually How they work inside kubernetes So the other things which I wanted to share is that how do you get better at this kind of stuff? Well, the only way to get better is to actually try and break it try and And and become better at it. So the two resources that I would recommend is the cka practicing practice environment Which is basically just a collection of challenges And they're also useful if you're interested in in being certified And then the first one is a just a collection of Useful I would say useful challenges They are not some of them are not particularly good, but it's just You know good challenges for you to to tackle and then in practice. So I think something that I would recommend And then how do you protect against things like this? How do you protect against, you know, someone going inside a cluster and dropping the connections? Or you know noticing that you won't be able to reach a particular application because the routing table is stale Now there isn't a simple answer because it depends on what application you're building You know, what are your constraints? How you tweak the flags? Uh, but but basically all boils down to how you monitor your infrastructure and How you control your parts and and everything else So I think I want to be able to give you just one link and say this is the holy grail go and do it You will need to understand how your application works to actually fix it And that was it. Thank you very much for listening to my talk. I hope you enjoyed I've got a couple of us. So if you like to talk I wrote the same thing is as a blog post. Okay, so this is the blog post that we wrote And then it goes for the same example and you can also check out the code and try it on your own So maybe if you've got a production class that I probably wouldn't advise that But if you like to try that at home. Yeah, please do so I've got stickers Okay, so just Be careful with the sticker because the last time I did that some there was a guy who just Went there and grabbed all of them All of them and it wasn't fair for everyone. Everyone else. Okay, so be mindful of others I guess so but just just be mindful of others as well, please um Any question the question is going back to the situation one has click brain syndrome When you have No communication between the workers In the networking world The voice is kind of stuff You have a condition you have to have a redundant network connectivity between Systems all currently as I see right now the current is networking is based on the Assumptions I think to answer your question, I hope I got the question right if the question is Can I actually use a redundant network for my Kubernetes cluster or who That's Two words for what you described um, so To answer your question first So kubernetes is built around three basic rules of networking And the three basic rules are any application can talk to any other application inside a cluster Any node can talk to any other node in any other application and the third third rule is The application see itself with the same ip address that everyone else sees with okay, so these are the three rules Now if you satisfy the three rules, you can implement any type of network you want So usually what we see um when we work with We've with clients is that they ask us for A kubernetes pod a kubernetes application to have two ip addresses one which belongs to the One network which is the network of the cluster and then the second one is actually a network somewhere else So you could have like multiple networks attached to your pods. I don't think it's quite what you asked for My question was specifically related to the nodes The nodes are communicating with each other There's a loader internal node network But that's one Pretty much because you do not have a two network interfaces per node That's That's That's correct, but it's also not always the truth Okay, so sometimes for example if you're using amazon if you're using eks for example Um the network for the nodes and the network for the applications are on the same network, which is the vpc Okay, but if you take the same configuration in gke which is google cloud platform then the node exactly Exactly. So I think as long as you satisfy the three rules of your networking Then you can design a network in the way you want And eks having one and and and and gke having another so I think Depending on what you're building probably I've got very little understanding of what you're trying to achieve But I think as long as you can satisfy the three rules, you should be able to have redundancy in in the network itself So can I So actually you can as he said and he gave the public cloud example only Even in private cloud you can build multiple card and do that And the other thing So you talk about straight brain situation when they start here. So split brain situation in this case The entire thing works on the core out. So you need to have three to start here So for example, if you are When you need to have three Instances to have core out if you want to achieve And then one goes down there is way to they elect by themselves one master and they can continue to work So split brain problem here was when it was This is more active active cluster that is one thing to start with what Even if one goes down The other useful are going to let and do it So that's how this and other thing that works here is the eventual consistency he talked about Over the time things are able to settle down the entire design premises in this case is That we call for tolerance rather thought resistance So something goes wrong Other things will continue to work For some time with whatever information they have and once the other whatever went down comes back It will try to sync up and try to figure out What are the best So that's a little design A philosophical difference in here called tolerance versus for resistance And that that that's what that's what plays a role in this Any other question, okay, cool that was it. Thank you very much