What's the word, y'all? I hope all y'all had a merry Christmas, happy holidays, bro. I just love this day, mostly because it's basketball from the very early part to the very end. The clock has went past midnight, ladies and gentlemen. We've had like 13 hours of straight hoops. And it was insane. I know a lot of people were talking bad about it. Before it even started, people said man, the NBA gotta be better at picking their games. And even after just wrapped up, people are still saying it was a bad day and I'm just being honest with you. I think you just don't enjoy ball. The first game of the day to the last game of the day, it was not the best straight bangers. Even gave you a little moment in one of the games where you can take a nap. I think that's the perfect holiday season and I'm here to talk about it. I think the title of this video is like Yolkich is inevitable because he really is. I was affirmed, believe it, before the season started that no matter how good of a season Nikola Yolkich was gonna have, I think the politics of everything was not gonna allow him to win an MVP award. There's only a few people in NBA history to win three back to back to back. And I just thought that the voters would be like, we don't know if we want to put him in that category just yet. He's making it hard. I mean, if you ask me right now who's the MVP of the season, I don't have a legitimate answer between Tatum Yanis, Yolkich, Embiid, KD. I would argue those are the top five guys. Actually, when we talking of your piece, I'm not even going past five anymore. Those are my top five candidates and I don't know the order, but now the Denver Nuggets hold the sole spot for the number one seed and you know, things are looking better. If you look it back on my video from maybe three weeks ago or so, I was trying to figure out what teams are contenders or pretenders. And I had a really hard time once we got to the Denver Nuggets. And my main argument against it was the fact that their defense had been dreadful this season. At that point, I think they were 28th in basketball. They allowed the highest field goal percentage at the rim. It was just all gloomy and bad and not to mention the bench had been terrible as well. When if Yolkich is not on the court, then every time he comes back in and feel like he playing catch up and that's still kind of the case. We're gonna talk about that in a second. But since that video, they've kind of put it together defensively. They've jumped up to 24th in basketball in the season. But better than that, they have a top 10 defense in the last couple of weeks, which is a great, great sign. This game today, obviously upset that book didn't get to play as he wanted to. You saw each other go out there and do his thing. But Landry Shaman stepped up. Damia Lee hit some huge shots down the stretch. So they held their own. But I feel like if they had Devin Booker, you know, things might have been different. But I still enjoyed this game so very much from Yolkich dropping, what, a 40 point triple double on Christmas to Aaron Gordon catching the body and showcasing why he should be in the conversations for an all-star appearance.  So he probably will not make it, but he should be in consideration because he's having an amazing season. And to see Jamal Murray have another really good game when Ashley TV had put it all together for it to be the perfect game for the Denver Nuggets. But the thing that scares me about the Denver Nuggets as good as they are, like I predict that they would have the most win in the Western Conference because I know that Yolkich is that good and with Jamal Murray coming back to form, Michael Porter Jr. being back, I knew they were gonna win a ton of basketball games. The thing that scares me the most about them though is the fact that when Yolkich is on the bench, it seems like people just running around and not doing nothing. And then someone said recently that they think part of that is a product of Yolkich being so goddamn good that the system is him. So when he's not in, they don't have a secondary system to run with the bench people, which I guess is a theory worth talking about. But in general, I just think they need a couple guys. People kind of assume that Bones was gonna be that guy, but people have to realize Bones is a year two player and it's not like he was a top five pick or something. Even though he has a bright future in his league, I didn't expect him in year number two to be like, oh, I'm about to be the sixth man and I'm gonna take over the offense when Jamal's on the bench or Yolkich is on the bench. But they need to figure out something because there's a lot to be desired with that second unit. And I was projecting a little bit. If you watched the videos early this season, I was like, oh, before the season start, I was like, hey, it's gonna be Zignagi. Zignagi gonna get minutes and they gonna need those minutes. He don't really play. He played today and it just, it looked bad. Like I'm saying, the bench looked bad. I think he had what, four fouls in less than 10 minutes. Like it was rough out there. But both versions of their starter unit are great. Jamal Murray, KCP, Bruce Brown, Aaron Gordon, and Yolkich, of course. That's what Michael Porter Jr. been out with his injury. And then the one with Michael Porter Jr. in replacing Bruce Brown is even better than that first one. So like both of their starter units are so amazing, but it seems like everything outside of, I guess six, I'm putting Bruce Brown at six because he's been a great pickup for them. Shout out to that. They basically stole him away from the Brooklyn Nats, gave him, it seemed like nothing for what his production could be on a really good team. But everything out there has a struggle. And I don't know how you fixed that, how you remedied that. I don't know if they're not usually a team that will make a big time trade, even though the Aaron Gordon deal, the Aaron Gordon trade has been amazing so far. But still, I got a lot of love for the different nuggets. They gave us a great Sean Christmas and we love that. The second best game of the day for me was the Philadelphia 76ers beating the New York Knicks and the Mecca. I'm mostly because it gave us a lot of different story lines to talk about. Before this game he was starting, this man, whoa, just talking about James Harden might go back to Houston. What, on Christmas? This is what we doing right now? What was he reporting on this man who got a hoop in 15 minutes? That's what he did. And early in this game, it felt like it was the New York Knicks game, bro. They was, you know, obviously on the A game, once you get a little wild and going down there in the third game, Louis the Street got to this one. And it felt like they were just about to pick up that same momentum because Julius Randall came out very early and he was hooping Jaylen Brunson into a couple of big shots to start off the game. And yeah, the first quarter school was 37 to 25.  And then I remember specifically on the broadcast, this had to be middle through the second quarter. Whoever was commentating, I don't remember the names because it's all blend together, we had five games. And they were saying Joel and B it's been relatively quiet so far today. From that moment, that man, Joel and B put on that MVP cap and said, I'm here and he really took over this game between him, Harden and George and Yang. That's all they really needed today to win. And Harden is playing really good basketball. So obviously not the same basketball he was playing in Houston where he was averaging 35 points per game and doing the ridiculous stuff. But this is a better version, the best version that we've seen of him in like the last season and a half. And that is a good sign because they need stuff like this 29 points, 13 assists and what I was so impressed with is some of the turnovers that he caused. He knew that he was on national TV. I am a firm believer that when players know that they own national TV, they just perform better because he knows that like 75 to 80% of the people watching this game won't be able to watch his next game because they ain't got league pass. So I need to get him a show and let them know I'm still him. Maybe I'm making that up but that's what it felt like for James today. And even in the post game presser, George and Yang was like, hey, they was playing drop coverage on me and I ain't seen that since college. And the Yang game was out the minivan. They set it 100,000 a broadcast, 16 points from him and they turned that game around in a split second. We still don't know exactly what happened to Jaylen Brunson. I hope he's okay because again, he's been playing great basketball for the Knicks and this would be terrible for them to have this eight game winning streak and then now in a blink of an eye, things changed like rapidly. So see if they put it together. The next game goal to stay wars with the Memphis Grizz. Now I know that the Warriors controlled this game for a great majority of it but I think what made it interesting was the antics. The thing that I hate it the most about it is that the refs, the refs also know they're on national TV. How about we talk about that? They know when they're on national TV because they just bloated. There was no reason for it to be as many texts as it was today. It just wasn't. I understand the tech if people get start throwing punches, pushing each other maybe even yapping at the ref when we get to people talking to each other that is not, that should not be a technical fact. And I don't agree with Stephen A. Smith on a lot of stuff. I'm enjoying this stuff, I don't agree with Stephen A. Smith but that was one thing I did agree with him on saying that the refs think that we are here to watch them and that was not the case. But this was a huge win for the Warriors because obviously with no Steph Curry, a lot of people with me included predicted that they were about to go on a huge slump because we saw what they looked like without them on court. They had like an offensive rating was like a 98 which is God awful when they're off the court and before the injuries. But since then they've been waking up obviously Jordan Poole kind of slides into the Steph Curry role. Even of course he's not Steph Curry but he can play it where the off ball movement is so great getting himself open shots. Jermaine Green with three points, 13 assists, 30 rebounds just Jermaine being Jermaine. On the other side, I was very disappointed in Jaren. I was very disappointed in Jaren. This was your opportunity to show the world that you belong at DPOWAC Conversations. If you to come out there and have 5,000 what 10 minutes or something like that at one point in the game, it's unacceptable to Jaren, it's unacceptable. Boston will occasionally just come out and hit every shot in the world and you have no chance. And that's what happened today. They hit eight threes in the first quarter alone, 36 total points from Jalen Brown to Tatum having a great game on Christmas with Deuce.  But this was the conference finals right here. This is a conference finals preview. Obviously Chris Middleton didn't play and I know a lot of people gonna discredit the Celtics because of that. I mean it's a part of the game. Everybody's gonna have injuries. I mean we will see once they're fully healthy. I'm not gonna say because the Boston Celtics won this game that I'm picking them to win the Eastern Conference now. It's not, we're not going that deep. But they did come out and they showed the world. Hey, I know our offense have been struggling over the last couple of weeks. I know we lost our last three home games but we still actually liked that. And they showcased that to the world. I was disappointed in the Bucks though. Again, no Chris Middleton. So obviously the offense is gonna be iffy, but like I was more disappointed. Maybe it was the defense. Yeah, I think I was more disappointed in defense than their lack of shooting. This team in general just doesn't have a lot of great shooters, which is a scary thing if you Yanis or whatever. Yanis ended up with 27 points. Didn't feel like he had 27. One of his worst games I would say at a season, 27 points deal with 27 points. You get what I'm saying. When it comes to overall impact on the offense or defense side of the ball, I feel like I didn't get a lot of Yanis that's credit to the Boston Celtics because they packed the paint. Hey, we gonna let y'all shoot. Y'all got people that we might trust, but at the end of the day, we really don't. And Yanis needs those people around him to maximize himself and to maximize the team. And they don't really have a lot of great three point shooting. I mean, I don't know if the number's backed it up. I guess I'll look it up. They have the 20th best three point percentage in basketball. They're shooting the ninth most. So you can maybe do some math to figure out if that's good or bad. I mean, let me compare it to like the previous season. I feel like the last season, they were really good three point shooters and obviously the championship season too. Yeah, the season before that they were ninth in percentage. Obviously the volume is super high as well. Number six in value with ninth percentage is actually insane. The last game was the Lakers versus the Dallas Mavericks. This was the game that you could have fell asleep on. And that's exactly what I did. In the middle of the third quarter, you know, you know what I'm saying? It's Christmas.  And that's what happened in the third quarter. Actually, I saw a tweet. Hold on, let me pull up this tweet because I thought it was insane and I must have been asleep once this man, Darvaham, did this. The Lakers have played a five guard lineup against the Mavs with Russell Westbrook, Dennis Schroeder, Austin Rees, Patrick Beverly and Lonnie Walker. I must have fell asleep because I did not remember that. And apparently, according to the comments under that tweet, Christian Wood was lighting their ass up when that five was out there. So the Lakers is just such a rough spot without Anthony Davis being there, bro. LeBron gave us a great performance, hit all his free throws, 38 points. And he was excited. He was saying, I'm fun for it to be Christmas and stuff. And I think he really enjoys it. 17, 17 games on Christmas so far in this crew, which is ridiculous. But that's, yeah, that's all. That's too much. I appreciate y'all watching. Hopefully y'all had a happy holidays. I got a link in the description. Two, I should have probably said this at the top, but thank you so much for watching. I have another channel. I don't know what I'm saying. Can't even believe you're doing whatever. Actually, go subscribe to all the channels. There's a gaming channel. There's another channel, another NBA channel. But this new channel is what I'm calling my burner channel. We're like, I'm just gonna drop videos. No themes, no like common themes. Like this one is talking basketball. The other one is also talking basketball. And the other one is playing a basketball video game. This one has no correlation to all of that. The first video I dropped is about manga. Like, like brother, I'm reading, I think I mentioned that in the video before. I'm reading hella manga. So the first video is about me getting into manga. I'm gonna do a video about soccer slash football this week. So like, you just want other Kenny content that's outside of this room? Go subscribe, I guess.


Warning, HMCL and O5 approval required. The file you are attempting to access is available to personnel with level 4 forward slash 2000 clearance only. This clearance is not included in General Level 4 security protocol. Attempting access beyond this point without necessary clearance is grounds for termination of foundation employment and cancellation of all educational, medical, retirement, and mortality benefits. By submitting your credentials, you hear by consent to exposure to a known cognitohazardous image, and verify that you have been inoculated against that image. In the event of unauthorized access, this console will become inoperable. Security personnel will be dispatched to revive you and escort you to a detention cell for interrogation. Attempting to access this file from any computer not connected to the Foundation Internet will result in immediate termination regardless of clearance. At login credentials, level 4 forward slash 2000 clearance required. Security cognitohazard activated. Scanning for neural activity. Consciousness confirmed. Retrieving file. Item number SCP-2000. Object Class Domual. Special Containment Procedures. The entrance to SCP-2000 is disguised as a disused park, ranger station, and Yellow Stone National Park. Despite several civilian trespassing attempts, the entrance has yet to be breached into installation's recorded history, and no further physical containment has been deemed necessary. Protocol Plain Sight 201 is in effect for SCP-2000. Necessary supplies and replacement personnel may be delivered via unmarked road vehicles or civilian helicopter as appropriate. No personnel below level 4 forward slash 2000 clearance are permitted access to documentation regarding SCP-2000, or any protocols associated with its containment and upkeep. No personnel below level 5 forward slash 2000 clearance are permitted access to SCP-2000 below sub-level 3. All personnel assigned to SCP-2000 must submit to neural archetype scan on a monthly basis. Personnel stationed on site must submit to weekly scan to be stored locally. Level 4 forward slash 2000 personnel or above stationed on site are not permitted to leave Yellow Stone National Park during the course of their assignment. In the event of transfer, either elective or compulsory, Class A amnestics must be administered, and false memories implanted consistent with assignment to other high-security or keeter-class SCP objects. Additional personnel may be assigned to SCP-2000 and granted temporary level 4 forward slash 2000 clearance at the discretion of the item's HMCL supervisor, currently Dr. Charles Gears, and O5 Command. The exterior surface of SCP-2000 is surrounded by Scranton reality anchors, SRAs, every 20 meters, arranged hexagonally to prevent incursion by hostile anomalous interference. Each SRA's function must be checked semi-annually and replaced as necessary. Technicians servicing SRA components may reference document SRA-033, revision 1.0.7, 5 X-Yank, Anastasicus, Constant Temporal Syncs, or X-Axt, capable of maintaining stable tachyon flux across the expanse of the facility, maximum output rating of 100W each, have been installed and are to be maintained monthly. Technicians servicing X-Axt components may refer to document X-Axt-864, revision 1.3.0. One pseudo-Romanian manifold has been initiated at the entrance to sub-level 4 and must remain open at all times. In the event of the manifold failure, procedure dead Euclid-101 is to be executed immediately. Other non-anomalous life support and utility systems may be maintained in accordance with the Standard Foundation and Maintenance Protocol, Section 101.5, Mission Critical Components. Whenever possible, non-anomalous materials and resources are to be used for SCP-2000's maintenance and repair. In the event of any K-class scenario which does not compromise the existence or function of SCP-2000, procedure CYA-009 is to be enacted as soon as possible. Many Foundation installations globally are to monitor the scenario as it unfolds, preserving what material resources are possible under the Genomead Protocol until such time as all remaining sites respond all-clear to SCP-2000 curies as to find and document 2000-XKAC-1.9. Upon receipt of all-clear code, procedure Lazarus-01 is to be implemented.  Some things are just as important. This device is absolutely not an excuse to let down our guard, or take greater risks with SCP objects, or cross-test them or whatever you might have in mind. Primary containment is still our best chance at survival, otherwise there would be no reason to make the cover-up so extensive. We can only suspend God's disbelief so many times before the Universe just says no, and considering what we've had to deal with in these past few decades, we may have passed that point already. Former Administrator Dr. William Fritz Description, SCP-2000 is a subterranean Foundation installation originally constructed sometime in the last ██████ years for the purpose of reconstructing civilization in the event that a K-class end-of-the-world scenario could not be averted in time to prevent humanity's extinction or near extinction. Since its inception, SCP-2000 has been activated at least twice. Foundation records regarding SCP-2000's construction and history prior to this assumed first use has been lost. Whether this information blackout as a result of accident or design is impossible to determine, the mission critical portion of this installation begins 75 meters below ground level and extends to 100 meters depth. Although the scope of engineering required to recreate SCP-2000 in its entirety is impossible to execute while maintaining secrecy, all subsystems of SCP-2000 have been successfully reproduced in laboratory setting. The installation on all procedures involved in its upkeep are mundane in nature. The facility documented 2000-SS-EX for information regarding esoteric Foundation technologies necessary for SCP-2000's function. Primary power for the facility is a liquid fluoride thorium reactor rated for 1 gigawatt total output, with a reactor life of 70 years at maximum capacity. A geothermal generator has also been installed to take advantage of the region's volcanic activity. This generator is capable of powering the facility in standby mode indefinitely. SCP-2000 also contains water treatment facilities, air purification and recycling systems, hydroponic production wings, and housing necessary to permanently sustain up to 10,000 personnel. To fulfill its primary mission, SCP-2000 includes 500,000 bright, Zarsian-hominated replicators. At peak capacity, SCP-2000 is capable of producing 100,000 viable, non-anomalous humans per day, with a warm-up period of 5 days, utilizing an underground Romanian transit pipe to collect raw material from various hot springs, an underground magma flows in the area, and a computer memory bank housing data on all known human alleles. This system is capable of recreating any lost human genome, or generating as many new and unique genomes as necessary to repopulate human civilization. Researcher note, use of the BZHR system is currently suspended outside of maintenance testing and emergency situations. CYA-009 is still go. Possible hostile incursion is still being investigated, and this database is proving particularly difficult to debug. We're still seeing a distribution of congenital and genetic defects far above baseline numbers. Right now, it can only guarantee about 60-75% viability in new specimens. See Addendum 2000-1, Dr. Christopher Zarsian MD, Biotech Research and Development. Humans produced by this process can be advanced to any age desired without extending the 5-day incubation period. In addition to construction features, the BZHR also has the ability to implant memories by administration of Class G hallucinogenics and developmental hypnotherapy. Life histories, neuro-archetype scans and genomes of many Foundation personnel, including all personnel level 4-4-2,000 clearance and above, are maintained to ensure the SCP-2000 may be activated and procedure LASR-01 can be initiated by as few as one surviving human. After the implementation of the Ganymede Protocol, indicating a failure of the Foundation to prevent a K-class scenario, SCP-2000's security systems will unlock, allowing any Foundation employee to initiate procedures CYA-009. If after 20 years SCP-2000 remains inactive, security will be relaxed further, allowing any non-anomalous human being to access the facility and initiate the procedure. Once activated, SCP-2000's internal monitoring systems will attempt to locate all personnel level 4-4-2,000 clearance and assess their condition. Mission critical personnel not found will be replicated using the most recent neuro-archetype scan on file and awakened prior to the initialization of any other systems. After these personnel are revived, security logs will resume normal function. For a complete list of contingency options available, level 5-4-2,000 personnel may access Document 2000-CYA-09. Note that receipt of the all-clear code as defined by Document 2000-XKAC-1.9 may be waived only if all other Foundation facilities have been rendered inoperative. Otherwise, security and mobile task force elements provided on the procedure CYA-009 will be dispatched to all remaining Foundation facilities to confirm their function and the integrity of local reality. After Lazarus-01 will begin when an authorized level 5-4-2,000 Foundation employee inputs the desired resume date into SCP-2000's BZHR control unit. Available units will then begin production of prominent political and cultural leaders of the time period using descriptions, genetic information on file, as well as replication of a global populace consistent with the chosen time period. List of SCP-2000 force bases dedicated to storage of building materials, construction equipment, factory machinery, agricultural equipment, and computer database storage. In addition to infrastructure concerns, a wide cultural base with copies of thousands of famous works of art, music, literature, and a full backup of the worldwide web are kept on site in the event that other repositories are destroyed. HMCL Note Discovered this note in previous iteration records at Lazarus-01's conclusion. Researcher Note If we ever have to do this again, do not set the resume date further back than 20 years before the event. Not only can we piggyback on a lot of undestroyed structures if we do, but it will make continuity a lot easier to resume. Years Is Too Many We're straining personnel such as it is about having to rebuild the chronological specifications just to save time on the population and agricultural demands. Besides, how much of the 20th to 20th centuries do we really want to rewrite, and how many times? Isn't one great war hard enough to keep track of? Dr. Henrietta Eisenhower, Historian My tenure as SCP-2000's HMCL will honor this request, currently pursuing official documentation update to account for this change. Two world wars is plenty. We do not need the hazard of third. Dr. Charles Gears HMCL Supervisor The first replacement humans housed off-site must necessarily be informed of SCP-2000's existence and function as they are being created. This strategy allows newly constructed humans to assist in construction and recolonization efforts directly, and skill sets appropriate to reconstruction have been pre-selected for increased prevalence in the first five million individuals produced. As global population increases, the process of diaspora and reconstruction will accelerate geometrically, allowing economic and agricultural infrastructure to recover as quickly as possible. While it is feasible that some replacement humans will not survive the initial renovation period, such individuals can be recreated indefinitely until all major population centers and foundation facilities have been completed. Modern administrative assets during this period will focus on the falsification of dendro-chronological, astronomical, and radiometric dating records necessary to maintain the appearance of historical continuity. Please see Document 2000, RETCON, version 2.3.3 for details. In the event that significant portions of natural habitat are also destroyed prior to the project's completion, refer to Document 2001-Tier, version 3.0 for approved rapid regrowth methods. It is estimated that the world population, manufacturing capability, agricultural production, and culture can be reset to 2000 CE levels 25-50 years after the procedure is implemented. At the conclusion of procedure LASR01, amnestic agent ENUI-5 will be released en masse, causing all reconstructed humans to forget their affiliation with foundation assets. History will then resume from the chosen date. Each procedure will necessarily alter the course of human events due to the enormous complexity of human-social interaction. Further research into predictive historical modeling based on observations from prior completions of the procedure LASR01 is ongoing. HMCL Note No further proposals for behavior or cultural modification will be accepted at this time. Previous attempts to amarylliate violent or sociopathic tendencies in humanity as a whole have already been implemented and deemed successful. Experimentation using second iteration subjects indicates that further modification would undermine tenacity to such a degree that technological and social progress would be noticeably inhibited. See experiment log ██████ for further information. Dr. Charles Gears, HMCL Supervisor, Document 2000-SS-EX The following information establishes basic operational parameters of technology developed specifically for the SCP-2000 project. Although this technology may appear to be anomalous, it is based entirely on verifiable scientific principles currently in use by the Foundation to affect containment. The invention of the Scranton Reality Anchor appears to predate the first activation of SCP-2000 and is credited to Dr. Robert Scranton in 1889. The main body and much of the circuitry of the SRA are constructed of a conrosan resistant beryllium bronze alloy. Inspired by artifacts recovered ██████ effectively eliminating the appearance of virtual particle anti-particle pairs required for type-green reality-bending phenomenon to manifest. Due to the expense involved in producing the beryllium bronze alloy required for the SRA's construction, Foundation-wide implementation of the device has been limited to units capable of an area of effect less than two cubic meters. Note-1 Use of MSRA Scranton Boxes to Provide Mission Critical Document Security El Pidment et al. Foundation Vol. 106.8, pp. 10-14-1988 Researcher Note The mechanism of the SRA's function and the source of its inspiration must be kept secret from all possible reality-bending entities for reasons which I hope are obvious. The qualified level 6 forward slash 2000 maintenance technicians have been cleared to access this documentation. If any member of SCP-2000 staff reveals to you that they are a level 6 forward slash 2000 maintenance technician, please report to them to O5 command so they can be reassigned and submitted to amnestic therapy immediately. This is not a punishment. It is a legitimate safety concern. If these devices are ever compromised, so too is our lifeboat. Dr. Lowell Henry Piedmont Esoteric Containment The ex-Yank Anastasicus Constant Temporal Sync is a device designed to stabilize the flow of casualty across a given field of effect. Ex-Ax use high-power electromagnetic radiation in the radio band coupled with a tachyon field emitter to create a permeable event boundary allowing organic and electrical systems to pass through unaffected while maintaining a static casual environment. Static motion and superfluids for use in tachyon emission and storage TX-Yank A Anastasicus Foundation Volume 10.4 PP-141-143-1892 In other words, temporal anomalies which might normally prevent SCP-2000 from being constructed will have no effect, so long as at least one ex-Ax remains in operation. There are no plans to implement Foundation-wide use of ex-Ax devices. Future Note Temporal things can be useful for a lot of things. Containing SCP objects for which you need one second to last 300,000 years is a good example. Holding a point of reference constant during temporal repair missions so that you can meaningfully record your progress and undo series mistakes is another. But natural-causal relationships are flexible in a way that human mind is not equipped to deal with meaningfully, and creating more than a small handful of isolated static causalities will do more to damage temporal integrity and security. Ex-Ax will not be implemented Foundation-wide. Yes, we have tried it during a past iteration. No. Further increase in the results of that attempt will not be accepted. Dr. Thaddeus Ex-Yank Temporal Anomalies The use of a pseudo-Romanian manifold allows SCP-2004 plan to extend into negative depth, providing 10 km² of floor space. Normal documentation of the system's construction prior to previous SCP-2000 activations has been lost. While this phenomenon has traditionally been indicative of spatial anomalies, it is the determination of doctors Robert Boyd and Tristan Bailey that the manifold interest is consistent with an advanced implementation of modern physics. Footnote 3 Transit Portal Dynamics Stretching the Brain T. Bailey et al Foundation Volume 115.2, pp 23-37-1997 This negative space is maintained via a non-gravitational singularity generated through focused ███████ particle emission across the manifold's desired entrance. In the event of the singularity's failure, the installation will remain intact in isolation and will not suffer structural collapse. Initiation of the manifold is estimated to take less than 10 hours of protocol-dead Euclid 101s enacted immediately after failure. The isolated portion of SCP-2000 will remain operable and inhabitable for up to 36 hours after the manifold fails and is recoverable indefinitely. Addendum 2000-1 Daring containment breach of SCP-████ on ██████, SCP-2000 experienced failure of several SRA and XX components which coincided with activation of the BZHR unit on site. For 25 days following this incident, BZHR units produced over 10 million humanoid entities with internal biology inconsistent with modern humans. Differences include an additional heart chamber, perfect polydactyl of the hands and feet, increased endocrinial volume and height, and the presence of an abdominal organ of unknown purpose, which emits and responds to radio frequencies in the 2.4 to 3.6 GHz range. These humanoid were neither dozed with Class-G hallucinogenics during replication nor submitted to developmental hypnotherapy. All remained unconscious until expiration five weeks later. Classification of SCP-2000-1 for these entities is currently under review. After this event is the direct result of trans-temporal interaction between SCP-████ and SCP-2000, sabotage, information leak, or non-anomalous equipment malfunction is of yet unknown. Diagnostic checks and structural repair are proceeding within acceptable risk. SCP-2000 expected to resume normal function as of January 2020. SCP-2000-2, while making repairs to SRA units in Sector-3382 on ██████, Technician ██████ reported the discovery of human remains in an advanced state of decay. Analysis of clothing fragments discovered with the remains indicates the remains are 450 to 700 years old. Valid Foundation security credentials for Dr. Alto Cliff were discovered nearby, although a genetic match could not be established. The following note was recovered from a hermetically sealed plastic document sleeve. Why did we have to build this thing? When did we do it? How long have we been doing it? Do we even know? Subsequent interrogation has verified that Dr. Cliff had no knowledge of this event and is ignorant as to the purpose of the message.


My name is Dan Williams and I am a member of Roshid's team who just presented before. So it's the networking services team. And this presentation is multi-networking Kubernetes containers with CNI. I'd like to give a special shout out to Doug Smith from whom a lot of the slides are adapted. Part of this was also presentation that we gave at KubeCon Seattle back in December. So thanks a lot to Doug for helping out with not just the main event of this presentation, which is Maltes, but also a lot of the slides too. So beyond Doug, I'd also like to thank Tomo Fung from Red Hat on the NFV partner engineering team. And then also Karal, who is from Intel and is also a member of the upstream Maltes community and has done a lot of work on the Maltes projects and the Plumbing Working Group as well. So and then of course there's the upstream Maltes community. It is an open-source project on GitHub and there's a lot of people who collaborate in that particular project. And if you're interested, join the community, help out. So with that said, we're going to go through a couple of things today. We're going to talk about exactly what all the acronyms that we are using here that I'm using today are. We're going to talk a little bit about the Network Plumbing Working Group, which is a group that we started upstream in Kubernetes and why that's relevant to this. And then we're also going to talk about how we take the stuff that the Network Plumbing Working Group developed and bring it into practice in Kubernetes. And then finally we'll talk a little bit about what's next, what's coming up in the Plumbing Working Group, CNI, Kubernetes networking itself. So some of the acronyms or other things that we're going to talk about today. Hopefully everybody knows what Kubernetes is. It's container orchestration system that's fairly popular today. And that's when you have container orchestration system like Kubernetes, you probably want the containers to talk to each other and you probably want the containers to talk to the outside world. And that's where something like CNI comes into play, which is the container network interface. And that is basically a specification and a set of reference plugins that allow network plugins to set up and tear down container networking. And that's going to be an integral part of the presentation today because the CNI interface is how all of these pieces fit together. Also POD, that's Kubernetes terminology for a set of related containers that share a network configuration, network namespace and setup. And then CRD, Custom Resource Definition, that's another Kubernetes term and that's basically just a way to describe an object in the Kubernetes API that anybody can create. It's not an official part of the Kubernetes API, but anybody can create one and that's one of the ways that Kubernetes enables extensibility of the system. And these only showed up fairly recently in the last year or two, but they've already created kind of a huge explosion in how third parties and other components of the Kubernetes ecosystem that are not officially part of Kubernetes projects interact with Kubernetes itself and the rest of the ecosystem. And finally, CRI. The CRI is an interface that Kubernetes developed that's kind of an abstraction layer between Kubernetes itself and what actually runs the containers and sets up the network namespace for the containers. One of those is you might have heard of Docker. There is a CRI for Docker, but up and coming is also the CRIO project and there is a CRI abstraction for CRIO as well. In fact, CRIO stands for Container Runtime Interface. I forget what the O stands for, but there are many talks about CRIO this weekend as well. So if you're interested in more of kind of the details and the guts of container networking, especially in Kubernetes land, check out those talks. So what's the general problem? Why are we even talking about this today? Well, the first problem is that Kubernetes really only has one network interface or one network that a container can be connected to.  If you're just running nginx, this is great. It works fine, however. That's not the only use case that a lot of people have. And over the past couple of years, we've been seeing a lot of use cases, a lot of customers of Red Hat, but also a lot of interest upstream in being able to have more flexible networking for your containers. Things that don't really fit well into that Kubernetes model, very high bandwidth, like media streaming applications that need to push tens of gigabytes per second, or gigabit per second out of a container. If you have specific latency requirements, a lot of the default networking plug-ins for Kubernetes don't really have strict guarantees or the ability to provide guarantees for these things. So, and segregated networks and legacy networks, those two are, if you have, for example, old legacy things like databases that aren't really a containerized model. Those might be over here on this network that might have a particular IP address. You need to talk to that. It's segregated due to privacy concerns or legal concerns. You want your container to be able to talk to it, but you can't hook that thing's network up to the rest of your container cluster. So, maybe you have a physically segregated network that you have to connect containers to to be able to talk to that resource. So, these are a couple of the things that don't fit quite as well into the Kubernetes networking model, microservices, et cetera, as, you know, just simple web servers or, you know, databases and web apps. So, what's the network plumbing working group? That is a group that we formed to kind of tackle some of these problems. We worked with a number of other upstream partners and groups in Intel as well. I'm trying to think of some of their ones well. Anyway, Red Hat helped form this group about a little over a year and a half ago. And its focus is on enabling some of these use cases that might require multiple network attachments per pod to enable these in Kubernetes, but at least initially in a way that does not modify the Kubernetes API officially. There had been discussions going on a lot in Kubernetes Network Special Interest Group around how to enable this and some POCs and things like that. But there's some resistance upstream to it, some, you know, for some good reasons. And so, the network plumbing working group was formed as a forum to be able to talk about these things, prototype them, and figure out what we need to do to maybe get some pieces of this upstream solve these problems before proposing something upstream that may or may not get rejected, might need a lot of work, et cetera. And it turned out there was a lot more work than we thought here. So the plumbing working group focused on creating a specification that bases itself off CNI that anybody could implement to provide multiple networks per pod. We did some POCs. We refined those. We learned a lot from the POCs. We developed a specification over about a year or so. We refined it. We did a first release mid to late last year and also been working on a reference implementation of that, which is Maltis, which we'll talk a little bit more about in depth quite soon. If you're interested in helping out with this group, joining this group, I have the link to the community right there. And that includes like leading times, the purpose, some of the things that are being worked on. And there's also in the slide deck a link to meeting recordings on YouTube. All of our meetings are public. All the meetings of Kubernetes network specialist group are also public. So it's a very inclusive community. Feel free to join. We like everybody's ideas and we'd love to have you join. I'm going to skip that slide for time reasons. So the spec view one, again, short-term solution. There are other groups that are exploring much longer-term solutions, like network service mesh, if any of you are familiar with that. But the network plumbing group was focused on what can we do to enable some of those cases that we talked about sooner rather than later without changing Kubernetes API, because that's very hard to do for various reasons that I won't necessarily get into unless you really want to know. And basically a lightweight standard that anybody could fairly easily implement. Beyond that, it does use CNI, but we found while developing the specification that there were other people who didn't want to use CNI plugins necessarily to do the specification. So we worked with those people and we tried to figure out ways to make sure the spec would work for plugins that didn't necessarily use CNI. We also want to coordinate with the resource management working group, and that's a working group that's focused on some things like scarce hardware. So if you have SRI or VNICs on your nodes, that's something that you only have a certain number of, and they only have a certain number of capabilities. And so you need to figure out, well, if I oversubscribe this node with pods that require that capability, can't do that, things will fail. You want to stop that before it actually happens. So we're working with them to try to figure out how we can best do some of the resource management on the nodes and make sure that we prevent these problems before they occur. So we'll quickly go over the specification. It has a couple of parts. The first one is an annotation. So in Kubernetes, you define everything through usually YAML files, and so you'll add an annotation to the pod object when you create it that says, I want to attach this pod to network A, or in this case, network foobar. When that happens and the pod is created, the node will actually go off and it will attach that pod to the cluster-wide default network, which is the normal Kubernetes network, but then also to foobar. And then the implementation, for example, Maltes, will take all the information about MAC address, IP address, other characteristics, and publish that back to the Kubernetes API. Currently, the only thing in the CUBE API that gets reported about a pod is its IP address, and it turns out that that wasn't actually sufficient for a lot of the cases. So you can kind of see here that there are a number of pieces of information that get published. Another thing you can see is that pods can have multiple IP addresses. That's something that Kubernetes upstream itself only is really starting to deal with, and that was only because of IPv4 and IPv6 dual stack. So we kind of tried to incorporate those kinds of things into the specification already so that it would be compatible with future versions of Kubernetes. So you can see that, so that's the second part. The third part is that the specification defines a custom resource definition, which we talked about earlier. And the custom resource definition just says, this is what my network needs to be created for the pods. Here are the properties that this network should use when a pod is connected to that network. And you do that through the CUBE API. There are some additional components. For example, you can think of, you might not want every single pod in your cluster to attach to a given network, so we need to make sure we have access control for these networks. And there's also admission controllers for validation. An admission controller is simply a component in Kubernetes that allows validation, allows access control of things before they get added to Kubernetes itself. So it kind of routes your request through the admission controller, if the admission controller says yes, allows that to be added to the Kubernetes object store and ecosystem. There's also some upcoming stuff to help other implementations. So, let's talk about multis. We call multis a CNI meta-plugin, because essentially what it is is a shim between Kubernetes and a number of other network plugins. And it kind of multiplexes things, which is kind of where the name multis comes from. So it allows you to attach more than one network to any given pod in Kubernetes, and it understands the network plumbing working group specification, which allows you to do all these kinds of things. So, again, the problem, just to recap, each pod only has one network interface in normal Kubernetes. That's not particularly dynamic. You only get one thing. We need a little bit more flexibility. Let's help with this flexibility, or help with this, and provide the flexibility. Well, you define the CRDs that define all of your networks for the cluster. Multis looks at those, it reads those, it figures out when your pod is born on a particular node which networks it needs to go attach that pod to. It looks up those network definitions, and it actually makes that happen, which we'll go into in a little bit more. So in this example, you can kind of see how multis will attach two different networks to the same pod. And you get, you know, Mac, VLNAG. The second network is going to be any CNI plug-in. It doesn't really matter. So it's fairly open, it's fairly easy to specify what kind of networks you want beyond the default one. So, key concepts. The specification, okay, let's back up a second. Kubernetes requires a cluster-wide default network. It has certain guarantees or certain things that it expects out of a network plug-in and so the specification calls that the default cluster-wide network. And that provides the backwards compatibility between what multis does with multiple networks and what Kubernetes expects. It always attaches the default or the pod to the default cluster-wide network, but then all of the additional ones are secondary sidebar networks. And what that means is that they're always going to be additional. You have the default one always, or more of these secondary networks. The secondary networks don't have quite the same guarantees as the default cluster-wide network. For example, you don't have microservices on those. You don't have any kind of network policy on those networks. We're going to work on adding that in the future and exploring how to do that. But at the moment, these secondary networks are very targeted and very focused. Custom resource definitions. Basically what happens is you say this is a description of my object. In our case, that would be these secondary networks. And you tell Kubernetes what this particular object looks like, how to define it, how to validate it. You add that to the Kubernetes API and then anybody later can create objects using that kind of type, using that description. For the Networkline Working Group specification, you can kind of see the example of what the pod annotation looks like to select multiple networks. Right here, you can kind of see this is an annotation that's kind of defined by the Networking Group. And it has a name for each network. And so you can say, okay, well, I want to attach this pod to the control plane network. And I want to attach the pod to the data plane network. So these names, which are the pod specification, map down here to the actual object that you have defined for that network. And that object has a couple of properties as well. And this is basically the CNI configuration for that network that describes how you're actually going to attach pods to that network. This is a little bit more detailed definition of how this works. This is the object that we've been talking about the network attachment definition. And this is what you create the CRD for. So the CRD tells Kubernetes how to interpret this particular object when you've added this object to the CUBE API. So you add this once, and then every single node on the system is able to see the configuration for CNI and to be able to create pods that attach to this network. So, how do you start a pod with one of these additional interfaces? Pretty easy. You use an annotation, and you say, this is the network that I want to attach to. This name maps back what you're just looking at that describes that MAC feel-in network. So that would be here. And you can, there's a couple of formats for this annotation. You can use the short format which is a lot more user-friendly, and that just says the name. But there is also another format that allows you to describe things like what's the MAC address that I want this interface to have? What's the IP address that I want this interface to have? What is the network interface name inside the pod that you should have? So that it's not completely random and your application inside the pod can expect a certain network of your face name. So then, of course, after you attach this pod to a number of different networks, how do you even get those results back? The specification defines and multisimplements a way to publish this information back to the Kubernetes API so that you can inspect it from your other applications that you want. And once you see the information here, you'll see the secondary network interface and that's the MAC vlan1. And I guess we'll show the status in a second. But now we have demo time. So, and we'll just do a really quick demo of how this works. So you can see here I have a small Kubernetes or small OpenShift cluster and it has two nodes in it at the moment. There's a master and there's a second node right there. And this is just showing that multis is running in that cluster and managing the network configuration. Yep. Is that better? Okay. Good point. So, what we're going to do first is we're actually going to create a pod and we're just going to use nginx all and it's really simple. So, we create the nginx pod and just run it. Create itself for a second there. Sorry about the wait. I actually had pulled this image before but apparently that's not the case anymore. Anyway, we will come back to that and hopefully it will be where it needs to be. So while we wait for that what is next for the network plumbing working group? We have some minor specification updates. Obviously, not everything is perfect the first time around. So we found some changes that we need to make. We found some errors in specification. There were a couple of small problems that we had to address. For example, what if you want to specify multiple static IPs? We had to add multiple static IPs but with a network prefix. So if you want to do like a slash 24 slash, you know, 16 whatever you want your static IP to be, we found that that wasn't in the specification. We added that and we also found that that wasn't possible with CNI due to some of the conventions that CNI had and so we had to also update CNI. So there's kind of been a cross, you know, pollination I guess between CNI and the plumbing working group. We work pretty well together. Also some of us on the plumbing working group are maintainers of CNI so it's very easy to make these changes back and forth. Some of the other minor spec updates adding some of the capabilities that Kubernetes allows and making sure that those were expressed in the specification. Kubernetes allows things like port mapping, bandwidth QoS type stuff and it pushes those through into the network plugin but we also need to make sure that the specification allowed passing those through to something like Maltis and to those subplugins. We also, as I talked about before because these secondary networks are not really full citizens yet and that can be a problem. So for example if you want to run a media streaming service but it needs high performance you might have a second network interface that's dedicated to media streaming but you want to have a service on that particular network so that clients don't have to connect to a particular IP address they can just use a domain name and Kubernetes to figure out which pods it goes to. That's not currently possible because of the network plumbing groups because the plumbing working group is attempting to not change the Kubernetes API yet. So what we need to do is we look into PoCs and do a little research to figure out what's possible there. One of the problems with that is that if you have a second network but you expose all of these kinds of things like the pods IP address, the service virtual IP, those things to the Kube API, how does something that's reading the Kube API and trying to talk to it know that it has to use this completely separate physical network to access the pod. So we have to solve those kinds of problems. We also have to take a look at network policy on these second interfaces because again the network policy is talking about can pod A talk to pod B can anything in the network name can anything in the project over here talk to the project name space over here which at least an open shift is one of the ways that we implement multi-tenancy. That's not so easy because if the pod is on two networks at once how do you know what can talk to each other how do you know that these things over here are supposed to be able to access the network. What happens if you can't actually talk between the two networks on the physical level. So that's an area of research that we're trying to work on. And also dynamic interface attachments. Those are right now Kubernetes expects that when you start a pod it has your cluster by default network and through Maltis and the specification you get these additional networks but you can't add and remove them on demand because Kubernetes really does not expect that. Well it turns out that some people want this. There's a lot of interest upstream in being able to change the pod definition after it started and have those networks automatically attach and detach. And because we have this shim for example Maltis in between Kubernetes and the pods themselves this shim can sit there watch the Kubernetes API and decide oh hey I noticed that this network is now present on the pod specification let's add it to the pod. Or for example removing it. That's a use case for the particular use case that somebody is very interested in is dynamic routing and so they're kind of building an architecture where some of the routing logic is actually in the pods but to be able to do that you might need to add network interfaces remove them from the pod to be able to dynamically update that system. So we're going to work on that that's actually not really that hard because it's just doing the same exact operation just at a different time. So that's going to come up as well. And because not many people have done this type of thing before we're going to try to figure out if there are implications for Kubernetes there might be we'll see a lot of this stuff you really don't know until you try it and not a lot of people have tried this kind of thing before. So for CNI itself there are two parts to CNI the first one is the specification that anybody can implement and there's also a set of reference plugins for CNI. So next up for CNI we're going to release a new specification for CNI in the next couple of weeks or maybe a month or so and as part of that it adds things like check support which is network health checking previously Kubernetes has not really had the ability to say hey does this pod actually is its network actually healthy does this network actually work. It has a higher level health checking where it actually queries the service inside the pod so if it's a web server it'll actually query the web server and say hey does this web server still healthy but there's not a way to say is the network itself that the pod is attached to actually working. So that's something that we add to CNI and then eventually we'll also add to Kubernetes itself to call that functionality of CNI and when the network is unhealthy Kubernetes will kill the pod and we start it somewhere else or maybe the same note doesn't matter. Finally cash results in the helper library so CNI has kind of a helper library that Kubernetes or any other runtime can use and that currently what happens is Kubernetes calls the add request for the pod it gets back the IP address throws everything else away that doesn't work that well in some of the cases for example when you want to check the network health again so we added support for caching that result from the pod network setup so that it could be used later that you get more information that Kubernetes could use later because again right now it only stores the IP address and that's not really sufficient we also have some more reference plugins there's a firewall plugin that works with IP tables also works with firewall D that will help in some cases you need to punch roads through the firewall to do certain things the firewall plugin will allow that fairly easily there's also a new source based routing plugin that helps with some of the VRF which is I think virtual routing that was contributed upstream and recently merged MULTUS itself what's next for MULTUS we want to like I said before try to figure out how these secondary networks are actually going to work with services and network policy so there's going to be some POCs going on right now about that we also want to do enhanced security currently the way that access control works is if this pod is part of a namespace you restrict the network to the network definition to that namespace too and if you're not in the same namespace you can't actually add the network that's not sufficient so we're also going to investigate how we can make that more fine grain so you can give specific users perhaps or specific cluster rules access to certain networks there's also going to be refinements network plumbing working group specification I mean like I said it's kind of the small fixes we have to make sure that the MULTUS reference implementation is updated for those small fixes we also want a conformance test framework for the specification because MULTUS is only one implementation but there actually are others out there and so we want to develop a conformance test so that we know that certain plugins will actually implement the specification and implement it correctly but that also works well for MULTUS because we've found in the past that MULTUS itself didn't correctly implement the specification so it would be useful all around and then again continue working with the device management group on things like SRIOV if you only have the ability to have 32 virtual functions on your NIC well don't start 33 pods that require a virtual function to be inside the pods network namespace because clearly that's not going to work and unhappiness results the other thing we might want to make MULTUS a library because the functionality isn't something that's like earth-shatteringly complex so if we make it a library it could potentially be integrated into some of the Kubernetes container run times like CRIO and then you wouldn't necessarily need this shim because CRIO would automatically understand by default that if it sees that this pod spec has a couple of networks that it should be attached to just go off and do that basically fold the shim layer into the actual container run time itself it's totally possible not sure if that's actually going to happen it or not we're just kind of exploring and thinking that maybe that's the direction we could go and the last thing to talk about is network service mesh that's an upcoming attempt at solving this problem in a much grander way if you are interested in network service mesh happy to talk a little bit more about it in this particular presentation so let's actually go back and check out the demo if we can see if that got where it needs to be nope, it sure didn't so unfortunately we will not have the quick demo but if you're interested in any of these topics we would love to have your help love to have your input I've put the link for the programming working group community right there we have meetings every other week and we love any kind of help or input that anybody has so with that and minus the demo I'd like to open up the questions Jerry you said that there are some problems with getting the work upstream what are the problems right yes, Jerry asked I said that there were some problems with getting some of these features upstream into Kubernetes what are those problems and why after two years or more are these sorts of things upstream I don't know I'm assuming most people are familiar with networking in general in this room at least at a basic level well it's very complex and so one of the problems is that because everybody has something different that they need out of networking because all of the ways by those needs are different which interface type do you need to use do you need to use a particular vendors hardware and software combination to get your cluster networking working what methods do you need to use routing or is it going to be like layer 3 or do you need to use layer 2 all these kinds of things Kubernetes really does not want to be in the business of defining a certain set of capabilities that networks should have for your containers so they kind of want, because of the complexity to just wash their hands of it and push the kinds of things you need to do off to custom resource definitions like we've described here push it off to the network plugins themselves that's kind of why the CNI layer was added to Kubernetes a couple of years ago was to get Kubernetes out of the business of defining the properties of the network then just move it all into a simple add this container to the network, remove this container from the network call so all that stuff is pushed down in and the problem is that when you encode those kinds of things like those kinds of ideas about what a network is and how a network works into the Kubernetes API it's API, there's guarantees there's stability guarantees there's backwards compatibility guarantees and because it's so complex you don't want to have to deal with those kinds of things and formalize that stuff into the Kube API so we're kind of left with how do we make these things happen outside Kubernetes and maybe take some of the things that we learn and bring those back into Kubernetes but in a much more generic way and one of the thoughts there is what if you describe the things that your application needs out of Kubernetes like, okay does this application need a ton of bandwidth maybe what is the minimum bandwidth requirement for this application or what is the minimum QoS guarantee or what are the minimum isolation guarantees that you need should this product network talk to these other networks or should it not talk to that network and then based on those kinds of generic properties or requirements of the container maybe the Kubernetes ecosystem would figure out which actual back end network to attach the pod to without having to say this needs IP address you know, 10, 1, 1, 3 on this network with this particular MAC address on this particular card that can deliver a 40 gig per second so that's kind of one of the reasons why how we're going to approach that problem but yes it's a very long road and everybody needs something different from networking I'm sure you're not as well so, yes developing like trying to make second day networks first class citizens do you have in mind also networks that want to stay second class like the two networks yeah, they could I don't think that those things are mutually exclusive you know you don't have to define services on those secondary networks if you don't want to it's just that we want to allow that possibility to happen because there are some use cases that want that possibly sorry, the question was these the networks that multis attaches beyond the default network are secondary when we talk about trying to make those secondary networks first class citizens are we also accounting for the fact that maybe some of these networks don't want to be part of the part of Kubernetes in general or use the constructs that Kubernetes gives to networks by default so is that going to answer the question yes any other questions so the first question was how do we interact as like network multi-community C&I etc how do we interact with the Kubernetes network special interest group and what's the cooperation there and what's the maybe timeline for getting these kinds of improvements upstream into Kubernetes the second question was have we played with the hardware side of things like SRIV and that kind of stuff so first question I'm also I'm a co-chair of the Kubernetes network so I'm in both meetings and there's a lot of other people that are also in both meetings back and forth so there's a large degree of cross-pollination between those two groups just based on the members being similar in a lot of cases other than that we regularly bring specific issues back to SIG network so that we can talk about the way or get feedback on those proposals those ideas that's also a forum that a lot of us on the plumbing working group use to kind of keep track of what's happening in Kubernetes in general I'll give you one example there's been a lot of discussion back and forth about how should Kubernetes deal with IPv6 and so it was decided that yes Kubernetes should deal with IPv6 because it's kind of important well that means that pods might need multiple IP addresses because you'll have a v4 and you'll have a v6 or one or more so there was input back and forth we want to make sure that as a plumbing working group representative as I'm standing over here as a plumbing working group you want to make sure that when Kubernetes makes changes to its API they don't adversely impact you but standing over here as a SIG network representative I want to make sure that what gets added to Kubernetes is generic enough that anybody can use it and that it's worthwhile for anybody and so it's kind of taking those things and trying to make sure they happen it's a little bit challenging but it actually works out fairly well and we've been able to in the IPv4 v6 case kind of chart that path and make sure that it will be useful for plumbing working group and others but also useful in general for Kubernetes so I guess that's one specific example of kind of the cross-pollination that we have for the second example for the second question SRIOV for example Kural from Intel is working pretty heavily on SRIOV they have a device plugin so without getting too far down the rat hole of device plugins device plugins are what the they're basically built for managing hardware resources on a node so things like SRIOV InfiniBand things that aren't normal network interfaces that have some finite resource a device plugin is what knows how to configure that particular piece of hardware, knows how much of that hardware exists and that's what Kubernetes talks to to actually bring that hardware up at pod creation time so the device plugin is something that Kural and others at Intel have been working on and they have software that will actually handle SRIOV parts and interact with Maltis to make sure that you can do SRIOV with Maltis in the Kubernetes cluster and that's actually it should be available already and it works there's some Git repos at this link right here not the community link but if you just take off the community part and now we're pulling the working group there is a SRIOV device plugin, there's an SRIOV admission controller and if you grab those things they also publish images as well because all these things actually run as containers in the Kubernetes ecosystem and so you can use those today it may only work with Intel cards at the moment but there are others I think that Melanox is also working on device plugins for their stuff NVIDIA also has a device plugin for more of the Infinivan side too so does that answer the question? Yes, on the end so when you have the service running to a bunch of pods which have several several interfaces how is the end of the list calculated? So currently the endpoint list only includes IP addresses from the cluster wide default network so you do not have endpoints on the second networks yet and the problem there is you have to keep the guarantees of Kubernetes API compatibility sorry the question again was how are the endpoints when you have services and you have pods on multiple networks how are the endpoints calculated what does that endpoint list look like it does not include any of the network interface IPs on secondary interfaces at this time because something reading the Kubernetes API sees a list of the endpoints and assumes currently that it can reach every single one of those endpoints but if some of those endpoints are on a separate or like physically separate network or even logically separate network something reading the Kube APIs can be able to necessarily talk to those now you can get around this through like proxies between the different networks or like some other kind of connection between those networks but at the moment that is not possible we are not going to try to jump into that mud pit at the moment but that is something that we are looking at trying to solve how to solve there are some ideas around a fully connected cluster where you have essentially two separate networks but every single node is connected to both of those networks that is a use case that is a lot more easily solved than if some machines are hooked up to this physical network and some machines are hooked up to this physical network and they can't actually talk that to each other so not quite yet but hopefully over the next year or so that might happen yes so I missed the middle part of your question could you repeat that again sorry it's a little noisy that for coming out streaming they serve for example a right of course that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not that they are not important range. That currently is hard to configure. Are we thinking about making that easier? That's one of the issues, but instead of TCP or UDP, maybe we'll have to support for the kind of protocols, low level. Yeah, so also about protocols, not just TCP and UDP, but other protocols. SCTP support was recently added with the QBAPI. That may or may not be interesting to you. And so that's one example. We know there have also been requests, at least on the open shift side, to have port ranges. So again, another example there. No, Kubernetes does not make that easily available. But we certainly want to make that easier. We know that those use cases exist. I don't think there's a plan specifically for it. But if this is something that is interesting to you and that you need, I'd say get involved in SIG network or network-only working group. Let's figure out what that use case is and let's figure out how to address that upstream. Okay, last question. Can you explain a little bit about the SCTP support? On the SCTP? Yeah, yes. Okay, the question was, can I talk a little bit more about SCTP support in Kubernetes? Kubernetes only really cared about TCP and UDP as protocols in the API. It was recently updated, like the Kubernetes API objects were updated for SCTP support. But of course that requires that you implement that support in the proxy layer and also potentially your network plugins. So just because Qube allows it now as part of the API, there's kind of a little bit of lag between when the network plugins and the proxy and stuff actually end up supporting that. I believe Qube proxy does support SCTP now, but not every network plugin actually uses Qube proxy. Yes, exactly. So yeah, those plugins, yes. So it's going to be a little bit before some of those plugins support it. Some plugins might not actually ever support it. But if there's certain plugins you're interested in, you might need to contact that project or that vendor or something and ask them to add support for it. I think it was like the Kubernetes 112 release when it was added. So that was only mid-last year. So it's still pretty recent. Any other questions? All right. Thank you very much and let's see if we...


Good afternoon, everyone. I'd like to welcome you to today's virtual panel titled Genomic Data Sharing Policies, Global Challenges and Innovative Solutions for Sharing Human Genomic Data. My name is Erin Ramos. I'm a Deputy Director of the Division of Genomic Medicine at the National Human Genome Research Institute. I also serve as the Co-Chair of the NHGRI Genomic Data Sharing Governance Committee, along with Barbara Thomas. And today I'm joined by my colleague Elena Ganiam, a Health Science Policy Analyst, who oversees and coordinates our genomic data sharing activities at NHGRI along with four terrific speakers that I will introduce in a moment. The responsible data sharing is the central tenet of the National Human Genome Research Institute and was emphasized in our recently published 2020 Strategic Vision as one of the guiding principles and values for human genomics. We've assembled this panel today to describe the present day perspectives on genomic data sharing, current efforts to create new and improved genomic data sharing policies, and efforts to ensure better data sharing in practice. So today we'll hear from Mary Enderlich-Majumder, Professor of Medicine in the Center for Medical Ethics and Health Policy at Baylor College of Medicine. Colin Mitchell, a Senior Policy Analyst at the Ph.G. Foundation. Jonathan Lawson, Software Product Manager at the Broad Institute. And finally, Alyssa Pritchup, lead for shaping the future of health and healthcare at the World Economic Forum. So each of our speakers will give approximately a 10 minute presentation. We've reserved time at the end of the session for a Q&A with the panelists. So we encourage each of you to pose your questions in the chat, both on Facebook Live and Genome TV. Thank you again to everyone who joined the session today and I'd like to turn it over to Mary for our first presentation. Thank you, Mary.


Hi, this is Yoho Sapnibhatiya and today we have with us Webbrown, CEO and co-founder of KubeCost. It's great to have you on the show. Great to be here. Thank you so much for having me today. I was actually looking forward to this discussion because you might be surprised or not surprised that I was at KubeCon. We did a lot of interviews there. Your name will come up in almost every other discussion. So I'll tell people that, yeah, I'm going to talk to him in a couple of weeks. So I'm looking forward to that. So there are so many things to talk about. I was just working on a story and community's complexity and cost, it does become a big topic. But before we go there, you are not at KubeCon. We will see each other at KubeCon, but I'm pretty sure that you were keeping an eye on it remotely. What was your observation? Because when we look at KubeCost, the kind of discussions that are going on there, the kind of folks that are there, that gives you some indication where the market is heading. Yeah, so I wasn't able to attend this year. We talked a little bit about it. I actually had my first child right before, but looking forward to being at the next one. But we did have a team of 14 that was there and came back with like really lots of interesting observations. And it really feels like energy is kind of reforming in this ecosystem, if you will. It felt like with the pandemic, we all kind of were really focused in our own silos in a lot of ways. And now it feels like just this reemergence of this community. And it's really interesting that like a lot of digital transformation and cloud native adoption acceleration has happened in that period. So it feels like we've really moved the ball forward in terms of production kubernetes, production kubernetes at scale. And I see more and more teams starting to think about these day two like problems, right? So now really thinking about operability around security and cost and reliability, et cetera. Which is the phase after you've really got your hands around kind of your first set of microservices, if you will. Now let's talk about open cost, which is an open source project for kubernetes cost monitoring and management. What was the driver behind this project? Why you felt, it's actually quite obvious, but creating an open source project specifically for that. And if I'm not wrong, there is FinOps Foundation, which deals with a totally different problem but which is also in terms of cost. So talk about the origin of the project, why you created it. Yeah, I think it really goes back to the origin of Kube cost, which is born out of our experience at Google, where we'd spent about five years doing various like infrastructure monitoring projects, like mostly focus around performance costs and health. So my co-founder and I worked on like internal observability as well as external observability for Google Cloud and a DevTools team. And when the Kubernetes effort was born, we saw really even early adopters at scale, struggling with even just how to like understand and monitor and allocate costs. Because all of a sudden you don't have a dedicated VM per team or application, et cetera, you have these like really dynamic services and pods containers being deployed by the Kubernetes scheduler. So that was the kind of the genesis of Kube cost. Over the last about three years now, we've helped thousands of teams get monitoring and then optimization in place. And two things kind of emerged during that time. One was that there's actually a lot of teams when we start working with them that have like implemented their own kind of DIY monitoring, if you will. Sometimes it's a simple Grafana dashboard. Sometimes it's like, you know, doing something more advanced with like Prometheus. And what we saw is that like, basically all these teams were doing it different, right? Like there's just tons of ambiguity and no like agreed upon standard in terms of how to measure costs. And then secondly, all of these partners were coming to us saying, we're hearing this demand and are, you know, problem from our users and we also want to give some visibility. And so we came together with this amazing group of partners and launched the open cost effort just last week. And it's this brand new, you know, vendor agnostic standard, which comes with a detailed spec on how to implement it. And then also comes with a going implementation, which is the Kube cost core allocation engine, which we've contributed. Because it's an open source project. So when we talk about and people talk about ecosystem, you know, when people are community, there is no community, there are communities. It could be users, it could be vendors, it could be maintainers. So if you just look at open costs, talk a bit about what kind of communities are on that or what kind of community you're...  But I do understand what kind of ecosystem is around it, who are supporting it. Just give us a picture of the community. Yeah, so it's early days and we are very much actively welcoming other contributors, but we've got this amazing mix of cloud providers, ecosystem partners, and then end users. So like AWS and Google made really big contributions from the cloud provider side. Then we have others like Adobe and New Relic and D2IQ. Those were the lot of like deep Kubernetes experience. So just tons of input from around the table from that really broad array of ecosystem partners. And now we're just in the stages where we're welcoming kind of even broader community input now that we've launched open costs 0.1 and already getting tons of great input in the almost one week that we've launched it. So we are looking for and hoping to get just input from a broader community. That includes the FinOps community. That includes the CNCF or like cloud native Kubernetes community and others. We feel like this is just the beginning. And again, we're super focused on containers and Kubernetes now, but we also think this framework can apply and this implementation can apply to other areas. Talk about the scope of the project and how do you actually help organizations or developers, I mean, depending on who really is the target there to kind of keep an eye on where they're spending and going and of course, you know, optimize it as well. Yeah, lots of great points there. We view Kubernetes itself as this incredibly flexible and powerful platform, right? And that can be applied in a lot of different ways. And that our view is one of the like core reasons why it's been so successful was that enterprise needs are complex. And, you know, Kubernetes has developed as a complex platform to try to meet all of those needs. So if you look at the core scope of the open cost project today, we are targeting Kubernetes environments that are like where the Kubernetes API is available. So, you know, that could be, you know, managed service at any of the major three cloud providers. You know, that could be on-prem clusters. That could, and we're starting to discuss this as a team, but technically the implementation will support environments like K-Native that are, you know, abstractions on top of kind of, you know, underlying Kubernetes environments. So, you know, it's that core use case. Already we're discussing around, you know, containers being run in other places and already we're discussing around, you know, external cost or cloud services that are consumed by a core Kubernetes tenants. So those will be discussions that are, you know, unfolding now or, you know, will unfold in the coming weeks and just starting now. When we talk about how this really helps teams, this, the flexibility of the spec starts with understanding cost in real time at the container level. So once you have a cost at that very granular level, no matter how you're using Kubernetes, you can aggregate it in a way that's meaningful for you. So you can look at cost by microservice or by cluster or by namespace or by deployment or stable set or job or label or anything. You know, that is a lot of the core power is this open cost model is truly built up from a per container basis.  Project is very, very new, but you folks have already planned to contribute it, you know, to CNCF. It's too early as you earlier mentioned, but just give us kind of, you know, what is your long-term goal with the project? Where do you see it, you know, evolve?  And ultimately, this is about, you know, similar to Kubernetes, which is, we want to see a neutral governance model, you know, where the community can be very actively involved and ultimately help address a lot of this ambiguity around measuring cost. Because our view is again, once you have a real-time cost language that is, you know, agreed upon, then you can start doing really powerful things like real-time alerting and dynamic optimization in ways that just weren't possible because you didn't have that, you know, complete, accurate, fair, et cetera, language for measuring costs.