My first realisation that there was a huge amount of waste of food going on was when I owned a little bar and at the end of service every day the chef would scrape all this product out of the fridge into the bins and that was kind of when my interest was first peaked and I realised before that I was just walking around asleep at the wheel. In fact the equivalent of 561 semi-trailers of food going to waste every single day in Australia and that's just in the commercial and industrial food space. So we need to step up and that's how I came up with the idea of technology being a real solution for the problem of food waste. UMEI is an online marketplace for the sale of surplus food. Sustainability in small business I think is vitally important, a trend that we've really seen consumers wanting to know more about what we do. For small businesses in particular if you are doing something in a sustainability space it's such a critical thing that you advertise that. From the viewpoint of Australian industry we've returned over five million dollars to Australian farmers and Australian manufacturers. Product that might otherwise go to waste so it's a win for the environment. We've got a win for the buyers because they're able to buy products at a reduced rate and it's a win for the supplier because they are able to see a return for product that they would normally have to pay to bury. So it's a no-brainer really, you wouldn't think it would be hard but it appears common sense as a rare commodity today. Mission Creep is one of the single biggest killers of social enterprise and for-purpose businesses. I understand why it's such a challenge. When you're on your knees financially which they will guarantee be a time and you suddenly get an opportunity to maybe make some money but it's not really in line with your mission saying no to that is really hard. But it is so important because that's where your credibility and your point of difference comes from. You've got to back your mission. So the single most important piece of advice I could give anyone is to surround yourself with people that share your passion because it's when those sort of dark moments happen that you need to have people that can dig deep and that they believe in that mission and that's what will carry you through those dark times and just don't ever give up. You know life is super busy and there's so many things going on for people. If you can make it easy for them to feel that they're making a difference then they've got this shop that's doing something sustainable and has great coffee and they've got this shop that has great coffee not doing anything sustainable. Where would you walk into? I know where I'd walk into. At the end of the day someone's got to solve this problem so it might as well be you.


Namos ka hikkharti hokal, aji ame e diskasyanat, fuzzpepperar, second modular, next partner bhi khe alasana kre. Ame e patir, mantel and cognitive development and language development of early childhood stressor bhi khe alasana kre. Ame yar purbo diskasyanat, agotya by local starot, hikkhu hokalor, kid horonor, haririk aru hansala mulak bhi kak hoi, hedi khe alasana kre su. aji ame e diskasyanat, agotya by local starot, hikkhu hokalor, kid horonor, manohik aru hengat mok bhi kak hathon hoi, hedi khe alasana kre. At the age of 2 years, the brain reaches 75% of its adult weight and 90% by the age of 5. Orthat, agotya by local starot, hokalor, kid horonor, haririk aru hengat mok bhi kak hathon hoi, hikkhu hokalor, hikkhu hokalor, kid horonor, kid horonor, hikkhu hokalor, hikkhu hokalor, hikkhu hokalor, hikkhu hokalor, hikkhu hokalor, hikkhu hokalor, har candidates hokalor, har freisem. Jump. Ni kan cram togo piki no hidha At the age of 8-9 years, hikkhu hokalor, hikkhu hokalor, nahit khe f10 njhet cikibi, hikkhu hokalor, నానే లాకుకణినిస్త్ను మిసానుసినియారెడ్ట్ఫోమనుఎస్నివిన్యారెనును అఔంట్యాను. మాను మానన్యానినిసాయా నినేథ్నారినంచినిన్. he understands basic functional relationship between things and events Switch on the fried 오픈. Access that you used to have closet to clear up the tea? מ allá, we hold up than you. then they development of  né.  né. at the month of six months, all these are tabling He will either say queremos babe glamour He will haveothe us after eight of one year he will complete her whatever you want


YouTube what's the word on back? This is actually a mutt head game mutt head the mutt head league season 7 It's probably probably the best season they've had so far full of big names and actually I'm playing the Arizona club series Champion that we watched earlier on my podcast and we also watched him win that event Which is T Dave has been around the man community a long time been a competitive player for a long time So it's definitely a good player to play a good player to watch And if you haven't checked out that podcast where I go over him winning the um Arizona club series Check it out my YouTube videos man My podcast is every Tuesday 6 p.m. Eastern this aired live on my twitch channel that link is below You can also click that and you know check out the podcast all the previous podcast so far are on my YouTube channel Make sure you check those out and get into a minute. It's really picking up some steam. You guys are really a Approving of that and I appreciate it. You know in this game. I started this in my head. I have night train Lane I have Michael Vick. I have full Pat Pete definitely some goons in the second I know he's gonna run a little bunch a little west coast and I gotta be able to stop some Stop some passing plays and that's what I'm prepared to do But I also believe he has for net it running back So we got to make some plays on that. So that's why I pretty much have night train I think I want to play him at the safety position We go starting to skunk it got to be prepared for that get a better catch from tied girly there and First drive I'm gonna dump it off the Coleman. Let's just fill it out. It's a little three three five odd I I feel pretty comfortable now against three three five odd I mean it just pretty much pick a sideline and play against the sideline is it's really a my main method of Playing against three three five odd But even though even though I feel comfortable against every once in a while you'll get screamed at You know the detackle will come free or you'll get hit off the edge right there the mill and I get a third and 17 What I'm gonna do is just dump it off here to girly good good tackle by him And I thought he did a good job tackling me get to a fourth and eight I think I could dump it off and fight with girly but he gets wrapped up and I turn the ball over on downs My first drive he comes out with for net for net breaks through the night train lane there But I'm Adrian Amos cleans it up. So we got stopped to run early We sent a lot of pressure that time we get a sack by my man Adams. I don't know who he is He's a 10 cat, but a sack from him is big huge player on third and 16th on under Patrick Peterson's face Here we go for net spinning off a night train lane right there Got a slowdown man. I turned the ball over first early get no pressure there But Tavares more able to drag down Michael Vick on a quick little scramble. That's why you have Vic Those are on a need a lot of little baby dots man He did a great job this whole game just taking underneath passes take little short ends and outs That's a big sack right there from Vic Beasley junior to 25 cap to gold get him to a third down I only rest three send the spy you playmakers and we click on Jalen Ramsey here jumping from the pass That's 45 cap Jalen Ramsey with the pick I'm on the one yard line to start what sucks, but go ahead just take that little flat sneak in a little flat Pass there get me eight yards. I'm five for five right now feeling good on offense underneath the girly truck over milton truck Sideways dubbies the only person in the truck sideways not up once again. We hit girly again Good job of tackling by him. He did a great job every time I mean I was in the flat He was able to click on and made the tackle more often than not here We get great pass protection right there from my offensive line able to hit the playmaker underneath the Randy Moss Eight for eight. I believe right now a lot of little things They're going once over once again right over the middle of field Tyreek Hill make a couple plays nine for nine We're feeling good mixing a little inside zone. Just keep them honest also keep your wide receivers fresh man You gotta run down the field all the time. They're going to fatigue So every once in a while you got mixing a run so you can hit a little baby dot over here Once again a great tackle by him. I mean, I'm I'm really completing passes and speaking and completing passes We're gonna run one in for a touchdown. That's Tevin Coleman. That's inside zone man Obviously, you want to keep some energy your wide receivers, but anytime you can bust the run. That's also great.  I feel like I didn't stop him He just made a mistake. He forced to pass and Jalen Ramsey made him pay He goes, uh, that's Shaki a tight end and Apke oils and wraps him up Nice spin move there by Leonard for net who fights and somehow gets I think 11 yards on that run right there Looked like I played decent run defense there at those underneath there a lot of people would have waited for the big play But he took the little in route wound up gaining him almost 20 yards on that little in route Get to a first down here. He has moved to come back route A lot of people are running the comeback routes now come back in a drag Like I showed you on an earlier YouTube video guys You can check it out below and here we see is tight end Jalen Ramsey I actually was on tackling aggressive right there and that was a big reason why I missed that tackling went into the end zone And so from the 20 yard line the little whip route to the tight end get a touchdown is is bad defense Though a laser over the middle to learn Randy Moss and that's officially makes me I believe 11 for 11 right now feeling great on offense I want to loft this one over top of his purple zone or his linebackers bad lineback number 52 I believe is nausea good Another dot tiger get out of bounds. We tackles me there got to use a time out Which I would have got out of bounds there next play. I got 20 one time out I got to make sure I get some points here to ivory kill underneath huge broken tackle there to get out of bounds and save a time Out that's a huge situation right there being will save my time out because I can still run Though the doubt over there came back and got he was in the end zone, but then caught her on the one We're gonna go for this. I got a quick play. I get rid of ball fast I snap and though the inside slant over there to Calvin Ridley holds on to the ball Touchdown 14 to 7 going in the half. That's how I feel a half time. I'm kicking the ball up I feel good having a point having something point leaves always great Okay, well if you're got to keep playing defense man if I get one more stop I feel like I can go down and get three and make it a two possession game Just don't give up any more big plays, especially not broken tackles I might honestly switch back to conservative tackling that way doesn't get any more of those plays where he does a whip route and gets a huge play on it Here's a little play action underneath throws at the Jeremy Shacky We bring him down just swarm around the ball My boys are fast enough to go and make tackles and we see Calil Mack just playing the tackle for me really getting off the blocks Really making a play. He gets a bad animation there gets him to a third and long Once again throws underneath we a strip haul, but the ball does not come out Apki cannot strip Tim Brown right there little a draw and your ball sheared blows it up right there for me That's not underneath and Apki just pulls them forward for another four yards on the play gets him to a third Throws over the top. Oh night train. I need you to make that play for me kid first down the Moss Those underneath here to Leonard for net just trucking and spinning trucking and spinning trucking and spinning I mean he's 12 for 13 as well. He's doing very well Although he does had a one interception was the difference in the game and speaking of interceptions Jalen Ramsey jumps that corner route man. Just the height the jump Everything about Jalen Ramsey. He's been the second best DB in the game to me behind Patrick Peterson Here we go almost lurked me right there, but Apki's just too slow to keep up with Calvin really. That's a great Calvin really card I believe he's 32 cap 36 cap something like that But he's fast enough to make plays and go to come back route 18 for 18 right now Feeling really good. He blows up my draw. I just want to get in feel a range if I get in feel a range Two possessions is guaranteed Playing a great pocket right here. I'm able to hit Tyree kill on the slant Once he does a good job tackling me holding me to 13 yard line We'll head in run twice here and take it to the fourth quarters Then the fourth quarter starts we get sacked the first play brings the crib on the 10 yard line did not expect that 351 drop back here. I'm cool taking my field but I'm gonna hit this curl over here and Tyree kills I'm gonna fight through Dion and 23. I believe is ladder more Tyree kill pulls them forward We go up 21 to 7. That's huge because that means he needs to hold touchdowns to tie this game I'm feeling I'm playing all right on defense. Just keep making them work and keeping them some time off the clock It's four minutes left. Hopefully I get the ball back under two minute warning in the first play He runs a draw, which I'm perfectly happy with because that's gonna be 20 seconds of the clock Is he 415 he snaps the next one in four minutes Thunderneath that's cool night train gonna rip the ball out and somehow we don't recover that but it's okay We're gonna go ahead and that's another 15 seconds off the clock Cover the drag that time goes over the top and night train lane goes and takes that one I think Jeremy Shacky being slow there there run the deep blue off and night train lock clicked on and just made us Just work to the ball. And that's what I mean. That's why I put that 55 cat free safety out there Who that's my first incompletion went for the high ball had Moss wide open That's definitely my first incompletion then he goes ahead and just mans up everybody and makes a mistake When you're down 14, it's impossible to play defense You have to make high risk plays in that time. He just called the wrong covers I was able to get a touchdown man. That makes the game is pretty much over now. I'm up 21 points. That's huge. Obviously man anytime you have three possessions let alone with three minutes left That's pretty much stick a fork in them. Once again, now we get to the point I'm getting a gas with a little Mac. I mean, it's gonna be a long night for you But now you get to a point where you just want to make them use some time make them use some time Hit this corner out over here to a Randy Moss Patrick Peterson could not get there in time. Those a little playmaker. We actually get a sack to varus more gets the loop So I was looking really dark for him here He runs a little bunch a little in route hits the post back and boom Troy Apke the best user in the game the most underrated cat player makes the pick and that's pretty much Want to end the game man is I'm up 21 points I want to run here and just go down here and get my field just so I can make it a max point differential in Mudhead is 24 points. So that's pretty much what I wanted to go for now Obviously if this was not a point differential situation, we just run the clock out and wouldn't take any points But in playoffs and everything's seating like that point differential matters a lot I mean, it's not something where you run the score up or try to disrespect an opponent But you want to get that max point difference to try to sneak in the playoffs not that many teams making this way to go I believe when I made this video right now, I am I think I'm one in three right now So we're not looking very good, but it was big to get this win against a great opponent Arizona club series champion way to get this game. He made a couple mistakes. I didn't feel like I played great defense I just feel like he made mistakes That was the difference in the game and and I played great offensively I thought I played like that I should be able to win most of the games here on out of my head league So please check out my twitch channel if you want to check out some of these games that I play live against the best players in the world


Talking to Richard Arras with the little Mark 1 escort beautifully prepared and it looks like it got you there in fine stead. Yeah, not bad for a farm workshop vehicle is it really. Why the silver firm, what was the appeal of the event? Well we've done the East African Safari rally, Norris and I together, successfully, we like the endurance rally with right time of year for us, we're both farmers in England so November's a good time to go rallying. And the Mark 1 BDA, what's the choice, what was that based on? Arras 2000 this is, so it's only a single cam, so we're up against the BDAs but we were good on times. And have you enjoyed it? Yes, apart from we had a day visit to the hospital midway which just put the cold water on things a bit. What do you think you would tell people back home about this event? It's definitely an endurance event, we are on the edge all the while, it lives up to everything it promises.


Great, thanks for having us. All right, so let me share my screen. So I just want to kind of give, have a little discussion about this Anvil project to try to introduce it and talk about where it is now and where it's moving into the future. And I think, you know, it's talking mostly about Anvil, but some of the themes are much broader than Anvil, where there's sort of, you know, there's some emerging technologies I think everyone should be aware of. And I just want to kind of open a discussion about where they're moving. I also wanted to credit NS, he's not on the call right now, but he created like more than half the slides here. So he's, he helped me out enormously. And then behind the scenes, there was a huge team of people that have been focused on this. And I'm really grateful to everyone that's contributed. So just to kind of open with a very broad introduction. So the Anvil is this analysis platform that's sponsored by NHGRI. So for those of you that aren't really aware of the politics. So I'm sure everyone's heard of the NIH, right? That's the National Institutes of Health. That's organized into, I think it's 27 institutes and centers. And there's a lot of independence between the different centers.  So they kind of get organized into into themes sort of centered around either a disease or an organ system or, you know, some sort of major component like that. A relatively new institute is NHGRI. That's the National Human Genome Research Institute. It was initially formed with kind of a singular vision, which was to sort of organize and launch the human genome project. So it's a relatively new institute. Over time, it's morphed considerably. It started with just, you know, kind of establishing the reference. And now it's much broader. So here's just kind of a picture of its research areas. In my mind, it's sort of the molecular biology branch of NIH, where they've been really, really innovative at new sequencing technologies for genomes, but also kind of, you know, everything, omics, so gene expression, looking at protein binding, you know, looking at anything kind of at the molecular level. It's increasingly thinking about a singular cell analysis. It's increasingly thinking about what other data types, other modalities should be augmented with just an omics to kind of make sense of, you know, what's sort of happening in development and disease in this very sort of fundamental molecular level. In terms of scope, it's a, their annual budget is something like $600 million a year. So it's not nearly as big as, say, NCI, the National Cancer Institute, but it's still a very sizable institute. Historically, NHGRI has funded Galaxy, you know, the initial RO1s. And more recently, the center grants have been funded through NHGRI. So I consider them like a great friend and a benefactor to the Galaxy projects. So, you know, there's sort of a lot of goodwill between them. So in response to, you know, because, you know, initially started basically with one genome, the initial reference genome. But since then, their mission has grown enormously in scope. You may have heard of these very large consortiums like GTACs or the 1000 Genomes Project or the Centers for Mendelian Genomics or Centers for Common Disease Genetics. I don't have a precise number on this, but something like on the order of like hundreds of thousands of genomes a year are sequenced through a variety of NHGRI projects. Today, it's a real challenge where, you know, huge amounts of data are being generated, you know, many petabytes per year. But those data get locked away into various silos. So the number one silo is like the SRA and DBGAP. But that's mostly for published data. For data pre-publication, it gets siloed away at like different institutional computing centers where, you know, there's one at Hopkins, there's one at Broad, there's one at Wash U, there's one at Yale, you know, kind of like major research institutes have their own data centers where these data get locked away. So a few years ago, about five years ago, NHGRI had what I thought was the good idea was to try to figure out solutions where data doesn't get locked away in these silos. We want to have sort of maximal exposure. We want to have maximal scientific impact from these data. Ultimately, these are taxpayer dollars that are funding these projects. We want to get as much sort of, you know, much bang for your buck as much science as possible from these data. So the idea is rather than having, you know, these huge silos where you have to copy where it's expensive, it's time-consuming to move data around, what if we could flip it around where there was a centralized resource where these data would all be sort of co-located and then users could remotely connect to it, you know, through the cloud to be able to access these data in sort of a standardized, harmonized way. Some of these data are open access, like 1,000 genomes. You can just freely post it on the internet. You can download them. But the vast majority of the data are controlled access data sets where there's individual, like patients, you know, are being treated for different diseases. They've consented to sort of release their genome data and other measurements for research purposes, but they don't want to just like blast it over the internet. They want it to be, you know, a lot of sort of safeguards put into place so that only those that are approved access those data can do so. That's the vast majority of those 100,000 genomes per year. So, you know, there's a centralized resource, but there's a lot of, there's a security perimeter that has been established to kind of add all kinds of protection. So there's encryption at rest. There's encryption between services. There's auditing of who's accessing. There's firewalls, intrusion detection systems that are constantly being activated, you know, to just to kind of monitor the perimeter, make sure that only those people that are authorized are able to get into these data. You know, I know that in, you know, in the galaxy, there's a great tradition of open access and open sharing. And of course, when we can, that's what we, that's what we, we move for. But there are these really important data sets that for really good reason are require these sorts of levels of protection. So that's sort of the initial motivation of and then, you know, to put some sort of numbers behind this. So the Anvil was launched through a few two sort of major awards about four years ago, one to Hopkins, one to Broad with a bunch of sort of subcontracts. And it's taken a few years, but it's really working now in the sense that there's a platform, we've been able to kind of start it all kind of starts with the data. We're trying to aggregate these huge numbers of data sets together. Currently, we're approaching five petabytes of data, about 600,000 genomes have all been adjusted into the Anvil. Again, a lot of this starts with these mega consortiums that are looking at, you know, common diseases, rare diseases.  Right now, we, I would guess that we capture I don't know the exact number, certainly more than 10% of data from NHGRI, maybe as much as 50%. I mean, I think that the vision is eventually, you know, in the next few years to basically have all of the data generated through NHGRI adjusted into the Anvil. We'll probably never get 100% because, you know, there's a long tail project, some are very big, some are small, some are, you know, looking at different types of data. We'll probably never get every single project, but the goal there is to get a huge fraction organized into a central platform where researchers can look at it. Mike, can I can I ask some questions or you want to do it at the end? Yeah, please jump in. So is this data as it's produced is directly deposited or is this sort of, you know, your lab does something. And then when you want to publish it, you just scramble and put it on there? Yeah, I think this is, I think this is what's something that's really special about the Anvil is a lot of these data are being loaded as they're being generated pre-publication. So for a lot of these like mega consortiums, they're using Anvil, where, you know, get 100,000 genome sequence, you need to do a mega analysis of it. You know, let's get it sort of cloud first. I'm with you though, Marius, there is this great tradition of just scrambling at the very end to upload data. We're trying to bridge. But maybe that's not going into end, but let's actually go into the public ones. Yeah. Yeah. But I think it really is sort of changing. We're breaking out of that old tradition of just uploading at the very, very end of this process. So that's what's going on inside of Anvil and NHGRI. But kind of more broadly, this sort of trend is emerging, not in all 27 institutes of NIH yet, but in several of the major ones. So like NCI, the Cancer Institute has established their cancer research data commons, heart, lung, and blood have established bio data catalysts. You know, other institutes are sort of joining this effort to have cloud platforms to sort of organize and analyze these data. From NIH points of view, it's a way to get kind of the most bang for their buck. If I'm a little bit cynical, I think there's what they're looking forward to in the future is actually cost savings in the sense that right now a lot of the institutions, including Hopkins, apply for so-called instrumentation awards, where we can buy, you know, a million dollars of computing. Doing that at every single university, you know, the United States starts to get expensive. So instead of distributing and building all these separate data centers, you know, let's effectively have one mega data center that all of NIH researchers can use. So these efforts are being sort of organized through this thing called the NCPI, the NIH Cloud Platform Interoperability Effort. Today we have sort of buy-in from a few key platforms, but my full expectation is this is going to grow and grow over time. In aggregate, there's something like 11, well, this is a little bit out of date, but there's more than 11 petabytes approaching a million genomes that are all kind of available through these different platforms and will be kind of, you know, kind of leading the way here with sort of the most data available. So it's just, you know, kind of a testament to the types of data that are being organized now and into the future. So we have this huge amounts of data. It's all organized in the cloud. A lot of it is on GCP. A lot of it is on AWS in the near future. A lot of it will be on Azure.   This is a substantial data footprint. They wanted, you know, best in class. They wanted the security. They wanted, you know, the distribution network. They wanted to sort of tap into all these capabilities. But data at rest is kind of boring, in my opinion. It's sort of necessary. It's essential to have databases. But to me, what gets exciting is when you actually have an active platform where you can have these ginormous data sets and then actually do interesting analysis on top of that. So ANVIL is, yes, a data repository of sorts. But to me, the exciting part is in addition to these huge data sets, there's also a variety of different tools that are available to kind of make sense of it and look at it in new and interesting ways. So here's kind of what we call the ANVIL Neal, which is kind of a really, really high-level overview of some of the capabilities that are present there. Capabilities for sort of programmatic access in different forms, visualization of different forms, user interfaces into the data specifically, and then sort of downstream analysis that are available there. In the kind of interesting time, I wanted to spend a minute sort of reviewing some of the key analysis platforms. This is going to be a very sort of superficial highlights tour of some of the things that are possible and some of the things that are on a horizon in the near future. So on the left here, we have sort of Jupiter and RStudio and Bioconductor. I think these are very familiar technologies to everyone on this call. So I'm not going to really spend any time on it. I think they serve an important purpose. If you want to do really customized analysis, really customized visualization, you need to write some code. I think these are great platforms. They have some trade-offs in terms of what you can do. I think they're kind of necessary but not sufficient in terms of what can be done, but they're fully featured. They're working in a very robust way. Another sort of key component is this workflow description language or widow. This is sort of the primary technology that's been adopted for some of these mega consortiums. So if you have a project that has thousands of thousands of genomes, if I'm honest with myself, this would be the preferred technology to do it. I'll say a bit more about it in a minute, but it's extremely scalable. It's extremely flexible. It's very technically demanding in the sense that there's sort of a lot of pretty low-level scripting that needs to take place to use it and there's a lot of cost considerations that come with it, but at the same time, if you have a mega project, that's a very effective technology. In addition to that, the Galaxy is sort of a first-class citizen in this universe. I'll have a lot more to say about that in just a minute. So to give you kind of a sense of what's possible with Widdle, I wanted to spend five minutes talking about a recent project that I was involved in through the Telemyr to Telemyr Consortium. This is that project that's been getting a huge amount of buzz in the last few weeks here. So in this project, I'm sure everyone is aware, there is this reference human genome. The first version of it was established in some 21 years ago, sort of first published in 2001. And that reference human has been sort of iteratively approved over the years, but sort of the big challenge there is that it is still missing quite a huge amount of sequence. If you open up the reference human today and you sort of look at chromosome one, instead of beginning ACGT, ACGT, it actually begins NNNNNNNNNNNN. There's millions of nucleotides just at the beginning of chromosome one that are unresolved, where we can look under a microscope and use other technologies to see how big the chromosomes ought to be. And we know how much of it has been assembled, so we know that there's huge chunks that have been missing. So the ends of chromosomes are the so-called Telemyr regions. The middles of chromosomes are so-called centromyric regions. And just because of the way that they're organized, historically, they've just been too complicated to make sense of. The sequencing technology just did not exist to get in there, but the amazing thing is the sequencing technology now does exist where we can look inside of those regions and actually resolve them for the very first time. So we recently published this assembly. It's a special sample called CHM-13, where we've been able to assemble every single chromosome from Telemyr to Telemyr, from kind of one tip of the chromosome to the other tip, which brought about 200 megabases of additional sequence, and it also fixed megabases of errors that were detected through the old reference genome, GRCH-38. So now we had this new assembly, and one of the key things we wanted to do was look at it and look at how it would impact our analysis and our understanding of human genetic variation. So kind of the idea is we have this new assembly. That's great. But, you know, we have an old assembly. We have a new assembly. We just want to do kind of an apples to apples comparison. That's when you look at other data sets, other human genomes, you know, what changes about our interpretation using one versus the other. So we put together a team of analysis. There's something like 50 people that were part of just this one team. We're kind of looking at trying to answer this question using long read sequencing, short read sequencing, you know, looking genome-wide, looking in clinically relevant regions, and just trying to, you know, be able to ask the question of what changes by using this new reference genome. So the data set that we ended up looking at is this very famous data set from population genetics. It's called the Thousand Genomes Project. That's the name. It's a bit of a misnomer in the sense that today it's actually 3,202 samples. It's called Thousand Genomes, kind of for legacy purposes. At the time that was a very aspirational goal, and now it's been far exceeded. So it's more than 3,000 samples. It has representation from five major continental populations. Within those sort of continents, there's 26 individual populations. That's what these different stars are sort of highlighting. So it's a nice collection because we get, you know, representation from Europeans, Africans, Asians, sort of Native Americans. You know, we get sort of a nice initial view of human diversity through this collection. Another sort of nice attribute of it is that all of these 3,000 people have been consented for open release of their data. We can just go to the website. You can download their data. It doesn't need to be done in a protected way, which also makes it nice where any results we have can now be freely shared with the world. We don't have to sort of set it up behind any sort of barriers that can limit access to these data. So it's a great collection. It was very recently resequenced by the New York Genome Center. So it's sort of fresh, high quality data. The one sort of technical challenge of it is, so it's all compressed data. It's something like almost 100 terabytes of input data. So we knew we wanted to look at this, but it's a pretty sizable collection. We actually started the analysis at Hopkins and we realized it was going to take like more of a year of compute. So we needed to kind of do this in a more efficient, more effective way to be able to look through this. So what we ended up doing was we developed a workflow that could look through these data in a pretty rapid form. In the grand scheme of things, it's actually not that complicated. It's sort of a few major phases to kind of organize and do some preprocessing, a few phases to align the reads and kind of get them organized. Not shown here is then once we have the alignments organized, we can do downstream variant calling using GATK. The idea was to kind of follow the protocol established by the New York Genome Center so that we can get an apples-to-apples comparison where if you just change the reference genome, what changes in our interpretation of all these sequencing data. I really wanted to credit my student, Samantha Zartar, who actually wrote all the whittles for doing this. If you haven't seen it before, a whittle is a pretty simple concept, at least from the sort of user experience. So it's a workflow language. It's organized into sort of a series of tasks. These tasks can be linear. You can also do branching. You can also do aggregation. You can also have some conditionals, but by and large, it's a series of tasks that get sort of chained together where the output of one task becomes the input to the next. So it's super simple. The way I think of it is like a turbocharged bash script. So this is one of the tasks for aligning the reads. So there's a bunch of input data that describes what the read files are, what the reference genome is that we're going to align to. There's a series of shell commands that are then executed. So this is sort of the fancy bash component. Anything you can express in bash then gets executed. There's a runtime environment that says, oh yeah, this is the Docker container I want to use that has the Unix tools preloaded into it with how much RAM and how many cores we want. And then there's sort of an explicit listing of which output files do we actually want to keep. For a lot of these workflows, you're going to be creating a gazillion intermediate files that are actually not that interesting. So we just want to sort of decorate, oh yeah, these are the key steps, these are the key outputs that we actually want to save away. And then those get actually sort of recorded. So it's a fancy bash script. The key to it is their execution engine called Cromwell that the behind the scenes will kind of orchestrate a whole cluster where it'll boot up virtual machines of the specified type. It'll load the Docker containers that are necessary. It'll stage data out of, in this case, Google buckets onto the local disk. It'll do the bash processing. And then it'll take the decorated files right back to the buckets. So from a user perspective, it's super simple. Behind the scenes, there's some pretty nice engineering to actually orchestrate this at scale to be able to connect these different paths together. So the computer doing was quite substantial. This is a snapshot of the Google Cloud console where we're like routinely using like 10 or 12,000 cores for a few weeks time. So we're able to tap into a few million core hours of computing over a few weeks. It did cost money. The computing part of this was about $50,000. So it was a pretty substantial compute. But the scientific outcomes from it have been enormous. So I sleep very easy at night. I think this was a very good use of resources in order to support these activities. At this time, I won't go through all the details, but just very quickly, some of the scientific outcomes. So using this new genome, which has resolved 200 megabases of variants, 200 megabases of additional sequence, we find about a million additional variants in the population. So there's all kinds of new opportunities for discovery about how those variants impact our health, how they impact disease susceptibility, how they just explain differences amongst different population traits and how they look and how they behave and so forth. In addition to finding a substantial amount of novel variation, we were able to demonstrate and prove that the variants that we were able to detect are actually of higher quality. Where there's a reduction in false negatives, there's a reduction in false positives. And this includes within clinically relevant genes. So when we started this project, we thought the whole paper was going to be about all the new stuff. But a big part of the paper as well has been improving parts of the genome that we thought we had correctly assembled before, realizing that there were some issues there and then being able to reduce, in some cases, a tenfold reduction in false positives in certain key clinically relevant genes. So basically in every way we can possibly measure, we see improvements using this new reference genome. We're really, really strongly trying to advocate for its adoption just because it makes everything better to use it. So that was all orchestrated through Wittles just because of the scale with some downstream processing in our prime end Python to some degree to make a bunch of custom plots to be able to execute this custom kind of analysis that it just never been done before. So I'm excited about it. In the grand scheme of things though, this is a relatively modestly sized project. This was 3,000 genomes. A lot of the mega projects are at the scale of like 30,000 or 300,000 genomes. So it was the biggest single project I've ever been a part of. But in the grand scheme of things is actually not, by far, not one of the largest that are ongoing in the field right now. So it's kind of exciting and also terrifying to think about what the future is going to hold here. So that takes us back to our main story about where things stand for Galaxy. So it's working well. It's working quite robustly. Any of us can log into Ample today in a matter of a few clicks, you know, bring up an instance of Galaxy that is fully featured can do a lot of sophisticated analysis though. Now I mentioned early on, you know, one of the key requirements for Ample is, you know, great sort of mindfulness for security. And, you know, one of the sort of original design decisions was, well, how are we going to make out? We knew we wanted to bring Galaxy into Ample. That was like a certainty. One of the great sort of choices was, you know, what's the right mechanism to do so? On the continuum, one extreme would have been to try to mimic something like Galaxy main where there is sort of a more or less permanent instance of Galaxy that was always running that any user could kind of immediately connect to. That I think at a technical level, you know, works well. I mean, obviously main works well. It has, I don't know, 10,000 users a month. But we were concerned about the security policies for so-called multi-tenant system. So a multi-tenant meaning, you know, many users can simultaneously access the same software stack.  I think we're like in pretty good shape. But, you know, through those audits and kind of through our own reflections, you know, I think there is some work that needs to happen. If we're going to be able to, you know, kind of confidently be able to have a multi-tenant system, it is accessible for protected data sets. I think it could be done. If we want to reach that so-called FedRAMP certification, there's a very long checklist of items that we need to address, some of which are just procedural in the sense that every, there's a set of requirements about how code changes are committed, how they're reviewed, how testing is executed. I think we're kind of moving in the right direction. But if I'm honest, you know, it would be a pretty substantial cultural shift, you know, if we wanted to go pursue that. I think there are, you know, there are pros and cons to that concept. But in the here and now, it wasn't practical to execute in that sort of scenario. It's sort of the timeframe that we had available with the sort of level of funding that we had available. As an alternatives model, and this is what's available today, was instead of having a multi-tenant environment where many users would connect to one centralized galaxy, we've moved into a single tenant model where individual users can boot up and manage their own version of Galaxy. This way, there's sort of protection where if I'm using Galaxy with my data, you know, none of you can see it, even if, but you're allowed to have your own independent versions of Galaxy. If my Galaxy needs to sort of interact with your Galaxy, you know, you would have to have an explicit export, you know, of one that can be done through the workspaces, that could be done in the case of tooling, to say Dock Store, where, you know, there are ways to kind of share information, but it's a few extra steps, so that security is really sort of a central there. By the way, that's sort of just one layer behind the scenes, and I really credit Ennis for putting together a lot of these diagrams, is the kind of user facing component is so-called Terra. There's an API layer called Leo, where through a Leo command, there's a single Hamilton chart that launches a Kubernetes cluster, you know, that where you can kind of dial in how many nodes, and how many cores, and how much disk you want to have there. That then actually, that sort of boots up Cubeman that can kind of bring up the whole Galaxy stack. And then like I said, if you want to kind of work with that, because it's kind of a fresh installation, you can bring in data from workspaces, you can bring in tooling from, say, Dock Store on demand as needed. Just one more sort of diagram here is, you know, because we're using Kubernetes, so if I orchestrate all of this, you know, we have a lot of flexibility.  You know, that can be kind of booted up. It's all kind of executed through one command. We could basically get effectively any number of job handlers, any number of sort of resources behind the scenes to be able to do the analysis there. Couple more screenshots. So this is a screenshot I took last night, kind of showing how you launch Galaxy inside of Anvil. So, you know, you kind of click this button, create a cloud environment that brings up this sort of configuration panel where you get to pick, you know, how many nodes, how many CPUs, memory, how much disk. The basic deployment is 53 cents per hour.  What's one thing that's like really cool to me is, you know, you can just sort of dial in how many nodes you want. So if you want your own dedicated cluster with 800 cores or 8000 cores, it's very accessible and easy to do so. It does cost money, but again, you know, for some of these scientific projects where we're going to be making great discoveries, I think spending some money is probably a good use of resources. Obviously, we don't want to be wasteful, you know, if we can minimize the spend that is in there, obviously that's something we want to do. We want to be aware of those costs, but sometimes it's definitely kind of an appropriate use of resources. Once that's booted, you'll get a very familiar version of Galaxy. It's branded Anvil with, you know, kind of the color scheme and some of the logos, but it's just, you know, it's exactly the same code base that's executing there. So everything you could do inside of, you know, any other instance of Galaxy is available. One really special part, and I really credit Luke and others for kind of building this, is that, you know, so this is running in a Kubernetes cluster inside of Google. You know, one of the main things you want to do is look at the data that you have available in your workspace. So this could be, you know, all 1,000 genomes. This could be all of the centers for Mendelian disease genomes, you know, basically unlimited, effectively unlimited amount of data you might be interested in. You can go in and just browse that, you know, right through the Galaxy UI. Once you've kind of identified the data sets that you're interested in, you know, in a few clicks, you can then ingest these data out of the Google buckets into the Kubernetes cluster where Galaxy's, you know, kind of really running.  It's working really well, in a really robust way. And again, I really credit a ton of people that have contributed to making this stack just as awesome as it is today. There's also nice features where it can auto pause, you know, and then once you're done with your analysis, you can kind of totally turn off the cluster. You can put it back in a residual state that costs like one penny per hour, or you can delete it entirely if you're entirely done with the analysis.  We get secure computing with direct access to huge numbers of data and growing every day. We can use Galaxy without any sort of fixed quotas. There's no competition for queues because it's kind of, it's your own private, for instance, a Galaxy that you can run with. We can connect data in novel ways and we're going to make those scientific discoveries. Just last week at NHRI Council, they have approved the renewal for at least five more years of Anvil. I kind of think that this is going to be the sort of project that, you know, in a hundred years and a thousand years, it may not be called Anvil anymore, but this is, in my mind, clearly the right thing to do from a sort of a government point of view. So I think there's going to be continued interest to build out these resources and make them even more powerful, even more capable in the years to come. Okay, so that's sort of the exciting part. A couple of reflections. So one sort of mega reflection is, you know, I think historically Galaxy has really led the way in terms of, you know, making tools accessible, making science reproducible, but there is effectively fresh competition. You know, through Terra, through Dockstore, you know, there are workflows that can be executed in a very reproducible way. They are getting hardened. You know, they at kind of a high level, they have some of the same virtues that we've been highlighting in Galaxy for more than a decade. So how do we maintain our competitive edge? In the here and now, in terms of accessibility, in terms of ease of use, Galaxy wins by a landslide, by a landslide. It takes a thousand clicks. It's very complicated to write a whittle. It's sort of very cumbersome to modify them or do anything interesting with them. But nevertheless, it can be done. And for users that are comfortable with, you know, batch scripting, you know, we have to be really mindful of that competition there and how we maintain our competitive edge. And then kind of the other mega reflection is how do we optimize the Galaxy on Anvil experience?  You know, whittles are ridiculously simple because they leverage all the cloud technologies about buckets and cloud-native APIs. You know, behind the scenes, there's a lot of cool stuff that's executed. You know, how can we tap into that sort of amazing capability to kind of just make the Galaxy all the more successful, all the more powerful? So a couple of wishlist items. And I think basically all of these are in progress right now. But this is just sort of my top of the line wishlist. You know, number one is for that TDT project, step one of the analysis is like moving around 100 terabytes of data. You know, I'm excited to hear, you know, remote data type activities are on the near horizon that will immediately be transformative to the experience. Kind of a couple of this, now that we're talking about projects in the hundreds of terabytes to petabyte scale, you know, I think we've got to be really mindful about, you know, which data are kept.  So kind of leveraging these closer object stores, I think in the long term is going to be really essential. Things like optimizing the launch, you know, in the here and now it takes a few minutes, which is, you know, for analysis, it's going to run for two weeks. Five minutes doesn't really matter. But if we're, you know, if we're trying to, you know, really harmonize that user experience, is there some way that we can bring up, you know, in a matter of a few seconds, even if it's a very minimal UI, you know, just some sort of, some way to give exposure, you know, very quickly so people can see what's going on there. And then in terms of tooling and workflows, you know, trying to harden the existing suite and more tools for human genetics, I'm very excited to hear that, you know, things like user installable tools are on the horizon. You know, sometimes when you're in the middle of a project at 2am, you know, I don't necessarily want to write a whole wrapper for a tool. I just want to get my analysis done at 2am. So that sort of capability, I think it will also make the platform a lot more attractive to a broader range of analysts. I love this. It's amazing. Thanks for stating this because that was not at all a unanimous view.  But I think we have to be kind of flexible and open-minded about this. You know, maybe not every single tool needs to end up with a wrapper. You know, maybe sometimes we can acknowledge this is only going to be run in a once. If we want to make this a platform for everyone, including pretty sophisticated analysts, you know, I think having this capability is going to be really transformative. Yes, I think the moment I started working, the moment I started working on Galaxy and talked to other people about this, they were like, but I have to be admin, right? And I mean, that's another thing that works around this, which is not a problem on Anvil, but just in general, it should just be possible to plug in your script. I agree. I agree.   You're not, you know, you're really not sure what it is exactly you want to do or how it's going to look like. There we just want to be super nimble to support that sort of analysis. I'm wondering if others have any kind of thoughts or comments? This, you know, this is just my, my personal impressions here.  It's amazing. It's super solid. You know, my goal here is just to kind of, kind of plant the seed that, you know, let's, let's consider this, you know, in a really, really deep way and what sort of opportunities are there to make it even better for the future? Yeah. I mean, you know, I mean, so if I can say something, when you talked about the reanalysis of the re-sequenced 1000 genomes project, I mean, you've answered this question already sort of, but I was wondering, okay, so why not take the extra step and do it in Galaxy? And I mean, of course you mentioned some of the limitations, but I also think we need these big projects to be in. I think a lot of improvements are coming out of the VGP project, but it's, it's not the same sort of considerations, right? So for this project, a lot is, a lot of focus is on like efficiency, efficiency of transfer of data. And it's, I mean, it's relatively removed from the core team of doing an analysis like this, right? So if we said, okay, we want to be able to redo this and maybe be dwindled in terms of execution time, I don't think that's impossible, right? But it needs more knowledge now than we could easily do with like, you know, get started. So I guess my question is like, what do you think we need to do so that this becomes an option to people that right now would prefer writing with them? I totally love the, the, the spirit of what you're saying. And I should comment, this is, you know, the, the genomes just published will be the first of many new reference genomes that are put out. You know, there's through something called the human pan genome reference collection consortium. There's something like 350 additional reference genomes that are being created. And that's just NHGRI. I anticipate in the near future, there's going to be, I don't know, thousands of human reference genomes and beyond. And you're right. I would love in say a year's time to be able to do this whole analysis entirely in a galaxy. I would, you know, I'd be open to if that was galaxy in anvil or galaxy on main, you know, it's open access data. So some of the usual privacy considerations are, are not even a factor here anymore. The reason that we picked widdles was, you know, was, was kind of this number right here intermediate data was like five petabytes. I mean, it's, it's a very interesting thing that you mentioned that we didn't even put on the roadmap. But I mean, it's obviously something that, you know, we've ignored for a long time because nobody really said, well, but we really, really need this, right? And I think this is actually more, well, I mean, I shouldn't say this, but this is as valuable as keeping the initial import step remote, right? So that, you know, the, the, you know, 100 terabyte of input data doesn't need to appear in Galaxy's object store. I think it's just as important that intermediate files that you don't need don't need me to be transferred back, right? Yeah, you know, clearly some of them need to be transferred back, but we probably don't need to collect every single intermediate. At least not forever, at least not forever.  There's new technologies all the time. There's new questions, new types of questions. So I think this is just sort of the progression of the field. And I think it can motivate us in Galaxy to, you know, adapt, adapt to the changing landscape. I'm wondering if anyone else has any impressions? Like I said, I know many of these things are, you know, kind of in flight already. So that, that's awesome. I think we're, we're already kind of pointing in the right direction and, you know, anything we can do to go even further, because we're, this is just the very beginning. This is just the very beginning. I mean, at this, at this point, I think it's also very good to just have like concrete use cases that drive development, because sometimes you're working in a bit of a vacuum and you don't really know, okay, if I'm going to spend the next six months of my life doing this, will it be worth it? Right? Or is it something that, yeah, it's nice to have, but that's, that's a great idea. I mean, one, you know, natural thing we could do is sort of, you know, kind of formalize this as a use case, and it's highly scalable. You know, we did 3,202, but there would have been value to do 30 genomes or 300 genomes. So, you know, I think we could, we could set up some intermediate goals that would be scientifically valuable. And then, you know, from those, we could kind of look to the future to tie all these pieces together. Has, has there been any work to convert the sort of reproduce the workflows that you've used here in Galaxy yet? Would that be interesting at all? That's an interesting question. I think many of the tools are available. Maybe not all of the components of GATK that we use were, but I think, I think many of them would have been, yeah. I mean, there's also the ongoing effort to support CWL, where I understand there is some conversion possible between the two. So that would be another option for workflows that already exist. Again, there was a big discussion, is it even worth doing this? Because, you know, then you end up with parallel pipelines, pipelines defined by somebody else. But I think this is just, you know, this is a production pipeline. And for reproducibility sake, you may want to execute it the way it was written. I agree. And we've had subtle bugs introduced when we try to, we've converted from like Snakemake to other workflows and subtle bugs pop up. So, you know, being able to just run the native, hardened, tested workflow brings a lot of value to it. I mean, we always have, in general, there's always going to be workflows that people publish that aren't going to be polished by the IWC. But that's kind of the point of the IWC is having those specifically hardened workflows. But that's not to say that there's no benefit to having things outside of that. A random thought is that I really like this wish list. I'm 100% with Marius that user installable tools are so important and don't ever get sort of, they don't ever sit at the table in the way I think they should. And this, you know, purging the intermediate stuff and workflows, we spent a lot of time, like, I mean, again, I just wanted to echo what Marius said in terms of like, the initial data is not so, I mean, we've done it, we made a bunch of progress towards it. It's good that we did that. But we could run a whole workflow on a node on Anvil and just collect the outputs of the workflow and where, I mean, it feels like that would be, I don't know, it feels like there needs to be a pipeline to get from Anvil, from these like Anvil wish list things that are things that would really just help Galaxy in general into like the core team or like onto the roadmaps and stuff. I just wanted to like, I don't know, showed up those two particular things, which I think, I mean, obviously our big, important, exciting developments that should happen and presumably could happen in, you know, we could have both of those things done in the next year if we put our, if they became priorities. So I just wanted to sort of echo Marius, I think those should be priorities and it's really good to see an awesome use case for them. Yeah. And I don't know about ongoing projects and I mean, everybody's drained or maybe I'm just extrapolating for myself. I mean, we're pretty busy, but you know, I mean, if another thing like this comes up and, you know, you say, okay, now we want to scale an analysis of that size or more, we just, you know, want to see how far can we get with Galaxy. I think that's something where an engineer's time is well spent. I love it. I love it. Let's let's do that. Let's do exactly that.  So we're perfectly pleased to do this. And if we can kind of tie analysts with some of these use case or engineers analysts with some of these use cases, I think that could be enormously impressive. Awesome. Awesome. And you know, I also appreciate, you know, this is all these are not five minute jobs. So it's the kind of thing where, you know, months or years of work are going to be necessary to fully achieve it. But I think it's useful to have a long term vision as to where things are heading. All right. Thank you. Thank you, Mike. We are over time. But if there's any additional comments or questions, you can take them. Yeah, we should probably wrap. I would say just poke me on Slack or other if you want to chat more. And I'm looking forward to the workgroup meeting and what a month's time or so. And then I'm especially looking forward to seeing many of you in person at GCC. Thanks, everybody. All right. Thanks, Mike. Thanks, everybody.


July 23 2009, Mark Burley is tossing a gem in Chicago. He's perfect through eight. And the day didn't start out that perfect. We got an article here from Lee Jenkins says that burley always bought a rockstar energy drink from the Shell gas station before every start when he was 10 minutes from the stadium. He looked down at his cup holder and noticed it was empty.  He texted his wife, Jamie, forgot my drink. I'll definitely lose today. Even more interesting is the backup catcher Ramon Castro showed up and found his name in the starting lineup. He rushed over to burley to get the signs said fastball cutter change and burley said, you know, I throw I throw a curveball too. So Castro was a May acquisition from the Mets who was catching burley for the first time ever had no idea that he threw a curveball. And the last one is that the White Sox were playing a day game after a night game with a double header coming up the next day. So they tried out a lineup that looked more like a split squad. Gordon Beckham was in the minor leagues two months ago and he was starting at third base, which moved Josh Fields to first base for the seventh time in his career. Burley's command was so poor in his pregame bullpen session that he told Castro, don't worry, it'll get better. And it did. He's perfect through eight during wise comes in to play center field in the ninth inning. And we got the leadoff hitter up ball outside. Nice little cutter in one in one count outside and soft foul ball. Oh, you know who this is. This is Gabe Kapler, our good dear friend.  Gabe actually puts together a decent at bat one to takes the high fastball. Then that fastball he turns on but turns a little too much but got good wood and then here's the play everyone remembers to to bombs it out there and wise tracks it down grabs it tumbling catch.  Okay, what's going on? This is what the fans are going insane. Burley couldn't believe it. Wise picks up his sunglasses casually put some back on his head. Oh hum. I liked how you can see like when you can see he's he's got it I'm going backwards now. Alright, so he's got it peaks at the track in the wall to find out where he is quickly gets it again and never looks back at the wall after that peak never looked back knew exactly where he was grabs it. Oh my God, did you see the kid behind home plate never noticed this when I was editing it. This kid behind home plate gets sat down by this foul ball. Anyway, this next batter now wise has made the catcher two outs away. It's like fuck, I better just do it now. He falls behind him three one. And that dude doesn't swing at the three one. What are you looking for a walk so it goes from perfect game to no hitter get a hit dude. Three one swing the bats a perfect game be a bully. Instead burley's like fine, I'll just go off speed because you have to swing now. And just pulls the string on him makes him look bullish. Now they're standing up in Chicago one out away from the perfect game. There you go 81 right in there. His next pitch called a ball all the fans wanted it. A lot of fakeouts are out there one and one cutter low two and one. I mean burley was the most fun to watch pitch he goes off speed gets it to short but bam. Also burley's reaction here just glove on head in awe kind of like holy smokes what happened and then the teammates. I love perfect games because everyone needs to be perfect sure like 99% of the credit goes to the starting pitcher but like everyone had to be perfect in the field. It's a team accomplishment but 99% is there. So anyway says Haida was wife is baby. She got cute dress on and then says thank you to wise hey thanks for making that fantastic catch. I'm gonna go shower now and then wise says, you know, you guys saw that right? That was pretty cool. I mean defensive replacement to come in. Oh, here's a sign. I'm a socks fan. I always make the wise choice sponsored by OfficeMax. Wow. Great job by OfficeMax sponsoring that sign. What's up with her arm? It's like she's she got Tommy John surgery and we got pie to the hair in the head. Slick it back mark looking beautiful. One of the cooler defensive plays in a perfect game because everyone tuned in because everyone knew it was a perfect game. So I remember being home and not watching it but then ESPN broadcasted everyone tunes in and you're hoping to see something exciting not only do you see that that catch every perfect game no hitter comes with a defensive gem but for it to be the defensive replacement in center field in the ninth inning just insane.


Hi everyone, welcome to the Jenkins governance meeting today is November 4th, we have several contributors on the call, me, Mark Waite, Uli Haffner and Vip Haffner, so we have a few topics and agenda today, we will firstly start from news and various ongoing events in the Jenkins community and we will talk about Jenkins elections, Mark has an update about the PFA video, I have an update about Trademark, which is a no update, we will get there and also we wanted to quickly discuss, search for Jenkins IO and our options there. Okay, let's start from the beginning, so for Jenkins elections we have announced 2020 board and officer candidates, so you can find the list on the website, there was a announcement about that, you can see that there are nine candidates for governance board, you have two positions after election, so if you want to support any particular candidate, now it's a great time to actually register so that you participate in the elections starting next week, so there is a big register button here, you can also find ways to register here in the bottom, we will also have elections for the Jenkins release officer, we have three candidates at the moment, for the rest of the officers after the initial discussions with the nominated contributors, we ended up with only one contributor running for the elections, what it means that there may have been more contributors, but then they decided not to compete in the elections and finally we got four uncontested officers, so elections will happen on the photo positions. Okay, so key dates, November 8th voting sign up is over, so it's this Sunday I guess and there is still some time to register, so again please do so and after that we will start communicating with all contributors so that everyone can participate. Okay, any questions about the elections? Okay, so more updates, we have recently had Jenkins releases, Mark would you like to summarize them? Sure, 2.249.3 released today, it's a relatively small update compared to 2.249.2, two regressions fixed and one additional fix, we've released 2.264 with the tables to divs implementation and we've got some additional upcoming bold changes coming, oh and plugin security release, good observation, yeah. Yeah, I also forgot about that, so yeah important to note for weekly releases, there will be a period of instability, maybe a few weekly releases, because we have a number of changes coming down the line, so Mark has a summary there, please be ready to that. Right and we're looking forward to those, each of those is a good step and a valuable step and today we had an announcement of plugin security releases for several, particularly if you're using the Active Directory plugin, please upgrade. So yeah it's here, you can see that the list is quite big, so it's not only Active Directory, Kubernetes plugin, Kuryl plugin, Subversion plugin, all static analysis plugins, you find this incorrectly, the new warning synchro stack is not affected, right? No, it's only the old ones, because nobody is developing anything, so. Yeah, but all of them are marked as deprecated at US, yeah. Yes, well and proudly we've removed them, the old one from ci.jakin.idol. Okay, so still, if you use deprecated plugins, it's definitely time to upgrade, because you won't be providing security fixes for the new, for the deprecated plugin. Okay, so Mark, would you like to comment about these changes? Sure, so where Jenkins has used for many, many years an old version of a security framework that we're now upgrading to replace with current version of Spring Boot Security, and very grateful to Jesse Glick for doing that work, he's doing that work through a Jenkins enhancement proposal, and it's looking like within the next one or two, maybe three, three weeklies we should have that in, it won't be visible to users of the LTS until March of 2021, most likely, but it's looking very good and very, very positive. Yeah, and this change includes incompatible modifications in APIs, so there is ongoing fixing for plugins, and the users of LTS may actually get the versions with updated plugins, but it shouldn't impact the behavior according to the JEPs. So, yeah, in ideal situation, there should be no impact. Yeah, there is a lot of other changes happening here and there, but it's nice to see that the technical depth in our dependencies, because each of these dependencies have been a huge maintenance button for us, so once we get them over the line, we will be able to pre-adopt new dependencies, which have been blocked, and also we want to need to care about maintaining our forks. Yes, the whole process of unforking, deeply grateful for Jesse's work, and Felix's work on the jQuery upgrade. Those of us who are users need to be aware that the changes are happening and be ready to report issues as we detect them. Okay, yeah, so any other news to share? Okay, then let's move on. So, Jenkins selection status update. Again, we have the blog post, maybe a few updates there, mostly to highlight the registration button. Also, as we discussed at the last governance meeting, we spent some time to facilitate the registrations, so most notably, thanks to Olivier and Mark, we sent emails to 2019 election participants and to plug-in maintainers, so it led to putting the registration numbers, and hopefully, we will get more by the deadline. And yes, this is the main update. We also started doing pre-tweets for announcements from candidates in Twitter, but we didn't do it consistently. Not every candidate even has a Twitter account, so it's just additional assistance here and there and improving visibility. So, if you haven't registered yet, please do so, because again, the deadline is August 8th, and almost every Jenkins contributor is a digital vote. So, if you did any contribution before September 1st, you're a digital and not only quad contributions count, so if there are any reports, etc., you can still vote. Okay, any questions before we move on? Okay, moving on. So, if you have a video. So, this one is still on my list. Apologies, I'm back. I've got to do it still. We had the agreement two weeks ago. My apologies, I haven't taken action on the agreement yet. The agreement, I believe, still stands. It'll be part of the Jenkins IO route page, right below the jambotron. Thank you. So, you also have action items on me from the previous meetings about trademark page updates. I still haven't finished the pull request. It goes stuck somewhere in the middle, but yeah, definitely by the next meeting it will be addressed. So, just quick update. I guess that's all these action items we had before. I had a quick update regarding a web search for Jenkins IO. I'm not sure whether it's the best venue. I will just summarize with the discovery status. So, yeah, as you know, we don't have a search on Jenkins IO. One of the reasons is that they were concerned about using any particular search engine because of vendor neutrality and because of data collection. So, at some point, when we were discussing to the contributor summit, there was consensus that just we don't want to go to use a standard engine like Google's one at the same time. If there is a search engine which we could embed, which we could use, which wouldn't collect the data, et cetera, it's something we would be willing to do. I spent some time on trying out the engines available in the market. Actually, there are not so many of them, but I found one with a potential sponsorship opportunity which we would be able to put. I was about to do a presentation on the documentation seek, but yeah, finally, the situation that we had a quick discussion about sponsorship, but we didn't have the agreement so far. So, it's not an option for us. And as we discussed that before, the main reason is that we still have difficulties proving that we are not a commercial organization. That's why sponsorship is available for some profits, and yeah, there is a nonstop level there. So, my question to the governance meeting participants is, would you like to revisit the options with search parts which actually collect user data? And for me, I don't object to that collection. I would object to having a search on our page inadvertently display something from some other CI system as an ad or something. That would bother me. In my case, I'm using Google to find something on the Jenkins website anyway, because it's hard to find anything without search. So, there for me, nothing would change in principle. What about others? I think it's really helpful to have a good search, because now the Wiki is dead, and I used the search in the Wiki a lot, and it would be really helpful to search the developer documentation on our webpage using a good search. So, Uli, I didn't hear an answer to Oleg's specific question there. Is it okay if we use a search engine that collects user data with you? Yeah, okay, sorry. I thought the question was, first, if I think the search would be good, and yeah, I don't care about that. I don't know where I lose my data in every form in the internet today, so I think it doesn't matter on our page. It's good for me, yeah. A cram, any opinion? It's fine, it's okay. Yeah, on our side, so we are joining also the question about the Jenkins operator. Moving, yeah, this is the point that we have added to the agenda, so this is about the roadmap in the future of the operator. We have raised an issue already like, it's almost two months ago now, and we would like to know what we can do about that. So, we will get to there. Let's just finish the search topic. So, if there is no more input there, so what I can do, I can do a new prototype, let's see, for Google search or for whatever other embedded search. If you want, I can put bin to there, but yeah, we'll definitely do that, and then we can just review it, maybe at the next meeting, and see whether it's good enough. Yeah, thank you very much for being willing to do that. Mm-hmm, well, I wanted to do it for a long time, and yeah, since there is a lot of free user feedback coming from the lack of search, and yeah, actually the prototype I was doing, it was able to collect from multiple sub-domains. So, for example, the same search was working for the plugin site and also for the main site, and just for fun for Jenkins, it's the way I go. Yeah, I'm not sure what I will be able to reproduce that, but it has some advantages. So, for the plugin index, you can definitely search here, but there's an assumption that a lot of documentation is injected on the plugin pages now, which haven't integrated the search engine working across multiple sub-domains, so it would be useful. Maybe even two searches, one for user documentation, another one for developer documentation, but it's more complicated. Okay, so any comments, questions? Okay, then Jenkins operator roadmap, they will try to find a mailing list thread, so somebody will summarize the service. And so, this one. So, again, the public conversation here is just the tip of the iceberg, because there was a lot of various background discussions. So, it started from hard to make it a proposal for open governance of the Jenkins operator project. So, the reason that the previous maintainers with Toshlap, they have been inactive since May or even April this year, and there is a lot of start-to-requests for the project, and basically there are obstacles which prevent the people from contributing there. And there was a proposal to actually produce an open governance model, which would include multiple parties, so that by default, it's with Toshlap, Red Heart, and it's governance board, so that there is no at least three entities to break a tie if needed. And this proposal was sent to the developer mailing list after that one of Jenkins board members, Alex Erl, it took an action item to reach out to the Toshlap director. Alex did it, and we got a response at a more than one month of waiting companion that Toshlap would be interested to continue in the open source, and that they intend to respond. So, the service update from Alex was on October 21st, so almost 30 weeks, and since that we received no communication from Red Heart staff, so I think we have to think again, but ultimately it will bring us to the issue what we do with this project, because we spent two months without any clear progress, and yeah, this is definitely a good talk discussed today. Yeah, actually also there are some users who start to be upset, so at least there is one issue, someone asking what's going on in the project, and why is it still not maintained and answered regarding issues. So, that started to be like indeed a bit of an annoying situation, and on the other side at Red Heart we still continue working on our four-color project, so for a while we have maintained compatibility with the end-to-end tests that were initially developed by Toshlap, so we were sure that we have maintained the same future, but since a while ago, since maybe one month or something like that, we started to operate a master refactor on our codebase, which basically breaks the compatibility with the original one, so we were first to do that because we have to move forward with refactoring and developing new feature, and yeah, so this is the current situation. Yeah, so we discussed multiple options in August, because our main preference remains that sure that the current upstream works fine. We have an option of creating a new upstream, especially if there are breaking changes, because at some point in the Red Heart team I made a proposal about the Jenkins-operated roadmap, just a second I'll change the screen again, but yeah, from what I understand this proposal basically got no response from Toshlap, right? This one, so the only responder is basically me, and yeah, so if we assume that there are major and potentially breaking changes, what we can do is say that, okay, there is Kubernetes operator, and there will be Kubernetes operator two, basically coexistent and parallel. It has obvious disadvantages because it will skip to the user base, it will create confusion, but at the same time it has a chance of becoming a new, more healthy upstream. This is what we can do basically even now. I will also, I can also ask Alex to interrupt about this point, Oleg, because we have discussed already all the options, and I remember that my own opinion was to indeed move to another fork and create our own version, but it has been decided, I mean, at Red Heart level with the product management and so on that we prefer to remain with the single community of users, and so this is what led us to propose a new governance model which is more open source compatible. So yeah, changing this direction would be also a huge effort for us right now, and so yeah, we have to find the solution which is suitable for maintaining the community, and at the same time, like the project being not active for several months, that's completely normal that there is some changes in terms of compatibility and that we cannot maintain the same future such long, especially that the APIs were on alpha stage, which means as an operator perspective that being alpha, we are still able to change the API and do breaking changes, actually, which would have been different if the API had been published in beta or stable already, but this is not the case right now. Yeah, so there were some changes and integrations from Thomas? Yeah, actually it's more about documentations and probably the Helm chart change, some changes on the Helm chart that installs the operator, I think this is that the last changes that happened on October. Okay, so yeah, I will just follow up on the thread we have with the Jenkins governance board, because we have a considerably longer thread there, which is not public, yeah, let's see whether we could proceed. Okay, apologies for that, it takes so long, but ultimately we cannot get a response, and at some point we will need to think what we do, because with the assumption, so if we don't get a response, neither from Thomas, who is officially a maintainer, as far as the Jenkins project commission or from Virtus.Lab, which is de facto the sponsoring company for the current upstream, then yeah, in the Jenkins project, we have a few options, but ultimately we don't have a process of transfer and permissions when a maintainer is actively against transfer and permissions. So yeah, it will be a tricky situation, and I think that finally we will end up voting at the governance meeting, but yeah, I think that taking the release, it's this way becomes more and more probable. So I suggest that we do the last attempt, and if we don't have progress by the next meeting in two weeks, then we actually start a formal process, which is not clear to me, but yeah, just looking at whether we could transition the current upstream. Yeah, it's my suggestion, I'm not sure what Mark and would you think about that? I think that sounds reasonable, so Akram, it seemed that your position, your strong desire is to retain the current artifact ID and thus the current community, and therefore you would need to be allowed to take maintenance of that repository, co-maintaining with Virtus or sole maintenance for a new set of maintainers, right? I think that's what you're describing. Yes, that's what has been requested by the Global Red Hat team when we discuss this. So we still want an open governance model, we want to maintain the same artifact ID, so-called artifact ID, because it's an operator, and we want to be able to federate the existing community and support them. One potential concern there is that basically we have no control over the operator hub, so what it means that we can transfer permissions on the Jenkins projects side, but all operator hub, even if we come to a decision on the Jenkins side, I'm not sure what will be the process there. So for operator hub, I don't know also completely the process, what I know is that it is maintained also by Red Hat people, and we didn't want to interfere or to bypass, like requesting the permissions or the rights to commit on the on the on the files to deploy on operator hub. So we just keep this one as is, but I would say that if we have an agreement from the community, then we can ask for this one, I mean, but the most important is to have an agreement and a proper decision, I would say. Okay, what's your opinion? It's hard to say, I don't have any knowledge in that topic, so I really don't know. Okay, so let's continue in the governance about my increased, because we might definitely need some advice, probably even from a continuous derivative foundation on this model, but here we have two weeks until the next meeting. So let's see whether we could get everything in place by this time. Okay, but yeah, thanks for bringing up this topic, because Jenkins Kubernetes is a really important project for the community, so we definitely need to resolve this issue. Anything else for today? Any other topics? Nothing from me as well, then we can close the meeting earlier. So the next meeting is November 18th, I guess nothing specific is scheduled on this, well, it's cube content, right? But I guess we do the meeting, can you wait? All right, and I will miss the next meeting, but go ahead. Okay, so then let's just do that. I will probably be at cubecon for a while, but yeah, governance meeting shouldn't be a problem. Okay, then I guess that's it for today. Thanks everyone, and I'll talk to you later. Thank you everyone.


Hi everyone, today I'll be talking about web-bush channels, which is a Pub-Sub paradigm and channel-based push service. My name is Mansimhar Kaur and I work as a software developer intern with the storage team of Mozilla and I got selected for it through a program called Outreachy and I also contributed to Brackets, which is a co-editor by Adobe. Now what exactly is Outreachy? Outreachy is an initiative of the Software Freedom Conservancy and it's open to participation for women, transgenders and other underrepresented communities throughout Outreachy, free and open source software organizations, hire interns to work remotely for three months on projects in collaboration with mentors. So what I'm talking to you about today is basically my Outreachy project and the organization that I'm working with is Kinto. So a couple of points about Kinto and why it's so amazing. Kinto is basically a minimalist JSON storage service which is being used and developed at Mozilla and it has been released under the Apache V2 license. So through Kinto you can create great user-facing interfaces which can be synchronized over multiple devices and when you create a web API there are multiple number of times you know you're worried about handling cross-origin resource sharing or maybe you want your web app to be very secure, you want to take care of the privacy of the users through encryption. So when you use Kinto the path between your great idea and the deployment to production will be quite short and we at Kinto affirm believers of the fact that user data belongs to users and not necessarily you know the application so we believe that the application should be decoupled from the storage location and now coming to the mutualization of services and self hosting part which basically means that the backend is deployed, scaled and secured only once and which is awesome. Now those of you who want to you know start contributing to Kinto or want to use it you can join our ISC channel which is hashtag Kinto on isc.phenote.net and you can also join us on Slack and you can find us on GitHub on github.com. And now coming to address the elephant in the room which is my project what is web push.  So you know maybe you're busy doing some other stuff you're folding your laundry and you're connected to the internet or you're surfing some other website and you've allowed another website to let you push notifications. So they can send you notifications with whatever new content is available. Now why is it exactly called web push? Because the data that is being sent to you is basically being pushed by the web app server to you and it is being displayed to you in the form of a notification using the show notification API. Now for a website to be able to push notifications it's very essential that the website has an active service worker. Now what is the service worker? A service worker is basically a JavaScript file which acts as a background worker and it is able to control the web pages and sites it is associated with. It basically acts as a proxy server between your browser and the web app server by routing data between them and a service worker is active and keeps working even when you're not on that particular web page or website to which the service worker is associated and it even keeps running during browser resorts. That's quite powerful and when there's power there comes responsibility. Service workers can only be registered over web pages that are served on HTTPS and since a service worker is not associated with a particular tab or just page it cannot handle DOM but it can send events to it and in future service workers will be able to support geo fencing which is basically geo fencing is an API in which there will be geographic boundaries which can be marked and when your device crosses that you'll be able to receive notifications. Now that basically when you use a visitor web page which has a service worker associated with it what happens is through service worker container or registers call you basically a promise gets resolved through service worker container or register after this an object is returned that object is basically the registration of the service worker once the registration is done the service worker file is downloaded on your browser and this happens after every 24 hours or so and can be done even in a lesser time and if the file that is downloaded is found to be new it is installed and if your browser is not using the previous service worker file then the new one will be activated and once the activation is done the first step to be able to receive push notifications is done Now coming to the next step basically when you visit a website you must have seen that you get this pop-up kind of thing which asks your permission whether you'd like to receive some notifications you can always choose to receive notifications or to block them once you click on always receive notifications what happens is pushmanager.subscribe is called what this does is it subscribes the service worker with the push server so that it's able to receive push notifications what happens with pushmanager.subscribe is basically a promise is returned which resolves into the subscription now that I mentioned the push server what exactly it is it basically acts as a middleman between your web app server and the client routing the payload between them and what happens is the service worker has a communication channel with the push service now talking about the UAID UAID is the user agent ID what happens is there is a communication channel between the browser and the web and the web push server that channel between them is recognized with the UAID so if a particular user has basically given permission for the push notifications the server will be able to send information with the help of this UAID to your browser now when the browser receives the information it has to be sent to the service worker that is involved now to be able to send that information to the service worker the browser identifies it through a channel ID so through the channel ID the service worker is able to receive the information now coming back to the part where we have sent this request for subscription to the push server now what the push server does is it returns a capability URL and encryption keys now talking about the capability URL on the internet there are two ways in which we can access information the first is through passwords and the second is through obscure URLs which contain information and people who should be able to access that information only those have those URLs so capability URL is just that and encryption keys are for encrypting the payload that will be sent across in the push messages so the push server sends back the capability URL and encryption keys to the service worker and the service worker sends them back to the sends them across to the web app server so the web app server can send notifications as in when they have to be sent now talking about the web push channels that I am creating in that what happens is one to be able to this is a sample request which sends across the subscription the push notification subscription to the service so this is basically a post request in which the endpoint URL which is a capability URL and the keys are being sent across here I have used HTTP Pi to send the example request and this is my server URL and the endpoint is slash subscriptions and what you can see in the form of auth token is basically basic auth which is used for authentication once I send the post request this is the example response that I get which is 201 created what happens is this subscription has been created and saved for future reference when a subscription has to be when a push notification has to be actually sent to the subscriber now what basically the service that I have created is a PubSub service as I mentioned in the starting so a little bit about what PubSub is now that the subscription has been done and we have allowed the web app server to send us notifications we need to inform the web app server about what things we require notifications so how this is done is through PubSub channels basically PubSub is a paradigm in which there are publishers and then there are subscribers and the publishers are the ones which send out information and subscribers are the ones which want that information and both the publisher and the subscriber I mean the publisher and the subscriber basically the publisher is able to send across information on a channel it is not aware of how many subscribers are there or not directly sending it to each one of them basically there is a channel just like a TV channel on which you can just send across the information and the publisher publishes the information and the subscribers which have subscribed to a particular channel they can access that information now coming to the part how to subscribe a channel to subscribe to a channel you basically have to send a put request on the end point slash channel slash channel ID slash registration where channel ID is basically a channel name and for example of you for example sake of use the form builder update form builder update channel and channels basically after you send the put request you get an accepted request which means that the servers acknowledge the fact that you want to be informed about whenever there is a whenever there is a message published on this particular channel the one which the user has just subscribed to now coming to the part when there is some changes and some data and you know you need to publish to a channel and to inform the subscribers about whatever changes have taken place so what happens is a post request is sent to the push server and the data payload is encrypted using the encryption keys which were sent with the subscription the subscription that we received initially when the user had allowed us to send them push notifications along with the end point URL so those encryption keys will be used to encrypt the payload and send across the push message in the form of a post request so the end point here is slash channel slash channel ID now the example response will be 202 accepted and as I told you that data payload will be encrypted and it will be authenticated to the end point and for encryption we use web pusher and when this push message will be sent across it will be sent across with a TTI header which is a time to live header which tells the push server for how long the message that it has been sent has to live on the server when the user is not online so once the push server receives it it can send it across now to summarize the entire process that takes place what happens is you get a pop-up saying would you like to subscribe to notifications you say yes so what the service worker does is it sends a request to the push server basically and the push server sends across the end point in the encryption keys and after you get the end point in the encryption keys you send it to the web app server that subscription is saved and now you have to inform the server about what channels you want to subscribe to you send across a put request and you inform the server about what channels you want to subscribe to and after you've subscribed and see some data changes happen and now you need to be informed about it the data will be retrieved and the encryption keys will be retrieved and the encryption will be done and the payload will be sent across to the push server and then the push server will store it until it can directly pass it on to the browser if the user is online if the user is not online the message will live on the server for the time specified in the TTL which was the time to live header and when the payload is received the payload is encrypted and sent to the respective service worker and the service worker decides whether it has to be shown in the form of a notification through the show notification API or something has to be done in the background so that's about it thank you