June the 15th, 2022. It's Wednesday. It's 11 o'clock. That can mean only one thing. Time for American Issues. Take one. I'm Tim Appichelle, your host. And with me today is a bit of a reunion show. We have Winston Welch, who's been on assignment for many, many months. I know that he's been working on a lot of stories. And Cynthia Lee Sinclair. Welcome you guys. Welcome back. Morning. Aloha. I kind of went out there with the title for the show today, because the title lend itself to what's the topic of the day. And the title is Trump goes with drunk Rudy. Big lie is born. You know, what's in the title everything. And what have we been finding out from the select committee's hearings a lot. A lot of that we've known but I think the value of the hearings is that they're putting it together in a story that we can actually get our arms around. Remember the Mueller investigation. It was so convoluted and there are so many exits and on ramps to what was going on that I don't think the American public could grasp it. And it didn't take much for Bill Barr to decouple the reality of the Mueller investigation from its complexity. So here we are with a very complex issue of January 6 how that happened who was involved. And, but the hearing select committee is doing I think a wonderful job of putting the pieces of the puzzle together and telling a story. So without further ado, let's go to that question. Winston to you first. What, what new things do you think we've learned from this from these two hearings. Well, I, you know, I was talking with someone about this asking have you been watching the hearings and she said no, there's no new information in there I don't I don't think we heard anything but that's not true there's a lot of new information from all of these these guests they've had a year and a half, basically to do a methodical plotting. So this is an analysis of what's gone on and that actually you know, they've been criticized for not coming out sooner for not doing this, you know, we're a year and a half out on this thing but that said this sort of work needs painstaking research it needed to have the people that they had to come forward and what we're seeing out of this is this was not insurrectionist or rioters this is being shown for what it is which was a full frontal coup attempt on overturning the peaceful transition of power from one election to another. And it's being led in as bipartisan manner as could possibly be done because the, you know, the House Republican leadership would not really participate in this because some of them are complicit in this as we found out as well in these in these hearings. But that said, having the way that this is gone about and Liz Cheney is a brilliant leader in this and the way that she laid it out and I had heard on one commentary that she'd been a federal prosecutor for 20 years but the way that this was laid out. As one commentator said was textbook classic case of what they were going to do, how they were going to do it, it was not hysterical it was not dramatic it was very matter of fact laying out what it is. I want to go to that point, I want to go to that point because one of the things that impressed me is a lot of times in the hearings and I think we saw some of this in the impeachment hearings is a Tennessee for committee members to grandstand a bit. And what I haven't noticed is any committee members are asking your questions, any hint of grandstanding or trying to paint this for a political advantage. What I'm seeing is a outlay of facts and testimony and witness testimony. Your impression on that point. Yeah, I would agree I'm not seeing any any of that I think what may have surprised people is just sort of laying this right at the feet of Donald Trump but it's not just Donald Trump there's an entire movement behind him that wanted to support this so while he you know is rightly being put where it is, they're calling out the other people behind it and I think part of their really brilliant strategy here is to have people that were involved in his inner circle, the bill bars, his campaign chairman who had, you know his wife had the baby the day of the testimony earlier. These are these were deep. They had, who was the Ginsberg the lawyer I mean that had that fought Bush v Gore. These are died in the wall, Republicans and not only that these are part of the inner circle of Donald Trump from the very beginning people that that we probably have a little bit on some of the actions that they taken but those have been. I think all sins are absolved on some level because they are actually standing up when bill bar stands up and says this is the way that it is when you have Liz Cheney who is was number three in the House leadership on the Republican side. daughter of Vice President Cheney I mean these are not liberals, she voted with Donald Trump 90% of the time but when she sees a coup attempt that's just out to destroy what she understands as her nation. This is really remarkable this is not liberal Democrats banging that the pan this is going to the source of the Republicans that were all around it. And most important is what one of the witnesses or one of the committee members said was this is not a Democrat issue this is not a Republican issue this is an American issue. And these folks are standing up as Americans and saying, this was wrong, and this is what I saw, and this is the factual presentation of what's happened. Right yeah great points Winston appreciate that Cynthia you know Winston brings up the point the fact that all this testimony is coming from more or less Trump's inner circle. I would think that lens a little bit more credibility, and to, I'm sure to Republicans they don't like to see that they don't like to come out against other Republicans but when it comes to attempted coups I guess that's what you might expect. But anyway, to you a same question. What have you learned out of these two hearings that maybe you didn't know before. To that point. Well, um, first off, let's just define what a coup is right a coup is the overthrow or alteration of a legitimate government by abusing power, or through violent. It's an ongoing coup right that some of these people are are going with. I think the guy that struck me the most made me the most effect I thought was that Ben Ginsberg guy who was involved in the 1990 or no the 2000 Florida. And recount stuff investigation and all that, and he just stood up and said there was no evidence. No, and he told Trump that. So I think that's an important thing. They're trying to establish if he did know if there was anything, you know, behind what he was doing. Did he just believe that he really did win, or did he know better. And that's when I see them very carefully laying out is how many times he was told that there was no fraud. Well that I think that's the point is that you had I think is Alex Cannon, who was kind of the data guy who who looked at election night results and said, not looking good. And I would not recommend going out on election night at one in the morning or two in the morning. I was up to watch that and say that you've you've won the elections, I would not recommend that. And then Mark Meadows chief of staff said okay so there is no there there. And despite the constant advice that he got Trump wouldn't have any part of it didn't want to listen to it so what did he do he tried it out, and went with yeah I won the election and if I don't win, then this whole things a fraud. So he said, you know he set the stage on election night. And I think maybe, maybe they had announced Arizona at that time. And that's what I think angered Trump the most is that see it excuse me, Fox News actually aired the, the Arizona results and I don't think they were happy about that. No, they fired the guy. They did for telling for telling the truth firewall. Yeah, yeah for telling the truth they they terminated them. And guess what that's what every news organization should do is fire your your your executive director for telling the truth. Wow. So any other thing Cynthia that you thought that caught your eye on these last two hearings that that you really didn't know before or other than Ginsburg. So, as far as things that we didn't know before. Here on think tech we really follow the news. So we are pretty much familiar with all the stuff they're presenting. Right, but the average Joe just watches the evening news, he's getting all kinds of new stuff. So I'm not really seeing a lot of new things that we didn't already kind of know happened, but now what they're doing and I think that another thing that's behind all this is one of the questions is who are they trying to reach with this right who's their target audience. This is my opinion. History is their target audience. All of this is getting, you know, transcribed, you know, black and white history, and there's going to be no denying it there's no changing it there's, and they're being very careful to stay away from opinion, and just deal with facts. And, and I think that's really great and I noticed the same thing there's no grandstanding, right. How wonderful to be able to air down all of that. You know, look how great I am see over here let me gaslight you now kind of stuff it's gone, and they can just go right straight to the heart of the matter and I think that's brilliantly put together and really well executed. You know if I was in the audience as a candidate for 2024 for the president United States. I'd be watching this very carefully and watching this report that's going to be produced because as an audience member, I might want to take the report and plop it down on a judge's desk and say, Here's my brief. President Trump, former President Trump has no legal standing to enter into President United States under the 14th Amendment paragraph three. And here's my brief to prove it. So there might be a future audience that we haven't really thought about yet but any candidate was standing certainly could good run with this. We'll see time will tell. Yeah, I agree with that. So any one of us here on the show could run for President United States and what's prevent us from filing a suit. When are you running Winston when you're running for office. And even then, I don't want to catch talks. It's hard enough just to be an informed citizen honestly that run and stand for office with noble valuable principles are honorable people for the huge most part and we have to applaud them and they've taken so much abuse and flack of the last couple years, even we're seeing that that that fellow that testified. You know, we haven't even gotten to that part yet of the hearings coming up which are going to be going drilling down to that level in Georgia and the harassment and threats of local Rosenberger died in the wall. Yeah, a broken functionaries in these states that are just designed to count the boats. Yeah, there's some real heroes that have emerged out of this in the last 17 months and Liz Chinese certainly is one of them, regardless if she retains her seat in Wyoming as a senator. She'll go down in history as a true patriot, certainly not with the principles I line up with but as a patriot of democracy in the Republic should go down in history. No two ways about it. Speaking of history, you know, William Barr is coming out of this is, I don't know, I'm trying to think of back in the days when Robert McNamara. He tried to redefine his role in the Vietnam war and, and not really telling the public how the war was going he was in charge of you know, you know, he was the Defense Department Secretary of Defense. And he was partly responsible for the cover up of what was occurring in Vietnam and how we were not winning the war there. And then he came out you know years later and kind of tried to. You know polish it up spin it that his role and I'm getting the sense that Bill Barr is doing the same. What are your impressions of Bill Barr and his testimony concerning Donald Trump and the insistence of his big lie. Well, you know, like I said he got, he got a lot of stuff wrong over the last, when he was in power and he really had some policies that I don't think were in the best interest of America but he was the Attorney General at that time but for this last one. And this, this really important thing which was our time here is done. We lost Farron Square. He told it to Donald Trump many times in many different ways, as he stated, very clearly, he stated, you know talked about the different lawsuits he sent out his own investigators really looking for something is there any there there. He categorically found there was nothing there I thought when when the when they rolled the comments of the federal judges, a huge number which was pointed by Donald Trump saying this is just pure she buy right and and the comments that they were rolling from the judges was so telling these are people who have access to all types of information that are in a position to judge these things very carefully. And then they did. Bill Barr is whitewashing of himself. Well, you know what, it just is but if he's coming through at the end here and saying this is this is actually what happened and you know city was saying that the target audience is history I think that is one target audience. David Brooks had very good column in the New York Times a couple days ago and he said I don't care who sent an email or text when on January 6 I want them to address the ongoing coup attempt of these these elections we're seeing right now where you're having people, these acolytes it's just as if you know that the beast was unleashed and it was it's a hydra so you know and they're literally running for office saying, if I don't like the results. I will simply ignore the will of the voters and and put these these folks in office. That's really where the damage is that the value of these hearings is to say, you know there's ongoing attempts to start a free democratic election. Well that's where the dots have to be connected because I think it's probably be on the purview of this but not necessarily because it's a logical. It is the next step that says here going from every every level go into school board rooms or school boards on to you know your local city councils on to state government levels and and put enact these policies that are detrimental to a free and functioning society so I think it is natural to make those connections but hopefully, hopefully, like Cynthia says we probably pay attention to this more than the average Joe get 20 million people watching it so how many people in this country let's sort of round up to 400 million so you've got 5% of the people but are those. Is this going to be reaching the elusive soccer parents right who who were sitting there and they're like yeah well I really don't like the swamp and I appreciated Donald Trump trying to drain the swamp but this was too much putting this all together in a logical coherent historical fact to laden series of hearings, I want to buy the commemorative addition honestly but if just people watch this and they see wow, I thought that guy was his, his acolyte that was his number one deputy and now he's saying wait a minute here. He didn't bill bar at once when say he thought he'd lost it or that he was delusional or something to that effect like. Okay well let's let's go to this because you know what came out of this in this testimony and certainly on the second. You know the last hearing was this notion that there was team normal and team crazy. And you know that kind of catches the American public's interest like team normal team crazy, who was on team crazy well it was drunk Rudy on election night and Sidney pal.  It's still election night and nothing's been formally you know closed out, but Trump didn't want to listen to that. So I guess the question is, if there was a team normal, and if bill bar was part of team team normal. They certainly didn't come out and say anything. As Trump started to pedal out the big lie, they were complicit with the big lie because I don't know maybe as a consultant you just want to stay on the payroll and dam the democracy you're getting paid, your thoughts on that Winston. Team normal basically went along with team Trump on his big line, you know, narrative. Well, you got to look at when. Go ahead. Oh, I just just got to look at when did they quit. When did they really express their reservations about this and you know what day did bill bar quit it was January. First or fourth or I can't remember when he quit December something. Why didn't they come out very forcefully in the media saying this was wrong, but you also have to remember that at the time, you have this entire bellows stoking the nation saying this was stolen this was fake and here's our all our information but if there had been a concerted attempt by team normal to come out at that time, maybe we would have had headed this off already so I think that's the point, you know, this whole thing could have been head off. And I think you make a great point on that, Cynthia you were you had an opinion about this topic about team normal versus team crazy. Then question to you. Why do you think team normal didn't spark up and say, I mean I know they'd be have a job they'd be terminated immediately, but didn't they have a duty to the American public. So there must have been some sort of threat behind it that wouldn't let them something that would compromise themselves. Right. Well, Donald Trump can't have dirt on everybody because that's, that's the kind of the mantra of why people fall in line with Donald Trump that he's got some kind of hold or dirt or hypnotic effect on him. Can't have it on everybody though Kenny. Well, anybody that didn't come out when they should have he can say, well you're complicit because you didn't say anything back then so you're just as guilty. So I think the threat element is there. And you hear them say the reason that they didn't do it was because they wanted to be able to stay and keep breaks on the crazy. And to think that they could somehow keep it from getting so outrageous. And I don't know if I really buy into that too much because, like as per came out with this book saying all this stuff and it's like well why, why didn't you come out and say well I wanted to see if I could at least, you know, keep it from going out of control completely. And I guess I kind of give that to him but it doesn't ring true to me that factoid that they're trying to put out there as to why they didn't come forward. And some of them, it's true, they just thought they could help if they stayed and then won't lose their jobs, or maybe they really were complicit in part of it. And that's all it would take is to maybe it's a lesson for government and those who work in an administration I don't care if it's Democratic or GOP is, you know, it's the john deans that knew they're crossing a line they didn't know it well enough, they were doing something wrong and they went along because they didn't want to fall out of favor. There's there's a intoxicating effect that you're you know you're working with closely with the president United States, and you want to make the president United States happy. And you know john dean of the Watergate fame was certainly that attorney that didn't want to put up roadblocks and then be thrown out of the game.  How do we stop that. How do we stop that. That's the one power, the entire Republican party sacrificed their integrity to get their judges and, you know, to get what they wanted. They were willing to sacrifice their integrity for that. And so any Republican or any member of the guys, you know, administration was power hungry with this they've already, they've already sacrificed their integrity so they're all in at this point. So maybe that's part of it too. So the only thing left is their historic reputation maybe. Well, partly but so they thought it could succeed. And then they could stay in power, right and it's that that draw the power out there. Like, well maybe we can make this work and then we'll all be you know set for life with this job. I think that might have been part of it too just that that power, absolute power corrupts absolutely and I think these people were corrupted absolutely. Yeah, part of it. Yeah, no that's a great point. Winston, we had 20 million viewers on the first night of the hearings. Clearly, I think that's a bigger audience than anyone anticipated beat out the NBA championship games, which is, you know, that's quite a high viewership. I think the interest of these hearings is just confirmation and of what people already know and makes them feel better. Or is there a new audience that independence and or some of the old GOP that have some integrity left. Are they watching. Who do you think our audience. I know we talked a little bit who the audience maybe but what do you think this 20 million might comprise of. And I didn't see a breakdown as to who was actually watching this yet in elaborate breakdown but let's just assume that Fox News got the message because they carry the other hearing now they said oh we couldn't carry it because our the first one because our host control their shows okay whatever but they carried the second one which is quite interesting and I saw the. I did watch Fox with Brett bear debriefing it with to blonde ladies and and basically they were they had to tread very gingerly because their colleagues are implicated in this I mean they're they're they're text to Donald Trump are there so shine Kennedy comes to mind. Exactly and at the end of the day though you know what we're watching is is we we need to look at this as Americans and say okay what kind of society do we really want to live in here. Do we value free and fair elections, regardless of who is in office and is. Let's just say okay. This is the mess that we've got right now. So this is helping us understand where we're at. What are we going to do from here. How are we going to make it better. How are we going to all get on the same page that we can understand that elections are not stolen or the switch flipped and all the theories that have been debunked I think that that's where maybe the benefit of these hearings will be that they're being presented in a way that is. The public to understand this happened. Now what now what are we going to do with this and so it's not to prevent I don't think Donald Trump will run again. But he's got a lot of other people out there who are still part of this whole denier thing and they'll be holding to him but as we saw in the elect in the, you know, election results of yesterday. You don't necessarily have to hold on to Donald Trump anymore to win as a Republican you can go back to the conservative values. And it's sort of skirt this for right now until he's out of public consciousness enough I don't know that that's going to happen, but these hearings if it does happen will be part of that you know that happens. Sometimes when you get into these hearings and you hear testimony things, one thing leads to another and I saw the door opening on that on this week when you've gone from sedition and Trump's involvement with sedition to basically take an avenue over to the big lie became the big rip off which is to say they raised 200 million, 250 million dollars for the Trump defense of the big lie stopped to steal was the name of it, and 250 million dollars was raised, only to find out that a lot of that money went to Trump hotels. What's Trump juniors girlfriend's name. She got 60 K speaking role and for 60,000. You know Al Capone never went to prison for any involvement with a murder, he went to prison regarding tax evasion. Does this take, take on another flavor with a prosecuting attorney and say New York or somewhere else Winston. Very well may will have to see about that. I would still guess that at the end of the day Donald Trump is not going to be in prison. He might be indicted though, and he might undergo some sort of trial, but it will have to be undertaken very carefully with the same sober people that we're looking at right now but when you have people that just say this is just total fake news and more fake news I'm not sure that we have to find out what is the best way to reach those folks to let them know this there were, there were crimes committed here. And, but if they don't believe it it may not make a difference. He's not going to go to to prison but he might be indicted and unable to run. That said, you've got 40% of the country that still thinks his approval rating is higher than those on January six which is really shocking and just speaks to the, the need to educate our fellow countrymen and women as to, you know what's gone on but maybe after this and coven, we're just kind of needing to turn a new page and if we can just turn this dangerous corner coming up in the next year or two, three, we might stand a chance but we're looking at a fundamental, I think change in the psyche of American society that is being played out in the Supreme Court and the composition of that as well as what's going to come up in these next elections so we just have to stand by and stand by. Yeah. Okay, thank you Winston. Cynthia we've come out of run out of time but I want to get your, any of your last thoughts on this topic. I know you had a couple quotes, things that you think that are important to point out before the show concludes. It's very powerful to have all the capital policemen sitting in the front two rows during the entire thing. I thought that was a real important statement to make. These were people that got hurt. They were law enforcement. So all your blue wave love is wrong, you know this whole blue line staying with the blue line we're all about law and order. It was policemen that were injured. And so, instead of just having them come in to be there for the questioning part, they were there in the audience just, and I wonder if they're going to be there through the whole thing. I think they might be. So a couple other things that I think are important and just crazy. It's another part of the crazy coming out of all of this. Peter Navarro has now had made an official charge has a suit against the January 6 committee, and Nancy Pelosi, because he's saying that they are maligning his character or whatever, just because they're stating the facts of what happened. And so he's trying to grow shade out there. His actions maligned his character. Exactly. Thank you very much. And I think it's important to remember there is so much stuff that has come out of this insurrection. And just was it yesterday or David points was yesterday. They stopped was it 20 old father guys that were going to I can't remember what they're not keepers there. Yeah, no. They're out of the Hayden Lake area. Yeah, they came all over from everywhere. But there were all a bunch with riot gear in waiting in a, in a van in a U-Haul van to go and make trouble and attack and who knows if they would have hurt people or they were prepared to get plenty of stuff to hurt people in that that gay pride in a pride that they were having. And it just, I wouldn't just struck me this morning. Yeah, but this isn't just about January 6. This is about what's happening in our tree that's profoundly affecting the electorate in in dangerous dangerous ways. Yeah, I think this one that thank goodness they've got a good handle on investigative intelligence to be able to know it was going to happen. And it was just a tip from someone. So I really want to enforce that if you see something say something right to people so that if you see something say something. It wasn't anonymous tip from. Yeah, good point. I just saw it. Okay, so my closing closing quote is from Stephen Colbert is as you guys know that way I handle all of this is I watched the comedy at night and it somehow makes it so that it's a little more palatable right. So Stephen Colbert says, so 20 million people watched and he's so great we just want to express my gratitude to the committee. We all know about all of it, but watching the way they put it all together reminds us that we're not crazy. Yeah, we're not. Thank you Cynthia Winston you get the last word. Well, I'd say stay tuned. Watch what's going on, educate yourselves, ask your friends okay USS 331 million people I rounded too far up but let's just say we need a lot more than 20 million watching this we need 100 million or 200 million watching this. This is real history like Cynthia said, the bleed over into our society right now, and all these different areas where people feel that it's, it's fine to just.  That's the message that we can't really have here. That's the message that we need to rein in people and say, this is not the country we want. We want a country that aspires to its best and highest ideals. And that's what this commission is all about just finding that the facts and letting the American people know what they are so that we can course correct as a nation. Yeah, great point. The world is watching our society, our democracy, we are that shining city on the hill, hopefully still, and your point is well well taken Winston thank you so much. All right, we've won our time in 28 minutes certainly isn't going to be enough time to cover this topic, but we'll have to take another crack at it again so I hope you'll join us. I'd like to thank Winston Welch for rejoining us here on American Issues take one, and I'd like to thank Cynthia Lee Sinclair for always for great comments. Thank you Cynthia. Thank you for both of you to join us. Would you please join us next Wednesday at 11 o'clock, and I'm Tim Apachella and we hope to see you then. Aloha. You can also follow us on Facebook, Instagram, Twitter and LinkedIn and donate to us at thinktecawaii.com. Mahalo.


What's up everyone it's your girl Kenzie here from Leighton Sports Cards, and I'm ripping up some personals for John L We've got two orders here.  All right, John best of luck. We're gonna start here with your Star Wars We do Dean for ship it only. What's up, Luke? How's it going for ship? I See Justin's in queue. You're gonna order something Luke. I know you and your brother like to do like secret box wars We've got Ezra Bridger up first We've got the Mandalorian That's a cool. I don't know if I've ever seen that one yet And then we've got blue hunter Hunter that one is gonna be 42 out of 75 And I see some color here. It's a nice one too. I see some orange Orange of Saban Ren that is 23 of 25 of Tia Sakhar Sabine Ren 23 of 25 Nice. We'll take an orange there. Yes, Elijah. I do see your order again It's gonna be closer to about the two-hour mark so I get to yours as there is probably a good like 12 people in front of you, but I Do I do see it though Elijah M for two Bowman chrome hobbies. Yep. All right. Now. We've got your two 2022 tops chrome hobby boxes here for you, John Two 2022 tops chrome baseball hobby. We got your silver packs here that we will do at the end Good luck here, John We have a rookie Otto of Connor Siebold Connor Siebold is gonna be your Otto here, John red socks Kyle Tucker Mr. Kowar. We have a Brandon Marsh rookie Fracture see here rare refractor Carlos Correa and sunny gray Mr. Posey's at Brinkey. We've got Aqua wave of Paul Goldschmidt. Nice one there. It's part of the Buyback 194 of 199 Yeah, what's I'm saying Nice one there for you, John Paid for like one of your boxes almost Kibrian Hayes Into a base judge yesterday. Oh, yeah, I saw that. Yeah, didn't you you have the Yankees? Yeah, I was just joined too late prism John means Shane Bieber as Kyle see your refractor and the child. Yeah, the bases are going for like 20 I know like the fuchsia is we're like a hundred bucks or something like that. So between a 50 and a hundred dollar card But may you Enjoy Vado Tristan Mackenzie Let's see. We've got a Jarender ran rookie refractor Kyle Lewis new classics Wanda Franco rookie Chris Bryant refractor was the Altuve So we've got heart of the city of Josh Hader Paul Goldschmidt base Got like $70 worth of Goldschmidt cards already Tatees Kyle who we've got prison Pete Alonzo nice one there. I'm gonna see ya That's the hundred dollar one. Yeah Nice 120 dollars from to Paul Goldschmidt cards Rice Harper Clement refractor of Tapia Zack Wheeler. I think the fuchsia of Aaron Judge was like what close to $200 rookie auto for the Phillies Luke Williams Luke Williams And a burger Man, we've got feature stars or factor of Bobby Dahlback Nolan Aronato Ragman Hannah Graham rookie refractor of Alexander Wells with bets. I still remember when Sam pulled on a Cunha junior rookie Auto out of a blue out of one of those silver packs. We've got pinstripes Mickey bets Walsh Yeah, I definitely can Prism of Shamanaya base. I personally haven't pulled anything huge out of those refractor hobby bias Stuff from out of there We have a Trevor Larnack on the negative cool in there New classics Jonathan India a break you refractor Brantley londor blast back here. We'll get into your second hobby box Prism Lucas Jolito and Jirinda Ranbase It's all a box there with the Goldschmidt. So it's two cards almost pay for the box itself I'm not even counting your autos. All right. Hobby number two One tune it into the stream so far today. Hope everyone's having a great Sunday so far Got rookie refractor of Josiah Gray And you can check out all the personals we have available at leahsportscards.com underneath the personals tab Thank you. We've got blue wave auto a Cedric Mullins. That's such a sick card Love the way raves there Beautiful beautiful card that is 104 out of 150 Go John killin these uh, oops tops chrome hobby I haven't ripped any of this in a minute either We've got Joey Vado stand Let's grow we've got to 3.99 a fuchsia albert pull hosts Dodger's pull hosts Martina's refractor Ian Hatt So we've got aqua rookie of max crannick for the pirates 45 of 199 Mayhew Vado prism Alec Minoa Mount Castle through refractor Matt Chapman and Zimmerman Thompson Jose Abreu Seth beer rookie New classics. Um, Randy Rosarino. We've got louis heel on the rookie refractor We've got chicago heart of the city tim anderson bellinger Brian dala cruise on the prism rookie second half here another auto refractor wilson churros We have a lejo Lopez on the rookie auto for the reds lejo Lopez Santana base future stars of dylan carlson refractor jayden de groin calrally ricky refractor max scherzer launching air and judge pinstripe dj. La mayhew the cutchin prism jake kronenworth padres rookie refractor brandon marsh brandy Tim anderson machado New classics. Elohim and as brandon lau Pax left here Got areas air and judge base It's $20 car there. Carlos Last half got prism of john gray. It does have a print line right through his like cleat there and al-tube All right, silver packs here Good luck, john See short green. Come on Plakes now brandon belts Josiah gray rookie and refractor is a dylan carlson Harrison batter vader Shohei otani We've got a george sprayer and refractor rookie wander franco.  Thanks again for hopping in and we'll get these cards right out to you


We lived in the darkness long enough, don't you think? Soon it'll be dawn. And Plato would have us step into the light. Now, we might find the cave comforting. The shadows and the darkness of the cave don't represent foreboding evil. Rather, it's ignorance. And even more specifically, the ignorance created by the certainty of our own beliefs, by the certainty that you can have, or the confidence that you can have with mere shadows, with just mere knowledge of particular things, there's a deeper comprehension to be had according to Plato. You know, the darkness, the moonlight, can be a bit more comfortable than the sun. After all, the sun can be a little hard on your eyes, especially for me. But you can't see nearly as much by moonlight as you can by the sun. You can't see nearly as much by a campfire as you can by the sun. It's gonna be dawn soon. It's time for us to step into the light. It's not quite yet dawn. So we have a little time before we really encounter the symbol for Plato's truth. So it's probably helpful to remember what Plato's trying to do here. He's trying to account for existence, right? He was still dealing with that project, beginning with Thales' Annexment or Annexments all the way through, still trying to account for existence. Now, his answer for what it means to exist is to participate, right? To be form, to participate in form. So all these particular trees out here participate in the form of tree. Now, this is very much, what Plato's doing is not only the project of metaphysics, he's also very much got the skeptics, there were some skeptics at the time, but very specifically the Sophists. He's very specifically rejecting what the Sophists had to say. And the Sophists were living in that ignorance. They had just the certainty of appearances. Truth is whatever you believe it is. Now, he's trying to bring us out not only trying to bring us out by giving us account of knowledge, but also giving us account of existence, but also knowledge, also knowledge. And for Plato, these two are wrapped up into one. They really are. So for what it means to exist is also his answer to the question, what is knowledge? It's form, right? This knowledge of the form. Now in the allegory of the cave, right? We had these four parts in the allegory, the shadows, the puppets or the cutouts, the solid objects outside, and then the sun. And you've heard this before, but it's worth repeating, right? The shadows are what they mean, or are just the appearances of particular things. Just the appearances, huh? The puppets are the particular things, these concrete particular things, right? So the appearance, the shadows for this tree bonds, that's the greens and the browns. The cutout is the tree, right? That's the tree. But you might say, well, hey, now I know the tree. What else is there? And Plato says, you're not done yet, right? In order to really understand tree, that tree, you have to understand what it means to be tree, right? The definition, the meaning, the essence of tree. And that's the form, right? So in the allegory, you have the shadows, which are the appearances, the puppets, which are the particular objects. The solid objects outside, well, that's the form, that's the meaning of tree, but you're still not done. In order to really get the meaning of tree, you have to understand the truth, the good, the true, the beautiful. Now that's represented by the sun. Okay. So these forms are the, you know, in these meanings, these essences, and for Plato, they have certain characteristics that make it very different from either particular things or the knowledge of them. Right? So starting with the knowledge, right? I know the form through reason. I don't know it through appearances. I can't see the form tree. One of the reasons why is because, you know, so I know the forms to the reason is because forms are universals as opposed to particulars. Now that's a particular tree. That's a particular tree. Over there is another particular tree. Those are particular things. The form of tree is what they have in common. Now you can't see that. You can only comprehend it through reason, right? So the perception is that tree, that tree, that tree. But through reason, I understand the form of tree. So forms are known through reason. They are universals. And known through reason as opposed to perception, they're universals as opposed to particulars. Now, since they're, you know, these particular things, they come and go, right? These trees will dive into, there's a point in time which they did not exist and there's a point in time in which they will cease to exist. And since, you know, they, since, you know, there's a point in time that they cease to exist, right? They are temporary. They exist in the timeline. Okay, well, the form does not cease to exist when a tree dies. The form exists independent of the tree. So these forms, now they're eternal, right? They're eternal as opposed to temporary. These trees are temporary. The form tree is eternal. It doesn't cease to exist with the death of trees. All right, and one other really important thing to remember for Plato, knowledge of the forms is objective, right? I know the form as opposed to subjective. With subjective knowledge, I just know what I think or what's happening in me, right? So for instance, by the way, appearances are subjective, right? That's subjective knowledge. I see my appearances, but nobody else sees my appearances. I don't see your appearances. You know, maybe the appearances are the same and are individualized. Okay, maybe yes, maybe no, but it's not like I can see what's happening in your mind, right? It's not like I can hear your thoughts. Only you can hear your thoughts. Only you can see what's happening in your mind. Well, if that's the case, then that's subjective knowledge. But the form is, when you know the form, you know the object, you know the form. I don't know that particular tree objectively, but I know the form objectively. I know the form, not just what's happening in me. So in the allegory, we got shadows, puppets, solid objects, the sun. That represent appearances, particular objects, the form, and then the good, the true and the beautiful. And forms are known through reason and it's objective knowledge. Forms themselves are the universals and they're eternal. Now that's just a quick breakdown of everything that we said, right? Oh gosh, we had to hear this again. No, okay. It's one thing to hear it. It's another thing to actually try and start using it. And to understand the differences between these. So let's go on our hike, right? And by the light of the true, the good and beautiful, let's see if we can figure out how to use what Plato has given us.


The River State Commissioner of Police Ebokha Friday has ordered a full-scale investigation into an explosion that rocked a popular May 3 motor park on Monday morning in Porta Cote. CP Friday gave the order while confirming the incident on Monday afternoon. He said the explosion was from a locally made dynamite planting the fish stop inside the park saying the son of the lady who owns the shop was injured along with others. The investigation was to ascertain the cost of the explosion and ensure that those behind it face the full weight of the law.


Hi, I'm Alessandro Ferrari, one of the authors of the paper entitled Expanding the Substitute Scope of Kite-Oligo-Saccharide Oxidase from Fusarium Graminiarum by Structure-Inspired Metagenesis. The flaboprotein Kite-Oligo-Saccharide Oxidase from now on called Kite is a carbohydrate oxidase with a peculiarity. It's the only of its kind capable of oxidizing an acetylated sugars. In fact, it can oxidize the constitutive element of chitin, anacetylglucosamine, and its oligomers. The oxidation takes place at the C1 position, resulting in the formation of the corresponding lactone which subsequently hydrolyzes to the respective aldonic acid. In a previous study, a point mutation in the active site, Q268R, changed the substrate specificity of Kite from Kite-Oligo-Saccharide to Glucol-Oligo-Saccharide. In this study, by comparison with two other known carbohydrate oxidase, namely the Glucol-Oligo-Saccharide oxidase from Acremonium Strictum from now on called Gox, and the lactose oxidase from Microdochium Nibale from now on called Laot, Kite was redesigned in order to accept other carbohydrates as substrates. By superimposition of the model structure of Kite-O with the two other known carbohydrate oxidase, we took a close look at the active site at the substrate binding pocket, and eight residues were identified which were different between Kite-O, Gox, and Laot. So we designed eleven mutations in order to either change the residue in Kite-O with the residue type found either in Laot or Gox, or change the physical properties of the selected amino acid, for example from a methionine to a tyrosine. After screening with 23 different carbohydrates, two mutants G270E and S410R turned out to be interesting. In fact, they improved the catalytic efficiency towards oxidation of cellobios, lactose and maltose two times compared to the wild type. Then we decided to combine these mutations together with the previously characterized one Q268R, generating three different combinations, G270E with S410R, Q268R with S410R, and the triple mutant, which included the combination of all the three together. Well the results were way beyond our expectations, the double mutant G270E S410R improved the catalytic efficiency of Enacetyl Glucosamine by 16 times compared to the wild type, generating the best Enacetyl Glucosamine oxidase so far reported in the literature. And the results were even more surprising for the triple mutant. The oxidation of cellobios improved by 200 times compared to the wild type, and the oxidation of the lactose improved even more than 500 times. As you can see, by rational design of mutation, the certain scope of Kaito was successfully changed. We generated variants with superior activity towards different carbohydrates compared to the wild type. You can read in more detail in our paper in Biotechnology and Bioengineering, and we can read how we achieved this and the reasons that might be behind these drastic improvements. For any questions, comments or requests, please feel free to contact Professor Marco Freyhe at the mail address below. Thank you for your attention, and we hope you will enjoy our paper.


and kick things off before we get into the main presentation, but give it another minute.  Just see, you can see this. Are you seeing that in presented mode or are you seeing that in the, it isn't- The main screen. We can see the main screen. Excellent. And then the presenting mode again. Oh, good. Fantastic. Okay. Well, let's slowly get going. And then if people are a little bit late in and they're a little bit late, I'm sure people will pick up what's going on very, very quickly. As we've got a few presentations to go through, I want to allow a better time for that discussion at the end. So thanks, everyone. It's been a while since we had our last Our Validation Hub meeting. I changed the name for it because realistically it's not been quarterly. It's probably been about six months since we last had last had a meeting or more. But that doesn't mean that we haven't been active as a hub. And if you're not signed up to the mailing list, I'd encourage you to sign up to that because we do share when we put our new blog posts and so on. And if you want to sign up for that and we'll keep checking the website as well, you'll see some of the latest activities that we've been working on as a hub. What we're going to do is because we've got this very focused session today and there'll be another one on May 24th that we're going to follow up with. So part two on the same theme will occur then. I'll send an invite out afterwards. That hasn't gone out yet. And so if you're on the distribution list, if you're signed up to that, which you can do through the website on the Contact Us page. If you're signed up to that, then you'll get an invite later today from me to join that next meeting. And these two meetings really are with kind of our theme for the year. So those of you who've been a member for a long time will have seen we've put roadmaps in the past and talked about what we're trying to do. There's an initial focus on a white paper and what I kind of thinking our methodology was. That was followed by a focus on tools and we're still building some tools and you'll get some updates via the distributionist following this meeting separately from the main thing today. But the main thing today is around how people and the main theme for the year with the group is how people are using the framework. So what are people using that's working well? What are where are people adapting our framework? And so really it's about communication again, but it's about making sure we understand how people are using what we've put out and where we might need to provide some more information potentially through future white papers and hand or tools. So that is essentially the majority of my intro. We've got four talks today, Roshnav artists, Merck and GSK and we'll get the, he presents to quickly introduce themselves in their case studies. So they're gonna walk through case studies at their company about what they're trying to do with some sort of highlights of things that have gone well with implementing the framework and areas where they've adapted or changed or feel that there should be improvement. At the end, we'll bring it together for a sort of full discussion Q&A. So some of the presenters may give a little bit of time the Q&A during their presentation, we'll have a big Q&A session at the end, which I'll try and manage for everyone. So just before we get into the main thing, so the first one will be Colleen Zevolos from Roshnav, so I'll hand over to Colleen in a second to present, but just whilst I have your, hopefully have some of your attention, just to note that this Thursday, so April 28th, this Thursday that our consortium are running a session called reporting table creation in R, and you can see the advert for it there on the left. You can sign up for the R consortium homepage as a webinars link that you can quite easily get to from the top menu there. If you're not aware of this, not signed up. It's not a R validation hub activity, but we are part of the R consortium. And if you're here, you're probably in some way involved or interested in R adoption within your organizations, and the theme for those webinars is very much about R adoption. So the next one is on tables, and if you may be aware that there is an RTRS, tables focused working group through the R consortium as well that runs in parallel to this one, so we consider those one of our partners. So if you wanna hear about that, Gabe is going to be leading presentation talking about something called R tables, which many of you probably be aware of as an open source package from not just R tables, but our whole ecosystem around that. And the other thing I've already mentioned today is the R validation hub, all hands part two. So 24th of May, you'll get that invite soon. You shouldn't, you won't have it yet, so you'll get that before the end of the day. And we hope for more companies to do their case studies. Okay, well with that, I will stop sharing and hopefully Kalina's online and ready to, ready to, Kalina's not. I'm jumping in, Andy. This is Doug. We have, I'm Shaman, representing Roche today who's on the call. Fantastic. I love it when problems solve themselves very quickly. So that's fantastic. So Shaman, I'll let you take over sharing. I tried to do it. I first have to allow my Zoom to be taken. I will have to restart the application due to like my new mag does not allow me to share. I have to restart Zoom. So we'll be back in 30 seconds. You know, Keith, if it's longer than 30 seconds, I'll switch the order. So second on the list is Matt Fiddler from Novartis. So Matt, if you could be on standby, just in case. Otherwise, I will try and give it 20 or 30 seconds more of a fill until he's back. Are there any questions that anyone has immediately kicking off before we start? I'm not expecting it, but someone wants to bring from my voice, you're welcome to say something. Sorry for that one. But fortunately it was that another problem that's also instantly, so on the nicer part. Okay, so are we able to see my slides right now? Yes, and the presentation is good. Cool, thank you. So as we already know, my name is Shuman Maximruk and today I will present our package validation by Rosh Farma validation team. Led by the Colin Zebius and Doug Cahillhoff and the members of the team are also Ranzo Brasci and myself. So first I would like to unite some design priorities we put in front of us while preparing the process. So first is leveraging best practices for our package development. We believe that those practices are some kind of guidelines that adhering to each will allow us to minimize the risk of potential flaws in the package. Then we wanted to automate the process of getting those checks done, but at the very same time avoid special cases for particular class of packages. So if we ever decide to allow some exceptions, we want to allow those exceptions for every possible package, not just for particular subclass. Then moving forward, we want to minimize the packages scope. So as you are aware, a lot of packages in R does not necessarily provide a unique functionality. They are rather interfaces for other functionalities like databases, APIs, and so on and other. In that case, the team that is writing those packages are responsible only for the R language part of this package. So we are not interested in the system they talk to. This will be handled by another validation process. Then important part in our process in the human and we want the human to be in the middle of it. So for instance, if some automated checks are failing, we want the human to have the last decision what to do with such package. Thanks to that, we believe that we balance also risk mitigation against automation because for instance, if a package is failing for some like false positive even problem or a minor RCM V-check node that is not applicable to any regulatory use case, then we want this package to be permitted. And thanks to human eye in the middle of this process, we can leverage that one. Going forward with our validation, we target this for systems, mostly containers. It's because containers can be shipped for once device to another, saving all of the system settings. And this ensures that a validated package that was that was valid in a container will work the same way regardless of the system it's running on because of that container. And last but not least, we want the whole process to be transparent, to encourage in-house development and iterations. Here's a small graph that presents the sprints of our process. I will briefly go through it and then explain each step in detail. So everything starts with a validation, with a request. So package has to be requested to us and it can be either external or internal package. Externally developed package can be CRUN packages, BIOS packages, public GitHub, public GitHub, whatever. While internal packages are those developed within Roche and those are usually submitted via CI-CD. Then we run our to that first automatically gathers package metadata and just after triggers automated set of checks. The later steps depend on the results of those checks. If the checks passed and everything is fine, then we run reverse dependency check against other validated packages to make sure that the upgrade or a new package will not crush anything. And if that package is also passing, then we are publishing the package to a validated package repository and generated a report. If the automated check failed, then we have that human in the middle, I've mentioned previously and that human eye represented by two, by a pair of eyes on this graph actually assess the severity of the failure and whoever it can potentially inflict some of the regulatory work. If it's not severe, we justify those failures, why we can permit to happen and proceed with the later validation as it were passing, as all of the checks are passing. So reverse dependency check and so forth. As for the submission, the submission really has only two prerequisites. The first prerequisite is our dependencies. We require all of the dependencies to be validated first and every unmet dependency will immediately stop the execution and the execution of the validation process won't resume until all of the hard dependencies are validated. And the second prerequisite is that the source code of the package has to be reproducible. So it has to be either a Git repository, a Git hash or a checksum associated with a parable. Later, the submission triggers a CID in the workflow managed through merge requests. This actually allows requesters, developer but also a support team member to see and monitor every part of the process, every part of the validation process. So every step is recorded and can be witnessed from inside this merge request. This also provides a grant for discussion. So developer or a person that requests the package can ask questions, raise concerns, basically tell whatever they have in mind regarding this validation inside of the merge request associated with the validation. So everything is one place and it's kind of public. Public actually are obviously to the internal extent. And as a result of that one obviously, submissions are overseen by the support team member and we are ready every step to help with the submission. And the next step is actually the package method data collection. So as I said, we have an automated tool that collects data about the package that we later used to populate the report. And we start with a basic method data. So name, version, alpha, description, documentation, basically everything that can be used to present this package in a later report. Then we execute RCMD check and record those results. We execute tests and calculate coverage. Then we map unit tests to the documentation objects building a traceability matrix. And last but not least, we record a system information about basically about the system where all those checks happened. And answering the question of how they are collected. So they are first collected, all of the checks are executed in a control to a producible analytic environment. So basically in like similar words, an environment that already has an R validated in it. That's simple as that. And secondly, we strictly manage dependencies. So we allow only dependencies that were later violated prior and they can be installed only from a validated R package repository. And here's some table that summarized those and the data we collect. Source control and documentation, our thing that's clear. If it comes to RCMD check, we, this is our hard requirement that it has to pass without any errors. Ideally it will pass also without warning and notes, but this can be later addressed. As for tests, they all have to pass. The coverage has to be above 80%. And as for traceability, every exported functionality has to be evaluated at least by one unit test. But it can happen. Sorry, warning in progress. Okay, never mind. But obviously it can happen that a package may have some gaps that has to be addressed. And it can be, those can be really minor. So like coverage is for instance, 75 instead of 80% or some small note about enhanced package not available for checking. In such case, we can address those gaps. As I said, using that human in the middle, I will repeat it over and over again, but I simply think it's very important for our process. And those addressing gaps can go into paths. It can be either CRAN comments, margin files, so shot out to CRAN or remediation procedure. Unfortunately, I don't have much time today. So I cannot explain those processes in detail, but feel free to visit our pharma case studies GitHub. There will be this exact presentation, but with slides added about both of those sections. And answering the last basic question. So what comes after validation? So the divided packages are uploaded to a validated R package repository. And we think that it's also, although it seems like it's end of it, but we think that's an important part of our process because it actually allows us to leverage CRAN style rolling package release. So we can use snapshot, we can use archive, we can use simple remotes install version to sometimes roll back if the upgrade is not sufficient for us. We can also, it also helps users because users, we embed the appropriate option repos to the image. So then users using it does not, don't even have to know whether they are using a validated package or not. They simply will always get a validated package. They will, they can focus on the purely on work instead of thinking, whether I got a validated package or not. Oh my God, what to do? They don't have to do it. They just get a valid package using install.packages and everything works. And the second step after validation is report generation. So report, our report is a mix of static and dynamic document. So static parts are basically part of the process, extensive vocabulary used during the check process and so on and so forth. Why dynamic part is those are simply made data collected during the check embedded in the report and rendered together. And all of this is uploaded to the document management service. And to summarize all of this, we believe that we created automated strict set of rules, but a set of rules that only highlights gaps to the support team members. And important note is that validation applies only to a specific version of the package run on the specific system. So if we like upgrade the R version or change the architecture, the validation does not holds. And to say it at the end, we believe that the process with developed assures high quality of the packages for regulatory analysis reinforces internal developers familiarity with best R programming practices because of those possible reiterations of the package after addressing gaps and reduces risk by making our analysts and developers more capable contributors. And I think that will be it and I really appreciate your time and thank you for that. Thank you very much. I already have at least five questions for so, but we're gonna save questions for the end because we're on the sort of 10 minute mark. So we'll go to the next question, the next presentation, then we'll come back and some of those questions will probably be generic to many of the presentations. So thank you for that. Next up, we've got Matt Bidler from Novartis, I believe. Are you here Matt? And is it you that he's presenting for us? I'm here, can you hear me? Yes, you can, hi Matt. Great, let me see if I can share my screen. I don't know if I need special permissions or not. I think you should be seeing my screen. Yep, we can see that. Awesome, okay. I'm not a Zoom user, so I'm trying to move things around here. Okay, my name is Matt Bidler and I'm presenting on behalf of the R Governance Group and Dave is also here with me. What we're talking about is our package risk assessment and validation that we're currently using at Novartis. It's a little bit different than some of the other approaches I've seen here. So I think there's some discussion. So the basic approach here is a risk-based setup with pre-installed packages that allow ad hoc install packages as well. So there are four different types of risks that we have. We have high-risk packages, medium-risk packages, low-risk packages, and then intolerable risk profile packages. And we'll talk about what goes on in each of those cases. And for the user, we have two different use cases. We have one where everything's pre-installed and that's always available to the users as they start up their R session. And then we have an ad hoc installation to address a specific analysis need as long as it meets certain requirements for regulatory work. So as we talked about here, we have three different types of risks. First is the low-risk types of packages. In the low-risk, we wanna make sure that it's from a high-trusted source, for example, BaseR, recommended software from the BaseR distribution like NLME, for instance, or the Tidyverse dependencies. If it's something that's from a reputable source, we believe it's also a low-risk that includes something like RStudio or Microsoft. The other thing we assess for reputable source is if there's a large amount of community usage, which includes a very high download rate or a large number of reverse dependencies. And then also some packages just by their nature are low-risk. For example, a display package where one of the display packages in R is praise. It's used for test that. But its only purpose is to praise the user for doing a great job. There's not really any risk to analysis by telling them they did a good job. Or for instance, data packages where they only have data within that package. There's no risk to analysis for those particular types of packages. So there are low-risk package. So if all those are false, we go to the next tier of risk. This would include coverages that are queried from the package. Our current threshold is 60%. If you're above 60% coverage, we assume that you're a medium-risk package. The other risk assessment that we look at here is this package in a peer-reviewed journal with credible citations. If that's the case, we also consider a medium-risk package. High-risk packages must maintain some certain criteria. They have to have documentation, they have to have releases, they have to have frequencies of updates, but that's not sufficient to be in our system. We also have to have a user to not just installation qualification, but additional PQ tests. So these additional tests and what's used in testing in our environment at Novars are categorized based on use case. So for the use case one, the pre-installed packages, all packages undergo formal IQ and OQ testing by IT. Low-risk and medium-risk packages do not require a business unit testing or no PQ testing. We still believe there's adequate evidence to consider them sufficiently trustworthy to use without the standard PQ. All high-risk packages require business testing to or the PQ testing to verify the most important functions that are used in analysis. The documentation for all this process is done with standard lifecycle and management tool and with QA approval. For the ad hoc usage, we make sure that the user installs the package and runs all the package tests, vignettes and examples. We make sure the user does a risk assessment. All that information is stored in whatever project location they have their data in and their analysis in. For high-risk packages, business user asked to store additional evidence has proved the credibility of the package, review the evidence and the evidence has to be reviewed with the project team and the project team ultimately has the accountability to sign off on any high-risk package used in an actual analysis. We also have a tool to help with this ad hoc installation and risk assessment. This tool is an R package that ensures the proper ad hoc package installation. By default, it uses whatever snapshot date we have in our system. Currently, we're using M-RAN, although we could use RStudio in the future. But once it has that M-RAN date, we have the user install whatever package version that existed at that M-RAN date to ensure maximum compatibility. We also install all the suggested dependencies and the tests to be able to run the examples, the tests and the vignettes. All those are run during the time of the ad hoc install and all that information is stored into a zip file. The zip file is saved for evidence of that successful installation was performed and you can look at it and see the review, the data to make sure that it makes sense. It also stores the package version information if you're using a different version or a GitHub or GitLab or a Bitbucket type of source to make sure that whatever you use is reproducible. It also conducts the risk assessment that we just talked about using metadata from the R package ecosystem and uses that to infer what the risk is based on our current practices. And that is stored in a Word document with embedded R information in it. And with that, that's everything I have for this brief overview of what we do at Novars. Is there any questions? Please feel free to ask. Thank you. Timing-wise, I'm gonna move on with the questions but I think there's already been one question in the chat. I think that's another thing. If people wanna put the questions in the chat at least then we've got there. I think we're gonna run out of time to be honest to get through like a ton of questions at the end, because there will be lots. So put the questions down and if we've got them documented and this couldn't, as we mentioned, a couple of the presenters have mentioned already, we are looking to capture this as well as the meeting here today. We're looking to capture this on the site in GitHub. So we will have questions that we can address through that in kind of formal written case studies as well. So all the questions you've got will be captured and answered in some capacity, whether we send our answers after the meeting or whether we address them in the meeting or if we do them within the papers that will follow. So please do document the questions, put them out just in case you don't get a chance to answer the question at the end. With that, I'll move us on to our third presentation, which is Pretherm, if you are online. Can you hear me? Yes, we can. Okay, and can you see my screen? Yes, we're not seeing the presenters, you were seeing the start, now we are. Yeah, that's for you. All right, thank you for the opportunity to present at the meeting and I'm Pretherm, I represent BARTs from Mark. We are being trying to implement the R package qualification based on the different frameworks that we've been exploring. And so this is just a simple case study that we are gonna be demonstrating using GGLE package. My co-authors are Paul Bernanke, Jane Leo and Elon Zhang. So starting off with respect to the package qualification implementation, we just looked at first the different guidance that's available from both regulatory authorities as well as cross-industry initiatives. So we focused on both the FDA guidance as well as the ICH guidelines for the different validation requirements. And also the different frameworks that are available from the cost industry initiatives, both our validation hub and transfer rate MSA initiative. So we have taken elements from both our validation hub and transfer rate framework to implement a qualification framework that minimizes both risk and increases the contents in the package being qualified. So that just a quick overview of what the qualification framework at Mark is being implemented as a risk-based package, qualification which includes a risk-based strategy which looks at the different types of analysis and reporting deliverables that are being used, that are being delivered using the package. And the goal is to create the documentation that contains qualification details related to that package based on some pre-specified criteria. So there's some examples that are available that can be used to look at the different risk levels is the deliverable. So if that package is being used for some sort of an external deliverable, such as an ECTD required deliverable, including CSR requests and submission packages, then that package has to be qualified under the low risk. If it's non-ECTD or other requests such as data monitoring committee requests or manuscripts and publication requests, it could be either in moderate or low risk. And if it's purely for data exploration or exploratory analysis, then it could be in the open risk or any of the other categories. Based on this, we have also defined the different criteria that are required for the package qualification. So we have specified about five different criteria that are required for the different levels of classification. So a low risk package needs to meet at least the first four criteria listed here, C1 to C4. And one criteria is sufficient to qualify it within a low risk category. A moderate risk can be qualified using any of those C1 to C5 criteria. And an open risk is any package that is available on PrAN or Bioconductor and is not qualified under either the low risk or moderate risk. And we have guidance in space within the organization that open risk packages cannot be directly used for regulatory purposes. So that is something that is already in place and it's a part of the standard operating procedure. Now, within this context, we also have an implementation of the global R library with the mark, which is a library that contains a set of directories with all the R packages and the dependencies installed. So the goal of this R library is to establish the shared baseline approach that RStudio approaches within the R package management and helps with providing all of the users a uniform and a shared baseline environment we can start with. And they don't have to worry about different dependencies or any of the other issues that pop up with package management. This global library is updated approximately every three months or as per business needs required. And in order to update this and also maintain the R library, we are using the RStudio package manager, the RSPM for the package repository server and also to install the packages in the global library. Within the global library, we have the three risk categories as I specified before, the low, moderate and open and it is a nested structure. So with all the low risk packages are part of the moderate risk category and all of the moderate risk are part of the open risk. The open risk category packages are anything that a user installs or downloads within internally within work from either from the RSPM as the RSPM is already synced with the CRAN or by a conductor. So they have access to most of the packages that are available online. So when we come to the package qualification workflow that we're implementing, this is a quick overview about how the qualification work from works out is we use a snapshot date within RSPM to pick a particular date where all the packages are frozen and then use, then also have a document about which packages need to be updated within the different risk categories. This package list only contains moderate and low risk packages that need to be qualified because the open risk are already part of the library and they don't need to qualify separately. Then we create a request form that can be used to inform IT about the installation of the different packages with risk categories. Before we send it off to IT, we do a driver installation of these packages to check if there's any issues with respect to installation or any dependency issues. Once that is complete, we send the request to IT and they start the installation process and once that is complete and simultaneously, we also generate the qualification documents that are required to qualify that package into a low or moderate risk category. With that in mind, the GGLE package was a case study that we would like to present. The GGLE is our package that extends GG plot to functionality and is requested by some of our internal teams for a moderate risk category for publication purposes. And so at the time of qualification, this package was in the open risk and so the process is highlighted here. It started with us reviewing the package documentation to determine which qualifying criteria the package could be qualified under and then we performed a driver installation for the global library update. With this in mind, I'd also like to point that we have an internally developed R package that does automate most of this process and it's not a manual process. We just use the functionals from that package to automate most of these steps. So once the driver is complete, we also do a check of installation log for any errors or warnings regarding this package installation and then also do a package code coverage and associated STLC documentation check. So you can see here two screenshots about the different STLC documentation. The one on the left is from the GitHub of the GG plot too and that is replicated within the RSPM because it's synchronized with the CRAN and the Bioconductor sources. Once this package coverage and associated STLC documentation is complete, we also cross check against an internal database which contains a list of R package authors that are in the trusted category and we call a white list. So these are authors that have had a history of package publications or whose have had package downloads more than 1,000 within the last two years. So we have a set criteria which includes these trusted authors. Once this checks are complete, the qualification document is generated. That qualification document I would like to point out if I can open it really quick is a HTML file that can be used to... And this is the qualification document that is actually generated for the different qualifications and it includes a complete documentation of what risk levels including the qualification date and the criteria. So it has information about the packages including the STLC as well as the different dependencies for the different packages that are dependent on this and as well as the qualification details in detail about how this package is qualified into moderate risk. Go back here. So in conclusion, we have implemented a risk-based qualification process at Mark to classify the different packages based on the deliverables that are generated using that package. And we demonstrated the package framework qualification framework using the GGLE package which was qualified under the moderate risk. There is still work in progress in the internally developed R package for automate more of the processes including checking for the different installation errors as well as dependency checks. And there was also ongoing work into defining the pre-specified criteria to qualify an organization or group of vendors as trusted source. This was brought up during a recent internal QA team discussion where there was more feedback given about how we could be using a specified criteria to qualify an organization as a trusted source or trusted vendor. So we're still working on that particular part of the process. So with that, any questions or feedback? Here's my email and that's pretty much my presentation. Thanks for being done. We're a little bit behind on time so I'm gonna quickly get on to Ellis and Becker. We'll probably have about 10 to 15 minutes maybe depending on how quick you guys are for questions at the end. Remember, all of this is feeding into documents afterwards. So I'm just gonna keep filling guys until you've got your slides up on screen. But everything will be shared through those talks afterwards. If you don't get a chance to ask a question today, you'll get a chance to do that for them. And in May 24th, when we repeat this exercise, we'll do some number of presentations as we go on a good half hour safe for the discussion. All right, with that, I'll hand over to Ellis or Becker whoever is going first. Well, thanks, Andy. Ellis, do you wanna lead us off? Sure thing, yeah, so today we'll be talking about our package assessment and the general ideas behind the way that we're approaching assessing various R packages. My name's Ellis Hughes. I'm a data science lead and the innovation team at GSK. Becker, you wanna introduce yourself? Sure, yep, I'm Becker Kraus. I work with Ellis. I also wanna give a shout out to Teal O'Blank who's also on the call and he's working with us on this and advising us as well, so. Okay, let's go on to the next slide. So the way that we're approaching package assessment is that we're really trying to do risk mitigation and it really is along a spectrum. So we're trying to balance case specific, time and effort involved with package assessment with how quickly can we get this done and how much information can we gather quickly? So there's two sides of this. It really depends on the risk the organization is willing to take on and their interpretation of the documentation that exists that's been presented in prior presentations like the ICH E9 which is you need proof and documentation that the software is appropriately tested. And so we're kind of taking into account advice from the RValidation Hub and just generally observing how the pharma community is approaching this. So there's one end of the spectrum which is complete reliance on automated assessments which is really only trying to use measurables really produced a lot by like risk metric or things like that like test coverage and whatnot. You execute the included tests. You look at general package development criteria. So does it pass the R command check that's included with CRAN? Is it on CRAN? If it is, there's usually a lot of trust already with the package itself but really we're trying to gain empirical evidence to prove why we should trust a package. And the value in this is that it's really scalable it grows really quickly. You can assess a lot of packages with minimal effort but it does have the possibility of you just start generating data without looking at what does this mean? What is the implication of this assessment? And there's not as much nuance in it. And then if there is any situations where if you've completely automated everything and something crashes, adding a human back into the loop can really gum things up and really make this slow. And so your fallback methods for complete reliance on automated processes needs to be really robust. The other side of the spectrum is complete reliance on expert opinion. So this is where you have an expert go in, look at the package, look at the functions, look at how you're planning on using the package. What is the package doing? Is it statistical in nature? Or is it data manipulation? Is it used for utilities? And that's really focusing on what is the package doing and relying on your experts to do this? And they can look at additional things like what is the development plan? What is the software development lifecycle? How robust is the code? How robust are the tests? The issue with this is that you're really tied by how much time your expert has to be reviewing these packages. And there's a limited number of experts, just there's a limited number of people in the world that have this amount of expertise. And then you don't really want them spending all their time just looking at packages because the reason you hired them you want them to be spending their expertise not on does this package work but actually using these packages. And so our goal is to hit a middle ground if you want to click space. And this is to leverage both of these ideas where we can use our automated assessments to try to target our experts so they're not wasting their time on things that's not valuable where they're not adding value to the review. So they don't need to worry at looking at like what is the test coverage? Does it need a minimum bar? Those sorts of things. It's given the way this package is behaving. Let's look at that and look at the way we intend on using the package and through the lens of the information we gather through this automated assessment what is the final conclusion of the package? Do we trust it? Do we not? Do we want to push it forward? And so, yeah. So we're trying to combine these two and reduce wasted time. So we do this by establishing a baseline level of trust. So we're using the ideas that exist that have been compiled by the ARM validation hub using risk metric, using tools that already exist. Test coverage, you can look at CodeCov and whatnot but then you also look at other things like the documentation quality. So that's a little bit outside of the risk metric world but it is still pretty straightforward to do where you start looking at vignettes. Are they helpful? Do they answer the basic questions? Can they get a user really going and using the package? The help pages themselves, are they instructive? Do they provide good examples? What are the main functions that exist inside of the package? And then you can look at the community itself. Is there a paper that they wrote that was submitted and was not accepted? Do the rate blog posts, do they respond to issues on the GitHub page? Things like that to really help derive the initial assessment of how they look at the package and that then drives this next piece that Becca's gonna talk about. Yeah, so when we have this baseline level of trust we're equipped with it, that informs really the next step in the process. And for us, that is really focused on testing and test quality. So with that baseline level of how much we trust it that dictates how big of an investment we're going to do in assessing tests and really doing a deep dive there. And so that involves figuring out test quality. It's a little bit different concept from test coverage and whether there's tests or not which we gain in that first part of the assessment. So figuring out test quality really does require us to look inside the code itself, inside the code base. So this step involves actually looking in the files and we need to either obtain the tar ball or the go to the GitHub repo, click around and see what we need. At this point, we've already identified key pieces of functionality that we're concerned with. So the most important functions are the ones we would focus on. And from there, we're trying to navigate the different files, you know, the source code and the documentation and the tests and so forth. A package and the package directory is not really meant for doing this easily. So what we've done, what we're working towards is having a nice interface for doing this more easily. So we have a prototype that we've built that allows you to whatever package you're assessing allows you to actually peek inside and navigate these pieces a little easier. So I just have a series of screen screenshots for the app here, for this prototype. For this example, we're looking at the read Excel package. We have a dropdown here on the left of the various exported functions we can toggle between. Let's say I'm interested in looking at the read underscore Excel function specifically. So that one's selected and I have three tabs here for some key pieces. Test code, source code and the help page. With the source code, it's literally just the code file for that. So we can see, you know, exactly how it's being programmed. Perhaps of a little bit more interest for this exercise is the help page and the test code. So the help page allows us to see exactly what the function is meant to do. It's inputs and outputs and really the different scenarios, different use cases that can come up. And then from there, we can actually look at tests and the test files and see how well what we can see when I clicked over to the test code tab, there's a little dynamic portion of the UI that pops up. All of the test that files, all the test that scripts that call our function of interest, some place in there. So you have access to all the places it's tested, not just the test read Excel function. So another thing that we make this process easier is if we can direct whoever's doing the assessment, if we can have a visual, a little more visual direction, visual cues to make this quicker so they can more easily find the places that the function's being tested. So here you can see the code highlighting that helps with that. So I wanna step back and say that the risk assessment app that Marley's spearheading is really a nice way to take the risk metric package and add this human component. So we're talking with Marley about adding this in potentially to the risk assessment app, this ability to manually navigate the actual package contents and be able to perform this exercise in the same place as the rest of the assessment that we mentioned earlier as the rest of the assessment that involves more of those automated pieces. So that's all we have as part of this presentation but we're happy to take any questions in the next few minutes. Thank you. Thanks, Becker. Thanks, Alison. Thank you to all of the presenters today. We were perhaps a little bit ambitious with the four presentations to take as we've gone to a point where we've only got nine minutes left of the call. So what we're gonna do is pivot slightly with this. And so next time when we meet on the 24th, we will focus that more on the discussion side. So for now, if there are kind of clarifying questions for the presenters, it'd be good to hear those. Otherwise, as I say, please use the chat and document those and hopefully we can get an export of that chat later so that we can address those within the discussion because there are no doubt plenty of questions. Julianne's just posted in there. Can you please send the questions to her email address there? So if you can post the questions to Julianne and thank you before we do run out of time, I just wanna say thank you to Julianne for coordinating this and getting all the presenters together as well for today. So thank you, Julianne for sorting that out and thank you again for the presenters today. So are there any quick sort of qualifying questions that anyone has based on the presentations today? I'm not sure if everyone can unmute themselves. I think people can self unmute. If not, I will ask one slight, one question that verges on discussion. And that is with respect to when packages enter in this process. This question for all presenters. Who governs what packages are allowed to enter the process? So can any package be put through your processes or is there someone kind of, is there a pre-gateway before you even get into the process decide whether a package is allowed to go through that or not? And this is freedom. So within Moraq, we have a process. I mean, we have some of our governance committee that actually looks at the different use cases for these packages before they're qualified. And also during the qualification, we do have a review panel of comprising of one programmer and as well as statistician that do the review of the different qualification documents and make sure that the packages are properly qualified. So we do have some gateways that actually govern these processes implementation as well. For Rosh, so internally developed packages. So those packages that were developed for Rosh employees and the source codes available on the either Rosh GitLab or GitHub, so any like source code platform. So those packages can be valid free. So basically on demand. So whenever the maintainer decides the package is in a place when it can be validated, the process starts. As for the crown packages and exterminate packages, they also can be requested by anyone at the very every moment but those are required acceptance by the team, by the validation team. From my point of view, this is Matt at Navarra. So we actually have our automated risk assessment forms look at all the packages that are evaluated for the ones that are high risk. That's where we do the most reviewing because we just wanna make sure that these high risks are not an entire risk category. We also make sure that we review the PQs in that point. So in general, we're kind of sorting this out as well. We're gonna have a team of individuals that are experts that will be reviewing the packages just for use case. And then there's an additional team of people that will actually be doing the evaluation of the individual packages that'll be giving feedback as to whether we'll be accepting those packages. So it's kind of like a two-fold approach. At least that's a general thought right now. Thanks everyone. Just to give a flavor for the sorts of things that we'll continue in this discussion. There's been a couple of other suggestions around taking discussions of Slack and so on. We do have a Slack group. We've not been heavily used by this group historically but that's another area where you can take some questions to potentially as well. So we'll try and work out the best way to get that conversation kind of flowing continuously with the presenters. So my idea was that these things shared on GitHub and there will be a public record of like all the quick Q&A that will be there. So if you're not able to make a discussion for example on the 24th of May, then you will be able to access some of the information, the questions that have been asked in some other way. I can at least promise that. Other questions that are coming in, things around like how have people worked with their quality teams around this process? Do people have like new SOPs and the likes to govern those? So that one's come in. We don't have to answer all these now just to give you an idea of the source things are coming up. How do you look at, if people are using combinations of packages that are not part of your testing process, if people are doing things that, you can only test so much. So if people are using things in an ad hoc way, does that matter? How do you account for that in the process? The question that hasn't come on by I wanted to ask was around if let's say a study team is looking to use a package or you're looking to use a function that's in a package that isn't in the current set, how would you accommodate that? Would it have to go through the process or do you have a means for teams to kind of all by self certifying it? It's okay, I know this one function is fine or would they have to kind of write that up, write the function themselves in their own code? So how would you handle packages that are not part of your base set? Because obviously if you install a new package, for example, if you had, I think it was a Roche presentation, we talked about containers. So if you had a new package to a container, you kind of changed what's being tested, mandated, qualified. So how do you account for that? How do you account for that?  So with that, with three minutes left, I'll give everyone about time back in their calendars. We will pick this conversation up. Thank you again to the presenters and we'll see you guys all soon. Fantastic turnout today. So obviously a lot of interest in seeing these case studies. So we look forward to a couple more cases and a big discussion next time. Just a quick question, Andy. Will the case studies that we sent out, we posted on the GitHub? Julian, do you wanna talk a little bit about the process? Yes, so we have a GitHub repo and you should be able to create a pull request and we should go through and actually approve them. So we put that on our to-do list as well. Sure. Okay. Thank you, everyone. Clearly a lot of interest in this topic. Nice to see everyone back again and we'll see you all in a month's time. Thank you. Have a good rest of your day. Thank you. Thank you, everyone. Thanks, Andy. Thanks, everybody. Bye-bye. Thank you. Thanks, everyone.