Comments for David. Yes, Heidi. So, David, I'm curious, you know, we all sign out cases all the time that are inconclusive, variants of unknown significance. So I'm curious to think about or ask you, as you think about whether to report out that last variant as, well, here's a candidate, but we don't have conclusive evidence to say it's causal versus reporting it out as negative because it didn't meet the bar. Like, you know, in thinking about the risk-benefit ratio to that patient and what may or may not be done subsequently to try and solve the story, and like, how do you think about when you decide to not report anything versus put something out there that's inconclusive in the context of a genome exome sort of study?  So we actually have pretty tight rules. I think one of the things that we've been very aware of is the amount of damage that's been done by variants of uncertain significance reporting and things like BRCA1, where people have had mastectomies based on the lab reporting of a variant of uncertain significance, and it being misinterpreted as causal. I think in a typical genome, you know, we get four million single nucleotide variants, about half a million interesting structural variants. Just by random chance, certain of those will fall in genes that are interesting. And typically, I mean, when we've done mapping this way of interesting variants that fall into interesting genes, we typically find between two and 20,000 variants that are quote, unquote, interesting. We are, I think, very shy of reporting out new genes, even with functional evidence. As a clinical lab, I think with my research head on those are the ones that are most interesting to go after and try and prove. But as a clinical lab, I'm not going to want to be the first person to call a new gene a pathogenic gene. So I think we fall very strictly into this box. This is the kind of the box. And we were talking earlier, you know, we're relatively strict about following the rule. So we won't report out category X variants. We will often throw them over the fence to the research lab and say, hey, you might want to look at this in this patient and a cohort of other patients that we have with a similar phenotype. But as a clinical lab, we won't report out. Other comments? I shouldn't have stopped you quite so soon. Let me just ask, as a dumb epidemiologist, I hear a lot about HGMD and the challenges and the problems with it. And I guess in my own mind, I'm thinking this is probably a group that wants to capture all the information that's out there and have other people do some judgments on it. But is there not some way to have it capture some of the judgments as well that we're, I mean, has anybody approached them about that? Yeah, we have. And they've said it has to be published in the literature for them to put that reference in there and they will not override what's been published. So it's been very frustrating. Or is it there isn't some way to work with that? Again, I, you know. Well, they just say they're very honest. They have no, they don't have the resources to do that. I mean, they would have to operate a 24-7 call center to be taking info from all of us who are doing this stuff about what we think about their variants. Yeah, I mean, as it is, it's $27,000 a year to use that database for clinical use, for clinical service. Just in building our pharmacogenomics resource, we think we've thought about this a lot. And I think the only thing that works is we curate as the author would have curated on the day they submitted the paper when the sky is blue, the sun is shining and everything seems to make sense. And then you have, so if you curate that way, then you can build on top of that an infrastructure to say, not so much, time has told that this is, but if you start making the initial curation filled with judgment, then you just have, it's untenable going forward. So, but HMGD and any other database can do that first level and then on top of that have mechanisms. So why not have a super HMGD or whatever that says, okay, here's where we started and here's the evidence that we've evaluated. Some of these filters that, in the examples today, a variance 20% frequency in Europeans, it should be pretty straightforward to pull out all 1,000 genome frequencies or all EVS frequencies and annotate them in HMGMD. So obviously there's the corner cases. A variant was reported, but there were no appropriate controls, so who knows exactly what's happening, but these extreme ones, it should be possible to flag all of them. Not have to go through them repeatedly. I mean, HMGD actually did work with 1,000 genomes to flag some of those 1,000 genomes variants, the show up again. But do they get flagged? I mean, I know, we talked about them in the project, a few times. So it's weird, they have some internal flag, but it's not distributed in the public release. It's really bizarre. To academics, it is distributed for some reason. Yeah, because I asked them about that. I said, you know, there's all these variants that are at like 90% in the ESP cohort, can't you fix that? And they were like, well, it's not in the download that you get. So maybe an action item that we take out of this is we don't necessarily have to make HMGD change the way they do business, but if we took it upon ourselves to make sure that a community action we undertake is to draw from 10,000 or 20,000 exomes that have been sequenced in the last few years collectively in a very accurate allele frequency and then publish that, some of our colleagues over here, as a single paper with a single supplement that actually articulates this, then maybe we'd have more traction in getting that uptaken by not just HMGD, but any database. And I would say just as a clinical lab, we functionally do this. We just have a way of filtering automatically. So we don't report anything that is over a 5% allele frequency. 5%. I'm sort of intrigued by the thought of the sheer number of in-house curated or fixed versions of HMGD that exist in the clinical population. I mean, I have no idea how many that is. Is there communication between clinical labs that could be used to help to merge those efforts into? Yeah, I mean, that's something that we're trying to work on. I've actually had contacted HMGD and said, because some of the U01 funded collaborative groups are all sharing these stories and we all have our own in-house databases where we've gone through all these efforts and we have all the HGMD variants, or I'm not all of them, but we each have our lists. And I actually went, because HGMD is planning to put together Scientific Advisory Board, so they'd asked me to be on it. And I said, I'd be happy to organize some efforts to bring everybody's data that has conflicted with the HGMD classifications and try to bring that to you so it could be put in there. And they haven't agreed to do that yet, but I think at least agreeing to form a Scientific Advisory Board, I think there's some hope that we can get some traction there. It seems like the interpretation of very rare stuff is very much a numbers game, right? And individual, I mean, one of the things I liked about one of the slides we had up there where it almost looks like a medical note, right? Where you're writing a note about a variant, right? And I just, it seems like, I mean, maybe HGMD isn't the appropriate sort of nexus for kind of organizing an antagonistic community about the problem with HGMD. It seems like what you want is something where everyone can contribute annotations from wherever they are about variants they observe in their patients in some addendum type way to get those numbers, to be able to add interpretation over time. Yeah, absolutely, and maybe Donna can speak from the NCBI ClinVar viewpoint in terms of creating a truly publicly accessible environment to be able to do that, and we've been trying to work with them, and hopefully make some inroads in that area. Other comments? My concern is that the utility of the HGMD, despite all of its problems, is because it's curated, and that sort of what distinguishes it from DBSNP, right? Where you can just put anything in. So I think that there does need to be some curation. Maybe the curators are identified as people who have a particular interest in one gene or another, but that seems to be an essential component of maintaining the integrity of such a database and engendering the trust that somebody would have if they were testing lab that you know does their homework when they report something to you. And I can add to that if you could put up number 14 on the screen. So in working with Donna and thinking about how to annotate the curation level, so that who's curated that data? This is sort of the scheme that we came up with where within ClinVar there may be data at the bottom that is uncurated like data in DBSNP and ESP cohorts, but then the laboratories, the clinical laboratories that we're working with to try and get their data in there would come with clinical assertions, but it would be marked as single source curation. So somebody said something about it and you see who said it, so you can perhaps pass some judgment on who that was. And then there's this interlaboratory multi-source where you're actually collecting multiple different curations that may be the same or may be different, but there's some ability to find consensus because there's multiple ones, but they're still done independently. And then you get to the expert curation where you truly bring a group of experts together who in a consensus evidence-based manner collectively decides a level of evidence on a variant and classifies it. And then there's some variants that now today exist within clinical practice guidelines that can be marked in reference to those guidelines, but that was our attempt to really set up a scheme so that you could have a place that has all data, everything from the only population frequency on up to the most clinical grade data and not have to go to eight different sources that are different grades, but yet still have some way to show a difference in the amount of curation that's happened on that data. So it's just what we've been thinking about. I think some of it is simply crowdsourcing. I mean, there's a number of places. We have several people doing just what Heidi described as that happens in her lab happening every day, what David describes. Some of it is just gonna be creating standards in a forum like here and then bringing together all the people who are doing that to speak to the same standard and just pull that data together, because it's happening. I think we just need to harness what's already happening out there and bring it together into one thing. David, Demak, you made the point that you're totally dependent, obviously, on what's in the research realm. So if you had your druthers, how might you re-engineer what we do in research aside from single papers being published on individual things that often conflict? Well, I am a researcher as well. So I think that's a hard question to answer. One of the things that really would make a huge difference is just putting the bar higher for publication. You know, I review papers where they say we found this variant in a kid with this disease and it's like, and the proof that this variant affects the function of this gene, just because a patient has PKU doesn't mean that you found a polymorphism in phenylalanine hydroxylase means that causes the enzyme not to work. And I think for too many years, we've accepted that kind of, should we say, sloppy assumption. And I think we need really as a community to tighten up the publication standards to really show people, you know, if they're gonna show causality, which I know is what we can talk a lot about tomorrow, that really they show causality, not just an association. And if it's an association, that's okay, but it needs to be described as an association, not as causality. And the trouble is association with GWAS has almost assumed a different meaning. So I guess we maybe almost need another term for it as something that we happen to find in an individual, but we haven't proven that this specific mutation leads to this phenotype. And so I think, you know, with the paper that everyone saw with the retraction for the missense mutation from the mitochondrial group, I think that was a very brave thing for them to publish. I think also it was very important lesson for us all, but just because a child has a variant in a gene and the gene doesn't function, it doesn't mean that that variant causes the gene not to function. There's not too many negatives and positives in a sentence.  And I think maybe also, once again, on the literature side of things, actually getting towards a more structured approach where curators can more easily pick out what an author is causing pathogenic versus associated, because I think, you know, one of the things we criticize the HGMD curators, but I think there are times when I look at papers and I try to work out what the author thought was pathogenic and what they thought was neutral. And I understand why the curators had trouble because, I mean, they're human and it's hard to read some of these papers. So I think also moving towards more clear-cut requirement that authors come out and say they think something is pathogenic or that they think it might be associated and why, and providing them to put in a structured way the proof for why they think that's the case, I think those things would make a huge difference. And then the other things that have already made a huge difference, the ex-embarian server has been a godsend to all of the clinical labs. It's, I said to Bogeet who works in Heidi's lab, this will change your life when we stumbled upon it. And it really, it's changed our life. It's changed all of the clinical labs that I know is live. It's just in a radical way. It's really, really useful. And that's just from a relatively small number of exomes. I mean, it's respectable, but imagine if we brought together all the exomes. And imagine if we actually had phenotypes to go with them as well. Yeah, I can really start dreaming now, right? Can you clarify for me? So this is from the, I mean, Gonzalo can probably speak to the exact number right now, but about 6,500 exomes from the NHLBI exome sequencing project from a, 6,800 from a diversity of heart, lung, and blood-relevant phenotypes. Gugli, EVS, Exome, GS, Washington. They have a really excellent variant server in Google. It comes up. But it has allele frequencies broken down by ethnicity, but not no individual level data and no phenotype data. The other nice thing about the ESP cohort is you can actually determine that a region was covered to be able to say that a variant was absent, which is incredibly useful also. What he's doing is changing his resolution. Mike has advised us that we should be at 1024 by 768, which is a kind of a wimpy resolution if you like a high definition screen, but that's what the display likes. So Mike also needs to test everybody's machine around the table. He's tested only 7 of 32 so far. So some of you could stay over a little bit later, and he can check them or check them tomorrow morning. But Les, go ahead. I just thought it was worth pointing out this paper. Oh, it was worth pointing out this paper, which came out late last spring, I think. David Rosenblatt is basically suggesting going kind of the other way with a lot against the arguments that we're hearing here today, which is to say publish everything. Just sequence them and throw it into a paper. It doesn't matter if it's causative or not. Just put it in a paper. So really the question he's posing is whether or not you should publish it, might change our question into whether, not whether we should be publishing things, but what we're allowed to conclude about our research. And so we're going in opposite directions here. I'm not arguing for this. I just thought people ought to be aware of it. So I just wanted to amplify in the last two comments. I do think that if we have a more structured literature where we can publish something as English text, but we also have structured ways of saying things, people are going to be able to mine that lots of these observations and get consensus kind of properly statistically and so forth. Oh, I just wanted to echo something that you said, David, about the fact that we shouldn't set a bar so high that things don't get published. Because for somebody who works with extremely rare diseases, I'm searching around for that second and third case. And I'm happy to look at something where there's not great evidence, but maybe those researchers and I can put that together to try to create better evidence. So I wouldn't want those to be excluded. I think it's just important to document what the level of evidence is. But we do have to be a little careful with that, right? Because that's creating yet another multiple testing problem, right? If everybody just throws the stuff up and you find, oh, here's two people with inflammatory bowel disease and rare variants in the same gene. You have a huge multiple testing problem there. Yeah, it's a similar point. So it's a really cumbersome way to, first of all, it's a cumbersome way to look for that second or third patient.  So surely the way to actually think about doing this is to have some kind of a centralization system that's independent of publication. I mean, that's really what we want to do. You know, I might just note that six years ago or so we were in the same situation with GWAS. And that was one of the things that, you know, there were two things that I think came out of the discussion that Mark so ably started with the despondent gene was that we came up with some standards. And we also, I think, with many of the journals came to an agreement that, you know, if you're going to publish GWAS data, you need to submit those data to DBGAP. And several of the journals really got behind that. And now DBGAP isn't the be-all and end-all, but it was a start, at least, to having some of the data available. And I think we're, in many ways, we've kind of moved beyond that to get to a structured format. And so maybe something that we can be talking about a little bit tomorrow is what kinds of databases would be useful to develop and how might we go about doing that, as well as what the standards should be. Yeah, I totally would like to emphasize and highlight exactly that because we have this sort of now unique opportunity. Like the discussion has already sort of gotten back into trying how do we retrofit our interpretation and evidence based on a huge backlog of literature. And that's a notable and laudable goal to try and do, but we should also keep in mind exactly what the, if we sit in a room, like we were doing, and we try and imagine a future that we'd like to have, what is that future? What is, if we had talked to our future 20 year old, 20 year future colleagues that said, if you had done these things now, what would those things be? And I think Terry's exactly right. We have to imagine what kinds of databases we'd want. Maybe what kinds of things constitute evidence and what kind of category storms for particular variants we want to rate in terms of evidence. Because of course, statistically, that's gonna be the first thing we're gonna look to. And I think it's possible to do things like rate variants and be mindful about what kinds of rigor those particular variants have been, whether or not they've passed a multiple testing threshold or what kinds of evaluations given studies that have been done and discoveries that have been made on those things, what that actually means. We can put that into a database, into a way of structuring information and then start to gradually build and document evidence that's non-statistical on top of those things. I think whether or not, sort of, I think what Mark will probably tell us about tomorrow, sort of integrating evidence, I think clearly has to be a way of doing that. And so we have to imagine what the structures of information that we have to sort of hold that information and then what kinds of queries we might do to that. I don't know how else we're gonna, I think that's sort of what we should think about framing these questions. Because if we try and go backwards, I think we're gonna get stuck. That's an excellent point. And I think we were hearing while you can look backwards and find the history that makes you really uncomfortable. And I think the stories that David Valley and Heidi and David Dimmick have told us should make us all lose some sleep tonight. Because facing those patients is a really scary thing. And yet, we're really just at the beginning of this. I mean, how many of these have been published so far? Not very many. And there's gonna be a flood of them. And so this is our chance. So get energized for tomorrow. I would say that pseudogenes and paralogs don't just deck me from sleep, but they give me nightmares while I'm actually sleeping as well. I'm worrying about them and calls we're making for patients. I was gonna say one theme that we had been touched on briefly that was very relevant in the GRS here and is relevant here too. I mean, causality is often a multiplex thing. And so it's not just the significance of the causal relationship, but the magnitude of the influence. And sometimes that can be almost impossible to quantify with such small numbers, but it's not something we should forget when we're thinking about what we've traditionally known as single gene disorders, single variant disorders, almost certainly are oligogenic. I agree that we're just at the beginning of this, but the pace at which it is rolling is unbelievable. And I know that at a place like ours, which has a fairly prominent genetics group, there are clinics that are ordering whole exomes from a variety of commercial sources and using that information to deal with patients. And I have no earthly idea how they interpret the data, what they tell the families, anything. That's a sobering thought, goodness. That's called Hopkins Hopkins. That's a jake issue. Well, and the scary thing is obviously everybody wants to practice up-to-date state-of-the-art medicine. And if that becomes the standard of what that is, that's real scary. David Demick, you have your mic on. Is that to make us more nervous Okay. Well, it seems like we're at about a stopping point now. Daniel, did you wanna make any progress? No, I thought it was incredibly constructive and really sets the stage for tomorrow. I mean, we have a lot to get through. But I think if we can focus on the points that were raised in this particular session about the sheer importance of getting this done and getting it done quickly, then I think we can really make some serious progress tomorrow. So just again, to lay out the schedule, we'll have one-hour sessions from each of the different working groups. I would like to really try and focus on the discussion as much as the actual presentation. But of course, this is an opportunity for the working groups to show off all the work that they've done prior to the presentation in laying the groundwork for this discussion. And we would also ask if possible that the presenters in those sessions may be finished with one slide, laying out maybe five or six points that they see as absolutely crucial to help guard the discussion just so we can make sure that those critical points do get covered in the subsequent conversation. And on that, I think that's it for me. Yep, I think that's it. Any closing comments? We started at eight o'clock tomorrow, so, yes. This is the first time we've actually done live streaming and we're very excited about it. It's not that we're behind in technology, it's that we're trying to do it at no cost. And so, this is the no cost version of it. But we may not have that capability. Maggie, so they were asking, could it be posted two days? Yeah, so it won't be in time for the meeting. We can circulate the slides though from all the presentations, so that's what we're going on. Yeah, but you want them like tonight, right? Yeah, yeah, yeah, that's it. So, do you have the slides now? Okay, so we need, so we have Marks so far? Yeah, Heidi and David. All right, so if Heidi and David, you can send them to Chris and then you guys can send them to everyone. So you have the emails of everyone? Yes. We're done. Break.


Okay, I think we are at day 41 here for the Aidu and the old reliable arrow gardens. I got my tomato plants here that are just growing like crazy and as you can see they are taking over the whole growing space here. They have a lot of extra branches and as you can see here on my arrow garden they are sucking up a lot of water. So first of all I am going to give this arrow garden some more water right away and then we are going to start pruning down these tomatoes. Get rid of a lot of extra foliage that we don't really need and get them ready for pollination. We have a lot of flowers on here so I am going to show you how I am going to get that set up.  And if you recall I have a basil way back here that is getting totally blocked out of the light and in the front we got some time which is still hanging in there pretty good. Okay so we are going to start down at the base here of the plants and try to get rid of some of these branches, get a little airflow going in through here. And I am just using regular scissors here that I have just sterilized with just a little bit of hydrogen peroxide. I probably should get myself a real pair of shears down here but these seem to work. So you just kind of start hacking away at your tomato. It really is pretty tough and can take some really harsh trimming here. So I am going to just try to start at the bottom, get some of this little foliage out of here. Once we get a fan going the air can flow through a little bit better and as you can see I had to add water again. I think having all these excess branches on here really sucks up a lot of water so don't need to have them here. You just kind of watch for the branches that have the blooms coming off of them because you want to make sure that you are not cutting those off. The ones that I am cutting off don't have any signs of blooms or anything coming on them. I kind of want to be able to get this basil out in the open again too so it can keep growing. I like to take a little harvest off of it every week. So that was a tiny Tim so I am going to start by hacking away at the bottom a little bit. So over here is the yellow cherry and it is starting to show a little sign of stress here. So I think it is a little bit overdue to prune it down a bit. So you are just going to have to walk around and have a look at it from different angles. See where you are cutting off. See this one here does have some little blooms on it so I am not going to chop that one off. Such a thick plant these dwarf tomatoes. They really got pretty heavy thick foliage. As you can see I have some blooms coming here. I am going to chop this one off down here. Let's try and give you an angle here from the top. I said some of these branches are looking a little weak. Cutting this one off. It does have a lot of suckers coming in between some of these branches. Just going to watch which ones. You can trim some of those off as well. Just going to have a look and see what we got left here. We got branch with blooms here. Some blooms here. Back here up here we got lots of blooms. I think I am going to hack off one more branch and then I am going to stop. Let's see what happens. I said it looks pretty thin and gangly now. Just by getting rid of a lot of those branches that are just basically sucking up nutrients should turn out to be a little bit healthier plant I hope. Very tempted to cut this one off just because it seems so big. It does have a little sucker coming here that is probably going to have some blooms on it. We are going to do it. As you can see here I have really hacked away at these two tomato plants pretty good. I have thinned them out quite a bit. Lots of room now at the bottom for air flow. I have lots of branches with flowers coming and blooms happening. I hope it can make it through the shock of all the chopping I have done tonight. Here is what I took off of the tomato plants on my aero garden here. The basil has got some light again. Time has got lots of room to grow. Now it's time to work on the Idu tomatoes and get them trimmed up. Growing in the Idu with my tomatoes here I have the mint and the chamomile at the back here. They are pretty good growing good. I could be harvesting some mint already. I want to try to open up and get rid of some of this foliage on the tomatoes just so these guys are getting some more light. At the front here I planted some arugula but it's kind of a big plant that needs a lot of space. I'm just going to actually pull this out and cover that whole up.  I didn't cover up this pod. I see there is some mold growing on it as opposed to LJ or maybe that is kind of an LJ. So I'm going to maybe cover that up as well and my chives never did come. So I'm going to just close up that hole as well. I'm just pulling out the arugula plant here and you can see that it's making the other plants move because all the roots are intertwined with each other. I'm just going to keep pulling gently and see how far I can go. If I need to I can always... So it's released here. These roots obviously belong to the tomatoes so I'll just gently push them back in here. So I bought some of these a little while ago that kind of do the same job as that cool noodle that you can just put them around the stem of your seedling and it'll cover the whole, keep out the light. So I'm just going to use that right now on here. As I mentioned here this oregano has a lot of what looks to be mold or LJ coming here. You can see there's a seedling actually ended up on the outside of the pot.  I'm actually going to try one of these that I bought. I might as well see how it works. It's pretty slick. So you just basically open it up and wrap it around the stem of your seedling. Get it in place. There you go. I like it. I have to get some more of these.  I think they look way happier. They've got lots of breathing room. They've got lots of blooms here that we need to get pollinating on. Over here on the eye do I see that nothing is blooming yet but there is still I can see lots of little flowers and buds forming here. So hopefully in the next week or so those will catch up as well. And as you can see there's lots of space underneath. My chamomiles got lots of light now and mint. And hopefully this little oregano guy will take off too. So I'll probably keep trimming away a little bit throughout the week. So we'll check back in about three days, four days, just to see how these plants have survived the butchering that I gave them tonight. So it has been about five days since we've done the pruning on these arrow garden and eye do tomatoes. We are at day 47 since they've been planted. And these plants are looking really good. So you can see they've got lots of airflow, not too many excessive branches on them. We've got tons of blooms coming here on both the arrow garden and on the eye do plants. So that's great. So I'm going to be running this fan here. I used to set it for two or three hours every day. And just let it blow gently on the plants to help with pollination. Come around, give them a little flick every once in a while. And they seem to do pretty well with that. And hopefully, as you can see here, you can always see we have a little tiny tomato coming. So we should be able to see some fruit probably in the next 10 to 20 days. So thank you for watching my part three of the arrow garden versus the eye do systems. And stay tuned for upcoming videos where I will show you some of the harvest from these plants. Thank you for watching and happy gardening.


Either this is a contradiction of the Trinity, or there's an easily explained answer. We believe because the Bible teaches such that Jesus is God, but there's a problem that some people believe is a problem, or possibly a contradiction. And that is, in James 1, it says, verse 13, let no one say when he is tempted that I am being tempted by God, for God cannot be tempted by evil, and he himself does not tempt anyone. Wait a second. If God cannot be tempted, and Jesus is God, doesn't the Bible say that Jesus was tempted? And if that's the case, then we have a clear contradiction. A contradiction is to say that this is, and then to come back and say this is not. So, A is, or A is not, those are contradictory terms. And so, can God be tempted? The Bible says in James, no, but then it says that he, God, or Jesus was tempted. So let's go and see. In Matthew 4, 1, the Bible says that Jesus was led up by the Spirit into the wilderness to be tempted. So he was led to be tempted. Again, it says that Jesus or that God cannot be tempted. And then we have in Hebrews 4, verse 15, says, we do not have a high priest who cannot sympathize with our weakness, but one who has been tempted in all things as we are, yet without sin. And of course, this is the very same Hebrew passage or the book that tells us that Jesus is God.  So Jesus has called God in Hebrews and said that he was tempted, but without sin. But the point is, or the question is, how could that be? That seems to be a contradiction. Well, it's not understanding what the word means is the point. If we go back to James, let's just look at what that word, paroxo, means. The word paroxo means to be tested or to be put to the test. It means to attempt, to put a trial in front of, to be tried. And what the Bible is saying is not that Jesus could not be put to a test. It's not that Jesus did not have to go through trials, the fact that those trials did not work. Here's what I mean. Let me give you a practical example. To me, if you were to tempt me with something that wouldn't would maybe tempt some other folks, but not mean. It doesn't mean that I'm not being tempted. It means the temptation will not work. If you were to tempt me with asparagus, if you were to tempt me with carrots, if you were to tempt me with something that I really don't like, it's not going to do very much. You try, you put me to the test with something that I could overcome. That means that the tempter did not try. In this case, the devil tried to tempt him, but tempt him with what? The very things that he created. He tempted him with power or the world or with food. You could not tempt God with what he created, and so he was put to the test. He experienced it, but it did nothing to him because it was not a trial for him. Now you could tempt me with sweets. Yeah, then that would work. Some other folks, you might tempt with sweets and it wouldn't do anything for them. And in this case, Jesus went through some similar states that we went through yet without sin. Whatever was brought before him, nothing could cause him to sin. And so it's not to say that he could not go through a trial. He could not be tested. What James is trying to say is that you can't really get away with testing him. He will pass the test. That's the point that James is making. You cannot cause him to sin. You cannot cause him to feel as though he's missing out on something. You cannot get him to desire things that are not of him. You could not get me to desire asparagus. You could not get me to desire some other things. Really with God. And so in this regard, no, we do not have a contradiction. We have a statement of fact that James makes. You can try, but it doesn't work. And it didn't have to. And his point is it doesn't have to with us because we are led away by the lust of our own heart is the point that James is trying to make. There is no lust. There is nothing ungodly. There's nothing that God can be or in this case, Jesus can be led away with. And so in that regard, we can kind of see, now it makes sense, okay, God or Jesus cannot be tempted. Amen.


Alright guys, so firstly the thing you'll notice is my mental little set up here. It's a bit messy. I've got my drawing batteries on charge I'm gonna be flying around today Anyway today's question is from somebody who would like to remain anonymous and they say hi Stefan I'm subscribed to your YouTube channel and I'd like to answer these questions I'd like you to answer these questions in a video, but I would like to stay anonymous Number one. How can I do a layer four and even a layer three? Lucidreams without the risk of accidentally changing my thoughts and beliefs Number two. Are there any dangerous risks similar to this? This pose is an interesting question and an interesting concept and that concept is Can you change your beliefs and thoughts in a lucidream? And if so, how easy is it to do it? Can you do it by accident or is it something you have to really intend to happen? Now I'm going to share my personal experiences with this which are as follows It with lucidreaming and through lucidreams I have managed to change a lot of my personal beliefs thoughts and my belief system which makes up my sort of Personality and what I believe about the world, but it's not something I've done just in a lucidream It's something I've done in waking life as well so you can't you can use lucidreaming and I'll get on to the layers in a second because the layers do have an impact to you know What effects it will have on you you can use lucid dreaming to affect and even bolster your beliefs or you know thoughts about the world But you this is very deliberate It's you know, it's very rarely spontaneous and it very rarely happens You know by accident as this person has described You can rarely accidentally change your beliefs want to be able to change your belief, right? It requires so much willpower and practice and persistence Because beliefs are really strong, you know, if you have if you hold a certain belief about something The longer you've held that belief the harder it will be to change And you can't just accidentally have a lucid dream and like wake up in the morning with that lucid dream without belief changed in Most cases now you can do it You certainly can do it, but it's not an accident some sort of persistent sort of action moving towards this It's not something you can just do accidentally in one group in in one lucidream now the different layers of lucidream Let me explain how they tie into this. We I've spoken before about the layer one two three and four I don't think I've made a layer five video just yet, but it's coming now layer four is Essentially just a lucid dream a dream in which you're aware that you're dreaming and you're in control of what you do And to some degree you're in control of the dream world as well You are not in control of your subconscious mind Because you can't be right. It's your subconscious. You're not you're very rarely You know, there are exceptions and it's sort of like a It's a it's a an emerging field of study, but you can't really control your subconscious mind You can influence it and you can definitely work with it, but you can't you can't directly control it because otherwise you'd be able to do things like Voluntarily raise your body temperature or decide to digest your food slower or faster You can influence those things, you know Like for example with digestion if you decide to go and do something that will release adrenaline Like a skydive or like putting yourself in danger that will Temporarily slow down your digestion because you have that fight-or-flight response and your reptilian brain takes over and you slow down your digestion Why you deal with the threat You know, we know that stuff we can influence our subconscious mind in that way But we can't control it consciously was just our thoughts alone and for that reason going back to the original question You can't very easily and you certainly can't accidentally change a fundamental belief that you hold It's gonna take a bit more work and it's gonna take a bit more consistency. It can be done, but it can't be done overnight So don't worry about that don't worry about having a lucid dream and then waking up like I've completely changed person Although it you know a profound lucid dream is going to change you in some way It might change your outlook on a certain person or a certain idea, but it really will completely change a belief system You know it can but it is it's rare is what I'm saying It's not gonna happen all the time so regarding the different layers obviously the higher up the layers you go the more Lucid you become right up until layer five which and even beyond layer five, which is like super lucidity where you're real you're hyper alert you're hyper aware and It requires a balancing of your hormones, which you know it when we get on to this advanced stuff in later videos It requires a balancing of your hormones meaning it has to be done at a certain time of night There's not really any way around that it has to be done during your longest period of REM sleep when you're just starting to produce The right amount of serotonin that's going to get ready to wake you up and increase your alertness for the day But you've also got a significant amount of melatonin still in your system enough to keep you asleep and sort of like sedated I guess you could say Where you're going to be held in the lucid dream and it's going to be a deep Immersive experience that's for advanced stuff, right? I'm not going to get on to that just yet because I'm still trying to figure out like My audience on this channel and how how far you've progressed in lucid dreaming and you know What sort of level you're at and by the way, it would really help me out if you could actually just comment letting me know in a sentence or two how lucid you you feel how Advanced you feel as a lucid dream. I do you think you're a beginner Where you maybe have and by the way, don't rate this on how many lucid dreams you have I want you to describe your level of skill based on when you have lucid dreams What level of lucidity are you attaining not how many you have but what level of skill you're attaining in the lucid dream So meaning like how stable is the dream? Are you definitely lucid? Can you like work with the dream and interact with different things and set yourself challenges? Can you for example enter a lucid dream with the with the intention of doing something specific?  These things are going to be useful for me to know so I know what level of skill I Need to produce videos for in the future because as you can imagine with almost sixty thousand subscribers It's hard to produce a video that's going to be useful to every one of you that being said I'm sure there are most of you just watch these videos anyway Just because you you know like my hair flicking around or you like hearing my rants. I don't know I'm back to the original question.  I mean like when I started it was difficult at first But then the more advanced you get the more you can use it for and eventually it just becomes like becomes part of your life just like a tool Just like you know that going to bed early and getting a good night's sleep is good for you You know that eating vegetables and drinking enough water is good for you And you just sort of use these things as tools like if you want to feel better you drink more water You eat healthier you go to the gym you work out you see your friends You do things that you know will make you feel better in the same way that once you've built the lucid dreaming tool box I guess you could call it the toolbox You know that if you want to work on some deep subconscious problem You have a lucid dream if you want to overcome fear or release anger or Get inspiration you have a lucid dream It just becomes a tool that you can use like meditation if you're feeling stressed you meditate If you're feeling hungry you go and have some food if you want inspiration You have a lucid dream and in the same way if you want to work with your belief systems or you know overcome phobias or do sort of emotional Emotional work on yourself You use lucid dreaming so that's it for now guys. Please go ahead and leave a comment You do know how much I love my comments on my videos and I'll see you next time Thanks for watching guys this video and this channel was supported by my patreon followers who are updated in the description of every Video at the time of uploading. Please consider giving just a dollar a month to support this channel Or just click the links in the description you'll find links to various lucid dream products articles techniques and tutorials If you did enjoy this video, please click the notification bell and subscribe and I'll see you next time Why are you still watching this? You should have clicked one of my related videos by now, right or subscribed or gone onto my website or something like that


So I'm here at KDE Academy where we're learning about all things KDE. For those of you who don't know, there's a Minecraft plasmoid for KDE that allows you to plug Minecraft in on the desktop. Our community member AIX is going to do a demo of that tomorrow. I'm super excited to be here and encourage anybody who's in Vienna this week to come out and say hi to the KDE folks.


a little bit differently because this is the last panel. We're going to put the last polling question up now. So if I could put the last polling question up and then you'll have the 30 minutes during the panel to think about this. So we're hoping this time you get the right answer. So the question is, what is the biggest barrier to widespread use of real-world evidence in biopharma, lack of data standards, interoperability of data collection systems, data science talent in biopharma or concerns over data security? So this will be up for the next 30 minutes. So we can now come back. So it's a great privilege to actually have been displaced because typically I've sat on this R&D head panel now for many, many years. As many as I've been in R&D head, which I'm now in my seventh year and it's always been the Hallmark event, the last event of the USAIC summit, but no longer data and digital sciences have taken over and they're now the event that will take us home to the finish line. It's a great panel. I'll introduce the moderator and then the moderator, Sashtri Chilakuri will introduce the panel. So Sashtri is a rare intellect. He spent his early part of his career at McKinsey where he was helping companies figure out what their digital and data sciences strategy might be. And then he realized, hey, I've got a lot of really great ideas. So he started his own company, a company known as Acorn AI. Is that the name of the company? Yeah. And then you've been serially acquired, which is a great testament to how successful you are first to metadata and then to the salt systems. So Sashtri, I know you have a really exciting panel plan. So without further ado, I'll kick it over to you. Great. Thank you so much, Andy. Really excited to be here to talk about digital and data science. And we have an illustrious panel. The panel needs no introduction. We have Rob Califf, Daphne Kohler, Karen Najat, as well as Craig. So why don't I kick it off, Rob, with you? It's been an incredible year for many of us over the last 12 months where we've seen amazing innovation. It's a miracle that we have a vaccine that all of us are functioning as a society back again. And we've seen tremendous advances both in the use of digital and AI. However, it's also highlighted tremendous amounts of inequality as well as opportunity in terms of the broad healthcare and the opportunity in front of us. Why don't you frame it for us in terms of what do you see as the greatest areas of opportunity as well as the greatest areas where we've fallen short? Well, thanks, Senate. It's great to be on with a panel of real experts. I'm sort of a middleman of data science. I'm not a real data scientist myself, but I've always worked closely with data scientists whom I admire. And by the way, it was exciting to hear the heads of R&D all agree that their companies will just be data science organizations in the future. That is a real area of opportunity, I would say if the leaders of those groups say that. But I think while we're hopefully coming out of the pandemic now, and certainly many of us are feeling better about the direction of things, in many ways the pandemic just shown a very bright magnifying light on the problems that existed in our society with our healthcare systems and with the way that our pharmaceutical industry works and serves society. We're in the midst of an onslaught of common chronic disease at the same time we're having a pandemic. In fact, in the US, more people died of heart disease and cancer than of COVID-19 during the same timeframe last year. And this was on top of a steady decline in life expectancy in the United States over the last seven years, five out of the last seven years we've seen a decline in life expectancy. And this is not because of people like us on this call. We really live in the US at least in eight Americas and I think around the world there's a very similar set of issues going on where health and biology intersect with social determinants, which are dominant over biology in most aspects of health and longevity. And so I know the panelists will talk about drug discovery where in my view, it couldn't look better. The ability now to use digital technology and the amazing aspects of computing to look at systems biology and to refine targets and to find new treatments is truly profound. But we have other issues that we need to attend to. There's been a lot of discussion at this meeting about more efficient clinical trials but I also wanna say more efficient does not necessarily mean smaller. If the purpose of a clinical trial is to inform what should be done in clinical practice and maybe that we need larger clinical trials just done in the right places in the right way without wasting time and energy on, for example, manual human data collection that doesn't add anything to the assessment of benefit and risk. And of course, the biggest part of the use of pharmaceuticals is a post market phase which in many cases I learned as commissioner goes on for 50 or 60 years with new discoveries yet to be made. And so I think the intersection of digital information in the post market phase is going to be a huge area in both drugs and devices. And the one thought I have about the big opportunity I think we have which I hope Alphabet my current employer will be involved in is a creation of essentially a COVID tracker for all of diseases. I don't know about you guys but most of the last 15 months I've gotten up every morning and looked at the COVID tracker to see what's going on in the world, in the United States, in the state I happen to be in, in the county I'm in and even down to the local hospital with up to date information fed in from a variety of sources. And knowing now that just having a good drug is not enough we actually need to deliver the drug to the people who need it in a way they can afford it and in a way that's convenient. This ability of modern computing and digital technology to look at the intersection of health, biology, behavior, social determinants and even the economic wellbeing of local areas related to that I think it's gonna be the biggest area for us to look at. Right, thanks Rob. Why don't I turn it over now to Daphne and Karen. What you work on is extremely exciting and when I talk to both of you you talk about how this using deep learning and data science on high resolution data can allow us to address a lot of these challenges that Rob was posing. But at the same time there are challenges and what I'd love to get from you both in terms of what do you see as the greatest opportunities and challenges? And then the second piece is I use this example often of the pole vaulting curve and there are large periods of time where the pole vaulting curve didn't really move that much. And then there was suddenly a breakthrough be it in the use of the materials that they were using or a new athlete came along and then they saw a step change. So what do you see as those technologies that are right now poised for this breakthrough that'll allow us to be able to achieve this step change in productivity and innovation? Daphne maybe we start with you and then Karen maybe you can chime in. Great, thank you Shostri. So I think the exciting thing about the moment in time in which we find ourselves is that I think there's opportunities throughout the pharma R&D value chain. I sometimes hear people talking about machine learning and data science as a point technology kind of like X-ray crystallography perhaps that might improve a single step in the pharma R&D value chain. And what I think is really remarkable about this technology is that it can help in so many different ways all the way from a better characterization of human disease where I think the current characterization that we have is based on very coarse grained clinical phenomenology that really doesn't capture a lot of the underlying biological mechanisms. Oncology is one of the areas where we've moved away using our understanding of genetics so much finer grained, more mechanistic understanding of sub types of cancer that are driven by certain molecular mechanisms and that's created an incredible innovation in the kinds of treatments that we can provide to patients. We haven't done that across the board in multiple other diseases that are not oncology and I think that's a tremendous opportunity but that's just the beginning if you will of the pharmaceutical value chain. We've seen tremendous advances in accelerating and de-risking the design of novel therapeutic novel chemical matter. We've seen and also heard from Rob about better clinical trial design whether it's the selection of better clinical sites on the selection of patients that are more likely to respond or in better tracking of those patients so as to determine which of them are actually benefiting in a much more granular and quantitative ways then let's write some stuff down in a notebook that has characterized some of the earlier ways in which I think some clinical trials are still designed and then over time I think there's and we'll hear from some of the other panelists I know about opportunities in manufacturing and supply chains and even in sales all of which I think can be improved by this incredible technology that we have available to us. So I think those are some of the opportunities we can certainly go deeper on all of them but you also asked about the risks and I'm going to highlight two of those that I think are really important. The first and foremost I think is data both the quality of the data and the biases in the data that are often very subtle and easy to miss. In some ways modern day powerful machine learning is very much a two-edged sword in the sense that well while it's really incredibly good at picking up on subtle signal it's equally good at picking up on subtle artifact and that artifact can really make the models highly misleading in ways that are both biologically implausible and relevant or in some cases we are hearing more and more about are biased in the context of particular racial bias or gender bias and again it's really important to try and address that and the only way to really do that is by having higher quality data and this is really the epitome of garbage in garbage out. The other big risk I think in the context of delivering on the promise here is the lack of availability of people. There's already this huge sucking sound from so many industries tech but also more broadly in terms of qualified professionals in AI and machine learning it is becoming increasingly hard to hire people with that skill set and for us in the biopharma industry it's not just any machine learning person who will do we need someone who has the capability to speak across the chasm between the technology world and the life science world and not everyone is interested in putting in the effort that it would take to really understand enough about human biology or chemistry to have a meaningful dialogue with one's counterparts in the other discipline. So I think this is a place where the biopharma industry really needs to make a big effort not only to attract the right kind of people but also to create a culture that can make them be as productive and as impactful as they can be because a lot of times people with that skill set find themselves relegated to a little corner where they're like the tech corner and that doesn't let them really have the impact that they can and also doesn't make it a particularly attractive place for them. So those are to me the big opportunities which are many and also two of the biggest challenges. Karen, I can't hear you Karen, can the others hear her? Looks like I was still on the, sorry about that. Yeah. Can you hear me now? Yes, we can. Yeah, I mean, I think one of the things that we have to say that it's become really obvious during this pandemic is that really the amount of data that's available to us, the area of data that's available to us as Daphne and others said not just in drug discovery but throughout the entire value chain and post market is vast. The area is huge but the perimeter of our ability as individual scientists to analyze that data has really kind of gone beyond what any individual or any team can do. And I think the promise really of data science and that combined with machine learning is and I often rename AI which is currently going through some sort of height curve right now. It's really about augmented intelligence, accelerated insights that help us to make better decisions. So that's really the opportunity that I think is in front of us. I think what we've also spoken about today is a sort of impatience that we all now have especially again, seeing what we've been able to do as an industry over the last couple of years with COVID there's an impatience. Why can't we do more of this accelerated understanding, accelerated hypothesis testing around unmedical need? And so the area that I'm focused on on a daily basis is not technology for technology's sake but technology that allows us to ask more questions, to test more therapeutic hypotheses. As Andy said, I think really we need to be a little bit more unconstrained in taking on some of these technologies to empower us to speed up the process and to change what we have been doing for decades. So the opportunity again, I think is embracing these fundamentally validated approaches again because I think we live in an age where AI seems to be touching everything. What are those validated approaches that really allow us to accurately test hypotheses? And I think the combination of, for example, physics-based methods that are highly accurate combining those with machine learning gives us an opportunity to sample, in this case, chemical space. And I'll just say that when it comes to real world evidence, historically we've been using subsets of chemical space captured in a high throughput screen or in a fixed library. We now have the ability to interrogate in a real world sense all of chemical matter to find the molecule that has the best properties to go ahead and test that therapeutic hypothesis. That's very exciting. I think we're seeing more and more people embrace that. On the challenge side, I think we're seeing the use of these terms AI as a panacea that's sort of gonna solve for some very complex things that frankly we don't really understand. And I think Daphne touched on this, really getting to the fundamentals of understanding biology so that we can go ahead and raise a hypothesis and then go ahead and test it. I mean, those are, we're really at the very beginning of that. So I think that there are many challenges in sort of knowing where these technologies can be applied with a level of accuracy and a level of validation that allows us to move forward with confidence and balancing that with the unfortunate, I think excitement and sort of rush of investment into an area that really is not a panacea. So that's one of the challenges. I think the other challenge is that we don't have all of the substrate information. We don't really understand human biology. We need to carefully apply these technologies in a way that allows us to keep asking questions, keep figuring out if we're on the right path. And in the same way, we need to essentially not sort of be tunnel vision about one approach of doing things but trying a lot of these different approaches. We now have so many modalities to go after. Gene therapy, biologics, CRISPR, mRNA, small molecules, AI and machine learning can be applied across many of these but there are some places where it works and there are some places where fundamentally it does not work. I think we have to own those areas that where this really isn't making a huge impact and find out how to sort of reconfigure our energies that we're not wasting time and sort of own up to the fact that some of this is not really having a meaningful impact. Daphne touched on training. I do think we're in a period where we're redefining what is a team? A team I think traditionally was comprised of people with very similar training, perhaps a team of people who were interested in therapeutic hypotheses working quite separately from people who speak the language of data science and machine learning. I think we're at a point now where team is defined as truly multidisciplinary, touching on physics, math, computation, biological insight, therapeutic and clinical insight, bringing that team of people together at the very beginning of programs, I think has the potential to really unleash our understanding, the types of questions we're asking. And so I would say that one of the challenges we have is in reconfiguring what these teams look like in very traditional environments. I think it's much easier in smaller companies. I think we're getting to the place now and I'm seeing a lot of this obviously amongst my colleagues that we're redefining team in the moment now. Again, partly because of what's gone on with COVID, but also broadly across by a farmer embracing these different disciplines and bringing people together early on. Yeah. So Najat, the question for you is how do you take these bleeding edge capabilities that the panel has been talking about and wire it into a 130-year-old organization like Johnson & Johnson with 10,000 plus researchers in the R&D department? I would maybe frame the question a little bit. How do we transform the discovery and development of medicines by actually advancing the technologies versus accepting the technologies? Because I think the reality of it is anywhere in the ecosystem we're all still figuring out how to do this well. And I think Andy said that well earlier on, really around innovation and you have to accept some of the things that are going to fail and some things that will accept that will succeed. So how do you take those calculated risks? So in order to actually embed this, I would say there's two main components, right? One is really around, everybody likes to see that there are wins. If you have wins at the end of the day, people can fall behind that, come behind that. But what does a win, what does success actually look like in a large pharmaceutical company? It's all about the pipeline.  So I'll talk through a few of the examples there and then I'll get to another aspect, which is the how, which is the people, some really great topics that have been already raised. So as Rob was mentioning, one of the aspects is how do we find the right patients? How do we ensure that they're the right segments that we're focused on? So just a few examples, we're looking at, let's say in oncology, I think Daphne mentioned as we're looking at more precision medicine, genetic mutations, these patients by definition of precision medicine are rare. So we're leveraging imaging. How do we actually use imaging of different modalities, whether it's radiomics or pathology, histopath images, to actually be able to identify which patients have certain mutations and then not just publish a paper on it, but actually require it in trials and to the real world. That requires a complete reconfiguring to Karen's point of how we work. The second aspect's example for COVID-19, I know there was a lot of conversation about the good and the bad. And when we started on the journey with the COVID-19 vaccine program, data science and digital was central to what we did and it's kudos to the need of 15 for doing that, because it's tough to actually take recommendations on, I think Rob mentioned this, not making the trial smaller, but the making the trial better. And for us, better meant, where exactly do we go? I mean, the reality is it takes us time to set up sites. There's regulatory hurdles, but how do we use not just to Karen's point, traditional real world data, but also a lot of social data, right? Like people, like at the end of the different modalities of data sets to be able to predict social compliance and where you might see hotspots of COVID. And that's what we used to really be able to precisely go to the right locations that accelerated the timelines of the trial. But also another point, which is around diversity. We factored in, you know, over indexing on areas of high diversity, African-American, Latinx, but also socioeconomic status makes our trials better. We had a very diverse, more so than we've ever had trial. And the other thing I'll say, I think Rob also just pulling the thread there mentioned about long-term outcomes. How do we measure long-term outcomes of patient sub-population? We do that with registries and post-marketing commitments. What about the patient or participant sessions that we have in our trials? We actually, actually some of the work was done with Shostri State, worked on creating a concept called tokenization where you can actually connect real world data to clinical trial data. So actually being able to better understand the pre and post of a patient's longitude and long journey to then be able to follow through the outcomes of actually participants in our trial. And each of these examples, as I mentioned, they've had significant impact in terms of how we do the story and the development of our programs. And, you know, I have to say, it couldn't be done just by the data science team. And this is going back to the reconfiguration of the team. It had to be done in an integrated fashion, having the data scientists with an equal voice around a clinical development team, take the current workflows, but ensure that data science is not only a voice, but is around the table for the decision-making very early on. For COVID, we started from day one. And for a lot of our other programs, that is what we do systematically. So, Chastri, to your question, it's hard to have that sort of impact because you're in large companies and it's debt by pilots. You want to make sure it's systematic and you're changing everything from the SOPs to the team structure. And then from the top down, right? When we have development committee, we actually look at what's the data science strategy. So it comes from many different angles of how do we impact change. The one other thing I do want to say that is really important is a piece around talent and bilingual data science talents. The talent that understands data science and Karen beautifully mentioned a lot of the difference of disciplines, but then also understand clinical development, understand chemistry and biology. That overlap in this amount of time inherently is hard to find because it's in the space. So how do we develop our right teams? But the reason why, and I loved what Daphne said, the sounding sound of talent going away, the way we keep our talent in the bio-pharma space is by focusing on what is the question we're trying to answer. It's for a program, Parkinson's disease program, or lung cancer, really being able to resonate to the impact that you can have by embedding data scientists in the right way. And maybe just one last comment. I think in general, large companies struggle with the fact that data sets are fragmented across the board. It's not consolidated in one place. How are we actually using the data? Is there the right appropriate distribution in the data set to avoid bias that Daphne thinks is a huge risk that we always have to look for? How do we do that? One of the first things when I did this rule a little over a year ago was I ensured that across just in order to pre-clinical, clinical and wearable data sets are connected, there is one place where all the models are being developed. And you know, this is a conversation that Anne Heatherington and I have a lot in terms of how do we integrate all of these data sets? But that has been really, really important to be able to feel the work we're doing. So there's talent, there is data, and using data to stewardship in the right way. But then also partnerships. We've done a lot of different partnerships because at the end of the day, and this is how we started, technological advancements live outside and inside. And that's true for R&D broadly, not just data science. And we need to keep leveraging that, especially in a space like data science and digital, which is earlier evolving faster. So I think that's what I would say in terms of sort of what's been our journey. It's both having the impact and wins, but then also some of those in nature. Great, I'll keep us moving here. And Craig, you sit at the intersection of the two most important topics over the last 18 months. Therapeutic development and supply chain disruptions. So give us your perspective from your role in supply chain in terms of the role AI played in terms of keeping the service levels up as well as ensuring that we didn't face drug shortages as a society. Yeah, thanks Astri. And I'll try to be succinct in my remarks. I want to leverage something that Daphne mentioned earlier, which I think is very important for everybody to keep in mind. We can't glorify just the notions of artificial intelligence and machine learning by themselves. We have to recognize that these are additional methods that we have because of computing power capability now and algorithmic capability that need to be directed at specific real world problems, whether they're for access development, supply chain or whatever it might be. We certainly saw those problems in abundance during the course of last year. And I would say, as some of my other colleagues have pointed out, these are problems that actually occur chronically all of the time in supply chain and manufacturing in general and specifically last year. But we saw plenty of opportunities last year and plenty of cases where artificial intelligence was able to respond to dramatically different situations and demands quickly. There are many circumstances where the normal transportation methods that we would use downstream of manufacturing became unavailable to us. And we relied on pretty significantly complex and well-trained machine learning to redirect shipments where we needed to be much clearer about where product actually needed to get to and how it might get there. We also saw plenty of opportunity using the same set of tools to make sure that we were able to protect the product for patients as it extended around the world. This became very true for products that required cold chain shipment where you needed to be very, very clear that you could maintain the temperature conditions all the way to the destination, even when there were disruptions to the regular supply chains. We've also seen plenty of opportunities in the upstream of manufactured portion of the supply chain as well, where image recognition technologies have accelerated the ability to do biological assay analysis and also the ability to just improve productivity for manufacturing where you've got certain circumstances where the demands just far outstrip our normal capabilities. I think finally what I would say just in this space too is beyond artificial intelligence and machine learning, there are other technologies including technologies like blockchain and decentralized databases that have helped us also provide a greater degree of security and surety for product as it goes out into the market and has allowed us to make sure that the product that we promise is the product that is actually delivered. So there are many, many opportunities that we've come to be able to capitalize on as a result of newer and more available technologies now. And as some of my other panelists colleagues have said, those problems and those opportunities are only now more fully available to us than they have been previously. And I'm quite confident that we're going to see a new expansive use of these kinds of technologies to solve problems that otherwise haven't been readily solved for us. Great. Thank you. At this time, why don't I bring in Chris Banco and get the audience question in. Chris, are you on? I am. Saustra, can you hear me in signal? Yep, we can see you.  Guys, this was a really, really great panel and I love the balance of experience that's represented here. Anjad, I thought you spoke really eloquently about something that's near and dear to my heart, which is the collaboration of data science and let's call it the more traditional engine of pharma and how important that is. And I know, for example, for you and Craig, you have senior executive positions in a big pharma where you're trying to drive that kind of collaboration. Daphne, I'm hopeful that as a CEO like me, you just tell people to do it and it happens because you got your own company. But I'm curious particularly to hear from Karen and Rob. You are experienced biomedical executives who work at what are predominantly tech companies. What is it like getting pharma to the table to be a partner with a more traditional technology and data player? And is that something that is happening seamlessly or there in particular frictions or lessons learned about how you blend the mindset of a company that's built on technology and data with a more traditional industry that's still figuring out how to utilize it? Yeah, Chris, great to see you. I can start. I would say that a little bit like I mentioned before, I think that when technology is the first and last word in the conversation, it's very hard to focus on what really biopharma needs out of these technologies. And so I'd say that there clearly is some friction, but I think there's also a lot of enthusiasm to sort of shift, if you like, the outcomes that we're all seeing. And I think Najad touched on this. It's really about the outcomes. How can these technologies accelerate what we're all doing? How can these technologies move the needle in terms of, for example, the quality of chemical matter that we're able to find with balanced properties? How can these technologies accelerate our insights? And I think when you focus on these outcome-based discussions, testing therapeutic hypotheses, coming up with molecules that everyone can get behind, I think that a lot of that friction goes away. I think what I've certainly seen since joining Schrodinger, where I'm now based, having come from a large farmer, is that there's an excitement about the idea that we can work together with these technologies in companies where, frankly, they have a lot of time to dedicate to sort of fixing the kinks, embracing the cloud, working through all of the sort of bathroom operational stuff that frankly I don't think is necessarily the bread and butter of a large farmer company. That's all being done sort of by the technology company. But I think where the conversation really takes off is around this idea of jointly reaching outcomes that I think we're all striving for, which is testing more of these therapeutic opportunities and accelerating our way to molecules that allow us to do that no matter what the modality. So I do think there are challenges still because at some level, and I think others have said it, at some level you're speaking very different languages, but there is a common language and that common language is about how do we get to these medicines faster? Well, Karen is an optimist and I tend to gravitate towards the problems. I'm actually an optimist, but I do gravitate towards the problems because I think they're more fun to try to solve. And I would just say there's no lack of enthusiasm at the higher levels because as evidenced by this meeting, everyone's excited about the potential of tech and biomedicine creating something that's new and different in as these different types of entities interface. And I'd say a lot of progress has been made sort of in the backroom areas as Karen referred to. And I'm pretty sure that the massive amount of funding going into early drug development is represented by people like Daphne in her company. It's gonna pay off in the long run. Right now though, I would say the culture of a tech company and the culture of a biomedical company, I don't know of anyone who's actually figured it out. And when it gets down to the actual working project, there are a lot of misfits. And I like when Karen said we're speaking different languages and I was taken aback at first and then I realized it was right. Amy Abernathy is just joining us that barely. And when I asked her what do you think is most important, she listed having a glossary as one of the most important issues in getting a tech company and a biomedical company to work together because of flat iron. She said she learned that people will use a term and they mean entirely different things in the two industries. And there's also the fact that in pure consumer tech, the project manager, the product manager and the lead engineer run the show. The executives like me, I mean, I came out of being FDA commissioner, I arrived at Verily, I might as well have been the food service provider because there was no real recognition that that had much importance in the tech world. And to a large extent, the executives are there to make sure that the food service is there and that the computers work and that sort of thing. Whereas the biomedical industry is much more hierarchical in terms of recognition of experience and perhaps the wisdom that comes from it. So we've got to work through these issues. I'm sure we'll get there. But if you look at the intersection of big tech and biomedicine right now, the turnover rate of the biomedical people is really high because the cultures are quite matching. And then I'll just mention in closing, one real big success, Najat referred to and we were partially involved in this massive effort to identify, in the setting of a pandemic, how do you find the right site at the right time where the pandemic is headed? And that wasn't all Google, but we participated along with the J&J data scientist and MIT data scientist. And I think it really works. So that was a good example. If I can just build on that really quickly, I think Rob is right. It's a journey and Karen, sort of beautifully, very different languages. But the minute, when we started, for instance, the example that Rob mentioned, the clinical team versus the operations team versus the data science team spoke completely different languages. But I think it is once you go through the storming and then hopefully there's a little bit of norming to see what are the outcomes. Because how do you put trust in something that has never been used before? How has this been found? These are all of the questions that come through. But once you actually get one of those examples, it really does create a level of respect for the different disciplines. And I think we're not fully there yet. But I can't think of anything other than some of the interesting examples that actually create that. And then being able, and then just one comment I wanna make, that a lot of the times in large pharma companies, when one area recognizes that something was done in a different way to make the trial better, there's a lot of pull from other aspects and other areas too. So this is true human behavior anywhere. And definitely good to leverage that in this state. Fantastic. I think we're on time here and we're encroaching on Andy's closing remarks clock. But thank you so much to the panel for your time and your insights. It was really excellent to talk to all of you. I look forward to meeting together in person next year. We're probably all at somewhere near the peak of the hype curve around AI and machine learning. So it'll be good to get together next year in person and compare notes, both in terms of the new successes as well as in terms of the next set of challenges that we've all uncovered. With that, why don't I turn it over to Andy for the closing session. Thank you very much, Sastry. And thanks to all of the panelists. We'll hope to see you back next year as we continue along this amazing journey of integration of tech and healthcare. So before I bring this 15th annual summit to a close, let's look at the polling results from the last question, please. We can get them up on screen or maybe not. All right. Well, I'll start with my profound closing comments and then at some point in the middle. They are there, Andy, it's there. Oh, I don't see them actually. Are they up on the screen, Karuna? I don't have time. Maybe you can bring the other screen. Oh, there we go. Oh, there we go. Good, good, good. Okay, so the question was, what is the biggest barrier to widespread use of real-world evidence in biopharma? And we had semi-convincing, though probably not statistically significant, playing through the data science panel answer, which was the interoperability of data collection systems. And I guess in a relatively close second was lack of data standards. So thank you all for participating in these polls. I think Karuna will be published in BioCentury if I'm correct. Yeah, they are going to write it and plus we are going to put on the website also. Good, good. So you'll have a chance to make an impact beyond the setting here.


If you're looking to master a new soccer skill, in this video, I'll show you how to do an Elastico. In this video, I'm gonna show you how to do an Elastico, a Flip Flap, a Ronaldinho, whatever you call it. There's a few different names for it, but I wanna talk about how to do the skill and more importantly, when to actually do this type of skill. Because as I always talk about in my skill videos and I have to start out saying this, don't get caught up in flashy skills that look good in practice or look good by yourself, but they don't work in a game. This skill has a bit of flash, but it can also be effective. So if you master it in a match, you can do it in the right situations. So let's just talk about the basic version. I think it would be an outside touch to an inside touch. So there's two touches here, but to make it look really good and what you want is you're trying to turn those two touches into one touch. So I'm pushing the bowl away with the outside. When I get to full extension, I'm trying to get it with the inside. I'm trying to slap it on the other side of the bowl with the inside. I'm trying to shift him that way, and as he goes that way, my touch goes the other way. Okay, so that would be the first version. And again, a couple of things to help you do this more effectively. So I'm trying to push it away and bring it back, the bend in the knees. If I'm doing this, standing up like this, it's difficult for me to get back around the ball and bring it to where I want to go in the opposite direction. If I get my body more on top of it, I push it, and then I can cut it. I'm on top of it. I'm gonna have a better change of direction, and I'm gonna be able to get away from the defender easier. So just focus on getting on top of the ball more rather than up here and start slow, start slow. But the better you get, the quicker you wanna make it. But a good place to start is just kinda tapping the ball back and forth, and you'll make mistakes, come back. But building comfort is just kinda on one foot inside, outside. And then over time, you're gonna have that a little quicker. So as I said in the beginning, there's a couple different ways you can do this. I believe this is elastical. I don't know if this is the same, but it's the same type of touch. It's a double touch. You're turning two touches into one. So I'll put this in the same video. This one, just like inside, outside, or outside, inside. It's the same outside, inside, outside. Okay, but again, this is better if I'm on top of it. Because even if I do have a bad touch, it doesn't work out how I want, just like that. At least I'm on top of the ball. If I'm up here with my body positioning, and I lose it, I can't get there quick enough. So with all your skills, with all your dribbling, just play a bit lower and look where that's a bad touch. Look where my body's ending up though. I'm out here outside of the ball. I'm not on top of it like this, okay? So is there another one? I mean, realistically, we used to mess around with this. This is a trap, guys. You can get caught in doing all these fancy skills. They're fun to practice. But in the game, do they work? It's like a triple, triple elastical. One, two, three. So you're doing, you're combining those two into one. One, two, three. But again, if you're gonna do this, don't just do it on the spot, because that does nothing. He's gonna just stand there. Even though you did a good bit of skill, he's just gonna stand there and tackle you. You want a little more range on your touches to get him moving. But again, you gotta have a lot of confidence and good technique to actually do that in the game. Okay, so keep those in mind. Keep the technique in mind. Let's talk about where to actually use this skill. So, personally, I would not use this skill when you're running at someone. If you're in the midfield, first of all, be smart where you use your skills. Yes, it's good that you can beat players all over the pitch, but it depends on the type of skill you use. For me, this is more of a standstill skill. I'm not gonna dribble at him with speed and try to do that touch. I mean, it's possible, but it's difficult. You're much more likely to beat someone if you're using your speed to your advantage. Where I believe, and I'll try it one more time. It's possible, but it's tough to do. Where I believe this is more valuable is when you receive the ball and maybe you're in the corner and now suddenly I'm at a standstill. I'm in this position here and I have to shift him because when we're both standing still, it's easy for him to defend me. If I'm running with speed and he's standing still, I have the advantage, but if we're both standing still, I have to do something to shift him. So that's where a move like that could be beneficial. So I would use it more in this situation, like a standstill we're here, rather than running at speed. Okay guys, but a couple of different ways to do this skill. And you see, I always like to practice with both feet, but whatever happens, whenever you make a skill, this part is good, but most of us will make this, will perform this good piece of technique and we even shift him. But after that, we don't get away. We're slow in our movement. Remember, technique is one thing, movement is another. So you're slow in your technique.  So after every bit of skill, I want to see quickly accelerate, change position. Even if you have that bad touch, if you have the bad touch and the skill didn't go how you wanted, but you were quick, then you're going to get away from more players. So keep those ideas in mind. This is how I do an Elastico. And I hope you found this video helpful. If you like this content, check out the Soccer Success Planner. You can download it for free. There's a link in the description below. And for advanced training, check out the Online Soccer Academy.