Yes, I have a question to Dr. Dominik. You said that the limited state is full of contradictions. Would you be ready to accept the limited state as a step towards gaining the anarcho-capitalistic society, or would you say that there is another way to achieve anarcho-capitalism? OK, thank you for that question. I think it will just cool us down a bit, because we have this format of the debate, and we're supposed to argue for the opposite positions. But I think that should be stated that we both come from libertarian, although you're not willing to use the word, we both come. So maybe let me change the word. So we both come from the respect for individual rights. And we want to analyze the possible systems, the political systems or social systems that can consistently and to the high extent, I guess, also that would be important consideration, provide protection for these rights. So although there is some disagreement between us, as you can imagine, I guess, this agreement is really insignificant in comparison to how many things we share and how many ideas we share and in how many concepts and the rights and liberties we believe in. So, well, let's say, outside the format of the debate, I would, of course, say that limited government is highly desirable. Let me maybe quote a friend of mine, Walter Block. He has this really funny joke when a person asks an economist, how's your wife? The answer is, compared to what? So how is limited government compared to what? Limited government, of course, compared to what we have currently, is an amazing solution. So I would say, well, I would absolutely welcome that. I would even welcome a limited government as not leading anywhere else. But as far as principles are concerned, I think that the right is on the side of anarcho-capitalist position. The right is on the side of full respect to individual rights. But this is comparatively insignificant in comparison to how many things we share, I guess. So I'm going to disagree. I think it's a big deal. I think the disagreement is a big deal. It's one of the reasons I won't call myself a libertarian. And I think, at the end of the day, we haven't talked about this, but it probably results in the fact that we don't define individual rights correctly the same way. And that we don't apply individual rights the same way. And that they are much deeper philosophical issues at bay where we disagree on. Because I don't think you can come to the anarcho-capitalist position, or the anarchist position, without, I think, making deep philosophical errors. For example, in the role of violence in human life and what individual rights actually mean. I mean, since you mentioned Walter Block, I will ask the question, is it OK? Is it rights-violating to have sex with a five-year-old child? Now, I would argue, absolutely it's rights-violating. It's a horrific rights-violation, and that person should spend a big chunk of their life in jail, if not forever. But they are libertarians. They are anarchists who say, well, why? It's consensual. The child might have said yes, but that's meaningless, right?  Now, what does one do if there is a protection agency? Let's say I'm offering services to pedophiles. I am a right to protection agency that protects the rights of men to have, or women, it doesn't matter, women are taught it, but to have sex with a five-year-old child. What do you do with that? How do we solve this issue? How do we combat it? How do we get rid of this agency? People voluntarily are signing up. People voluntarily are willing to participate. Maybe even parents are voluntarily willing to have their kids participate. It's all voluntary. How do we deal with it? Well, I mean, I've got creepy crawlers down my spine just thinking about that, right?  Now, that means there's no such thing as individual rights. That means the negation of individual rights and the negation of what even means to be voluntary. How can a five-year-old be voluntary or anything? And how can a parent abuse a child and get away with it? So these are complex issues that anarchism defaults on and ultimately has to lead to massive injustices of these individual rights that supposedly we agree upon. But I'm not sure we do agree on them. And I certainly don't think we see them as coming from the same source and applying in the same way. So I think, and this is my resistance to saying we agree on a lot of things, I'm not sure we do if once you start peeling away the rhetoric into what it actually means. Okay, so I have a question to Dr. Brug. So what's your last comment based on this taste? Would you be equally moved? Sorry. What's your last comment based on, I don't know, moral distaste or repugnance? Would you be equally moved? I don't know, if I just come back home now and enjoying my property and all my liberties, I masturbate, watching pornography. I don't know, would you call it unjust or, I don't know, are you against effectively principle of voluntariness? I don't know, does your position imply that if- I mean, your masturbation is not violating anybody's rights.  Yeah, because I'm trying to test your approach, like is it based on distaste, repugnance? So let's assume that- I mean, my point is this, my point is that application of individual rights is complicated. And applying it, for example, to children is very complex. But children have rights. And those rights are held in trust by parents. But somebody has to supervise their parents to make sure they don't abuse that trust. And that, for example, is one of the roles of government, to protect the rights of the child from, often, his own parents. And there is no way to- I just gave that example as a way in which Anarchy cannot deal with that. And indeed, Anarchy would encourage the creation of societies, of pedophiles, with parents who want to abuse their kids to join together. Now, I consider pedophilia a capital offense. It's as close to getting, you know, shot as I can think of.  I mean, I'm actually very much pro-masturbation. I think masturbation is wonderful. You should all do it if you're not doing it and do it often. What does that have to do with molesting children? It doesn't have any relationship to the two. So what about gays then? What about what? What about gays having consensual sex? It's consensual, right? They're adults. They get to make their own decisions. Again, a child is not an adult. And there has to be differentiation between children and adults. Which again, my argument is Anarchy cannot embed. You have to have a government that protects children. Okay, let me just push you out a bit more. There was this anecdote by Walter Block, like just to follow Dr. Mignac's style. Like, he used this anecdote. Like, there was a conference. There were two neo-Nazis in the audience. And he was like peculiarly keen on sort of converting them to libertarianism. He said, you'll have a really good deal with us with libertarians, you Nazis. You would have your goose marches.  Like, what's more, you could get some juice to the gas if they willing, if they volunteer. That's disgusting. That is absolutely disgusting. But that's voluntary. No, you don't, it's not, if you want to commit suicide, that's fine. You cannot volunteer. That's exactly a source to suicide. This is exactly the point where you guys don't understand what individual rights mean. You cannot voluntarily assign yourself into slavery. There is no such thing as a contract. Contract means something. It is a legal document that assigns certain rights and certain responsibilities. You cannot assign yourself into slavery. If you want to commit suicide, you have every right to do that. You can commit suicide. You cannot have a right to contractually assign some Jews to come and get gassed. You can goose march. You can do all the paraphernalia of Nazism. You have every right to do that. You can exclude from your property anybody who is not blue eyed and blonde.  And this is why, I mean, I'm disgusted by Walter Block and why I find him an enemy of liberty. Not a friend of liberty, but an enemy of liberty and on the side of the Nazis and the communists. And that's why I view gang warfare on that side. I don't view that as being consistent with freedom. Contracts mean something. That's not whatever you want. It's not whatever you whim. This is exactly what individual rights are trying to get away from. It's to get away from the idea that whim is primary. Whatever you feel like is primary. By the way, those Nazis, even if all they do is good steps, I'm still gonna call them evil.  And I still think they're evil, morally evil, for holding those ideas. So there is such a thing as... I'm not masturbating, but I'll be moved by now. Because I don't think you masturbating is illegal. Sorry, is immoral. I think you masturbating is moral. I think, I mean, it depends why you masturbate and how you masturbate and all that. Maybe it is immoral. But generally masturbation, I think, is a moral activity. I think that holding certain ideas, holding certain ideas like Nazism, holding certain ideas like Communism is immoral. Do I violate anybody by holding ideas? Morality is not about whether you violate rights or not. Morality is about how you live your life. This is exactly what I argued before. We don't agree on a lot. We disagree fundamentally. Most anarchists are moral subjectivists. Most anarchists don't believe in individual rights. Most anarchists don't believe in an objective morality. We disagree on the most fundamental questions about human engagement. And this is why I don't think, oh, we agree on most stuff, we disagree on just this final outcome. No, for most, and not everybody, there's some anarchists who I think we do agree on a lot of stuff. But for most anarchists, we disagree on a whole slew of philosophical questions. And that disagreement is reflected in the fact that we also disagree in this political issue. If I can jump in for a second. So, well, I think that was quite important what you said, because I agree that this is what distinguishes us, or what divides us. Because libertarianism and, well, anarcho-capitalism as a form of libertarianism is a view that can be called a thin philosophical view. So we don't really try to impose on people any so-called objective morality. You just pick and choose. This time you picked and choose. Masturbation is actually okay. And marching Nazis is not okay. We're not doing this. What we're doing is we try to distribute spheres of freedom between individuals and provide that they don't violate each other's rights, each other's domains, borders of jurisdictions. They can do whatever they want. So we are respectful of freedom. Whereas you've just shown, in my opinion, that there are some limits to freedom for you. The limits being that there is some particular, so-called objective. I can't see why it's objective. It's just particular morality. You pick and choose some things and you call them unethical. I'm not, as a libertarian, as a libertarian, not as a human being, but as a libertarian. We basically are not judgmental. We don't try to impose on people any particular morality. And we see the distinction between right and morality very, very clearly. And to this point, I'm actually referring to the problem with children that we refer to.  But I guess, as a human being, of course we would condemn it. But as libertarians, we would basically point out to the difference, the difference between rights as a protection of interests and different sorts of protections of interest. There are different sorts of protection of interest. There is morality that you mentioned. There is ethics that you mentioned. There is repugnance that was mentioned by one person in the audience. And of course, we would refer to these elements as human beings as protecting children against these kind of abuses. So basically, I can't see the problem here. I think that there is, of course, some disagreement between us about what rights, in this case, we would like to assign to children. I guess there are some rights you wouldn't like to assign to children. For instance, you wouldn't assign a right to a child to sell a real estate. You wouldn't like to assign a right to a child to perform any other legal actions. The only right you would be willing to assign to a child, I guess, would be a right to be protected against physical violence by other people. As far as I'm concerned, I think libertarianism has resources to do exactly the same. Although we wouldn't go further. We wouldn't go into thick theory. We wouldn't tell people what to do if they don't want to. We wouldn't pick and choose this and that as moral or immoral and refer to so-called objective morality. No, this is good, because this is exactly why I do not call myself libertarian. I find much of that repugnant. And the fact is that I don't want to impose my morality, but I do think there is a true morality. I do think there's a right morality. And that morality... What if I disagree?  But don't claim them. Don't claim that we agree on individual rights because we do not. And this is why this idea that libertarians believe in individual rights, we just think this is how we arbitrate them. You don't agree on individual rights. That is, individual rights are not random. Individual rights are a particular moral perspective. Individual rights are a moral concept. They're not even a political concept. They're a moral concept. The concept that bridges morality with politics. So let's be philosophical, right? We want to be philosophical. The whole idea was a moral defense of anarchism. Well, if morality is completely subjective, if morality is whatever you want it to be, then there is no moral defense of anything. It's whatever you happen to agree with. Whatever you feel like. There is no truth. There is no one moral truth, right? It's random. It's whimsical. It's whatever, right? So I believe that there is a moral truth. I believe that some systems are moral and some systems are immoral. But you say, you can't say it's immoral. It depends. Some people might like it. No, there is an objective truth here. So the concept of individual rights and how we come to it and how we apply it and how where it applies depends on having a proper conception of morality. And if we do not have a proper conception of morality, as I think most libertarians do not, then the concept of individual rights is flawed. And therefore their security agencies are all going to be flawed because they're trying to protect the wrong thing. The whole thing, the whole agenda is logically false and logically cannot stand because you won't accept the existence of an objective morality on which to base your moral case for X. Well, just one word if I can because that's really interesting. Well, I said that we share many things. I said that we share belief in individual rights. Yet I agree with you that in some respects, our definitions of individual rights are different. One of this respect, of course, that I identified during my speech is that you think that there is no right not to be expropriated by the state. There's no right, individual right. So what I said and you know that. There's no right with the protection service on a given territory if there is a compulsory monopolistic agency called the state. The second thing is that, well, libertarianism is really respectful of the fact which is usually called moral pluralism that people can reasonably differ as far as the moral doctrines are concerned. We have Christians, we have Catholics, we have Jews, we have various reasonable moral doctrines that differ and because we have this moral pluralism, exactly because of this reason, libertarianism is a preferable way of distributing between people domains of freedom to decide for themselves which moral doctrine they want to follow and not impose on them so-called objective morality that they can disagree with, basically. So one quick comment. Most moral doctrines, most moral doctrines. Most, I agree. Almost all moral doctrines think that using force on other people in order to make right is a good thing. Most moral doctrines do not recognize individual rights. Certainly the Judeo-Christian Islamic moral tradition has no respect for individual rights and believes explicitly that it's okay to use force on other people in order to straighten them out. So I think moral pluralism, great, right? But as long as the morality of individualism is not the dominant morality, then you will not have freedom. And as long as you advocate for moral pluralism, you will never have freedom. Moral pluralism is the way to go down in flames to the status, to the Christian status, the Jewish status, the Islamic status and all, and the secular status to use their moral code in order to justify their rule. It is a way to fail if you believe in liberty. Okay, thank you, Jen. What is the difference between objective and non-objective law and what are the consequences in society of each? That is one of the most important questions today. And objective law is a law which defines objectively what constitutes a crime or what is forbidden and the kind of penalties that a man would incur if he performs the forbidden action. Objective means definable, graspable by irrational consciousness. Therefore, an objective law would be a law which a man can understand and apply so that every man ahead of committing an action would be able to tell what is the crime forbidden, what penalty would he incur if he commits it and can make a decision accordingly. To be a law-abiding citizen, he should be able to understand the law and apply it as guidance to his own social actions. Now, a non-objective law is one which cannot be defined. It means a law without specific definition which may have as many different interpretations as there are men. Under a non-objective law, a citizen cannot tell what is permitted or forbidden. He cannot tell what action is socially accepted, what action will be punished and what will be the nature of the punishment. A non-objective law is left strictly at the interpretation of the authorities. Usually the judges under dictatorships, it would be the commissars, but in any case a non-objective law is one which a man cannot interpret himself, a law that is not defined and is in fact undefinable. The best example of it is, of course, anti-trust legislation, where a man cannot tell actually what is permitted to him or what is forbidden and make a legal crime without knowing that he's doing it. This ran a very popular legal doctrine, holds that law is actually what judges say it is and that legislative enactments are only sources of the law which the judges use to derive what they believe the law is. Do you believe this is a primary cause of the present state of non-objective law? It's not the primary cause, it's one of the manifestations. I believe if I'm not mistaken, it was Justice Holmes, Oliver Wendell Holmes who originated that doctrine. He was the worst philosophical influence on American law. That is a statement of pure non-objectivities. It's the formula for tyranny because if the laws are whatever the judges interpret, I don't see the purpose of having any laws at all. It would simply means that whatever the judges or the authorities decide at any given moment will determine what happens to the citizens of a country. It is not a formulation of law, it's the destruction, the negation of the concept of law.


We're not in Candice any more Dorothy and this is not the Wizard of Oz. I'm out with John from the fish locker who you'll all know. And if you don't know now you know and he's brought me here. Look at the state of this. I don't know if it picks it up on the camera but it is crystal clear like the Bahamas mate. It's stunning. Can we see where we are John? We can't really can we? North coast of Cornwall. We're on the north coast of Cornwall and that's all you're getting. I've got my new fins, new wetsuit top. I look like I forgot my PE kit. Let's get down there and see what we can get. There he goes look. Trepid explorer. Like a billy goat. It's going to be hard of going back up. I'll tell you that much for now. And look at these cliffs and John tells me that these are the old pasty mines of Cornwall. Where they used to mine there. Pure unrefined pasty. Pure unrefined pasty not even cut. So yeah look at the absolute majesty of it. And we're going to head around this little headland so that no one can steal our underpants while we're in water. And I hope you won't get cut off because if you remember what happened last time, roll clip. This look I've sort of got myself stranded down here so we're having to use this fancy pulley system of a rope attached to our stuff. And I'm filming it just in case something goes horrifically wrong. It's working. It's a bit like Murphy's law, apart from it's called the Hayes effect. Come on lad you can do it. Hey Paul you've forgotten something.   We don't want to repeat that. Oh a little shark purse. Bullhoose eggs yeah. Bullhoose eggs. Great spotted cat shark eggs. And what sort of beef have you got with bullhoose? You don't want to see that in my video. Alright well it's a nice little advertisement isn't it? Yeah in another video I mentioned that if you ever get the opportunity to wrestle with like a double figure bullhoose in the water, possibly think twice about it. I'm like bike marks on my thighs where it's completely dominated me. John sent me the footage of him wrestling with a shark. So if you want to see John wrestling with an actual shark in British waters go subscribe to his channel link below. The one thing you don't want to leave behind is your weight belt because it's well heavy. And I left it up there on top of Cliff's. I've just had to run up in wet suit pants mate. Warm to say at least. Warm.  So I'm going to just get a bit zen, take on board some water and then go out and see John in big blue. Big turquoise bike looks a thing.  John has stealthily come up behind me and all I'm going to say is thank God he didn't get me on bingo wing again because look at this. I'd like to say it was an absolute battle book as far as extractions go. Straight forward. For folks that don't know much about this is a male. It's two things that stand out straight away as a male. He's by the size of these claws. I mean his claws are as big as my hands. Look at that. You can tell by the tail. A female of this size or a tail would be that wide.  Look at that man. Big boy. He's of that age that I don't think we can keep him. I think he's too big. We're going to take him and put him back where you got him. We'll get a photo of him when we take him back. Okie dokie. You can see the difference between the claws there. He's got a crunching claw and a cutting claw. If something comes past he'll get rid of it with this one. Crunchy and kill it. I knew it was the serrated one. Look at him. I was worried that John was going to get me on bingo wing. Last time. For those of you who don't believe I'm holding the camera still. Right on bingo wing. It really hurt me. Ow. Did it go through the suit? Well there you go. That's going to scar. That's going to bleed. I'm bleeding. I'm medic. I had recorded that. That really hurt. Right so that's the first. Nipped by a lobster on bingo wing. Hold on. Let me get me composure because I was going to say something there before that happened. Did she just grab me on? Did you do that on purpose? What I'll do is I'll overlay a little photo of the damage that's done now. I'm going to get saddled up. We're going to get him back in sea so he doesn't suffer any more than he needs to get some photos with him. See what else we can pull out. So here I am. Hugging this rock like some sort of underwater Rambo. Admiring the wrasse. I never wanted to shoot it. Just having a look at it with camera.  I can see it in the distance now. I let it go on its side. I take the shot and I've left the safety on. Absolute muppet. Underwater muppet. So that's my bad. And this was the last deep ish dive that I did. Because I was struggling to equalize in one ear and it was wrecking. It felt like my head was going to pop. So I decided not to do any deep diving and took to the shallows to see what I could find. Right I'm out. Tongue is absolutely mullered. Some hot salt. I think we've been in for about 3 hours. It was just lovely to be back in sea. The visibility was changing. I had a bass in my sights. A keeper. You'll have seen that on the film. Went to press trigger. Safety was on. Then it just bolted. Get away! I'm always nervous about bingo wings as you'll know. And then I had a shot at one as I was coming in and I missed. Still. Tell you what, you're honest about it. You're honest. I'm honest, there I am honest.  Unfortunately a pair of us just as well came up to this which was right at the end. My hands were freezing. Yeah. So this is just a... A better sized one. Yeah. Yeah, male. The other one was just too big. This one is about 100 mil. Yeah. What's the two little white dots underneath that say it's a male? That one and that one. Look like it's... And they're not it's nuts are they? Because they don't have nuts do they? Lobster bollocks. I don't know you've got to be careful about what you say don't you? Not on my channel either. All right. Whee. You can say lobster bollock. This is a male lobster. It's a male lobster. I don't know if they have bollocks or not do they? I don't know. We don't... Do we need to have this conversation off camera? As you might have not had the person based off of it. The lobsters and crabs. They love each other very much right now. Arm-a-plated bollocks. This is a male lobster. Arm-a-plated bollocks is what he's saying. I'm real nervous around him because he's been going with me didn't he? So yeah. It's a keeper anyway. Yeah, definitely. We've deemed it not worthy of going all the way back up to the car getting the stuff to make a fire and cooking just that up and sharing it. So we're going to go and hit another mark and see if we can add to the pot. Three cornered leeks. Good eating. The smell. The smell is lovely. And you can tell they're three cornered leeks because guess what? What? Three corners on them. Lovely wild edible and lots of them. Loads of them. There's also some Alexander's they're edible and this is a... It's like a bramble stick but if you marinated that in an oxo cube for overnight I'd say and then put that in a baguette your happy days. We've come to a second location. We've just chewed our way down a cliff brought me weight belt. We don't think we forgot anything this time. Conditions are looking pretty good. You reckon, John? John reckons we might be on to a spider if we're lucky. I'll probably get bit by it but we'll see. The spider crab. Healthy looking creature there. Yeah, we're looking for them twice that size. Right, okay. Happy days. Good sign. So tight goggles were mate. Turn it and goggle on. We've been there for quite a while there mate and it weren't very fishy. There weren't many fish on go visibility weren't other clever. Spider crabs everywhere. We chose a decent size. When we had a couple but they were just too heavy for me to to drag them. Dragging them on my floor it was like having an anchor on back of me mate. Couldn't handle it. Couldn't handle it so we oiled one and we've got one. We've got one for a cook-up to go with our lobster from earlier. I can't even talk man. I'm absolutely knackered. Went right round to another headland over there and then on the way back the tide was against us and they had that big crab on back. He's saying hurry up and come on. He's saying hurry up. He's saying hurry up so I better hurry up right. I'll see you back at fire. We're just going to do this really simply Paul is worried that we're going to get cut off by the tide. I'm not. BTW by the way I'm not. I can climb like a mountain goat but yeah. Smell like a mountain goat. I've got a bit of a fire going. All we're doing is we're going to build up a bed of coals. We're going to cook all of these in a bed of coals. This crab is already on his way back to the sea. We have a spider crab and Mr. Lobster from earlier.  I'm going to wrap them in tin foil with a bit of butter a bit of rock sand fire I'm going to try and steam some of the sea bait and some of these allium. Some of these three-cornered leek. Right happy days and we've foraged this wood and it looks perfectly cut well done. John's battened all that down it's taking him ages. It's just off cuts from the fish locker workshop. Yeah that's good then. Crab pot bottoms, bird boxes all that type stuff all the little projects that me and James make and the last ones we made are made in like a load of shields and swords and a lance and all sorts. We started off being medieval knights and then he became like Megatron Jedi knight and you're all still stuck in stone age. We're transformers of course. And there you go.     Yeah difficult to handle just because the shells are so spiky you know. Most of the meat you'll find inside one of these is inside these hoofing claws in all the legs and inside the knuckles inside of there. Now this is a male you can tell it's a male just because of the sheer size of his claws. The easiest way to dispatch these guys is you simply lift up this apron put the knife in. Lift up and go straight in there. That's it. There he's He's no longer of this earth. Well just look he's just so floppy now. I know you feel. He's out. He's gone. Now there is a second way you can do it but it's hard to get it right and all you do is you hold on to them all at the same time knock it on the edge of a rock and flip his whole back off. Now all I'm going to do now is just separate the back of the apron and then peel the whole back off. Yummy. Quick bit of anatomy that's most of his digestive system in there. These bits on the sides here called dead man's fingers. These are the gills. We'll take those off. Those are the gonads more of the guts.  I'll never advise to eat internal organs because any harmful bacteria or chemicals or anything that this crab has ingested in its life have been scientifically proven to be concentrated in its internal organs. The eat dead rotting matter that's all going to be in there as well. I'll wash all that out. So we'll take these off we'll wash that out and then we'll come back to it. These spider crab ink see there look. I'm tired mate. Sorry for the low energy I'm just absolutely knackered from days of just camping and mucking about ink see. He takes it out of here. All of them I've just washed out all the insides. There's none of the guts left. And that was there where I went in with a knife to kill it. And all you need to do with that is just got a natural seam net just break it to two pieces. That's it completely butchered off. You can keep this if you wanted to wash all the insides out and if you pick the meat put it into there as a bowl but we're not going to use this. Everything that we're discarding everything that we're not going to eat is all going to be returned to the sea and none of it goes to waste because it goes straight back into the ecosystem. All the little crabs and blendies and gobies they'll all pick through it and they'll have a feed as well.  Lobsters biology, lobsters nervous system isn't like, well you consider like a mammalian one being that there's one single point where you can cut it off like we do with the spinal cord. They've got 12 ganglia consider their nervous system like a web like a net over their whole body so unless you're going to cleave it completely in half like Rambo they don't know there's no real way to put it out straight up.  These are all the internal organs this is its intestine line that runs all the way down its back so you'll want to scoop this out because that's basically its poop line. We'll scoop that out I'll scoop out all of this it's inside the head for the same reason as the spider crab. Once I've cleaned it up then we're ready to get wrapped in tinfoil.  Perfect citrusy and oniony garlicky and all that will do when you wrap it up it'll all steam with whatever juices it's got in there I mean your wrapping skills I'm just going to say can you tell that I'll wrap all our Christmas presents all you need to do is you just need to remember which way is up and I try to like to leave a little tiny bit to pick them up with because otherwise if I was smart enough I would have probably brought some gloves with me but considering that I am well you know you're good in other ways aren't you I'm still trying to fathom that out but right there's one that's how we're doing lobsters let's have a go on spider crab what I like so we're going to be eating a feast here it's going to be an absolute culinary delight but what I like to make my food a little bit better is the risk potential risk of drowning so we've got the tide constantly coming in as we're prepping to eat and we'll be eating so we're sort of against the clock aren't we this could be a game show actually you've got to get no it couldn't be a game show see you later imagine foresight down here ah a big brucey ah you've got to get it done before tide comes in lads little bit of that in there sandy greens shall I wash greens just add a little bit of crunch just exactly the same all I've done is I've given the big claws a bit of a knock with a rock put some butter on there a couple of sprigs of rock sand fire and I've put some sea beating there to try and steam that as well and I'm going to try and wrap this up gently without piercing in tin foil which giving a spider crab's anatomy is going to be quite hard we didn't have enough course to cook them both at the same time so we're just elected to do the crab and then the lobsters lobsters are not going to take as long as the crab either but come and have a look at this here this is the money shot steam coming off it smells delicious and this just being steamed wrap that back up and open all up at once it smells amazing oh look at that it does smell amazing though there you go and after a few hours in the seat don't have to fill up an appetite yeah so what are you going to see see on me right those lobsters wow what are you going to eat look at that it's just red hot I'm going to have a bit of this I just need some I've used all the butter I should have brought twice as much butter to go up at top after just a little bit top look at that cheers mate nice look at that oh this waltz waltz he's coming in look at that absolutely delicious I've not cooked an eating spider crab on here before so I want to just tell you what it tastes like look at all that meat there sweet tender buttery perfect battery's going so we're just going to end up we're going to finish this while we polish off these crustaceans and what look off that side's coming as well for this one we'll close it it's been an absolute pleasure as always mate as always go follow John I'll leave all links below if you don't already go and check it out I'll put a link for your second channel as well Fish Locker Workshop downstairs as well go check them both out if you like this sort of stuff and take care thanks for watching Au revoir I'm getting one of these days


Hey, welcome everyone to grand rounds. I'm super excited today to have Michelle rebar visiting. She was planning on being in person but wasn't able to make it and so. But we're really grateful that she's able to to speak to us virtually today. Dr rebar is an informaticist with really deep expertise in ophthalmology she's worked a lot with ophthalmology EHR data with phenotyping from electronic health record data. And she works at the at OHSU and then she's also been a data scholar with the National Eye Institute. The National Eye Institute really is pushing and trying to improve informatics for disease and for ophthalmology. I know her I am part of a kind of a small glaucoma informatics interest group with researchers from across the country. Dr rebar attends that and has been really helpful and really taught me a lot about data standards and using electronic health record data and so just super grateful that you could come and speak to us today. So we've got some online. I think maybe you're able to see the the folks online and then we have a number of people here in our auditorium. So I have a mic that I'll run around for questions if there's if there's time for question or comments at the end. But otherwise will you can start speaking and thank you again for coming. Oh thank you Brian for inviting me and I apologize I really wish I were there in person this would be much more fun. But I'm glad I have the opportunity to talk to you virtually today. So, as Brian mentioned, I do a lot of research in the secondary use of each our data, particularly in the field of ophthalmology and vision science. I'm a data scholar at the National Eye Institute and this is a new program that the NIH has started out of the Office of data science and strategy, and each center can have one new I, or data scholar per year, and, but they can stay one to two years you can have two at a time. I am the inaugural run for the National Eye Institute. So I'm going to be talking to you today about the reuse of each our data for research. And that'll touch a little bit on the work I've been doing at the National Eye Institute, but the, the seminar at noon today will focus a little bit more on that be a little bit more specific. So, to start off with let's see I get this to fans. I just want to mention that I don't have any financial disclosures, and that we're going to be talking sounds like the title, each our data, what it is, and then the challenges for you for reuse and research, but then some opportunities. And really the important thing here, I have to put my informaticist hat on is to say that while the focus is on eye care and vision research, these concepts really apply across medical specialties and a lot of the things that we struggle with in ophthalmology in terms of reuse of our data is common across medicine so just want to keep that in mind. So let's start off talking about each our data. And so health care data are, you know, collected in the HR during the course of clinical care is just huge. It is estimated to be about 30% of the overall huge amount of data worldwide 64.2 zettabytes in 2020. And really the holy grail of all of this data and the reason that, you know, we have the high tech act that really, you know, encouraged each our adoption and we have, you know, very widespread each our adoption now is the promise to learn from all this data you know when you have, you know, clinical data that's collected if it's just in paper charts, kind of gets stuck there it helps that single patient but we don't. It's very difficult to learn from from that and maybe apply what you've learned from some patients to others. And that promise is slowly becoming a reality, not quickly as some of us would like, but I'll talk a little bit about some of the things that we have been able to do today. So this each our data is often called real world data or observational health data that sort of falls into that category, because it's data that's captured outside of a controlled or experimental setting for the most part sometimes people do capture some research study data in each ours for but for the most part it's, it's there for clinical care. But you can also be collecting as part of monitoring so that could be, you know, devices that are monitoring patients in an inpatient setting, but that monitoring also applies if we think about, you know, patient reported or collected data such as like Fitbit monitors or continuous glucose monitoring data. And also, Billings billing and claims data is also considered to be part of this observational health data, and it's really useful for studying patient outcomes during real care and over a much longer extended time. The trials are still the randomized controlled trial is still going to be the gold standard for, you know, really determining the safety and efficacy of a treatment. But this, you know, observational health data research can really be helpful for figuring out what happens over a much longer time than try trial could feasibly cover what happens, you know, how are patients responding to treatments that are different from those trial cohorts we all know that patients that meet inclusion and exclusion criteria within a clinical trial are really, those are optimized to really show the efficacy of a treatment, but most of the patients that you treat are often, you know, quite a bit different from that you know, how do those patients respond over time to medications and how do different treatments compare. And also, it can be very useful for studying and learning from a variety of practice patterns and sometimes so that variety of practice patterns really becomes a natural experiment. So if you're all familiar with the clinical data that's captured in the EHR, and this contains sort of that core data that's, you know, demographic data vitals exam history notes that kind of thing. And also on the left side, some of the data that right now isn't necessarily integrated exactly into the EHR but we still use it potentially during patient care. So in psychology, imaging data is a really important piece of data that you collect about patients that they use for clinical care, but it's not fully integrated into the EHR there's often connections and ways, you know, links you can click to see it but it's not fully part of your EHR. And on the right is data that some of the medication data might actually be considered to be core EHR data but also data that like labs and monitoring data that exists outside of the EHR. But gets pulled in and that you use doing clinical care. But really, the important thing about this is that data that's in that EHR is really optimized and stored in a way that it really helps you access a single data, single patients data very quickly. And this is not necessarily the best way for us to do research. So, when we use EHR data for research, we're actually using data that's coming from the EHR's data mark. And many of you may already know this but I've also is an informaticist not themology realize that some people don't know that the data that you use for research is stored separately from the EHR the data that you use in clinical care. And part of that is because the data that the way the format that of the data in the EHR is really not optimized well for accessing data about multiple patients so overnight. And this is part of your EHR system so you have epic I have epic it we've epic it OHSU is that data gets pushed from the EHR into a relational database called the data mark. And then that data is used for things like reporting for querying data for research. It can actually be used for financial reporting as well. And part of that is because you know it's stored in a better format but the other, you know probably more important piece is that it's kind of, you know, it doesn't slow down the production each our database. We really don't want to be, you know, interfering with clinical care when we're running our queries for research. And, but also to do research using each our data we could be sending that data to other places such as clinical registries. And I'm not entirely sure if you talk contributes to the iris registry or source, but those are ophthalmology specific registries, and this really allows us to access multiple patients data for research. So in ophthalmology research using clinical data in I care and vision has especially machine learning and AI is really focused on imaging. And the reason for this is it avoids some of the challenges of using other clinical data in the HR that we're going to talk about today. And while these models have been using imaging, pretty successfully, there are still some challenges that remain for doing this at a wide scale across the United States or even the world, and being having being able to have very large and diverse data sets and we'll talk a little bit more about that. But we're all familiar with, you know, the first AI system in medicine that gained FDA approval was for one that performs autonomous diabetic but not the screening. And the other addiction models have been felt for AMD glaucoma and RP, and it really has made ophthalmology a leader in artificial intelligence. And in it that's exciting and it's focused on imaging. And the expertise is on using HR data. And this is still much we're kind of lagging a little bit in terms of doing using unstructured and structured data that's in the HR for machine learning and AI. The data or the research is often limited to just a very few and at a single site or very few sites and studies. So we've done things like looking at IOP after anti bed injections, surgery prediction for glaucoma patients, looking at IOP changes after cataract surgery. And, but really much of this research, both imaging and the HR data was performed using data from a single or few sites. And even in imaging, the data sets that are used where you usually took, you know, years to curate in order to train the models. I work closely with colleagues at OHSU that did a lot of the work in the ROP AI models. And, you know, I have seen the amount of work that they've done to curate their data set. And it's a lot. So, I think we'll talk a little bit now about the reuse of each our data for research and what those challenges are. So, many of you are probably familiar with a lot of the challenges with EHR data and a lot of this are challenges not only for research but for clinical care as well. You know, data is missing. Sometimes data is only entered in notes which makes it difficult to extract for research. The data that was entered is unusable it's in a format, not sure what it means. Sometimes data is incorrect and it can be very difficult to figure out if data is incorrect because a lot of the data that we collect during clinical care we don't necessarily have a gold standard for. One that I often struggle with is the fact that data is fragmented among multiple institutions and we'll talk a little bit more about that. Data isn't standardized so it can be very difficult to do research between multiple sites. I had an experience where I was helping some researchers at OHSU in pediatrics work and pull some data in conjunction with UCSF. And UCSF uses Epic just like OHSU does. They sent us their queries that they used to pull together the data set. We pulled that data sent it off to them only to realize that we did not store data exactly the same. So it looked like our data looked like we had something like a third of our data, a third of our patient visits look like they were self pay, which very weird thing to have happened. So even when you use the same EHR at two different sites, the way you implement that EHR can really have big effects on your data and how you can harmonize that data. And then sometimes data is not accessible and data is not representative and that's a big challenge for any healthcare research because we can only see the data from patients who actually, you know, access healthcare and we all know that not everyone can access healthcare in the same way and equally, you know, at every single institution. So there are some very smart people in informatics that have come up with this framework for data readiness for research for EHR data. And it has four components to it. Data quality, data availability, data interoperability and data provenance. And so we're going to talk about those four different aspects of data readiness and how that plays out in ophthalmology today. So first we talk about data quality. Okay, so first we'll talk about EHR data quality. And there's a group that is from Khan Al there's a really sort of, you know, benchmark paper that talked about this, this data quality assessment terminology that has really been widely used. It talks about three different data quality categories. One is conformance on the data devalues adhere to specified standards and formats. Completeness are data values present or do we have a lot of data that's missing. So we'd think that the third C should be correctness and it is in a way, but that relates back to this idea that it's very difficult for us to determine whether something is correct, because it really isn't a gold standard. Even looking at a patient's chart, all you know is what someone observed someone measured in their, their conclusions and you don't necessarily know if that's absolutely correct. We talk about plausibility instead, you know, based on the evidence that's in the chart are the data values believable. So to demonstrate this a bit, I think it helps to look at examples so there is a paper by a bug who from Northwestern University, when he was at Northwestern University, looking at what visual acuties were recorded in the HR. And they looked at almost 300,000 outpatient visits at Northwestern. And the good news is that the completeness of this data is very high 99% of those visits actually had visual acuity measurements. The conformance, however, was a little bit not is wonderful to expect. You know, there were in those 300,000 visits, there were over 5000 unique values in those. So, and when they talked about the unique they talked about sort of each one being exactly the same so you have the ones they expect like 20 over 20, 20 over 10 that might be something. But then, you know, you get ones that have that measurement books.  So because of that, there has to be some sort of processing that makes that data, you know, useful because we know that there's going to be, you know, some data that doesn't look exactly the same but it's still usable. So, what they found was where that 80% of the recorded values were an exact match for what they expected so that's good, but 20% needed processing and that's that 20% that had that huge variety of all those different types of values. And you can see here this was just sort of a snapshot of the most common visual acuties, and you can see that for some of these like no light perception or NLP. You know the exact there's really was a very big exact match, but for counting fingers, there wasn't that needed some processing and that's to be expected since kind of fingers often has. You know some information about is particularly about the distance. Now, when they did this processing. The values on the left are the ones that they processed to that they wanted to get to these kind of standardized forms. You know the important thing that we wonder, and this is something that that I struggle with as we're working and standardizing this data is, are we losing information when we process this and we're forcing it into these really clean categories, or you know, is there something should be paying attention to that plus or minus one after that I guess it really wants me to go on. So, the next example of data quality is possibility and this is with medication list. So, I think many of you know that each our medication lists are notoriously error written.  It's very so bad. Why am I asked all the time about these medications that, you know, were prescribed years ago.  They're often not updated. And when you're looking at a patient's chart you'd have medications that were prescribed elsewhere they're not often recorded. So you can have that sort of fragmented view as well. And over the counter medications are often not recorded in that medication list. I had a PhD student a few years ago who did a review of 150 glaucoma patient visits, looking at specifically glaucoma medications not all medications. And he showed that there was a discrepancy in 36% of those visits in terms of what the medications were that appeared in the notes and what appeared in the medication list for that patient. One of the good news is he was able to develop some NLP methods to reliably extract the medication from the notes. Okay, and so that second concept that we talked about is availability. So, one way I like to think about this is not necessarily the way that a lot of people think about this but as an informaticist. The data that is stored in the HR can affect its availability so structured data that is is collected either through, you know, things like radio buttons, check boxes, dropdowns. That data that day is called structured and it's very easy to access on the back end we really don't have to do very much we can pull it into report a query very easily. And that would be data that would be entered into like a progress note, surgical note something like that, that data, while, you know, we can pull it kind of as a big blob. We can't get information from it very easily that requires a lot of processing sometimes natural language processing to extract what was in that note, and what was meaningful. Sometimes you can, you can do it using some so simple filters but other times, it really needs some really complicated natural language processing, because you have to make sure that what you're finding it's not negated we're not finding it as part of, you know, patient history or family history, those kinds of things. And then you have this thing in the middle. These are sort of these free text fields and an epic if you're using kaleidoscope. Any of those fields where you can type data in those are free text fields so you can wind up with any type of text in there so even if you're entering in an IOP which should be a numeric value, you can wind up with numeric values numeric values plus text or even text. And that can make that kind of difficult and that's the reason why in that visual acuity study, you have those differences with conformance because you know you you don't have it's not forced into this nice format. So that can affect it, all this can affect availability because depending on how and who's accessing your data from your EHR, you can sometimes have you know it can be if you have a data analyst who just wants to extract structured data you won't have really good access to some of this other data. So there's other examples of EHR data availability. And one is in this one I've often run into when I worked with clinicians at KCI Institute at OHSU is that in the end, we're very lucky that all well, I guess lucky in one way that our major healthcare institutions all use Epic. So what that means is they're all set up within Epic care everywhere. So it looks pretty seamless from the clinicians view and they ask access patients record during a visit that they can kind of see all this data that is from, you know, legacy and governance all these other health care systems and within Portland. And then, but they're always surprised when they asked me to pull that data or they I pulled data for them like but where's all this other stuff and they well, you know, I can't really pull that from legacy, because it's not in our institution. We need data use agreements to access that data and sometimes they'll be stupid that data will have to be to identified with me, which can make it difficult to line up with the data at our institution. So this, this can mean that data that's available for research is often incomplete. And there have been some studies that have shown this is study Massachusetts showed that 31% of patients who had at least two hospital visit visits were seen in at least two different hospitals, which is pretty fragmented. And we did even did a study, looking at systemic data in ophthalmology patients. So, we're trying to come up with a cohort of patients looking for risk factors for glaucoma, and we're using cataract patients just trying to get a cohort that we felt, you know, were didn't necessarily have glaucoma. We were looking for risk factors, and we were trying to look at systemic factors such as hypertension, particularly anti hypertension medications. So we started looking in the chart and in reviewing and we came up with some robots so we actually went in and for 134 patients. And we found that roughly half of them had a primary care physician at OHSU. So then we went through for those 134 cataract patients and had a couple different physicians review their charts to figure out, you know, whether or not we thought this patient had hypertension. When we compared the patients that had primary care providers at OHSU versus those that didn't, there was a significant difference in terms of the percentages of patients that had an accurate diagnosis of hypertension in their charts. So this made it difficult for us to even be looking at these systemic risk factors for our patients, knowing that we don't, we aren't necessarily guaranteed to have a complete look at their data. The next one is interoperability challenges. And these challenges affect everything, essentially. So it affects, it involves the ability to connect other data sources, but also to harmonize that data between sources and link them up. And this requires physical access of connecting of connections, standardizing and harmonizing that data so make sure we're talking about the same thing. And, you know, having the permissions to access the data and making sure that that data is linked correctly between patients. And this is really necessary for multiple, multiple institutional studies and large data sets. But I think a lot of you have also run into this even in clinical care. And because we can't, we have interoperability challenges, even with connecting to imaging devices. So right now a lot of you I know are probably not either pulling in PDF reports for imaging or visual fields into your EHR or manually entering elements in there. And, which is a challenge. And we are challenged in particular so there's some national data sets that the NIH has created. N3C stands for the national cohort, COVID cohort collaborative and that was a large national data set with a spun up in 2020 in response to COVID. And it was it's collected data about all patients that were tested and treated for COVID. And also the all of us data set, which is also another NIH initiative that is trying to collect data about a million patients. And right now, ophthalmology specific data is not included in either of those data sets and some other ones as well, because of these interoperability challenges and I'll talk about that in a little bit. But it's people here. Okay. And then finally, we have this EHR data prop not so it's really to understand the origin of data and how it changes over time and how that affects research is that. So for example, if you train an AI model in, you know, using some data that data can change. And patient did it can change over time that's called data drift. So that means you really have to figure out how frequently you need to retrain those models to make sure that those models actually are working for the patients that you're seeing or that you're applying it to. And another example of this is that, you know, data that we know that was entered for billing may not be as accurate as data that is in the stored in the clinical record. So, and this can be a real challenge. We all know that when you enter in codes, you know, that are diagnosis because they're used for billing, they're not necessarily completely accurate or representative of what you're seeing in the clinic. So one example of this that we've found at OHSU was that we were trying to develop a cohort of newly diagnosed amblyopia patients and we found that, you know, in that time period about 4000 had a billing code for amblyopia on their first visit. When we went in and did a manual review of the charts, we found that another 1000 patients really did have amblyopia but they did not have a billing code for it. And this is really because amblyopia is often a secondary diagnosis. And it's not necessary that it wasn't necessary for the provider to enter the amblyopia diagnosis to get full reimbursement so it wasn't entered in. What does this all mean so unfortunately it means that with this poor data readiness, we can have bias data and this can definitely impact the models that we that we develop so as I mentioned select we can have what's called selection bias which only patients who use healthcare are included. And the, you know, the sicker patients have more data, and this can definitely bias our results in our models if we don't handle that data carefully. There's also measurement bias or information bias. And this is really relates to that lack of a gold standard disease. You know, we don't necessarily know exactly what a patient has but, you know, we can look at the different evidence in the chart to try and figure that out. But also disease staging can be subject, subjective, and that also can influence results or models. And then they also have something called confounding bias and this means that a factor influencing the outcome is missing from your analysis your model. And this sometimes happens because relevant data is missing that might be because it's in unstructured data was entered in the note it was an extra structured data, but also the data fragmentation that we talked about and the end result is that we can have bias data models and algorithms. I'll talk a little bit more later about how we can get around this but really one of the things that I want to point out that this also impacts clinical care because if we can't accurately identify patients. And, you know, who has a disease, it doesn't just affect our research affects our ability to use and take advantage of electronic digital data so cannot, you know, effectively trigger practice alerts, we can't integrate those models and workflows. Often can't fully track our patients progress over time. You know, if we think about that imaging data if it's not actually those measurements of the metadata isn't being pulled into the HR day we can't integrate that into visualizations very well. And we can't necessarily fully automate or support care, some care management activities. And we also don't have complete data about patients and those can, in fact, our, our clinical care as well. So, not to sound too discouraging but there are actually opportunities for research and opportunities to sort of get around these factors. Each data has been used for research and many examples across healthcare. This drug surveillance comparative effectiveness research, doing patient level prediction I'm going to be talking a little bit about clinical workflows, and the best results really are when data sets are large and diverse. An example of data of research I want to talk about our audit log time stamps, and this is an area that I have focused on and when I started in it over 10 years ago was really sort of this emerging area of using this data that's automatically generated during clinical care. And this is part of ONC's regulations for each our certification is that each ours have to track who's in the chart and who's in the record for what patient, when and where they are in the chart and what they're doing. Vendors can provide some reporting of this data and this use data kind of aggregating it and trying to show how people are using the EHR and so for example, signal reporting in Epic that some of you may be familiar with is it's, it's a little separate from their regulatory audit log. They're very careful about saying that, but it's, it's doing the same kind of event tracking, looking at where you are in the chart and what you're doing it and aggregating it into these results so you can actually figure out like for example, how much time you're spending documenting or how much time you're working outside of clinic hours that sort of things. And this data in its raw form or in its aggregated form that vendors provide can be used for research and there are some notations in terms of how the vendors implement it but for the most part it's, it's a very robust data source. And one way that we've used this is that we studied EHR use over time so we studied over 600,000 visits with 70 ophthalmic providers over a decade at OHSU from 2006 to 2016. We found that EHR use increased over time. Now I do need to go back in update this to see what's happened since 2016, but it was really interesting how at the start of EHR use, you know, the average amount of time that was used per visit was pretty close about 4.2 minutes. And it sort of hit a peak in about 2014 it sort of double that. And then sort of seems like it's kind of leveling back down to about six minutes per visit. Now it was really interesting when we showed this data to our epic data analyst who had been actually before he became the data analyst at KC he had actually worked in the implementation of epic back in the 2005 and 2006. And he looked at this and he's like, meaningful use worked, which I thought was a really interesting take on this. Most people physicians look at this and say oh man, each are just uses just getting, you know, the requirements for use just become untenable but so it's kind of an interesting way to see how it hers are being used over time. Another study we use this for was to compare the EHR use for providers when they use describing when they didn't use describe. And we found that, you know, as we expected the total amount of EHR time decreased for the provider when they had a scribe didn't have a scribe. And the most of that benefit came from during the visit when they were actually in the exam room with the patient. But in some cases, the, the providers actually spent more time in the chart after the visit.  And you can see that when you looked at each of the seven providers that we studied that it was different some providers actually spent less time afterwards and some spent specifically more time. And we thought this was very interesting. Of course this data tells us nothing about how, how this worked for the provider and their level of satisfaction and some of them may have actually preferred this to have that one on one time with the patient even if they meant that they had to spend a little bit more time cleaning up their notes afterwards. And then finally we did a very interesting study where we use this data to, to create simulation models of clinics that allowed us to test different different scheduling strategies and what happened was we came up with a scheduling template that really decreased patient wait time by about 15%. At a time when our clinic volume was actually increasing. So, and how this works is that we really try and schedule the patients that are going to take a short amount of time early in the day versus the patients that are long having them later. Now, we didn't put the long patients at the very end because we didn't want our clinic to run over, but even just putting those short patients at the very start really decreased patient wait time and the idea is that if you start with the longer patients which a lot of clinics were doing because they didn't want the clinic to overwork, you know to run late is then you can wind up actually creating delays that you just can't recover from. And I am part of a national effort national group of researchers and I believe we have over 100 researchers that use audit log data and about three years ago we came up with some measures that we thought that each are should be measuring with this audit log data that can allow researchers to do better studies of EHR use. And you can see here what they are just totally each our time work outside of work. Time on note and note documentation time of prescriptions inbox, how much teamwork was associated with an order and the sort of aspirational metric for undivided attention. And the reason I mentioned this is that what has happened is over time is that vendors have adopt have adopted these measures, and they have also released these really large data sets of each are some of its audit log data some of its actual data that allows us to study this at scale and multiple sites looking at things of how physicians use each hours and how they compose notes comparison of work patterns. And the point of this is that this audit log data well it's not perfect. It doesn't suffer from some of these things like data quality issues. And when we actually made an effort to say hey these are what the metrics are that you should be providing. It enabled us to actually do these large multi select data set so it's really kind of, I think a good model for us moving forward with other each our data. So we really want large diverse data sets in order to do research, and the really to represent all the diversity that we can study with patients practice patterns and data and allows us to do studies of where diseases and outcomes. And there's a couple different approaches to this we can pull the data in and registries or we can study it in distributed manner. So pulling the data would be looking at atomic data registry such as iris and source. There's some limitations with this did you have to contribute to the registry to use the data and getting the those data use agreements in place can take sometimes years. And pulling and harmonizing this data is really resource intensive and the imaging data isn't included. There are some prospective data sets that have looked at pulling data. And this is all of us data that we I talked about. There's a bridge to I projects, the UK Biobank. And these can really help us if they're prospective so they can involve they can avoid some of the challenges of the EHR data because we can have better data quality. And they do require having some standardization. And that standardization that they are using is what's called OMOP the OMOP common data model. And we are looking at using this data model and this is a group that Brian mentioned through the Odyssey which is observational health data sciences and informatics. This is a international collaboration of thousands of researchers that are making trying to make this observational health data is particularly collected in EHR is useful and sort of networked and so the aspects of it are the common data model, which just sort of identifies and standardizes the naming of the data how it's stored, especially over time and making sure that everything is kind of normalized and so you can, you know, when you're actually referring to IOP in one institution it's the same data value at another institution. And what we're working on is getting that alphalmic data that's in this common data models it's already widely used it's used in all of us it's using in 3C it's used in these bridge to I projects. And we were trying to get the alphalmic data in the EHR to be included in that and that is what we're working on now. And I think due to time I'll skip this but the first thing about this data is that has really changed practice so it during the pandemic Odyssey studies were able to show when there was a concern about the AstraZeneca vaccine, having the Blair, the blood class it was initially pulled from the market Odyssey studies were able to show that it really was safe and they have it returned to market that was an Odyssey study. And there are some new studies coming out. I don't know if they've been released fully yet, but that have really shown that there is a reduced risk of along COVID for vaccinated patients versus unvaccinated patient. So, what we're trying to do at the National Eye Institute and within this Odyssey work that we're doing with OMOP is we really want to create this alphalmic data network where each of the sites that are part of this network have this data that has been standardized to an OMOP standard. We're also looking at ways to integrate that imaging into that standard. And then they can come together and do studies doing multi site studies. And they'll be sort of a data coordinating center in the middle. But the important thing is that the data will stay at the home institution which allows a little bit easier data use agreements. Hopefully we can integrate imaging into that and sharing data or pooling imaging is almost impossible so we're that's another thing we're hoping on. But we'll be sharing code and models between sites and then the results will be returned. And this is interesting to you. I encourage you to join us. Brian can give you information about how to join the work group I can to this. You know you can follow this link here. And we meet on the summer meeting one time a month but normally you meet twice a month. And it's a very welcoming community. And if you're looking for an area of research to get involved with this is great because you just come in as a subject matter expert in your field and then we can think about this data and how do you standardize it best for use in clinical care and research. So EHRs are a rich source for longitudinal real world observational data, and while ophthalmology is leading the way in AI and machine learning using imaging. EHR data is still an emerging research area and this is true across medicine. And so we're really working on these large diverse data sets that can include all these different facets of patient data systemic ophthalmic and imaging data. And I think I'll stop here. So, and I'll stop sharing because I can, or maybe I'll keep sharing and look at you. Great, thank you. This is just a really amazing summary of EHR data and ophthalmology and all the issues with it and also the opportunities with it. And just really some good just teaching and that too about EHR. I think we'll open it up to questions if people have have thoughts. I wanted to lead off with a question. If that's okay. Yeah, your information on the fragmentation like that really striking and I you know I guess fits with with what we'd expect. My question is, is that in some ways it really like that fragmentation is hard to fix when there's so much like that that's just like built into our healthcare system. And in some ways puts us at it at a disadvantage compared to other countries trying to use this use AI or build algorithms. I wonder if you had any thoughts on that like, you know, maybe from a national health system like like England or like the UK, how like how that's coming out of there. If you've seen differences with that and what's been published. Yeah, yeah, you've hit on one of the things that is really a big sticking point. But that I get frustrated with as well and yeah it is true that for countries that have national healthcare systems, you know the data that is collected within that national healthcare system, you know, is, you know, there's a single patient identifier right. I'm not sure if that same patient identifier persists across. You know, for more of the private healthcare institutions are not. I do know in the US there has, you know, been for least as long as I've been informatics people that I'm pushed for that national medical record identifier. But I believe Congress has stated stated pretty unequivocally that Social Security numbers will only ever be the single identifier so I don't think we'll ever have that. So, especially Regent Strieff has done a lot of work in trying to match patients. So, for in that and 3C, the national covert cohort collaborative. They have done some work in trying to link patient data, because in covert a lot of times patients that were tested in one spot got, you know, cared a different spot so they did do some type of, you know, linking that patient data. Also the iris registry does patient linking as well. So, if a patient has received care multiple institutions. I've worked with iris data. So I know exactly how a ride that can go. So it's not perfect, but you know there are attempts at getting around that and that was is in area I would love to see more work in especially from the perspective because if we are going to be doing things like using up some imaging as a way to either detect or monitor systemic conditions. I think we really do need we, you know, a focus on that improvement in that fragmented data, if we're really going to be using that observational health data otherwise we're just going to be stuck using perspective data sets which aren't bad. But we'd love to take advantage of that decades of observational health data. Can you hear me. Yeah. Okay, we've got another question here. Oh, she's got her own mic. Can you hear me Michelle. Yes, I can. I lean long I'm a retina researcher here. Thanks to Brian stag doing the years of work to bureaucracy to get us onto the source database. Many of us are just starting to do this kind of research so this was a really great and helpful talk for us. I'm just wondering, you know, you mentioned some examples of the limitations of the HR data, specifically like the systemic data regarding hypertension and then the notes versus the billing diagnosis regarding amblyopia. At what point is a problem so big that we just shouldn't try to ask that question or use that data and then, you know, how did you handle those two specific examples like, you know, are there ways to like say okay I think this is not affecting my, this is a limitation but it's not affecting my ability to answer this research question without with a reasonable level low bias, you know. Yeah, and that's a really great question and luckily we have highly trained statisticians and epidemiologists who can help us out with this. This is not my area of expertise, but I know enough to know that I need to include those people. So one of the things that I didn't touch on this in this talk, I'll talk a little bit more about this at my talk at noon, but the Odyssey community has come up with these wonderful tools that, and they have been developed by epidemiologists that really help figure out, do you have high enough quality data to do this type of study using this particular, you know, data site.  There's also a lot of work that's gone into developing phenotypes and testing them so, you know, we can't necessarily expect to identify all of our, our embryo patients with diagnosis codes for example, what else could go into that phenotype that would allow us to identify that, you know, can we find other evidence in the chart that would lead us to think that that patient has amblyopia and what are, you know, what's the reliability of that evidence and that's something that I'm working on right now I don't have the answers to. But it really is trying to figure out, like you said, which questions and which studies we really can do what data is good enough. And when do we just say, okay, this isn't good enough and let's not try because anything would come out that will come out of the study just we wouldn't rely we wouldn't think the evidence was very reliable. So I think that's probably, that's the way we feel that is appropriate for approaching that. So but yeah, that it is a challenge. At a time does anyone else have a question or anything they'd like to talk about. Well, great. So we're looking forward to hearing more from you, Michelle and really appreciate your presentation. A lot of great teaching points there so thank you. Yeah, I wish I could see all of you but nice to be able to speak with you today.