Okay so we're live now. Hello everyone, welcome to Esmar Conf 2023 and this workshop on screening studies for eligibility in evidence synthesis. My name is Bibiana Vetankur. This workshop is being live streamed to YouTube and has a group of participants also taking part live. A very warm welcome to all of you. If you have any questions for our presenter today, you can ask them by the ES Hackathon Twitter account by commenting on the tweet about this workshop. Also if you registered for the workshop, you can ask your question here on Zoom in the Q&A facility or you can open your mic and ask the presenter directly. You can also comment in the chat and chat with other participants on our dedicated Slack channel. They should have been sent along with your registration information as well. We will endeavour to answer all your questions as soon as possible. Also we would like to take time to draw your attention to the code of conduct. This is available at the Esmar Conf website at www.esmarconf.org. Our workshop presenter today is Jacqui Eales from the European Center for Environment and Human Health at the University of Exeter. She's also a research scientist at the Earthwatch Institute. Jacqui, over to you. Thanks Bibiana and a very warm welcome to everyone and I hope you enjoyed this workshop. So this is a workshop which is really aimed at people who don't necessarily have a full knowledge of the systematic review process, perhaps just starting a review relatively new or perhaps you just like to know more about the process itself. It can also be useful for people who have done this before and would like some tips. As we go along I'll be hopefully answering some of your questions and at this point it's quite a good time to say that please do interrupt me with questions. There are lots of options as Bibiana just said to ask those questions, lots of methods but for those of you in the meeting today please do feel free to unmute to put up your hands whichever you prefer to do and just to really interact with this session because I think it's a really good opportunity to ask those pressing questions. Now so as many of you know systematic processes for deciding which studies to include is a key stage in any evidence synthesis and this workshop's really going to present the key principles of screening which are to be transparent and objective with using some worked examples, some exercises and as I said lots of opportunities for those questions. So the importance of transparency we're going to cover that we'll also go through how to design eligibility criteria so what should you be included including in your eligibility criteria what should you not. Any of the common pitfalls that many people fall into and how we can avoid these and also what tools, softwares and new developments there might have been in the recent years. There are quite a few of these available to aid the screening process and these are just becoming more and more. A quick look on SR Toolbox will show you that there are plenty of softwares which are being developed all of which have their pros and cons and many of us have our favorites and are not so favorites so we'll be looking at some of those a bit later as well. And also this is a good opportunity for any of you who are working on some tools perhaps using maybe using text mining to help aid the screening process. It's an opportunity for you to talk about what you're doing as well in your work so please do feel free to unmute and pop things in the chats. But really I do want to make sure that you have covered the key questions which you have burning so what if I can't make a decision on a particular study I can't find a full text or how detailed should you go with eligibility criteria. Now these are some of the key questions which often come up when doing screening for systematic reviews and other evidence synthesis and we'd like to be able to cover those as we go along so please do shout out. So in the process of doing an evidence synthesis article screening or eligibility screening or study inclusion they often get called the same things is one of the most time consuming processes in conducting your evidence synthesis. So in some cases there can be tens of thousands of search results which need to be screened at the title or title and abstract stage leaving several thousands of abstracts to be screened and this can lead you to hundreds of full texts which need to be found retrieved read assessed and then carried through to the next stage. But we know that this stage this quick screening stage is really vital to ensure that all of the relevant evidence that we can find is included and that the really careful carefully curated balance of sensitivity and specificity in the search which was designed doesn't go to waste. So during this session we're going to discuss some techniques that might help us save time during screening as well. So the screening process now we have our search results which we've carefully curated and put into perhaps one library of references perhaps if you've got a lot of great literature it's not possible to do that you have to put it in manually there might be several different libraries you're having to look at. You've removed duplicates where possible you've checked the web you've checked organizational websites and you've retrieved all the great literature you can. You are now ready to start the screening process I'm perhaps I should say at this point also that it's not necessarily that this screening process is standalone it might be that it bleeds into other parts of the reviews. I'm doing a particular review at the moment where I have I'm doing a lot of snowballing which is where we're looking at the reference lists of existing articles to see if there are any more relevant ones so it's kind of like a snowballing technique or a pile growing technique sometimes and this is going on all the time every time that I'm including a study I need to go and look at the reference list of that study and do forwards and sometimes backwards citation chasing so this screening process can be really iterative and of course if you've done reviews before you might know that screening and testing screening or scoping is part of the developing the protocol stage so once you launch yourself into the full stage of screening it might be that you've already had some experience of doing it because you've already done some some screening in your protocol development. So the question we're trying to answer here is really how do we decide which of our studies are eligible for inclusion in this review well first of all it's quite useful to underline the importance of transparency now this of course carries throughout the whole of a systematic review process or evidence synthesis process and particularly in the screening stage it means that our methods are easy to understand and whilst we're going through that once we're undertaking the methods but also for people who are going back and reading those methods later we can prove that the methods were followed by being detailed and transparent about how they were undertaken. We can go back to particular articles you may have a situation where there might have been a so-called mistake made in screening where so an example is where an author has come to you and said well I know that you're doing this evidence synthesis in this topical area have you seen my paper and you look at the paper and you think oh that's really relevant well have we missed it somehow has it has it been not picked up in the search if it had been picked up in the search it would have gone through to screening and that's where it can be really useful to find out what happened to this particular article why was it screened out and that's where transparency can be really important so keeping really meticulous methods I can't underline that enough due to a painful person experience and it also helps updates to be undertaken updates to your evidence synthesis sometimes that's needed to be done before publication it can also be a lot more efficient by being transparent so you've often got many search results and by keeping proper records that's the only way to keep track of what's going on in your review by using a preset number of eligibility criteria across different stages of screening this can really help us be much more efficient and we'll talk a lot more about what those eligibility criteria might be later on and it also enables the full reporting decisions and these to be openly accessible for future works we try to make these open access particularly when it comes to the full text screening decisions and general recommendations for most of the evidence synthesis guidelines are that decisions should be made at full decisions made at full text should be presented somewhere so that they can be traceable and of course not forgetting the importance of objectivity in any evidence synthesis so this is following the key systematic review principle of trying to be as objective as possible and of course we know that there is a degree of subjectivity that creeps in but it's about being transparent about what you've done so that we can assess whether a review has been done to a different different levels of objectivity so during the screening process this is where we're really trying to minimize the risk of selection bias and also the differences in selection bias between different reviewers and this comes in by having multiple reviewers multiple rounds of consistency checking between reviewers and this is also known as inter-rater reliability as well and on that point where we've got multiple terms yeah I just wanted to put this up here because I find well I certainly found when I started doing systematic reviews systematic maps that there was quite a lot of terminology flying around and I wasn't necessarily sure what each of those meant and it turned out that quite a lot of them were meaning the same exactly the same thing so I just wanted to put this slide up here and kind of go through some of these and this is particularly important perhaps if you're in English isn't your your first language so the screening and relevance assessment this describes the stage that we're talking about the article screening stage sometimes called relevance assessment a reviewer and a screener that's essentially the same person if they're undertaking this process so you can use those you can see those terms used interchangeably and sometimes these terms will be used interchangeably across the method section of one particular paper if you're reading an inhibited synthesis paper so yeah it can't be quite confusing inclusion criteria exclusion criteria relevance criteria eligibility criteria these are all the same thing as well essentially although I would say that inclusion and exclusion criteria are set into two separate you might see them as columns in a table so you might have a list of study study examples or parts or facets of a study which are which would denote a relevant review study and then you might have a list of those which would denote a excluded study a study that's not relevant so that's where those two differ between those and relevance and eligibility criteria and then we've got interator reliability inter reviewer reliability or consistency checking or the level of agreement essentially this is how how consistent each screener is in looking at each of the studies so you might have a test set of say a hundred studies and you'll see which which review is on which studies was their difference in opinion as to whether that study should be included included or excluded and then we've got eligible and relevant these mean the same thing essentially a study that is to be included in the review scoping and testing I've put these as similar or the same things because yeah typically scoping is a testing phase of the review and we often talk about this when we're doing screening with your scoping stage to test out your eligibility criteria and also to train systematic reviewers as well and then in the last row we've got record article and studies and these these are referring to the the documents that you're looking at and your rating but it might also be that within one article there might be multiple studies there might be the case where you have one study which is reported by multiple articles so that's the difference between those two but then we've also got record and record is something that you might have come across if you're doing a lot of searching so this is your search hit whichever whatever that is that you get back from your search strategy you might have nine thousand hits and those each of those hits would be a record so I hope that's that's cleared up some some terminology just a quick pause to see if anyone's got any questions yet great so far there are no questions in the chat no questions on twitter and no questions on slack I'm not sure if the participants may want to open the mic and probably ask you something directly yeah so do feel free to open up your mics any questions we're just getting into the meat at this work now so let's move on to eligibility criteria so at this stage at the screening stage any of your articles the screening should be taken undertaken according to particular eligibility criteria and the aim really of having this set of criteria is that each article should be judged on the basis of a set or predefined set of criteria which should avoid selection bias as much as possible and this is one of the I would say this is one of the major criticisms of traditional reviews is selection bias it's where the authors have or have not sometimes just have not reported how they've decided to include studies in reviews so it's a really important stage to be transparent and repeatable at and for you to be able to make your decisions in an as objective way as possible so we need to treat all articles in the same way it should be repeatable and any articles which are included should be included on the basis of their relevance not on the significance of their results and this particularly comes in when you're reading full texts of papers sometimes it's a good tip to try and avoid reading the results section of any full texts until after inclusion because then we're removing the possibility for subconscious changes in our behaviour and sometimes this isn't possible when you're looking at abstracts and the articles report the results and the conclusions and the abstracts but that's certainly been something that's been talked about so we're assessing each potentially relevant record or article against this set of eligibility criteria and it also helps us to prevent changes in our inclusion decisions over time because sometimes we get what's called decision drift which is where anyone despite your best intentions interprets the eligibility criteria differently as time goes on now there may be points at which the eligibility criteria do need to be altered as time goes on and that sometimes that comes because only after a certain point do you realise that there's a set of studies or there's another type of outcome which is being reported by studies which wasn't necessarily picked up on when you were doing the scoping or the protocol development so hopefully that shouldn't happen because you should have done comprehensive scoping but if it does that change any changes should be recorded to eligibility criteria and how do we define them well we often usually base them on aspects of the review questions key elements so the typical pico question the population intervention comparator and outcome but also I would recommend that you would also base them on things like the study design perhaps the location the geographical location the perhaps the date there might be date limiters as well that you want to but also you do need to make sure that you justify any of these limitations why should it be a particular region or date and that will come through not within the eligibility criteria themselves but in the description in your report so let's have a look at an example of this eligibility criteria this is taken from a review which was undertaken several years ago now but I've tweaked it a little bit so this in this example the systematic review was trying to answer the question whether marine protected areas are where effective tools for sustainable fisheries management so we're really looking at the biodiversity impact of marine protected areas so areas of marine reserves sometimes and this was actually in temperate zones so in in those areas so the study eligibility criteria we have listed the key points about those criteria here so the population of any study should have included or could include any of the marine biota the intervention was a marine protected area in a temperate area now you might actually say that this could the temperate part could actually fall into the population as well or it could just sit on its owner's geographical location that doesn't really matter the comparator they wanted to include less protected or unprotected areas to compare the protected areas with and then the outcome was the a measure of the biodiversity impact so that's here density and abundance measures biomass species richness and also they've also got study design here as well so each study should have been an primary study collecting empirical research um now this is another point um just an interesting point actually because I think there are some differences in how how people perceive what empirical research um is so typically for me it's something where the the um the authors of the study the researchers went and deliberately collected research for their own purposes um but it might also for some people this means a a re-evaluation or a reinterpretation of existing evidence so I've seen this happen in different fields where there's a bit of a difference in interpretation even on things like a study design which I thought were quite um were quite well recognized as the same thing so it's definitely important to make sure that there's consistency and understanding across the whole of your research team so how this is one of the questions that often comes up how detailed should you go with your editability criteria um now the example that we had back here is actually not very detailed at all um it's really basic um and I just wanted to show you that because um we've got an exercise using that in a few slides time um and to give you an idea of how how it is to undertake assessments when you don't necessarily really know um the details of the editability criteria and a short answer to this question I would say as detailed as possible um and sometimes this detail might need to need to be added in as you go along the review so as an example I have um taken a screenshot this morning um because I didn't know whether this was actually quite um difficult to understand how detailed you should go okay so these are six pages of um inclusion criteria for one systematic map that I was undertaking um so you can see that this one's the six pages and it's got a number of notes um and points and different categorizations within it um so you might think this might be overkill but it was really helpful for us and the whole review team when we were doing our screening and I've just taken a screenshot of the first the first page here of that eligibility criteria um so you can see that in terms of the population we were looking at individuals, households, communities living or working in areas uh closed deliveries in Southeast Asia so we had to define what countries were in Southeast Asia so we literally had to go through and decide well there were several different definitions of what Southeast Asia is so economic definition the geographical definition uh we also needed to define what coastal area was so how do you define that some people might see it as you know 100 or 200 meters from the beach maybe that's what an ecologist might uh might determine a certain coastal zone as but actually we were we wanted to be a bit broader than that um so we defined them as areas which are adjacent to and heavily dependent on or impacted by the sea so that could be in economic, sociocultural or ecological terms we also said that each of the studies must clearly state that they're focused on that relevant population so you can start to see here how um how detailed we're getting with these eligibility criteria um and then when it came to the intervention we had a number of different categories or subcategories um so just for those of you who are looking and have more time to look at this um offline this intervention was about um marine management interventions so we had to try and classify those into different types of marine management because again it's something that could be interpreted by different people in different ways so I don't want to scare you but um it's it's more a point to just to say please be as detailed as possible and you're probably not going to realize the level of detail you need to go into when it comes to do this so this is a point where scoping and testing is really important so in terms of how the process works it generally gets undertaken at several different stages so these can be um title abstract and full text we've got those sort of three levels of information that any one study might give us hopefully um and this helps us save time by consolidating the similar activities um and also by screening out at title stage you sometimes are able to remove a large number of irrelevant titles if you are not able to do that by um automated text mining um methods um typically though I would say a lot of the um screening gets undertaken at title and abstract and this again this will sometimes depend on what your tool or your software is that you're using and how it presents um each of the records so if it presents the title and abstract together it's a lot easier to just screen titles and abstracts so just screen the title first and then if you can't decide make a decision from the title then you screen at the abstract level um so often you'll have multiple hopefully you'll have multiple reviewers doing screening that's the most um sort of objective way and the most reliable way of doing screening um you should be recording your reasons for exclusion now I would say people can do this at title and abstract stage but if you've got anything more than a few hundred results then you you're really going to be struggling to do this and um manage your resources at the same time so typically just recording the reasons for exclusion at the full text level is sufficient and it's typically included as an appendix and supplementary information of any published and evidence synthesis so here's another one uh what happens if I can't make a decision so this is very typical um we often get people doing title screening and title and abstract screening and exercises and there's just maybes across the whole lot well maybe or I can't tell yet um so if you're screening at title and or abstract stage um if you have any doubt about inclusion you should be including that review it might if it's potentially relevant even if the information is not provided that's the moment at which you should be inclusive and that's to reduce the risk of eliminating any um non-relevant studies um yeah what happens if I can't make a decision at full text level um if there's not enough information then you may need to go back and contact authors if possible again it's all about being transplanted at this stage okay so we're going to have a go at screening um some titles against some eligibility criteria uh we'll use an example from again this was the a real systematic review but it's kind of slightly condensed a little bit um and that review question how is the level of protection in a marine protected area affect populations of marine living organisms okay so I'm now going to ask those of you who are in the meeting today um to speak up because I want some answers to this question I'm going to ask you when you identify the question components here so the question components this is a keychain question population intervention comparator and outcome so would anybody like to tell me what the population is here it's all the marine living organisms basically yeah absolutely right thanks valentina all the marine living organisms marine biotech as we said before the intervention then protection that's right yeah thanks depra it's the level of protection um in the marine protected areas so what would the comparator be here well the level of protection absolutely yeah thanks valentina in this one it's um it's one of those reviews where there'll be different levels of protection so you'll have won't just be uh no protection it could be different levels and finally what's the outcome can be density can be because it affects population so density live or death or I don't know something like this yeah absolutely right valentina um affecting populations so when we say populations we mean the sort of absolute numbers density and things like that so yeah a little bit easier that one because we've already seen that one before but thank you so much everyone for uh helping um and now let's move on to the exercise I'm going to so here we've got the inclusion criteria and I'm going to put up five or six I think different studies okay um and these are just the titles so what I want you to do for the next few minutes um we'll give you let's say four minutes here um to look at each of these and decide whether you would include it or exclude it based on the title alone so off you go okay um I'm gonna open it up to you guys now so who would like to tell me what your answer was or number one is it include or exclude exclude exclude thanks Peter yeah so why did you exclude that one because Nepal doesn't have a coastline excellent thank you so that's uh number one it's the wrong population for that one so and if you were doing this at full text you would write down no population you could do if you wanted to as a reason at this stage but for title stage it's probably a little bit too much information uh for the next one number two same wrong population but in this case because it's not um marine biota biota is poverty so it's probably human population yeah okay so you've so Valentina you've excluded on the basis that um it's on human populations which is which is a fair assessment now I don't know anyone else has uh have a different opinion on that one yeah I would say maybe but you'd have to go to the full text to see if they've included um outcomes that are of interest beside poverty yeah I think that's what I'd probably say as well Peter that's a really good assessment so although it's only mentioned for reducing poverty here it may have also I mentioned some of the biotic measurements so I would say you're right but the only other thing is that we are looking at temperate marine protected areas and these ones mentioned Costa Rica and Thailand which are tropical so probably on that I would exclude so it is quite useful to actually know what what reasons we're excluding on so thanks both of you excellent um for the next one number three I can go again I see no reason to reject this some marine reserves doesn't say anything about where it is um yeah except yeah I would accept that one as well definitely that looks like a pretty spot on from the title only so we'll take that through to abstract and full text the next one number four well for this one I would probably go to read the abstract because I don't actually understand what is about from the just from the title yeah I work in I work in Africa so I know Zambia is landlocked so I'd reject it on population great absolutely it's a bit like the okay cool yeah thanks thanks Peter um yeah we're probably not the only one you've got that um a little bit wrong there um Valentina um when looking at these sometimes we do need to have a map geographical map up just to make sure that we will um we know what we're doing so yeah excluding from the wrong population but otherwise I would say that's a very um it's a very good point um again not really not really being sure as to whether um whether they've got the right uh outcomes so number five I've worked with this person so I know what this paper is about but uh but um yes you reject this it's not about um marine ecosystems at all it's about terrestrial organic farming systems yeah I would say yeah for me it was rejected too because I mean organic farming and well I don't think that it is the right population yeah it's not it's not a protected area um so yeah excluding on those basises and that's a really good point actually Peter that you bring up um sometimes we do come across studies which are written by colleagues people that we know or even ourselves sometimes if that does happen we tend to say okay um let's uh for those ones we need to get somebody else to do the assessment on it because we don't really tend to review our own work in terms of eligibility and finally number six included I don't see any reason why to exclude this one marine reserve canary island perfect yeah it looks good it looks good on the face of things one thing I would say is about again it's about the temperateness of the canary islands so they are they're a Mediterranean climate um but in this case we're not sure whether we're only told that we have to include temperate marine protected areas so in some in some classifications people group just temperate um and tropical um boreal into like really big groups of geographical zones so if those are the zones that we're talking about then yeah I would say that this is probably in the more temperate zone but if you're classifying them in a much more detailed ways perhaps using the the Copenhagen classification system um these are going to sit separately to I mean the temperate ones and I've done a paper like this before and had to define exactly what the feed the kind of Copenhagen temperate zones were and I needed to make sure that each of I had a map with me every time I was doing my screening so I needed to make sure that the particular location of that study was placed in um the right zone so well I hope you found that exercise interesting a little bit fun I guess um we are now going to do it another time so I'm going to give it another go on a slightly different topic though so here's a systematic review question what are the human well-being impacts of terrestrial protected areas um and let's um let's go forward to the actual exercise how I'll give you a little bit less time maybe a couple of minutes um so yeah take it away here it is about population is humans living in or around a protected area your intervention is that protected area establishment or just presence of it the comparator is a less protected or unprotected area um and your outcome is measures of human well-being and again we're including any empirical research Jackie there is a question on slack why would you do the title and abstract screening separate is it not more efficient to screen it in the same time yeah a really good question um it again it depends on how you're you're um seeing your okay so it depends on two things so first of all how it depends on what the tool is that you're using so some tools um don't allow you to see the title and abstract together most of them do but some of them it's easier just to see the title um and so in that way it can be quicker just to screen screen the titles first um and also particularly when combined with the fact that um in some reviews and because of the way that some searching is undertaken or can only be undertaken for some topics you might know that you're going to get quite a lot of irrelevant records that you can't you're unable to screen out now maybe text mining is going to help us with that a little bit more in the future but if that does seem to be the case that you're looking through and you're actually you're bringing up the abstracts but it's not not helpful I would say then just go to just screening the titles first typically I would tend to do titles and abstracts together because that's the the softwares that I use show them both and I would say it is more efficient to do title abstract together generally I'd say also just try out and see how you get on which which method you prefer for some reviews I have gone and done titles first I quickly looked through the titles in fact there's one review I'm doing at the moment which I'm doing that just give it a few more seconds so people can finish these Jackie so far there are no further questions here on the platforms also I've not seen something on the chat if the participants maybe want to ask something that would be great yeah do feel free to to speak out at any point um let's move forward so who would like to tell me what they gave for number seven um I can go I would include this one you want a really difficult strain product error management seems good yep it's about humans it's about protected areas that's all we can tell for the title so yeah let's include it absolutely number eight I'm pretty sure it's not just Valentino and I understand I'm cold so everybody else who's trying to pitch in but I would include this yeah I did the same include that one not quite sure exactly what it's about but it certainly got relevant elements in there great number nine I would say no so this one because international trading status um is um species protected species but not product in the area so I would say no yeah it doesn't look it doesn't look relevant from first sight however I would probably say I would just include this one because it might be talking about talking about the role of Thailand it might be talking about some protected areas within Thailand or maybe Thailand's um a designation of areas reserves which might be impacting societies uh reptiles and amphibians there might be human well-being aspects in in that as well if it's it's talking about how people are exploiting animals so I would say it's very unlikely to be relevant but if you wanted to be really inclusive you could include it at this stage but I generally agree with you Valentina number 10 somebody else okay looks like it's me again reject this this looks like it's some kind of um paper about the genetics of wild canid species I don't think it's about doesn't seem to have any links to humans or human well-being or even well the reasoning to rather might be a protected area but I think it fails on population and outcome probably yeah I would say the same um we've also got the next one which is number 11 socio-economic impact of the karu national part now I would say that's probably a probably an include we've got national part we've got a protected area we've got socio-economics it's got to be talking about humans so I'd say that was exclude and then the final one number 12 I think we could probably all say that one is going to be a definite exclude that's not in our inclusion criteria so let's move on um I've also got another screening exercise which is looking at the abstracts um but I'm actually going to skip over that right now um and just kind of move forward and talk about um in fact no we'll go we'll go there we'll go there um so if you want to go to this link um I think the the piano might be able to pop it into the chat as well um it's an osf site and it should take you straight to a page where there is a pdf document um which says as mark on 2023 workshops screening if you click on that um it should pop up or it should pop up automatically hopefully so do let me know if you've got any problems accessing that um but again I just want to give you about the same amount of time you've got four abstracts in that document um and if you want to look at that document and trying to assess it's on the basis of the same exclusion inclusion criteria from before so it's about human well-being and protected areas um yeah so just go to that one and I will just put up the the next slide we'll also have the same information so you've got the website there you've got your inclusion criteria and you've got those four studies so those are the titles in fact there are some of the ones that you've already looked at just in the last few minutes um I'm going to give you just a few more minutes again about the same amount of time because you don't actually get you can't really take your time about this you've kind of got to to move forward if you've got a lot of um studies to get through so it's really working out that trade off between how much time you're spending on it and how accurate you're being and of course you're going to get way more efficient as you go along um so I'm just going to give you a couple of minutes to look at this now thanks Bibiana for posting the link I'm not sure if you're able to do that on YouTube as well certainly be there um after the live stream I've also shared it on Slack so I think that we can be reaching more people okay so that wasn't very long but um hopefully you've got a good feel for some of those studies um pretty sure you'll get very used to um skin reading things I certainly do I think even more so since I've been working in systematic reviews right so who would like to tell me what you thought of the abstract for Anita tell include or exclude or not sure I think in general I find it um difficult to know if there's comparators in these um from these abstracts so for Anita I thought it really doesn't sound like there's a comparator it sounds like they're just focusing on describing a single protected area um now I don't know if you want to be conservative and say but you need to check the full text to really understand if that comparator is there or not um so that's what kind of I thought yeah it's about humans that's a protected area and this issue's around human yeah I think human well-being probably um and it seems to be empirical research so I think that's all fine it's just a comparison for that one I apologize it seems that Jackie has some technical issues okay now she's back Jackie you are muted hi everyone sorry about that I just dropped out with my internet it's it's raining here in the UK um for a change so I'm sorry about that um I managed to get to this stage where um we decided that Peter had excluded the uh Anita study sorry included the Anita study and I would say I would also agree with you Peter to um include if unsure um whether you're whether there is a comparator or not okay let's move on to the next one Barney Jackie probably you could share your slides again we are not seeing the abstract thank you so who uh so hopefully you're seeing my screen now we're back online um so who thought Barney was an include I thought it was similar to the previous paper and that they've looked at protected areas and their benefits to people it's just unclear if there's a comparator in their analysis or not yeah I would say the same unclear if there's a comparator does mention under managed protected areas so maybe that could be a comparator not sure if that would fall into our um inclusion criteria but we'd have to have a look thank you Peter uh the next one uh niche man well this is the one which I said let's be um let's be a bit inclusive about this and it turned out as I was almost certain that it wasn't relevant it's not about protected areas um I don't think there will be anything more about that in the full text so I think we can exclude a bit the stage and then the R school by Simon now this one looks this one does look really relevant um it's about a national parks we're assuming that it's protected sometimes some these national parks can just be paper parks but um we're assuming that it's protected again I'm not entirely sure of this comparator so this is it's a good point you've picked up Peter it's typically the the comparator which isn't shown through um the uh in the abstract or the title so yeah definitely if it's not there I would I would include it and just to look at it at full text just in case because we don't want to be overly exclusive so I mean any reflections on that process so far Jackie there is a question Jackie there is one question by Ismaila Salaw not about the exercise precisely but the question is sometimes in Africa we do not have access to some journals because of subscription fees what do we do we include a study we can't have access to what implications does it have for publication bias a really good question um I've actually got a slide about that later on but we'll we'll cover it now um well it's not a slide it's just a prompt for me to answer that question it can be really difficult to get access to a lot of papers um there are a number of methods that can be used um so um you can you can always ask the authors if you can get hold of the authors from the email address you can ask them for that um there are a lot of a lot of work is being published open access these days you will actually find that's that's quite um encouraging and preprints can also be um uh like draft versions can also be made available by by authors too um there's becoming a bit of a backlash against the the the paywall so there's another a number of ways in which people can help as well there's there's a hashtag a twitter hashtag called han has pdf so i think it's hashtag c a n h a z pdf i think that's work um and that can sometimes help identify you by trying to find the pdf of a study um but it is a problem if you've got collaborators working in in different organizations on the same project that can be also really helpful um because you can ask them to to share some of the the work that they're doing if you're working on the same project so that's why international collaboration can be can be really useful and really important um your library might also have a little interlibrary loans section um that can be expensive but um it's one way of obtaining difficult defined um studies as well so i think one of my main reflections on this is that there's a huge variability in what people um determine is relevant and not relevant particularly when your inclusion criteria aren't very well defined so that makes a really good case for trying to define them a bit further but sometimes you just don't know and you have to go to the next level to find out so some key things to remember um if you're uncertain retain the record um it's easier to apply if your inclusion criteria if they're well defined so as i mentioned some of mine have been defined to the nth degree um the other important thing is to avoid vague terminology or at least if you're using terminology to explain exactly what you mean by that so we said by marine management interventions we explicitly um listed and we had some examples of the types of um of marine management interventions we were interested in you could be trialling your eligibility criteria um and that trial should involve everybody who's going to be carrying on carrying on doing that systematic review and screening process and again recording and reporting and the final review is really important um here's a question that also comes up quite a lot so what about how many reviewers do i need to include in my in my evidence synthesis do i actually need multiple reviewers i've only got myself sometimes um if you're working on your own project phd students sometimes don't have access to a lot of people to help them um it can be a problem so one thing that you can do if you're you're struggling for people is to um to sort of work share so there's um you can do this informally with um your colleagues um where you help them out on one with you and you might um and then they can help them out on another one help you out on another one um there's also a there's a task share um kind of forum i suppose or group which you can access there's one which runs through the Cochrane collaboration so i'm not sure if that's relevant for for many people but if you want to go and look for Cochrane collaboration and task share um there's number of other ways you can you can share work in that way um how many of you is doing actually need well i would say minimum of two if you're going to be doing um double screening or uh consistency checking so it's also very useful to have a wide um breadth of experience in your review team when you're doing screening so topic expert is obviously really useful but sometimes a methodology expert um can be useful as well perhaps your topic is spreading across many different themes and in that case you might need somebody for each of the themes um so it really depends on the on the type of project lucky there is a question on slack peter stewart asks any wisdom on free tools for automating the title abstract screening process oh that's a really good question um i will go through tools at the very end um um but i would say automating title abstract selection is a very dangerous process to do if you're going to rely on it completely i would say there is there are definitely some ways in which the tools have been uh developed with this in mind um and so i wouldn't necessarily try and develop or use a tool which completely screens out um all of the the studies that title and abstract level because i would want to go back and check um there's a few which do uh a ranking which i find quite useful um one of those is sis rev and i'll i'll talk about in the next um next uh at the end of the session and there's also um yeah there's a couple of others but i'll get into those later um right okay so we know that we're going to get differences between reviewers so we do need to assess the level of agreement um and you can do this using a consistency test and then if there is this agreement you can refine your definitions and repeat the assessment so you can do this with with a consistency like a test which i'll show you one in the next few slides um but probably one of the best ways to do this is literally to sit down with people in the same room or to get together after a certain number of screening records double screening or triple screening records and um having a discussion about the reasons for differences um and that process should of course be recorded and tested and scoped um put a slide in here for any questions um in case any wins not uh not quite a holding on at the moment um but i want to just cover quickly uh consistency checks so these are really important so you're going to have multiple reviews in your study um and even if you're not having multiple reviewers and you don't have the time that's not recommended but if that's all you can do you should also be consistency checking across the time period that you are um screening and that's to minimize the risk of this decision drift so um it can be also be really fun by doing consistency checks i did a recent review once where um prizes for those who were getting through the screenings and having the lowest consistency um problems with consistency um keeping notes regularly uh discussing we tend to do a 10 percent sample as a very minimum but this um is i would say it's a minimum and i would repeat this across the the process perhaps um two or three times it depends on the scale and the size of your review of course so one of the tests for consistency which is often talked about is the cohen's kappa test um i'm going to flip through this quickly because uh i want to get on and talk about um things like tools and give a chance for people to discuss this um so the cohen's kappa is a measure of this inter-rator or inter-reviewer agreement level um it basically in a nutshell it tries to assess the proportion of agreement after an element of chance has been removed um this is the calculation for it you can find plenty of information about it online and some tools that will help you calculate it as well um so typically this is what you would have if you've got say two reviewers and you can do this um what's a slightly different version of this test with um more than two reviewers so you've got the includes and excludes by both reviewers so where both reviewers agree to include both agree to exclude when agrees to include and exclude and vice versa and at the bottom here you can see there's a um sort of like a table of um interpretation of the kappa value so if your kappa value is in this case it's falling at 0.5649 it falls within the moderate agreement level um now sometimes cohen's kappa can be a little bit misleading um and this is particularly the case when um you have rare categories where you have um you're mostly agreeing on some studies then there were some which were um only a few say like five or ten percent which you disagreed on and in these cases the kappa can underestimate the agreement on that rare category um and this is due to the way that the statistic is calculated um the difference in um quantity and allocation between these different uh boxes that we just saw in the last slide it's typically and sometimes can be considered a overly conservative measure of agreement other ways of measuring this are percentage agreement um on particular how many you agree to include or exclude and there are a number of other alternatives as well there's scott's pie which i haven't used fleece's kappa this is a to be used when you've got multiple reviewers not just two um but you can also use weighted kappas as well what i would say is any of these tools are just indications of agreement levels and you can use them i'd certainly say it's better to use them than not use them but also use it alongside um your own um sort of sensible judgment typically if you are repeating your consistency checks across the the course of your project um you'll be able to get a good feel for how much decision drift there is or how much difference there is between reviewers and you can adjust and talk and um provide more detail on the inclusion criteria according to that jackey there is a question on slack at which stage would you test for inter-rater reliability i can imagine results will be different in title screening compared to full text screening yeah absolutely so let's take title abstract um screening um together because that's typically what's done well you would do them at both stages so you'll do it both at the title and abstract stage and the full text stage that's really important um so yeah you've got to do it at any stage that you're doing um screening um so you might be really confused now what am i going to do i don't know how many reviewers i've got if i do have lots of reviewers i don't know how i'm going to to record all of this or or to overcome this potential selection bias um well just remember that everything you're doing in one of these evidence analyses if you're recording it you're already doing a lot better than any other review type so it's all about transparency i would say just keep rechecking all the way throughout keep taking a random sample that's another really important point is that when you're doing your sampling so say you're going to sample 10 percent of your studies at the very beginning um you need to do that randomly to if you're doing a consistency check between different people because um if not if you're doing it by author for example you're just taking the first chunk that appear in first 10 percent that appear in your list of records um you might be getting a load of them that are undertaken by a few authors um and that can that's not a random sample of your of your full data set um so just running through um the the next part is about how to document what you're doing here so you might be familiar with flowcharts here's one example um and we've also got the example of the roses in the next one just to mention here this is also where articles and studies um delineate so where you might have multiple articles in one study or multiple studies in one article and that's a really important definition to make which will fall out when you do the sort of later bit of screening and data coding and data extraction this is the roses um flowcharts set up so many different ones that you can use this is one which was developed for um in collaboration for environmental evidence mere synthesis um and just a note about full text screening it will need to follow the same principles as the title and title and abstract screening um so that's in terms of doing the consistency checking making sure you're using the inclusion criteria includes an exclusion criteria and fully um but also to mention that sometimes it's you can find that actually when you're doing the screening at full text you're reading the full article this might be the most efficient point at which to um do your coding so that's where you're taking um variables information out of your um each of the studies and recording it perhaps in an excel sheet or something um so just wanted to make sure that people uh are aware that this can be a really efficient way of doing your coding doing it at the same time as screening so we've had this question already about not being able to find find or access a full text um research gate can be another really useful um source of studies um sometimes i would find find that actually i've had a really good response rate from authors by asking them really nicely in emails so now that we typically get email addresses with the publications of papers uh it's a it's an opportunity for you to request information from from authors and this also leads into the kind of the coding and data extraction i've often found typically that there's not enough information in a lot of the studies i'm looking at um and i've sent well structured but um copied and pasted with copied and pasted parts uh emails to authors and i've had a really good response rate so if you're you're asking nicely if you're saying what you're doing it for and also um but then if they're going to help you they would get a citation in um of this of their study that can really help you um so other common pitfalls here um these are quite practical ones so not taking regular breaks when screening so this can lead to decision fatigue where you kind of get a little bit sloppy in terms of your decision making when you're screening it can also really burn you out and make you demotivated on the project particularly when you've got a lot to do um and it also leads to decision drift so um that's one of the common pitfalls do make sure that you kind of struck to your your screening and it's good to do it in chunks but not too big chunks so you'll find out what your own tolerance levels are trying to code in too much detail so this might be when you're just making decisions and so for example at the full text level try not to put too much information in there because it's only going to slow you down and do make sure you get some as well um this is something that often happens and it means that you might need to go back and re-screen some of your studies it's when you miss potentially relevant or potentially interesting studies um and I say potentially or sideline because um these studies might not have been anticipated when you first started doing your review and it might be that you've suddenly realized you've got a whole chunk of studies I've got this happening in a review that I'm doing at the moment I've got a whole chunk of studies that I'm interested in but I don't think they're quite relevant what I've done is I've tagged them so I've said um these studies are included but I've got another column in my excel sheet where I'm saying okay we're tagging these because they are on a particular topic or they are perhaps they're undertaken in a geographical location which isn't relevant to your review but there's a lot of them you might want to include them as a separate group to come back to if you have enough time or if you've not got enough studies in your full review and again that's perfectly fine it's just a case of recording that what else did I want to say about that don't remember uh so yeah not keeping track of your consistency or not keeping track of anything um is a really common pitfall and I found that out to my own detriment so so do try and make things um rather than repetitive and tiring more fun and competitive um again as I said I've done this where I've given um given prizes to people for doing various different screening um minimizing amount of detail to record and tag interesting articles those are some key things so in summary uh about this part of the screening before we move on to the tools we've seen uh that we should be using these really well-defined eligibility criteria for efficacy efficiency transparency and repeatability and we know that there's going to be a degree of um decision difference between reviewers your aim is to reduce this and for any situation in which there might be um difference in decision that the authors should be more inclusive than exclusive and that will account for that difference in decisions we've talked about assessing for repeatability and checking um criteria documenting everything using flow charts um and keeping a detailed record of any record any reasons for exclusion so I just want to finish off by talking about softwares now um so there's lots of different softwares to manage articles I don't know if anyone's looked at the systematic review toolbox um aimed at systematic reviews but relevant to many public instances and you can go on this and set tools that you want to have the stage of review that you want to look at if you look at screening there are tens of um tools that are available so let's start off with Excel and other spreadsheets they can be quite easy to use your many people are already familiar with them so if you've got a multiple reviewer review team then uh it can be um you know it's quite easy because they already know how to use it they can be um they can be clunky though um they're not designed specifically for doing evidence synthesis but generally they're inexpensive they don't cost anything you can also use reference management software so thing like EndNote, Zotero, Mendeley um these can be really useful and I know some of these the searching group like to use uh reference management softwares when it comes to screening as well these also can be quite easy to use once you're used to the system but they can be expensive and it might be that different reviewers on your team prefer to use different reference management software so some are really anti some people who use EndNote are really anti Mendeley for example um so that can cause a bit of friction but then there are review management software so we've got things like Covidence, EpiReviewer, Kadeema, Cessrev, Ryan these are all specifically designed for the purpose of undertaking reviews um however they can be expensive um and that's a very brief overview of what those ones are let's look a little bit more at a couple of these um so I would say that EpiReviewer is one of the ones which has quite a good bit of functionality um it was designed by a group of researchers um in the UK and it does a lot of different things they one of the good things about it is they have a fantastic support and help desk um they are very quick in getting back to you um with any problems you can always access your review so even though you might not be subscribed you can still get back to it um and um it's relatively relatively inexpensive so that's one worth checking out there's also a completely free one this is called Kadeema this was developed by the wider Collaboration for Environmental Evidence Network that's one of the networks who produce guidelines for systematic reviews and environmental uh syntheses um it's also based on the EpiReviewer model but it's slightly tweaked and designed a bit differently that's also one worth checking out particularly for those who um want access to the full kind of uh the full software without having to go just to a limited free version so I would also recommend having a look on systematic review toolbooks I've picked out a few of the ones that I've used personally before here um all of them have got their pros and cons and I'm unable to kind of go through those um with you right now um because there's quite a lot of them but I would say that one of the some of the recent ones I've used are um well cysred I've used is the most recent one that I've used uh it is freely available I don't know if you have to pay for it I don't know if there is a paid version I'm not paid for it if that is the case but it's got good um functionality there's you can use it on your phone it's quite easy to use on your phone um I think there's an app for at least a couple of these I can't remember which cysred's got an app but Ryan does um so it's been quite useful when I've been actually sitting on a train or something I've been able to just go through and use use these tools and screen um quite easily um Ryan has been useful there's a bit of a quirk in the way that it exports some of its um um data I'm not sure if that's been sorted out I know they've had a recent update and revamped so there may have been some changes there um and a lot of these have free trial so co-edents for example that is quite an expensive one but I've used it with um on some commercial reviews that I've done and it's been pretty pretty useful actually um but for example this one for co-edents for a single review it's 240 US dollars per year um Ryan for example one of the cheaper options to go for is four US dollars per month for students um and for professionals it's about eight US dollars per month so there's big variability in the in the price um that's available um what do I want to say other about these but yeah so a lot of them are using now are using either machine learning um or text mining I think mostly machine learning so for example with cysred it uses and I think with Ryan as well it has a ranking system so it uses machine learning to determine based on these studies the titles and abstracts that you're looking at which are the most uh which are most likely to be more relevant and then it pushes them up in the list of review of studies to be assessed so that I find is quite useful um cool and as I've mentioned this is these are becoming more and more available um we've seen that actually by doing using machine learning or building it in two um tools for systematic reviews the cost of there was one study which worked out that the cost of a review could be potentially halved or even less than halved by um using single screening with text mining as compared to double screening so there's a lot of opportunity there and just to also shout out to Martin Westgate GitHub rev tools um this is a package for article screening I do believe it uses some uh text mining or text analysis and there are plenty more talks on the more technical aspects of um text mining used for screening and text mining itself on the Esmalconf website if you want to go and have a look at those and some of them have been presented in previous years so I just that all that remains for me is to say thank you and to to field any questions before we need to sign off so do feel free to ask any questions Jackie there are so far no questions on Slack, Twitter, YouTube, here in the chat okay so there is one question on Slack now one eligibility criterion is language not all studies are published in English in my experience translating documents from German, French, Russian, Spanish, Korean or Chinese to English using Google translate works fine in principle would you recommend collecting and screening non-English documents as well as English documents although this certainly depends on the purpose of one's research yeah I think you've hit the nail on the head there it really does depend on on the research that you're undertaking that topic area if you think that it's likely that there's going to be a large suite of studies written in a particular language I would say yes go for it um but otherwise we typically tend to only search in English and only um only only look for studies in another language if there's a good reason for it and that's perfectly justifiable um in in a in the right of review okay so I think that we are all set and the questions have been answered successfully yes great thanks so much okay thanks thank you so you Jackie so that's it for today's workshop on screening studies for eligibility in evidence synthesis we hoped you enjoyed it as much as we did thanks so much Jackie for this wonderful workshop we learned a lot and see you all soon at the next session


I want to thank each of you for being here today, a very special day, a very special month as we raise the awareness for diabetes, of course, in South Carolina. I'm happy to have with me the director of our South Carolina Department of Health and Environment. Simon, thank you so much. I want to thank Mr. Traxler for being with us. And of course, my colleagues on the council, Councilwoman Herbert and Councilwoman Dr. Brussels. Thank you for being here today. This is special because diabetes in South Carolina, once upon a time, you referred to diabetes as sugar. Now, one of the things I want to say, I want to say very emphatically, sugar ain't sweet. And if you know the numbers, you know the statistics for diabetes in South Carolina is very troubling. As you know, I am a diabetic. And of course, I'm a type 2 diabetic. And I have DHEC with me today. And they're going to talk about some of these statistics. And as I close this conference today, I want to say something very personal about diabetes, the effects of diabetes, and how impactful it has been in the state of South Carolina. Particularly at 29203, it has hit tremendously. And of course, these two persons from DHEC will bring us and say to us some of the glaring statistics that are already, that's already there. So please, Ed, if you would come and say a word shortly after that, Ms. Traxler will come and I will close this press conference. It's great to be here. Thank you all for coming. And especially Councilman McDowell, Councilwoman Herbert, Councilwoman Bussells, thank you very much for allowing us to do this, for highlighting this very important topic that is of such great importance to many people in the city of Columbia and in Richland County. This is American Diabetes Month, so it's a perfect time to highlight information about diabetes.  It's challenging. Now, the good news is there's hope. There's good treatment today for diabetes. It's not like 50 years ago when we called this sugar. Now we can help people live with their diabetes and overcome their diabetes. And although it never goes away, great treatment, as I said, is available so people can live long, full lives, even though they have diabetes. So that's the good news. But even better is when we can prevent it. So at DHAC, our focus is on providing access to care and services, both to prevent diabetes and to help those who have it. Information so that people can make good choices about things like diet and exercise that help prevent diabetes and also help those who have it. And finally, resources, because even if you have the best information in the world, if you don't have the resources to use that information, it doesn't matter. So let me talk to you briefly in some ways that we're working with many partners across the city in the county to do this. Because I can tell you, DHAC does not do this by ourselves, without our community partners, without the providers who support us, without folks like all of you. This doesn't work. So thank you for that. So we're very focused on encouraging people to have a healthy lifestyle. We're the health department, that's what we do, right? But we're very focused on healthy lifestyle.  And also, not using tobacco if you are using quitting and if you aren't using, never starting. Because it turns out tobacco use actually both can make diabetes more likely to come and makes it worse if you have it. So our division of diabetes and heart disease management is working with numerous providers across the county. And in fact, 29203, the zip code where we know there's so much problem, not just with diabetes, but also with some of the effects of diabetes, especially including heart disease and amputations, especially in the lower limbs, which are just devastating for people once they have them. Our community systems teams are providing information through, for example, a women and children program. So that new moms can both work to prevent diabetes for themselves, but also can raise their children on a diet that helps prevent diabetes. Because one of the number one causes of diabetes is childhood obesity. If we can help children have a good diet up front, we can prevent problems the rest of their lives. So that's really important. You may know that our headquarters is right down the street, 2100 Bull Street, the old state hospital building. In front of that building, although they're a little dormant now because of the weather, we have two community gardens, two demonstration gardens. And we're encouraging folks to put community gardens all across our state, providing seeds through our program. We can help them with tools, with training, so that people can grow their own fruits and vegetables. Because it's one thing to give people fruits and vegetables for a season. It's another thing to help them grow fruits and vegetables for themselves for a lifetime. So that's really what we're focused on. Right here in the Midlands, we're focused on access to healthy food, as I said, both in terms of growing your own, but also increasing the number of places that sell healthy food. You know, too many parts of our community don't have access to a grocery store. They may have a dollar store and nothing against dollar stores, but those don't often have healthy food choices. So we're working both to get more healthy food choices in the dollar stores, hopefully working to get more grocery stores, farmers markets, those kinds of things that people can use to have healthy choices. We're also working on providing childcare and transportation to appointments so people can get the care that they need to determine do they have pre-diabetes so they can take steps to prevent it from becoming diabetes. And if they do have diabetes, they can get the treatment they need so they don't get the things like heart disease and eye disease and amputations that can come with it. We're also partnering with the Diabetes Action Council. You are out here on Sumter Street where the pilot bus station is. You don't have to stay there very long and you'll see the diabetes bus go by. We have wrapped the common bus, again, working with the Diabetes Council, Diabetes Action Council. And that bus has diabetes prevention messages, how to manage diabetes, and also points of contact so you can reach out and get more information and get support. Because that bus goes all over the city and is a great traveling messenger. As I said, we're also focused on helping people not use tobacco because that helps their diabetes. So we sponsor the South Carolina Quit Line which offers free counseling and free resources to help people quit whether that's smoking cigarettes or other forms of tobacco or vaping because even the nicotine vaping can cause problems. So if you call 1-800-QUIT-NOW-Q-U-I-T-N-O-W, you can access all those resources and again, it's free, you don't need insurance, you just have to be in South Carolina and you can use that. So finally, in conclusion, we believe that connecting individuals with local organizations across our city and county and with providers who are right here can help reduce the impact of diabetes on our community both by prevention, most important, and also by helping those who have diabetes to live with it well because when our citizens live well, that is our ultimate goal. With that, Councilman, thank you very much and I will give the floor to our public health director, Dr. Dressel. Good afternoon and I want to thank Councilman McDowell, all of the Columbia City Council, particularly those here today and all of the city leaders and community partners that are here today for us for holding this event and shining the light on the impact that diabetes does have not only in our state but around the country. Diabetes is a serious life-threatening disease that affects more than 37 million Americans. This includes more than half a million South Carolinians and that's South Carolina adults, more than 500,000 who have been diagnosed with diabetes. And if you don't have a fact sheet, please get one from our PIO, Laura, in the back. It gives a lot of this information. There's another 120,000 South Carolinians that are estimated to have diabetes but haven't been officially diagnosed. They don't know they haven't. And there's probably another 1.4 million South Carolinians, so almost a quarter of our population that have pre-diabetes. In 2020, there were 1,943 South Carolinians who unfortunately lost their lives from diabetes, making it the eighth leading cause of death in our state. A person is diagnosed with diabetes when their body either doesn't make enough insulin or it doesn't use the insulin that it does have as well as it should to help the body regulate the amount of sugar. So sugar still isn't played, but amount of sugar in the person's bloodstream. Insulin is that hormone that regulates how much sugar is in the blood, that blood sugar level. By helping the body either use or store the sugar called glucose that it gets from the foods we eat. Over time, the body's inability to either make or properly use insulin can lead to serious health problems such as heart disease, vision loss or kidney disease. And diabetes can ultimately, as I noted, result in death. There are three main types of diabetes, type 1, type 2, and gestational diabetes. Type 1 diabetes is thought to be caused by an autoimmune reaction, meaning the body somehow accidentally attacks itself by mistake, usually the cells that make the insulin, and so your body then stops making insulin. It can use insulin properly still, it just doesn't make it. Approximately 5 to 10% of diabetics have type 1 diabetes. Symptoms often develop very quickly. It's usually diagnosed in children or teens or young adults. And people with type 1 diabetes have to take insulin every day in order to survive. Again, their body can use it correctly, but they just can't make it. With type 2 diabetes, the body makes insulin, but then it can't use that insulin well, and therefore it can't keep your blood sugar at normal levels. And this makes up about 90 to 95% of diabetics. Type 2 diabetes develops over many years and usually is diagnosed in adults, though unfortunately we are seeing it develop more in teens and children and young adults. Gestational diabetes develops specifically in pregnant women who have never had diabetes before their pregnancy. And people with gestational diabetes, their babies are at higher risk for health problems and complications. Gestational diabetes often goes away after the baby is born, but it does increase that woman's chances for type 2 diabetes later in life. The baby is also more likely to have obesity as a child or a teen, and to develop type 2 diabetes later in their life. There's also this condition I mentioned called pre-diabetes, which is where a blood person's blood sugar levels are higher than normal, but they're not quite high enough to be diagnosed with type 2 diabetes. But having pre-diabetes diagnosed does raise your risk for type 2 diabetes and for heart disease and stroke. There's no cure for diabetes, although many people can learn to very well manage their diabetes by following the treatments and recommendations of their doctors. Of course, however, you can't properly treat your diabetes if you don't know you have it. So that's why so much of our messaging is focused on not only those lifestyle changes that can help you reduce your risk of developing diabetes, but also making sure that people get tested to know if they have diabetes or pre-diabetes. As I noted earlier, our data suggests that there are 120,000 South Carolinians who have diabetes right this moment, but do not know that they have it. Certain people are at higher risk for developing diabetes or pre-diabetes, including people who are overweight, who are 45 years or older, have a parent, brother, or sister with type 2 diabetes, are physically active less than 3 days per week, and if they ever had gestational diabetes or gave birth to a baby who weighed more than 9 pounds at birth. So we encourage anyone, if you fall into one of these categories, these higher risk categories, or anybody that has any concern about their risk for diabetes to talk to their medical provider. Diabetes and pre-diabetes can be diagnosed through a simple blood test ordered by your doctor's office, your provider's office, and performed by a lab. And while it can be more difficult to prevent type 1 diabetes and gestational diabetes, there are several key things we can all do to reduce our risk of getting pre-diabetes and then type 2 diabetes, again, which is the most common by far. Those include living an active lifestyle. In general, most of us healthy adults should aim for about 30 minutes or more of physical activity, at least 5 days a week, eating healthy foods and maintaining a healthy weight, quitting the use of tobacco products, managing stress, and maintaining a safe and healthy blood pressure. So our approach to publicly messaging the impact of diabetes on South Carolinians really is too pronged, as Dr. Simmer noted. It's that prevention, but then also the diagnosis by people getting tested to know if they have it. By sharing with our friends and family these risk factors for diabetes and how to get tested, we can help our state combat this deadly disease and help our fellow South Carolinians live healthy lives. We encourage anyone with questions again about their diabetes status. You know, do not wait. Visit your healthcare provider. Talk about your concerns. Ask if you should be tested. You can also get more information from our DHEC webpage or the CDC webpage, or other trusted partners such as the American Diabetes Association. I want to again thank the City of Columbia for holding this important event, which coincides with November of this year being the National Diabetes Month. The more we talk about diabetes and its prevalence in our state, you know, our city, our county, our country, the more we can hopefully encourage each other to evaluate our risk factors, get tested, and follow a healthy lifestyle of prevention. Thank you. Dr. Chancellor, thank you very much. Dr. Simmer, thank you very much for imparting upon us the real facts about diabetes. From a personal note, when you talk about diabetes, you always want to put a face on diabetes, of course. One out of six African Americans in South Carolina is affected by diabetes. Amputation in certain parts of our state is extremely high. Why am I so interested in health issues in this city of ours, particularly diabetes? I'm a diabetic. I am a 10-year amputee.  Well, trust me, I am. I say all of that to say it is extremely important. The facts that were just given, it helps and allows folk who find themselves in situations where perhaps a kidney is being destroyed by diabetes or eyesight is being destroyed. Make sure, make sure this is the most important issue for our city, I think. If it's health-related, a healthy city is a safe city, and a safe city is a healthy city.


Now we're going to talk about a guy who used to be a prince, Harry. And Greg won't tell us about the videos we're going to watch. Yeah, this is Hoda interviewing Prince Harry at the Invictus Games and asking him about his situation, about him living in the US, about him wanting to go home. You obviously made a lot of news recently. You came home to the UK. You saw your grandmother. How was that? It was great. It was really nice to see her. Be able to see her in some element of privacy was nice. I hadn't had a chance to go back to the UK for a couple of years, apart from those two times, one for my grandfather's funeral and one for the unveiling of statue of my mum. How did it feel being back? Being with her. Being with her, it was great. It was just so nice to see her. You know, she's on great form. She's always got a great sense of humour with me and I'm just making sure that she's, you know, protected and that the right people are right. All right, Mark, what do you get? He's almost a Ricky Gervais character now. Some element of privacy, and there's what I will call a social status eye block, which came from Chase's idea of somebody who shields their eyes to kind of put... That's exactly it. To put themselves above everybody else. Some element of privacy. And his head moves away from the conflict. So there's clearly this conflict there around the privacy, a reticence on the feelings of being back. And then it was great being with her. Lip retractions also. Right at the end, here's the most important thing, disdain and contempt for the right people around her. If you ever want to see a great image of disdain and contempt, it's at the end of this clip. Chase, what do you got on this one? Yeah, it's a shift to internal dialogue, which is his eyes are moving to indicate that he's rehearsing or going through what he's going to say. And there's hard stress and a mental struggle around the word privacy. You can see it clear as day here, but there's a contempt micro expression on top of that at the privacy topic. And you'll see the face, like the kind of one-sided smile, you'll see that there. And when she says, how did it feel being back with her? We have vocal hesitancy, a shift to internal dialogue, lip compression and retraction, which is when we're withholding opinions or we're kind of holding some information back. There's question repetition. And when he's saying it's so nice to see her, there's a single shoulder shrug, which indicates that we lack, typically, lack confidence in what we're saying. And I thought he said, she's ungrateful. I thought those words came out. You can hear it in the clip. It's crystal clear to me. But when he's saying surrounded by the right people, a strong facial expression of contempt, like Mark was just saying, the appearance of this entire clip is strong emotions around privacy. I don't think the trip went well at all. These are behaviors that are reliable on their own, but when all of them stack up, just like we see here on a single topic, there's a massive increase in the likelihood that someone was being deceptive. And he's hinting that he's concerned that she's surrounded by a bad people who are maybe influencing her perceptions in a way that he would like to change. Great. Yeah, I have a little bit different approach on that whole thing, but we all are about negativity around that right people around her. Let me give you one thing. If Hoda knew what I know or what you know, what the four of us know, this could have been a different interview because she asked him a question that basically says, what does it feel like to be the person, the hairy formerly known as a prince?  He's trying to work through it and she throws him a life ring. And when she throws him that life ring and talks about the grandmother, then his face lights up and he starts to talk. There's contempt for sure when he has that smirk or and there's actual amusement in his face when he says the right people around her. My guess is that's from some other conversation. That's, he's prior to a conversation where somebody said, well, the right people are around her and you're not the right kind of people or something. There's something in there is the reason you see that smirk in his face. But he should know that he's going to internal conversation and that's a deep, well, he's digging into and let him go. We were kind of a very different conversation. That's what I got, Scott, what do you got? All right, I think this answer was prepared. I think they had a time I said, here's what you're gonna ask you because it's lazy and almost monotone and has no emotion whatsoever in it, that first question. And I think it's probably been asked that question about 400 times since he's been back. And at the end of the clip, like you were saying, geez, was it Mark? I don't know if it was Mark or Chase. Okay, talking about the contempt. That's the classic expression of contempt. As one of the seven universal expressions that were nailed down by Paul Ekman, happiness, sadness, anger, fear, surprise, contempt, disgust, those are the big seven. And we're gonna go into a deep dive on those in Las Vegas at our event here in just a few weeks. You obviously made a lot of news recently. You came home to the UK. You saw your grandmother. How was that? It was great. It was really nice to see you. See her in some element of privacy was nice. And I had a chance to go back to the UK for a couple of years, apart from those two times, one for my grandfather's funeral and one for on the very statue of my mom. How did it feel being back? Being with her. Being with her, it was great. It was just so nice to see her. She's on great form. She always got a great sense of humor with me and I'm just making sure that she's protected and got the right people around. Well, you make her laugh. That's what she always says. Did you do it again? Yes, I did both Megan and I had tea with her. So it was really nice to catch up with her. And home for me now is, for the time being, it's in the States. And it feels that way as well. Does it? Yeah. We've been welcomed with open arms and it's got such a great community up in Santa Barbara. So you feel like that's home more for you. Is that weird to say? No, but I'm sure it would become a thing. You agree? Chase, what do you got? We see his eyes going down left again, which is a shift to talking to himself, rehearsing some lines. They probably went through this a lot and just going through these lines and rehearsing it. And when he's saying it, it was nice to catch up with her. I think we see an anger expression flash on the face very briefly, in my opinion. And when he's saying, welcome with open arms, there's a super stress response. It's a big, pacifying things on the legs there. You guys can talk about that. But biggest of all, there's a loss of language fluency. The fluency of his normal speech starts going away. It gets more difficult for him to talk. That happens during deception for some people and just some fidgeting. But it's like she's asking, this is home for you. Weird deviation from his baseline behavior. There's a struggle with saying home for him. And he's saying, it's not hard to say, but it needs to become a thing, which is meaningless. Like the whole ending of that statement is meaningless. So all the avoidant behaviors here suggests that there's a similar situation. The visit was about the opposite of what we're actually hearing. It's the opposite of all the words that we're hearing. That's all I got there. Scott. All right. Well, that first question about making the queen laugh, we see his tongue flapping around in the front part of his teeth, like circle around there. Quite often we see something like that.  And when they do that, usually the head goes down, that chin goes down to the chest. And it's almost like they're getting away with something. So when he says, home for me, for the time being as the states, you can hear his hands rubbing together. So it's starting that, starting an adapter there. And he's rocking back and forth. And that's another form of adapting. And in that sentence, when he says, like you were saying, Chase, now when he says, now there's a micro expression and it could be argued that that's an expression of anger or it could be pain. But I'd lean toward anger, but it looks more like, yeah, I'm gonna go with anger on that one. And just after he says states, his grief muscle engages. So I think you're right, Chase. I don't think this went well for him at all in the United States. And I don't think it went well for him at home either or at his old home either. So he says, we've been welcome with open arms. He runs his hands down both his legs and this is another self soothing behavior. And this one's huge. I mean, it's really big. So I get the feeling he's starting to regret the decision he made about separating himself from the family, the royal family and starting to dawn on him that he's made a big mistake at this point. Mark, what do you got? Yeah, where in my interview, I would dig into that welcome idea because that's a massive cluster of self soothing. I'll take you through it in a moment. Where in my interview though, I wouldn't get to interview him again because there is clearly a massively sore point on this. Why? So he does do the open arms gesture. Very, very good. But then hands come down, slap on there, down the inside of the thighs, down towards the ankle to protect the delicate ankle joint there. We often, we've seen that in other interviews with him when he's talking about subjects which he doesn't really wanna go to, touchy subjects, he'll go for his ankle.  It's a huge stress indicator. I'd be right on top of that and I'd go, so Harry, like tell me a little bit more about this welcome, like explain it to me and I'd get him deep into that. Yeah, I'd never get another interview with him. Greg, what do you got on this one? I'm sure he'll never interview with me but he's a 12 year old to me. The way he moves is not like, what is he, 35? I have no idea how old he is. You know, I don't keep up with the Royals quite as much but I don't know how old he is but this is young boy kind of body language. I mean, this is 12 year old boys at camp kind of body language for me. Now I'll also say he does a little bit of brows up in the center in the beginning we associate that with sorrow but he's also a ginger and he's in the sun. I know the pain, trust me. They're pictures where I look like my eyes are all scrunched up because of it. Light color dies out. So I'll give him an out for that. The for the time being, when he says for the time being to me Scott, that's profound because that's the trigger to everything else that happens. All that adapting, that slamming his legs and that pushing those things down. Now when we talk about adapters, people call things soothing maneuvers, adapters, whatever but what you're in effect doing is making the unknown known, making the uncomfortable comfortable because if you do something all the time and it becomes a habit, it comforts you. If you put people in a cage, those things come out even more. Animals do the same. He had a genuine smile, did you make her laugh? That was good. I think he probably had a great visit with her. I think he's probably feeling some remorse as I look at this that drop and rub and then all of that stuff is 12 year old body language for discomfort. We don't know why, we don't have to know why. And then when he says the whole stuff about welcome, that's, we see a lot of that clusters of body language and he does this lips of disappointment kind of thing and his shoulders go up like so, but what am I to do? I think this kind of sums up what we're seeing from him. You make her laugh, that's what she always says. Did you do it again? Yes, yeah, I did, both Megan and I had tea with her so it was really nice to catch up with her and home for me now is, for the time being, it's in this phase and it feels that way as well. Does it? Yeah, we've been welcomed with open arms and it's got such a great community up in Santa Barbara. So you feel like that's home more for you? Is that weird to say? No, but I'm sure it'll become a thing. What's the best thing about her? She has a sense of humour and her ability to see the humour in so many different things. We have a really special relationship. We talk about things that she can't talk about with anybody else. So that's always a nice piece to it. But I think she's, I think after so many years, you get bored of birthdays. You do? You think she's bored of her 96? She won't be bored of the jubilee, will she? No, I don't think so. She's had a few jubilees now. So everyone's like, everyone is slightly different. But I'm sure she's looking forward to it. Do you think you'll come? I don't know yet. There's lots of things with security issues and everything else. So this is what I'm trying to do. Trying to make it possible that I can get my kids to meet her. All right, Greg, what do you got? Yeah, so I just have a few things. The brow tips are up again. This is a ginger in the sun. That's what it is.  He genuinely is happy in this. You can see his face engaging until he gets to this point about whether he will or will not come back. You can see that. And there's internal dialogue going on there as he figures out what he should say. I'm sure there's probably the party line. There's probably a line his brother or whoever the handlers have told him. Mark, I have a note right here that says not overly sophisticated in his body language and message and could use the work of Mark Bowden. So if you're listening, Harry, and you need a coach, we know one that was really close to you might want to call. But then I see when he's talking about the effort required to get his children to his grandmother, there's absolute disdain there. There's disgust. There's all that lower face, all that disdain around the hurdles he has to go through to get these kids to the to the grandmother. Mark, what do you got? Yeah, I love this one. She can't talk about with anybody else stuff. She can't talk about with anybody else. Listen to that else. There's massive vocal fry on that. It is so different. I'm going to say he's being utterly deceptive about. She talks to him about stuff that she can't talk to anybody else about because of that vocal fry. It is so different. And then he goes, so that's always a nice piece to it. But that means nothing. It's so unspecific. I would dig into that and I would go, well, specifically. I mean, I know you can't talk about your private conversations with Her Majesty, but give us the generalized area that she talks about with you that she cannot talk to anybody else. I would imagine he would be instantly in trouble on that. Too unspecific. He's playing this idea of confidence status against the right people. It's a status play throughout this chase. I managed to get in the word status for you there. But what do you got on this one? Thank you. Always appreciate a little status. These facial expressions from Hoda are bizarre, I think. I've never seen people make those facial expressions before in this interview. But let's talk about Harry. What did she say? Or Harry says, talk about things that we can't talk about with anyone else. Mark, I totally agree with you. I think this whole thing suggests some kind of special access and privilege that no one else that no one else has implies maybe an element of trust above all the other children says. That's always a nice piece to it. Meaningless, hollow language there. But when he's saying I get my kids to meet her right at the very end, there's a little bit of contempt, just kind of contempt on the face. All the behavior that we're seeing here in this little clip is showing a very difficult time, I think, coping with the loss of security and maybe a need to assert a degree of specialness because he's lost some kind of specialness, which I think he feels is now gone from his life. His behavioral response to the topic of security specifically suggests more than just physical protection and may lead to something deeper in terms of security that he feels is lacking in his life. I'm not going to go to pop psychology. I already did. So Scott, what do you got? I think you nailed it, though, man. He's he's he's just struggling for relevance at this point because all he talks about is how important he is and security. And he talks to the queen like nobody else. And she talks to him like it's it's it's it's over. It's done when he went home and he saw her those people booing. So I think that's what's happened. Just just a struggle just grasping straws to be relevant at this point. What's the best thing about her sense of humor and her ability to see the the humor in so many, so many different things. We we have a really special relationship. We talk about things that she can't talk about with anybody else. So that's always a nice piece to it. But I think she's I think I'll just say that you get bored of birthdays. You do. You think she's bored of her 96? She won't be bored of the jubilee, will she? No, I don't think so. She's had she's had a few jubilees now. So everyone's everyone's everyone is slightly different. But I'm sure she's looking. Do you think you'll come? I don't know yet. There's lots of things with security issues and everything else. So this is what I'm trying to do, trying to make it possible that I can get my kids to meet her. All right, I think this is a good one, fellas, and we'll see you next time.