Okay, so we're on live stream now Mark and should I start. Yeah, absolutely. Good. Hi everyone my name is Mark Lejeunesse and welcome to my very informal workshop on how to wrangle large teams for research synthesis. Before starting and maybe you need a little bit of introduction of who I am. I'm an ecologist at the University of South Florida, and I've experimented a lot with including many groups of people at various stages in their career into research synthesis project so for example right now I'm teaching introductory biology I have 250 students, and I have all these students at helping me out doing a scoping review on a topic. And what you'll find is a lot of the details I'm going to talk about today have to do with my many many failures in getting consistent outcomes and I can't say I quite have a solution to streamline the whole process and including a lot of people but I've certainly learned a lot of things along the way, and I'm hoping to share these with you. And if you're willing to take on a large group and what I mean a large group. Well, I'm getting ahead of myself here when I mean a large group. I mean, more than 50 to 200 or so individuals. It gets really difficult. And at the heart of all this we want to create a high quality synthesis. And when you include many, many people, again with various backgrounds and whether or not they understand research practices. And a lot of challenges with consistency. And so a lot of the details I'm going to talk about today is make trying to make the whole team on the same page so that you could get consistent outcomes. And maybe walk away, making decisions on whether or not you want to include a very large team. And without further ado, let's just jump into it and begin with what I mean by team. And when I was a graduate student, a large research synthesis team was essentially collecting my friends to be a participant in a research synthesis project. Some teams are absolutely streamlined. Every member is highly qualified expert. They know the role. They know how to complete tasks with little or no training. Ideally, this is the best team you could have when everyone is on the same page. Everyone is motivated. Everyone understands the tools. But now I because I teach large classes I always get excited about including undergraduates in research synthesis projects. And so, for me, I typically have a giant cohort of minions that I really want to have them participate in research. And the work involved in getting them to participate is really on my end, in that I got to make sure that they know exactly what to do so that they could succeed. No matter which team you have, right, either it's a bunch of colleagues you're part of a work group you got some funding to parachute people together and work on a project. You are part of an existing team where you just crank out a systematic reviews or meta analysis, or if you are, you have the opportunity to include a collection of people that aren't typically part of a synthesis project. The same. And the beauty of it is a lot of aspects of research synthesis can be a broken down. Research synthesis is complicated, but it is a highly structured process. And that's really to our advantage. That's the golden thread that connects the entire process is every stage is very well understood. And, and so I'm going to cover a lot of these stages and I'm going to discuss which ones are most amenable to including many people in terms of like the dividing jobs and handing out distributing effort to complete those tasks. Right. So here, for example, this you could see this this is this applies to any type of multi member group task assignment where you got a big job.  And then with those little parts you distribute it to many people. And so research synthesis. And like I said earlier, although very complicated and time consuming. The aspirational goal is maybe you want to include many participants in the process so that you could have a greater scope in your synthesis project you could cover more studies, or maybe you're interested in farming some of the meticulous things like extracting data from figures. And, again, unlike other types of research projects research synthesis is very open and has many opportunities to include participants. However, not every synthesis project should have a giant team. And if you feel like you're including a lot of people because you want to increase efficiency the pace of a systematic review or meta analysis. You may actually find yourself slowing down the entire process by including all these people because you got to train them. You got to catch them up. You got to evaluate validate their decisions, maintain consistency. And of course you got to communicate effectively and wrangle all these folks to try to get the project done. So let's say, let's have a little bit of fun here. And what I'm going to do is, you might have seen these things before right these. You know, putting systems on YouTube where you discuss things and you place them on a scale of how amenable here I'm going to put all these various research synthesis tax tasks on a scale and how amenable they are to including many participants. Later I'm going to talk about in more detail the actual composition of a team. But right now, I'm going to focus on, you know, aspects in the tasks involved in research synthesis, where you could include non experts, where you could include people not essentially evolved in the development of the project protocol or the project aims or any conceptual details. But they have little pieces of work where they could tinker with make decisions and send it back your way in hopes that you could aggregate all that information and move along to the next task. Okay, so one of the first stages is the scoping stage. Now, depending on how far along you are in your project sometimes you have an idea and you just need to figure out what's been done. And whether or not there's many studies out there whether or not they're complicated. I think this is an opportunity yes to include many people. I will put scoping as, you know, kind of a B, B or a level as being the best as being the most superb at tasks research synthesis tasks that is open to a broad participation of people scoping is okay. I, I'm doing it right now with 250 students. And what I'm doing, having them do is just kind of wander around willy nilly in an unstructured way to try to find studies that are relevant to our topic. Now my my aims are very different than probably what you're interested in my aims are to teach them science, teach them how to navigate references and how to understand how science is reported and published. And the ancillary benefit is I'm getting a feel on a real research project. And so they have a task where I s I tell them. We are doing this topic topic X.  Provide me the citation and a PDF. I'm going to just quote a little bit of information about that study related to our topic. Overall that's fairly successful but again, I'm not using their decisions, or the information they collect to do a proper synthesis project. I'm just scoping out what's out there, searching. So really a key stage.  This I don't think is something that you could distribute to work unless you have many, many, many databases you can put it on. But the overhead of that is actually fairly difficult, because you know for each database the keyword search string will have to be modified. And that's probably not something you want to delegate to a group. That's something that you should be doing on your own. If you're the presumably the manager, the lead on the project. I would say important stage and to be consistent and to be clear. And so I don't feel like this is a part of the research since since I says tasks that would be useful to have many people work on screening on the other hand right you've downloaded all the bibliographic information from multiple databases you combine them you duplicated you did all that business. Now you have 20,000 references you need to screen. And you want to do it in a, in a way in which you have high confidence that things to include and exclude are correct, are exactly what you want. This is really one of the important stages of including many people in a project. However, trade off. Yes, there is the possibility of processing thousands and thousands of references. You could have a dual, a double screening design where multiple people are putting eyes on the same abstract and title, making decisions on whether or not to include exclude. But the more participants you include in that. Many more challenges emerge, like how do you resolve conflicts. How do you resolve individuals that are providing inconsistent responses. My personal experience and I talked about my experience with having undergraduates screen many thousands of studies is they are incredibly inconsistent. It is not a population of participants that will yield high quality outcomes. I include them because again, my goal is slightly different from a proper research project and then I want to train them. I'm teaching them aspects of science aspects of biology. But I'll get to the challenges of including many participants in the screening part later. So I would say screening. Yes, fantastic. But there's many caveats associated with creating high quality screening outcomes. Coding. So you have to do quality assessments across studies. You have to code potential moderator effects across studies. Coding details to evaluate the quality of each study. I guess I said that already. This is also something that's amenable to including many participants. There's another one of these decision tasks. And depending on how you train people, unless they have a rigorous understanding of the concept that you want coded, you're going to get inconsistent outcomes. Sure, you could build in a built in again. Dual screening. You know, double coding of each study. Again to measure consistency. But then you've, because there's many participants you fall under the same challenge as screening efforts in that how do you, how do you reach a consensus across those decisions, how do you reach a consensus across disagreements. If the things you're extracting are numerical. If you address different values in those numbers, it gets very complicated and in the end, even though you've delegated these tasks to many groups and maybe you've processed a lot of stuff. You end up spending a lot of time or maybe you assign a new team to validate and double check that information. It's another iteration in the process. Where a lot of the work that's done may get dropped because it's not of high quality. And so, for many parts of the workshop that I'm going to talk about later is trying to avoid these multiple iterations of reassessing redoing these repeated tasks over and over again because some participants are not quite achieving the desired outcomes. Another important aspect if you're doing a meta analysis more than a systematic review is you're again extracting numerical information extracting outcomes. Yes, I feel there, there's mixed opportunity for extracting information from studies. I know that undergraduates should not be involved in the extraction of study outcomes numerical outcomes. They just don't quite have that sophisticated understanding of the scientific process yet. They're understanding on their end of experimental design of statistical tests of interpreting and understanding what's relevant from a finger. And so you the team that you want to be involved in that heavy extraction process which is by far the most tedious and difficult, because every study is unique, a different puzzle to solve in terms of extracting the outcome. That is not something that should be delegated to someone with someone that is a novice scientist. And I'm including undergraduates as a novice scientist here because they just haven't read enough they haven't taken many courses they don't. In my case they don't understand the biology yet and so having them extract like a contrast a control treatment contrast out of a study is a lot to ask them for. However, those tasks can be further cut up into more generic repeated tests like if you make a decision on what's important to extract from a study. Right, and you know that there's a valuable information locked in a diagram in a figure. Then, collecting all the figures that you need to extract data from is something that you could delegate to someone else to someone that is inexperienced because all you need to do is really is train them on how to do the extractions. And you don't need to have them understand or make decisions on what to extract. And in fact my advice, or that kind of stuff is I just have them extract everything from the plot. Right, don't let them make a decision on what's important or relevant have them extract the whole thing.  Okay, we're getting there slow study. Hypothetically, all the outcomes have been extracted. And now we're at the synthesis stage. This is not a task that can be delegated or distributed amongst multiple members the the analysis and the interpretation of the results that's kind of left to the core team. There's people involved in devising the protocol or devising the aim of the study. And that's not anything you could farm out. Right, so the synthesis, synthesis part is really like a low level task in terms of delegating to many, many people. Right, I'm not saying you can't delegate it to a statistical expert. That's fine. Right. But would you distribute your analysis to a dozen statisticians to a dozen people to analyze and synthesize I feel like that's a mess that you shouldn't need to worry about. Finally the last part, which is the writing and the recording of the project. I include many people involved in that process. I'm mostly because it's a it's a good training opportunity for people to experience how to effectively report a synthesis outcome, how to discuss potential biases and issues with the data set. When people involved in that process while you're generating the text and discussion and the results. I feel like is important. However, I don't think that's also something that you could distribute to hundreds of hundreds of participants that say you're not going to get a anything that's readable emerging from that. So reporting kind of falls on the low end to. What I'm hoping that you get from this process right ranking these different tasks is there are many opportunities to include many people. There are many tasks that could be distributed. But the remainder of this workshop is not so much about, you know, whether or not these tasks can be distributed to many people, but more of how do you manage that process. How do you make sure that they're doing high quality jobs, right they're getting little jobs, are they doing it effectively. And then, and then what do you do with all that stuff. Once it comes in, and I could tell you right now I've experiment experimented so much with this, and I still haven't quite resolved the problem. And so I'm hoping what you get is me describing the challenges that I faced these ad hoc solutions to getting many people involved. And then, so that you could avoid some of my mistakes, so that you could maybe better strategize use better tools. I tend to limit myself and our right now there's a lot of a lot of cool stuff out there that could facilitate the collaboration of these tasks. But this workshop is not a tutorial on how to use those tools, but I'll provide a listing of what's available. All right, let me just take my shoes off here and get more comfortable. Okay, so let's move on. So the remainder of the workshop, which will essentially be me describing my experiences with including many, many participants. I'm going to focus on some management approaches. I might be useful. You may already be familiar with a lot of this stuff, but I think sometimes it's useful to hear how someone else has approached this problem. And by managing I mean you are coordinating the participation of many people, making sure that they're on track, making sure that they are submitting their work. And then, most importantly, you're the one that's assessing and evaluating the consistency and quality of their efforts. Then I'm going to talk about team composition. Again, your management approach will probably differ considerably depending on who is part of the synthesis project. Really right. You have like the soccer team, where everyone is a high level expert in and in research synthesis, everyone understands conceptually what research synthesis is everyone understands conceptually the subject, you want to synthesize everyone understands the stats, right. And so there's a lot of short hand discussion that occurs, and you could be extremely nimble and get what you want fairly rapidly and still be high quality. On the other tail end, you have hundreds of undergraduates with no expertise whatsoever. Some may be eager to participate in research. Others may just treat it like this is an assignment or an exam. And not give it the attention it needs. It's just like a task in the week they need to get done so they get a grade right so the motivation of the participants can be very different. I feel like this is what would happen if you would say farm out to the broader community, these menial tasks, or these repetitive tasks, because not everyone is fully motivated or interested. And, and, you know, gamifying all these things or say hypothetically you invite hundreds of people to extract data from figures. Opening years, I feel like you're opening yourself up to a world of pain. If you do that because you really need to think ahead and organize a system that allows for repeated validation of decisions. And I'll talk a little bit about how I've had to kind of change gears, switch my total management approach to farming out these repetitive tasks to non experts. Finally, I'm going to talk a little bit about tools available to distribute effort, distribute tasks. I focus mostly on in our because that's what I love. But between you and me are is not the most optimal way to do this stuff. And you may just end up using like a collaboration, a canned collaboration software where people could work on the phones people could work on their desktops in a nice clean UI. The HTML or JavaScript or something like that are provides ways to make nice UIs also. But the challenge is like how information is returned to you right so you may create a nice shiny app that is meant to code, or help people input their study extractions. Issues will come up. Issues will come up where there's like you didn't fully populate a drop down. And now you have to go back and edit revise the shiny app to include more drop downs to include more cells to input because the ugly nature of research synthesis is that there's huge variation in study design. It's reported. And so it's very difficult to have a one size fits all extraction or coding form. Because that means you really have to do the work beforehand to assess what is needed for inputting, and how to make things kind of readable for people to do their job. If you just have a giant form that accounts for every single edge case of reporting practices. That is totally overwhelming. I mean, imagine if you have a single form that includes all the cells needed to input every variation of treatment control contrast effect sizes. Every variation of how correlations are reported every variation of how potentially odds ratios are reported on a single form. That's a lot to navigate a lot of opportunity from people to make mistakes, or to just get totally disenfranchised with the process. But this, this is, this is what I feel our opportunities. This is our opportunities for people to to develop tools that are kind of generalized that allow for quick. Augmentation for special cases in terms of data extractions or coding. I'll get into that this is really really the exciting stuff. If you're interested in, if you're, if you want to use if you're like a Cochrane author you got a ton of awesome resources and, and software for you to achieve these things but if you're a regular vanilla ecologist like me. Maybe I'm kind of a weird case and that I love developing this stuff so I, I don't mind getting my hands dirty. You may not want to take it this far. It may just be quicker to just do it the old fashioned way where you're. You have a giant Excel sheet with many columns, and that's where they people input their data. But let's let's, let's keep moving here. Because there's many things I want to discuss in detail. So let's, let's sit back a second here and talk about why we want to include many people in our synthesis projects. And I think the idea is it's very enticing to want to take a big bite out of something to have like a really broad scope and process thousands of studies to have like a broad conceptual mapping of related but diverse research topics and just farm a lot of that stuff out. In hopes to be able to that to synthesize it to provide a broad overview of what's been done. And, and if you've done a research synthesis project you could see oh yeah okay. You know what, when I did this ex research project, I really spent the bulk of my time in the extraction task. In the process pipeline, extracting data was the most tedious and time consuming. If I had help in doing those things, I could have done it quicker. I could have done it more efficiently. I could have covered more. If I had brought in my research topic. I think that's the aspirational goal, but in practice, it is difficult. And when you pull in many people there's many things that come up that you don't anticipate. I could have really stall many tasks along the research synthesis process that is not efficient. Right, you're spending a lot of time, making sense of what they've done. You're spending a lot of time catching people up. You're spending a lot of time organizing resources making sure everyone has the tools or what they need to succeed. And so even though you're like oh I'm going to save so much time by farming the stuff out. Now you're spending a lot of time doing these additional and managing tasks. At least for me are not super exciting to do. So here are some pros and cons of having a large team. I'm sure there's many more and I will overlook the bunch but I think I distilled it down to these. These few examples. Research synthesis the task, the tasks involved in research that synthesis is stable that, you know, there's a real structured process. If you're approaching an experiment, and then you have to account for contingencies, the research synthesis process is highly predictable in what needs to be done. And so the long view of including many participants in that process, you can see the advantages of that right. There's advantages of having many people screen references, because they could process many things. Right, you could, because you have many eyes. You could expand the scope of the research topic. You could have broad coverage, which you couldn't probably address beforehand because you had limited time to achieve things, including many people means moment maybe, you know, rather than doing this narrow topic. You could scale up hierarchically to a broader concept and include that, that little topic, but also include many more related or indirectly related topics. Task efficiency, right, and this is where I kind of have mixed opinions on. I'm never really interested in doing things quickly. But I understand there's a whole branch of research synthesis that wants to push things through fast, especially if like they are related to important emerging issues you need a really quick assessment of what's happening. I'm having many people in a team can be a way in which to process and complete the tasks quickly. I feel like the best team to do that is a collection of experts. Right, you're not going to zoom through a rapid review by including non experts in that in that process, right everyone involved in that are just like Ferraris, and they're not people on bicycles. When you take on a large project a con is, at least for me, for real is my role changes.  Deeply, right, more than just a manager more than just a supervisor I want to be the guy catching a dragonflies. I want to catch dragonflies to. And it's the same with my research synthesis project is I want to be part of the screening I want to be part of the extracting phase.  I cannot be doing these tasks. Because I have to focus my attention on resolving issues that emerge, as opposed to task completion. And I'll talk a little bit about that becomes a different managing style. Even though deep down inside I want to be the person doing a lot of this stuff. And at its heart to I feel like I could do all these tasks very quickly. Right. Sure, I have 250 students scoping a research topic. Right, there's incredible opportunity for me to get a good I feel for what's out there, but I could also do this on my own. And, and if I do it on my own I can do it quickly. I know exactly what I want. I know exactly what's needed. You know, 20 plus years of experience doing this stuff I could zip through it quickly. But in the end, in terms of my role, it's a last opportunity to train a lot of people. A lot of lost opportunity to get young people excited and involved in science. And so again I got to change my role I can't be the one that does all the work. I got to delegate. And for the risk of dilution effects when you distribute work across a large team that that the flow of tasks are not being addressed in a timely, or, or correct way. Because there were hiccups in how they were trained hiccups in the tools hiccups in how information is accessible for them to make good decisions. And so as tasks get passed on to a group. There's no way you could like individually respond to things, especially if there's hundreds of people. And so there, there might be a dilution of information that you need to account for when you're aggregating everything that's been done. And this is this is takes up time.  My, the biggest thing that I've failed at by including many people is setting up a program where things are consistent with undergraduates again you know their motivation may vary quite a bit. Their interests will vary quite a bit. Now, I know there's students in my class that are not interested in participating in the scoping review. I'm breaking up here, I think. Okay. Is everything still okay can you hear me. Yeah, we can hear you Mark. Okay. I just got a warning saying that you know there was an issue. You're okay. Consistency and I last year at the this conference I talked for 10 minutes about you know my challenges with consistency in screening efforts by including many people. And so I'll talk a little bit about that, but in the end, it's difficult is really tough. Okay, so your role right is to manage.  For me that's a tension internal tension that I struggle with all the time. It doesn't make me I could be honest right you right now I'm not a great effective leader because I have problems, delegating things and communicating to people. But I but because I've done it many times I have some tips and some some guidelines on how to avoid a certain management approaches. Right. So the idea is you're the manager you're you're managing complexity. You have the broad view of the entire project. You have the broad view of what needs to be done. What tasks need to be developed so that they could be distributed. How much time you're going to need to allocate to make sense of what's been done. And you have to be there for the participants. You're going to be the one that troubleshoots, they're going to ask you a question you're the one that's going to be there to make a decision. Even though you're not directly involved in the screening or the extracting indirectly, you will kind of be involved because they're going to come to you with difficult examples in ecology right experimental design and what gets reported is highly heterogeneous. You could have a nice clean study where they just do a contrast control group and treatment contrast. And then you have another study that has multi level multi factor analyses on a process. And you're left like trying to figure out what the outcome is what's the relevant thing they're going to come to you going to be like Mark, we need this we understand that this is a relevant study but the key outcomes are difficult to pull out. And so then you come in, you're the one who has to make the decisions to try to figure out. But your goal though is to try to make the tasks that they're involved in as simple as possible. Your job is to just strictly screen title abstracts third job is to strictly code. Single things out of studies, their job is to strictly extract data from bivariate plots. Okay, do not source out critical decision making things do not have if you have a collection of non experts, don't let them be the ones deciding what outcomes to pull out of a study. Don't let them decide what outcomes what studies to include or exclude. They're just not quite there yet to be able to make those key decisions, you have to have an expert group, or you have to have significant training prior to that whole process. So that you could assure you could buy some insurance that whatever decisions they make are useful. Yeah, my experience is I go through. There was one year where I went through six iterations of abstract title screening with my students, right. We screened all the studies in one about half of the students were highly inconsistent.  Let's just do it all over again. It's like a cohort of participants totally inconsistent, and it's easy to figure out who's consistent and who's inconsistent because again you're using statistics like Kappa statistics to verify the agreement.  They really have no clue what they're doing. And then, and then you spend time trying to catch them up. They're just disgruntled because there's a collection of students that aren't doing consistent outcomes. It gets ugly, it gets real ugly, but I got some tips for that. All right, so here's some management tasks for you, or some tips as you would as I would refer to them as how to help yourself out. When you're taking on a large team. You need to have your ideas in place right. If you are just starting off you got an idea. You haven't even scoped. You're not ready to have a lot of people participate. Put in the work and the ideal scenario is at minimum you've that you've thought through a Pico statement, or best case scenario the study is pre registered right you've thoroughly thought out. Every detail of the study, what exactly you're going to do. I feel like if you're at that stage, then it's easy for you to assess how many participants you need. If you're just starting off a project. You need to put in the footwork to evaluate whether or not it's even necessary to include many people, and I would say if at such an early stage like that don't spend time thinking about who to include. Because you have a ton of work ahead of you to try to figure out whether or not the project is tractable in the first place. Once you have a good idea good vision of a synthesis project. We need to start thinking about team composition and I haven't really talked much about team composition but really I'm going to split them up into collaborators, core members of the project. Right these are the experts. And then you have participants, people that had that are going to be working on these repeated tasks. We already covered the three main repeatable tasks that you can include many people on the screening, the coding and the extracting collaborators are important. Right, it's, but you're not going to get a super large team, even if you include many experts. So I've been part of many work groups where, you know, someone gets funding and then a collection of experts get shipped to an area.  But it's all experts right you have a statistician you may have a librarian you have a domain experts. And it's easy to see how these people fit in the project. It's the participants is like where it gets difficult like what you hear things you need to think about. Since you're farming out work. How are they included in the final report. Are they, is that enough work for them to be granted authorship is if you happen to have a few key participants that were really valuable should you give them credit. These are questions that I can't answer I tend to be absolutely inclusive. Right, when I teach a class and I say, if a publication emerges from our research synthesis, everyone that's participated irrespective of the magnitude of participation.  I want to ask them to be part of a project. So, and so I need to provide incentives on for them to do high quality stuff. And that's one way like I, I don't have money to distribute to pay them to do these tasks. And so this is like the carrot on a stick that allows them to facilitate or at least be ambitious because they're like, maybe an endpoint is an actual publication. I spend time thinking about resources that are available to you.  But there's a ton of stuff out there in terms of canned software for systematic reviews that are that can be expensive. They may have really enticing features. Right. They may be highly collaborative. They may have the flexibility that you need to delegate these tasks. You need to kind of figure that stuff out. I mean you can't just like invite a bunch of people, and then figure out what tool to use. You need to spend time to figure all this stuff out before you make a decision on whether or not to include many participants. Money is important money and time time. You don't want to waste people's time. I remember some of the reviews I got from my students is I wasted a lot of their time by doing these multiple iterations bouts of screening. I just wanted screening outcomes to be consistent. I would experiment weekly on different approaches to train people to have people catch up to try to identify the problematic students. But that's the participants when they are part of that you know they get disgruntled because they're repeating the same tasks over and over again. You know, in terms of screening they may be looking at different studies like making decisions on what to include exclude differently every time. But to go through that entire process over and over again and not seeing the project mature and achieve later important synthesis tasks. Well, maybe that's the that's the learning outcome is like science is hard, and it's difficult to anticipate what's going to happen. So I'd hate for that to be a situation where you just didn't quite think out in detail how people should be involved and how their time should be spent. And that just delays the progress of the project use collaboration tools right there's a lot of stuff out there that's free, or you could spend some money to have a central hub for everybody. Yes, there is a lot of systematic review software out there but those aren't those are like hubs where people ask questions, or, or have like tutorial resources, or there's it's not a place where you build a community. And so if you could establish a place where if someone needs to catch up on a task, they could just log in and see multiple discussions, multiple examples at tutorials, and to warm them up to help them finish a task, you save yourself because you don't have to answer emails or ask questions or answer questions. There's a central resource for them to find all the stuff they need to succeed. Right. So the way I've done it before is I've used teams, which is institutionally what we use here at USF. And if my team is local, right everyone, every participant is at USF, then I use teams because it has discussion boards, you the meetings are recorded everything can be placed in a central hub. So it's very nice. But if you're limited to something free you know Google Hangouts is a great supplement I mean everything you need is there to work out. The idea is to minimize managing time on your end to answering questions, often enough answers to questions can be delegated to participants participants will answer each other's questions in chats or discussion forums. I do a lot of time and I feel like that's has been a saving grace for me with my many undergraduate participants is that they kind of, they kind of teach each other how to troubleshoot and how to deal with these things and I could, you know review and verify their questions, but by most part they tend to be correct and prompt, and, and provide a lot of useful information, even for people who don't have questions right you may have someone who just like. They don't know what's happening, and they don't know what questions to ask. But if they visit a discussion board they could see a collection of questions that have been asked and how they have been answered. Fantastic tutorial and catch up resources I'm going to talk about that. These is really important, especially if you have many, many participants everyone needs to be on the same page. Everyone needs to know exactly what to do in a high quality way, and their strategies you could use to kind of facilitate that other important things right you want to have opportunities for people to live edit documents, or be involved in the research synthesis is great because you know there's a there's a really specific timeline, you cannot go to the extraction phase until you've collected the PDFs or found the main text of all the studies right so there's a real synchronic synchronicity to activities that need to be done. This adds a nice structure devise a communication strategy and so we're here again we're going to have a little bit of fun in describing like what's the most effective way to talk to a large team. My advice well let me just go into it now, but we're going to do another one of these rankings. What's the most effective way to communicate to team to make sure that everyone has what they need to succeed and to answer questions. Email is the least effective way to communicate to a large group. I mean people won't read their emails, your email is like distributed to a group, it may end up in people's junk mailboxes, it's text, it takes time for someone to read interpret. Avoid if you got 50 or 100 participants avoid email as the way to communicate to them. Again rely on the central hubs discussion boards, having documents or videos to help them navigate the process is really the quickest way for people to get done and so my advice video videos the top way to communicate stuff right. You can record the meetings people can attend, not a big deal the meeting is recorded. Just to show someone how to install arm. Just create a video. Don't write a document on how to install stuff. Just do a video tutorial. Right. This is takes less time than writing up a document. It's easy, you could just record do a screen capture right if you're in Windows there's like the Xbox screen capture stuff. You can record it in PowerPoint. And it's visual, the communication is visual you have audio at the same time. And you have the point and click navigation they see exactly what they need to do they can revise it they can remind it. It's way more straightforward than sitting down and writing a text to guide on how to do stuff. Right. And, and for me, when I'm delegating tasks to my students, it's all video. Here's what you need to do to install art. I go through it. Here's what you need to do to start screening studies. There's a two minute video to help them out with that. Meetings are important also right with many participation to participants. This actually gets difficult to organize and have everyone participate so again pre recordings is key, or if you are going to meet. centrally, then recording that process is important so that everyone has an opportunity to understand what's going on. Documents. It's a mid range way to communicate with people right. Having guidelines or examples to read is great. What if you could record a video. Right. And, and articulate how you made decisions. Right.  Describing your, your decision process. That is way more illuminating than providing a bunch of examples in text form on like, why you included and why you excluded stuff. You could record on your end than having to develop these giant documents. Finally, chat's important. Well, I should rank put it up a bit higher here. A lot of questions are going to come up and having these open chat pre pre set times to chat with people while they're meant to be doing their tasks is like, you just have the chat window open while they're doing a task and it could see everyone's a real time discussion on what's what's happening how they're making decisions, everyone benefits from that. Okay, the resources emphasis pipeline is essentially your timeline and your meeting rhythm. You know exactly when to have rigid meet ups, because they appear at the tail end of completing tasks. Right here here hypothetically is the entire process of a research synthesis project. Large groups are only involved in a few key moments during that process. Right. But these are the opportunities where you, you meet right two weeks a month. See if you have a if you have many participants you got to build in that buffer on time to get those things done. Because you got to train them. You got to give them time to install the tools they need you got to give them time to learn the tools to have practice examples.  And, and the idea is having a rhythm where you meet at a preset time by monthly, for example, gives people time to anticipate when the work needs to be done. How to navigate their lives when they have to screen a ton of stuff, right not everyone's going to be able to screen stuff right away. So they're going to be like, okay, this is not due for two weeks. I know next week I got some free time I'll do it done. Team composition alright, I got to speed up a bit because I'm talking a lot. Right the idea of a team is, again you got collaborators, and then you have participants, right. The ideal team is everyone's an expert. There's no overhead for the manager because you could delegate stuff and you know that whatever is going to emerge from that is going to be high quality. Everyone knows what to do. The statistician understands what to do the librarian, librarian understands what to do the domain expert understands what's needed. It's the, it's the repeated tasks participants that are the challenging group. So my philosophy is like anyone can participate in a research synthesis project right. But do you want to include a bunch of novice people, because, even though there's the opportunity for like increase scope and efficiency in tasks and completing tasks. You are opening yourself up to a lot of training, a lot of catching up and making sure that the novice participant has everything they need to succeed. So, in the end you end up spending a lot of time managing and training people, as opposed to having a collection of experts, where you could skip that whole phase, and sure you're not screening things with as much intensity in terms of eyes, but you know that they're doing high quality decisions. And you all that morasses you don't need to think about other things you need to worry about local teams are probably the best because you just bump into people you could chat with them. And it's incredible how much work you could do when you're just sitting at a round table getting things done, you can ask questions directly boom boom boom things happen fast. Remotely, things are slower. Right, but the opportunity to include more people is very high right because now they could be anywhere in the world. And if you have a central hub to communicate and to organize training material and documents. There's many opportunities to include participants. But again, you know, as a manager you're going to be spending all your time trying to make sure everyone's on the same page in order to avoid issues of consistency of task decisions. Again just to reiterate video tutorials better than text guidelines. My impression is people tend to be more visual learning folks then reading active practice is better than text examples right so in your guidelines.  That's not a the most effective way to catch someone up on what are high quality decisions, have them do a bunch of examples as they would, they're in a task. But do you know the answers already and a lot gives them an opportunity to like some introspection and evaluation of like why they made the wrong decision. Group activities versus or just better than individual activities this is more challenging if you have hundreds of people clearly working as a group to finish a task is not effective. Right. And so this really is dependent on how many people you want to include on your team. This is like try to synchronize smaller groups together together to get things done. So in my class for example we I have some time allocated during the lecture period for people to work together in person to help them finish their tasks. Discussion boards are absolutely key right chat chat rooms whatever anywhere where there's like a recording and keeping track of questions and answers is crucial, because not everyone is going to be like active at the same time a day. So we're not going to have that synchronous communication, the chat rooms the discussion boards is allows for asynchronous learning of what to do. So here's an example of like a video tutorial that I would post be like, okay your job is to extract data from figures, you're going to install the software, you're going to bring up this image, and you're going to try to replicate what I'm doing. Let's see if you can recover those and numerical data. This is an opportunity to discuss a bias associated with extractions and a way to kind of develop these consistent rules to extract what you need.  So here's a my typical workflow when I have many participants and it's ugly, and it doesn't always work. But here's what it is. Okay, so I record a bunch of videos on how to access tools like our how to download it. How to install packages. And then there's a time period where the everyone needs to achieve that before we move on to the next stage. So if someone has installed what they have. Right. Then we do additional training on conceptual, because the people are making decisions after screening things they need to know what the inclusion exclusion criteria are, and they need practice examples so that it gels in their head on how to achieve those things. They actually do the task right you're assigning them collections of abstracts to screen, you get a little warm up phase where they repeat some of these pre known abstract decisions, so that they did. And so they don't just jump into it cold, but they have a few iterations that are thrown away beforehand. And that's because you know, this is not well understood stuff but decisions you make really varies at what stage you're at, when you're screening things at the start you may be hot but at the tail end you may be cold. You know, your goal is to kind of make things small and tractable for people and this is one way to kind of make sure that they're creating high quality decisions. All this, when you're signing tasks right you have this dual screening design. Multiple eyes looking at the same thing making decisions on your end. It makes it things difficult and complicated because you have to verify consistency. Right you got Kappa statistics you got to make sense of you got to evaluate whether or not some people are just clueless and whether deciding right the decisions are random or totally wrong. And if that's the case.  You still want to include them. They're still participants, although between you and me I have totally dumped the work of some people, knowing that it's impossible to get consistent outcomes from them. They can get involved later on so that's not much of a big deal, but you got to retrain them, re expose them to different examples and exercises again to make sure again everyone's on the same page. And last year's conference I spent 10 minutes talking about how difficult it is to get consistent screening outcomes to like delegating tasks to undergraduates it's hard and really a my advice is it's not. It's not an effective way to get what you want. To me it's like a key training opportunity right, they get their hands dirty and science. It's an opportunity to improve science literacy. But in terms of generating high quality research projects. It's a lot of work. It's really a lot of work, maybe more work than it's worth. All right, so let me end with opportunities to, in terms of tools to facilitate the whole process. Right central hub is important for communication. What tools do you use to actually delegate the tasks and for them to do again I'm concerned mostly with our, and there's some stuff out there but there's not that much in terms of the entire process and I feel like those are open opportunities for some people to develop. Right if you're using can software so for example if you're using like a web platform for collaboration on like screening and extracting coding data there's a ton of stuff for you out there. I mean there's a lot. Some of it's free some of it's paid for they vary considerably and what sort of tools some to provide automation tools. This is stuff you need to figure out before you start the project don't start jumping from one tool to the next.  Ballooning growing field of like accessibility this is just like the ones that are web platforms. There's way other things out there that like this is a very incomplete list of a collaborative dissemination approaches. And that is what I'm mostly interested in because I like to tinker or provides you the most flexibility because you could have, you know, maybe this is not an advantage but you could have ad hoc solutions to problems. And since you know, I'm the manager, I could take her with the tools to get what I need. Whereas if I was using a can software I wouldn't be able to manipulate the HTML or the JavaScript or whatever is going on right here. So I don't have that type of operational flexibility when it comes to delegating tasks. So my conventional dissemination strategies is very simple, very simple, and I've done it all at some point. Right now I use Medicare. What it does is it just kind of creates a bunch of files filled with say hypothetically the screening tasks but it also could be a collection of studies that need to be coded, or a collection of the figures need the data extraction. What it does is it just kind of splits up the database and for multiple people and if you want a dual design it does it all for you. And then they're left to plug in this a sample data set into. And if you want Medicare the abstract screener. And once they complete the task, they could submit it to you. And then you could assemble and aggregate all the stuff this is basically what I use. Most of the time. Right and the same with the data extraction stuff and are there's many opportunities to delegate extraction from data, data extraction from figures. So my juicer is just one example of what's out there. One gap. However, in our is the coding and extraction phase in terms of like pulling out numerical outcomes from studies. The stuff is like amenable to just having like a form where people fill out in the input. They fill in the cells of information like the means and standard deviations of what gets reported from studies, but there's I feel like there's no clear tool out there in our at least that that is generalizable that allows you to account for edge cases of difficult extraction studies. I feel like this is an open open thing and I approached this problem different ways using JavaScript and HTML, and then when they submit the form it gets pushed into a Google spreadsheet. Another approach I've done this is have like a form fillable PDF. And then they submit me the PDF, and then I pull out the whatever things that he integrated into the form. All this is really nice for just like coding information. But when it comes to extracting numerical stuff, there's really no clear framework to get this done. I think most people do is just kind of populate an Excel sheet. But there's way more if you if you understand the process you can create some cool tool to really pull this off efficiently. So so I'm going to end with a bunch of aspirational tools that I feel could be augmented or pushed in in our or other ways to facilitate the extraction and coding of studies, which I feel like is something that is and not like developed underdeveloped in our. Okay, so our right now much in terms of coding and extraction, you can have a shiny app aspirationally. The shiny app should be something that someone could add is a generalized and someone could provide inputs on how many cells, or how much detailed information they want to code. Drop downs open numerical cells that gets complicated for for someone to pull together, especially because every research project and every study has kind of like its own problem to crack. But I think it could be generalized. And which is a really cool way perhaps to help validate whatever gets inputted because you could have conditionals and cells to communicate to someone saying hey, this is this is meant to be a count, and you've included a fraction. But Python could also be pulled into this to you but again, you're using Python and are as a back end thing really what you want is an interface and so in the end you'd be using some sort of a gooey package to create a front end for Python right tickle TK whatever. And so that that becomes a way more difficult task to pull together, but totally doable. So I have a couple spreadsheets. I've used it before I put it up here. Again, it allows you to have a collection of like something that has a form structure. Plug in data, a lot of people could just have access to the website plug in the responses to studies and it automatically populates a spreadsheet. Nice. That's also good. But here's the hiccup with form fillable PDFs recording and extracting is, you have no control on their end. What is the software they're using to actually render the PDF, and how to save it. Right so like right now most browsers have their own PDF. They're not using a tool right you're not using Adobe. So that means like once you fill out a form and you save it, the internal structures of the inputs are very different. And that causes a lot of headache for you sometimes a PDF is saved it's just an image. Right you there's no way to easily extract what's been filled. It gets ugly. You could use a canned package nothing wrong with using a can passion but here's ran. It allows you to collaborate and disseminate effort right you lose some flexibility though right you're really dependent on what they offer. And if you have some technical understanding then you know I'm telling you right now there's opportunities to develop something cool. Right HTML JavaScript. That's what all those web platforms are based on. And that could be rendered in our also right you don't necessarily need shiny. You could render HTML JavaScript CSS directly in our, and I feel like that's an opportunity also. And finally I, because I'm dealing with a lot of students, and we have our own dashboards or our own hubs to teach classes. And bamboozled those educational systems to do systematic reviews. Many opportunities out there. But in my case, every solution that I have used so far. I have been ad hoc to specific projects. Nothing other than say using meta gear has been generalizable across multiple subjects. So the person that solves that, that can create a generalized solution for extraction and coding. You're set. This is it. You're going to help a ton of people and facilitate the participation of many, many people and again to me. That's a wonderful thing, because it's a training opportunity. All right, well, I think I talk way longer than I should have. And so. And so I'm going to leave it at that, like I don't know if you're going to walk away from this being an expert on how to manage a team but I hope I provided you some, some thoughtful discussion on how difficult it is, and how what to anticipate, if you want to do it.  If you are interested in farming out or micro tasking or crowdsourcing.  If you do not spend the time to train people. And so you, even though you, your, your goal is to do a increase the efficiency of a research synthesis project. You're slowing yourself down quite a bit because you have to develop all these resources to make sure everyone have everyone has what they need to succeed. All right, I'm out of breath.


Hello everybody, he here welcome back to castle talk with so sorry there have been no videos this week We were in Hoover, Alabama at a convention if you guys saw Monday I was able to upload one of the panels. I have three more three more coming with a four or five six Something like that. I got a couple more coming. They're gonna be more. I don't know. Yeah They're gonna be sprinkled around throughout the channel Also, I'm going to be changing the schedule for Thursdays It's gonna be Thursday theorist followed by this show castle talk Just make it a one Stephen King day because I have plans. I had plans to begin with for Wednesday So starting Monday will be back on that schedule. Anyways, let's talk about episode four of castle rock on Hulu the box Yeah, the very first thing I thought you hear dog bark and I apologize, but the very first thing I thought when I heard the box was Wendy's button box the The story set in castle rock that King wrote with Schismar. So it's up there on the shelf, but um, I had no idea It's gonna end up going obviously at looking back hindsight. The box had absolute I don't think the box had much to do with this episode at all I mean other than the coffin and he was kept in a cage. Is that the box? What was the box? I think it was the coffin and And where he was kept Henry, you know, but I wasn't like the point of the episode though, wasn't I mean And maybe the prison maybe the prison Yeah, um, the thing the thing and I'm confused about with that whole episode is that there was no cohesion with like the story of that episode um, that episode went some of everywhere and none of it I think it went a little bit of everywhere so it could start pulling together everything from like the first episode on No, that episode doesn't stand alone by itself. No, he would have to actually watch the other episodes who to get it Which I kind of respect that at least now they're giving us something to chew on and actually talk about Yeah, um, one thing that I will make note of is I read an article from I think dread central or bloody disgusting one of those places That said actually, I just read the title. I didn't actually click through because I don't like either one of those sites but It said cast a rock to be an anthology series like American horror story So basically it's going to be standalone series for each and every you know each and every season is going to have a standalone series um It opened up very well The the episode did the tom weight song clap hands with salinsky going in Um, it's funny because I made note That that salinsky wasn't I thought unintentionally creepy. Uh, this is denis right denis salinsky's When henry's on the phone, he calls him denis but uh Again spoilers run away if you haven't seen that one because the end of this episode was a trip Yes, it was um There is a running theme once again like I've brought up in my steven king theorist videos About psychics in in main.  You got henry beaver Um, you got molly whole new home doing was seeing stuff. Yeah. Yeah molly. Well, I think his was actually just dreams He's just having dreams flashbacks Still though For me that one counts. Let me let me let me drink some water He ate the folks in it making us wait on him. Sorry. I got choked up. Anyway, I think the whole thing with henry Counts as part of this whole psychic thing or you know special abilities wise. Um Couple things I want to make note of molly strand lives in frank dodds house. So the the Uh serial killer from the dead zone. She lives in home news house Why did she tell the couple that was there to see the house y'all what in the world talking about lacy killing himself Yeah, well, technically she was right. She didn't have to tell him because he didn't die in the house I was talking about that just that though. She told them where she lived though talking about that. Oh, yeah Why didn't you know that? They don't need to know any of that. She's super awkward. I like her character Um, I don't remember this. I I'd have to read it's been a while since I read needle things It's been a while since I watched the movie was polychalmers a real estate agent I'm pretty sure she was hollering me down in the doobly-doo if I'm wrong Um, there was a sign for the house on maple street And I'm pretty sure if we reach back here smile at the people while I while I Hey, y'all how y'all doing? Uh, let me I just need to look in here to see what Yeah, the house on maple street. There's a maple street. This is nightmares and dreamscapes There's a short story in here called the house on maple street if you haven't read it spoilers spoilers spoilers The house literally lifts off into the air like it's a spaceship Um, but and there was a very very very brief glimpse of that when henry turns right He's anyway, I don't have to show you but he turns onto a street called maple street And then just yeah, they show you the the street sign for a split few seconds maple street No, I I caught that I be I be I be on it I be on it Yeah, um, so another huge one is desjardin That was creepiest You know That was creepy Uh, when he's talking about when he leans in and home dude says I never touched you with that smile on his face. I was like Oh There yes, how am I supposed to believe that when you were this close to me? They didn't going down a couple things with the disjardins Vince disjardin it was one of aces crew in the body or stand by me if you know it by the movie adaptation. Yeah Um, but uh, so he's a he's a part of actually his brother whoever joseph joseph desjardin Um, I only remember that is because when he's talking to ping born about henry Um killing his father and I remember him specifically the way he said joseph was kind of weird to me The way ping board did almost like he had a bad taste in his mouth. Oh, yeah joseph But everything this ping born says seems like um, he's He's got a bad taste in his mouth. This ping born is an axe Um, he I still like him though because even after even after the argument with henry He still goes and talks to henry's mom and said, you know, maybe going down there and then she she wild out Wilds out on him with the with henry with the night. I ain't going anywhere unless I leave in a box. That was pretty cool Um, we never had this conversation. Yeah Uh, so going back over this one one more time The place feels castor rock feels a lot more like dairy than it does like castle rock Um, there were problems in castle rock, but it didn't I don't know that it ever felt like a cursed town there is little bits and pieces Um in I think it's not it's not kujo, but it's the dead zone where um What's his name? Uh, well just a narrator talks about, you know Having a serial killer and a killer dog in the same town Um together that was kind of I was a koo. Yeah, it was at the beginning of kujo He's talking about dawg the the monster of the town Um, let's see here. This episode's soundtrack was on point. Um, it's straight out of a steven king book the The tom weights the uh, this what was the name of the song that came on toward the end of the episode while he shoot We'll get to that the shoot out. Yeah, I can't remember. It's the old almost doo-wop song Was that it I think that was it that wasn't it leave the name of song down there and doodly do and uh Okay, so we got to of course we got to talk about salinsky shooting up the prison. Um I've said spoilers. I don't know how many times that was the best scene in the entire show, but once again We got like psychic imagery In the first episode of all and I knew as soon as he pulled out his gun and walked out that I was like That's this is where the scene comes into play from the end of the first episode So they've tied it all together. My problem is it felt like a Season finale. It felt like the end of a season the end of a story How's it go from here? It felt like the end of salinsky story and maybe the beginning of a different story with Henry That would feel like if we were to watch that in the next episode episode five That would feel like the beginning of a new season. Yeah, why would you do that? Like halfway through This is this is a good point Um, and I agree with you completely The the thing that it now Henry has lost his man inside He has no idea. Nick is still in the cat Nicholas cage still makes me chuckle every time I think about it. Um, But so now he's he's lost his man inside Dennis But the best part of the whole shootout for me Was home dude walking after he plugs him and he walks in on Henry He looks at Henry and says I want to testify and I'm like, yeah, it's just like Okay He's gone. He's but you see it come over him Right then when he clicks from crazy person back to Normal, I guess you want to say what does that do to Henry and maybe his mindset with his relationship to the town because yeah, they don't they already don't like him He's got mommy issues now He's still trying to discover what the heck happened when he was a kid Does he really kill his dad or when see what I want to interject throw this in there because I made me think of it when When Pang born tells Henry He said Henry did it. Yeah, we don't know what Henry did exactly. What did he do? Yeah, what? Okay, so maybe we don't know that Molly off daddy, right? So Did pain born asked dad like who did it just before he died and he said Henry did it Or was there just a random Just out of place Henry did kind of thing that that does has nothing because as a writer I know that's the kind of red hair and you want to throw out make people think and even Henry's believing in at this point He's like, you know comes and of course he got busy with Molly if you didn't see that coming Maybe he gets the truth from Molly. Maybe they get some conversation going and she tells him A little something more. I don't know. I'm I'm grasped. Yeah, but Molly Molly reminds me a lot of Pauly challenges and probably because it's cast a rock probably because she's got that necklace that That Pauly coveted Molly Pauly. I mean come on This is just so much that we can ignore I know I know Stephen King came out on twitter and said forget the Easter eggs and just enjoy the show But for me the best part of the show right now Are the Easter eggs because the story is all over the place. Will they bring it together? I don't know if these writers are that talented. We won't know until the end of it There are some people and this episode really polarized people I got comments like if you didn't like that one, you're gonna hate this one if you That this is the best episode yet. It just keeps getting better It just keeps getting worse you guys are on both sides of the fence and ain't nobody in the middle So it's either you like it at this point or you don't I think I'm kind of with I like it But there is the caveat that It's it hasn't gone anywhere and now we have almost an end Almost a season finale. Yeah, I I feel like they actually just kind of killed off the show at the moment like Okay, yeah, we're done like wait You have somewhere at this time It's also a lot of storytelling Uh significance in that killing off a big character like Dennis in almost the middle of a season And how shaken up you can make your audience with that But I think that's everything for right now. You got anything any closing arguments? No, no, we're good. We're good. All right. So until next time. I have any you've and you This has been castle talk with shell. We'll talk to you guys later. Bye. Bye


Hi everybody, and welcome back to our podcast from the Kama Sutra to 2020, where we look at your questions, your concerns, even your worries around all things to do with sex and sexuality. So unfortunately, once again, we don't have Dr. Anvita Madanbehel with us today due to medical reasons, but we have in her stead the fabulous Nina Claire. And I am absolutely delighted to welcome Nina. Nina is a well-being crusader. And in a previous of Tar, Nina used to have the editorial team for the well-being magazine at BW Business World, but she's shifted her focus since then. She's now looking at psychosexual dynamics in connection with mental health. And today she brings her perspective to the advice that the Kama Sutra has to give. Welcome, Nina. Thank you so much for the introduction Seema and thank you for having me. I'm so excited to be here. I'm very grateful to Anvita for connecting us. Well, it's a delight to have you because as you know that this is a platform where we try and respond to all the questions and the messages that we get around sex and sexuality because people have very few other places that they can go to for advice, for good advice anyway. And today we've got you on here because we really want your advice on a particular issue which I've been getting a lot of emails about and we haven't had a chance to address it yet. And that is that, you know, when you look at sex and everybody says or sex is a taboo subject, we found that within that arena of taboo, there are some things that are higher up on the list of taboo. And I think the top most taboo is women and pleasure, particularly as women get older. So we've been getting a lot of emails from people like women who are in their 40s particularly, you know, once you have a child, then you're just supposed to not be a sexual being anymore because it's like, you've got a child and you've got a teenage child and how can you be thinking about sex? Or women who have been in a marriage for a while and their husband is really, really bad in bed but they're never supposed to want anything different. It's always like, well, he's a very good man otherwise.  You're not supposed to want it. And when you become single, it's like, that's it. If you're 40 and single, you're not supposed to be a sexual being altogether. And actually you have a great term that you coined for that particular situation, don't you? Tell our audience it because I love that. So I coined the term recycled virgin and I kind of realized, you know, when I got divorced when I came back to India that when you're single, you're supposed to be absolutely virginal. When you get married, you're supposed to be a goddess in bed and then when you get divorced and you're single all over again, you're supposed to be a recycled virgin all over again. All over again. So that's what, and that's where the taboo really comes in, right?  That when you're in your forties, you're peaking and your desires are at their all time high and most likely you've been able to discover what you like and what you don't like. Yes, you're so right. Forties is when women really come into themselves and this is where judgment really starts to pile on on them. Sorry, I just wanted to say that I'm sorry if the light on me is changing drastically but as many of you know, I am in the UK and we are having a particularly cold and gray and awful winter and today just as I managed to sit by the main French windows by my garden, the sun has come out and all its glory and it's blinding me. So if you think that the light is looking weird, please excuse me for this. Nina of course is in fabulous cassoli at the moment. Nina, are you all sunny and bright and beautiful over there? It's been a bit of a tease today. The sun's been playing hide and seek coming and going but it's a glorious day over here today. Well, okay. So it looks like it might be a cold and sunny day over here. So I'm sorry. I'm sure it'll go behind the cloud any minute. So I'll just carry on. Nina, actually this is what I want to talk to you about today.   Now you said you mentioned that you are single, that you've been through a divorce recently, is it? No, no, it's been many years, it's been many years. Okay, so I guess what I want you to start with is let's begin with the idea that you're still in a marriage and the sex is really, really bad.  What do you recommend for women to do? I mean, a lot of them just go very silent but it's obviously playing on their minds. Absolutely, I've given this topic a lot of thought and what I've realized is over the years of observing people and having conversations and interviewing people is that it's not so much the what, it's the why. Why are we in this situation? And I think that we as women come from a position of being sexually repressed in society, right? And we're not really wired to communicate our desires, to express our desires. And I think if one takes a step back a little bit it's not even about expressing because so many times you don't even know what you want. So one of the things that I did when I came back to India because my background is in psychology and I'm a complete therapy junkie, I went into therapy and I tried to find myself, to discover myself. And one of the things that worked for me beautifully was that I did an exercise of listing down what my wants were in life, what I wanted. And I realized Seema that once my thoughts became clear about my wants, it wasn't so easy even to communicate them. So then you start with the smaller things that Seema if I were to ask somebody that I want a particular act in bed, I don't even know how to verbalize that. So started even the most inane things, the small things to learn to express oneself. So when somebody I feel is caught in a union whether or not getting what they want my first and foremost recommendation always is marital therapy, counseling therapy and without generalizing Seema, more often than not I've seen that men are not really up for therapy. Is that what you've observed as well? Yeah, unfortunately, there is this thing about it's almost like an espersh on the masculine. Most of them won't accept that it could be a two-way problem. Yes, so what I realized also is over the years that it's a function of your very being because your sexual life is a representation of your very being. So evaluate what's happening with you. I think that's where it really begins. And if your partners who need to go for therapy, you yourself go for therapy. And many people say they can't afford therapies then I say work on yourself, evaluate really connect with yourself. And the more you become in connection with yourself is when you realize what your desires are. So Seema, if I'm an aggressive person personally when I'm in bed and it's more likely that I'll be aggressive in bed as well. And if I don't have the vocabulary or the confidence to communicate in my day-to-day life, I'm barely going to have the confidence to communicate that in bed either. So I think it's a lot of introspection, evaluation, talk therapy, writing helps a lot is what I feel. And worst case scenario Seema, I mean, I don't like to stand on moral ground. So worst case scenario, if it isn't working then look at maybe opening up your marriage. And if that isn't an option, then maybe step out of the union because sex is a very important part of one's life and well-being actually. I agree with you. And I think actually this point that you made about seeking therapy, even if your partner won't agree to it. And if you feel you can't afford it or you can't go to it without more judgment being put on you, there's a lot of online courses and therapy sessions that you can attend for free, aren't there? Absolutely. In fact, today I received an entire database of 60 places where you can get free therapy. I was so impressed. Good morning. That's amazing. That is really fantastic. One of the other things that's been coming up in these emails is a lot of women saying that because they have young children or teenage children that it's almost like, how can you even be thinking about sex? You've got grown up children. What will they think? You know, I think it's rather unfair but that's actually the reality.   It still exists. So I think it's very important to assert oneself.  It's part and parcel of who you are.  Sorry, Sima. I mean, I have to make this point that it's really important that you don't look at yourself in a unilateral sort of way. There's so many facets to your being.   Doesn't mean that you can't be like a CEO of a company. You can be all of that together. They're all parts of your being. Yeah, absolutely. But I was also going to say that if you at that point decide to repress it, I mean, nobody's saying that you should go out and shout about it to your children or shout about it to the world. But if you repress yourself completely, I'm sure that impacts the psyche of your child as well. And then we're just perpetuating the narrative. Absolutely, because then what I'm telling my child is that Mama only has this facet of her personality. Mama works and Mama's a Mama and Mama's, you know, Nani Nanu's daughter and P2 sister, but that's not true. Mama's a woman. Mama has desires. Mama has wants.  And that is actually a conversation I've had with my daughter quite recently. She's 13 now. And I'm actually not allowed to talk about her because, you know, it's not, there's no consent over here, but I can definitely divulge this much that she's understood the fact that Mama is a woman. And that's happened very recently actually. So... Well, talking about consent, actually that brings me to the next thing quite organically.   And then this idea of consent becomes a little bit of an issue. Do you think? You know, I've really observed this, you know, Seema and shockingly, not only in people who are in their 40s, 50s or 60s when they're talking about being in unions where they're not comfortable and they're not being asked what they really want, but even in my friends who are in their 20s and 30s, that they don't know that it's okay to say no because we're brought up even the youngsters to be people pleasers to a large extent. So we're not educated. So yes, I'm very happy. I've been observing for years now that, you know, in schools one talks about good touch and bad touch, right? Where it's all over the place, at least in private schools that I see. But no one teaches you about consent. No one teaches you about the importance of being able to say no. Yes, say no to a bad touch, but if you're with somebody, it's like, no, you know, that person might feel bad. And I've had friends who in their first experience of sex, it hasn't been good and they said no and their partners have just plowed through. I think it's so awful. You're so right. I was just going to say that funnily enough, you know, in the Kamasutra, now people think that this is a book written 2000 years ago.  You wait either three days, six days or nine days before depending on the kind of person she is.  And yeah, you don't plow through, but we do have this terrible sort of idea that if you are married, the moment you become Mrs. So-and-so, you no longer have the right to say no. You know, Seema, I actually have a friend who's in her 40s and she has an open marriage and switch me, she has multiple partners. So she's open to that extent.   And she said, oh my God, Ninda, I felt so violated. Just because I'm open and I haven't opened marriage doesn't mean that I'm open to everything. He should have at least asked me.  And she had a problem saying no. You know? Wow. Yeah. Yeah, it really is an issue. I mean, I find that, and these things are so subtle that a lot of times when they're said in company, in groups or at a party or in a discussion between friends, a lot of people will either laugh it off or make a joke of it. And I think that it's so important because people are reaching out at that point. They've got this problem and it is something that's bothering them. So I think it's really fantastic that we've been able to address it today. And Seema, you know, I'd like to add to that. I feel it has to be a multi-pronged approach. So we're talking about, you know, how education is important. And that's how the narrative actually changes. You know, you and I are hardly going to change. We're going to probably, you know, include things in our repertoire, but the actual script changes when you talk about education, right? So education, not only at school, not only in college, but even your gynecologists. And if somebody reaches out to you and tells you that they have a problem, as a support system, family and friends should be educated about being supportive and that there are actual places where you can go for redressal. There are cells where you can go and get support if something's done to you against your consent. I'm talking in extreme situations. So if you're in a marriage and you go, you know, and your spouse has abused you or you're in a domestic relationship with somebody and you've been abused, you can actually go and seek help with the domestic violence wing, you know. I think that's a really good thing to know. And are these also like free cells and things where you can go? Absolutely, absolutely.  There was a lot of emotional violence and a lot of other stuff that had happened. And I thanked God a million times Seema when I went there because there were women and I'm not exaggerating. Women walking with their legs apart because they had been so brutally mutilated. But yes, it's a little difficult. I mean, it's the domestic violence wing. It's supposed to be, you know, a wing where anybody and everybody can go, but even to get entry sometimes is a problem.  Oh God, yeah. I tell you what we can do is that when we finish, when we actually put this episode up, we will put down some of the links to these places that you mentioned in the description. So maybe that would be useful for people. Okay, one of the other questions that's been coming up a lot, and this is really interesting as far as I'm concerned, that the guilt that is attached to wanting your sexual desires to be fulfilled and not just simple straightforward or I feel guilty because we are supposed to be good girls and good girls don't want this. But these are women now in their 40s, they're emancipated. Many of them are single women in relationships and so on. And yet this point of guilt is still coming through to the point where I've been receiving, you wouldn't believe how many emails from people saying that, you know, I'm fine with it. I'm a sexual being, I have a great partner, but my family is very conservative and now I'm starting to feel guilty about having sex on certain days. So if I've been to the temple in the morning, is it okay to have sex at night? Or if it is a particular festival or a fast, can I have sex on that particular day? And I'm actually blown away by this question because I've been trying to say to people, this is not a bad thing. It's not a sin. Why feel guilty? But this guilt is actually multiplying. Yeah, you know, if we go by our calendar, Seema, I think we're never going to have sex. But yeah, yeah. We have a festival on every day. Every day. But jokes aside, I think, you know, when there's a topic that's associated with morality, the higher the morality attached to a topic, the more the guilt that is attached to a topic. And when we come to religion, I think there's no logic. But I mean, I would invite whoever is writing into you to consider a modicum of practicality. That if you're engaging in sex, it's an act of love or passion or whatever it is. So consider the fact that there's nothing wrong with it. First of all, and secondly, an element of self forgiveness. I think we're so unforgiving of ourselves, you know? And I feel that, and thirdly, if you really feel guilty about it, don't do it on those days then, right? And I have to share, you know, when you started off by this whole like bad girl thing, that I mean, I love to party, I love hanging out with my friends, and although I can't handle alcohol much anymore, there have been days where I've been shitfaced surely, I've been walking the streets of New York, I've gone to crazy swingers clubs with my best friends and, you know, seen all the sexual acts possible, and stared and gawked and been like, and my friends been like, and I'm like, and I'm like, and I'm walking back at six, o'clock in the morning and I'm got my heels in my hand and I'm stuffing my face with a pizza slice and then I reach home and then I'm like, I will not be forgiven. I've sinned and it's happened to me that I've bathed and bathed and bathed and then I realized, okay, because I'm under the influence of alcohol, I'm feeling really guilty still. Wake up in the morning and the handover makes you feel horrible, but then I allow logic to sink in because guilt increases your anxiety, it affects your mental health.  So that self forgiveness, I think is really important. And believe me, God forgives everything. God loves everybody just the way they are. So self forgiveness and a modicum of practicality, I would say. I think that's extremely good advice. I couldn't have said it better myself. Self forgiveness and a modicum of practicality and just logic and to stop the guilt. I think if everybody from this particular episode takes even that one thing away with them, I'll be delighted because that is certainly the best advice that I have heard so far on this subject. Okay, now there's something else that comes to mind. A lot of people go on and on. Like I said, you know, when we started off about how bad it is, make you feel guilty, make you feel judged. And a lot of women out of sheer frustration will go towards celibacy. What do you have to say about that? I know we talked about celibacy.  If I'm allowed to say that. And so what happened was that, yeah, so my mental health suffered tremendously. I almost had like a nervous breakdown and not of my own doing, but one of my closest friends said, Nina, please make a vow of celibacy. And I hated her in that moment because she was weeping.  Can you just refrain? And I think the thing is when you end a relationship and it's a nasty relationship, it's so easy because you're damaged.  But that ends up being so much more damaging is what I found. So what I did do was I made that vow, Seema. I made that vow. And as weeks turned into months, I realized that more than a physical and sexual detox, it was a mental detox because it allowed me the power of discernment. So if I met somebody, you know, all right.  So it allows you this power of discernment and mental detox. So I recommend it to anybody, especially if you've broken up, especially if you're going through a trying time, it's wonderful to try celibacy. That's really good advice. We get a lot of people on both sides of the spectrum saying, is this good or is this not good? And a lot of people in recent times, particularly the younger age group and amongst men in particular, have been talking about this no-fap movement, you know, this thing of don't masturbate, be completely celibate and so on. But I think that the way that you've just put it really puts it into perspective. I like what you've said, that it's not about taking a vow of celibacy to do away with your sexual desires, which is what most people think it is, but to actually use celibacy just for a brief while, for a contained amount of time, to detox so that you can get better discernment, so that you can actually see things more clearly. You know, I don't know if you know Sarvesh Shashi, the guy who's the founder of Sarvayoga. So he's one of these startup guys and he's doing very, very well. He's partnered with Malayka Arora and I'm so bad with these names, Malayka Arora and her sister, whoever these actors are. So he talks a lot about no-fapping and especially for men, that it's supposed to sort of, you know, bring up your spiritual powers and your intellectual powers. I don't know about that, but I think that if you're being celibate, I think self-pleasure should not be ruled out because it's a wonderful benign tool for you. It doesn't take away problems, but at least it empowers you to not feel that sense of victimhood that comes with, oh, I'm single, I can't be satisfied.  So that benign tool that you have of self-pleasure, use it till that kingdom come, is what my advice would be to everybody. Nenna, that's been a really enlightening conversation and thank you so much for your openness in suggesting and talking about things. And I think, I mean, I certainly have got a lot out of it, so I'm hoping that everybody out there listening to us today will feel the same way. But just before we wrap up this conversation, there's a couple of things that I'd like to get your advice on for our listeners. So women in their 40s, single again, once again looking for relationships, once again being out there in the world, the dating scene has changed drastically. I mean, I find some of the terminology absolutely alien to me and I am in the field of talking about sex and sexuality. So what I'd like you to talk about is just very briefly address the idea of polyamory, which is suddenly really in fashion at the moment.  There's a whole different vocabulary around it. And there's a whole set of rules that have built up around it. Like you were telling me about something called ghosting. So if you could explain some of that for some of our, maybe not for the younger audiences who are listening to us, but certainly for a lot of the others, certainly for a lot of the others who wouldn't know about this. You know, so I've always heard about the word open marriages, but now what I've been seeing off late the last year or two is a lot of conversation around polyamory. And what that essentially means is that you're open to having multiple partners. You might have one person who you're anchored with, but then you're allowed to experiment. And there are many rules in place where if you're sleeping with somebody else, you have to inform your partner, there has to be a huge amount of disclosure is what one of the basis of polyamory is. And a lot of trust, a lot of exchanges, a lot of confidence in your primary relationship. So it's not about cheating on the other person but actually doing it with consent. With consent, openly saying, listen, I'm going out today with so-and-so, there can't be secrecy in it. There can't be secrecy because in the person who you're in a primary relationship in will be uncomfortable. But I have to say, you know, my personal take on this and my sort of opinion on this is that I feel that monogamy in the first place, monogamous relationships in the first place are difficult enough to deal with in terms of its work, right? And there's so many dynamics. So when you bring somebody else into the equation or many other people, I don't know for the life of me how the dynamics work because there's so many tangential sort of, you know, dynamics and conversations happening. But having said that, I mean, I'm noticing a lot of young people not only openly talking about it but even putting it in their bios on Instagram. So saying I'm poly, I'm queer, I'm bi and I think that's really brave. And I'd love to learn a little bit more about it and I'm open to my opinion being changed but so far, anybody I've seen in a poly relationship and without being judgmental, I feel it comes from a place of confusion because you're not sure about what you want because human beings, some people might say that you're born monogamous, that human beings are not supposed to be monogamous but I feel that we're territorial. I'm not okay with sharing my friends half the time. How can you be okay sharing somebody that you're not being intimate with? So that's polyamory in my opinion but I think it sounds very sex-citing but again, I feel like a really bad girl if I did it. And then sexting, Babaji will not forgive me but sexting I think is a wonderful tool and I advise everybody to do it because there's a whole amount of playfulness, there's a whole amount of fantasy playing that can happen that you start dropping messages in the morning, hinting to your partner, even if you're at home, it's locked down, drop a message. I wanna do this to you. I want to do that in the afternoon or just dropping hints here and there and you have this sort of psychological advantage of being behind the phone. So you can say so much more and but I think safety is really important. There's a lot of cyber crime that happens out there. So no distinguishable marks should be there in the photographs or videos that you exchange. Your face shouldn't be there and consent again, you don't want anybody sending you new deeds without permission. So please don't go around sending new deeds without asking and you said about ghosting, right? So yeah, I was just gonna say that this idea of sending little messages is as old as time itself. So that's sent in the morning to prepare for the sexual act at night or even to sort of keep the excitement alive for a week ahead is something that they did in the time of the Kamasutra. They just didn't have phones in those days but they sent little other types of messages. So yeah, yeah. So, you know, for instance, this whole idea of cinnamon, if you sent a little pouch of cinnamon, it had a particular message with it. It had a sexual message as a pouch of almonds had a whole different type of message to go with it. A piece of jewelry had a whole different, different type of manicure. Yeah, yeah, yeah. So, you know, we always advise people from the Kamasutra that if you get a manicure done, because love scratches were a very important part of your food play.  It's very exciting and tells them that you're thinking about them in a very exciting way. Wow, wow. Yeah, and it's safe as well. It's not traffic. Absolutely. Yes, yes, yes, yes, yes, wow. Okay. But you were telling me about ghosting. Yes, I have to confess, I'm guilty of it also. I think it comes from the, from two places. I think it comes from the problem of plenty and it comes from the problem of having difficulty, having difficult conversations. So there's so many options out there, right? When you're talking about the dating world. Tuck, tuck, tuck, tuck, swipe right, swipe left. I can block you. I can delete you immediately, you know. So the problem of plenty, you don't want to have difficult conversations. So you know what? If you don't fall in line with what I want or what I feel that I desire, then move over. So it's very unfair actually when we end up doing that to people and what actually ends up happening to the person that you ghost is that they end up feeling that there's something wrong with them. So what I really want to advise people who are at the receiving end is, it's nothing to do with you. It's to do with the person who ghosted you because they have the inability to have difficult conversations. Or even to commit, I guess. To commit, absolutely, absolutely. That's interesting. So ghosting is where you just block the other person after having had a few conversations and you think. Yes, so you block them or you stop responding and yeah, and it's very painful. It's very painful because you get no closure really. Oh wow, okay. Well, thank you for explaining the ghosting to me. And in closing, if you have to give three bits of advice to our listeners out there of how to just maintain their balance, how to look after themselves, what would you say? Seema, did you realize that my new platform is called Pursuit of Balance? That's so cool. I like the name Pursuit of Balance. Because I don't think we're ever completely balanced so we're always in the pursuit. So my advice to people would be love yourself, invest in yourself, be kind to yourself and to everybody around you as much as possible. Invest in your support systems judiciously because that's what you're gonna fall back on eventually.  And the most important I think is know when to walk away because that's respecting yourself and respecting your dignity. And I think that's extremely good advice. Thank you for that. And just one other thing that you mentioned earlier which really, really struck a chord with me and I'd like to repeat that in closing. And that is, as Nina says, use the platforms of therapy that are available to you. So if you feel you cannot afford therapy, there are free places that you can go to. We have listed some of these links down in the description. Take a look at them, use them. A lot of people have been judgy about therapy. It's like, oh, there's must be something wrong with you. What is this nonsense about therapy? It's not. It's about your well-being. It's about you feeling good about yourself. So use the platforms of therapy. The other thing that Nina mentioned was about using celibacy for detoxing yourself. I know that a lot of people out there are talking about celibacy and anophap and so on in many, many different aspects of it and many different platforms. But I love the idea of just using it as a detox tool. It's an amazing thing to try. It gives you more discernment. It gives you more perspective. Do it just for that. Do it for yourself. And on that amazing note, Nina, I want to say once again, thank you so much for being with us. We are really, really honored to have had you on the platform with us and we think that people are going to get a lot out of this conversation. Thank you so much for having me, Seema. I'm honored to be here, to be honest. Thank you so much. I really enjoyed myself. So as always, do comment, like, subscribe. Your questions should be addressed to info.seema.anand at gmail.com. If you need to get in touch with Dr. Anvita Madan-Behel, she is on anvitamadan-behel.com. And if you wish to reach out to Nina Clair for her advice and guidance, you will find her on Instagram, on Nina Clair, but spelled N-I-N-A-K-L-E-R. With that, have a wonderful week. We'll see you here next week.


So my question to each of the witnesses is pretty simple. Is the spectrum management process in the United States today broken? And I'd like to get an answer from each of you. Senator, I think it's hopefully on the mend. We're seeing signs today with the announcement of the new MOU between the NTIA and the FCC. I think we can ask policymakers to continue to take other steps to ensure that the discipline needed to address the technical issues that need to be examined and determined on how spectrum works and whether there's interference is followed. We need leadership to make sure that there's a long-term commitment for those agencies to be the final authority on the technical specs. And then to follow that science and figure out, as I think Chair Kentwell was saying, what is the need and how we make sure that all of those needs are met as best as possible. But we have the technical capability to do it through sharing, unlicensed, and licensed spectrum, but we need the cooperation.