Just a quick walk through our table of competence. First, we're talking about what is reproducibility, followed by a crisis in reproducibility and the consequences of that crisis, some of the causes and drivers of that crisis and the consequences, why it matters for you. I'm hoping this is becoming obvious about given the title of this session, this should not be coming as a surprise. And finally, time at the end for questions and answers. So there's plenty of options for interaction throughout, but there will be time at the end dedicated to question and answers. So if you get right to the end and think, wait, I don't understand this key concept, go ahead and share. And because you can do it through mentometer.com, it will be anonymous. So what are we talking about? Let's dive right in. Broadly, reproducibility is about getting the same results as published work or at least previous work. And exactly what counts as the same is tricky and everybody will define it differently. Some people want exactly the absolute same down to five digits past the decimal point. Other people are talking about, do you show the same general trend or pattern in the data? Do you get the same, does someone else's results with a completely new set of observational studies concur with previous results, that kind of thing. But broadly the same. And it's an essential part of the scientific method. In fact, it is what puts peer into empirical research. But a bunch. Joe, I didn't hear you laugh. I did laugh. Okay. Right. But really, like the whole point of the scientific method is that you sort of, you know, document your whole process and you put it out there so that other people can go, huh, I didn't know it would do that. And they can do it on their own. They can get the same results and go, he wasn't lying. Now, that might be broadly about repeating an entire scientific work from data collection right through to creating the graphs that you put in the published piece of work. Alternatively, it might be just about narrowly recreating an analysis. So you might accept their data and do your own analysis with it, maybe changing a couple of things and seeing does that really matter if the end result or these sort of inconsequential changes. So, you know, reproducibility has a couple of terminological distinctions that some people make. Repeatability or replicability is broadly associated with the data collection and experimental steps. So kind of the early stuff, how you got the data, you know, what experiment you set up. Whereas reproducibility is often more closely tied to validation and the data analysis or interpretation of the results. But I'm going to use reproducibility to mean any or all of these parts because they're not entirely separable and it doesn't really matter for the concepts we're talking about today. So I'm going to say reproducibility, it means all of this. You can shake your fist and tell me why I'm wrong later in the Q&A session. Importantly, reproducible research is like a good recipe. It is not like the technical challenge in Great British Bake Off. You see in this image from the Great British Bake Off, they were all supposed to produce the same thing but with like a not very distinct recipe. And clearly they've all come up with slightly different things. They're all different sizes, some are darker, some are lighter, some are round, some are oval. You know, that variability is fundamentally not what reproducible research is looking for. So a good reproducible research recipe should include things like data, the details of how you got it, how representative it was, things like that. The tools you use, including the materials, the software, the packages, all of the decisions you made after you got the data, so how did you clean it, how did you process it, recode it, all of that kind of thing. Your results in as clear and objective a way as possible. So not the interpretation, that will be very hard to reproduce. People just have to believe you on that, but your results, the descriptive statistics, the statistical models that you got, things like that. And access, if it is at all possible to the data itself in both final and raw versions, any code or notebooks or recordings, transcripts, all of these kind of things, if it is possible. Won't always be possible because of confidentiality reasons, disclosure control, but if it is possible, please share. So I want, before we go anywhere else to ask people, have you tried to reproduce scientific research? And these are sliders, so you can put them from, it was a complete failure to it was complete success for reproducing the work of others, reproducing your own work or never even tried. And I don't know where on that slide are you gonna put never even tried. That is I guess a complete success. But yeah, lots of people not even tried. This is encouraging. It's also interesting that some people in efforts to produce their own work have not had resounding success. That is a useful thing to point out. And that is in fact one of the reasons that reproducibility is so important. So yeah, we've got some good answers here. People participating. Some people definitely have tried to reproduce scientific research of others. And I find that interesting. I don't know that I've ever really tried to reproduce scientific research of others. I've definitely taken their methods and applied them to my own research, but I didn't test it on their own research question, their own data to check the method, I guess. Just kind of trusting that they know what they're on about. And maybe that isn't very useful. All right, so moving ahead. Let's take a moment for Q&A. So you can do this through Mentimeter or Joe can enter the questions you've answered in other platforms into Mentimeter.  Everyone will be able to see them, but they will be at anonymous questions and I will try and answer them. So maybe you're unclear on the terminology or maybe on what the difference between trying, you might have a question like reproducing your own research. What does that even mean? Is my research, of course, I reproduce it. I don't know. Go ahead and ask questions. No one has any questions. How useful do you think conceptual replications are? That's a tricky one, conceptual replications. I assume this means where you sort of, rather than try and reproduce the exact results, maybe even observational studies, you're never gonna get those same people to walk past you so you can observe them again. But you try and observe another set of people and produce broadly the same results. I think that is very important. And it is also very important to be as clear as possible in the original about how representative your data collection was or if nothing else, what decisions you made about your data collection. No one will be able to conceptually replicate your findings if they don't understand what it is or sort of what it's about. You're kind of without any hope of replication, you're just hoping people trust you and it'd be nice if we all trusted each other but I think we have reason not to all the time. Oh, hang on. Don't know how to see these other questions. Oh, there we go, Mark has answered. Why do you think reproducing the work of others is more successful than reproducing my own? I suspect that that's because when we reproduce our own research, we kind of trust our memory whereas when we're doing it from something else, we're really explicitly following an instruction that has been published. I could be wrong. It could also be that things that matter, you never thought to write down and so you don't include them in your reproduction. I don't know, I'd be interested if anyone else has thoughts on that and you can enter them into this Q&A as well. When we're speaking about scientific research, are you also referring to social science? Absolutely, all scientific research should be replicable to some degree. Social science is more likely to do things like surveys or questionnaires or semi-structured interviews or observational studies, things like that that cannot be replicated in the same way as maybe chemical studies where you add a certain amount of chemicals together in certain lab conditions and observe the result. They are different kinds of reproducibility but they're both very important. Actually, this is a very prescient question that comes up later in the session. Where would you recommend we publish the recipe? Is it something you would add as supplementary material to a publication, et cetera? In theory, the recipe should be the method section of your paper. Not all journals require a good method section and supplementary material, absolutely I recommend. That's where I would put your raw data if you're allowed to share that, your finished data if you're allowed to share that, any code you use to move from raw to finished data, yeah, any kind of software or absolutely. I am a big fan of putting Jupiter notebooks and data files and things like that on GitHub repose. Other repose are available, Google Drive in theory, Dropbox, things like that. You could make these publicly accessible so that people can just have a nosy. See what you did? You can be as explicit as possible with no word limits. So that can really encourage you to be clear. That looks like the end of the questions. Here's comes another one. So if it applies to social research, are you talking about reproducing the methods with different populations? That I think is what I meant with the first question about conceptual replication. If you're reproducing the methods with a different population, that is arguably more of a conceptual reproduction of the work, whereas a very strict kind of exact, exactly the same reproduction would be the same methods, the same populations. That's not always possible. We do get into this a little bit more about how social science research that is different maybe from hard sciences and how reproducibility differs. Yep. But yeah, reproducing the methods with different populations is still very important but getting the same basic answers using the same methods on similar, if not the same populations is also important. They come up with different things. One is more like validating the concepts and one is validating the methods. Do you see what I mean? So those are different things and they're both important for reproducibility. And I'll just give this one more minute to see if anyone else has a question. If not, I will move on. Yeah, I think you're not wrong to point out the difference between social science research and maybe some of the more classic hard sciences research and how reproducibility is a bit of a contentious issue there. So I'll go ahead and dive in to specifically social psychology, that's a journal that set out to reproduce a bunch of the classic published articles from like the 1950s. So all of these psychology and sociology and sort of behavioral studies that had been published in the 50s, 60s, 70s, 80s kind of thing, sort of in the early 2000s, this journal thought, Peter would be really great if we deliberately set out to reproduce all this stuff. Unfortunately, they couldn't reproduce most of it. It was really embarrassing and it came to be known as a replicate. Now, more than, more recently than that, people noticed this replicate thing and they reached out and they interviewed a bunch of people and they really tried to find out what's going on and they found that 70% of researchers had tried and failed to reproduce another scientist's experiments. And that's probably broadly reproduced, maybe the method, maybe the population, maybe the findings, in some way they tried to reproduce. But 50% of those had also tried to reproduce their own experiments. So these are all failures as well. It's not just that they should try to reproduce it and it worked great. 70% had tried and failed to do someone else's work again. 50% had tried and failed to do their own work again. This is why in that poll I ran early on, becomes important. Furthermore, very few of these people who did reproduce someone else's research of their own tried or failed or succeeded, very few published their efforts. Almost no one published failed efforts. A few published successful reproductions. It's very odd. And moreover, very few contacted the original researchers with questions about why their reproducibility efforts had failed. So it just, this sort of adds to the replicate thing. It's like something's going on here, even with ourselves, where we know what we were doing. Hopefully. We're not really telling anyone what we're doing about our efforts to reproduce and we're certainly not asking the people who should know about our efforts to reproduce. In response to replicate and in fact, lots of other disciplines, not just social psychology, other journals, other fields, they sought to find how reproducible are we as a discipline. A lot of people found it's not great. And so there's a lot of more recent activism or movements about open science. There's a discipline called meta science, which is basically the science of reproducing science. It is quite meta. And webinars like the one you're attending right now. Now, the sort of crisis of reproducibility has consequences. A big one that I think you might feel strongly about is that when data that cannot be reproduced is taken up and accepted as true. For example, Alicina 2010 paper was published as sort of proof that austerity would help struggling economies grow by lowering state expenses. And it was accepted by governments that thought, oh yeah, that's what I believe. So I won't try and reproduce it. I'll just accept it because it's published scientific papers like pretty true, right? Accepted it and applied it. And since then, people have not been able to publish it and real world experiments have not agreed with the concepts that the paper suggested should have happened. Beyond that, of course, wasted time, money, research efficiency, lots of things where people say, oh, I've got this brilliant new idea and someone else tries to says, oh, that does sound brilliant. Let me just find out about that. And it turns out not to be a very brilliant idea. And there's a loss of trust.  Well, yeah, there's just some mistrust about what publishing work means, how much it can be trusted, what you can really believe or take away from it. Some real consequences for society. There are also specific consequences for science and scientists, including of course, the wasted time, money, and research efficiency. This all really matters when we're, you know, trying to cling on by the skin of our teeth to research budgets and get things done and get things published. As consequences for a research culture that can become distorted by pressures to publish shiny new things, even if they can't be reproduced, there's lots of trust for specific scientists, which also comes out as reputational damage. So researchers might, you know, if they publish something flashy, can't be reproduced. People either like malign the original research or they align, you know, malign the people who tried to do the reproducibility efforts and there becomes like fighting intention between who do you agree with and whose camp are you on. It's not very helpful for collaborative efforts to achieve things in society. That's my opinion, I'm sticking with it. So let's talk a little bit about what's driving this. Blatant self-citation here. I did an article about how alchemy now. And a big part of alchemy was that you could write down interesting sort of recipes for how you achieved the gold from lead or a new kind of blue dye or whatever it was that your alchemical goal was, but you tried to do it in code. So you sort of used a lot of metaphors and sort of symbolism and like internal aid memoir rather than really clear instructions for someone else to do. And that's still following true, I think a lot of researchers, especially in sciences, like maybe social sciences, I don't know, maybe I can't back that up. It's still true that a lot of researchers kind of want to be the only one with this magical knowledge, that we have the solution and you'll just have to believe us because we are who we are. And that's not necessarily sharing the knowledge, it's protecting knowledge, it's protecting your position as the person with the knowledge. And here's our committees leaping out of a bath. Who gets the credit? A lot of science has a problem with wanting to give credit to the first mover and not to anyone else, even if subsequent research validates it or extends it or applies it in an important new way. So the issue of credit becomes really important, especially with funding and grants and position and status and it's, these are longstanding problems of research culture. Now, there are of course modern problems within research culture. So ignoring alchemy in our committees and all of that, there's problems of how we educate people, like how much emphasis do we put on the value of reproduction within the scientific method? Are issues about hiring and promoting? Do we hire people with all the flashiest publications? Do we hire people with the most reproducible publications? Do we promote people who train their PhD students to produce reproducible work or to reproduce the work of others? Or do we promote people who have the widest number of researchers who out there grab in the grant money? It can be a distortion in modern cultures. While publishing, obviously, someone earlier in the Q and A brought up where are we gonna publish this stuff? Yeah, a lot of journals you'll find will not be very interested in your paper if you are successfully reproducing something that maybe is a little bit old, even if people have accepted it as true and you're reproducing it saying, yes, it is true. They're like, ah, we already know, they don't wanna know. It's a problem. Similarly, funding and post-publication engagement, you know, how the question about how many people have reached out to a research if they were unable to reproduce their results. Some researchers are really pleased with that. They're like, oh, you thought my idea was interesting and you're writing to me and I feel really, you know, noticed this is great. Other people are like, why are you bothering me? I'm really expensive. You know, my time is so expensive and I have better things to do and I've got three grant applications to write my lunch. Get out of here. It's an issue, but a big issue is questionable research practices. Now, some people consider questionable research practices to be just how it works if you look under the hood.  You don't wanna see how a research publication is made. Other people say this is soft fraud and it needs to be rightfully exposed for being problematic. Questionable research practices could include anything from not sharing your data properly. I'm not making your raw data available when it is safe to do so. Write through to cherry picking results and sort of massaging statistics and, you know, really problematic stuff. So here, please, you know, move. Tell me if you are aware of how felt pressured to have heard of anyone else doing questionable research practices or soft fraud. Let's just give some examples. You don't have to admit to having done it yourself. This is all anonymous. I mean, right up to falsifying data although that's pretty hard fraud. Let's see. This doesn't, I thought this was supposed to be a Q and A session panel but it seems that something has happened with my mentor meter and it is not allowing me to enter them. So if you want to enter them into the Zoom chat. We can still enter them. So it's your answers, it's a short question so it'll still pop up for you. Okay. Oh, I do have a question here that Joe's shared. Let me just check and see if I haven't missed any more. No, just one, a question. Were those non reproducible studies where an experiment cannot be repeated or where all data cannot be used to repeat the analytical results? All of this, basically the non reproducible studies from the social psychology era, some were like fundamentally flawed as in the experimental methods are highly unethical and we just are not allowed to do that to children. We're not allowed to put them in a room and shout at them until they cry and see how long it takes them. That's that kind of thing, not allowed. Those are unreproducible for a clear reason. Others were unreproducible because, yeah, the data was just inaccessible. You couldn't be repeated because the categories and subjects that are no longer accessible, something that might have been representative of the majority of people living in a place at a time, populations have changed, those people are no longer representative. There's all kinds of reasons why they may or may not be reproducible. But yeah, some of these reasons that you're producing as well could be reasons that those studies were not reproducible in that they were not very honest about the data sources they used. They were not reporting all of their findings and this is one of the distorted research culture issues is that there's a real problem that if you do research and you get a sort of non-result, you find like there's no interesting interactions between this condition and this output. Nobody wants to publish, nothing to see here. Like that's not a very flashy journal article but by not publishing that, someone else might come across the same question and reproduce your efforts, also finding a non-result. You'll have no idea that they concur. Both of you will have wasted time. It's not helpful. Hiding data processing steps which led to a result. This is a very good one. Do you mix two, collapse two categories together? Do you throw out outliers? Things like that. Retrospective control group, that is a very good one. I like that because yeah, we don't always have control groups. Certainly we go exploring data sometimes without a good sense of what the control group would be. And if we find it later in the data by finding who meets this criteria, everybody else will count that. It can be very questionable. Might be the only possible way to do things but you do need to be clear if that's the only possible way. Not including caveats. Yeah, selection bias. Selection bias is a real problem. I know everyone knows it's a problem but the fact that people aren't saying it's a problem and therefore they can't counteract it in the future. P values, tweaked P values. That was a huge part of the replicate is that people were really throwing out individual data points and things so that their P values were certainly significant and that's not allowed. You're not allowed to fudge the input to get the right output. Inappropriate stats test, yeah. You should in theory justify why you make all the choices and decisions that you do, why this test and not that one, why these factors and not those. Not including copies of data collection tools, e.g. surveys. Yes, that's very clear. If you have no idea what they actually asked on a survey, it's very hard to feel robust that their conclusions match your intuitions. That's really a tricky one. Not fully informing participants, that's a good one. Yeah, informed consent is definitely something that is much more recent and therefore we may not be able to reproduce old research because under modern ethics standards of informed consent people would not be as honest or they would not be as, yeah, they wouldn't answer in the same way. Again, implied consent, that's good. Shared method variance, that's absolutely true. Even within really hard sciences like chemistry, not every bottle of reagent is exactly the same as every other bottle, there is variance in there. Sometimes you get a bad one, it happens. Okay, super. Well, it doesn't look like any more are coming in. So these are all really good examples and show that you are aware that questionable research practices out there. Certainly some of these are definitely much more fraud-y than others that they're easier to see as fraud. But even the ones that are well-intentioned, you would be best to be clear about how you got the things, what decisions you made, why you made those decisions that people are not always clear. Okay, so let's move on. So to round up a few reactions to replicate and some of the other things that followed afterwards in other fields, not just social psychology, some people responded by saying, really it's just a misunderstanding. This whole crisis is overblown and the changes that you wanna make, they're too broad brush, they don't take account for the specific things that matter to my field or my discipline. Some people have even suggested that if someone is going to try and reproduce it published research, they should not be allowed to do so without the permission or collaboration of the original author. I shouldered to think that someone thought that was a good idea, but their theory was that it's so hard to be specific in the method sections of journals that there is no reasonable way you can really describe your method properly and people if they want to reproduce your method should come to you and really hash it out in detail. Bonkers to me, but somebody said it. On the other hand, a lot of people have said, let's call for a revolution, let's make soft sciences more like hard sciences. And some people of course have suggested market-based solutions like incentives or apps or journals that only publish reproduction research, things like that. So let me just take a quick reaction from you all. How big a deal do you think this crisis really is? Is it? So on each side is who is responsible for this being a problem? So on the top you might say it's a minor deal, but the original researcher should make the changes that to resolve this minor issue. Or you might think it's a major deal and really the reproducing research has bulk of work to do here. Or you might say it depends on the field in question and how that plays in. Yeah, yeah, this does not surprise me. Oh, very few people believe it's a minor deal. That is reassuring. I'm glad to hear, especially since you are attending this session that you do not think it's a trivial nonsense. That is encouraging. Okay, so I've got some good answers. There's a couple of pumps here. I can see that some people think the original, even if it's a major deal, some people think the fault lies with the original researcher and some people think the fault isn't quite right. The responsibility for making improvements lies with the original researcher and others for the reproducing researcher. The same kind of mixed response applies to you. It depends on the field in question, presumably because different fields, sometimes the original researcher has to do some more work and sometimes the reproducibility, yeah, the reproducing researcher. Good, glad to see that we agree that it matters. Okay, so here's a time for some more discussion here. And I want you to answer questions maybe about that it depends, you know?  I deal with personal data and vulnerable adults. It is not appropriate to publish all of that. You know, maybe you want to tell us about that. That's an example of why it really depends on the field. Alternatively, maybe you agree, like social sciences, but they're just too soft, too hand wavy. We need to make them more like hard sciences. Maybe you think the responsibility for making changes, it's not the original researcher, it's not the reproducing researcher, it's the journals. They should do a better job of enforcing standards about how methods sections go. Go ahead and ask any questions here. In fact, you can ask just questions. You can ask questions about anything in the workshop so far, not specifically about. About just the last slide. I do realize I'm speaking quite quickly and maybe everyone's a little bit like overwhelmed and therefore has no questions. It's all right. If we don't, I'll leave this for just another minute. And we got one. As a journal peer reviewer, I've never been asked if I think a study is reproducible, but I have been asked if research is novel. Yeah, this is a really good point. This exactly points to the distorted culture possibly. Novelty is important, but so is reproducibility. And the fact that people are specifically asking about novelty, but not reproducibility does suggest a bit of a distortion there in how journals approach new research to publish. That's a great point. Thanks. I will be working with interviews. So I would expect a reproduced piece of research to draw a lot of the same conclusions, different experiences will come up. Is that worth noting in the methods? That is worth noting. Absolutely. I mean, we all kind of know that if we know anything about how interviews work, but in the sort of discussion and conclusions part, you can absolutely say it's important to note that these are the lived experiences of these specific people. And hopefully they're representative of wider groups within the culture that their conclusions, their sentiments, their experiences are generalizable. It's worth noting that other researchers may not find this, that maybe they're quite unique people. You can say that as both a pro and a con, that other people won't be able to reproduce the exact interviews. But on the other hand, you've captured something very important to Novel, and we can all look at it critically and discuss what does this mean? How do we go about using this information to better society or to learn more about society? Thanks for that. Do you think journals with small word limits need to make authors publish additional notes on methods results somewhere else? Gil's details here are the first to go with small word limits. I absolutely agree. I think journals should really require either supplementary materials in the way of data and code and the status do file. So there are scripts or things like that that the people use to produce their graphics. They should be asked to include that. And if they do not include that, they should be asked to make a short statement about why they have not. It's entirely appropriate that someone would not be able to include original data. But they could say, I've included a short synthetic data set so that you can run my R scripts and get similar graphics. I would like to see more of that. And I do feel that small word limits, you have a nitty gritty real detailed stuff in method sections. It's not super. It's not what most of us want to really spend our time reading. I can see why it gets the boot first. Some data is not very accessible even with reproducible methods. How could we solve this? Well, I guess it depends on the data. But yes, I mean, it's absolutely the kind of thing where you could publish good supplementary resources and say, here's a data set, a cleaned or safe to publish data set. And here's some of my R code files. And here's how you can reference the data and play around with it and show how I got this results and how if you make different changes you get similar results and why I chose the one I did instead of those. It does involve more work and it does involve more interest on the part of the other person. But it's going to resolve some of those issues about trust and reproducibility and generalizability. It is certainly one solution. Absolutely talk to me in the Q and A after if you want to talk specifically about what kind of data you have and whether you're concerned about how to make it more reproducible, we're happy to talk.  So this one's, yeah, this feels like a reproduction of the question we had before. Yes, small word limits, the methods being the really dry portion is going to get the boot. I believe journals should strongly encourage if not require people to publish additional resources. And academic work does not get published. One of the factors involved is the perceived robustness of the data. This is a huge problem in publishing and I think it's one of the reasons that there's a distorted research culture. It's very hard to know the real reasons why academic work does not get published. I've had loads of things returned to me saying this falls outside the scope of the journal. And I find that confusing because how are people ever going to do novel research? I consider my own research by novel, but also reproducible because I make the files accessible. How are people ever going to get novel research published until something is well enough established that there's a specific journal for that field? It's a bit backwards, but I don't know, cynically it could be that they didn't want to publish my research because one of the reviewers has a very different opinion to me, just ideologically didn't like something I said in my paper, or cynically you could think not established enough or not male enough or not wealthy enough or whatever it is, whatever factors you think give someone's prestige and status and therefore more likely to be published in a given research culture. It's very hard to know and transparency about who gets published would be great. I like it, I think it would be good. I don't think it, turkeys are gonna vote for Christmas. Is it required to link to full transcripts of focus groups? I work in a sensitive area and moving words and phrases to enable this would be a huge task. Do you think linking to coding's et cetera is enough? Generally, probably linking to coding's would be enough. That said, there are computational methods to make full transcripts safer. You know, if you use, if you're going by hand and changing things, removing words and phrases that is absolutely a huge task, I do not recommend you do that, but you might look at some computational methods to make it much easier. And if that you think would really make people trust in your research or engage with your research more or make it more applicable, you know, do you think people might find new and interesting things in your data? Then maybe that's worth doing. But in the short term, linking to coding's is probably enough. It's certainly better than linking to nothing. Thank you very much for that. That's a good one. That's a really interesting approach to a unique kind of data that I think most of us would just assume can't be reproduced. Nothing to be done, but you found two ways to do it. One, which seems very manual and heavy and not recommended to do manually. And yeah, the question is if sensitive data is used for a study, could a synthetic version of that data be used to reproduce the study or vice versa? Or does that entirely depend on fidelity of the synthetic data? I would say if you're trying to exactly reproduce the specific, you know, graphs that you produced from your data, that comes down to the fidelity of the synthetic data. Not a bad thing. But yeah, that kind of one for one, like for like, but completely synthetic data would be used to reproduce your entire research. That might be useful if you want to demonstrate not only how you got, you know, what raw data looked like, all the different steps that you took to recode that data or, you know, combine subcategories and things like that. That might be quite useful just as a clear demonstration of your work process. But yeah, maybe just a simple example of a similar-ish, you know, not very one for one, not a pure replica kind of synthetic data set would still be useful to demonstrate, you know, your analysis process. So I think it's, if you're using very sensitive data, a synthetic version would still be useful just as an example case so that people can really see, here's what she got as an output from the data collection. Here's what she did next. Here's what happened after that. Here's what happened after that. Here's the final result. That's still very useful. Joe has shared, I can't stop thinking about that example of transcripts from interviews. How is public social media data different? What would it take to make that data open? Yeah, I mean, Joe is on quite a kick recently about social media data. What are the ethics of using social media data? Like, is it personal? Is it public? Like, how useful is it? He says to ignore his spiral, but no, I won't. I will join your spiral. And it's tricky because you could argue that what people say after they've signed, you know, an informed consent document, they ought to know that that what they say is being used much like people know that what they say on Twitter could be used because they have to sign something saying, you know, I'm over 18 and I agree to these terms and conditions. It's kind of an informed consent. It's not great. It's also very easy to forget that informed consent when you're furious about somebody on Twitter. But yeah, I think certainly if you were to look at lots of individual tweets, but you didn't want to share any real tweets because you were concerned about ruining the lives of the people who would be identifiable from those tweets. You could create a synthetic version and use that in all of your examples. That way you could protect individuals while still being able to talk about the process and the kind of results you found, the kind of things you got. Summer shares that she sees some analysis of tweets. Yeah, I mean, we could look that up and see how they treated their tweets today, hide the names and all identifiable information from the tweets where they just talk about big sort of descriptive qualities of a body of tweets or where they talk about individual things that individual people said. It's quite complicated. There's a lot of ethical implications. That's one of the reasons that being reproducible is important because if we think, well, they only got that result because they did something very unethical, well, we ought to know moving forward. Oh, no, the cat is in the toilet. But metaphorically, it's because hard sciences are not exempt from reproducibility problems. In fact, all of those results about the 70% of people that could tried and failed to reproduce someone else's research and the 50% of people that tried and failed to reproduce their own research included physics, chemistry, biology, medicine, all kinds of things that we think ideally should be hard and reproducible and well documented. After all, biology labs have lab notebooks for a reason to improve reproducibility. Well, it turns out they're not working or at least not working as well as people assume they're working. There was a study in which very high impact journals taking on board this crisis of reproducibility made really clear efforts to ensure that work published in their journal, their high impact journal was reproducible. And analysis of that showed that there was some improvement but that it was not perfectly reproducible. So even people who have all the respect and all the power and potentially, all the money, I don't really know how journals get their money. Even when they were well motivated to make it reproducible, they could not make it fully reproducible. And what's more is lots of people in response to the sort of crisis of reproducibility debates are really resisting. They're saying like, you don't know what you're talking about. Maybe your solutions work in chemistry but they weren't working my field or maybe your solutions will work for students but not for me. I have these kinds of demands of my time, whatever. Nobody likes being told that you're doing it wrong and you want to do it this other way instead. Sometimes for very good reasons because of that other way is irrelevant but sometimes just because people are defensive and grumpy, I'll admit to that myself. So here's some solutions that dig a bit deeper. A lot of people and all of the citations that I got these solutions from are in the sort of lower right quadrant, the one with Mac 2014, all of these articles. A lot of these solutions were repeated across these articles. A lot of people have come up with the same solutions. A lot is better training. We can encourage students rather than trying to do novel, interesting, brand new research from the jump, get them started with reproducing research that gives them practice of established methods, established research questions, established analysis tools, things like that. Also potentially produces something quite interesting. Maybe this has never been replicated before. Maybe it has been replicated several times but yours failed and it's interesting to look at why. Better mentorship, if principal investigators demonstrate good reproducible practices. If we encourage our students or early career researchers if we give them experience doing things, they're more likely to do those things. Bonkers concept, I know. There's the idea that people should budget in their research time, a certain amount of time to replicate themselves or that research teams ought to pair people up so that someone else within the research team makes an effort to reproduce your stuff. Likewise, there are some journals out there and more proposed that publish nothing but replication studies. So basically making it mandatory or at least highly encouraged and rewarding to do the replication stuff. Yeah, someone's recommended. Plus, they have a replication journal. Consolidate or standardize methods, materials and protocols. Now this one's tricky. This is one that works maybe better for some fields and not for others. We're not going to all agree on what's the right software and moreover software changes quite quickly. So something that you used to work in this version doesn't work in that. You have to do it slightly differently and maybe that matters. It's not like reagents in chemistry.  That's not how it works. But still, protocols, you know, making sure that your interview questions are checked by several people. That's a good protocol. Making sure that your selection bias process has been reviewed by someone outside of your immediate research team. That's a protocol that you could do. I mean, similarly, more time, reward, support incentives. If you're going to require that people replicate their own work or a buddy's work, someone has to pay them for their time. They're not going to replicate their buddy's work for free. Publishing practice, so again, journals, but also, you know, changes to how method sections are done or that articles have to require supplementary materials, things like that. Shared data code, et cetera, even if the journal doesn't require it, you can make it, you know, accessible. You can put up a GitHub repository. You can create a DOI for your repository so that people absolutely have the version of data and codes that you used. They can get it. It's reusable. It's, you know, it's out there. It's a marker. It's a flag saying, this is what I used. You can use it too. Attitude change. This one's a hard one. We've got to stop demanding to be our comedies leaping out of baths. The sole discoverer of something that probably everyone already knew. We've got to collaborate. We've got to value sharing information rather than keeping it to ourselves like how precious. Field generated or field specific solutions. We have to accept that what might improve reproducibility in one field will not reproduce, not improve reproducibility in another field. There are issues to concern that are make some research different than others. But I'd like to hear your solutions. This is always interesting. If feel free to, you know, expand on some of the solutions that I've already suggested here or you could come up with completely new solutions or you could share solutions that you know don't work. Please mark those as known to not work. You know, it just like publishing negative results. If you have a known failed solution you can share that with us. So yeah, tell me a bit about if any of your research teams or your universities or your institutes have imposed anything.  Have they demanded that you follow a checklist when writing up the methods section? Have they given you time to replicate someone else's research within your group? Update informed consent collected to enable access requests from certain journals and groups. That's a good one. Yeah, because absolutely when you interview people you want to make sure that they have consent to be interviewed.  It could absolutely be part of original informed consent. It could be a secondary stage of informed consent. It kind of depends on your field but that's a really good one. I like that. Do we have any other suggestions for how we might go about starting to fix the crisis of reproducibility? Because we're not going to fix it completely. If I did, this would be a charged workshop. Could finders play a role in imposing, assume that means funders, play a role in imposing reproducibility, e.g. fund smaller reproduction studies after original? Yes, they absolutely could. They could make it a part of their funding budget that someone from outside the team, that they have to pay someone from outside the team to reproduce the studies or that you know, a certain amount of some of the researcher's time has to go on reproducibility. They could again just fund reproduction studies. As far as I know, not many people are funding reproduction studies. There was a couple. There was the University of Toronto where something like that would give $5,000 to people who wanted to publish reproducibility studies. And yeah, that's not nothing. And maybe for early career researchers, that's a very valuable kind of fund to win and they can put their effort there. It's great. My college at University of Exeter are going hard in regards to the reproductive replicability crisis. Pardon me, let me just take a sip. They have workshops to talk about it and encouraged to put everything in OSF. That's Open Science Framework, I think. My first study in PhD project will be a replication of very recent work in my field. This is great. This hits a couple of the points. So the workshops is about training and mentorship. I'm setting an example. Allowing part of a PhD project be a replication sort of is changing the culture of what it means to do novel research. Replicating things can be considered novel research. If you're replicating it absolutely one for one, it's hard to see the novelty there, but it's sort of an accumulative confidence. And the more confidence you get, that's still novel. But if you reproduce almost all of it but apply it to a new population, that's also novel. It's very good. Thanks for that. Maybe we should teach more coding to ensure people are analyzing data in R, Python, et cetera. This means researchers would have data scripts which can be checked rather than hoping people selected the right buttons in SPSS. I like this very much. I would give you a sticker if I could because this is absolutely, I think, one of the best ways in the near term to make clear gains in reproducibility is to share your R scripts or your StataDue files, things like that, to show people the steps you followed exactly in order so they can do it too. Also, it's much easier than trying to write that out in a method section. So it's a win for everyone. Encourage journals to formally withdraw articles found to have used fraudulent data. Until this happens consistently, there's no penalty or no perceived disadvantage. So people just submit and publish with little consequence. Ooh, that is contentious, isn't it? How are they not required to withdraw articles? I mean, I've heard of articles being withdrawn after fraudulent data was discovered. I didn't know that was just a choice some journals could make. That is crushing. But again, yes, this points to the culture, the research culture and the publication culture. There's real pressure to publish or perish thing. If people think like it's fraud or perish, probably a lot of people are gonna use choose fraud. So we need to make that not an option, both because the fraud will not be tolerated, but also because research that isn't shiny and new and totally original and flashback and wisdom is not the only way to get published. So very good point. Thank you for this. Yeah, Joe also agrees on the point about teaching coding as a solution. He specifically says that SDSS has a playbook style report generator, but a lot of people are still using Excel in academia and Excel is extremely hard to document what you've done, what choices you've made. Have you sorted this field? Have you moved these columns the other way? Have you recoded the data? There's kind of no good way to capture that in Excel. Are not very good ways in SDSS. There are much better ways in R, Python, and stuff like that. And of course other things that I've just learned about Atlas earlier today, and I don't know if it's any good, but I assume it's probably better than Excel. All right. Well, I'll give you another quick minute here. If there's no other suggestions or failed suggestions or just further points you wanna elaborate, I'll move on to my big hitter suggestions, which they're not unlike some of the ones you've put up to here. My big perspective is that I want to redefine the problem and goal. 100% reproducibility of absolutely every single thing is not the goal because that will alienate a lot of social science researchers who think, well, I cannot get it 100%. Therefore, I'm gonna completely sidestep the whole thing and ignore reproducibility as a goal at all. Instead, what we want to do is be as clear as possible about what we did and why and how our main findings and conclusions might be replicated or validated or supported or applied to new areas of what it means if they are or are not found to apply elsewhere. Go cautiously. This is another one about reproducibility. 100% reproducibility is not the goal. If someone finds an amazing outcome and they publish it and someone else tries to get it and fails to get it, it doesn't mean that the first people were fraudulent or using questionable research practices. It might just be a fluke. Sometimes that happens. And sometimes you misinterpret the data. It turns out to be a null result. It's still published. It still promotes someone else to do work. And the investigation continues. And maybe you end up with something very valuable at the end of it, but it's not 100% reproducibility of the original study. Let's go cautiously. Be curious and critical about things. Embrace complexity. This ties in with that is that not everything will have a simple answer. Like we're not going to get the same answer every single time. And also embrace collaboration.  It's not how science works today. Arguably not how science ever worked. And it's just sort of a historical quirk that we think so many things are discovered by a single person. Strive for open science. So strive to share knowledge instead of hold on to knowledge. And that's for the betterment of society, arguably. And that's a bit altruistic and maybe we're not all equally altruistic, but if we make open science rewarding, then even non-altruistic people can get it behind it and automate the boring stuff. So this is about using R or Python or Stata 2 files, things like that to make it easy to do as much reproducibility as possible. To be as open as possible, to share as much as possible. It does not have to be a tedious extra chore. It could be a bit of extra work, but it actually makes your process much easier and faster because when you go to reproduce your own stuff, it will be so much easier if you think like, oh, I've produced this wonderful chart, but it's not in colorblind color scheme. I'll do it again. If you don't have those scripts and files, you may not be able to reproduce your original chart and that's gonna be a bit embarrassing. So it's for your benefit, it's for everyone's benefit and it looks really cool. When you say, oh, yeah, I've published all my code on a DOI to have repository. I mean, it's cool to make, but then I'm that kind of person. Here are my references. These slides will be available after the talk because I doubt you're gonna want to take a picture of the screen, that's weird. And you're not gonna be able to read all of this all along. Here's my contact details. You can talk to me afterwards.


A major fire burned a large church in Burlington, New Jersey to the ground on March 20. Its roof collapsing is more than 100 firefighters and emergency personnel battled the flames, local media reported. The blaze erupted at about 6 p.m. Eastern Time at the Fountain of Life Center in Florence Township, New Jersey, northeast of Philadelphia, local ABC 6 reported. Dark plumes of smoke could be seen for several miles around. It's a devastating loss, Russell Hodgins, a senior pastor at the Fountain of Life Center, told the station. Hodgins told local media that crews were still attempting to save an attached school building. More than 100 firefighters and other emergency responders were at the scene. Firefighters did a tremendous job of trying to stop the fire interiorly. It was just the size of the building the amount of fire that was here, we had to switch our operation to a defensive operation, Robert Tharp, the administrator of the Florence Township Fire District, told CBS 6. There were no immediate reports of injuries and it was not clear what ignited the blaze. The Cherry Hill Courier Post said the church, which opened in 1974, has 1,000 members. It's campus includes a school, worship space and family life center, the newspaper said. A cause has not yet been determined, the newspaper said.


What's up everybody, Jason here for jaspyscasebrakes.com. This was our last and only filler, guys. We needed to unlock the last seven spots in the Big Hit Express case break that we have going on, guys. Series six, guys, usually annually, once, twice a year, we bring back our repack product that we create here, which is called Jaspys Big Hit Express Multisport. So there it is, it is now live today. And like I said, the last seven spots will be given out in this break here. So for you guys, you guys are paying a small fraction of the price to get a chance to win up to seven spots in that break for only $35.99. Now, we're doing a one box legacy break. So of course, we'll do the break first, randomize the customer names and the teams. Then at the end, we'll re-randomize just the names I bought in, and the top seven get into that Big Hit Express will be coming up in a little bit, guys. So here we go, let's do the break first, roll it. It is a one, a two, three times, good luck. One, two, three, three times, three, three. Kev down a Roy. Three times, three times. One, two, three. All right, Colts down to the Steelers. Now again, guys, of course, this is just a filler. What's up? Oh, are you the miller? No, I'm just doing a filler box. Don't think so. Let me ask you to print out my list. Oh, yeah, I'll print it out right now. Is it gonna be on the same Jaspies case? It should be. Okay, awesome, I'll do it right now. I hope so. Yeah, I mean if not, it's just probably on another Jaspies account. Well, I'll figure out a way to connect it to it. Well, yeah, we should have one of those over there too, eventually. Yours was update baseball. Nice, just make it all the same size. It doesn't matter if it's like that, right? It doesn't matter. Does anyone grow on labels too? Might as well do it. Actually, yeah. Might as well. Might as well, if you can do that for me. Make it a lot easier just to blast through the recap at the end of the night. Yeah, yeah, I know. That's exactly why we should do it like that. Today's the third, right? Today is the third, all right. 18 tops updates baseball. So one box, RT-1. RT-2. RT-2. I guess I should advertise it, but you missed it. Is it this one right here, Joe? What? Is it this one? Should be desktop and break labels, yeah. That one, right? Okay. Double click that, perfect. Oh, just because it's not like in the letter. Yeah, I don't know why it's all like a larger icon, but. Gotcha. All righty, guys, so yeah. Quick pack filler here. What? Hold on. I can't hear you, give me a second. Yeah, they're correct break clicks, like in progress. I haven't even changed the thing yet. All right, thank you. No problem. All right, guys. Yeah, I mean. All right, sweet guys. So let's open this box here for you guys. Actually, the knife, so this one here. So number blast from the past, Charlie Joiner to 100. Anthony Richardson, of course, some of these will have the team logo, some of them won't, but of course it'll be pretty much by the checklist too. I think they're all in there. Quentin Jackson, little mini. We'll love us, Tennessee. We got a Brian Branch. There you go, that is to 2.99. And whoa, how about that? Bijan Robinson, under the lights. That's a nice one right there. Three out of 25 for the Atlanta Falcons. Who has the Atlanta Falcons actually? Hold on. A-A-Ron, where are you at, buddy? Nice sit right there with the Bijan Robinson, man. Congrats. It's a nice hit, I have a little box of legacy. Deondre Hopkins, out of 50. And I think that's still, Deondre Hopkins Cardinals edition. I don't think they've been able to update it just yet, because he just signed. Now, they've got a numbered out of 75, Brian Robinson Jr. And Sean Tucker, little refractor. Royal Campbell, out of 25. A Trenton Simpson. Rookie there, out of Clemson. I'll double check and see who that goes to right now. And we got Marvin Mims, number to 35, with a nice Anthony Richardson there as well. All right, let me pull up the checklist really quick, guys. Trenton Simpson, which it looks like in the checklist, he's just under Clemson Tigers. So let's look him up, then, where he's currently at. Trenton Simpson is a linebacker for the Baltimore Orioles. I mean, Baltimore Orioles, Baltimore Ravens, sorry. Drafted in the third round, actually. So there you go, Baltimore Ravens will get that one, which is Zach, Zach Ewing. So some of them won't have the team affiliate on there, some of them won't. But for the most part, obviously, if that's the case, it'll go like college rules type of deal. So for the shipping team, of course, we got that. But a nice little box, though. B. John Robinson was really, really nice. Sweet. Now, of course, guys, we're gonna skip on over now to the randomizer for the break. Guys, obviously, top seven. Get into it, top seven. So there we go. Here we go, here we go. Roll it. Six into two, eight times. Good luck. One, two, three, four, five, six, seven, and eight. Boom, eight, two, six, eight, eight. And top seven. That was Roy, sorry, buddy, not in there. But let's see if you can have some spots up there. I know you got a couple. Allen, you are in. Congratulations, Allen Murdoch. Allen again, Simon, Stephen Flatt, Jeremy S, Allen M. Top seven, congratulations, guys. So you guys are now in. And like I said, coming up in a little bit, guys, once we finish packing this all up, you can get Big Hit Express Series six roll in, guys. Appreciate it, guys. Jasper's Case Breaks.com.


Good morning guys welcome to vlog week day two hosting five vlogs this week Monday through Friday So if you guys haven't already watched yesterday's subscribe you guys will get notified today is kind of a day at home We're gonna do some home updates. I've got to get some work done today. It's gonna be a good vlog You guys are having a great morning. I'm wearing a year of our set I think I'm just gonna wear out the easier all day because I'm gonna work out at some point I therapy at one I have a call at two so I'm gonna need to go over to dogs at some point Um, I can see the updates on her new place. It'll be good But first I'm gonna start off my day with some water. I Need to do my dishes I'm like, I'll get to that. You know what you mean? Oh, I need to pack to actually today I got this thing called eight greens not sponsored Lauren from summer Fridays posted about it and she Like has one tablet a day and it gives you like all the greens you need One of them contains as much vitamin B6 as six cups of spinach as much vitamin C as six oranges as much vitamin B 15 as 15 cups of broccoli as much vitamin B12 as seven cups of milk and as much And as much zinc as three cups of peas. So We're having this dissolve. It doesn't taste bad at all. I got a variety pack I haven't tried the other ones, but I have tried this one. So All right, I just finished editing yesterday's vlog I'm about to go to literally I'm going to tractor supply Because that is actually where I got the gym mats from they're technically like horse Stomach and really this is just a genius investment because one day when I have horses Which who knows about you will ever come I'm trying so hard to like make it a hobby of mine to go horseback riding But it is so hard to find Like where to go which I live in Texas. You think it'd be easier. I know I live in the city It's like totally different, but I just want to go like every week. I also this summer I'm gonna get into golf and tennis like I might as well join a country club. You know any meme just like new hobbies, you know Anyways, I just ordered a smoothie king. So that is downstairs. I'm gonna grab that. I'm literally not kidding guys horse mats Okay, but they're actually really good if you guys are looking for good gym mats. That's what I have already I just need more I'm getting those hopefully my little brother will come over tonight because they're so heavy It's like next to impossible like I'm not gonna be able to move them. I don't think I'm it's gonna be very difficult So hopefully my little brother over here tonight. He's watching the dogs from this weekend.  I was listening to country all the way here. So I'm not mad.  It'll be down below Change of events. I'm not getting these mats today. They're gonna cancel my order because they're really big Which is fine like we could if it the men the problem is normally they have like things that can like block your seats So it doesn't like get all the tar on your seats basically like ruin my seats. It's fine It was only a 15 minute drive But they're gonna cancel the order and I'm gonna have to try again to get them delivered Which was a pain because for some reason I didn't have the option of that yesterday or else I would have done it So false alarm. It was a really nice drive that I had the tractor supply today though. So don't worry about me okay, I'm home and I'm getting emo for some God forsaken reason and I have like an hour 20 until therapy I'm gonna go do my Pilates class with they let I Get questions about this all the time I take from her on zoom, but I sometimes just do the zoom recording So today I'm doing is your recording from yesterday, but I didn't get to do it because yesterday at 12 through 30 my workout routine Mainly is Pilates, and then I'll do you 12 through 30 I go through phases where like I'm like definitely a few months for like all I did was 12 through 30 Then I started taking daylight again, and I feel like that's what my body like really toned up Like that's my body like really toned up It's always Pilates for me personally like I find the biggest difference and specifically day let's classes Like I won't just take Pilates anywhere I'm very picky because daylight isn't the best future ever if you guys want to take from her on zooms We might actually be in the same classes. I'll have her linked below But she's my Pilates teacher in LA and my trainer who I used to work out with like I would literally do six-day training with her and then seven amp Pilates, but I'm not crazy strict. I definitely work out like most weekdays at least And then sometimes on the weekends if I need it for some reason I'm getting really emo right now So I need to work out so I can like snap out of it. Um, so we're going Okay guys I changed I'm going to therapy Exciting things everyone. Okay.  I Did yeah, yeah, and she's like Kinsey Shock but Don does this thing which I don't think this is like always a bad thing I think you took that a totally different way You just to me it's deeper than you calling everyone bro and dude You just like a meet you like someone and you don't really want to admit it But in the process you don't like yeah, but I don't friends on myself in the process That's the difference you go in and you are like their best friend And they have no reason to even think that you're remotely interested in them So they're like all this is like my really good because they say something No, but it's the way that you like treat them as just a friend like you don't treat them any differently like with guys like for example It's like well what about but because I because I get because what if they hate me? I can't handle rejection Okay, well unfortunately in dating you like have to you've never been rejected. Are you joking? I Know it's hard to believe I Thank you for that one Okay, but I know the answer to this but you just go into a friendship with a guy and you like him He has no idea you like him and then you get in this cycle and like I get it because I've done this so many times So that's why I'm saying like you just get in this cycle of like oh you're interested in him He has no idea that you're interested in him and then you're just there Do you see what I'm saying? Yeah So I think that we just have to figure out how to not get you in that cycle I know which I honestly I think is having more like boundaries with god, but my boundaries Now are to just be like well now. That's my problem. It's because so see this is a great This is a great specific because you know two minutes perfect. This is gonna wrap up in two minutes I have a boundary and my boundaries brew bro bro brother and then it's usually once I get to know someone that I'm like oh you're cool, but then it's too late, and then I'm stuck I never usually like anyone off the bat. It's through time of getting to know someone very rarely. Have I liked someone off the bat? Different than okay, because that's the only time I've ever Like that yeah, okay. Well, I think we just have to find like some sort of middle ground for you there Yeah, I'm learning maybe this book last week That was actually it's like so cringy looking like I couldn't believe that she recommended it to me But it was actually really good.   Just not for the best day. I am so exhausted I could fall asleep right now and it is 6 p.m. I actually need to pack for this weekend I've kind of planned out my outfits and we leave in the morning I have time if I wanted to to wake up and do that. I might just get a few outfits together, so But I'm gonna show you guys what I do when I'm having just like a really off day Sometimes after therapy, it's just like you're emotionally exhausted. You know what I mean good news everyone I actually just planned out all my outfits, so I will feel a lot better. This is good Lots of outfits for a very short time of way, but it's gonna be okay. I have to pack a bag anyways Hi, girl. This is what's happening here. Probably doesn't seem like it, but I just cleaned up my room I was getting a little bit messy. I just had some random stuff here. I showered. I self tan I'm trying to just have a really cozy night. My little brother is here He's staying at my house this weekend, so I was just hanging with him for a little bit I ordered us some dinner from shake shock. I have been craving it nothing But like really unhealthy food this week, so it's fine. I'm craving nothing But like really unhealthy food this week. Anyways, because I'm just not really feeling Myself I wanted to have a really chill night. It's only like 7 p.m. The new Kardashians episode is up So I think I'm actually gonna watch I've been watching since the last season I kind of wish I would have watched and like kept up I'm just really bad at keeping up with TV shows, especially if I can't binge them So I'm gonna watch that I'm gonna wake up at 7 We don't have to leave the house until like 9 30 or something So I can pack and do all that in the morning. I am like out of hangers. I've gone through clothes I just have so many extra clothes so well picking out outfits for this weekend was the worst And you know when you're like not excited about what you're wearing I feel like that's like the worst feeling. Also, this crew is like so cozy I'm gonna figure out how I can watch the Kardashians my sister always puts it on and watch that and then I'm gonna finish my book tonight I'm really excited.  Also comment below if you guys think I need to get an Apple watch I feel like I need one for like 12 3 30 and workouts and going on walks and stuff for like that health and fitness Benefits, but if you guys have one, please let me know which one I should get why you guys like it What you use it the most for I don't want a cellular one because I don't want text messages popping up I get annoyed when I have like might have my AirPods in and Siri starts telling me who texted me like if I didn't open the text, I don't want to hear it yet You know, like I'm not I'm not mentally there. It's like been so emo today for like literally no reason All right, it's 8 p.m. I have the Kardashians on my shake shock is here After I finish watching at the Kardashians I'm gonna read my book that I'm reading That's actually our book club picked for the month the perfect marriage. It is so good Like it is one of the best books I've read in a while. I feel like I've been kind of in like a book rut recently I haven't found I think I had a lot of bangers like the first two or three months of 2021 and then the past month It's been, you know, rough, but the perfect marriage So so good, they cannot recommend it to you guys enough, but I love you guys Hope you guys enjoy this vlog and I'll see you guys tomorrow. Oh my god. I love fiber Fiber so I really increased my fiber and take a couple years ago and my life has been completely changed because of fiber And I realized like there was that


Hello, my name is Daryl Hoffman. I am the city minister of the city of Sheboygan. Thank you for giving me time today to talk about the city's strategic plan and its updates. Some of the information contained within my report is based upon the 2017 to 2021 strategic plans, fourth quarter action items and critical measurements. I have been providing these quarterly measurement updates to the Common Council to ensure that the city's efforts are working toward supporting these strategic plans. Creation of the strategic plan action items and critical measurements are provided by resident input. Our goals to uphold the strategic plan with these measurables and tasks to track our progress toward meeting our goals. The strategic plan mission statements for the city of Sheboygan is dedicated to providing residents, the business community and visitors with fiscally responsible municipal services in an effective and responsive manner to meet the needs of our diverse community. The strategic plan's vision statement is the city of Sheboygan will be a family oriented and prosperous community with a wide variety of housing, business, cultural and recreational opportunities in safe and attractive neighborhoods. The six strategic plan focus areas are quality of life, infrastructure and public facilities, economic development, neighborhood revitalization, governing and fiscal management and finally communication. We begin the review of our quality of life related benchmark target goals and achievements for 2019. One of the key objectives in the quality of life focus area is support and enhanced programming and activities that encourage increased guardianship and crime reduction in our neighborhoods. This chart confirms a significant decrease in Part 1 crime rates, specifically property crimes. Between 2018 and 2019, a 12% decrease in these Part 1 property crime rates occurred. This chart shows an overall decrease of 34% in property crimes from 2015 to 2019. Although the city has experienced some ups and downs in Part 1 crime rates for violent crimes, the chart confirms an average of 144 violent crimes occurred between 2015 and 2019. This is a notable decrease from the 159 violent crimes in 2015. Another quality of life amenity is the city's new ADA compliant canoe kayak launch installed at Kiwanis Park. This launch was initially installed in 2018, but final work on this project was completed in 2019. This launch project utilizes grant funding from the Sheboygan County and the Wisconsin Coastal Management Program. In 2018, the city established an adopt a park program to empower residents to support efforts to maintain city parks and trails. During 2019, three parks or trail areas were adopted, including Coal Park, which was adopted by the Balrath North Point Neighborhood Association, Shoreland 400 Trail, which was adopted by RCS, and the Lake Michigan Bluff, which was adopted by the Friends of North Point. In 2019, the city of Sheboygan received multiple accolades. Several of these recognitions were national in nature, and some were based within the state of Wisconsin. The city of Sheboygan was number 18, best small city for businesses by the chamber of commerce.org. The city of Sheboygan was number 5, best place for millennials to move by review.org. The city of Sheboygan was number 7 in the nation for most unique art town by Cubesmart.org. SoulSmart Bronze Award was received by the city for its solar energy growth efforts. The city of Sheboygan was number 9 for best cities for older Americans in retirement by 24-7 Wall Street. And last, the city was number 10 within the state of Wisconsin for the best city to live by the chamber of commerce.org. In 2019, Shoreline Metro, the city's transit service, received its highest ridership totals since 2001. During the past nine years, the transit ridership increased 54%. The annual increase in ridership is substantial. Between 2018 and 2019, transit ridership increased 13% during this one-year period. In 2019 alone, Shoreline Metro ridership experienced a 14% increase. During 2019, the city took delivery of five new fixed route buses. These are the first new buses the city has purchased in over 10 years. In 2018, city staff combined forces with a diverse resident-based task force to work toward achievement of the American Association of Retired Persons, or ARP. It's age-friendly community designation for the city of Sheboygan. This designation supports the theory that an age-friendly community improves the quality of life for not only older people, but all ages living within our community. The age-friendly community provides improved living for all residents. The results of the work include the creation of a livable Sheboygan plan. ARP approved this livable Sheboygan plan in 2019, designating Sheboygan as one of six age-friendly communities within the state of Wisconsin. The age-friendly community recognition is an international designation affiliated with the World Health Organization's global network. The eight domains of the livable community are pictured here. It will have significant impact on the next update to the city's strategic plan. During a 15-month period, the city worked hard to renovate its 102-year-old city hall. In September of this year, the city moved from its temporary locations back into this 102-year-old building. Also in 2019, the Mead Public Library Plaza and its adjacent parking lot and the Halpern Fountain experienced a complete revitalization, establishing a connection in the downtown area to the city green. Focus area number two is infrastructure and public facilities. During our annual community survey, we received consistent feedback from residents to focus on street improvements. One of the key objectives in the infrastructure and public facility focus area is to construct, repair, maintain city streets, sewers, and other critical infrastructure to ensure public safety and commerce. During 2019, the miles of street improvement decreased due to the city's need to partially allocate funds for the total reconstruction of the Pennsylvania Avenue Bridge and the North Avenue area between North 15th Street extending west to Calumet Drive. These two large street improvement projects included complete street and underground utilities reconstruction. Both projects were primarily funded by grant dollars, which determined the simultaneous timeline for the completion of both of these projects. This chart confirms more miles of street improvement occurred in 2018 and 2017. However, the projects completed during these two years were strictly pavement overlays, not complete street reconstructions, as it was in 2019. In 2019, the city paving crews also completed paving resurfacing projects on Pershing, North 8th Street, and North Taylor Drive. Our street surfaces are rated on a PASER scale. PASER was developed by the University of Wisconsin-Madison Transportation Improvement Center. The 1-10 rating scale assesses road pavement conditions uniformly across the entire state. Then PASER rating is conducted every two years by community. With the strategic plan's concerted effort on road improvement, the city's PASER rating continues to improve. The 2018-2019 PASER rating achieved a 6% gain from the 2016-2017 PASER rating. Our continued steady focus on street improvements will ensure that Cheboygan's PASER rating continues to grow in the future. Annually, the city is spending approximately $200,000 on managing the impact of emerald ash borer beetle on city-owned trees in both parks and in terrace areas. In 2019, 535 new non-ash trees were planted and 800 ash trees were treated to combat the destruction caused by the beetle. In 2020, the city will finalize the purchase of 1.67 miles of former Union Pacific Railroad right-of-way. This land acquisition will connect existing shoreline 400 trail, which currently ends at Pennsylvania Avenue. The funding for this program is through a tax incremental district and not taxpayer dependent. Upon completion of the extension, the trail will extend from North Avenue, southward toward Union Avenue. The timeline for completion of the improvements on this extended shoreline 400 trail are yet to be determined. The focus area number three is economic development. Strong interest continues in the residential real estate market within the city of Cheboygan. More people than ever want to own residential property within the city. These buyers are willing to pay more for their real estate purchases. This is great news after property values plummeted in 2009 after reaching their peak in 2018. This graph shows interesting changes to residential property values within the city from the Great Recession. In 2009 until 2015, property owners experienced an overall decrease of value by 16%. This decrease equates to $353 million of lost value. During the past three years, residential property values have experienced respectable increases, 6, 9 and an additional 6% this last year. This equates to millions of dollars of increase during this three-year period. In June 2020, the city will receive reports from the state of Wisconsin regarding the overall sales and related real estate values for 2019. Please note that the values do not include multifamily properties which are considered commercial properties for valuation purposes. This city has and continues to work hard on redevelopment. Since 2015, the value of the city's tax incremental districts or TIDs have achieved a 77% increase, which is an $85 million increase. The South Point Enterprise Campus Business Park located on the city's south side was completed in 2019. Approximately 150 acres are available for business development and expansion. The River Bend Neighborhood Master Plan was finalized and approved in 2019. Many exciting things are occurring within this section of the city including the relocation of Lakeshore Technical Colleges, Sheboygan Campus, Domestecore, Carpet One, construction of Waters Edge Condominiums on the Sheboygan River, and interest from senior housing developers on the former Mayline site located north of Pennsylvania Avenue. In partnership with the Harbor Center Business Improvement District, the city developed a pop-up shop program in the fall of 2019. Three businesses were selected to open their doors in the pop-up shop relocation area. They include Hometown Barkery, Cultivated Taste, and Nest Adorn. At the conclusion of the pop-up shop program, all three businesses continued to operate in the downtown area. Sheboygan was successful in finalizing development agreements on three blighted redevelopment sites. Two of the three housing developments are already under construction including Kingsbury Village on North Tenth Street with an anticipated value of $6 million and Phase 1 construction of South Pier Condos with an estimated value of $6 million. The third development project is the Oscar located on the former Vandervaart Concrete product site adjacent to South Business Drive. This housing development is anticipated to be valued at $47 million. Construction on the Oscar is expected to begin by mid-summer 2020. In 2018, the city began a partnership with the Sheboygan Economic Development Corporation, or SEDC, to establish a redevelopment area near the corner of South Tenth Street and Indiana Avenue, known as the Innovation District. In 2019, the city purchased three additional properties near this location including the former Craft 30 building. The additional properties will expand the city's footprint for this innovation district. Last year, the city began discussions with a developer who is currently pursuing construction of a three to four storey office collaboration center. The SEDC hired a director who is responsible for developing programming consistent with the FreshTech Innovation District's mission. By July 2020, the former Badger State Tanning Company's building, located at Indiana Avenue and Maryland Avenue, will complete its restoration as the Badger State Loft Apartments. This development will include 118 workforce housing units. The total cost for the Badger State Lofts project is $30 million. According to the strategic plan's focus area number four, the goal of neighborhood revitalization is to encourage reinvestment in the city's housing stock and create solid neighborhoods with strong leadership and ensure quality new housing developments, creating new diverse and affordable options for all citizens. City staff from multiple departments continue working together with residents to strengthen these neighborhoods within the city of Sheboygan. In 2019, we saw some significant events related to neighborhood revitalization efforts, including the formation of the city's 10th official neighborhood association. The neighborhood association initiative began in 2012 with the creation of the first neighborhood association for the Gateway neighborhood. During 2019, the city residents within the official neighborhood associations received an additional incentive in the form of the first annual large item disposable program. 23 tons of garbage were collected from these neighborhood associations that participated in this program. In 2017, the city began a partnership with Habitat for Humanities by hosting the first Rock the Block event. In 2019, Rock the Block has expanded from a single day event to a three-day event. For the past three years, the city staff has been targeting historic neighborhoods to identify and stabilize them. Utilizing increased code enforcement does bring focus to abandoned vehicles, weeds, conditions of roof, and maintenance improvement issues. In 2019, five neighborhoods were mapped to collect this baseline information for future improvement reviews. The five neighborhoods include South Calumet, River Bend, River Watch, Southlake, and King Park. The first part-time code enforcement officer was hired by the city of Sheboygan in 2015. In 2019, 1047 code enforcement orders were issued as part of these code enforcement actions. The first key strategy of focus area number five, governing and fiscal management, is to develop efficient transparent processes systems to provide financial information and foster sustainability. Since 2017, Sheboygan has received annual recognition from several national organizations including the Government Finance Officers Association or GFOA and the International City County Management Association or ICMA. Regarding the city's budget documents, our comprehensive annual financial report, CAFR, and our popular annual financial report, PAFR. Our performance measurement program also received recognition. Within the state of Wisconsin, Sheboygan is one of 17 communities earning the GFOA Distinguished Budget Presentation Award, one of six municipalities receiving the Certificate of Achievement for Excellence in Financial Reporting for our CAFR and PAFR documents. Sheboygan continues to be recognized by ICMA as a leader in the performance measurement as the city was only one of two municipalities in the state of Wisconsin to receive this recognition. All of these documents are available in electronic version and also print format. The electronic version is available on the city's website, the hard copy is available at the Mead Public Library, City Clerk's Office. The city's main operating fund is the city's general fund. A large number of cities operating expenditures are paid via this general fund. One of the most critical fiscal policies that the city upholds is to maintain a minimum of 25% uncommitted fund balance. This graph depicts the uncommitted general fund balance in two ways, by dollar amounts and by percentage of general fund expenditures. As you can see over the past five years, the average percent of uncommitted fund balance was 51% well above the minimum 25% amount. At the end of 2019, the uncommitted general fund balance is estimated to reach 46% or $17.3 million. Another critical fiscal policy contains an amount mandated by the state and is affirmed by the common councils related to the city's ability to issue debt. This chart shows the debt management limits. The blue cylinder on the left represents the state of Wisconsin's mandated maximum amount of debt the city can issue. The statutory limit is $146 million or 5% of the city's equalized value. The red cylinder in the middle represents the borrowed limit established by the common council in a policy. This limit is $88 million or 60% of that state mandated statutory limit. The green column on the right represents the city's actual and debt outstanding which is $38 million or 26% of that state mandatory statutory limit. The debt amount includes debt associated with tax incremental districts as well as general obligation debt. As you can see, the city's historically has borrowed significantly less than what is allowed by state statute and the common councils policy. The city has a double A bond rating provided by Moody's Investor Service. Sheboygan has held a double A-2 rating since 2010. The higher bond rating provides the city with many benefits including lower interest rates and of course lower borrowing costs. Prior to receiving the double A-2 bond rating 10 years ago, the city had a double A-3 bond rating. Communication is the strategic plan's focus area number six. The city has worked hard to improve communication with our residents and businesses alike. Sheboygan has expanded all of its social media communication, implemented a monthly e-newsletter, Sheboygan Insider, and included the Transparency and Benchmarking Center powered by ClearGov to empower residents. In 2019, a communication task force was created with key city staff and communication experts from a variety of Sheboygan County organizations to establish the best practices for internal and external communication. The employee portion of the task force continued to make progress to implement these best practices within the city's external and internal communication efforts. One of the easiest ways for residents to provide continued input and feedback to the city is through our annual community survey. Online survey is currently open through Sunday, February 23rd. The survey can be accessed through visiting the city's website www.sheboyganwi.gov. Paper surveys are available at the clerk's office, Me Public Library, and the Senior Activity Center. I want to thank you for allowing me the time to share information with you regarding the city's successful 2019 activities presented in the state of the city message. The city has many new and exciting initiatives planned for 2020. I look forward to providing you with an update regarding these successes at the same time next year.