Hi everyone, good evening. Thank you all for coming to our session. We have a panel to present this evening on promoting open science and transparency across sub-disciplines of psychology and so we'll be featuring several editors from APA journals as well as our open science and methodology chair. So first I'd like to share a quick overview of open science and how it factors in the APA, the American Psychological Association. Open science improves the transparency, accessibility, and reproducibility of science and here I'm presenting in three different ways to ways that you can engage in open science practices. So for simplicity to provide an overview, you can look at how open science aids in confirmation of hypothesis, so things like registering your research, so being upfront about what you plan to do with your research study before you've done the research. There's open science practices that can aid with collaboration, things like data sharing so that others can do secondary data analysis on the data that you've collected or pre-print so that folks can see the findings of your study before it's been published in a journal. And then open science can also aid in transparency and so an example here is constraints on generality statements which allows you to be very transparent about how far your your research findings extend. Some open science practices that APA has engaged in over the last several years include things like open science badges and so here I've displayed the open data badge but one thing that was very important since a lot of the research we publish reports on human interaction was that we have the open data protected access badge and so that's something the APA was very active in advocating for and so that that allows you to state that you'll share your data but it's it's um there's protected access to it so you have to have that data requested from you so it's not completely open in the world. Another way that the APA has engaged in open science is through journal article reporting standards and so these are for quantitative qualitative and mixed methods help um authors really think about being extremely transparent and what the reporting in the research study. We have a APA journal branded data repository through open science framework so if you publish an article with us and you deposit you can deposit your data in that repository. We recently last year launched a joint psychological societies pre-registration of quantitative research and psychology so this is a template Fred Oswald was part of this as well. It was a group with British Psych Society and German Psych Society as well as Center for Open Science and ZEDPED and that was to create a template that that researchers could use to pre-register their research and then finally we created an open science and methodology committee a few years ago and as I mentioned Fred Oswald is our chair now and this committee is in place to consider open science particularly as it pertains to APA journals and make recommendations on how the journal should be thinking about open science. And so for example one change that came about because of the committee's recommendation was for the APA core journals to sign on the top guidelines and so that's been happening throughout this year. So today we'd like to specifically feature the ways in which different journal editors and our open science and methodology committee are engaging in open science practices and specifically we try to pick a subsection of journals to really highlight how the different disciplines engage sub disciplines in psychology engage in open science and so Dr. Ebe will talk about journal applied psychology and her work there. Dr. Leitch will talk about the interpersonal relations and group processes section of journal of personality and social psychology we have a recording from Dr. Yates who couldn't join us tonight but he's the editor of neuropsychology and then Fred Oswald will speak more broadly about open science practices and then we'll have a Q&A session so we hope that several of you will be able to to participate in that and so I will move on to Dr. Ebe now. Thanks Rose I appreciate that so I'm going to talk to you about our kind of foray into open science at the journal applied psychology so I'm the editor of the journal and have about have 15 associate editors that work with me next slide Rose. So just to give you a little bit of context about the journal applied psychology I pulled this off of our website we basically study the psychology work employment and so our journal focuses specifically on the cognitive behavioral and psychological processes at multiple levels of analysis that are related to the world of work. We focus on primarily research that's conducted in organizational settings although we also do publish experimental and simulation and some methodologically based work so our context is somewhat unique and that we're really studying people primarily in field-based settings. We do publish largely empirical research but we also do publish qualitative research which I think poses some unique issues for open science. Next slide Rose. So in terms of our policies related to open science I took over the journal in 2020 and at that time we knew that there was a at least some rumblings at APA about really signing on to the tops and also really promoting open science so early on several team members and I wrote a piece on methodological reporting and some developed some checklist we could use with the journal. This is not to replace APA standard jars for qualitative and quantitative but what we do find is that some of those checklists were kind of cumbersome and for the type of work we do we thought a more specific type of checklist would be helpful so I mentioned this because it was really a nice way to get our stakeholders thinking about open science in a way that they could really understand and so we didn't implement this as policy but we implemented it as best practice prior to the decision by APA to implement the top guidelines and be a signatory. So on our website we have pretty detailed information on submitting manuscripts and also open science and we actually spent a lot of time really crafting this message. Open science is not something that is that common to people and industrial organizational psychology and management and so we really wanted to make sure that our messaging was clear and concise and so several of my associate editors and I worked on a lot of language with the help of APA and really kind of tailored it to our unique audience and effective number first we will be implementing the top guidelines and you can see here just as an example the different levels at which we have decided to adopt APA I think very very wisely determined that the different editors could make decisions about top implementation implementation that worked for them and required at least a level one so on some of these we have level two we don't have replication list on this table because we've always encouraged replications at the journal but you can see here that we are getting ready to go live very soon so if you talk to me in a couple of months I'm sure I'll have some additional stories but we've done a lot of preparatory work and we feel really quite prepared. Next slide. Kind of collaterally with the movement to the top guidelines one thing that I did when taking over as editor is create kind of an ancillary website that's not so much associated with the journal and it's not something that is formally endorsed by APA but it was a way for us to really signal our values to authors to reviewers and really reach a broad audience and so this is just a screenshot of our of our website encourage you to look at it it's got a lot of great information for graduate students authors reviewers and we deliberately have a a sub page on open science and on one we have some really good people who are learning about open science we have a logical checklist also open science events that are happening we have links to APA and actually we also have links to what other journals in our set of psychology are doing and we've gotten really good feedback on this this website as a resource but particularly related to open science next slide and so I'll just I'll say a few things about let learn this was organizers asked us to talk about and one thing they particularly for our area sub area of psychology is that education is really important and luckily prior to APA's decision to implement the top guidelines there had been numerous sessions at division 14 talking about open science getting people to understand what it was about and and we realized that there's a lot of misconceptions people often think of it as kind of an either or like you have to share everything you have to pre-register everything so we've been really working hard in small ways to try to get people to understand that there's lots of ways to be open but also to really highlight the advantages but for us it's really been a culture change and it's a still a culture change in progress um another thing I would recommend is just starting small I mentioned that we started with our methodological checklist and some messaging around open science I do give quite a number of talks to universities about the journal and so I am inserting open science slides into those presentations to get people talking about it and I also think that considering kind of the unique issues facing your sub discipline is really important for for industrial organizational psychology in particular issues of data sharing can be can be fraught with problems and in many cases may not even be allowed we tend to class dietary data a unique issue with some unique ontological beliefs that in some ways run counter to some of the tenets of open science or at least the interpretation and so I think that poses some challenges I'd also I'd also encourage you to think carefully about the role of reviewers and associate editors we're a very high volume journal we had almost 1700 submissions last year alone and so the the thought of how to do this well without increasing tremendous burden on reviewers and associate editors is something that we've really given a lot of thought to we also have given a lot of thought to how to ensure that the system is fair for all authors and that involves for us making sure that reviewers are clear on their role as reviewers as it relates to open science to make sure that people are being treated fairly in the process and then the last thing is we did spend also a lot of time just really thinking about the nuts and bolts there's a lot of boxes to be checked in implementing open science particularly at at a journal and so we did spend a lot of time thinking about who's going to be checking which parts of manuscript submissions what is our peer review manager going to be doing what is the editor going to be doing what are associate editors going to be doing and also whatever reviewers are going to be doing and so I this will be helpful we implement mine in November 1st Rose Lillian I missed that last part but you're finished right yes okay good your audio held up very well until the end so we're good all right great I'm going to stop sharing my screen now because dr leach is next and he'll talk about journal social our journal of personality and social psychology um hello everyone rose can you just give me a thumbs up if my audio is okay great thank you um so yes I'll speak about from the perspective of the journal of personality and social psychology and as maybe some of you know social and personality psychology but especially social psychology has maybe a unique relationship to open science and that's partly because we were at the center of a number of highly publicized scandals in psychology and I think to have this discussion for social psychology in particular we have to first acknowledge um events about 10 years ago um that started the particular form of open science practices and social psychology and so that was um the discovery of of long-standing years-long research misconduct by Dietrich Staple in the Netherlands in 2011 um was a number of well-known publications um were discovered to be fabricated um and also what became clear in that process which was that in fact it was very very difficult um for any of the people who suspected that misconduct to actually expose it it took years um and so this this really was a was a was a lesson for us in um the lack of transparency in research practices um and then of course the other thing that happened which was a little closer to home at JPSP although as well before my time as editor there which was in 2011 in that same year um Darrell Berman colleagues published in our journal a paper with nine experiments um purporting to provide evidence of ESP and this of course um attracted a lot of criticism and um and uh there were editorials written and blogs and uh in many ways people um focused on this article by Darrell Berman um supporting using very traditional social psychological methods and nine experiments um to provide empirical evidence for something that many people um believe cannot be empirically validated and so this also led to a lot of hand-wringing and criticism and so those two cases really fixed the field of social psychology um on uh research practices and we've been pretty fixed on it for the last 10 years um and in the end I think it's led to a particular in some ways cultural shift toward open practices in social psychology but also you know quite a lot of um tension and discussion or what have you and many people even called the subsequent years after those two revelations in 2011 or those two controversies many people have called uh have said that the field is in crisis um and so a number of things have happened since and I think have been really productive and the first one is and I think this is relevant to to everyone no matter what sub-discipline or discipline you're in which is that our main society actually our main professional society the society for personality and social psychology convened a committee and distributed guidelines that were that were really um shared widely in 2014 and this was really a a kind of consensus statement um about the need for more open and transparent practices um and reporting in the field and I think in many ways it reinforced some of the messages of the journal article and reporting standards that that Rose mentioned a little while ago which was um produced by APA in 2008 but that a lot of social and personality psychologists hadn't really picked up um and I think it carried forward some of those messages and updated them to some degree and so therefore I think it's also really consistent then with the latest journal article and reporting standards published in 2018 and so now I think um at the journal we have the last few editors across the three sections have really been referring to the 2014 document by our professional society and also the 2018 journal article reporting standards to refer to what I think are are clearly consensual statements about best practice in the conduct of research and the reporting of research and then um from my team in my editorial 2020 and in every editorial letter we refer to those standards and we also refer to the publication manual by the American Psychological Association which is another very important and useful document for research practices and reporting and so in every communication that we have with authors and reviewers we try to reinforce this consensualization of standards of practices and reporting by referring to those keystone documents and I think this is a really important in many ways social and cultural dynamic of open science and that is um developing consensus and agreement whatever it is that we can agree on and consensualizing it and then all referring to some guiding documents or statements in in our communications and in our engagement and our evaluations of work and as as the previous speaker mentioned also getting reviewers and everyone involved in the process referring to that same standard set of standards basic minimal standards has been a major task for us at the journal okay so more specific policies we um all three sections of JPSP have endorsed the level two guidelines by top and then since January 2020 we were also required upon initial submission the completion of the open practices disclosure form which indicates where data would be made available where data would be made available and where materials are available and code the relevant code for programming or for analysis and so authors must explain why this is not possible or why it's not advisable so the standard is that people make data materials and code available and if there are reasons why this doesn't make sense then they really have to say so and so therefore now all published articles include in the author note reference to where the data materials and code can be found or some explanation in those rare cases where it's not accessible and so again now in every art in every issue people are also getting this message over and over again that this is the standard and it is something that all authors have to commit to upon initial submission and I think that's been an important change for us the other thing we've done specifically at the interpersonal relations and group processes section of the journal my personality and social psychology is um and I think this also goes to the previous speaker's point about the extra labor that goes into engaging open practices documented information and so for our team because I think of our unique relationship and social psychology to this it was important for us to have and so APA was very supportive of us having three methods and statistics associated editors who can scrutinize initial submissions scrutinize some of that additional evidence that's presented and in some cases reanalyze data or perform simulations of data and this is really empowered the entire editorial team to ask pertinent questions if things are unclear and to engage the material that's provided through open science in a way that's actually useful and productive in the initial editors assessment of the initial submission because otherwise the assessment of the material that's submitted through open science practices the materials the data would have you that actually relies on volunteer reviewers to make the effort and to have actually have the expertise to engage the material that open science is now presented to us and so we thought it was really important to get some technical support to our editors in those cases where it really matters but there is a question here of scale and of volume and how to manage this given uneven expertise and interest or motivation among reviewers and I think that's a really important question for us given the system of peer review that our journals use which is a voluntary system where is this labor and expertise going to come from so having lots of information and access to it doesn't serve anyone if there's no one to actually make use of the information so this is one step that we've tried to take the methods and statistics associated editors but it really is a stopgap measure it's not really a long-term solution to making the most use of the extra information that we have access to we do not require pre-registration of hypotheses but we have been encouraging it a voluntary pre-registration and we do as a matter of course encourage authors when they've been asked to conduct further studies or reanalyze data or what have you to engage in pre-registration at that stage so we've certainly been using that as part of the editorial decision making when people are conducting replications or extensions and there is a sort of a logic that's apparent to just make that early to make that commitment to actually pre-register replication studies or follow-up studies now because of social psychology's unique relationship to these issues I would say a good 60 to 70 percent of our submissions contain pre-registered studies actually and that number is going up every day but there too we have the question of quality of pre-registration not all pre-registrations are equal in their quality some are very vague some are very general and of course we have had some cases where reviewers or others have raised questions about deviations from pre-registration or the quality of pre-registration so that again that raises the issue of making use of the information that's provided okay so I just want to mention me but a couple things now about researchers and the broader involvement as I've already mentioned a key area of growth I think for us is engaging reviewers in the even application and understanding of open practices this has been a struggle so far even though we've tried to share what I think are consensual standards there's a lot of difference of opinion and at the moment I think it's been a little difficult because some reviewers are very committed to maybe what you might even call sort of level three top guidelines and I think those are the only acceptable ones and others to none at all and so this is introducing now another level of heterogeneity in reviews that editors have to manage and that authors have to contend with and so far I think the field at least doesn't have a good approach to dealing with that but at the moment voluntary pre-review is highlighting where there is a lack of consensus on what open science practices are expected of authors we've also really pushed and you require authors to commit to an a priori power analysis and also with an explicit rationale for expected effect sizes and this helps deal with not genuinely a priori power analysis because of course people can say that they've done an a priori power analysis but if they have no real rationale for the effect size that they expect and that a priori power analysis isn't really worth the numbers that it produces and so getting researchers to be reflective and conscientious about their practices and what design what number of participants what methods are most appropriate for reasonable expectations has been a really important thing for us to engage authors and reviewers on and I think we're making headway there but that again has been I think relying on us sharing information and educating all of those involved in the peer review process about the usefulness of power analysis and I will say I think that the COVID effect on slowing research which has been pretty dramatic in social psychology has actually given us an opportunity to more productively and constructively and less defensively engage authors because I think everyone's been slowed down and that slow down has I think led to authors and reviewers having a greater appreciation for the effort that goes into producing research and therefore in some ways it's kind of like the the slow food movement in many ways I think people have had an appreciation for the quality that can be gained by slowing things down and I think that that does open more people up to the gains of open science practices by sharing more by being more reflective and and all of us engaging in a little bit more careful assessment of what we've done why we've done it and and how convincing it is and essential part of that for us the last thing I'll mention is we've also been encouraging people to also engage in open and best practices of graphics and in fact the previous panel ended with a discussion of that and that's another thing that we really pushed that actually making data more transparent in graphical representation is another way to reinforce this idea that we all gain by sharing as much information as possible about what we've done and why and so I think I might be coming toward my end but that's what we've been doing at the Journal of Personality and Social Psychology to engage open practices and to try to to get all of those involved in a productive and constructive conversation. Thank you. Thank you Dr. Leitch. Our next presenter is Dr. Keith Yates from the Journal of Neuropsychology and I'll be showing his video presentation. My name is Keith Yates, I'm a professor at the University of Calgary and I'm really pleased to be part of this symposium on promoting open science and transparency across sub-disciplines in psychology and I want to thank the organizers and particularly Rose Sokolchang for inviting me to be part of this. Before I get started just by way of disclosure I want to acknowledge that I have current grants and research support from the Canadian Institutes of Health Research and Alberta Children's Hospital Foundation. I also get a small amount of book royalties or books that I've edited or authored and I receive an editorial stipend from the American Psychological Association. I'm sorry I can't join you in person, I'm speaking from my office in Calgary, Alberta, Canada. I find that many people around the world know where Toronto, Montreal, and Vancouver are but may not be familiar with Calgary and this is a picture of the lovely view across downtown Calgary to the beautiful Canadian Rockies and the reason that I'm not with you in person today is that I had the opportunity to take a weekend at the Issynavoid Provincial Park. This is a picture of that area. It's a lovely part of the Canadian Rockies and I'm really looking forward to spending the weekend there because it's a really lovely place with tremendous hiking and we're very lucky to have it in our backyard in the midst of this pandemic. Without further ado let me go ahead and move along. I'm going to start just really briefly by talking a bit about what open science is. We've already reviewed this previous to my talk but just to reiterate, open science promotes transparency, integrity, and reproducibility of science. In many cases this is done through the sharing of data with the establishment of repositories where investigators can share their data, their materials, their computer code, analytic code, and also the sharing of scientific plans which often translates into the pre-registration of studies including in many cases the pre-registration of data analytic plans. So how does this influence neuropsychology? For those of you who aren't familiar with neuropsychology, neuropsychology is the science of brain behavior relationships. It typically focuses on how the brain is related to human behavior and both healthy as well as brain disorder and population.  Although some neuropsychologists focus on healthy brains and what we can understand about behavior in healthy populations. So we've been heavily influenced by outside forces that have driven the open science movement, of course the Center for Open Science, but also funding agencies that have increasingly expected us as investigators and as journal editors to promote open science, the National Institutes of Health, Canadian Institutes of Health Research for example, have promoted and required increasingly that investigators deposit their data, make it publicly available, make their publications publicly available, and that's certainly had a significant influence in our discipline. Another area or another force that's been at work is medical journals. Many of the journals that we publish as neuropsychologists tend to focus on medical disorders or populations and a number of those journals have also pushed in the direction of open science, particularly when it comes to randomized controlled trials or systematic reviews but not limited to that. So what we've seen over the past decade has been the establishment of data repositories that allow investigators to share their data readily with like-minded investigators. Some of these are government initiated efforts such as FitBur, which is the data repository initiated by NIH for investigations focused on traumatic brain injury. Some of these are voluntary efforts to pool data and put them into available repositories such as the Enigma Group, which focuses on neuroimaging and genetic data and combining those across studies and making those combined data sets available. The other thing that we've seen is the establishment of many sites, websites that allow for pre-registration. In many cases, these involve, as I mentioned, randomized controlled trials. So sites like clinicaltrials.gov or systematic reviews and Prospero, a good example of a site that's available for pre-registration and systematic reviews. But others are choosing to use pre-print servers and other sites to pre-register their studies. And in many cases, journals are increasingly willing to publish study protocols that allow an investigator or a team of investigators to pre-register their study, including their analytic code. And we increasingly see that in neuropsychology. There are barriers to open science that are not limited to neuropsychology, but certainly do bear on our discipline. In some cases, research ethics boards, individual institutional review boards or whatever they might be called at a local level actually place restrictions on sharing of data. In Canada, where I'm located, it is not necessarily the case that all research ethics boards will endorse or allow for the sharing of data even when it's de-identified. And that's a barrier that we need to try to address at a local level. We certainly also run into, at times, research or reluctance for a variety of reasons. Sometimes it's simply the sense of, well, I worked hard to collect this data. It's my intellectual property. I don't want to give it up. I don't want to share it. That can be particularly true early in the stage of research before all primary study hypotheses, for example, have been addressed in publications. It's also the case that open science is not without cost, cost in both money and time. Preparing data, for example, for deposit and a depository often means creating metadata files and doing other forms of cleaning or removing of identifying information and similar activities. And in many cases, those are not necessarily incorporated into the budgets that are available in people's research grants. Some investigators don't have large research grants. They don't have the staff necessarily that can take the time to make it happen. And there are a number of other barriers. There's a nice article that I referenced here that appeared in the American scientist that talked about how open science isn't always open, that investigators are sensitive to the fact, for example, that if they're working, if they're junior investigators working with more senior collaborators, they may want to promote open science. But if they're senior collaborators are not open to that, there are power differentials that make it difficult. Some of the financial limitations, some of the time limitations may fall particularly hard on junior investigators, on investigators from less advantaged research environments. And we know that the cost of open access publishing is really prohibitive for many scientists, particularly from lower and middle income countries that simply can't afford to pay two, three, four, $10,000 to publish a paper. And these fees are becoming increasingly common, even in the so-called high impact journals, and really place a burden on, particularly on less well-funded investigators. And unfortunately, that may mean that we are open or vulnerable to perpetuating a number of systemic barriers for historically excluded groups. We really need a culture change to address these sorts of concerns that provides funding and support for open science initiatives, being able to include these sorts of costs in one's research grants, being able to have institutional support for these sorts of costs, and providing appropriate incentives for them. Our academic system in neuropsychology as well as other disciplines is really not set up to reward sharing. It's set up to reward individual accomplishment. And until we change that incentive system, it's going to be difficult in some cases to promote this. So the journal neuropsychology that I edit has a number of goals to try to promote open science and are consistent with the American Psychological Association's framework to promote inclusive science. We want to foster an open, transparent, equitable, and reproducible science within the discipline of neuropsychology. And we want to do that by encouraging preregistration, promoting the availability of data, materials, and code, explicitly stating author contributions as a way of equitably recognizing the contributions that each author makes, establishing reporting standards so that all of our, all the papers published in our journal address these concerns, and promoting equity and diversity and inclusion as a general goal, because I think it's fair to say that science really can't be open and transparent, unless it's also diverse and inclusive. So what are the specific initiatives that we're undertaking at the journal to try to promote these goals? Several, a couple years ago, we instituted open science badges modeled on the Center for Open Science and have begun to promote their appearance in the journal. Although we have limited uptake thus far with about 20 papers that have appeared with badges, we've moved to make it easier for authors to request badges and make it a more obvious opportunity for them to show that they have engaged in open science. As part of a larger effort by APA publications, we've also endorsed and begun to implement the Transparency and Open Science guidelines, again promoted by the Center for Open Science. It was very interesting to go through this process with the associate editors and deciding the different criteria that are part of the TOPS guidelines as to which ones we would institute as disclosures as opposed to requirements. And we settled on three that would be requirements in terms of reporting of the availability of code and citations as one example, but others that are simply disclosures as to whether or not, for example, data is available. But I fully expect that over the coming years, we will increasingly move from having simply the requirement of disclosure to the actual requirements of having data available, of having pre-registration. But I think we have to acknowledge that the field is not fully there yet and that it's unlikely that we would be able to institute all the criteria as requirements at this point. More recently, we've also initiated the contributor roles taxonomy author contribution statements or credit. This is a way of allowing authors to explicitly state their contribution to a particular paper. I think this is a really important mechanism for giving credit in an equitable way and being sure that all authors have reason to claim authorship. For too long, I think we've put over much emphasis on being first author or senior author, and then there's everything in between yet as we move to team science and collaborative science, it's increasingly important that we give investigators a chance to state clearly what their contributions have been and we hope that credit will be a way of doing that. This sort of author credit acknowledgment not only promotes openness, but I think greater equity and is something that we've seen adopted by most medical journals already and was one of the reasons we moved in this direction. Most recently, we've started implementing reporting standards in the journal as a way of trying to again promote openness and transparency. In addition to having the transparency and openness section in our methods sections of our papers, we have explicitly called for detailed description of participants and justification of the particular sample to try to move away from exclusively western educated industrialized white types of samples or the so-called weird samples to promote greater inclusivity and generalizability in our samples. We've also begun to require a constraint on generality statement or something along those lines that speaks to whether or not the findings in the paper are going to be generalizable beyond the sample population. If so, why? If not, why not? To be clear that we need to be more circumspect about how generalizable our findings are and promote science in a broader range and more diverse range of populations. The last initiative that I want to talk about is one that we're taking to try to promote diversity in the pipeline for neuropsychology at the level of reviewers and editors because we believe that this is a way to promote more scientists coming from historically excluded groups and therefore broaden the pipeline at all levels and fundamentally believe that promoting diversity in our discipline will also promote more open and transparent science in part because it's likely to be the case that scientists reviewers and editors from historically excluded groups are going to be more sensitive to some of the issues involved in generalizability, the particular sample justification, and some of the methods like participatory-based, community-based types of science that traditionally neuropsychology hasn't made as wide use of. So two programs that we're starting beginning in 2022. One will be a mentored review program for historically excluded groups or individuals from historically excluded groups. This will be a program where we encourage our regular reviewers and our editorial board to identify students from historically excluded groups and involve them in a mentored review process. For students who don't have a reviewer or editorial board member of a local institution, we're going to pair them with someone on our editorial board. We will have them go through a structured process to do a mentored review, a smaller group of mentored reviewers who want to commit to a more intensive program will be identified and provided with some more intensive experience. The goal is to obviously broaden our reviewer pool in the long run and to expose students and trainees from historically excluded groups to the reviewer process, demystify it, encourage them to be able to do reviews for not only neuropsychology but other journals and add them eventually to our editorial board. We will identify all the mentored reviewers each year in acknowledgements and hope to create a student editorial board or trainee editorial board that will appear on the masthead for those mentored reviewers who engage in the more intensive experience. The second program that we're establishing is an editorial fellowship program again for junior scientists from historically excluded groups. This is a program where we take scientists who are in the first 10 years of their career and pair them with one of the associate editors or with me as the editor to go through a year long process where they essentially do a fellowship that involves learning all the steps in the editorial process, shepherding papers through that process, as well as some learning experiences that are geared to both equity, diversity, and inclusion issues but also the scientific publishing process. Each fellow who completes the program will be added to our editorial board and obviously the hope is that in the long run this will broaden the pipeline for our editorial board and our associate editors and as I said a number of times I really hope that the next editor of neuropsychology is selected from one of the historically excluded groups as we try to broaden the diversity of the science which I do believe is a way of increasing openness and transparency. I want to thank you for attending the symposium and the listening to my talk. I want to acknowledge the various agencies, foundations, and others that have been involved in supporting my work in the United States first and then out in Canada after I came up here about seven years ago. I'm sorry I'm not there to be able to answer questions but I hope you enjoy the symposium and enjoy the interaction with the other presenters. Have a great day. Bye-bye. Great and our final panelist tonight will be Dr. Oswald who will talk about open science and his expertise within the discipline of psychology. Great thanks Rose. It's been great hearing everybody's comments from the perspective of their journals and I'm coming at you with a hat on. You can't see it but I'm the chair of the open science methodology committee within APA. I've been a member of that committee since it was initiated in 2018 when Steve Kozlowski was chair and then I took over as chair in 2021. Chair is kind of a glorified title in the sense that we really do work as a team with Katie Corker, Lisa DeBruun, my fellow academic colleagues and then Annie Hall as APA staff. We're all working together to exchange perspectives, ideas, have constructive controversies I suppose and really learn what is happening within APA journals as these practices evolve as the editors are learning themselves. So our ears are open, our eyes are open and I guess there's an opportunity to say please reach out to us if you have any questions. I'll just send my email it's foswaldfoswalde.edu and we'd love to hear from anyone listening who has open science questions who are interacting with APA journals and we just accumulate knowledge and in turn it's a two-way street obviously as we implement our own charge for the journals based on your input. So I'll really talk in broad brushstrokes even though the devil truly is in the details as the editors have reflected on things. But the message, the overall message does drive us forward and seeing at a broad level how efforts are related or not related is really helpful to understanding open science, it's helpful to understanding psychology, it's helpful to understanding the landscape of incentives and resources and processes that are that are going on within the APA ecosystem. Not surprising to you all, I suppose APA seeks to improve research in terms of transparency, reproducibility, and replication. Again the devil really is in the details but as Colin noted it's spurred by attention to the field in terms of the need for replication to have findings that are indeed credible and you know reproducible is a blanket statement just like open sciences. It has many facets I guess that's another point I want to make here is that in the service of transparency there's a whole host of practices that need to be considered in a multifaceted way and that's exactly what the top guidelines are that APA has established within its journals in establishing eight different standards in terms of citation standards, data transparency, I won't list them all but being transparent in terms of research and materials, pre-registering, and so on. So these top guidelines are I guess they're called modular standards which means there are pieces and journals decide on the level at which they would like to adopt those guidelines in line with APA policy so the level could be for authors to disclose whether they used a particular open science practice or there's a requirement at level two that they actually do adopt it and then a level three we'll see as the culture evolves to what extent our journals verifying the practice is meeting some particular standard within the journal but the idea the spirit of it is really important here and that is to develop culture through education through knowledge sharing and through open science itself right because open science materials the sharing of those materials the practice of sharing stands to improve the quality of science it stands to improve the way we communicate to each other and conduct research so it's not just the improvement of the journal article and the scientific outcome of course we want that but it's also improvement of the process of the communication and I mean I'll explicitly acknowledge equity inclusion in that reflection that we want to increase equity inclusion through open science through open communications the fewer barriers there are to research in building this community the better it is not just for again not just for sharing but for improving knowledge and skills we learn beyond the walls of our own institutions and even our own disciplines so the editors have already reflected on a number of points about open science that I think the audience can appreciate but I'll single it out anyway to say that as I'd noted open science is not a monolith it's multifaceted it's a culture as much or more than the practices involved the culture and the practices are are evolving and so is the infrastructure the resource requirements we know in general I think it was pointed out that you know open science sometimes sounds like it's free but it's not free in terms of money resources the time call mentioned the time spent that reviewers spend in terms of their their expertise that they bring to bear if they have that expertise so so we're all educating ourselves while we're reviewing the practices we're engaging in so it's a many faceted enterprise that we're we're nonetheless committed to with all our arrows pointed in in the right direction to stimulate a better research process better research outcome and better researchers again in terms of education so I risk as you can tell I risk repeating myself there but I wanted to kind of highlight what I think is important to the open science methodology committee that that we're here we're listening here we we want to learn and grow the culture as much as the knowledge we want to learn discipline specific concerns and those concerns will evolve as well right that sometimes something that may be shared one way may be shared a different way over time or different strategies for sharing materials and data come up over time as as people engage in open science practices I guess another point I wanted to highlight as as we implement open science is how those behaviors or the the incentives the reputation those can be viewed through different lenses multiple um fruitful lenses so so the journal is one lens uh and and that's clearly important I I serve as associate editor at journal applied psychology working with and for uh Lillian Ebe and her editorial team it's been really great um just fyi but uh it does require as I'm learning in the trenches um the assignment of open science related responsibilities to every part of the journal so you want to implement top guidelines well how what are the what are the what's the responsibility of the authors uh the managing editor the manuscript coordinator the reviewers and and so on and um and so I think you heard every every journal is dealing with with those operational uh decisions which are not trivial um again that's that's where the rubber meets the road and where um you know we have a broad view and a broad mandate um for open science methodology but we want to hear those details and learn across uh across uh every journal as they invest in those details um so I'll say uh two more things um one is uh that uh the open science in addition to implementing the top guidelines the open science methodology committee um has also uh developed in in collaboration with several other professional societies uh pre-registration template um those societies by the way are the British psychological society uh the Leibniz Institute for Psychology in tandem with the German psychological society and and the Center for Open Science uh this pre-registration template is fairly detailed in fact it aligns with jars uh there's sort of a wiring diagram between the components of pre-registration and components of jars and the idea there is to help ensure a more comprehensive research development process uh just like we we take a grocery list to the store so we don't forget the guacamole um we also want to use the template as a way to remind ourselves of the key components as we engage in planning because pre-registration is after all planning and planning is a good thing to focus on and improve right regardless of the top guidelines if open science stimulates the planning the pre-registration template stimulates planning um that that all sounds like good good work to be done in the in the early phase of your of your research um and then finally the road ahead for open science methodology where we're working on appropriate citation for computer software and code this is in line with the credit framework that was mentioned earlier that all parts of the scientific enterprise should be um recognized and um and uh rewarded we should we should be thinking about all this in terms of promotion and tenure committees as well um we're also uh you know api is considering more about the about pre-registration friendly journals ones that are ramping up their efforts even even further or those that haven't started um and want to start how do we uh create that on ramp um as as the open science methodology committee and then we're we're continuing our open science education and and exposure through through various ways of uh broadcasting our efforts through blogs through social media through through short articles uh through communication with with editors um and through authors themselves educating us we can then relay that to others and and further educate in that in that reciprocal relationship I've been I've been emphasizing um so I will stop there I look forward to the Q&A but I hope you all appreciate what all these journal editors are doing the hard work that that they're going through and it really is an honor to support and it's a responsibility to support um and it's enjoyable uh so thank you thank you very much um I'm going to turn things over for the Q&A session to Dr Peggy Christadis who's um in the science director at the American Psychological Association and uh we've been told that people can either raise their hands to ask a question verbally or you can put it uh in the chat feature yes hi everyone um I want to thank our panelists for their presentations and as well as our audience for being here on a Saturday um as Rose mentioned if you have a question you can raise your hand or you can type it in the chat box I also have um a ton of questions that I've kind of been gathering actually over the last year because I I do a lot of work in open science and so I've heard a lot of things so while the audience is maybe um thinking of questions they can type in I can begin with a couple of questions uh that I have um one of the things that I hear a lot about is um how perhaps the transparency of research although it's important is there a dark side to open science and the transparency it requires of researchers um could it open up scientists to bad things like data theft uh public criticism you know like twitter wars and things of that nature or even um misuse of data in the hands of the wrong people who may not have the skill set to work with these data what do you say to people who are reluctant to engage in open science because they have these types of fears and that's for anyone yeah I'm happy to chime in yeah I'm happy to it looks like okay there you are Lillian I really am Peggy and I think particularly sorry I just was saying I think the issues you raise are really important and particularly for people who are not familiar with open science I think this is unfortunately what they sometimes think open science is um I also think unfortunately there there are situations where these type of bad behaviors happen uh threatening requests for data shaming authors who don't share data on social media so I think there's there's really a need for some collegial normative standards for open science and actually Fred and I are working on just a short piece for our professional association around this very issue because sometimes the requests come from people outside psychology who may not have the type of expertise to actually analyze the data or work with the data and come from a culture where it can be very threatening to a junior author to feel like they have to share data so these really real issues I think the base rate is pretty low but all it really takes is one bad experience I think to make people feel really nervous about open science but I'm curious what what Colin and Fred have Colin Fred do you have anything more to add about this dark side I asked Brian Nocic about this as well and he thinks that it's not really that bad that you know there may be an occasional case here and there but usually any kind of criticism is actually quite constructive or people are just asking for maybe additional details or explanation so I hope that alleviates people's fears I can say maybe one one thing about if you like I think I actually wrote about this in an editorial that I that I posted and I think the concern is based on a view I think of science that isn't quite right and that is that one once the research is done or once it's written up or even once it's published that somehow it's done and it's never done it's it's it's a continuing conversation and so I think once we sort of view science in that way then even if somebody you know misanalyzes or misrepresents what we've done that's just the way that the conversation continues so I think we shouldn't shy away from from those kinds of things we should engage them head on and open up the conversation and not be afraid of of where it takes us and I think that's what openness and transparency actually enables so I think the opposite is true the more open and transparent we are the more openly and transparently and and principally we can engage in those conversations with whoever really is willing to to engage with us so I actually think it's freeing actually it's not it's not limiting in that way there's nothing to fear there yeah I think that's a really great way to look at it it looks like we have a question from the audience member um we're gonna see if we can get her to speak Claudia are you able to ask your question you might have to let me see Claudia are you there it looks yeah wait hold on one second guys I can't seem to get her no well unfortunately for whatever reason I can't seem to get that to work so let's go on with some additional questions some of this has to do with more open access related things one person has asked in the past the lack of quality control question mark new open access journals may not have the same perception of legitimacy as established printed journals open access journals do not always have the same level of peer review as established printed journals is this actually true and what are your thoughts on this personally I I lack the experience in working within open access journals I'll say that my experience in submitting the open access journals has been very positive and I say that regardless of the decision that that whether it was accepted or rejected my experience in you know navigating established open access journals has been I've received good good timely reviews I I don't have any I mean these are broad impressions I don't I don't recall the reviews being remarkably better or worse you know that they were they were fairly standard I thought it was you know I think those those processes were were in place there there's also the you know that we live in the in the internet ecosystem now where there's positive aspects and negative aspects for the science right the the negative aspect I'll I'll I'll do the bad first to get it done is uh is that is the misinformation aspect right that that's something did not receive a quality review and and yet still was published and then gets promoted as something that was credible research we see that all the time in peer reviewed research let alone open access that's a reality the positive aspect is that we we also within our ecosystem give papers reviews post publication to Colin's point that science is a continued conversation the journal might be like this you know the fly trapped in amber analogy comes into comes to mind but but with communication comes not only an understanding of that research but an understanding of what research should be following it and I just think we're you know we're all operating off of our histories and our historical ways of thinking about science so so I think with open access journals and new technologies will will you know we're still we're here we are trying to figure this out right but I think the ecosystem is is overall is overall positive but certainly that's certainly that's a concern I'd be curious what others think well if Dr. Leach and Dr. Eby don't have anything more to add we have a lot more questions can you say a few words about qualitative research and what are some of the unique issues or challenges related to open science for qualitative research anyone have anything to say about I'll say a couple of things here sure sure I think and one in particular is that do you have replicas of it being part of the kind of epistemological beliefs of that type of research also qualitative research tends to evolve over time so the idea of pre-registering I think is can be different interview protocols develop and change as you do your interviews or you do your observations in the field so it just poses some unique issues regarding pre-registration and I also think that one of the reasons that open science practices have become common particularly in my sub-discipline is around parking or hypothesizing after the results are known which isn't really that doesn't have the same meaning for qualitative research which often doesn't start with a priori hypotheses so I do worry a little bit about badges and other ways to recognize open science that could be disadvantageous to qualitative researchers particularly in a field like mine that tends to undervalue I think qualitative research so I think those are just some of the initial thoughts on qualitative that's great thank you I have another question in your experience what has been more difficult to do teaching our new generation of psychologists to engage in open science practices or convincing older psychologists like myself who went to graduate school in a pre-open science era to engage in open science where have you seen maybe the most resistance any any experiences with that I can I can say something about that I think I remember those controversies that I mentioned in 2011 when the first one broke of Dietrich Staples misconduct I was actually at a summer school and there was really a generational split and I think I've seen that over and over again and I think there really is a generational divide that younger people recent PhDs and graduate students I think mainly because they're actually engaged in these social media communities that discuss these issues much more often are usually much better informed about about open science practices and have a particular commitment to them and they've kind of had an alternative socialization to research then in graduate school that's probably also because of the way that science has has been popularized in ways for this generation that maybe wasn't true in the past and so I don't know my experience has been a real generational shift and there is there is a bit of a divide and we certainly see that in reviewers and in authors and it's something we're having to work through and you know it is tough and I think I think one thing that's maybe important to acknowledge is of course as you mentioned that you know the training and education that you get at whatever stage is what you build your whole rest that you scaffold your understanding onto that and so it's hard to let go if you let go of that then you let go a lot and so I think we all have to acknowledge that that fundamental changes in how we think about science what we think are good practices are maybe not so easy for a lot of us who scaffolded a lot on top of what we are really learned but I think it's also the case that the truth is that many of us were taught things that now look incredibly naive and wrongheaded and it's also important for us to be really honest and open as we can about common practices that we were taught informal practices that we were taught intuitive ways of understanding and practicing that we were taught that make no sense and I think it's really important on the older generation to be as upfront as possible about what was you know what was problematic in the way that we were socialized and the way that we also maybe even socialized younger people so I think there is a way that that generational divide is going to have to get solved for us all to really get on the same page and work together on these issues. Thank you. We have a question from the audience. They ask what role should tone take in open science conversations? Does delivery matter right? Some people may be very aggressive and demanding that you engage in open science. What do you say to people like that? What sort of tone should we have in order to encourage people to really want to be part of this movement? So I think there are a variety of tones but both positive tones but one tone would be the fact that open science practices accrue to the author as much as or to the researcher as much as to the journal and so don't you want as an author as an early career researcher to engage in these practices that do generally speaking again open science is not one thing but do generally make your work more accessible useful to others. There's research showing that those who share their data those data sets tend to get cited and that then brings credit to the researcher. Also in a researcher handling data requests if you've already shared in the open science spirit but also in practice it's perhaps even routine to honor data requests and kind of tailor what you've already shared to a specific request. So there are other ways to answer that question in terms of tone like one other I'll mention quickly it's just what we've said about social media you know being a being a subtext or a sub tweet for some some of the open science activities that are going on. I would like to think that the more we experience open science as a culture the better we will be at handling open science conversations like data requests and like any conversation that has some element of difficulty like Lily mentioned early career researcher feeling threatened there needs to be multiple people at the table talking not just not just two people duking it out there needs to be multiple minds at play to inform to inform those issues but I think everyone here is an optimist in improving our practices and taking on these challenges. Yes thank you I have another question can open science help or hurt scholars who come from historically excluded or disadvantaged groups I did hear Dr. Yates mention in his talk and he's not here but he did say that you know open science can be for example financially costly so how do we help these groups to engage in open science practices? So I'll just say something quickly here that I just I think there are a variety of angles to this one is simply money you know having money for the to be published in open science journal sometimes there are publication fees so how do we help each other navigate that barrier you know sometimes it's a barrier sometimes it's not it's like you know journals can be flexible libraries can provide funds and sometimes researchers don't know that so you know these are that's the sort of barrier for everyone but for underrepresented minorities first generation college students first generation graduate students they're trying to figure out these the the ropes and they may not come as easily in fact the data probably show they don't come as easily without role models and infrastructure so I think we you know open science is not a one person game we need to support each other and and you know with with our concerns and and need for diversity within our research you know together we need that mutual support to surmount barriers but embedded in that question is is the reality that there are needs to navigate open science you're not you're not born knowing open science so so we need to firm that firm firm that up and and and with the with the concern for diversity equity inclusion great thank you can also add something to that of course so I this is I mean it's like this is my favorite rhetorical trick and maybe it is but I'm going to do it anyway I think this is another example where the truth is actually the opposite of of the way that question of the way we might think about it and that is I think if you're working in a resource poorer environment then actually being very planful being very open and being very transparent is actually an advantage if you know you have few resources to spend then spending them in a more informed plan full open way is actually smarter and I actually think that the the worst abuses in the least open science have actually been performed by the most resource rich places of people because data is cheap participants are cheap and you get into a kind of a factory mentality so I actually think this is another example where you know the opposite is true that in fact fewer resources means you have to be more careful with them and if you're going to deploy them you want to deploy them in the best way possible the most convincing way possible and I think open science does that the other thing about that is and I think this is related to one of the questions in the chat by Gabriela and that is also if we're studying minoritized or hard to reach populations then the same thing applies if there's only a small group of people who were studying we may not want to generalize to everyone you may not be able to replicate 10 times but you know that in fact these participants or this method or this group of people is a precious resource and you want to treat them as a precious resource that they are which means that you have to think very carefully about your methods and what methods work best for maybe that one or two shots that you get to do this study right so again I think these are these I think these concerns are really really important but I think there's nothing inherent in open science that makes those things more difficult and if anything they can actually be resources to study hard to reach or small or otherwise not typical sort of university student populations and they can actually empower people with fewer material resources to do higher quality research. Yeah that's a great point I'm really glad that you're able to see things in a more optimistic way I think people when they see a challenge may sometimes think oh this is I don't know about this is hard and they look at it in a more pessimistic way but like you've mentioned a few times you have to look at it in a more optimistic way and thank you also for answering Gabriella's question in your response. So we have about four minutes left so maybe just a couple more questions. Another question that I've heard quite often is how have contracts with publishers like the one between Elsevier and the University of California that's one example how is that contributed to open science efforts anyone want to tackle that one? Well I it's hard for me to can make this connection I'll start and see if I can get there so I'm I actually shared the the library committee at Rice University where they themselves entered a negotiated agreement with Elsevier in tandem with a I don't know what their formal name is but a partnership of universities in the in the south and you know got saved money for sure on the collections to to then ensure wider access to the extent they would take some of that savings and expand other into other materials that are available. There was a little bit of trade-off with Elsevier and and maybe maybe some other publishers some other publishers they were working with in that some collections were the trade-off was like they they saved money but it was not a permanent acquisition they would have to consistently pay for for access. I don't think I bridged the gap to open to open science there but the goal is to you know in the spirit of open science just generally make science accessible to academics I mean that's what that's what academics is all about and to have a lack of access to publications to the very academics that that may well be contributing to those journals and and yet not have access that that's that would be a sad irony so so it is a it is a it is a struggle but I do think it serves the the spirit and and and the practice open science to have access to these materials you know preprints are helpful as well in and postprints to the extent those are allowable so so that that ecosystem is expanding maybe others have thoughts here. Well we have about a minute left so I really have to ask this one last question um if you if you could look into the future what do you think open science might look like in say 10 20 even 50 years from now um what new open science practices technologies or communities might you see taking hold that's a big question maybe that's something that we should all ponder what will it look like I think we've made tremendous advances just in the last few years so maybe in another 10 you'll see more use of the open science framework more open access all of the all this good stuff so I'm going to hand it back over to Rose and that's going to conclude the Q&A. Thank you for all those engaging questions and the answers that you all provided and thanks to everyone for coming out like we said on a Saturday evening I'm sure you have a lot going on this weekend so it's nice to spend this time with you all talking about open science and how psychology has engaged within that practice thank you to our panelists for being part of this tonight. Thanks Rose. Great thanks for Rose and join it. Thank you.


Yu-Gi-Oh! is on Pawn Stars? We're reacting now. What's up guys? We're back with a video. We don't do very often. We don't react to a lot of stuff on this channel, but this is one that I was like, I probably don't need to react to this and I was like, I need to react to this. Yu-Gi-Oh! Pawn Stars. Pawn Stars. It's time to do a first edition Yu-Gi-Oh! cards for sale season 20. So many of you have seen the original Gary from Pawn Stars with the Pokemon episode. Reacts for like half a million dollars. The dude on Pawns, I don't know his name. Whatever the dude, the main dude, the bald guy. He was like, hey, yeah, that's crazy. Like cards, har har har. And then like this whole thing happened with cards becoming very expensive. Everyone knows that cards are expensive or at least most people know. So this, I feel like this episode is not gonna be like as like interesting because that one was kind of like, What? You want that much for cards? That's so weird. This one they're probably gonna be like, oh, yeah, that's pretty reasonable for you to want that. So we're gonna react to it and this is Yu-Gi-Oh! Prodigy who we were on his podcast. He's been around in Instagram for a very long time. So we're gonna go check this out, see how it went. It's only five minutes and 16 seconds, so it shouldn't be too long of a video. Let's- Collection, I think you might be interested in. Yu-Gi-Oh! Yu-Gi-Oh! So I have a unlimited sealed booster box, also a glossy first edition booster box. All right, so we got a glossy box and an unlimited box so far. This is a booster box and then I have the- Oh yeah, we've opened some of these on the channel. I think I've opened six or seven of these. These are kind of weird because these are a third party package product. So they're first edition packs that a third party such as, you know, like MJ Holding or something like that that we have now with the mystery boxes. They just put them into blister packs. Not sure why they did that, like honestly, I guess just to sell single packs because they never made first edition blisters. If you guys didn't know that, they never made first edition Legend of Blue-Eyes blisters. After that, they did. They made unlimited Legend of Blue Eyes, then they made first edition Metal Raiders, but they never actually made official first edition Legend of Blue Eyes blisters. These are the only ones you can get. So maybe that's why this happened. I don't know. And the blisters and then two of the unlimited blisters. OK, so as you see, like this is what the official would look like. The official blisters that we've opened a lot of on the channel, like Metal Raiders for Honor Guardian, all that stuff. It has like the artwork on the back and then it has like the actual pack in there. But with the repackage ones, it's just like a plastic thing. I'm always disappointed that these don't exist in first edition because if you're like watching Pokemon stuff, they open these first edition Pokemon blisters and like they're legit. And like, you know, they're unweight because they're in a blister and they're actual official Wizards of the Coast products. So it's like if Konami had that, that would be so cool, but they don't. So this is what it would look like with first edition on the pack, which would be awesome, but it doesn't appear at the pawn shop today to sell my one of a kind Yu-Gi-Oh! collection. The products. OK, are they using the same B-roll? This doesn't matter at all. But I'm pretty sure they're using the same B-roll with like the saturation termed up as earlier. Let me go back. Look at this. I haven't unlimited booster backs. Watch this B-roll. I'm here at the pawn shop today to sell my one of a kind Yu-Gi-Oh! It's the same clip. Just turn the saturation up. Truly not the next one. It's the same. They literally use the same clips. They have a five minute segment. It's the same B-roll twice. That's so funny. I'm looking to get 100,000 today. It's 100,000. OK, well, I got a little distracted by the B-roll. OK, so what he's selling an unlimited box, a glossy box first edition and a wavy box first edition with 10 packs. Current prices, that's definitely high. But I'm assuming that they recorded this a while ago. I don't think that they do this like YouTube videos where you record it and then post it in like three days or whatever. I think that it actually probably takes a few months to actually come out. So if this was a few months ago, then this might have been a more reasonable ask. I need to do a little math. OK, he said 100,000. I'm thinking like probably 20,000 for the wavy box. I think there was like a 20, $21,000 sale recently. Maybe like 18 for the glossy 4,000 for the unlimited box and then 8,000 for those loose blisters. That's $50,000. So that's he's asking for double that. This is pond stars. So I figured that you ask more than you actually would take and stuff like that. So that's kind of what I think at the current price would be like maybe like $50,000. Let's just see what happens after this. Now, this is cool. Oh, wait, I forgot about the two unlimited blisters. So those are worth, I think, 300 bucks. So just add like another maybe 250. You just add 50,500 or something like that. You go with the original. Wait, is this an open box? Wait, just a second. Wait, what is this one? Did I miss something? Is this opened? Wait, we're going to go back. Wait a second. OK, wait. This box looks opened. This is not a sealed box. Is he just selling the empty box? This is certainly not sealed. OK, and then he says box and then I have the repackaged packs that you see there in the blisters. OK, so these two are clearly sealed, the unlimited and the wavy box are sealed. The glossy is not sealed. So is there packs in there or not? I'm so confused. And this one would have been the reprinting of it, correct? Correct. The reprinting. OK, that's an interesting way to say unlimited print, which technically is a reprint, you know, unlimited print is a reprint. But it's also like not, you know, like a reprint usually comes out later. So it's a little different. So basically to play the Yu-Gi-Oh trading card game, you would take two players. OK, I saw some of this earlier. I think it's hilarious how they just like explain how to play the game like randomly doesn't even matter. The player would create a deck of 60 cards, then you would start with light points. Not 60, max 60, which I don't think they even add. I would basically the point of the game is for each player to try to eliminate the other player from battle by taking all the light points. The explanation there is hilarious. Take a deck, we go to battle, try to eliminate each other until it's the end of the battle. It's very funny. So how much are you looking to get for everything here? I'd like to get 100,000. 100,000 dollars, huh? OK, that seems a little high to me, but OK, OK, this is. But the Pokemon Gary asked 500,000. Come on, Yu-Gi-Oh, let's ask for 500,000. I know it's only worth like I said 50, but that's what I thought glossy was sealed. So it's probably some of these boxes can be very expensive when you're talking about first edition. So I really need to have someone come down, evaluate the seals of the products and talk about the value of the products. And then I think we can negotiate on a price from there. Yeah, I have no problem with that at all. OK, let me. All right, so who are we getting to go come about? Is it going to be someone we know or is it just like some random person that's like sports card guy that, you know, you know how that went. But I'll be interested to see if this is somebody we know down here. It'll be just a few minutes. OK. I am looking forward for the expert to come in today. I feel like my value is the expert in Yu-Gi-Oh too far off. So I'm really curious to see what his opinion is about it. I want to know when this was recorded because I just and I want to know if that box is seal because that matters a lot because it's like a lot of the value. But I like if this was like back in the boom, you know, there's like 50,000 for a wavy box, like we're actually in the right range at this point. But I don't think it was that long ago. Hey, Matt, how are you doing? Hey, pretty good. How you been? Pretty good. Excellent. OK, who is this? Matt Quinn, senior grading finalizer, CGC trading cards. OK, some guy from CGC. I don't know who this is, but I also don't know a lot about CGC, at least like who's working there. So he's a grading finalizer, whatever that means. I guess that qualifies him. That's what I called you down about. We'll see. We have some blue eyes, white dragon. Yeah, this looks like a pretty cool collection, actually. You know, this stuff is incredible. Oh, that was a better look. Let's see if we can see. Yeah, this looks like a pretty cool collect. All right, guys, that's not sealed. The glossy is not sealed. So we're looking at, OK, 100,000 little crazy. We're probably more like the 35,000 range than the 50,000, I said. Collection, actually, you know, this stuff is incredibly rare these days as it gets more open. You know, there's less of it, so. Oh, here it is. Here's the glossy box. That's opened. I want to know if there's packs in there. Maybe you can get into them and tell me if you think these are all authentically sealed from Konomi. Konomi, bro. Come on, man. With the packs, you really want to be looking at the crimp. So you have the crimp right here. These crimps look absolutely genuine. There's nothing wrong with these packs. They're authentic first-dition packs. That was it. I guess it's only a five, a five minute like segment. But like there's no way that's all I said. All right, let me see this. Where's the booster bag? All right, this thousand dollar pack. This has a crimp right here. OK, good to go. Yep, you're good. OK, and then here we have a sealed unlimited booster box. You know, what you're looking for on these is the seals. Basically, you want to look at that where the plastic is joined together. This wrap looks great. This is this is very, very convincing. You know, I think it's completely authentic. All right, and here we go. Yeah, there's definitely cutting stuff out of this because obviously, you don't just look at it and be like, oh, cool, wrapping. It's good to go. But I think what he did say about like the seals and stuff like that seems legitimate, but there's definitely more things you would look at if you're looking at a sealed booster box. Here we go. The holy grail of the Yu-Hi-Oh collecting world. Yeah, holy grail and sealed booster box. These are incredibly rare. They very rarely come to market. I'm amazed to see one of my hands right now, actually. And where he looks amazed. There's also a caption right over his eyes. And I turned it over right away. And it's where OK. October twenty twenty first edition was my dragon card sold for over eighty five thousand. My one ad that was a PSA 10 and also that price has come down a little bit since then. But yeah, this was I what I wonder I don't know when this is recorded, but there's no way to know. You feel happy. There's a flap on the bottom right here. This flap is important because without that flap, you know, it's going to be resealed. In my mind, I look at me so happy. I was like, oh, yeah, baby, let's go. My mind, this is a legitimate first edition box. And it's it's from the first printing. So this is what's called the wavy set. The wavy you guys want to know a really good way to tell if these are resealed. First of all, the wavy printing first edition right here. The smaller text that this is, that's wavy. The larger text first edition is always glossy. The reseals and like the the fakes and all that stuff, they're almost always wavy print. So it would be this print. But you need to look at the Konami logo. Let's see if we can get a good look at it. The best way to tell what this stuff is with this Konami logo right here. If the O and the N are touching, this has nothing to do with resealed. This has to do with the fake. So if you guys have seen Simply Unlucky's video from like 10 years ago where you open that red back card, like he opened it, all the cards looked really weird. And they were like fake counterfeits of Legend of Blue Eyes. This is how you tell if it's one of those. If this O and this N are touching together, that means it's a fake because in the print run of those fake cards, they had like a mess up in the Konami logo when they were trying to fabricate and make a fake looking Konami logo, not fake looking, but a fake real looking one. The O and N were touching. And that's the case with every single fake. So if you see that, do not touch that box because it's fake. You're going to get those red backed fakes in it. So keep that in mind if you're ever trying to like look to buy a wavy box and stuff like that. That is an actual way to tell. I've told multiple fakes from that. So keep that in mind. That's an actual good way to tell. And then the reseals a little bit harder. So what value can we put on the whole collection? The ones that are sealed in the Hanger packs. Those are probably about a thousand a piece. OK, thousand a piece. That's actually pretty accurate. I had them at eight hundred a piece, but I think a thousand. Honestly, you might actually get that nowadays, too. So a thousand is probably pretty fair for them. And the seal booster box there, you know, probably four thousand. That's exactly what I had. I had four thousand for the unlimited. OK. This Yu-Gi-Oh box, this first edition box is incredibly important, obviously. I'd say the value of that box today. OK, thirty five thousand. So this was definitely, I mean, this obviously wasn't like last week. I think there was a sale for like twenty one thousand, like two weeks ago on an auction site. I think it was Heritage. I can't remember which one, but thirty five. That was the case, you know, a few months ago. So this could have been then looking at it all together. I mean, if I priced it out in my head, I mean, fifty five thousand dollars. All right, well, I think that answers it. Oh, oh, no, look at that. Well, there goes my hundred thousand. I do want this stuff, but I don't know that I can pay that much for it. I'm just going to try and offer you twenty five thousand for this first edition box. Oh, he's just going for the box. OK, I respect that twenty five thousand. Honestly, very good offer. Like even though it was thirty five thousand, that's really not a bad offer. I think Prodigy knows he could get more, though, if he was just selling it like on Instagram to like a certain collector. He could probably back when it was thirty five. Now he couldn't. But like back when it was thirty five thousand, he could probably got more than twenty five thousand. But nowadays, twenty five thousand would be a great offer because one just sold for like twenty one, so it'd be great box here and hope you take it. Oh, I've had multiple offers higher than that. Yeah, multiple offers higher than that. Yeah, he honestly probably had. So there's no way I can go that low. All right. Well, it looks like we're not going to make a deal today. Well, it was quick. There's no negotiating. He's like, well, that's it. There's no no haggling. If you have anything else that you want to bring by, I would love to take a look at it. Sounds good. All right. Have a good day. Thank you. Thank you. That's it. It just ends abruptly. Like you think there's going to be some sort of haggling and stuff, but there's not.  So that's actually a pretty interesting episode. I don't it's not as good as the Pokemon one because, of course, Chumlee actually knew what he was looking at. He knew what he was like wanted to offer. He's actually interested in the cards versus like the first time, whatever the main dude's name is, like Rick or something. He's like, no, you're crazy. I don't know what he's paying for this. And clearly people were and it got crazy. But this was fun to look at. I actually went on a lot longer than I thought just talking about some of the stuff. But I enjoyed seeing this. It's cool that Yu-Gi-Oh made it on to Pond Stars kind of like five years later after Pokemon, but it's about time that it happened. If you guys enjoy this reaction video, let me know if you want to see more of these. Nothing really happens in Yu-Gi-Oh. So our next reaction will probably be next year, but we'll see. Make sure to subscribe to the channel for more epic content. Shout out to Toe Info Show, Daxter, Tomato Juice, JT Show, TCG Trusted Cards, Puffins of Doom, Ernesto DeAnda, Dizzy, Flexi Boy, Hoppus, Choice 333, Missycle, Melancholy, and at his high show, Christopher Ward, Ian Musa, John Nolan, Junior Barting, Mike Nance, Mimigakko, Stanley, and Thomas McLean. Thank you guys for supporting the channel. And I'll see you guys in the next one. Peace.


Again, people straight to the point. If you've already completed your Titan Solstice armour, this video is not for you. This video is however for anyone who hasn't completed their Titan Solstice gear yet. I know many people work or study during a week, or so many people will be starting their second or third characters. If that's a Titan, this video is definitely for you. So the one challenge on the rare Solstice Titan armour I keep seeing people ask about is the 300 solar melee kills. Today guys with a couple of great strats or cheeses as many will say these will allow you to get these done in no time at all. How's it going guys? My name is DPJ and I'd like to thank you for stopping by and checking out my video. If you do enjoy it leaving a like really helps out and if you are new around here and enjoy daily Destiny videos be sure to subscribe.  You can't just run around and keep melee and enemies on a solar subclass that doesn't work. You have to have a fully charged melee. One decent strat is using the middle tree throwing hammer. It works yes but the other strats today I bring you are miles faster. So the first strat which works great is using a sword. Yes this works but only when using the hilt of the sword. So if you have heavy ammo burn this out and then just hit enemies with a hilt of the sword. This weirdly counts but actually having heavy ammo in your sword even if it's a solar sword it doesn't work. But the hilt of the sword is great. Just find yourself a decent sparse to like the escalation protocol on Mars with a shiro chain counter and a last wish raid and you'll get this done pretty quickly. Another strat which works best in my opinion is within the forge. You know those orbs or bulls you have to pick up and throw at the forge? Well if you hit enemies with these using your melee button these actually count too and you can actually hit pretty quick which is why I think this is better than that hilt strat. A few runs people and you should have your 300 solar melee kills done in no time at all. Shout out to my powerporter runner one for finding this out and letting me know. Now people I have posted many other strats which should help all characters out with the solstice of heroes armor and if you're interested check them out by clicking my channel icon or channel name. By that note guys we have come to the end of the video if you guys enjoyed it leave them like really helps out. If you are new around here and enjoy daily testing the videos be sure to subscribe and if you never want to miss a video I upload you can turn notifications on by hitting that bell button. But guys thanks as always stopping by hopefully you enjoyed the video and hopefully I will see you on that next one.


In this video, I will be sharing the top reviews of the book called, How I Saved the World, authored by Jess Waters. Jesse Waters is the co-host of The Five and host of Waters World. Before this, Jesse was a producer and correspondent for The O'Reilly Factor, which showcased his comedic street interviews, aggressive confrontations and entertaining adventures all across the country. This book is on the New York bestseller list for the last four weeks and has ratings over 4.4. Before I get to the review part, let's see what this book is about. Many things have made our nation great. An unparalleled group of founding fathers. A history of increasing freedom and equality. A tradition of muckraking journalism.  In How I Saved the World, Waters takes readers on a tour of his life from basement to pampered champion of right-thinking Americans. He has devined great truths about the nature of our country while stumbling across beaches asking oblivious college students basic political questions and while stumbling over basic political questions on Air Force One with the president. Interspersed are his thoughtful suggestions for overcoming left-wing radicalism, maintaining American democracy, moving beyond aging hippies, like his long-suffering parents, saving the world from social justice warriors in the deep state. All while rolling his eyes at Juan Williams in only the nicest way. Waters outlines the stark choices ahead of us, between all American hamburgers and leftist Green New Deal breadlines, okay, maybe that one is a no-brainer. And shows the way for order and fairness to be restored. A manifesto and a call to arms from a man for all seasons. How I Saved the World is a hilarious, enlightening, entertaining book with a reasonable chance of winning a Nobel Prize in every category, even chemistry. Now let's get to the reviews. Henry from New Jersey says most books by news reporters or pundits are pretty boring and self-aggrandizing or political diatribes. Jesse Waters, How I Saved the World, is mostly a memoir, but certainly does set forth his politics. However, whether you agree or disagree with those politics, this book is entertaining, funny and a refreshing breath of fresh air in the genre. Waters doesn't pretend to be what he is not. His humor is almost entirely self-deprecating and he never takes himself very seriously. Jeanette from Illinois says very much about his personality. Very little of the truly personal revealed. He is a tough cookie. Despite his mom emails, texts and youth stories, this is all about the TV news career, persona of his beginning, process in the media business. Not that that aspect is a bad thing at all, but just not what you would expect from a memoir of someone who is 41 or 42 at the writing. It's 90% after his failed jobs and interview at Fox. Loved some of the chapters, but most were name call, name dropped fodder and ambush detail, interview, stories, history. I thought he was at core honest throughout, but at the same time, his personality is so knee centered that it becomes a framed with savvy portrait more than a heartfelt bilge of feeling reckoning or some reactive to experience positive his job focus. Actually, I truly enjoyed it, but could barely care less about all the CPAC and media dinner extravaganza of he said, she said to be enthralled. So many of these people are not worth the read time. Truly. Most especially the Hollywood or music scene sports so-called icons. Most of them are mental midgets. The stories often confirm. But it also takes away way more guile and hubris to mix. So much posturing. Lori from Las Vegas says I loved and laughed in the beginning. I thought and stopped sometimes during the reading. I finished with I loved the book. I love Jesse. I love the five. Thank you Jesse for allowing us to see you in daily attempt to figure out Gutfeld. I never miss his program either. Cara from the States says how I saved the world is a sometimes funny, often perceptive memoir by Jess Waters, a political commentator on Fox News. He has his own show, Waters World, and is co-host of the very popular The Five. In How I Saved the World, he shares how he got where he is today, the jobs he's had and lost, and the ones where he feels he had the most impact. A good number of the chapters were about assignments he had for the O'Reilly Factor and his experiences ambushing various nefarious characters who didn't want to be interviewed by the press. He shares texts from his politically liberal mother who critiques his performance on The Five, and snippets of random comments from one of his co-hosts. My favorite parts of the book were the hysterically funny Chapter 1 in which a teenage Jesse is sent to survival camp. How I Saved the Great Outdoors, and Chapter 16, How I Saved the Election, which gives a good overview of what happened during President Trump's last year in office. Mandy from States says this book was really funny and did a good job of highlighting what Jesse has done since graduating college. I've forgotten some of his factor-sticks so those were fun to relive. There were really no personal stories though and those sprinkled in would have been nice to know that side of Jesse. Even if someone doesn't agree with his politics, they would probably still enjoy this book. Now we are at the end of this video and I would like to say that this book has been receiving a lot of good comments and some are bad too. But overall this seems like a pretty good book. So I think it's worth a read. I picked these comments from the Goodreads website, feel free to check the description. Thank you for watching this video.


Hi everyone, Joe for jaskiescasebreaks.com coming at you with a random number block randomizer for this big group of teams right here. There are the instructions right there on how it works, it's a serial number break essentially. Big thanks to this group for knocking it out. There are the numbers right there, those represent the block of numbers, and let's roll it. Randomize names, number snake eyes, easy. One and two, boom. Rob down to Anthony, and then snake eyes for the numbers. Left eye of the snake, right eye. Four down to two. Alright, Rob with the number four, Brian with seven, Michael with three. Rob with one, Jeremy with zero. Now remember, Jeremy zero as it says in the description gets any and all redemptions, including one of one redemptions, obviously only for that group of teams down there. But Rob you'll still get live one of ones. Rob has nine and five as well, Aaron with eight, Brian with six, Anthony with two. Alright, coming up in the next video will be the break itself, jaskiescasebreaks.com.


What we're seeing more and more is that states which see that so much debate and reporting and Descent takes place on social media platforms. They're often going to those platforms directly outside of any legal process in order to ask those platforms to take down content and we should be okay with Platforms that for example take down child sexual exploitation or incitement of violence I think it's completely fair for the companies to regulate that kind of content and they have a role in I think ensuring that their platforms don't become cesspools of hatred because that in itself interferes with the ability of people to express themselves But when the companies are moderating content, that is when they're deciding what kind of content is appropriate and inappropriate I think we expect a few things we expect that they're transparent about their process for deciding what rules are going to be applied transparent about what those rules look like and how they're made clear about the nature of those rules and they provide a kind of Appeals process and accountability for Wrongful takedowns of contents