What's going on everybody welcome back to another video. Today we're going to be creating a script to automatically take data from a crypto API. Now this project stems from an earlier video that I did where I walked through what an API was and how you can use it and in that video I showed you how to use coin market caps API so you can start pulling in their crypto data and in this video we're going to take it one step further and automate that process then we're going to do a little bit of transformation with the data I'm going to show you some cool stuff of how you can use it and maybe we'll do a little bit of visualization at the end but that is not the main point of this video it's mostly around the automation piece and a little bit of the data cleaning piece as well. Now fair warning this is not a beginner's level project it's probably more like an intermediate project and it's not even a complete project per se because we're not doing all the data cleaning we're not doing all the visualizations but if you follow along we're going to cover a lot of different things and you're really going to set yourself up to be able to do just about anything you want with this data or different APIs that you pull from so with that being said let's jump onto my screen and get started with the project all right so this is where we stopped in our last video so if you haven't watched it now is the time to go back and do that I'll have a link in the description also all the code that we're going to be looking at today and working through is going to be in a github repo below so you can go and get all the code and have it completely finished and just follow along or you can code it from scratch along with me I do recommend writing it from scratch if you can because I think you'll learn more and you'll make mistakes and you'll learn from that as we go through it but it is up to you so let's get started and as you can see we have the script right here and I'm starting basically from scratch I have a completed one up here I'm actually going to get rid of those and what we're going to do is we're going to start from exactly where we started in our last one I'm going to run the script this is going to pull from our API and we're going to look at the dictionary option and do our JSON normalize so this is where we literally left off from the from the last video so we have all of this data and what we want to do with it is we want to kind of automate that process right because we don't want to have to come in here run this and you know put into a CSV manually or something like that we want to automate this data collection process so that we can just have the data ready for us to use and it all be ready to go so we're going to be using this script but you know we we might want to add a little bit more to it before we do that the first thing that I want to do before before anything is something that I like to do when I'm creating these automation scripts as I like to add a timestamp and the reason for that is because I want to know when I ran or when each of those loops you can say runs through and does those automated runs right so if I do it every day I want to know what time of day I ran it making sure each run ran successfully and so all I'm going to do is I'm going to add a new column at the end just call it timestamp so let's go right up here and we're going to say a pd dot and there's something called to date time so we're going to do to underscore date time and then we're going to do now and what this is literally going to do is take the the date of the the timestamp of right now when it's running and it's going to show that now we need to of course add a new a new column for that so all we're going to do is going to say data frame whoops let's say data frame and let me see real quick we just have the data we need to we need to create this data frame right here so data frame equals and then this JSON normalized and we're going to say data frame and then we're going to do a bracket and we're going to say timestamp and we'll do well all these lowercase we're going to keep with the lowercase we're going to say timestamp we'll do that bracket and we'll say equals so what this is going to do is going to first off it's going to create this data or assign this df as our data frame and then we're going to add this timestamp and add this new column and so let's run this really quickly and let's go all the way to the right and this is our timestamp and this is the time that it is right now this is the day that I'm running it this is the time that I'm running it and so this is working properly now if you look really quickly there is a last updated in here and this is very close to the timestamp but it is not the same thing but if you look to do this data and you really dug into it a little bit there's this last update is coming from coin market caps api and this is when the actual cryptocurrency was updated in their system and so it is going to be really close but it's not going to be exact and so I don't like to rely on built-in ones that you know are coming from api or something I want to make one myself that's running on the system where I'm creating the automated process just like just something I do so now we have this original data frame created right we now have what we need but what we want to do is to keep adding data to this we don't want it to just go to you know create these 5000 rows we want it to create 5000 5000 5000 over time whether it's a day an hour a week whatever you want to run it so what I'm actually going to do is I'm going to limit this a lot I just want to look at the top let's say 15 that we're going to run through all this again so now I just have top 15 it's going to be easier to to see and it won't take as much time to run our scripts again you can keep as many as you'd like you want 100 200 all 5000 you do whatever you'd like but what we are now going to do is we're going to create a function using this original script so we again we have this data frame and we are going to create an automated process that is going to an automated a script to automate this that is going to append data to this data frame right here so that's kind of you know the big thing that we're trying to accomplish in this project so let's go up here and we're going to we'll just take from here all the way to here we're just going to copy this and I'm going to paste it down here now what we need to do is we need to create a function so we're going to say def and we're going to call this the api underscore runner this is going to run our api whenever we need it to run now when you are formatting something for a function it needs to be formatted properly and so what we need to do is need to go over here when they hit tap we're going to do this all the way down I'm just going to skip forward when it's all the way done all right so now we have this URL and what we want to add because this is again this is going to run through kind of this this automated process we're going to run this this function there what we want is to also add this right here so we need to take this and we're going to need to add this I'll just put it down here okay now let's do that so what we have so far is really close to what we want our function to be we have this function that we're going to be running through it's going to call this function it's going to call the the api we're going to use our key we are going to you know test it load it format it format it right here then we're going to add this timestamp and then we will have this now right now it's just call it's just going to print this data frame basically but that's not what we want right now what we want is to actually append this data so when it gets to here when it gets to this data that's going to be right right here what we want to do now since we already have the original data frame set up up top is we now want to say that this is going to be data frame two and we're going to say it's going to append it to data frame two and so the original data frame we're going to say data frame two dot append and we're going to say df2 all this does is this says this new data that's going to be coming in every time let's say it's a loop and it's just looping through pulling the data pulling the data point data we're going to create this data frame we're going to add it add this timestamp like like we want and then we are going to append that to this original data frame so as of right now this looks good i will run it in a second i'll create it so i just created it so now we need to actually create our script to automatically run this so we're going to do something called import os and let me tell you there's a thousand different ways to do this and there are better ways to do this but they are much more complex much more complicated and some cost money in order to do it i'm going to show you different options on how to do this in future videos on how to automate your python scripts but this one to me is when i've used a lot many many times for different projects and it works so i'm not going to show you the most complicated thing in the world i'm going to show you something that i've just used a lot and so we're going to say from time import time from time import sleep that one's important and now we're going to create our loop so what these um what the time and the sleep the os uh your operating system what what these are going to do is they're going to give us the ability to track the time and we're going to be able to run through and call this function in certain intervals that we want so let's create our for loop we're going to say for i in now you can create this specific part in different ways but what i'm going to do is i'm going to say range of one uh let's say three hundred and thirty three and i say three hundred and thirty three and if you remember from the first video in the api you only have three hundred and thirty three runs per day and so if i ran this three hundred and thirty three times today that would be our max and so that's why i'm using that three hundred and thirty three just for reference so now we're going to do api underscore runner so in this loop we're going to call this function up here and then i'm going to say i want to prove or show have an output to show that this is running through successfully i'm just gonna and you can write anything here we're just going to say api runner completed uh completed successfully successfully how do you spell that successfully that doesn't look right i'm just gonna say completed all right forget that i don't remember how to say uh spell successfully if that's if it's spelled it right you guys spell it that way but i can't remember now we're going to use this sleep right here now this counts it in seconds you can change it to minutes hours whatever we're going to have it run every minute which is every 60 seconds and so this is going to i'm just going to say it's going to sleep for one minute and then we're going to say exit so all this is going to do and this is again fairly simple it's just a simple for loop and what it says is it's going to call this api it's going to tell us that it ran successfully and then it's going to wait for 60 seconds and it's going to run again that's it so let's run this and see what happens see if what we did works so it ran at the first time now i'm not gonna i'm not going to bore you because i'm doing this live exactly what we're about to get is what we're going to use i didn't run it overnight or for a week so that we have a bunch of data and what you were going to work with i'm going to work with as well so i'm going to wait a few minutes i'm going to let this run i want you to do the same thing i'm going to let this run for me like five minutes or so and we'll work with what we have and we'll keep going with the project because again we're not the point of this project is not to create the final product we're creating all the visualizations that will most likely be in another video where we're taking all this data and doing all these things with it the point of this video is to automate it clean it up to where we have it to where we can really use it and then i'm going to let you guys loose and you guys can do whatever you want with it and i think it's really setting you up for a lot of successful projects in the future that you can do all by yourself without me having to walk you through it so as you can see it's already ran through twice i'm going to pause for a second i'm going to let that run through just a few more times and then we will continue with the project all right we are back and of course it's only ran five times um it has not reached the limit of three three so we are perfectly fine what i'm going to do is i'm just going to stop this by clicking this square up here and it's going to give us some error and then we're going to check it and we will see what we have i don't know why it's taking so long if i'm being honest all right so i interrupted it and let's run this let's see what we got i hope we have more than 15 so i'm gonna be very upset okay so okay well uh i made a mistake um i was supposed to put data frame right here and i had data frame two so um take change your script do not do what i just did we're supposed to be a append it's supposed to be data frame append and we're supposed to be appending the original data this data frame two the original data frame so um i messed up on that one let's rerun that let's rerun that um let's see um local variable df reference before assignment okay this is perfect because this happened to me before um we're running into all sorts of good stuff i like to keep the stuff in my videos i laugh because i hate running into mistakes but everybody says they they're happy that i do this um so i'm gonna keep doing it i'm not going to cut this out i promise um but what we actually need to do is we need to go back up to this function because what happened was is we called this data frame and now it's it's because it's in a function it's in what they would call a local variable what we need to do is we now need to state that this is a global um it's just called a global that's all it is um and so what we're going to do is we're going to do tab let's say global let's say df and what this should do is this should declare it as a global variable and it should let this run properly let's hope it does all right it's running um again i ran into mistakes let me tell you something while we're here for just a second this project i ran into probably a hundred mistakes or a hundred errors issues that i had to research for hours um and hours i'm legitimately on stack overflow and just googling and figuring figuring these things out there are a lot of new things that i had never run into before um just on this project and so um everything that you're seeing is from after i went through all of those things uh or after i fixed all of those things and had to really work through them it was it was very um it was frustrating at times i just i couldn't figure it out and so what you're looking at is kind of the polished version of that now that i have everything laid out it's because i i can't spend 10 hours on a project nobody would watch it so just know that if you are running into some of these mistakes or you run some mistakes later on when you're expanding this project that's completely normal so what we're going to do is we're going to let this run for a little bit and then after maybe three or four minutes we'll come back and we'll keep going with the project all right so let's run this and check and see if we have the data that we're looking for and it looks like we do let's go actually back up here really quick we want to set this to display max rows because i want to be able to see all the rows and not just a few of them so and that just instead of it gives us this scrolling so that dot dot dot that shows us just a few so there's our original 15 and then we have the next the next loop and then we have the next loop and let me scroll over to the timestamps and i'll show you what i mean um so this was ran on 526 1501 let's go down 526 at 20905 20905 the next one you can see was ran at 3006 31 these are all the ones one minute after each other my original one was from earlier 32 33 yeah so you can see 32 31 30 30 or 30 29 and this one was about 15 minutes ago when I first um ran the original data frame right all right guys this is alex from the future i've actually completed this entire project in the video and you're about to see all that after this but i wanted to show you one more thing that you can do in this function up here that i didn't show you uh originally that i'm coming back to show you and that's how to actually put it into a csv now all we've done in this one is we we've kept it all enclosed in a data frame and that's it and that may be great but a lot of you guys are going to want to automate this and put it into a csv and i want to show you how to do that all right so what i'm going to show you really quickly is right here in this uh in this folder right here i have all these different api threes and fours these were tests that i did before but what you can do is instead of just putting it into a data frame you can actually append the data to a csv and have that csv sitting out there for you instead of just keeping it all in the data frame and there's a lot of different uses for that you may want to have that file separately from here just in case something times out or something breaks which is a legitimate concern or your computer shuts off or or something like that that is a legitimate concern so what we're gonna do is we're going to say um if not and this could basically uh an if statement we're gonna say os dot path dot is file so what this is going to do is check if there's already a file under this name and we're gonna do r dot or r um if you have never done um if you've never done csv stuff before it's really important that you put that you're gonna get an error every time so we're gonna take this right here and we're gonna copy that and we're gonna put that right here and then we're also gonna do a slash and then we're gonna name it basically um let's name this api because i don't think i have that one in there yeah i deleted it yeah so i don't have api so i'm just gonna keep api dot csv and then i'm gonna close up parentheses and then we're gonna add a colon right here and we're gonna say if that does not exist we are going to write this to it and create it so we're gonna say data frames that's this data frame right here data frame dot i'm gonna say to underscore csv and we're gonna do that r and then we're going to copy this so let's just let's just replace it like that and then we're gonna say comma header header is equal to column underscore names so what this is going to do is if we run through this and what we would have to do is um i'll talk about this a little bit we'll have to change this up a little bit but what this is gonna do is gonna check to see if this file right here exists if it does not it is going to create it and create the column headers based off the this data frame that is what that does now what we want to do is say else and this next part that we're gonna write is saying if there's already the api file there we want to append the data we don't want to overwrite it or anything like that we want to append the data so we're gonna say we're basically gonna copy this maybe not the whole thing but i already did it um so we're gonna copy that i'm gonna say mode oops mode equals a and a stands for append and then we're gonna say header keep messing up header i'm gonna say false oops we're gonna say false which means when it depends the data it's not going to use those call the column headers every time which you don't want because every time you append it if you added the headers every 15 rows every 15 rows you're gonna have another headers that you're gonna have to like go out into that csv and filter out and get rid of them so we're gonna say header equals false now just a second ago i said you would need to mess with this just a little bit and you would because every time um you'd be putting in this data frame which it's already appending it to this data frame so every time you'd be creating a lot of duplicates if you kept it exactly as is what you were gonna need to do is basically take it back to its to its um bones um so you need to kind of keep it like this so what you need to do is just now run this and it would work perfectly uh let's test it really quick um to see if it works uh because i'm i'm promising you something i want to make sure it actually works let's run it this time okay so it just ran for the first time so it should have created this file let's go see if that works properly so now it just created that file and now we're going to see if it actually appends the data so let's wait just one time um and then i'm gonna stop it i'm gonna see if it works again i'm just verifying to make sure that what i'm telling you is actually working because if it doesn't i would feel terrible uh we don't want that and while that's running actually i'm going to add this because now i want to show you how to call it um super easy we're just gonna do pd dot read underscore csv do that we're gonna call this just like that and then we're gonna say data frame and we're just gonna do 72 something random because i've already done this whole project i don't want to mess anything up so i'm gonna say data frame 72 so now let's stop this and what we're going to do is once that stops we're gonna run this and see if it actually worked and to see make sure that this actually pulled the data in all right so we interrupted it the file is ready to be read in so let's read it in there's our file um let's see what did i mess up or did i mess anything up ah i didn't mess anything up this is the index for this file and we already have this in here we'd probably be able to get rid of it but if you see we have zero one two three four five six seven eight fourteen then we have zero one two three and if we look at the time stamp it should be one minute apart so it's one nineteen forty five instead one twenty forty five so this worked exactly as planned again you have two different options you can just keep it how it was before and i'll leave both of those options you know in the in the script so that you can kind of choose which one you want but um that's how you do that so then right here you're appending it to a csv file and then if you just keep this and you get rid of all this you're just appending it to a data frame now please continue with the rest of the video that i already have done um but again i'm future alex so uh please continue with the rest of the video okay so we have all this data we have we have so many columns we can do now you know if you want to completely just go and do your own thing you absolutely can do that i'm going to mess around with a few things um kind of show you something that i did that i thought was really interesting um in order to visualize this data a little bit and transform it a little bit to make it more usable um but we're not doing a full data claim that's not what this project is we're not doing a full data cleaning of this data that would be a very large undertaking because honestly this needs a lot of work one thing that i do want to clean up really quick uh is is this right here this the the map will be fine it's just the way that it's shown on here is in state the scientific notation and i don't like it so what i'm going to do really quickly is just um get rid of that so we're going to uh we're going to say pd.set and do underscore option and this is going to be uh do a parentheses i'm going to say display this is just this how this is formatted it's going to display uh dot float underscore format and we're going to say comma and we're now we're going to use this lambda say x colon and we're going to say percent point five f that right there and we're going to say percent x now if you don't know what lambdas are um i highly recommend looking those up um again this is not a beginner's tutorial whoops no such keys display floor format that makes sense uh this is float yeah guys this is not a beginner's level all right uh you can't use the floor format this is the float format all right so now let's take a look at this uh this df uh this data frame that we have so we're just going to hit df click enter and now our numbers are a little bit more easily readable i prefer it this way you do not have to do this i'm doing this just because this is what i prefer so let's jump right into it um something that when i saw this data i was like something that i really thought was interesting is this percent change of one hour percent change 24 hours seven days 30 days 60 days 90 days if you're not in crypto you don't do investing or anything like that what this is going to show us is how i mean it's pretty obvious how much the price of this coin has changed over the last hour 24 hour seven days so as you can see it's it's barely fluctuated over the past 24 hours a little bit over the past seven days a lot of last 30 days 60 days and 90 days 20 minus 26 percent minus 33 percent we're in may we just had a kind of a crash in crypto a couple weeks ago so i mean this tracks right but i want to visualize this see this and kind of see um you know how this is going to look and how if i can gain any insight from that information and just having it all displayed for me but in its current state um you know we really cannot do that um now another issue not an issue but another thing that we have to take into consideration is we have bitcoin net right here we have bitcoin right here after different polls now we just did it a minute after each other but for your project you may do it a a run each day a run every hour or something like that right and if you did that your data could be very different and so you may just want to take this first one but what i'm going to do for the sake of this project i'm going to group them so let's go down here and we're going to say df dot group by and so if you've ever done something like sequel this is how you group by in pandas basically we're going to group by the name so so on bitcoin ethereum together so we're going to we're going to do that on name and i'm not going to i'm going to say sort is equal to i'm not going to sort it uh you could say true there but we're not going to i guess you'll see why later we're gonna do an open bracket and now we need to choose what we're going to group by uh or what we're gonna what columns we're gonna have so i'm going to do another open bracket and i'm just going to copy and paste these so i'm going to start right here at quote percent one hour so i'm going to do boom and then go over one and we're going to take 24 hours paste that comma we have the seven day 30 day and we're going to do like that and i'm just going to do comma i'm going to do the same one but i'm just going to manually change it to 30 day that at the end i don't know what that is and then we're going to do 60 days and comma and we're going to do our last one which is 90 days and let's see what that gives us doesn't give us anything okay i know what's wrong here we forgot to add basically the what we're we have we're grouping by something we need to have like an average a mean a mode or something like that right so all we have to do is go to the end right here and let's just do we're gonna do an average um and so we're taking this number so let's say this is for bitcoin so we're going to take this number in this one hour for every time it's bitcoin it's going to group them all together um and then it's going to average them so in the past five minutes where it's been running we're going to take the average of that so let's run this again and so now this is our output let's take a look oops i meant down here let's run this now now what we have is all of these um cryptos these are all 15 that we have and this is the average um it's one hour 24 seven days 30 days 60 days and 90 days so now we have all of our cryptocurrencies over here we have our percent changes up top and then our averages here as well and so now what we're going to do is you know if you try to visualize this as is it doesn't really work because these percent changes are up here as columns and we don't really want them as columns because that it just doesn't work for visual we're actually creating the visualizations we really need these to be rows and so my initial thought when I was doing this was I of course I need to pivot um you know if you've ever used a pivot like an excel or or power vi or something like that that was my first thought and I tried everything and I could get not could not get it to work and I almost gave up until I ran across um something called stacking or a stack and so this was not something that I I think I have used it before but like I couldn't remember to be if I'm being completely right I couldn't remember how to do this so I just did um once I saw what it was I did stack let's make that or you don't have to do this you can keep this all the original data frame I'm just I like for visual purposes you can see like the progression that we're making um but I like to you know create it's a new data frame and I can always go back and look at this data frame three um as we go but you don't you don't have to do that that's just what I'm doing so now let's take a look at this now up here we had bitcoin and we had all these columns and we had these numbers as rows but now we have all of these as rows as well this how we have this is much much more usable and if you've ever done something like pivot or the stacking before you'll know that you you kind of have to do it if you really want to visualize this well but um you because we just stacked it it kind of changed it so if we look at um let's look at the type of type of data frame three this is before um before we stacked it this was in a data frame but now let's go look at data frame four so this is a series this is no longer a data frame so we have to remember that that's that's really important because we can no longer treat it as a data frame it's now a series so we want to get it back to a data frame we don't want it to be like that because you can't really use it in the series so what we're going to do and let me just create a few of these so it can be up here better so now what we're going to do is we're going to say data frame four dot and something called two underscore frame so we're going to make this into a frame and now we're going to specify the name and it doesn't mean um the name like right here we actually mean the name of these values right here this is part of the stacking process and in these columns are these two columns so let's go right here and we're going to call it let's just say values and let's make this data frame five and let's see the output whoops four data frame five and now so there's that values and now this already looks a lot better right so it's in this it's in this more um this is already a data so this is a data frame so let's look type data frame five so now it's in a data frame but the issue is is that this name is kind of acting like an index which we don't want because we want to be able to use this so it doesn't really have an index at the moment so we need to give it an index but typically when you give an index you'll do something like um we'll say data frame five we'll do set underscore index and then you'll do something like um name uh so let's just do data six is equal to we'll see we'll see what happens here it's going to give us an error oops what I meant is we're going to do data frame five bracket uh name that's a column right we're going to do that and it's basically going to say that that's not going to work and what we need to do is what I or at least what I want to do and what we're going to do in this video is I'm going to create numbers I really would just want it to be numbered one two three four five that's what I want um but we don't have that right now I can't just will it into existence so now what we're going to do is kind of create an index basically out of thin air so we're going to do pd.index and we're going to say uh you know we basically want how many um rows are in here that's what we want our our index to be we want it to count how many are in here now you can make this dynamic and I it probably wouldn't be that hard but I'm going to take the super lazy routes um and I'm just going to say let's do df.5 or oops df5.count and there's 90 values in here so I'm going to do is I'm going to do a range of 90 uh and this is not uh I would definitely make this dynamic but I'm again I'm just being uh being a little bit lazy I'm going to call this index is equal to and I'm going to put this index right here so now this is a number so now it's going to literally index this for us now I ran into this issue many times um so what I need to actually do is to reset this index and then do it properly the first time uh so let's do re let's get rid of this let's reset this index um and it actually fixed itself um so what was happening was is we were indexing something it was already indexed and were causing issues in a nutshell so we reset the index and now this is what it looks like and this is exactly what we want this is really how we wanted it formatted in order to for our visualizations we have multiple rows for the bitcoin each of these columns or is now a row with the value attached to it exactly what we wanted so um really quick I for whatever reason it makes that uh level one I don't know why but we're just going to rename that column really quickly so we're going to do data frame six dot rename and then we're going to do an open parentheses let's say columns equal to we're going to do one of these bad boys oops one of these bad boys this type of bracket and we're going to say level underscore one and do a colon and then oops and then a colon and then we want to change it to and I'm just going to call this the percent underscore change so let's call this data frame seven and you don't have to do that I'm just doing it so now this looks much much better now let's try to visualize this one because we haven't done any visualizations yet we've just been messing with the data a little bit I you know I kind of want to see how we can use this it's something that I personally am interested in so I kind of wanted to see visualize how these changed over these these time periods but we need to import some stuff in order to be able to visualize this so we're going to import seaborn as sns and if we need to we're going to import matplotlib as well I don't know if we'll use it right now or at all but we're going to we're going to add it in here either way so now those are added and so what we're going to do is come right here we're going to do sns.catplot and we're going to oops we're going to say the x axis is equal to and we want to do this as the percent change percent underscore change and then we have the y axis now we want the y axis to be these values right here we'll say comma y is equal to and let's say values oops and then we're gonna say comma and we'll say we want to basically create a legend I guess you can call it we're gonna say hue is equal to name I'll show you what it looks like without it and then you know you can see that we need that we're gonna say the data is equal to this data frame seven data frame seven and then we are going to say the kind is equal to and let's run this and see what we get and super quickly with just you know limited inputs here's what we have now this looks really good we can narrow this down if we wanted to a few less because there's a lot here and there's a lot of colors but again that's just because we have a lot of different stuff but there's a few they're doing really well I think this is drawn and then we have a few they're not doing so well but it's really hard to see if you look down here it's really hard to see this and that's just because of the the column names and so I actually want to change these column names or these values so that when we visualize it right down here it it doesn't look like that I kind of want this to be at least one good visualization you can take out of here this is definitely not perfect or complete any means but you know you can take take that away from here um so let's um I did alt enter which adds another row I could have just pushed plus I that's kind of a lazy way um what I'm going to do is I'm going to change these these values in here so I'm going to do that I'm going to do data frame seven and we only want to look at this one column so we'll do that right there and we want to say dot replace and we're in an open uh parentheses and then a bracket now what we need to do is I'm just to show you um one of them so I'm gonna say this one hour do that oops and then what I need to do is a comma another bracket and this is what it's going to change to I'm just going to say one hour one hour um and we'll do this one really quick and then I'm going to I don't want you to have to watch me type all this out but I'm going to go through and basically do all of this uh for those but let's let's see this really quick and so now as you can see that um the originally it said quote dot usd dot percent change one hour is now only one hour now this didn't actually do anything we need to apply it to this right here so I'm going to say data frame seven is equal to and then we'll run data frame seven again so now that has actually changed that value now I'm going to go through and I'm going to update that for every single one all right so I basically just put the other ones um in here that we wanted to change with commas underneath so I have 24 hours comma with the seven days 30 days 60 days 90 days and then this bracket over here which tells uh it what's a change to 24 seven days 30 days 90 days so let's run this I haven't even tried it yet uh and it looks like it obviously worked properly so now let's go back down here and let's run this again and look at that it looks so much cleaner so much nicer um and as I mean all of them with that one hour change has a very little change and then you can look back so we can see back within 90 days it's gone a lot of these have gone down which again if you're following crypto you know there's a big crash recently especially with you know all these altcoins um that you're seeing right here went down a ton so I think this is um avalanche or die or whatever these ones are you know went down dramatically whereas there's one up here this lone wolf um that's just that's just doing really well for whatever reason so it's really interesting um to see now this is a pretty specific um visualization that I personally wanted to see and I thought was interesting you can do absolutely whatever you want to do with this data I mean there's so much here you can do a lot I mean a lot with this data especially depending on how long you track it right I only did this over the course of like five minutes but if you set this up um and you can track it over a longer time now let's say you wanted to do something much simpler you just want to look at like bitcoin over that time that you you know took the data in that's going to be a lot simpler than what we just did and I'll show you how to do that really quickly so we're going to look at the data frame and we are going to say uh or we're going to take specific columns we just want um a few columns that we want to keep or or pull from so we're going to take uh oops we're going to take the name column we're going to do um might be easier if I copy them but I'm just going to write them out quote dot usd dot price is the price of the actual cryptocurrency then we're going to do timestamp and let's make this data frame and we're just going to do 10 for absolutely no reason maybe I should have made it nine but it'd be easier so now we just have these um these columns and you know we have all these separate columns so what we can do and uh the kind of the reason I want to show you this is you can just query this really quickly and just take the columns that you want so let's say we just wanted to look at bitcoin so we're gonna say data frame 10 dot query do open parentheses and we're gonna say name is equal and equals not like that uh when you're doing it like this you need to say equal equal equal to ignore that uh is equal to bitcoin and we're gonna do it just like that and we're gonna say data frame 10 is equal to let's start running that I think something's wrong with that try it like this all right let's try that there we go it was just the I needed a double quotation instead of a single quotation that was the issue so now we have bitcoin we have the price and we have these timestamps so this is the actual time when we ran it so this is the original data frame and then in the you know this this project it took me 52 more minutes to get this one and then we had it running properly for the next five minutes so that's you know that's actually what we have now if we want to just visualize this really simply what we can do is we're gonna say uh we're gonna do sns dot line plots that's gonna be like a little line chart or line graph whatever whatever you want to call it then we're gonna say x is equal to we'll say quote no actually we wanted the timestamp to be on the x-axis um and then we'll do y is equal to quote dot usd dot price and let's see if that works do not interpret time stamp for the parameter uh that's because it's not understanding that the data equals data frame 10 now let's try this all right so this is uh looks terrible let me just say sns dot set underscore theme open parentheses we'll do style is equal to dark this looks a little better now again we are looking just at a very very short time series but we can look at just bitcoin or look at multiple and we're showing this you know this line that's showing us this trajectory over time so you can get really creative with this you can run this for a long time you can show bitcoin over days weeks or months however long you run this and so that's really all i've got um honestly like i said this is not a i wouldn't say this is a complete full project but i'm showing you how to do something to enable you to kind of run with it and run with the ball and do basically whatever you want with this you can pull it from uh you know data from a different api you can use this exact api and data but i just wanted to show you just a few things that i initially saw that i might do with the data and you have so much let me go back to this original data frame uh right this one right here this one right here look at all this data i mean you have so so so much data actually let's go to this one this is better you have so much data so many numbers here um so many columns that we didn't even look at that you can use um and so you know there's a lot that you can use here and i'm really trying to just set you up so that you can run with it and do whatever you want i could have done a thousand different things here but you know i tried to just show you two things that you can do with the data that i thought were pretty interesting or simple to do and you know i want you guys to go out and do something way way better than what i did so i hope that this was helpful i hope that this showed you how to automate that process so you don't have to sit there and click it and append it and do all these different things that it can show you how to kind of automate this process and hopefully that will be helpful in your future projects so with that being said thank you so much for watching if you made it all day to the end you guys are fantastic if you like this video be sure to like and subscribe below i'll see you in the next video


Welcome everybody and thanks for being here. And so we're going to do a deep dive into preprints and open access. I'm Brian Cook, I'm at the University of Virginia, and I am the co chair of the moderation committee for at archive and I've been involved in with at archive which is a preprint server for education so I'll talk a little bit about that and we'll talk about other issues related to preprints and and open access. And I'll just, before I forget, throw out a quick, even though we can't see your faces, I think you can make comments or raise hands and things that we'd encourage you to chime in. If you have questions, comments, you know, would like us to expand on on something that we want to make this as relevant as possible for you so so feel free at any time to bump in and give us some input on. I want to try the chat and sort of interject but also say hi I'm Stacy. I'm a assistant faculty member and learning sciences and psychology at Worcester Polytechnic Institute. I'm also part of the ed archive steering committee and think my role is like liaison at the center of open science so I'm relatively new to the land of preprints, but I am a huge fan of open access and so I'm really excited to be here today with Brian and we're going to go over a couple of different things and sort of trade off talking, but please let us know if you have questions want us to talk about something in more detail or more depth. And if I think if we don't have the answer to your question we can always try and follow up with you later. But let me just, I can go ahead and start the side deck so today's agenda. I'm going to start us off talking about open access, but that is the rationale, publishing open access and some of these open access FAQs. I'm going to talk about preprints talking about the rationale benefits limitations and then like Brian said we'll, we'll sort of walk you through at archive. So just to sort of jump us off open access open access the rationale for this is several lines of thinking really. The idea here is that you want to make your work open and available to the public. You work hardest researchers, or writers or whatever it is right you want to make your work known and disseminated as broadly as possible, open access is a way to help do that. So by adopting open access promote scientific literacy I was just talking to someone in another session who said that they were in Ethiopia and no one has access to scientific literature over there because there's no library access. The publishing model usually makes it really hard to buy articles or buy access and so by sort of adopting open access practices you can make your, your work more widely available promoting scientific literacy. So people think this is the ethical way to disseminate work. And what I really mean by that is that researchers don't get paid to publish and reviewers don't get paid to review. Also, taxpayers largely front the bill for the research we do from our governments, or other sort of organizations and so it's kind of unfair to the taxpayer that their tax dollars go to funding us our work. They sort of, you know, are in the process of creating not knowledge but then they are charged again to access that knowledge so a lot of times when we think about open access or really thinking it's really ethical dissemination right we don't want to make people pay twice. Brian, can you explain what these various levels me. I think this came out of PVIVAR and I'm guessing I'm saying that right but I may not be. Did a, and colleagues did a very large review of open access publishing across different disciplines in science in 2018. Sometimes there's an additional category but these are kind of the traditional categories of open access. So someone would posit a kind of top category that sometimes called diamond or platinum open access and that way abbreviation is open access. So I'll start with gold and then go back up. So, gold is when a an entire journal is open access. So within the field of education for example, there is open AER a, and everything on that journal is is freely accessible by everyone if you can get on the internet. So there's, there's really no option to it if you're going to publish there it's going to be open access which sounds wonderful. Great I'll publish open access then. Well, open access is still. And so instead of having people pay to access the, the publications because they're behind a pay wall and people need to have a subscription or pay individually for them. They charge the authors in a PC or an article processing charge. And those range very broadly. I did a study looking at in my research, I did a study looking at the authors in APC or an article processing charge. And those range very broadly. I did a study looking at in my field of special education. We looked at journal policies and the APCs varied. They tended to be around $3,000 per article. So they are not trivial in in the news recently. Nature, which is owned by Springer has now they're developing some, some gold option journal options, and they are charging 9500 euros or, I think a little over $11,000 per article, which is raised a whole other issue or demand. 100 euros or, I think a little over $11,000 per article, which is raised a whole other issue or dimension of open access publishing so that ethical issue that Stacy talked about that we want to make our, our scholarship and our research as accessible and open as possible in terms of dissemination. But does this then limit who can publish in certain outlets and these gold outlets by who can afford to pay APCs and we'll come back to that a little bit later. Above it isn't on here but I want to mention the diamond or platinum open access is when journals do not charge authors so things are the publications are open access, but there's no APC for them. There's no article processing charge. Those tend to be small scale journals that are run maybe by a professional organization, and they just have volunteers running a website and they're more just kind of posting PDFs and and probably have a just a volunteer set of reviewers and editorial board members. And so those do exist they tend not to be very large high profile journals, but they certainly do exist. We'll talk about green, which is essentially anything that self archived so pre prints are an example of this, and there's different options that as authors, we can post versions of our own work on different sites and different locations on the web. Bronze is is actually, according to Pivovar's work in 2018 was the most frequently used type of open access. And so bronze is the work isn't licensed as open access. It is something that the journal or the publisher decides to make open access on their own volition. And this could be for kind hearted reasons because they want to make this a good research and scholarship available to the world. But sometimes it is probably because there's a special issue that they want to drive traffic to their journal website, or for whatever reason, but they decide to make things certain things open access on their on their website, and that may stay open access for a long time. It could change tomorrow and be back behind a paywall. So bronze open access gives us open access, but it isn't licensed which isn't then it's not clear about how you can use that in in different ways, whether you can freely use that. And it may cease to be open access very soon. Hybrid is becoming more and more popular, whereas a gold open access journal is the entire journal is is open access hybrid is where authors have the option and I see this more and more frequently. When something gets accepted for publications, they ask would you like to make this open access, and I would but then it also says well then you can do that you can make your article open access. Other things in the journal probably won't be in that issue or the journal generally, but you have to pay that APC then that article processing charge and again that tends to be in our field about $3,000. But it varies by journal and by publisher, but this is becoming an option more and more and some of the higher impact high prestige journals and it can be more expensive, and then closed is the journal is probably predominant, roughly probably 50% in in pivovar survey across disciplines about 50% of published articles were just not available, unless you were affiliated with an institution that that's paid for a contract, or that you choose to pay access to an article, and they're getting quite clear the publishers are getting quite creative about how to do that I see more and more like for 24 hours of access only $25 or something. Which is, I think is as Stacy mentioned is just a real shame that we have large oftentimes publicly funded research that we can't access without paying for. Oh, and so here is I've referred to it a couple times. This is a figure from the pivovar review. And so this shows it is, it's getting better there's more and more of these different options being done and this is a few years old now I would suspect that these numbers are continuing in the same direction, but even in the more recent years we see the majority of research is still closed with at any given time, there's a fair amount of material that is bronze access, some hybrid more and more gold, and I would imagine green is increasing in the most recent years more and more to So a lot of people have questions about open access because we want to make our work available, but like Brian just sort of laid out like it's complicated right like a lot of times to get that you know your work out if you go through a specific journal, you are going to have to pay an article processing charge 3000 to $11,000. The lowest I've ever seen I think is 700. So a lot of people try and find workarounds or ways that they can sort of identify the journals that are the most encouraging of open access and so one of the most frequently asked questions is how do you find these open access journals and articles. We all want the diamond one right like we all want the one that's not going to charge us it's not going to charge the reader it's just going to make it available but sometimes that's not always an option so we want to see what our options are. So, one really helpful resource here is the directory of open access journals, many of you probably heard about this. But what this is is it's a directory of open access journals you go in you can search by subject, or title. There's a lot of different filters you can put on and off like, you know does the journal require an article processing charge does it not. And this is a really great way to sort of find where you can publish your work in a way that makes it the most open access as possible. Another question we usually get is. Oh wait, Brian did you add this one. I did I snuck in a slide on you. This is a kind of the counterpart to looking for open access journals. This is looking for specific articles that may be open access. And so this is something developed, and they, they just search a whole bunch of different repositories for you. And so if you have an article name or a DOI, you just put it in and they search it for you. If they can't find it anywhere they actually have something that they'll set up you can email or you can contact the author. I'm assuming through the authors contact information from the article to to request the article from them, which is always an option which is a very good option. If you have on an individual basis, you can send your article formatted your PDF of the formatted article to people individually. Oftentimes though, you've signed the copyright with a journal you can't do that and just posted anywhere on the internet, but you can send it individually to folks so that's always an option, but I wasn't aware of this open access button until recently. I believe it is not 100% exhaustive. So there are things that didn't come back on the open access button that I've done searches, just through Google, and if I if I look around enough I sometimes I can find a version of it around to. And then is Sherry points out there's also the unpaywall option that that's out there. Yeah, so just to clarify Brian is this open access button to plug in as well to believe it is yeah okay. I can't wait to play with it. Yeah, but like as Brian mentioned, one thing that's really helpful. And I think if you're an instructor or deal with any students to recommend is that they can't access an article, don't ever pay. Try and email the author or look at the author's website a lot of times I found that works really well. Oh, thanks for sharing. Okay. Okay, so other questions people asked is their funding for open access author fees right these APCs can be a lot of money what sort of funding is around for them. Usually, pretty limited. So a lot of times you can write them into grants if you're trying to get funding for your research you can say I need X amount of money for article processing charges to make this open and available. Sometimes universities have funds specifically for this, but not all universities. Some journals have reduced APCs for students and authors based in countries classified as low income economies. There's a couple other sort of nuances Brian I don't know if you want to add anything to this. I don't, I don't know off the top of my head. But with the, a lot of the APCs have kind of a graduated that they'll have adjusted rates, depending on kind of career status and country that that you're that the university is based in so sometimes the APCs aren't as as expensive. But for journals that that require APCs to to make them open that is generally an unfortunate requirement there. There he is I think a growing back to the the APCs and as they seem to be growing higher and higher. It is almost just a transferring of the cost and we don't have time to get into it and I'll get all upset about it too but when you look at the, you look at the profit margins of the publishers that it's just remarkable. So, the people to charge the people that are reviewing the articles that are writing the articles that are conducting the articles, the to then charge them to publish it. It's a much bigger picture that that we're not going to go into here, but there, there is, I think a movement to kind of just blow up the system, and, and think about doing publishing in a very different way that is separate from the kind of traditional journal structure that that's been around for for quite a while now but may not be necessary. I wish we had more time to talk about that because there have been some new interesting models like collaboration psychology charging at cost but leaving APCs for like graduate researchers or early career paying reviewers like there's a lot of new models sort of happening around but I think for by and large, most of the time when you're talking about open access someone's going to try and charge you an APC. Okay, let's just pretend. You can't afford the APC, but this is the best journal for your work. Go ahead and publish in them but how can you still, even if you can afford to make it open access for everyone how can you still sort of make your work more open access for people right so if you can't do it through the journal through the cost there are other work around so let's talk about some of those. I think it's really helpful is research gate or academia.edu, whatever you you fancy in terms of research sharing. A lot of times you can post articles under your profile as a researcher and store the text privately. So, like one journal I had said you can send 50 free copies or whatever to people or if it's for teaching or instruction purposes like it's unlimited, and I would just hold a sort of a private text and then every time someone would hit that request button boom I'd approve it and it would shoot them out a copy of the article so that's, that's one way to sort of help make your work more open. Another is you can check Sherpa Romeo, which is a wonderful website that will basically allow you to look up the journal you're publishing in and see what you can post. And we're going to talk about pre prints in just a second but this is a great website to see sort of what can I get away with legally in terms of posting the work that I've done on this article. So if you go to Sherpa Romeo. This is the front page you can type in the journal title, or the ISSN. And then it will show you what the policy of the journal is. So here you have the published version so for this, this particular journal, you can post to any website the journal website etc. So this is the publishers PDF they formatted it it's very beautiful that's that's the published version. You also have the accepted version. So this is probably still that Microsoft or latex document but it's gone through peer review, but it's the accepted right all the material all the words are the same. And this accepted version, according to this website could be posted on archive bio archive social archive and a couple others. And there's this one I can't remember what the difference is accepted version pathway be I think oh this one for institutional repository or specific ones. You need you have an embargo of like 18 months. So there's just a lot of information here on what you can post and sort of where, but the submitted version which is what we typically think of as the preprint before peer review. So in embargo, you can pretty much posted to any of the preprint server so this website is a really great way to say like hey this journals policies are you can post this version of your article and these different places. Stacy can just tag on a second there. I'll highlight that embargo and it's here with a little hourglass and talking about time that we'll talk a little bit more about this but as you start to wade into some of the nitty gritty kind of legalistic elements of what can I do and what can I not do. It isn't. It is it is oftentimes a chore to figure that out for any particular journal. I wish there was greater clarity you could just go on the journal website and oh here are policies about open access but usually, you have to really search through their website and then oftentimes that takes you to. There's a link that goes to the publisher website and then you have to go through three more links there to go. I encourage people that they're not sure they can't figure it out. Email the journal editor or the the the the assistant for the journal. Oftentimes they're not entirely sure of all of these policies to but anyway one of the issues is that for certain things, you can do them certain things you can't and certain things you can do after embargo period. So, for example, you could post the accepted version. And so that's not the the fancy journal formatted piece just like Stacy said, this is probably a PDF of your word document, but it has elements of peer review that have been incorporated into it. The journal does have some say over what you can do with that because it's gone through their peer review, you've incorporated elements of that into the manuscript, but very oftentimes you are allowed to post that. And so here you can post that on a preprint archive at any point, and you can post it on different institutional repositories for example, but only after 18 months after publication, which in this is, I'm not sure if there is a rhyme or reason to it and the policies really vary across different publishers and journals so it's hard to just say, and this is the generally what happens. The submitted version where this is what you submitted it was it's a probably a PDF of your word document. It hasn't been submitted yet that you can pretty much do whatever you want with. And if it wasn't you, you haven't like dependent on peer review or anything like that. Right. Awesome. Oh, good. Oh, I was going to say, but there are there is a rare exception where some journals will not accept submissions that have been preprinted. It is fairly rare. I think it is very seldom policed, but there are some journals. I'm not sure of the rationale, but there are some journals that is at least their policy, not to accept submissions for preprinted articles so in that case if you posted yours, the submitted version. It's your right to do that. Legally, it may have you want to make sure that the journal or journals that you may be targeting don't have a policy that then prevents you from submitting there. Right. Yeah, that's a good point and we'll talk more about this. One thing I wanted to add though, talk to your librarian if you're not sure I learned way too late as a graduate student at UCLA that the entire UC system. Because I was like affiliated with them had all of these rights that I had no idea so like I could literally sign my first born away by giving my publisher or my PDF to a journal, and they're like we own this we own you whatever, because I was affiliated with the UC system, I actually had rights to upload that into their own repository so there's like a lot of layers here that are difficult to navigate. This is definitely one resource I think that that can be helpful but also your local librarians they might know things that are specific to you and your institution that are really helpful. It's a good point you see it the UCs University of California has have a very interesting history and recent history with negotiating with publishers and do have some unique arrangements with publishers. So let's talk about preprints. So this is sort of like where we're going to go from here so preprints we're talking about what they are the different terms that get thrown around why we need them the benefits, the limitations sort of how they're growing. If they're reliable or not, sort of how they relate to repositories. And then Brian is going to take us to the ad archive posting them consuming and then we'll do like questions and answers. Okay, so some terminology so we talked about this little bit of preprint is the most common term I think, but it's using a different and potentially confusing way so sometimes people call these white paper sometimes they're referred to as preprints. I think the open science community is pretty much adopted the term as preprints but there's a couple different ways of certain terms that get thrown around. So the preprint is the print that's posted to a repository before edits are made reflecting peer review from a journal. And to refer generally to both pre and post so if you have a preprint it's like what you wrote with your team or yourself or whatever. And if it gets posted it's it's considered like a preprint, it hasn't been printed by a publisher it hasn't even gone through peer review. The post print is the print that's posted to a repository after edits are made reflecting peer review from a journal. So basically, what this is is like, it's gone through peer review, but it's not the publisher's PDF version right. And then lastly this print is sometimes used to refer to generally just both pre prints and post prints. I think it's a little bit confusing but the idea here is that the preprint is before peer review and the post print is usually after peer review. Brian did I get that right. I think that's right the only thing to try to make it even more confusing is that I think oftentimes this is the kind of the sub bullet there mentioned preprint is often used, kind of generically and probably technically incorrectly people will refer to this kind of universe of print postings as pre prints and technically the pre print probably should mean just if it hasn't been submitted yet but oftentimes I think people just think about all pre print so when you hear pre print. It's just it's probably not clear what what exactly is being referred to unless you ask. Want to walk us through this. Yeah, sure. So lots of rationales for doing this that overlap significantly with the rationales for making work open access that Stacy talked about earlier and so a lot of it is just, we want to make our stuff freely accessible for everybody to democratize access to research and the knowledge base. In terms of for the researcher, there is research to show that making your work open access and in particular green access increases impact and I think that only stands to reason. So we're making it available to a whole new and broader range of researcher or other researchers and research consumers. So it can really extend the impact, especially I think the non researchers were probably less likely to have those institution institutional affiliations. That that's reflected that increased impact is is reflected in documented higher citation rates for for pre prints. Certainly combating publication bias where I mean I guess it isn't the file drawer analogy doesn't work quite as well in our digital age, but the idea of that that we just keep a lot of these studies with no effects don't get published they're just kind of on hard drives are in their file drawers. This allows us and there's actually been initiatives and other fields to try to empty the file drawer and get all of these unpublished all that unpublished great literature out there and available for people to search. So when you post to to a print archive, for example, it gets a DOI and then it turns up in Google scholar searches and things like that. So we can really make a big fruitful push for trying to flesh out the research base for things that that might not be published because the findings were insignificant or small or not interesting enough to make it into a journal. And so that's the next bullet point there. Making research available more quickly this is really made headlines and has become a bigger and bigger issue with the pandemic that in the hard sciences, getting information about COVID and the COVID studies out tomorrow is saves lives as opposed to waiting for months of peer review. And in the worst case. Well, I don't know if I should say worst case but in the extreme case, you know, peer review articles sometimes go through multiple rounds of peer review at multiple journals. So within a single journal I've had things that are in peer review for over a year. And so there are stories certainly that I've heard of things being in going through peer review for multiple years and so if this is a timely getting that out peer review obviously has a lot of strengths, but it takes a lot of time, and we'll talk a little bit more about that that obviously brings up huge issues that these prints have not necessarily been peer reviewed, but it gets it out there much more quickly. It can be a mechanism for feedback that people can comment on work before you submit it for peer review for publication. And so you can put it out there, invite people to look at it, or just have it's out there for people to comment on and get feedback for people to look at it. It's also a nice thing if you have say you're on the job market, or something like that and instead of saying oh yeah look at my CV I list a whole bunch of in in papers or yeah. And so here you can actually have here's a here's a DOI where you can look at some of my work that that is currently under review and have that available for both. Yeah. And just to jump in I remember like one tweet Brian knows like had was like this article just got published and it already has 17 citations because we published this as a preprint and so people were already like using the knowledge in their work and referencing that in their own work so by the time it was published it was already being used, which I thought was really cool. Right, which in kind of my view of how the publishing can be a nice kind of megaphone for work, but it. I like the idea of having a separate system for disseminating it making available making it available and people can use it, regardless of whether a publisher picks it up or not. And also, there aren't page limits, which most journals have associated with preprint archives so you want to add extra material, expand on your method. You can put it out there, have that extended on a bridged version on a on an archive or in a preprint, which I think is just fantastic. And I don't know how there's a bullet around the file drawer I put at the end because I already talked about that related to publication. Yeah, and I think that it ties into the slide someone wrote, when you post an empirical report on a preprint journal with no intention of submitting it to a journal for peer review. What's that called not really a preprint has determined developed yet to signal that a paper is a terminal preprint. I love this because, again, that idea of the file drawer like we spend time energy. If you're human subjects other people's time working and trying to collect data and for it to just sit there and never see the light of day is sort of like a huge disservice to all that time energy and resources and so I've had people who are like, this study like fell apart it doesn't have the findings to get published what I do and I'd be like, write it up as a preprint, write it up as a preprint post it there, you'll have a do I people can find it. Right, it's allowing people to find the work who are interested in the topic. And you know, it doesn't have the same impact on the CV as a published article but at least you're disseminating the work in a way that is open access. Yeah. Yeah, oh yeah great point share you can put those terminal preprints in your local institutions repository. Yeah, exactly. Another thing too is, we're seeing some shifts in the field so the journal elife recently changed their model to a preprint first model so basically this journal found after looking at who was applying to the journal and what was getting published, like 90 something percent of the people had been posting preprints ahead of time and so they thought well this is where the fields going we're just going to require this so with this. This journal, they actually want you to and require you to post your work ahead of time perhaps get feedback, have it open and then you could take that preprint and apply to the journals and I've seen a lot of energy behind and I'm just wishing that all journals would just accept preprints as submission so you don't have to like format it and do all these other things for you know specific to the journal but you could really just like post your preprint and then use that as submission. So I think there's a little bit of momentum here, which I really encourage, but it has yet to be widely adopted to my knowledge. Education I don't, I don't think it's been adopted in particular but I do that that idea of kind of blowing up the process a little bit this is maybe one way to do that and so publishing really then becomes just making it public and we make all of our work public and we have these print repositories where everybody's work is out there for everybody to comment on. But that then journals, then kind of can select and curate different works from that list that are available and in some ways maybe give them an additional megaphone but it obviates the journal as the the nexus of the dissemination of scholarship which I think is a real positive development potential development. Yeah, I think a lot of researchers would love the ability to just make their work with the click of a button available to everyone right instead of waiting years for peer review, and to get solicited by journals. That would be amazing. So I hope that is a direction we're moving towards. This is from people of ours review and I referred to it a couple times and this is so this is citation. There are other works that show other a an open access advantage in terms of like the, what's what's the social media impact on blanking in the name of. How often old metrics Thank you very much Wendy. Hi Wendy.  I was that in a era open, but a year or two ago a era just had a mishap with things, and they made a couple entire journals just open for a short period of time bronze open, and they showed a real bump in terms of their downloads, which kind of makes their downloads for free and so it just kind of shows you make these open people are going to come get them and use them more hybrid is really up there. Green is really up there in terms of making these things. I would suspect some of this is selection bias people probably are making a choosing to make green versions of their work that is probably going to have some greater access maybe or greater impact, and probably be cited more they may be more excited to do that with that work. So that might be behind some of why people are choosing to, yeah I'm going to pay for that piece to be open access. It's got it might be cited more anyway, but it was, I always found this curious that there's a lower rate for the gold. And I think some of this goes back to the point I'd mentioned earlier about a lot of the gold journals are actually kind of small journals that are associated with small organizations that are just run by the volunteers and they're just going to kind of be low, lower sided, because of the nature of the journal. I guess I should pay attention. We have a great question but I was hoping to hold it off till the end because it's a spark a larger debate which is an important one. Perfect. I wanted to talk. So, some of the disadvantages some of the challenges. It's not peer reviewed. And so I'm involved with at archive, and we just make sure there aren't the, that the content is moderately appropriate, you know, we've run across a couple where that's not the case. But we're not going in and was that adequately powered and they didn't list that as a limitation and I think they really should have. There's none of that. That's not the job. On one hand, I think there's a caveat emptor and that's maybe that's okay as long as you realize that, but I'll talk about some, maybe some other approaches to dealing with it. But that's a huge issue and limitation to this that that the prints that pre prints aren't peer reviewed yet. Some as I mentioned as we mentioned some journals policies don't allow for submission of pre printed manuscripts. I think that is not too many, but it does happen and it's it's it's important to be aware. Make sure when you're thinking about posting a pre print that you're not going to then target a journal that won't accept your submission. There's no journal policies there. And even though the journal policies on the websites themselves aren't always clear. One other thing that that I personally, I'm doing a lot of work actually right now in the in the area of open science and in my little world of special education. There is a limited pool of people that are going to review works on open science and at least some of those people are connected with on social media. So if if I post something on on at archive and say hey I just we just finished this go read it I put it on Twitter and Facebook and great people go read that. Then I'm worried though that when they get invited to review the piece they they have to say no I've already read that I know who the author is. So I actually oftentimes wait till the first round or after peer review is done and wait till something's accepted, which is a shame I think for people in broader who are who are doing broader work and there's going to be no potential peer reviewers, then then maybe that's less of a worry. I'd like to see us maybe. Well I don't know. There are different thoughts on whether peer review absolutely needs to be blinded in terms of knowing the author, but that's another issue to think about. Yeah, Matthews I tend to be the same I'm I think I'm okay with with open peer review but I know there are pros and cons and different thoughts on that. And so part of this this is a the I think I'm being aware of the limitations are is an increasingly important issue as as we start to look at just the. Just the number of green open access and pre prints is starting to really explode in different fields. Biology, for example, physics has been that way for a long time. And I think we're growing in education. And so hope I don't think our, we did one specific to education it wouldn't quite look like this but I think we're moving in the right direction. All right, so this gets to be kind of the reliability of the pre prints then and this starts to speak to that issue whether not peer reviewed. Actually, a lot of the stuff that's out there as a pre print is peer reviewed. And so it is something that was a pre print. Either the journal allows you to update those those prints, maybe after an embargo period, maybe right away. You can also update pre prints as I'll talk about in just a minute. And so, when it is accepted when it is published, you can put the updated do I in in the, the, the title page of the print. You can put a link to the published article. And so, and you can actually note areas of potential change, but you can certainly clearly indicate whether this has been published or not so people can compare the published and unpublished versions. And so, this actually they were looking specifically at co vid in the Gahanau at all study, and they found that 68% of the pre prints that have been cited around co vid were subsequently published so they're actually out there and and published, and have gone through peer review. This is just an early version of that piece so most of the pieces actually do get published and stand up to peer review, but some some don't and that's important to realize about 4043% of the pre prints didn't didn't mention the publication status on the pre print. And that you go ahead and put this is this is under peer review. It just takes a few seconds but it does take some time then to update it when it has been accepted when there is an elect a published version of it out there, you can put that out there. There's more and more reviews and studies starting to be starting to be done, where they compare the published version to the pre print. And so, in this study 56% had the exact same data in the exact same conclusions, but 15% of the pre prints that had been published when they looked at the published version, it had different conclusions. And that's all just just this year already in 2022. Again, looking and co the co vid related pre prints and publications are kind of the hot area. They did find that that between 717% of the pieces had some changes between the pre print and the published article, but the majority of those articles that did have changes. They didn't really affect the main conclusions, they were, they were smaller changes. So in some ways, and different people have interpreted this as being very supportive of the reliability the pre print look most of the stuff ends up getting published. Most of the published stuff isn't meaningfully different, but it also highlights that there is some stuff that doesn't get some stuff I shouldn't say stuff, some pre prints that don't get published and even among those that have some of the pre prints, or some of the publications, there's a real change from the pre print. It's a it's a relatively small proportion, but something certainly to bear in mind, and, and recommendations to watermark pre prints to put something in the margins that that clearly identify the pre print and whether it's undergone peer review or not. Great advice. I just want to be mindful of time in about 12 minutes left. So maybe Brian you and I can like speed get to the rest of the slides. So fast from here on out. Now the other stuff actually is the I think, okay, yeah, quick. So there's lots of archives out there some of them, a general or multidisciplinary that education folks can post to. Then there are some that are domain specific sci archive used to be a big one for educational researchers to post to. But we've been going for two years now we have at archive which is an archive specific to educational research. And a lot of them are hosted on OSF so a little bit about at archive up to 931 pre prints posted probably more because that was a day or two ago. So to post on it, it is actually I'm not going to go through these for the interest of time and to because you don't need to.  I can do this without a tutorial, and I'm not good at this type of thing so it is, it is pretty darn self explanatory from a consumer perspective you can also search the archives that are on there. They have OSF has built in hypothesis which allows you to comment on the on the on the print so there is an element of ongoing peer review, and you can download any of the posted manuscripts as well. Most of this, there is. Yeah, so this is just one that I was going to demonstrate this little I thing is the hypothesis if you wanted to comment on it, you can also endorse the work and so this was one of my advisees had made this kind of cool flow chart about how to post a pre print and was a supportive I applauded the work there. It also comes with some metadata what's interesting this is just the bottom half I couldn't fit it all in one screen, but you can download it. It also shows that there were two versions and the you can download the previous version, it shows you when it was submitted and last edited, there are tags that you can provide as an author and then it actually gives you citations in the different writing styles. We talked about the flow chart. And that's already. Yeah, check it out if you're interested. So basically this is what we'll wrap it up this is what we talked about today I think I have one more thing for you which is pretty pre prints which I really appreciate. These are made by Brenton Wernick. They are on the open science framework you could literally Google pretty pre prints, and their templates that you can download to make your Microsoft Word document a little bit more pretty, a little bit more fancy if you wanted to up your pre print game. And actually I fooled myself into thinking that the pre prints I have on my website where actually the publishers PDF and accidentally sent them before. So it's just, yeah it's a really fun, great way to kind of make your pre prints a little bit more polished a little bit more fun to distribute to others, and I highly highly recommend it. And we, I think that's it so thank you Brian for helping us walk through this we had some really good questions in the chat so I just wanted to sort of start with a question posed by a Vaughn, which was, what are your thoughts about the need to educate the general audience about what pre prints are, for instance it's not always easy for media to understand the differences between pre prints and reviewed articles. I might start by saying, I don't know if Brian's frozen I might start by saying, there's a huge problem when it comes to dissemination of scientific research by media, just in general, even for things that are peer reviewed. But when it comes to pre prints. Yeah, it's a little bit more tricky because people might treat them with the same like level of rigor as peer reviewed whether that's true or not I'm not sure. But it is a little bit challenging, just in general to work with media who really just wants the bottom line the height the headlines, and not really caring about the new ones. So Brian I don't know what your thoughts are. Yeah, I don't know if I know that there is some work being done and I'm not going to do a good job. I'm describing it because I'll forget but how the media is handling this, and it is something that especially has come up with coven and the media oftentimes kind of trips over itself to either not mention anything about peer review, or to mention it as kind of just counting it, and, and, you know, something isn't of high quality, because it is or is not peer reviewed it is just a step to try to promote and serve as a gatekeeper for quality but there's plenty of pieces that go through peer review that are then retracted and are of low quality there's plenty of stuff that isn't published that that is of high quality so it's certainly no guarantee one way or the other I think just being as transparent as possible and about the publication status of it the peer review status of it, and being caveat emptor about all of that is is probably about as as good as we can do. And it's a trade off between making things, making our making work and research, publicly available quickly available, but then not having peer review, you know in the, in the big, perfect world, I think we would build peer review into a print server, a preprint archive, where we could have ongoing review and commentary there, but but we're not there at archive yet and most print servers aren't that that'd be quite an undertaking. There's also a point in the chat from Charlotte who posted a lot of great resources, which is have people heard of PCI registered reports, a community driven initiative that facilitates the peer reviews of registered reports via preprints. You're nodding like you I have not heard of this. It's absolutely wonderful and I think Chris chambers is spearheading it, and the idea here is is registered reports which is a whole other topic, but doing registered reports works largely like dissertations and you submit just a perspective and methods and then that gets accepted in principle, but doing that and so they're taking the peer review out of the journals hands and reviewing these kind of study proposals independently that then can be picked up by other journals, and it is I think really a kind of first move towards taking the review the peer review process and owning it amongst a community of scholars rather than having it being run through and for the kind of to some degree, supporting the profit of publishers, even though the academics are still doing all the work. That's great I just want to let everyone know these resources you're adding to the chat. I'm pulling into our slide deck that will be on the open science framework so the very last slide will include everything that people are sharing. So we won't have time to go over all of it, but I do want to make sure that we all can access it later. Question about at archive. And have we been in discussions about accepting pre prints in other languages besides English, and I think the answer is no because only Brian and Jesse are reading through all of the pre prints on a daily basis, and you're probably at capacity. I, I would have to check and sorry this isn't going to be a satisfactory answer. I don't know what our policies are about that. I know I've interpreted pieces using Google translator, and I kind of think we have accepted pieces that are not in English, but I could be wrong on that. I think it's important to do so. It's just we do have kind of a mediator role and we want to make sure content is appropriate even though we're not peer reviewing per se. So we have to be have some mechanism to figure out what it says. But I anyway I can't yeah I can't speak to it without having our policies in front of it because in front of me because I don't want to misspeak but I kind of think we do accept things other than English but again I could be very wrong and if so I apologize. It's a good idea. I think we should within our ability to do so. Alright, and Matthew is saying OSF pre prints accepts paper in Portuguese so maybe if you can't find the subject specific pre print, if your field that accepts a different language maybe there's a more general one like the open science framework or archive that you could still post. So I know when I went on the job market, I added all the pre prints I could to my website because I knew people were going to be looking at them and my old work was in transfer students and that wasn't what I was doing but that was predominantly what I had published. So I created my pretty pre prints, bark them clearly as pre prints and upload them to my website, which I think was really really helpful when people were trying to Google me. But also I sort of like had control of the pre print, even though it didn't come with the DIY man website. There's just another way to sort of use them to my advantage. The institutional repositories to or something that I haven't sufficiently explored but I know a lot of people use and feel feel very positively about that more and more. Almost all I don't know of all but most universities are going to have an institutional repository where I think you're encouraged to post the green versions essentially pre prints. And yeah, I don't, but I should and it's something I need to probably look into and make myself do. Yeah. Well, it's just turn three o'clock I want to thank you all for coming if you have any questions feel free to email Brian or I, or find us I guess in one of the, the hang out meeting rooms. But I'm sure all y'all want to go see other sessions so thank you so much for coming. And thanks for the resources you provided. Yes, that was fun. Yeah, thanks everyone. Thanks Stacy. Thank you, Brian. I think I've already told you for this.