Beth, I thank you for the introduction and I want to thank the organizers for inviting me to give this presentation. This was really an interesting time for me because when Cordo came out and I saw some announcements, I thought it had a lot of promise and I thought a good way to learn it would be to learn it under fire by writing a book while I was learning it. That's what I did and it also taught me various ways I could extend Cordo and I would say it turned out to be a very successful experiment and I've been very happy with it. And since then I've rewritten two other books that are much larger, more than 500 pages in Cordo, converting from either latex or markdown and that conversion process was very successful too. So what I'm going to be talking about in our workflow are these areas here, including a little bit about report formatting, a little bit about importing data, getting an overview of data, including filtration of observations and missing data patterns and learning about your data and sort of a meta sense. So can we do sort of an overall or a higher order analysis of our data? A little bit about data processing and this is where the talk actually gets the most controversial because R gives you so many different ways to process data. And what I'm suggesting in this electronic book and in this talk is really a way of living. It's a way of living as a data scientist, statistician, data analyst that is very, to me, very logical and very linear way to think. And you'll see that those of you who use the tidyverse will see a drastic difference between this approach and the tidyverse approach. So that's probably where a lot of the controversy is. I'll get a little bit into descriptive statistics and analysis. I won't really cover very much. That's so project specific, but I will talk about some principles and a key principle is staying close to the data. And one of the ways we translate that phrase is that we never dichotomize ordinal or continuous variables when we're analyzing data. So that's the data analysis sin that we want to avoid, which mainly means that tables are no longer viable as a tool in statistics unless all of your variables are already categorical, especially your baseline or independent variables. So there's controversy there. And then there's chapters each on caching, parallel computing and simulation. I won't really cover any of that. Those things are more standalone topics. So if you look at the preface to this e-book, there's a lot of motivation and a lot about what I'm trying to accomplish. And then there's resources for learning quarto and some other details. And then there is a chapter two, which is a very detailed case study. You see this that comes up here? This is Mermaid. So you can use this book as a template for a quarto because if you click on that code button, you can see the source and you see how easy it is to specify little relationships. And in quarto, these are rendered instantly because Mermaid is pre-installed with quarto. And so you immediately get these very simple flow diagrams. So this case study illustrates a lot of the things that are in the book and a lot of the philosophy of preparing data for analysis and analyzing data. And I would encourage you to go through this case study first before really looking at the individual chapters. But what I'm going to be doing is going through things in individual chapters. I'm not going to cover any of the basics, but this is one way to help people learn R, who are not already very exposed to R. So let's get into the report formatting. And so there are many aspects to formatting, including how do you lay out figures and quarto has some really wonderful capabilities. As we saw earlier in the conference, you can also make customized HTML tables to make tables as advanced as you want.  So do you want to place things in marginal notes? Do you want to place them in tabs? Do you want to expand or hide material? I have this GitHub repository with a file called RepTools for reporting tools. And these are helper functions that were written largely to extend what Kordo can do, such as making dynamic tabs and dynamic marginal content. And I'll show you a little bit about how those are used. Then you have decisions to make about your overall format. I didn't list Microsoft Word because I use that as seldom as I humanly possibly can, but Kordo is really great support for Microsoft Word. And then there is a section in here about how to do multi-format reports because some of your content needs to be dynamic. For example, when you're producing PDF, you can't have an interactive graphic. So it needs to be. Let's say if you were using ggplot2, you need to produce PDF. But if you're producing HTML, it can be interactive, so you can use plotly. And so you might have code in your report that is sensitive to what the format currently being produced is and use different graphics calls, whether it's PDF versus HTML. A big theme is metadata or data dictionary. And so this involves at the simplest level, variable labels and units. And there is a section in the book about making making the claim that you really don't want to have highly descriptive variable names because that means the variable names are so long that they're unwieldy and your code is actually much harder to read. But you really want to have fully descriptive variable labels. And then those labels are attached to the names in a variety of ways and can be dynamically looked up and popped up in our studio windows. And I go into that in some detail. So I use the variable labels for the full description. I also make heavy use of units of measurement for better annotating plots and tables. So that is about annotation of reports. And then there's other material in this chapter that goes into detail about tables and multi output format. But the next thing I want to cover is analysis file creation. So we typically import things like binary files, could be a SAS dataset or STATA or we import CSV or we have an API that goes directly into a system such as RedCap and we may want to import 50 files at one time in a loop and I give examples of how to do that very, very easily. So when we're importing, we want to have variable, we want to have manageable variable names and we want to add data dictionary content or metadata and so we can import labels. We can have external metadata that's in another place that is then associated with your primary data to define labels and units and comments about variables. And then we need ways to view data dictionaries. So the book tells you how to easily call a function that will pop up a data dictionary into the RStudio view window. So you can use that data dictionary as a guide to help you remember variable names and help you formulate your analysis code. So there is a section about importing and I just want to mention that say this is one way you can import a dataset that doesn't have any metadata with it and you can associate the metadata on the fly very easily using this old function in the HMS package called Updata. So you can see that we're starting with the dataset D and we're renaming various variables and the .q function is just a quoting function in the HMS package so you don't have to use as many quote marks. Then we define a little function Y in because we have a whole list of yes no variables that are coded exactly the same way and we wanted to put them as factor variables with levels yes or no. So we're just transforming those as Updata is building your dataset. We're defining levels of a variable. We're dropping variables. We're defining labels. And then we're defining units of measurement. So this would be what you might do when you don't have any external metadata and you don't have any metadata coming directly from your data system. So we'll see in a minute when you're importing from RedCap you'll have metadata you won't be doing this like this except for units of measurement. So RedCap used to fully support units of measurement and Paul Harris took that out a number of years ago and I've been lobbying him ever since to put it back in because I think units of measurement are really fundamentally important attributes for continuous variables. So there's separate sections in here to how to deal with various kinds of input data whether it's binary like Stata, SPSS, SAS, Excel, reading multiple files. And for RedCap I'll just mention this is for people who are not using one of the at least two R packages for reading directly with a RedCap API. This is using this import RedCap import RedCap function in the in our GitHub repository. So when I say import RedCap parenthesis parenthesis that will read in the last exported file into your directory from your directory and it will read the R code and it will read the CSV file and it will clean that up so you know RedCap makes two copies of all your variables if they're categorical. We don't really want two copies of variables and we don't want the word factor to be part of a variable name. So this cleans up a lot of things and creates a very streamlined data frame with all the metadata that we can get out of RedCap when you're not going directly through the API. So it's very simple to get very streamlined ready to analyze RedCap files just using the standard R export into CSV and .R files. So that's all I'm going to cover from the analysis one. The missing data section you know we all have to deal with missing data and there's a lot of ways you can understand your data. We can look at the extent of missing data per variable or per observation. Those are both very important. We can look at patterns. So how does a missing this cluster how if you were sequentially excluding variables so which which variable is missing the most and then if you were excluding on the basis of missing on other variables how many exclusions would you get. And then there's interesting patterns you can look at such as the association between values of non missing variables and the number of variables missing per observation. So this is actually a statistical analysis that's automatically done where you run an ordinal logistic regression to understand the strong predictors of the number of missing missing variables and observation has and whether you have more missing data for the sicker patients or for the less sick patients or whatever. So that gives us a statistical analysis of missing this patterns and so there's this rep tools function that defines a lot of other functions. We're importing a data set from our data set repository that's fully annotated the support data set converting it to a data table and running the miss check function on that saying that we do want to predict the number of missing per observations. There's a variable we want to omit while we're doing that that's really kind of redundant. So you see the number of missings per variable and per observation they the minimum maximum and mean number. So out of a thousand patients we tend to have 141 people missing on a variable and we have the whole distribution of number of missings per variable or per observation and then the miss check function creates all these tabs for you automatically. It gives you sort of packaged analysis so the number of missing values per observation is a dot chart that was per variable sorry the number of missing variables per observation is a different dot chart. This one is has been handy for me the mean number of other variables that are missing when the indicated variable is missing. So when serum creatinine is missing that bottom dot there on the average nine other variables are missing on the same patient. So it's just one of many ways to understand the missing patterns. You can look at sequential exclusions. ADLP has the highest number of missings but you have additional patients that are missing on other variables. We can look at combinations all possible combinations of missingness whoops using this kind of dot chart and if you hover over things you get a lot more information about these combinations so you can hover this this is using plot length type of graphics you can see marginal counts you can see combination counts and more information at the top. So those are all possible combinations of missingness and this is where a model is run to predict the number of missing variables on the basis of the values of the non missing variables the variables that are never missing. So you can see the time until death has the strongest ability to predict the number of missing values so it turns out people that die quicker have more missing data at baseline because a lot of them are on a ventilator and they're unable to be interviewed to get certain data points. So that just gives you a lot of different ways here in these tabs to explore missing data and then data checking we'd like to do range checks and cross variable consistency and like to do this with minimum coding and get listings and summaries. So if you look at the way most people do this it takes a lot of coding and you can really capitalize on R to do this with a lot less coding because you can create expressions for your checks so these are conditions that should not happen very often. So in this data set someone who's younger than 30 years old or older than 90 it should not occur very often in the data. Somebody who's female and has a maximum heart rate above 170 I just made this up I don't know if that occurs often or not but it's just an example of a consistency base suggestion fraction is between 72 and 77 or is greater than 77. You just have this whole array of expressions and then you pass that to data check and it will execute that array passing through the observations and we can tell it to also give me a report by ID and so you get a separate tab for each check with the list of observations that met the criteria. See that was H less than 30 age greater than 90. So you see that here's one for accommodations and you can also get your summary by ID and site with the check that was triggered the values of the variables that led led it to trigger that as a positive check and you can also get a summary which is giving you counts of how often certain triggers were were met. So those are all just automatically produced by virtue of having data check create tabs that that quarter knows about. So that's data checking and then data overview we want to look at observation filtering so we need to create diagrams where we insert computed counts that we're making and we can do that many different ways but two ways are with consort diagrams and with the general purpose mermaid and then we also want to get more data about the data. So we have various missing value snapshots some of those were presented just a few minutes ago and we can have data characteristics and hopefully that'll make sense in a minute when I show you an example. So that includes breaking variables down a discrete versus continuous variables. How many ties are there in the data? How much information is in the data? This is something you probably not used to seeing but we have an information measure that goes from zero to one so a variable that would have near zero information would be a binary variable where almost every observation is a zero. So that would be a very low information content where a high content variable would be a continuous variable where there's no ties in the data. Measures of symmetry of the distribution rare values and common values. And so for filtering of observations we can here's an example of using a data table to create a clinical trial where as different criteria are met people are randomized and then they're randomized to treatment A or B. And then that's our simulated clinical trial data. And then we're going to set up the counts that we're going to need that are passed to the consort package using the consort plot function and giving it the labels for the various branches or nodes and certain other information will get a classical consort diagram and you have these options of putting in sort of large categories to organize your chart but that's just a standard standard consort diagram. And you can actually more naturally make a consort diagram if you use the building blocks instead of using the actual consort plot you use all these building blocks that the consort package comes with like adding a box adding a side box and I just found this a little more intuitive to calculate things myself and pass them to these building blocks and I'll get the same output but just the inputs are much different. And then with mermaid you can make all sorts of plots but mermaid is going to construct the plot the way it wants to so it won't meet your actual consort rules but one of the things that makes this pretty general is you see when I define the mermaid mark up here it's all in quotes it's all under this object X and so this is brace brace is a macro so this is using the net expand function in the knitter package we can pass macros so it's just like passing our variables and by having brace brace the variable values are inserted at that point in the chart so this is just a very general way to associate variable content with nodes or parts of nodes because it's parts of the node labels so if you use this little mate mermaid function that I wrote this is where you define all those variables that are referred to by by these symbols up here so like this one and that's just a count of how many people were on treatment B and they had the response variable measured whereas this is a count of people on treatment B whether or not they had the response variable already assessed so when you pass that through make mermaid it will create the mermaid diagram so this allows you a lot more flexibility than what you can do with a consort diagram but it doesn't doesn't meet the classical formatting criteria but I think it's still useful for a lot of things you can also have nodes that are more dynamic and and they will have more pop-up content so this is done with a callback where I have a table that pops up when I when I click on a certain area this is one one one of the few problems I've had with Cordo it's a little bit buggy and things like this so this is supposed to work that you're supposed to hover and see a pop-up table but I found this really browser dependent so that's not quite there yet but I I've had it work with some browsers but not with the main browsers that are used so that's supposed to be a live link you can also have URLs and link to things by clicking within the nodes of the mermaid chart descriptive statistics is a very wide area and we need to really respect the nature of the data and so we have various tools that I use over and over for descriptive statistics the primary one being the describe function which is a very old function in the HMS package for univariate summaries and so this will give you statistical summaries and also give you graphical summaries and for continuous variables it gives you these little spike histograms which are nearly full resolution displays of your distribution you'll see examples of that in a minute of course for categorical variables we just have frequency tables and frequency dot charts and for continuous variables we also besides spike histograms we use extended box plots and then for longitudinal data there's special displays that you need that I really want to be covering now and there's also special displays you have for multiple category event charts and timelines and various displays for showing relationships such as variable clustering so let's just look at a couple of examples very quickly this is reading this stress echo data set and I've made it so that Cordo hides that output by default and I click and I can expand it so I see statistics that are appropriate to each type of variable and you see it has different font for the label compared to the variable name and a different font for the units of measurement so you see quantals genies mean difference which is better than the standard deviation for measuring dispersion the mean the information quotient how many distinct values there are and then you have what's the most important output is is the raw data distribution with high resolution so either one or 200 bars you can see extreme skewness you can you can see extreme digit preference here in in blood pressure and and you can see you won't see digit preference when you make regular plots that most people use such as broad bars or CDF's you won't see it in a CDF very well but you can see by modality and digit preference and all kinds of things when you when you show the data at high resolution and you can see it doesn't take much space in the report to be able to do that this is just capitalizing on HTML output which can contain little little widgets little little graphs that are converted to base 64 encoding for use in standard self-contained HTML so that's what the describe function looks like and you see that for for binary variables and so on and then there's ways to pop up various things in windows to help you view the output which is a guide when you really get to the analysis and then here's an example where we use the make tabs function in the rep tools repository where we're plotting something that produces two plots it produces one plot that's labeled categorical variables and one plot that's labeled continuous so you can see the continuous variables are just proportions in each category and you see this is a variable has two categories here's one that has three categories and you have these three proportions here for history of cigarette smoking and then when you have continuous variables you'll see the spike histograms now that gets more interesting when you say let's make these plot late graphics and we're going to make tabs like we did before and make these dynamic tabs so now when you hover over something you see a lot more information you see the statistics that were in the table that the describe function created and and these are the categorical variables and you see always numerator and denominator so there's never any question about what these fractions represent if there was missing data these these dots would be color coded by the amount of missing data and then for continuous variables you see what we saw before but you can see the individual bar so this is a $138 these were bends to the nearest two millimeters of mercury there were eight patients at the 138 bend but then importantly when you go over here you see all of the detailed distribution of characteristics and number of missing values and quantiles and so on so that's just the plot late version which works in any HTML report and it gives you this hover ability to look up much more information there's a lot more ways to get summary data and the summary M function is for summarizing multiple variables so I give a formula like this that has lots of dependent variables and I'm making history of my currently on the fraction an independent variable so this is going to be my stratification variable and the rest of these are analyzed separately by levels of history of MI so that formula is past here and we're going to use make tabs where you can say the first tab is going to be empty so if you look down here you see a blank tab that just means that by default nothing's going to show so nothing will show to you click on one of the tabs the first tab is called table one which is the HTML version of the summary M function so let's see what that looks like when you show this on a wider screen all of this formatting is automatically adjusted to be very beautiful like you're used to seeing in a table and it's using different fonts and so on so that is the HTML version of the summary M and if we want to look at a categorical variable plot we see something like this if we want to look at continuous variable plots we see extended box plots so if you hover you'll see the definition of what the corners represent so this corner represents the 95th percentile and here's the 5th percentile down here which is 104 and you have the mean and you have the median the interval that contains 25%, 50%, 75% of the data so that's how you look up what the points in the extended box plot means that's a lot more information than a regular box plot you can also do summary interactive spike histograms that have the same information on them as a box plot and with plotly you can turn certain things off if you decide you don't want to currently display that you just click in the legend and that will take something away and you can click and put it back in so that is a whirlwind tour of various descriptive statistics that I use as standard and data manipulation and aggregation is where we tend to spend more of our time and here is where data table is really the central component so the biggest take on message of this talk is if you haven't already learned data table you need to start today because it's probably by far the most important add on package in all of our it gives you a logical way to work it is super logical, it is coherent, it is consistent and it's blazing fast so it has nothing but pluses but you do have to learn it and I think one of the things we face as data scientists there are people that want to get up and running as a data scientist without mastering the mastering their trade so if you don't really master the base of R the base R methods and code and to me if you don't master data table you're really not mastering your trade and so data table is not the fastest thing you'll ever learn but every hour that you invest in learning will pay off with multiple hours of personal efficiency so using data table and using base R we can do sub setting we can modify variables we can do various recodes and reshapes so this chapter goes through this and this is the schematic that really explains data table in a nutshell you have a data table which looks a lot like a data frame it's rectangular you have something that tells you what rows you're operating on what to do and what columns you're operating on and by grouping you can also have an on operator here when you're matching on certain variables that might have different names and different things and then we can have lots of extra arguments so this chart goes into a little more detail to help you with that when you type the name of a data table and a bracket that means you're entering into the environment that data table and everything you do is within that environment and so you enter the environment you're dealing with rows columns and by or on so the rows are the rows to fetch or to change you have the columns to fetch or the columns that you're creating or redefining and creation of new columns and redefining will be done with a column equals and then you can have by for grouping or on for matching and then when you get out of that environment you have to figure out what are you left with so when you leave that environment by closing it with the bracket if you didn't have a colon equal you might just have printed or plotted the output and you might have assigned the result to a new data table if you had a colon equal that means the data table in place and if you have a very large data table to be able to add a new column in place is a major time savings so you can add and modify columns leaving the original name original data table in place so that's really the setup for data table and then what I did in writing this book is I had a bunch of notes dating back 10 and 20 years of how to do common tasks so I had all these code snippets lying around and it decided they weren't useful to me or anyone else unless they organized them so I brought all these code snippets in from a variety of little files and annotated them and it's tried to write down what are the tasks that we need to do all the time such as finding observations in November finding all the observations from the month that's like MB in other words that's MB in the name of the month combination criteria pulling off one variable pulling off two variables out of a data table these are just things that we do all the time and just try to show prototypical examples of those and so you'll see a lot of examples here this one just is translating an existing variable to make it uppercase how do you analyze selected variables and subsets but really I want to get to the recoding because that's where we that's where we tend to spend a little bit more of our time or before that this is just how you change variable names using data table or you can use the dot queue function just to keep from quoting so many things so renaming variables very easy and we can rename variables by matching pieces of strings and that's just very very easy to do but when we get into recoding you have so many options and one of the most powerful and logical ways is with the F case function in data table and so it's going to recode according to the first condition that's met so if the patient died if this is true the result for this X variable is going to be quote death if X2 was either stroke or MI quoted we'll call that a stroke slash MI will be the category to assign if the patient was not one of these but was symptomatic it's this otherwise we default to none so F case is just one of many many ways to recode and it's a very logical and powerful way to do it sometimes you're recoding it's just a simple table lookup and you don't need data table or anything else for that you can just use named vectors so we can pull in state using state abbreviations we can look up the state name and then there's an example here for hierarchical recoding so we want to recode things where we have plants and animals within plants we have vegetables and fruit we have animals we have domestic and wild so we have a hierarchical tree and we can navigate that tree to do various recodes and without going into the detail that this will show you how to do that so recoding is a big topic operating on multiple data tables is a little more advanced topic but there's a lot of capabilities there that you can look at later to get into summary statistics we can summarize all the variables we can summarize subsets we can summarize using functions that return multi-dimensional results we can do marginal summaries and data table really shines for this and there's a new package that works well with data table and other systems which I think is called Collapse and I'll be adding that to this chapter because that is a very general way to compute summary statistics and it's ultra-fast for huge data sets but what's built into data table is pretty amazing and one of the things that's tricky for people to learn but once you learn it .sd stands for the whole data table so you can do operations on all the variables each variable separately the operation we're doing here is to run the nd function for a number of distinct values for a variable so we're saying to run the nd function separately for every variable in data table d and we just get a count of the unique values that way we can say do it for the variables that match these patterns the variables that have hx in their name or they have either d or m so that's an or symbol there so we're saying this is just the variables that match these criteria here and we're running that analysis just on two variables this is creating a function to calculate the main ignoring missing data and we're going to run that on every variable in the data set that is numeric so it's very easy to put conditions when you're running on a lot of variables let's look at this one here we're defining the function c-mult which is that the the variable is not numeric and it has more than two distinct values so that's a true false function and we're going to just run that function here to see which variables qualify for the analysis so that's like saying is.numeric but it's more general these criteria and then we're going to do a table and we're going to summarize each variable with a very brief frequency table hoping it doesn't have too many levels for us and that's the output that you get just more examples of creating functions and I'm going to make a major point here this won't seem like much of a point but when you write a little function like this and you get that function debugged you can use that function in the more complex context but debugging it here means you can understand it and make sure it absolutely works on one variable so you only have to debug it on one variable to make sure it's right and then you're using that same function for multiple variables so you'll see that philosophy pay off in many different ways with this approach and I'll show you a more advanced example so we're just sub-setting data and doing lots of different observations and then when you're using the data table function you also have access to multi-way summarization so the cube function when we're saying by-gender or let's say by-gender and the history of micro-linear infarction we're going to summarize the mean basal heart rate with mean we're going to summarize heart rate with the mean and with the number of non-missing observations so we define these little functions up here and then we run it by gender and history combinations but we're doing it with cube and not with a regular data table operation because when you're doing it with cube you'll get it for all possible combinations like you expect but then you get all the marginal summaries so here's a summary by gender ignoring history summary by history ignoring gender and the summary ignoring both which is your overall grand summary and then if you want to have more control over that you can use the grouping sets function in data table and you can tell it restrictions on how you want to use cross-classifications when you're summarizing the data so that was summary statistics and merging data I'm just going to show this this is a front end to both the regular merge and the data table merge merge with a capital M it helps you merge lots of data sets at one time but more than that it gives you a report on how the merge went first of all how do you do merging in base it's very very easy and very very fast if you want to use this front end capital M merge I'm merging baseline with longitudinal data merging on ID and you see I get a report if I don't say print equals false here I get a report so in the baseline data set there were two variables for observations there were four unique IDs in the longitudinal data there were three variables, 12 observations four unique IDs and only three of those IDs were in the baseline table in the merge table there were four variables total 13 observations five unique IDs and four of the IDs that are in the final table were in the baseline table and then you also have this information that's reported so this is just to help you feel good that the merging went well now longitudinal data is a huge topic I'm not going to cover very much of but we have various needs for processing longitudinal data and data table is just exceptionally powerful for processing longitudinal data and so you can deal with longitudinal data of various types and you can convert between various types and so you might have a uniform number of rows and you might want to do something like last observation carried forward which we usually frown upon as statisticians but there are built-in functions in a field and set in a field in data table that make it real easy to do that but in other cases we have variable number of rows and you might want to carry forward in that context means you need to create new observations that didn't exist before so you're going to expand the number of observations and this just shows you how to create observations past the last observation out to the maximum possible day you're creating these new observations and you're combining those with an R bind onto your original data set but this is where data table shines even more so you want to say what is the first day in which Y was greater than or equal to three so what is the minimum day such that Y is greater than or equal to three we're going to call that first three what's the first day in which Y is greater than or equal to seven we're going to do that separately by patient so this is what happens when it never did pass the threshold of seven puts in an infinite which we can go to a little trouble to make that be an N-A instead of an infinite but where this philosophy really shines is when you have more complex conditions so if you wanted the days to be consecutive you can say that let's say we needed Z to be greater than .5 and the Z on the previous day to also be greater than .5 we're going to do that separately by patient so that's just an N condition with a lag of your measurement but that's not still the most general way to think about it the most powerful and general way to think about it is to write a function that counts so even though you might be interested in the second day in which something happened let's count how many days in which it happened how many days in a row was something satisfied so this function will do that very concisely so when something happens in a row so this will give you a count of two but then the count starts over and this would be a count of one and a count of two there then it starts over and this is a count of four so this will calculate the number of consecutive true values for doing your table data table operation you can say give me the first day so this would be the first day the minimum day for which the number of consecutive occurrences of Z greater than .5 is equal to two so once you write this function and get it to work for one patient and get it debug now you're using it separately for all the patients instead of just writing condition that says Z is greater than .5 in the previous was also greater than .5 let's use it as a counter because we might want to have three days in a row and we just change that to a three so this means the second consecutive day was Z greater than .5 and so once you learn how to write counter functions and you learn how to debug a function for one patient which is much easier than debugging it for the whole dataset you see you just plug those functions in and use those in the data table context when you're doing overlap joins you can get a lot more complicated and data table is extremely powerful for doing that so without going into details what we're trying to do here is we have an events dataset with zero values per subject containing starting in times and a measurement X representing a daily dose of something giving you the patient between S and E starting in times you have a base dataset B that has one record per subject with times C and D and you want to compute the total dose of the drug received between C and D for the subject you do that by finding all records in E for the subject such as the C and D has any overlap with the interval S to E and for each one you compute the number of days in the interval S to E that are also in C to D so sounds pretty foreign but once you get your dataset set up we run the F overlaps function in data table and we calculate we do this by for the left-hand side the left-hand data table we do it by ID low and high and for the right-hand one by ID start and end type equals any and now this calculates the elapsed time for the right overlap intervals and the total dose over that correct elapsed time so this sounds like a pretty complex thing and this code is not the easiest code to read but you can see that the once you get a hang of it the amount of programming is is pretty minimal so lots more about manipulating longitudinal data and then graphics is a very general chapter with some graphics principles mainly and a few few tricks about using ggplot2 which I I use very very heavily and I also use plotly a lot and then the analysis chapter is really showing you how to really do better than table one by staying close to the data and I'll just sort of stop with showing you an example of a what I think is a pretty rich presentation that's a lot better than table one would be this presentation so this is using ggplot2 instead of showing the age distribution in treatment A and treatment B it's a randomized trial we don't need to know the age distribution by treatment because it was randomized it's irrelevant what we need to know is how does age relate to the outcome of the trial so the relationship between age and the probability of dying in the hospital is given by this non-parametric estimate and then using this gglayers function and add easily extended box plot and spike histogram to the ggplot2 standard output so now we have the univariate distribution of age which would be the combined treatment A and B distribution of age or the raw data for age instead of stratifying this by treatment and giving people confusing data we give them the overall trial of patient characteristics in our chart which is augmenting the relationship with the outcome which is data that table 1 doesn't try to show usually but it's a lot more valuable data than table 1 and this uses a little function called melt data which is just a way to say you have one dependent variable lots of independent variables and you're going to reshape that into a tall and thin data set to pass directly into ggplot so that you're faceting on the variable so you get separate analysis for each variable with a free x scale as you go across variables they don't need to have the same x range so sort of to conclude things I think data table is one of our most important tools by far I think you've seen how you can put things together with quarter and what I really didn't show you is how in some of these chapters I produce marginal graphics with some functions that make it really easy to put things in the margin quarter has all these beautiful formatting capabilities and quarter like data table gives you a unified way to think about reporting whether you're making a web page a book a report or a slide presentation quarter can do all of that without having new R packages so I'll stop there and see if I've stimulated any comments from anyone thanks for listening thanks Frank for always interesting presentation lots of chatter on the different things with data table so one of the questions is that when will we move beyond static paper and PDFs for research manuscripts and FDA submissions that's a wonderful question I think those are FDA submissions are probably going to move faster than journals I can't really speak for FDA even though I work for FDA now I think FDA will eventually have more flexibility because it helps their reviewers review easier you can do a global search within an HTML report you can drill down to see details I think journals are hopeless and are not going to change and the best way to deal with journals is to quit submitting papers to journals and just take things under your own control so when you're able to self publish or put something in a preprint archive or write a blog article you have total control and you can use the latest and greatest interactive graphics and another question is do you have a catchy name for your replacement for table one I need a catchy name or it's not going to catch on I would be open to suggestions but the purpose of the replacement to table one is to show that are not preordained you know we look at comparison of treatment A and B it's really irrelevant at a randomized trial especially one that has more than 30 or 40 patients in it but what we don't get is just you know who's in the trial and then what happened to them and was it if death is your outcome was that mainly in those over 70 or did the deaths get spread across all the ages that's the information people are not giving that is absolutely interesting and sometimes unexpected so Peter suggested baseline outcome plot bop yeah and it needs to connote it's baseline distribution because it's study population description but also outcome relationships combination yeah and then Peter says that he agrees on public publishing but it's easy for him since he's senior but hard for trainees mentees who are trying to build up their reputation and get 10 years you know any thoughts on how to do this without hurting a generation of junior people that's really probably the trickiest of all the questions and I think promotion and tenure committees are doing a major disservice now by promoting the wrong people they're not promoting people whose research is reproducible they're just reporting promoting people they have high volume output or have splashy splashy results but I think the tenure and promotion process needs to be totally redone but there was an announcement this week from NIH that's going to have a lot of ramifications which is NIH is really changing how they want reporting to be done and they want it to be much faster and not wait on journals they want the results more usable quicker so I think the step that NIH took this week which I need to read more about may get us going yeah and so many institutions use easy metrics to count publications and things like that for reviews for promotions and things like that and it's not necessarily quality right so have you considered using the code link true in this auto YAML setting it's hard to know that's my next thing to do because and the only reason I haven't used code link already I think it has a dependency that I didn't have installed so I need to look at that one more time but I think for those of you don't know code link it will when you have a function call in your code that's listed in your report you can click on the name of the function for that function I think that's what code link does and I think that's pretty cool so that's on my to-do list that's now on my to-do list as well yeah because sometimes it's hard to know what's coming from HMISC or data table where are things coming from so that might help with some of that and I've also really appreciated having being able to collapse code so that I can insert a figure but not disrupt the flow of the diagram so people can see the code of how things were created but not necessarily all the details if they don't want to and I also let me show you just one thing in this analysis chapter there are certain analyses that you do where you want some information I see your PR equals margin these are some denominators that define how the smoothing was done and that will appear in a marginal table so sometimes you put things outside the body that are supplemental information that don't get in the way of the body of the report and then other times you'll use a command or just hard-coded in Courtault to have an optional section that's a show or hide or collapse collapsible section to have more information and I think those are very valuable for reports too yeah and I like the use of having the figure legends in the margins I really like that yeah okay from Travis I don't know why we would want to analyze baseline variable relationships with outcomes and trials that aren't designed to evaluate those relationships could get misleading results for example if there's an interaction with treatment assignment so I never want to criticize doing an analysis because something more complex might be in play because if you don't try to analyze it you'll never learn anything about that interaction so the presentations that we do clinical trial reports now do nothing but hide interactions so I can't really agree with the premise of that question but I think most trials are actually designed to do exactly what I'm suggesting they usually have some kind of age distribution and they usually have a severity of disease at baseline and the study report doesn't even tell us whether those patients with minimal severity of disease ever had any of our endpoints so by showing the relationships between severity of disease and the outcome we're giving new information so I think most clinical trials have the information you need to create these relationships so the question I'm always left with in a clinical trial is what happened to the patients and which patients did it happen to and this is an attempt to solve that right any other last questions for Frank all again thank you very much for this presentation and we will take a quick break until 26 after the hour coming back with Peter and talking about wrangling medical data thanks for having me and thanks for the great comments and discussion yeah generally we've generated lots of chat great I have to go back and look at some more of it


Thank you chief Morris. Thank you chiefs welcome welcome back from lunch I trust it was healthy and nutritious and I'm going to struggle to keep everybody's focus after we had such a such a great lunch But I think we've got a really fun topic to talk about even though I only get to talk about it at an unclassified level but talking about national defense strategy that was set forth from the Secretary of Defense last year and It's ultimately changed the paradigm of how we in intelligence take a look at our problem sets and how we can solve these problem sets and it Gives us a little bit of a structure now if you've seen previous briefings that attack what we do on a global scale Maybe you've heard the the Intel equation of four plus one well now We're in a two plus three concept, and I'm going to Explain that here very shortly because as we see within the national defense strategy And I do want to take a poll of people of who has actually read the national defense strategy Very good response. I I hope everybody takes the time over Over the next few weeks and months to please go over that document because it really does outline how the paradigm is shifted in particularly over the past 20 years of conflict as we've seen As we have evolved from really fighting against an asymmetric warfare fight and against rogue nations and really seeing a reemergence of great power competition rising up against us trying to evolve within a post-World War two order and maintaining that as The great powers are really trying to pull apart our NATO alliances and greater alliances that we use That are very pivotal to how we are able to execute our day-to-day operations in our global Operations as it is a truly global fight. Where are my partner? nation represent representatives here today So great. Thank you so much for everything of what you do that really helps us bring strategic power To you know to the globe and everything you do is as we work together So so we see once again is the two plus three concept now as we talk about our great power Competitors of China and Russia and then we'll also delve in in more detail on the DPRK Iran and as well as violent extremism So let's take a look at the map and we see the physical area that these competitors take place But really it's when we talk about how cyber works in and through these capabilities And as violent extremism perpetuates throughout the whole planet really they're not constrained anymore to borders Really as far as influence in trying to propagate Technologies and capabilities it really is a global threat environment as we go through the multiple regions and functions And working through the different domains, and we've already talked about Multi-domain command and control and how cyber capabilities enables all of that. That's really what we're talking about So when we look at adversary goals We'll start off with China and how they're really trying to pursue a National rejuvenation and they're going through really in the same way that we talk about the American dream well Xi Jinping has established what he calls his Chinese dream and China is actually getting ready to celebrate the first of two 100-year anniversaries the first of which which will take place in 2021 is The 100th anniversary of Communist Party rule within China and now the centenary goal for that was for China to be able To be an economic power that doubles its GDP Over from 2010 and they've already been able to do that three years ahead of their goal Their second goal as of 2049 that will be the 100th anniversary of the People's Republic of China And they want to be the preeminent power supplanting us within within the world and They look to do this in multiple different ways Maybe you've heard of the Belt and Road initiative where it's a way for For China to not only be the regional hegemon and in control Finances and all sorts of logistics within the Pan-Asian region But also building capabilities that will link real really global commerce and be able to control When controlling global commerce at that level ultimately it's going to have an effect on a battle space as well When we talk about Russia we talk about a nation that went through a bit of an identity crisis coming out of Coming out of the 90s and through the 90s once the Soviet Union fell and trying to re-establish itself As a as a global power and we see them Ultimately trying to to flex their muscles and and corner the market as far as their their weapons development and proliferation and Using that to help spur on an economy that had been struggling which is now starting to To be able to regenerate itself and they're trying to assert themselves a little bit more globally We'll take a little bit more of a look at that in North Korea now We get into these rogue regimes and a lot of what they're trying to do in Korea is really legitimizing that particular regime and instead of having the world looking at Kim Jong-un as a despot Last week. What was he able to do? He was able to get the world to look at him in the summit basically on par with The the world's top leaders and that's really what North Korea is trying to do with all the flexing their muscle with their Nuclear capability and their military capability is really showing that you know It's an information campaign to their own people to show their own might and that they actually have a seat at the big boy table With with the rest of the main world powers Iran trying to extend their particular influence and we see this in multiple areas within the Middle East with their With their influence in Yemen as well as in Iraq and in Syria talking about materiel and fighters and other capabilities that perpetuate their ability to grasp onto that Key terrain as well as violent extremist organizations So when we're talking about both ISIS and al-qaeda and they kind of have a couple of separate different goals So when we talk about al-qaeda trying to destabilize nation-state authority that we've been trying to build up of the past couple of decades in Afghanistan as well as trying to ISIS trying to reestablish the caliphate that the combined joint task force operational inherent resolve has been able to Decline over the past three years So we'll let's take a deeper look at China and what they've been able to To ultimately accomplish on the nuclear side They are rapidly modernizing their nuclear capabilities and rapidly modernizing their force All the while trying to maintain that they are a non-first use entity Then the numbers are somewhere between 260 to 280 actual warheads that they have which really makes them the The second smallest of the top five nuclear powers as far as how many active warheads They have within their arsenal, but they are they are still very active in Not only Producing warheads, but also delivery mechanisms for that aim doing Intercontinental ballistic missile tests as well and that goes straight into the space discussion because space capabilities and ICBM capabilities sort of go hand-in-hand and also focusing on Satellite communications robust ISR capabilities once again getting into that command and control discussion China's trying to develop Much stronger capabilities in that realm as well as they try to control and influence the battle space to to their Effects within within space as well They're getting a very growing customer base for space lift In the Pan-Asian region and further We see a lot of smaller nations that are just getting into the space realm that are building small satellite packages that are looking to other countries to be able to lift their capabilities up into space and China has taken the forefront in In allowing them to use their rockets to be able to put their capabilities up in space and they're and they're looking to be a Leader with that well the more robust that space lift capability is you can it's immediately applicable to what they're doing with the ballistic missile development as well and China is actually very key on the fact that they are very much against what they would call American weaponization of space and trying to get Russia in on It's capability to say we're going to deter the US at every step of trying to weaponize space well it's mostly also to Move people away from looking at their own anti-satellite Capability and and they see they see that is more protecting themselves against American weaponization vice developing their own space-based weapons Within the cyber realm China is very well developed and think about it this way China is actually the largest English-speaking country on the planet.  Well, they have lots of capability that they have that they have developed through cyber reach to be able to reach into American networks and we We see this as we've got fifth generation Cyber development through the company Huawei and we already know that you know the United States Japan Australia to name a few No longer use any type of of Huawei capability For their government networks, but that's also something that you know potentially other partner nations could be using that type of capability and in China Obviously with a close-off nature of their regime. It's actually in their laws That force the handover of data from any company if it has any type of a military application So we keep that in mind when when we talk about 5g and data and that goes straight into the air and air defense portion of the chart because a lot of the the knowledge and capability of catching up to like fourth and fifth generation fighter development and weapon development within China largely has happened through through cyber means of Stolen intellectual property that China has been able to perpetuate now the struggle that China is going to have moving forward Ultimately getting to that fifth generation goal is really being able to develop their own capability that they can own and cultivate and And be able to advance As well as adding advanced munitions to those air-to-air capabilities whether we're talking Hypersonic capability is something that they're they're fast developing as well as unmanned aerial vehicles and which is really key for Throughout the region for them to build their capability and their fourth generation fighters actually They just recently had an air show where the J-16 has come online and talking about Once again, we we talked a little bit earlier about of small sat capability within the Pan-Asian region. Well, there are also Air forces within the Pan-Asian region and beyond that are looking to expand their particular capability and China's trying to Grow their ability in the market to be able to bring other air forces up to speed with Chinese capabilities And we take a look at Russia Many of us know that Russia has the the largest nuclear stockpile and now not to be Encumbered by the INF treaty which was just torn asunder just a few weeks ago So now President Putin in early February after after President Trump backed out of INF treaty Which is the intermediate range nuclear forces treaty now? President Putin has said that well We're going to redevelop those intermediate range nuclear capabilities that were previously banned under the INF treaty As well as modernizing all the delivery systems to that very robust capability that they have so about 4500 strategic weapons Both both strategic and non strategic weapons that they have within their stockpile very robust space capability once again, they're they're trying to recover their economy and and ultimately and Ultimately bringing that that space capability a lot of people still don't realize They're the ones who are actually taking our astronauts up into space so they can use that capability as sort of diplomatic leverage against us If should we actually increase? sanctions on on them Also very rapidly developing anti-satellite capabilities in concert with with China to deter us freedom of action within space very very sophisticated cyber and electronic warfare capabilities a lot of a lot of what our strategic Competitors would would use they a lot of them use Russian equipment and we've seen them use a lot of this capability as they've infiltrated Into Ukraine in Syria and also into Georgia with those conflicts that that they've been able to to use as a basically a first Initial into the conflict a type of a capability But within within air and air defense We see them using especially in Syria. This has been their proving ground for all of their capability We see a lot of of testing and development and operational actually using new weapons new capabilities Whether it's new radar systems, whether it's new aircraft capabilities Integrated integrated missile defense once again, this is a country that's very highly Reliant on their surface-to-air missile capability But getting really good with their TTPs and passing those on To their Syrian counterparts and doing a lot of great lessons learned for their command and control capabilities now we're going to get into our rogue regimes a little bit starting with North Korea and the summit as I just mentioned before just Just wrapped up last week. There was no deal that was struck and some of you may have actually read some of the headlines from today that one of the Missile production facilities up in North Korea is back up and running so what's not clear is if it was already running prior to the summit or if If this is a result of no deal actually being brokered between North Korea and the US in the lifting of sanctions on to North Korea, but Ultimately, how do we get to this point a very large? ramp up of capability, especially when we look back into 2016 2017 of North Korea's fielding of Intercontinental ballistic missile capability and we actually saw their first two tests that went off without failure With systems that have the ability to range the continental US So still very committed despite still trying to win the court of global public opinion of how the the US needs to lift sanctions on on them as soon as Kim Jong-un was able to declare himself a nuclear state that was ultimately his ticket to To the the seat of saying I am a legitimate nuclear power now Let's let's bring this to the table and and start talking negotiations naturally We we are looking for a lot more So no launches in 2018, but also looking to use that capability within a space sense and and looking to actually use those Use the development of an of an emerging space capability to Continue their ballistic missile Ames or within the cyber realm not as developed as China or Russia but still but still showing quite an aptitude for that as well as Most of their their attacks or most of the focus is south of the border into South Korea some targeted attacks mostly using ransomware and and those types of Abilities to extort money for them obviously for for them to be able to gain More access to materials and an air and air defense. We take a look at North Korea that We say that they have a lot of antiquated Capability when we look at their aircraft or their missile defense systems, but the regimented Ability for them to be able to exercise and be able to use this capability They are really competent and really good at using a lot of this older capability So we look past the fact that they're using outdated equipment They're very proficient at using outdated equipment And once again relying on a lot of surface-to-air missiles and in triple-a that they have in mass In order to be able to counter any threat and then moving on to the other rogue state Iran and another deal that ultimately fell by the wayside in the joint comprehensive plan of action which or as many of you know as the Iran nuclear deal and When that fell basically that that Removed any guidelines that kept Iran from the ability to be able to develop a nuclear weapon within a year for their stockpiles So previously under JCPOA it would it would take them five years or more with With the amount of centrifuges that they had on hand as well as the amount of highly enriched uranium that they had in Or for them to be able to build a weapon now without those without those chains on them Provided by JCPOA they have the ability to be able or with they have enough highly enriched uranium They could conceivably have One year out timeline from developing a nuclear weapon Very minimal space presence, but we did see them conduct a launch an unsuccessful launch last month But they are broadly exploring that capability in order to once again Crossover that ballistic missile capability in space capability. They share a lot of technology with North Korea So once again invested in each other in order to be able to develop capabilities and Trying to evolve their cyber capabilities then we've seen a couple of different instances where Where Iran has perpetrated very key facilities and key capabilities like Saudi Aramco back in 2012 as well as back in 2017 in 2012 a focused a focused hack from the The Revolutionary Guard Corps Actually knocked out 30,000 computers and stopped production from Saudi Aramco So it's a very a very considerable capability as well as They had an attack on a New York dam facility back in 2012 a very similar attack against a Security-controlled and data acquisition system that can control a large capability that has both military and civilian capabilities to that so surface-to-surface missile independent and are working very hard to Further develop that capability despite despite sanctions investing really heavily in unmanned aerial vehicles And we're seeing a lot of that capability especially down in in Yemen with with those types of those types of capabilities being used and and talking just like in North Korea some outdated outdated systems But looking to update their capabilities But once again very proficient with the outdated systems that they have and very reliant upon Other rogue states to help bring them up in their capability Now we talk about violent extremism and we think about the the ISIS problem Imagine The ISIS caliphate what you see in that map up there the gray areas what the ISIS caliphate was at its Largest strength about four years ago and now that red dot at around the Atomf garrison there in the tri-border region is really The last real stronghold of what we would call ISIS's physical caliphate But imagine that caliphate in dealing with it like squeezing a water balloon So the the harder you squeeze it you're going to get water coming through your fingers and it's going to ultimately perpetrate in all different directions and You know and all over and that's ultimately what we've seen with the caliphate as the physical caliphate has gone We see a lot more Mobilization if you look at the map down below it And a lot more ability through decentralized execution Decentralized leadership then be able to perpetrate lone wolf attacks and other coordinated attacks using that capability so being able to eradicate ISIS I It's sometimes seen as a misnomer. Well, we've won the war against ISIS No, we haven't really the war has just begun as far as what we're doing against these violent extremists organizations looking on the al-qaeda side and Talked a little bit about how they're really trying to legitimate or legitimize their role in What's going on in Afghanistan as the government is trying to? build its capability and be and become a lot more legitimate Trying to tear apart the the internationally recognized government there and Establish one that's more favorable to their aims and Perpetrating all sorts of recruitment bases and we see this going through There's a big push to be able to recruit into sub-saharan Africa and it's a major Area because now they're going into areas that are not predominantly Muslim But yet are able to exploit an area that is generally generally repressed and Sewing, you know, sowing all sorts of seeds for for chaos and destruction there in in places that are not traditionally Muslim so ultimately What what I want what I want everybody to be able to take away from this discussion is ultimately the way The national defense strategy has has shown our fight has really evolved over over the last 20 years And where we didn't necessarily worry about great powers. Those great powers are catching up to us Very rapidly But if we devote all of our attention to those great powers Once again, we're losing sight on a lot of those asymmetric capabilities So this is once again why we bring in a whole coalition Of of help and wealth and experience that we rely upon every single day And we appreciate everything of what you guys do So here's a very quick picture of what we're looking at with our intelligence challenge And I'll take any questions that you have this time And we have the post lunch lull I see But once again ladies and gentlemen, I really appreciate your attention. Thank you so much. Congratulations chiefs And uh, have a great conference. Thank you. Right. Look at that time Where was that this morning? Huh? Sorry, I didn't mean to say that out loud just kidding all right I think we are all set Got one slide to come up real quick for this afternoon. Just wanted to kind of show you guys the uh location for the elective electives


Satellite remote sensing is a powerful tool for monitoring flooding in the Mekong Delta. Invisit ASARWSM data was used to analyze the flood regime in the region, providing insight into the frequency of flooding and its impacts on various areas. The data revealed that certain areas were more prone to flooding than others, with some remaining dry throughout the year. Additionally, the data showed how dikes affect floodwater emergence, and how land use can influence flood patterns. This article was authored by Stefan Deck, Sheen Woolley, Patrick Leningkügel, and others.


The harm caused by cigarette smoking is primarily due to the toxins produced during combustion, while electronic nicotine delivery systems, ENDS, do not involve combustion and thus produce fewer toxins. This study found that ENDS users had lower levels of toxin exposure than cigarette smokers, as well as those who used both ENDS and cigarettes. Additionally, the amount of toxins in ENDS users was directly correlated with how many cigarettes they smoked. This suggests that ENDS could be a viable option for people who are unable or unwilling to quit smoking but still want to reduce their risk of developing tobacco-related diseases. This article was offered by Nathan M. Holt, Sol Schiffman, Rainer Black and others.


Welcome to Happiness Isn't Brain Surgery with Doc Snipes. This podcast was created to provide you the information and tools Doc Snipes gives our clients so that you too can start living happier. Our website, docsknipes.com, has even more resources, videos and handouts and even interactive sessions with Doc Snipes to help you apply what you learn. Go to docsknipes.com to learn more. Welcome to Happiness Isn't Brain Surgery with Doc Snipes, practical tools to improve your mood and quality of life. Today we're going to be talking about sleep and why it's important and how it really is one of our greatest vulnerabilities for depression, anxiety and irritability. We're going to start by learning about the function of sleep. Talk about those sleep cycles that everyone talks about, REM sleep, deep sleep, this kind of sleep. What are they?  And then we'll talk about how sleep contributes to feelings of depression, anxiety and irritability. Once you've learned all that, you might be motivated to figure out how to improve your sleep. I know I was. Sleep is the time for your body to rest and restore. Adequate sleep improves your memory and learning, increases attention and creativity and aids in concentration and decision making. So all of those things, if you look at them, well, maybe not memory and learning, but your concentration and your decision making and your energy levels, all of those things are actually symptoms of depression as well as lack of sufficient sleep. So lack of quality sleep, and I'm going to keep using the word quality, is really important in understanding maybe why you're feeling a certain way. Feelings that accumulate in the brain are thought to be cleared out during sleep. So, you know, if you haven't slept well and you get up and you're kind of foggy in the morning, that might be one of the reasons. Healing and repair of cells also takes place during sleep. So it's important to get this good quality sleep. And sleep also helps to maintain the balance of hormones in the body, including ghrelin and leptin, which regulate your feelings of hunger and fullness. So if you're not getting really good sleep, you may feel like you want to eat all the time, or likewise, you may have no appetite at all. But more often than not, people tend to be more hungry or more apt to want to eat and not know if they're hungry or not. Insulin, which is responsible for the regulation of glucose in the blood, is also affected when you don't get enough sleep, which is why it's really important if you have hypoglycemia or any type of diabetes or prediabetes to make sure you're getting enough sleep so your body can effectively regulate your insulin levels. Sleep deficiency is also linked to a higher risk of cardiovascular disease, stroke, diabetes, kidney disease, difficulty concentrating, irritability, fatigue, and loss of energy. Now, if you think about when you haven't had enough sleep and if it goes on for a long time, long periods of sleep deprivation, you know, when my children were younger, there were several months there. I didn't get good quality sleep for a while. And during those periods, I definitely felt a lot more symptomatic of, you know, no energy, irritability, difficulty concentrating. And this extra stress on the body from not getting enough sleep is generally what causes some of the problems, the physical health problems, that result from lack of sleep, because over time, the body starts to break down, which is, again, why it's important, nip it in the bud, get as much quality sleep as you can. Stage one, or non-rem sleep, is when you drift in and out of light sleep and can be easily awakened. It's kind of like when you nap on the couch.  This usually lasts for about 20 minutes. Now, don't hold me to these numbers because everybody's a little bit different, which is why I say it usually lasts for that long.  Stage three, non-rem sleep, also known as deep sleep or delta sleep, is marked by very slow delta brain waves. There's no voluntary movement, and you're very difficult to awaken. Think about a time, you know, if you've got kids, you've probably experienced this, or if you've got dogs, when you have been in a deep sleep, not dreaming in a deep, deep sleep, and somebody has awakened you, and it's taken you a minute to figure out, okay, where am I? Who are you? What's going on? Okay, got it. Let's proceed. You were probably in your deep sleep at that point in time. This stage usually lasts for about 30 minutes. The largest percentage of deep sleep comes in the early part of the total night's sleep pattern. Now, that's in normal sleep people. If you have depression, if you have high levels of anxiety, or if you are drinking alcohol or caffeine prior to bed, then your deep sleep may not come at the beginning of the night. It'll be when those substances wear off or when you calm down. And sometimes with depression, people have difficulty getting any deep sleep. So it's important to not look if you've got a activity tracker and look at your monitor and go, well, my deep sleep's coming at the end. If it is, you might want to ask yourself, well, when did I have caffeine last? Because caffeine stays in your body for 12 hours. That aside, it's important to know that when it comes, when you get your deep sleep, it usually only lasts for about 30 minutes at a shot. So you're going to cycle in and out of it throughout the night. After deep sleep is your REM sleep, which is characterized by temporary paralysis of the voluntary muscles and fast irregular breathing, inability to regulate body temperature, faster brain waves, resembling the activity of a person that is awake. Most dreams occur during your REM sleep.  But it's important to understand you're not regulating your body temperature really well, which is why you may wake up and feel sweaty and throw the covers off. REM sleep is what most people think about as deep sleep. And while it's good, it is not the deep restorative sleep that you need in order for your body to rest and rebalance. So how much do you need? Well, the hour requirement is a little bit loosey-goosey, because if you're not getting the deep sleep, then you may need more sleep than what's recommended here. So I'm just going to skip ahead to us adults here. And they say we typically need between 7.5 and 9 hours of sleep. And you may be laughing to yourself going, I can't remember the last time I got more than 6. Something to consider that may be contributing to irritability, fatigue, and foggy-headedness. Now, some people get a lot of deep sleep in six hours, and that's fine. Deep sleep needed for adults is between 1.5 and 1.8 hours. And, you know, again, if you use a fitness tracker, not an actual sleep study monitor, it measures your, quote, deep sleep in terms of how much movement you have. And we know that there are two different phases of sleep where you are not having voluntary muscle movement, so your movement's going to be a lot slower. For a rough estimate for myself, I take whatever it says in terms of how many hours of deep sleep I got, and I divide it in two, because I figure probably only half of that was the actual deep restorative sleep. So I aim for between 3.5 and 4 hours of, quote, deep sleep, according to my fitness tracker each night. Obviously, you're going to need to do what feels best for you. Some people, if they sleep more than 8 hours, they just don't feel like they can get started. Other people don't feel like they can get started unless they've had at least 8 hours. If you're sleeping a lot and you're not feeling rested, it would be interesting to get a sleep tracker of some sort and Fitbit, Garmin, there's bunches of them out there. And see how much deep sleep you're getting. But at the end of this presentation, we're also going to talk about things you can do right now to start improving your sleep and just keep a sleep journal of I slept this many hours and when I woke up, I felt rested or not rested. How did you feel at noon time and how did you feel at bedtime?  Estrogen usually improves the quality of sleep, reduces the time to fall asleep and increases the amount of REM sleep. Now too much estrogen can keep you up, but too little estrogen as you see in people who are going through menopause can also keep you up and mess with your ability to control your temperature and those sorts of things. Too little or too much testosterone may also affect overall sleep quality.   Cortisol is your stress hormone and it prevents restful sleep. When you've got cortisol going through your system, it's telling your body there's a threat somewhere and you can't afford to go into a deep sleep because you need to be able to respond as soon as I tell you to. And we know you're not going to if you're in deep sleep. So if you have high levels of stress and anxiety and you have ongoing cortisol pretty much throughout the day, you keep it up because of your anxious thoughts and things like that, then you most likely will not get restful sleep. So it's important to start addressing some of those thoughts and issues in your life that are keeping your stress levels high consistently where you feel like you're burning the candle at both ends. And thyroid hormones, which are too high, can also cause insomnia and too low can cause fatigue and lethargy. Again, this is one of those your doctor can measure and tell you, yes, it's in the normal range. This is probably not causing your problem or identify that there is a problem. So when you're going for your physical, if you're having symptoms of fatigue, foggyheadedness, depression, irritability, it might be worth talking to him or her about getting a full blood workup done. Nutrition, tryptophan is used to make serotonin. Serotonin is used to make melatonin, melatonin functions to help you feel sleepy. Well, tryptophan is a protein and it's found in most of your meats. Your body cannot make it. So if you don't eat a good diet, your body can't make the neurochemicals needed to help you feel sleepy. Some people will try to take melatonin supplements. That works for some people, for other people. It is awful. They wake up, they may feel good for 30 minutes, and then they're groggy the rest of the day. That would be something I would strongly encourage you to talk with your doctor about. The other reason for that is that a lot of the over-the-counter supplements are significantly higher than what the American Sleep Foundation recommends for melatonin supplementation. So before you take any supplements, always talk with your doctor. But if you eat a good diet, or make sure you eat a good diet, so your body can help you make the neurotransmitters you need to get sleepy. Caffeine is a stimulant and it has a six-hour half-life, which means it's not out of your body for 12 hours. So if you drink a espresso at noon, it is still impacting you at midnight. Just keep that in mind. If you're having sleep issues, you may want to stop ingesting caffeine at like 10 a.m., and then some people may be like, oh my gosh, I couldn't get through the day without it. You may consider weaning down slowly, going first to half and half, and then 25% caffeine after 10 in the morning or something, so you don't get that jolt. But when you're taking in caffeine, you're causing your body to run more intensely, hotter than normal, if you will, burn up more energy than it really has.  So you're going to be tired and it creates sort of a vicious cycle of needing more caffeine to keep going. As you wean, and I've done it myself, I went from three pots of coffee a day to two cups in the morning, and I feel a lot better. I don't feel like I need to have caffeine throughout the day. But be aware that as you wean down, you may start to get headaches and drink plenty of water, et cetera, and take it slow.  Nicotine is also a stimulant with a two-hour half-life, which means half of it is out of your body in two hours. It doesn't completely get out of your body for four hours.  Decongestants like pseudofed are also stimulants with a two-hour half-life. So be aware, if you've got a cold, you may need to take them so you can breathe. When you're laying down trying to get to sleep, I know for me, when I get a head cold, I don't sleep well with or without the decongestants, because I can't stand it if I can't breathe through my nose. So you may feel like you need to take the decongestants, and that's totally cool, but understand that it will keep you from getting the deep restorative sleep, most likely. Antihistamines, which are the ingredient in most over-the-counter sleep aids, like Benadryl, make you drowsy, but contribute to poor quality sleep. So if you take antihistamines, just be aware or sleep aids, be aware that you may fall asleep faster and sleep for longer, but it may not be quality sleep. Alcohol blocks REM sleep and can cause sleep apnea, so people who drink before bed may not get into deep sleep, because they keep waking themselves up as their respiration slows down and they start snoring harder. Alcohol, decongestants, antihistamines, caffeine, and nicotine are all things that if you can avoid them, it would be ideal, or minimize them so they can get out of your system and you can get a good night's sleep. Like I said, eating a high protein dinner, high in tryptophan in the body is really good, chicken is good, beef is good, eggs are okay. You can Google food sources of tryptophan and find something that works for you. Not everybody eats meat or animal-based products. Make sure you're getting enough selenium, vitamin D, calcium, vitamin A, magnesium, and zinc. All of these are needed to help you sleep, and you're probably going, I have no clue where those come from if I'm getting enough. Most of us don't. Thankfully there are apps that you can get on your mobile device, and most all of them will calculate your nutrient intake, so you can figure out if you keep a diary on your mobile device. You can figure out if you're getting enough of these vitamins and minerals, or if you need a multivitamin. Again, before taking any supplements, talk to your doctor. So sleep hygiene. Create a wind-down ritual, just like we did with kids when they were infants.  We'd read them a story, kiss them on the cheek, and put them to bed, and that cued their body in. As soon as they started eating dinner, the body's like, okay, wait a minute, I know, in an hour and a half or so, it's going to be time for sleep. So then it starts the process of making the melatonin available. So do that for yourself. Create two or three things that you do every night before bed. You know, work backwards from what you already do. You already brush your teeth. So all right, you brush your teeth before bed. What do you do before that? And so your body has a clue. For me, I play checkers on my mobile device. And anytime I start playing checkers now, my body seems to think it's time to go to bed, and I start getting sleepy. So if your body starts to associate a certain activity with almost time for bed, it will help you calm down. Reduce or eliminate exposure to blue lights one hour before bed. Now blue lights are things that come from computer monitors, TVs, cell phones, and some of your not soft white bulbs. So try to minimize how much light's coming in. Your circadian rhythms are partly set by light levels. So when the sun goes down, you're supposed to go to sleep. But if you have it right as noon time in your bedroom, your brain's not going to get the clue. You can get different filters, different apps that'll put a filter over your mobile devices, whether it's a tablet or a phone or whatever. So the blue lights are not available and it has sort of a red hue to it, which is supposed to help you not jolt out of that wind down routine. Try to go to bed at roughly the same time each night. Now that doesn't mean you have to do it every single night. Just roughly the same time for me. It's within an hour window, most nights. Every once in a while I'll get wild and crazy and stay up until 10 o'clock. But whatever works for you, but if you can try to keep on a consistent schedule, you'll find that it helps you a lot. Eliminate as much light as possible in your room, including considering a sleep mask if you have lights coming in from street lights or whatever. Eliminate as much noise as possible. If you feel comfortable, you might try putting on earplugs or getting a white noise device. Don't exercise or take a hot bath within two hours of bed because your body actually needs to be cool to go to sleep. Keeping the room cool 72 degrees is ideal is great, but it's really expensive most of the year. Regardless, try to keep yourself as cool as possible with having a fan on. Consider getting a cooling pillow and a mattress topper that doesn't hold heat. A lot of the old fashioned foam mattress toppers that are supposed to conform to your body are great, but they conform to your body by virtue of as they heat up, they become more malleable and they hold that heat. Some of the new generation mattress toppers don't do that, so you want to look for a mattress topper that's going to help you maintain your body temperature. Try to avoid anything that might get you upset at night, such as social media or the news or whatever it is that tends to get you all fired up because as soon as you kick off that stress response, then you're going to kind of have to start all over again at the beginning of the wind down. Reducer, eliminate caffeine at least six, preferably 12 hours before bed. Drink the majority of fluids during the day. I am guilty of waiting till dinner time and drinking too much many times and then you have to get up in the middle of the night to go to the bathroom. If you have to get up in the middle of the night to go to the bathroom, try to have a flashlight that has a red light on it or have a night light in the bathroom that has a red light on it so you don't trick your body into thinking it's almost time to get up. Keep an air purifier in the room if you have allergies, keep animals off the bed. This is one of the things I'm guilty of. You may not be able to do all these things, but if you start improving your sleep hygiene in two or three different areas, you're probably going to see an improvement in your sleep.   Keep a red light flashlight and a notepad by your bed to write down anything that pops into your head that you, quote, need to remember. I've had nights that I've mulled over. I need to remember to get X and so at the grocery, and it's kept me from going to sleep as easily because I keep getting stressed out that I'm going to forget that I need to remember to get that. So just keep a tablet by your bed, a notepad and write down things as you think of them so you can rest comfortably and know that they're going to be there for you in the morning. And finally, weighted blankets can help some people get to sleep easier because they give you a feeling of being secured. You can order them offline, you can make your own, you can go on YouTube and find videos on how to make your own weighted blankets for cheaper than you can buy them. If this is something that might be helpful to you, if you like to be under something heavy when you sleep, then that would be something to consider. Keep daytime naps to under 45 minutes. Once you pass that 45 minute mark, then you start confusing your brain about whether this is a nap or it's supposed to go into deep sleep. Consider diffusing essential oils that can help you relax. Lavender, chamomile, patchouli and catnip are all known to have sedative properties to them and calming properties to them. And select the right pillow. If your neck is all out of whack, if it's not an ergonomic situation, you may have difficulty getting to sleep and you may wake up the next morning with a kink in your neck. So getting a good pillow is really worth the investment. Insufficient quality sleep contributes to fatigue, difficulty concentrating, reduced reaction time and apathy. During deep sleep is when researchers think the brain kind of resets and rebalances. And over time sleep deprivation can cause changes in neurotransmitter levels and immune system functioning. Most people could benefit from auditing their sleep quality and doing things as simple as reducing light and noises, reducing the number of trips to the bathroom, reducing wakeups because of allergies, coughing or being too hot, and cutting back on their caffeine six, preferably 12 hours before bed. If you're only willing to change one thing this month to start being happier, more energetic and clear-headed, sleep is probably a great place to start. If you like this podcast, please subscribe on your favorite podcast app. You can join our Facebook group at docsnives.com slash Facebook. Subscribe to our YouTube channel at youtube.com slash all CEU's education or join our community and access additional resources at doc. Welcome to this week's book review. Each week I review a book that I believe would be helpful to the general public and or clinicians. I'm never paid to do the reviews. However, in some instances, I may receive a small commission if you purchase the item which helps defray the cost of our podcast and providing the free educational videos. The cost to you, however, remains the same. Today, we're going to be talking about self-esteem. And this is the fourth edition. I actually started out using this book when it was back in the third edition. And I absolutely love it. It's a very comprehensive guide to helping people improve their self-esteem. In the book, you're going to learn how to uncover and analyze negative self-statements. So talking about that internal critic, create new, more objective and positive self-statements. Let go of judgmental self-attacking thoughts and act in accordance with what matters to you most. So the book is 300 and some odd pages and it goes on for a while. But what it does is walk you through step by step understanding what self-esteem is because it's kind of one of those terms that we talk about, but we may not really know what it is. And why just telling yourself that you're okay may not work? Well, the key is you have to believe it. So he starts talking about why people don't believe they're okay even if they tell themselves they are. Then he introduces you to the pathological critic and encourages you to explore why you listen to it and learn how to disarm it. It moves on from that point to creating a balanced self-description, accepting weaknesses and embracing strengths, recognizing that we're not perfect, but nobody's perfect. So embracing yourself and cherishing yourself as you are. He moves on from there to identifying and combating cognitive distortions that keep you feeling bad because a lot of times when we start looking at our weaknesses, we magnify them. And when we look at our strengths, we minimize them. And that keeps us feeling bad about ourselves. So McKay goes into great detail about how to combat some of those cognitive errors or thinking errors. He encourages you to explore your thoughts about compassion and developing self-compassion. One of the reasons that many people have difficulty developing self-esteem and being kind to themselves is because somewhere along the way they were taught that having compassion for themselves or being kind can be seen as weakness or failure or something negative. So he really challenges you to look at your beliefs about self-compassion and self-acceptance. He explores how to reframe mistakes and accept fallibility. Because again, even though we know we've got strengths and weaknesses, we're also going to make mistakes. So accepting those and learning from them and not turning them into something huge but accepting that everybody makes mistakes. Then he talks about what to do with the constructive and destructive criticism that you receive sometimes on a daily basis to help you respond in a meaningful way. And basically it boils down to a bunch of techniques that will help you learn how to take what's useful and leave the rest, accepting that sometimes criticism is not as much about you as about that other person's stuff. And then he ends with helping you clarify what you want and need and using your skills to get those needs met more effectively. So creating win-win situations and learning how to assertively communicate with the people that are important to you. This book is written in really plain language with practical tools which is one thing that I love about it. You can read three or four pages, get something really useful and insightful and work with that for a while, then come back the next day or the next week and read another three or four pages. So it can be really useful for self-help or as a tool to guide individual or group therapy sessions. Visual learning clients like to have it as a reference to review markup and bring for discussion in the next session. So a lot of times when I do self-esteem groups, I'll have each client get a copy of this book. We go through it page by page, activity by activity. We do them sometimes in class, sometimes as homework. And then we talk about what they learned and then they can go back and review over the week between sessions what we talked about and get those visual cues that help them remember the important skills. I hope you've enjoyed this book review and please feel free to log on. It's self-esteem by Matthew McKay. It does have a Google preview so you can read a little bit more about what's in it and see for yourself and see if it resonates with you as a good tool for self-improvement. Thanks for tuning into happiness isn't brain surgery with Doc Snipes. Our mission is to make practical tools for living the happiest life affordable and accessible to everyone. We record the podcast during a Facebook live broadcast each week. Join us free at docsnipes.com slash Facebook or subscribe to the podcast on your favorite podcast player. And remember DocSnipes.com has even more resources, members only videos, handouts, and workbooks to help you apply what you learn. If you like this podcast and want to support the work we are doing for as little as $3.99 per month, you can become a supporter at docsnipes.com slash join. Again, thank you for joining us and let us know how we can help you.


Here's the wildfire update for Tuesday, August 14th, 2018. Over 12,500 firefighters remain on the lines of 11 large wildfires across California. Here's the latest on six of the largest fires currently burning. Firefighters made great progress on the Holy Fire in Orange County overnight, still just under 23,000 acres but containment jumped overnight by a third, now right at 60%. There is also great news to report on the Ferguson fire in Mariposa County. So much progress has been made on containing this nearly 100,000 acre fire that Yosemite National Park has reopened Yosemite Valley to visitors as of 9 a.m. Tuesday. Visitors returning to the valley may continue to see visible smoke and or urge to be mindful of firefighters, equipment and road construction delays. The Donnell fire in Tuolumne County grew by another 6,000 acres overnight, now just shy of 30,000 acres, containment holding around 25%. Monday night was another tragic night on the Mendocino Complex fire, the largest wildfire in California history. A firefighter from Utah was injured while working on an active portion of the ranch fire within the Mendocino complex. He was airlifted to a hospital where he died. There are currently three separate fires burning in Shasta County, the largest, the Car Fire, grew by another 5,000 acres overnight but containment also grew, now up to 65%. And the Hat Fire also in Shasta County held steady overnight, still just under 2,000 acres containment at 75%. For the latest on any of the wildfires currently burning in California, you can go to the Incident page on the Cal Fire website, that's fire.ca.gov. And if you need help, assistance or information on recovery from the wildfires, you can go to wildfirerecovery.org.