analysis. I'm going to talk about what is a specific data request. I'm going to talk about data cleaning again a little bit. I'm going to talk about preparing the datasets and of course documentation. So what is a research data request? Typically an investigator, a clinician, a scientist will come to the group, the statisticians and the data managers with a question. They have a question they want to answer, they have a hypothesis about what the answer will be and then they have aims for a specific project. And typically what they do is they write up a concept proposal which is a detailed plan for how the research project will proceed and that typically includes the aims of the study. So what are their goals? What are they going to try and achieve with this study? What are the specific associated hypotheses of the study? So like you might have multiple aims and have you might have multiple hypotheses what you think is going to happen. So this is really helpful because the statisticians know all about testing hypotheses given the data. The concept proposal will typically include a statistical analysis plan. The investigator or the researcher is not always the one that writes the analysis plan themselves. They do so in conjunction with the biostatisticians. Typically there's a detailed description of the cohort and we've talked about what a cohort is. It's defining the population or the study population that you want to analyze. So you want to include in your data set. And then also there are on a detailed concept proposal there's there's usually a specific list of variables needed and a description of the data needed to conduct this study. So let me show you a quick a concept proposal. So you can see that it can be pretty detailed. We have a description of the study that the the investigator wants to perform. We also have a list of statistical methods so a short analysis plan of how they're going to analyze the data. And then below is a list of the variables the specific variables that they need. The dependent variables and the independent variables. And just some other administrative information about contacts and so forth. So this is like our guidebook when the when the data managers start to put together an analysis data set. We use this to figure out you know what patients should be included. What variables should be included. What time period they're looking at. What what new variables we're going to need to to derive. So in this I mean in this example you can see that these are some of the things that we've seen in the the demonstration data set. I mean age and gender and CD4 count. The CD4 plus T cell count is the same as our CD4 count. The who stage which we've we've worked with a little bit. Opportunistic infections and these are like the diseases that patients get from HIV. And then a few other miscellaneous variables. And then the the dependent variable here is the time to treatment initiation. So this is looking at this particular concept proposal is looking at the characteristics of patients prior to initiating a ART. So the the changing characteristics of changing over time the characteristics of HIV infected patients who are getting ready to start antiretroviral therapy in East Africa. So this is an actual concept proposal that we've we've put together the data set for these these results have been presented at one of the international AIDS meetings. And we're in the process of writing up the manuscript. So that's our concept proposal. That's that's how things begin preferably. So how do we go about fulfilling a data request that's embedded in a in a concept proposal. I mean the first thing you want to do is to identify and resolve any questions regarding the requirements. So you you have to read through the proposal and understand exactly what the investment wants and what the statisticians are going to do with the data and make sure that you have that you have a good sense of what is required of the analysis data set. You want to determine what the data sources are. I mean sometimes data is not coming just from one source. You may be the clinician or the investigator may come with some information from another from the pharmacy or from another sub study or something and you have to be able to merge that data with that to determine what your all your data sources are and what variables are going to be needed. What those derived variables are and how you're going to how you're going to have a good idea about what those visits and applications are that it will be included these data sets and or properly prepare them for analysis. You'll need to prepare the actual data and how that's done. And then always you need to document. Okay so let's back up just a little bit. It's rare that it sass is not a very good tool for inputting data for actually keying it for a data inter clerk to key in data. It does have capabilities you can set up some simple interfaces but it's not as good as OpenMRS for sure or other SQL packages or even access. So you typically data are not collected in sass. So if at all possible you if you have access to the raw data you might want to do some processing and cleaning there before you bring it over into sass. So when you when you get ready to start data cleaning in the raw data set the first thing you want to do is back up the original data files. You don't want us you know you know you're not going to do this. There may be little studies going on collecting things and access databases around for little sub studies. So in those situations when you're going to when you're going to extract somebody's data you want to make sure that you back up that original data files before you make any changes whatsoever. You want to go through and look for blank records. A lot of times folks that don't have a lot of experience with programming if they set up an access database there'll be a lot of blank records in there you know entries with no with actual not any actual data or real data or records in there just that were were inserted for testing. So you want to eliminate those if possible. You want to locate any duplicate records sometimes I mean in access you can well if you set it up right you can't have to that there is that uniquely distinguishes every record in the table from every other record in the table. And so typically you might have identified that in Excel are great for this kind of thing because you especially if you have a small database you know with not too many observations you can just click on each column and sort that column. I mean you want to sort and you find somebody in there who has an adult by each column and it's great for looking at dates when you if you know when the study started and when it ended if you sort by date and you see future dates or you see dates in 1908 you know I mean you know that there's been entry errors there and you can fix them right there on the spot before you bring the data back. And then for categorical variables a lot of inexperienced developers will allow entry of upper and lower case coding for specific variables they may also allow they may have FM 1 and 2 you know so they start out for a field like gender they start out using F&M and then they switch to one and two midway through I mean so you can if you sort the data set if you sorted that column you could easily identify where the issues are and where the problems are. And so again you can fix those on site and it will actually I mean this kind of raw data cleaning is actually helpful to the person who's maintaining the database you know because it sort of teaches them how to to be more consistent in the future and and it also cleans up their data their raw data for them so that if they go to run generate some some frequencies on gender or something they won't get for they won't get for values they'll only get the two F&M because uppercase F and lowercase F are not the same thing in SAS. SAS understands those as two separate values not true with variable names but true with values and then also when you're sorting those columns you can easily review the missing data so if if you have you know if you sort the gender column and you see that there's five or six people missing gender I mean you can say do you know what should we can we enter this do you have this and they can immediately go yeah nobody should be missing gender we know what what gender is on everybody so you can easily go and grab you know you can easily go and grab those data and then you can just say ask for this variable the the other advantage of doing data cleaning like this in in the raw data set is that there are occasionally problems with transferring data to SAS depending on the format especially things with Excel spreadsheets there's a lot of quirky things happening with Excel spreadsheets that don't always come out well in SAS so if you can look at the data in its original format you can you can confirm that that those that everything's transferring to SAS correctly so even the an ASCII file I like to have a separate SAS program that just does the conversion that takes the is because you often make that that you might not want to be reading to SAS data sets and I think Eva talked about this last week for converting an access or an Excel data set you can use tables there are limits on how many variables or how many observations can be in an access table so make things a little more modular and they break things up in smaller pieces and usually those pieces can be combined recombined in SAS for the analysis for when creating an analysis data set so you need to merge or append or concatenate these tables as necessary you need to double-check that the merging process worked properly by looking at the number of variables and the number of observations here I'm talking about the number of observations look at the you know what the number of observations were in each of the data sets and we'll look at some examples and you also need to understand how the number of records is dependent on the overlap among the among the data set so you might have some patients in this data set that aren't in here which will create you know a few additional records in the final merge data set so you need to understand what the relationship is so let's take a look at an example here let's say you have oh wow okay so this is actually your the I want to I want to I want to I want to keep the ID the visit date age weight height BMI and CD4 from this longitudinal data set and let's assume that okay so in this particular example so it doesn't match exactly with our demonstration set there was about 933,000 observations in the original visit data set so in this data step where I'm keeping only the patients only page only the records for patient ID 1 2 3 4 and 5 I end up with 71 observations in this subset of data set so so so that means that there are multiple observations where I data set because I'm including patient 4 and 5 from both of these data sets okay is everybody kind of get a picture in their head of what what's going on so far so when I do that I'm still I'm still extracting the data from the same demonstration so the the original data sets has 23,000 observations new data set okay now what happens when I let's let's let's sort of count them let's list them okay so will patient ID be in there okay appointment date that's to age 3 weight 4 height 5 BMI 6 CD4 7 patient ID do I count that one again no because I already counted it right how about appointment date no already counted it okay so clinic hemoglobin and SAO 2 so there'll be 10 10 variables in that final data set can you tell how many observations will be in that final data set 117 how do you get that 46 plus 71 okay but I'm gonna merge these data sets right and the two data sets presumably have some observations in common for patients 4 and 5 cannot tell right you cannot tell how could you tell if there's if you knew something you could tell how many observations would be in the final data set and if the patient IDs were different what would be the final number right what there's the sum okay that's good there's something else if you know something else you will also would be able to tell how many were in the final data set for patient what right right but you wouldn't have to know all of them so she said the nut you if you knew the number of observations per patient ID then you could calculate the total but you wouldn't have to know all of them what how many would you have to know actually only six now for you wouldn't have to know four five because if you knew six then you could just subtract that number from here and or if you knew six all you do is add add that number let's say six had ten observations all you'd have to do is add that to this number right because there's complete overlap because we're not we're not restricting by any other variables I mean all all observations for number four and five are in this data set and and they're also all in this data set so you really would only have to know how many observations there are for number six to get the total and it could be zero right it could be that there are no observations for number six there so let's see if we were right about oh look at this no so so so when you do the merge you you read 71 observations from the first data set and 46 from the second and you end up with yay 10 variables like you said and 83 observations so based on that can you tell me how many observations ID six has 10 12 right we should test that see if you're right okay so I mean this is the kind of I mean we're really breaking this down but this is the kind of sleuthing you have to do to confirm that your merge works correctly and this is a very simple one right there's no conditional statements there's no you know sub-setting out other observations and so forth I mean this is this is really important to get to get these merges working right and not overriding data from one data set to another so when you're doing this you want to confirm that the total number of variables in the merge data set is correct like we just example and there's actually a specific formula that you can use based on the number of data sets being merged the number of key fields and the number of fields in each data set so in this in this previous in a previous example we had seven fields in the first data set we had seven we had five fields in the second data set I mean yeah five fields and we had two key fields patient ID and appointment date so that's this two here and you multiply that by the number of the total number of data sets being merged minus one actually this might not be right look at this again but it works when you have only two data sets I think this what what you subtract by maybe dependent on the number of key fields so we can look at that later though if the number of variables is less than what you expect then and you know that you have other fields that are common to both data sets and this should be strictly avoided you do not want to to get into a situation where you're overriding one one field with another from a different data set so if I had kept if I had also kept age here then this number would be six but this number would still be ten and you and you want to avoid that situation where you're you're merging two data sets that have fields in common that are not in the by statement okay the only variables in common that you want between the data sets that you're merging are what's in the by statement because sass will give you does kind of some strange things depending on how how the records align so when you're running sass program whether you're creating a data set or for research or or doing cleanup queries or whatever you're doing you always want to review the log before you go on and look at the output or before you send out the data set and you guys actually I've been impressed you've been pretty good about reviewing the log making sure that that things ran properly and some of the unfortunately they all popped up at once some of the certainly the error message they come error messages they come up in red and it's pretty obvious that you've got a problem you can't really continue without it there are some warning messages I mean that that allow you to continue the warning and note messages allow you to continue processing but you're not gonna you may not be getting what you expect you're getting in your data sets and this is just a list of five of the warning messages or notes that you get that really should be attended to before continuing to process your program some of them may not actually be causing problems with your code but they should be corrected because it means there's an error in your code the error may not be affecting the actual data sets that you're creating but it is indeed still a problem and particularly the first one is a serious problem and that's the note where you get the merge statement has more than one data set with the repeats of the vibe by values and what this means is is that you that there are multiple observations per by category in both of the data sets so it doesn't know how to merge them so if you if you're by someone only contains patient ID and you've got patient ID listed three times in one data set and twice in another it doesn't know how to link those three observations with these two from this data set okay so there's that leads to confusion it typically when you get this message what what typically is the problem is that you haven't enough variables in your by statement that you haven't that you don't realize that there's multiple observations per patient in this data set and multiple observations per patient in this case and whenever that happens you you cannot merge by a single variable you cannot merge by just patient ID you have to have some other variable to merge by visit date CD you know whatever a visit number whatever else is available but the point is that that you that you cannot merge these two because there are multiple records in each of those data sets for this for variables named in the by statement and if when you run the merge you get strange results and you should definitely not continue on to the next step or next procedure or anything until you resolve this issue and that air goes away then there are two other notes that are very similar where a variable is on a variable with a specific name it can be any name here they'll just list the variable that's uninitialized or has never been referenced and these indicate that the variable is probably not properly defined and many times it's just a typo where you've misspelled one of your variable names and it's kind of handy because SAS picks up on that the next one where care where you get a note where character values have been converted to numeric values and there's a bunch of other blah blah blah that this indicates that SAS has automatically converted a character variable to numeric because this can lead to unexpected results in SAS it's best to do this conversion manually with an input function which converts numeric a numeric variable to I'm sorry input I always get confused input is character to numeric and put statement is is numeric to character and then the last one I don't see as often where you have multiple links specified for your bi variable so if you if you get results from the lab database and patient ID is in there and you also get results from from the clinical database and patient ID is in there if these two if the people designing those datasets didn't communicate before the design was set up the length of this field may be 10 characters long and the list of the in the length of this one may be 15 so you want to be careful about trying to merge those two until you resolve that that length problem okay when whenever I run a SAS program and I typically rerun SAS programs before I get it right sometimes I recreate the datasets you know a year down the road or what have you whenever I run a SAS program I just go straight to the log and I search for these I search for these keywords so I search for you know merge statement I search for uninitialized I search for referenced I search for character values and I search for this multiple links you don't have to type in the entire thing but you know just a snippet from this from from each of these commands will give you will take you to those problems and sometimes I search for warning messages as well I'm not I always search for more I always search for error messages first then warning messages and cetera because because these don't show up in a different color they the warning messages show up in green I think the error messages in red but but the notes just show up in in in black so if you're like I mean you know I just it's harder to run through the program and easily spot them so I just search for those automatically okay so what else do you have to do when when you're setting up a permanent data set for research purposes I mean one of the things is you you may need to recode the missing values that were used in the raw data files as I mentioned earlier sometimes people use nines fill a field with nines to indicate missing these should should be converted to a blank or a dot and sass before proceeding unless there's a special value unless they they have meaning an additional meaning unless that nine means unknown or that nine means that that the patient refused to answer but if it's just general missing then it should be converted to the missing values a lot of times we have to calculate summary scores and and a simple example is the body mass index which is based on height and weight but there's also a lot of studies that that that do that use behavioral questionnaires that ask about pain and depression and you know they may have a series of 30 questions that ask about depression and there are summary scores that can be calculated from those questions sometimes there are multiple summary scores and we don't typically in our area we don't typically do all these calculations in the data entry system we typically do them at the end in sass so there's also the steps you need to add for for coding those summary scores audit three is isn't the summary of the alcohol questions I think there's four alcohol questions and there's a couple of audit scores you want to look at the difference between the dates such as time from enrollment to ART initiation this is this is also a cleanup query but the analysis they need to have the durations in the system so this is a good place to put to calculate those variables and and storm in the in the data set any variables that you calculate any variables that you derive any new variables that you create you must label and you must document in external just word I mean just documentation as well and then the final thing you want to do is attach formats to specific variables so like marital status or civil status we had that the values are actually one to seven the entered values are one to seven but the but they but one two three four doesn't mean anything to the statistician so that's why we we create formats for those values and attach those to to the data to those associated variable so once you have your permanent data sets created from the raw data sets you can have a separate program that does the data cleaning and generates cleanup queries so in the cleaning data set I like to have generate frequencies and means and sometimes univariate statistics on all the on all the variables I just look at every single variable in the in the data set that I got to make sure that there are no allies to make sure that gender has doesn't have four levels to check for and to check for invalid data some you can also plot the data often all plot age against weight especially if I have if I'm looking at kids you know to see how much how much work is going to be needed in cleaning up those those weights for numeric and date fields you can look at the minimums and the maximums to verify the values are in the expected ranges you can do this if you have if it's if you have few observations or there are limited number of responses you can use proc freak to look at the to look at everything look at the minimum the maximum but if you have you know if you're looking at weight and you've got 10,000 observations in the data set you probably don't want to run a proc frequency on those so that's where you can use the proc univariate and the proc means that will show you how many are at the lower end and how many are at the upper end and and you can figure out you can generate some cleanup queries based on what you see in those those descriptive statistics you want to locate of course locate duplicate records and fix those if you find observe multiple observations per patient in what's supposed to be a cross-sectional data set then you know you've got to eliminate those records I mean this is redundant a lot of this is redundant to what you did or could do in the raw data set but all of it has to be repeated here and SAS and then you want to compare fields where appropriate you know looking at at date of birth and age and visit date make sure that confirm that the date of the initial visit what they have defined as the initial visit actually proceeds the date of the follow-up visits okay then you want to kind of pick out the important fields such as summary scores and verify their values in more detail you know I mean there's there's going to be a lot of data collected for a research study and some of it is not important to the analysis how many phone calls how many times a person called in to ask about medication that I mean those kinds of things are may not probably may not be useful for the analysis so you don't need to spend your time on those variables so that's why it's important that's why it's important to understand the concept proposal so you know what the specific aims are for a study and you know what variables are going to be needed for the analysis to address those questions in the in the aims so typically you want to merge all the longitudinal data sets together you may not want to you may not want to store that as such in the final data set that you that you prepare for analysis but you certainly want to do that as part of the cleaning as part of the cleaning so you can verify there are no inconsistencies and how variables are formatted so the variables that are supposed to be in all data sets like an identifier and a visit number and a visit date you want to make you want to merge those data sets to confirm that that the format of those variables is all the same and the length of those variables so that what if the statistician needs to merge them that there won't be any questions about inconsistencies and then you always want to merge the cross-sectional data set the demographic data set with the longitudinal data set with the visits data set to the to identify subjects who are in one data set but not the other and you may also want to get the data birth into the longitudinal data set so that you can calculate age and other such variables but this is only a temporary merge okay you don't want to send the data set to the statisticians with gender in every field of the longitudinal data set okay and every in every record of the longitudinal data set okay this is a temporary merge where you're just merging to make sure that there's agreement between these two data sets maybe to do some calculations but when you actually send the data set out they need to be separate files and I'll explain more about why that is and okay so when you're writing SAS programs it's really important to save all of your logs and your outputs especially the the final version that you use to create the analysis data sets and even more especially if those data sets are going to be used for publication however if I give a data set to any statistician or investigator I have the log in the listing and the SAS program frozen and the data sets that were given I freeze those you know put those into an archived folder and don't ever touch them again I copy them into that folder so that if I'm back to me six months later and say you know I want to run some additional analysis on that data set you created can you add in these next you know can you do this then I can go back to that exact data set even if the other data set even if data is continuing to be collected and the other data set is growing I still have that data set as it was at that time okay and believe me this happens I mean even after publication they'll come back and want some additional information about that data set and if you haven't saved the log and you haven't saved the program then you cannot you cannot reproduce the results and you cannot answer questions about how that data set was created when you're when you're naming SAS programs we talked about this earlier it's it's important to name to give them a name that means something not just to just call them you know frequencies or study or what have you and it's also important to name the log and listing files with the same prefix as as the original SAS program so in this example study X is not study X it's going to be like bone studies project or it's going to be something like you know it'll be specific it'll be specific to a particular study but then the log and listing should be should be named the same thing with just the different extension log and LST and SAS automatically attaches the log the log and the LST when you save it through SAS only the program that generates the permanent data set should override it only the program that generates a permanent SAS data set should override it should ever override it you should not have two SAS programs on your hard drive that that modify the same permanent SAS data set okay this because this leads to a lot of confusion and a lot of problems if you have one program that creates a permanent SAS data set and then another program that modifies that SAS data set and overwrites it and you come back to redo to recreate this data set six months later and you forget to run that first program or you run those programs in opposite order then you're not going to get the right results in that set in that data set okay the other big disadvantage is the other advantage of not letting more than one SAS program override a specific data set is that you always have the date and time associated with the creation of that data set you always have a data set so if you have a data set that lots of other programmers are using lots of other data managers are using to generate other data sets and you let them modify this permanent data set then you're never going to know no one's going to know what if they have the most recent version you know if they just look at the date and time because of other programs even a proc sort will change the date and time associated with that data set so if you sort a data set and override it then all the other programmers might think oh I have the latest data set because it's from last week but that may not be the latest data set that's that's just when it will that particular file was updated but it may not be the actual data set that they want to be working with okay I even take it one step further and say even within one SAS program there shouldn't be more than one data step or procedure that that modifies that data set so you don't want to you don't want to have two data steps that that overwrite the same the create the same SAS data set I mean there's no point in that I mean if you just can eliminate that first data step or create a temporary data set for that first data set and then only then only output the permanent data set and in that final data step but then it gets a little confusing well what if you want that data set sorted in a certain way well then you have to sort it prior to creating that that permanent data set okay does that make sense okay so then you never you never want to override a permanent data set like I said even with a proc sort from any other program so only one program creates the data set and no other programs overwrite that permanent data set so let's go talk a little bit about documentation as we said earlier you want to internally document your SAS programs at a minimum you need to include a file name location purpose author date and revisions we also had when we're looking at the program design language we also had a header that included inputs and outputs as I said to include the names of any permanent SAS data sets created by this program that would be an output and then all your your SAS pronounce if you create a rich text file or just an ASCII file you want to put meaningful titles on that output meaningful titles that include the name of the project that at minimum include the name of the project but you may also want to put something else the treatment interruption analysis data set you know like the second generation or this is something about what the subset is or when it was run but at least at minimum the name of the study or the project would be very helpful and then you can also use the footnote option and I usually use the footnote option to display the name of the SAS program that created that output so this you can have so in SAS you can have an option statement with a footnote and then you just put whatever you want in quotes here in the in the footnote option and it and it puts that on every page of your output and I can tell you this more than anything has come in handy because investigators have come back to me months after I've generated an analysis or or a list for them they come back and say you know that list you'd put together for me can you rerun that can you redo that you know I'm like oh my god you know I've got 25 30 projects going at once and then I'll say hey is there is there a note at the bottom of that page that that that tells what program generate and then I go yeah and it says this you know and I can just go right to the folder right to the program where that that generated that particular output so it's it's very handy it's a good habit to get into okay like as you saw with the demonstration data set when we have formats attached to specific variables we need to make sure that we include those format value statements somewhere in the documentation you always want to generate format keys we talked about that make those available and you want to provide a detailed description of any variables included in the data set that are not found on the form key these would be all your derived variables your summary scores your duration variables any you know additional variables that you create you want to make sure that they're in the documentation okay when I review other programmers were the first thing I do is I look I look through the list of variables in it and they if it's not on the key form then I send them and it's not in the documentation I say I have no idea what this variable is even if I can figure out what it is you know I say you got to have you got to provide more detailed description and I send it back okay let's talk a little bit about summary scores and I'm I tried to find an example for you this morning without the internet I could not pull it up but there are often cases where you have questionnaires as I mentioned earlier like depression that measure depression or pain or other behavioral problems and there'll be a series of 10 20 30 sometimes 60 questions and typically these questions are interrelated and and can result in in multiple summary scores and typically a total score for that questionnaire as well and and these are very handy because the a lot of these questionnaires have been validated and are accepted in the research and so you go to the literature and you can compare you can actually compare your depression scores your mean and median depression scores for your population with with other populations that are published in the in the literature you know and it's actually very handy and as long as everyone is using the same standardized instrument and scoring the instrument the same way then these are comparable results so but scoring instruments can be a little bit complicated so you need to have you need to provide a detailed description of how you generated those summary scores and by doing so you're actually going to come up with the algorithms yourself and the end and the procedures for how you're going to actually code it in SAS so this detailed summary should should include the list of variables that were included in the summary scores with variables were included in which summary scores and then which variables were recoded and how they were recoded quite often the the the items will be will will have opposite meanings like if you strongly agree to one thing then you are more satisfied but if on the next question down you strongly agree that indicates that you're less satisfied with maybe like the service that you're receiving from the clinician or something so in order to you can't just add those two scores together you know and maybe strongly agree as coded as a five because that will just they'll just negate one another that they don't have the same meaning so you actually have to flip the the the responses for one of the items so that the higher score still means that you're satisfied with the with the service that you're receiving does that make sense so that's what I'm getting at here where you might need to recode you might need to flip the scoring of specific variables so that they all have the same meaning so that they're consistent does that make sense okay it's not possible to calculate summary scores if you have too much missing data and a rule of thumb is typically that if two-thirds of the data are not missing then you can calculate the summary score some people say that they need to have three-quarters of the data non-missing before they they feel comfortable calculating the summary score it and it also depends on the number of items I mean if you've got 50 items going into a summary score you know you can live with you may be able to live with a little more higher percentage of missing if you have three items going into a summary score then you know you may not be half you may insist that at least two of those three are there before you're calculating before you'll let the score be calculated in this documentation you also need to explain how the missing values are addressed so typically when calculating a total or some score the mean for the other non-missing values can be calculated and imputed for that missing data okay if the summary score is a mean itself like you know you want to calculate the mean score for a set of five questions or something then the then the missing data can be ignored right so if it's a total let me just go through that again if it's a total score a sum score you know like you add all the items together and you're missing one of the items then you can't really calculate you can't ignore the fact that one's missing because that person would have you know would have would naturally have a lower total or the potential of a lower total than all the other patients who have complete data so what you can do is you can take the mean of the other four items that are not missing and add that to the other four items to come up with a with a total so that sort of adjusts adjusts for the meat adjusts for the missing when you're talking about a total score but if the total if the summary score is a mean itself then calculating the mean of the missing value doesn't add anything to the mean because the four items the four non missing items would have the same mean as if you imputed the mean for the fifth item and then calculate the mean so it doesn't it doesn't matter okay but it's very important to understand that in both of these cases whether you're calculating the total or the mean that you cannot ignore the the minimum requirement for non missing data okay so if you have five items that go into a score and you say I have to have two-thirds non missing you know so you have to have at least three of the values answered you can't calculate the score if four of them are missing or even if three of them are missing you can't whether it's a mean or a total you cannot you cannot successfully calculate that score because too many too much of the data are missing and you can't really say you can't really come up with a valid score for that patient okay so even if the score is a mean and you and you can ignore the missing or if you can impute the the mean for the missing data in the case where the total is a sum I mean the summary score is a sum you still have to pay attention to this this minimum non missing requirement okay and then the final thing that you want to have in your documentation is is what's the meaning of the score you know and how is it scaled so you want to indicate and you want to include the the possible range how high the score can be and how how the score how a high score differs from a low score so you want to have something in your document documentation to say something like a higher score indicates more depression because a lot of times the the investing these certainly the bio statisticians and possibly the investigators don't know how the the the instrument is is is scored so you make sure you want to make sure you say what what a score means and what the scale is okay when you when you when you create an analysis data set and you're going to distribute it to the statisticians and to the investigators you want to make sure you include some what we call a data set cover sheet it doesn't have to be exactly like this but it but it includes the additional information about the data sets that you're distributing so what the project name is a lot of this stuff you can get from the original concept proposal who the principal investigator is again that's typically on the concept proposal when the when the original request was made and you can even attach a copy of the concept proposal and then the data sets created the names of the SAS data sets that you created the the name and location of the SAS programs that were used to generate these data sets that's always nice to have especially if you have multiple data managers working with the same in the same project or you you're going to pass on some of your responsibilities to somebody else and they don't have to go searching for things all the information be right here when those data sets were created who created those data sets I mean some of this is redundant with what the stuff that you're putting in your SAS program but it's but but this is what's actually going to be submitted to other people okay other people other the statisticians and investigators are probably not going to see your SAS programs and then we also just list who the bio statisticians are so we know who the data went to and then a description of the cohort what subjects are included in this data set and then the description of the derived variable so this is your documentation this is where you would put how age at ARV start was calculated and what it means okay for example and then your SAS formats with the with the PROC format statement and then typically we would do preliminary statistics just descriptive statistics like frequencies and you know so we would pick out the most important variables and maybe include them in this document but that's more that's sort of optional here if you if you create additional versions of the data sets usually I provide an updated version of the cover sheet as well okay so just some general notes when you're using SAS if the study is longitudinal you need to provide at least two data sets one that contains the longitudinal data the data that's collected multiple times and one that contains just the demographics of the cross-sectional data the data that you only have one observation per person okay statisticians can they know how to merge data sets so if they need to get the data birth into that longitudinal data set they can merge those two data sets and make it happen you you don't ever want to put cross-sectional variables such as gender into the longitudinal data set I sometimes have this argument with statisticians because they want that data in there I said okay well if I put gender in the cross-sectional data in the longitudinal data set how you're going to calculate frequencies on gender oh well I'm going to have to manipulate the data set to take out to find just one observation per person and then calculate the the frequency on gender and I said or I can give you the cross-sectional data set and you can just create the product that you can generate automatically on that data set without any manipulation you can create frequencies on gender so you know there there are trade-offs but I think it's just cleaner and more concise if you have if you're not mixing longitudinal data with cross-sectional data and the way you ask yourself this is is this if you if you wondering whether this is a longitudinal variable cross-sectional variable you say well does this change over time if I were to have it in a longitudinal data set would it be different for every observation or could it be different for every observation and the answer to things like gender and date of birth is no okay so that's a cross-sectional variable and should be in a separate data set you want to be careful about formatting your dates I've made some mistakes where I didn't where I formatted dates with just a two-year extension for the years a two-digit year and didn't realize that there were some really there were some years in there that would like 1895 instead of 1995 so you need it's a good idea to always format your dates so that you can see all four digits and when I'm working in East Africa I almost always use date 9 and I apologize I did not use that here in this these presentations but I almost always use date 9 because then there's no confusion about if it's month or or day first and then you want to default to the numeric type numeric types are just much easier to work with and the statisticians prefer numeric types over care over text fields so when you're distributing a SAS data set if if possible you want to have another data manager review the data sets and the documentation before distributing they don't necessarily have to read through every line of your SAS code that created those data sets but the idea is that they should be able to go to your SAS data sets open the SAS data sets in SAS view or in SAS and look at the documentation and be able to understand what the data sets are for what all the variables mean maybe they also need the the form keys but they should be able it should make sense to them okay and if it doesn't make sense to them then you probably need to to do a little bit more work on documenting or there may be problems with actually how you generated the data sets are throughout the variable so it's really handy to have somebody else work look at those look at your data sets and documentation when you're giving when you're distributing data sets especially to statisticians you need to include the following you need to include the form keys the electronic data dictionaries form keys whichever you have you need to include the appropriate data sets and all these data sets should have the extension SAS 7BDAT you need to include the data cover sheet which we were just working on the latest data request form or the concept proposal it's always it it's almost never the same I mean the concept proposal is submitted there's discussions there's meetings they tweak the aims they may change the hypothesis a little bit so you want to make sure that the most recent the final version of that concept proposal is made available to the statisticians and the investigators and then any other documentation that you need that that further explains the the data set in most cases the following should not be distributed you do not want to distribute distribute any protected health information and these include things like the subject's name and address and phone numbers of security numbers national ID numbers the statisticians do not need this information to analyze the data I mean they may need GPS coordinates for some sub studies but you need to be real careful about distributing those and even things like date of birth may not be necessary what I usually do is I usually calculate age for every observation and calculate ages at specific points in time and I round that age to the nearest 10th so they don't a session doesn't really necessarily have to have the exact age and so they don't actually need the date of birth and especially here in East Africa where a lot of people don't know what their exact date of birth is anyway you can substitute age for date of birth this you don't need to give your SAS generation programs to the statisticians at least in my SAS programs there's a lot of PHI in there there's a lot of protected health information things where I'm listing out people's dates of birth and there and that maybe even their names so I can distinguish men from women etc so there's there's there's I don't offer I don't hardly ever make those data set those programs available to the statisticians or certainly not the investigators so let's just talk a couple minutes about file maintenance and archiving for your own records what you should keep is a copy of everything that you give to the biostatistician everything that you've ever given out should be stored I mean almost everything is electronic now should be stored and saved and preserved in an archived subfolder on your on your hard disk somewhere okay because they'll they'll lose things they'll come back and say can you resend that data set and if you've overwritten it then you have to say no I can't and we you may be in trouble if they can't if you can't reproduce the same data set that you had already given them you want to make sure that you archive a copy of all the logs and the SAS programs especially those that create permanent SAS data sets but I would also say anything that creates SAS data sets that you're passing on to somebody else you should save the log in the in the program and even the output file too there's it's a good idea to keep copies of the grant proposals meetings notes from meetings the scoring algorithms instructions any manuscripts a lot of times you'll get the scoring algorithms from published from the published literature and you should try and keep a copy of that in your in your folder that's associated with this particular study and it may be helpful to maintain a subdirectory that for for each specific study that that holds all of these data and in addition you may also want to have a subdirectory that mirrors the data entry system and then this applies mostly to the the simpler data entry systems like like access or if you if you've actually developed the access data entry system yourself then you should keep that that separate on your computer and so that you know what so that you have a copy of what the what the data inter clerks are seeing or what the data manager is actually using at the site and then for launch student studies in particular it's really important to archive data sets and SAS programs which were used for analysis and abstracts and papers because in typically in a longitudinal data set there'll be interim analyses so you'll actually create analysis data sets midway through the study and then another data set at the end of the study so you need to make sure that that you preserve each of those in separate files and then just a few notes about working with the working with investigators and bio statisticians you want to make sure that you are seen as a team member okay I mean you want to be involved in the decision-making process you want to contribute because if somebody goes and designs a data collection forum without having input from a status dish I mean from a data manager they're probably not going to do a very good job and it's going to be really hard to develop a good data capturing and electronic system for that data for that questionnaire so you want to kind of make sure that you you know that you kind of become part of the team and and that involves attending the study meetings documenting taking notes at all this at all the study meetings certainly you want to comment on any proposed study changes I don't think I've ever been involved in a research study where midway through the investigator says okay now I want to add this questionnaire or I want to stop doing this on these group of patients you know and you really have to be you really have to to discuss have many discussions about this before you make those kind of decisions because they can really affect the outcomes of the study and and it's good to get the statisticians involved in those discussions as well because they may have comments about how it's going to affect the analysis you want to try and understand the analysis plan I mean sometimes it can get very very very complicated but understanding how the statisticians are going to use the data and what variables are most important will really help you focus your attention to those in those areas and making sure that you've got the data properly formatted to address the questions I like to review the statistical reports before they actually go to the investigator so I work in the division of biostatistics and I work very closely with a lot of statisticians and we use a form a team on each project and I like to review that the analysis that they do mostly to make sure that I have conveyed all the information appropriately to them because when they start doing the analysis they don't have the knowledge of the of the raw data that I have they may not even they may not have been involved in developing the data collection form they may have never even met the investigator you know or know anything about how the study is conducted so it's really important that you know I do the best I can to convey all my knowledge to them but it's obvious when I'm looking at some of the analyses that they've overlooked something or they've misinterpreted one of my variables or something so it's a good idea to meet with them on a regular basis and review their reports and then I spend a lot of time looking at manuscripts and abstracts those people sort of have come to rely on me to go through with the detail you know with a detailed head and comparing every value is this the right number is this the right mean is this the right value you know pp value and so forth so but but also in addition you can contribute to the to the narrative and even to the to the results section and the description of the study and so forth so it's good to it's good to have the data managers input to make sure that and sometimes when I read manuscripts I realize that I didn't understand exactly I didn't understand things exactly the way they are and and I may need to adjust some of my some of the variables in my data sets so just remember that your your contribution is extremely important in these in these settings with with research data analysis that's all I have any questions thank you very much I'll be here all week


Present to you the President of the United States. Thank you very much. Thank you. Thank you. Thank you very much. I'm delighted to be here, and special greetings to Stephen Greenberg of the Young Leadership Cabinet, Mickey Barron of the Young Women's Leadership Cabinet. For almost 45 years, the United Jewish Appeal has served as the main fundraising organization of American Jews, and you certainly have proven that to me this morning with the figures that we've just heard. Through the agencies it funds, the UJA provides vital social and economic assistance, including resettlement, rehabilitation, and development programs for Jews in Israel in more than 30 other countries. And through its Young Leadership Cabinet and the UJA, it trains the hundreds of young men and women for service positions of responsibility around the world. In recognition of your historic task and your great humanitarian achievements, I certainly commend you. In your lives, you must overcome great challenges. I know you draw strength and inspiration from the well of a rich spiritual heritage, from the fundamental values of faith, family, work, neighborhood, and peace. Two centuries ago, those values led Americans to build democratic institutions and begin their constitution with those courageous and historic words, we the people. And today, our democratic institutions and ideals unite all Americans, regardless of color or creed. Yet, as we enjoy the freedom that America offers, we must remember that millions on Earth are denied a voice in government and must struggle for their rights. They live under brutal dictatorships or communist regimes that systematically suppress human rights. Under communism, Jews in particular suffer cruel persecution. Here in our own hemisphere, the communist Sandinista regime in Nicaragua has used threats and harassment to force virtually every Nicaraguan Jew to flee his country. In the Soviet Union, Jews are virtually forbidden to teach Hebrew to their children, are limited to a small number of synagogues, and cannot publish books of Hebrew liturgy. Immigration of Jews from the Soviet Union has been brought to a near standstill. Prominent Jews like Iosif Begin have been arraigned in mock trials and given harsh sentences. Hebrew scholars like Lev Furman have seen their teaching materials robbed in their homes ransacked, and Jewish dissidents like Anatoly Sharansky have been put in mental wards or thrown in jail. We must support Soviet Jews in their struggle for basic rights. And I urge all Americans to observe the International Day of Concern for Soviet Jews this Thursday, day after tomorrow, March 15. In this world where so many are hostile to democracy, how can Americans best preserve and promote the democratic ideals that we hold dear? Ideals which are the keys to the golden door of human progress. Here at home, I believe we can move forward together toward a genuine opportunity society by meeting two important challenges. First, we must teach tolerance and denounce racism, anti-Semitism, and all ethnic or religious bigotry wherever they exist as unacceptable evils. And down through our history, American Jews have been in the front lines in our nation's great struggles for equal rights. A century ago, the 14th Amendment proclaimed the full protection of the law for all. In the 50s and 60s, the struggle for civil rights stirred our nation's soul. Americans must continue that great tradition because even today, vestiges of racism and anti-Semitism remain. Synagogues are vandalized. Jews and others are harassed and mocked.  Well, let us reject prejudice, turn our backs on bigotry, and stand shoulder to shoulder for equal rights. Our second challenge is to promote economic growth. Throughout history, civil and economic rights have gone hand in hand. For century, rulers kept Jews down by limiting their occupational choices. In our own country, many blacks suffered from Jim Crow prejudice, denied all but the poorest of jobs. But in a strong and growing economy, all groups have the opportunity to advance through hard work, enterprise, and heart. Just three years ago, our nation was an economic disaster area. Double-digit inflation, record interest rates, huge tax increases, and too much regulation were destroying growth, drying up opportunities, and freezing those at the bottom of our society into a bleak existence of dependency. And that's why economic recovery without inflation was our top priority. We cut taxes, reduced the growth of the federal budget, eliminated useless regulations, and passed an historic reform called tax indexing. Indexing means government can never again profit from inflation at your expense. Today, America's economic engine is pulling this nation forward again. Inflation is down from 12% in 1980 to about 4%. The prime interest rate has fallen by almost half. New businesses are the biggest innovators and job producers and from January to November 1983, more than 548,000 companies incorporated. That's nearly half again the yearly rate during the 1970s. It means more jobs and opportunities, so it's no accident that unemployment is down to 7.7% the sharpest drop in more than 30 years. More Americans are holding jobs today than any other time in this nation's history. Despite all we've accomplished, we must go forward to new goals to keep the nightmare of inflation from ever coming back. We must enact constitutional budget reforms like the line item veto and the balanced budget amendment. To make taxes more simple and fair and to provide greater incentives to our people, we must press for tax simplification, a sweeping and comprehensive reform of the entire tax code. Could I interject something here that when we talk about simplification, it sounds awfully simple, but the other day I got a figure from the Treasury Department that astounded me. If you were a young lawyer and deciding maybe to get into the area of tax counseling and advice and so forth, do you know how many books of regulations you would have on yourselves just to help you with the income tax? Well, you've heard of the Harvard classics, five-foot shelf of books. You'd have to have a shelf of 31 feet of books just for that one subject. Well, as we move ahead, we're determined to leave no one behind. Under this administration, more funds go to needy Americans even after adjusting for inflation than ever before. And total spending on social programs has increased by $71 billion during these last three years. And while I'm on this subject, I wonder if you who are intensely committed to social justice and Jewish charity would join us in questioning the relationship between greater federal spending and a healthy, prosperous and growing country. During the 60s and 70s, the great society and other federal programs led to massive increases in social spending. Why then, at the same time, did the number of Americans below the poverty line stop shrinking? Why did we see a drop in the number of males in the workforce and a huge increase in births out of wedlock? I believe the answer lies in the firm difference between the New Deal and the great society. The New Deal gave cash to the poor, but the great society failed to target assistance truly needy and make government the instrument of vast transfer payments erecting huge bureaucracies to manage hundreds of social programs. The great society failed in two crucial aspects. It fostered dependence on government subsidies, and it made the transfer of money from Washington bureaucrats to those in need seem like a mission impossible. I was a New Deal Democrat, and I still believe today that there is only one compassionate, sensible, and effective policy for federal assistance. We must focus domestic spending on the poor and bypass the bureaucracies by giving assistance directly to those who need it. We must end dependency, eliminate quotas, and foster a vital innovative economy that rewards all Americans according to their talent and hard work. If we do, we can enhance our democratic ideals and can make America a genuine opportunity society. To promote our democratic ideals abroad, we must also meet great challenges, and I see three that are paramount. First, we must keep America strong. During the 70s, the United States made a conscious choice to restrict its military development fervently hoping the Soviets would respond in kind. Well, during those 10 years, our spending on defense dropped over 20% in real terms. We canceled major weapons programs, reduced our nuclear stockpile to its lowest level in 20 years, and slackened in the training of our armed forces. Between 1968 and 1978, we cut our Navy the fleet by more than half. But far from responding to our good intentions and to our restraint, the Soviets launched the most massive military buildup in world history. From 1974 to 1980, they outproduced us in practically every category of weapons, three times more tanks, twice as many tactical combat aircraft, five times more ICBMs, and 15 times more ballistic missile submarines. By 1980, total Soviet military investment was more than one and a half times ours. President Carter, Secretary of Defense Harold Brown, put it very well. He acknowledged a bitter lesson about Soviet practice in saying, when we build, they build. When we don't build, they build. Since taking office, our administration has made significant headway in rebuilding our defenses and making America more secure. Perhaps you remember the 29th Psalm in which King David said, the Lord will give strength to his people. The Lord will bless his people with peace. Well, today, America once again recognizes that peace and strength are inseparable. But we've only begun to repair past damage. Make no mistake, if we heed those who would cripple America's rebuilding program, we will undermine our own security and the security of our closest friends like Israel. And I am not prepared to let that happen. After two decades of military expansion by the Soviet Union and a decade of neglect by the United States, we're struggling not to regain the superiority we once enjoyed, but simply to restore the military equivalents we need to keep the peace. A second great challenge is to defend and promote human rights throughout the world. Alexander Herson, a great Russian writer, warned, to shrink from saying a word in defense of the oppressed is as bad as any crime. Well, we who are blessed by the fruits of liberty have a personal responsibility and a moral obligation to speak out in defense of our brothers and sisters. We must not and we will not remain silent. Our administration has repeatedly and vigorously protested the persecution of Jews and others in the Soviet Union and other communist nations. We're also using our influence with countries that receive American assistance to give human rights firm support. In El Salvador, we're insisting that the leaders take steps to end human rights abuse. And although El Salvador is far from perfect, we've seen marked progress. We in the United Nations, Iran's representative, once called Israel, quote, a cancerous growth. And Libya's representative has referred to the people of Israel as, quote, the most vile people upon Earth. Well, this so-called anti-Zionism is just another mask for vicious anti-Semitism, and that's something the United States will not tolerate. As I wrote last month to Stanley Blend, the president of the Jewish Federation of San Antonio, the lesson of history is overwhelmingly clear. Silence is never an acceptable response to anti-Semitism. UN Ambassador Jean Kirkpatrick is our leader on this, and let me assure you of one thing about Jean. She is a very tenacious woman. She has defended Israel and stood up for human rights with persistence and courage. But just so no one gets any ideas, I will be blunt. If Israel is ever forced to walk out of the UN, the United States and Israel will walk out together. Standing steadfast with our allies in support of greater economic growth and a peace with freedom is our third great challenge. Our administration is working hard to do just that. In Europe, we and our NATO allies have shown the Soviets our willingness to negotiate and our unshakable resolve to defend Western Europe. In the Far East, we are strengthening our ties to the Asian democracies and developing our relations with China. In Central America, we have supported democracy and fostered economic development. And in the Middle East, we have strengthened our relations with a nation close to your heart and mind, the state of Israel. Now let me take a moment to describe our relations with Israel and our efforts in the Middle East. Israel and the United States are bound together by the ties of friendship, shared ideals, and mutual interests. We are allies in the defense of freedom in the Middle East. The United States was the first nation to recognize the state of Israel. And ever since, our support for Israel has remained unflinching. Today, when even our NATO allies vote with us in the United States only some 6 out of 10 votes, the alliance between the United States and Israel is so strong that we vote together more than 9 times out of 10. Since I took office, the U.S.-Israeli relationship has grown closer than ever before in three crucial ways. First, the U.S.-Israeli strategic relationship has been elevated and formalized. This is the first time in Israel's history that a formal strategic relationship has existed. The new American-Israeli Joint Political Military Group is working to decide how the U.S. and Israel can counter the threat that growing Soviet involvement in the Middle East poses to our mutual interests. Our cooperation adds to deterrence and improves and protect the prospects for peace and security. The negotiations have been positive and they're moving forward. Second, we're negotiating to establish a free trade area between the United States and Israel. This will launch a new era of closer economic relations between our countries. By substantially eliminating duties and non-tariff barriers between our nations, we will enable American producers to sell and compete in Israel while providing Israeli manufacturers unimpeded access to the free world's largest market. Third, the United States will soon be giving Israel military aid on a grant, not a loan basis. We have restructured our 1985 foreign aid package and Israel will now receive economic aid totaling $850 million and a military grant of some $1.4 billion. And this will ensure that Israel maintains its qualitative military edge. All in all, the friendship between Israel and the United States is closer and stronger today than ever before, and I intend to keep it that way. In the Middle East as a whole, the United States has three aims. First, we must deter the Soviet threat. As the crossroad between three continents and the source of oil for much of the industrialized world, the Middle East is of enormous strategic importance. We're the Soviets to control the region and they have expanded their influence there in a number of ways, notably by stationing 7,000 troops and advisers in Syria. The entire world would be vulnerable to economic blackmail. Their brutal war against the Afghan people continues with increasing ferocity. We must not allow them to dominate the region. Second, we must prevent a widening of the conflict in the Persian Gulf, which could threaten the sea lanes carrying much of the free world's oil. It could also damage the infrastructure that pumps the oil out of the ground, and we must not permit this to happen. Third, we seek to go on promoting peace between Israel and her Arab neighbors. In response to the growth of Syrian power and the rise of the Iranian threat, we must help to protect moderate Arabs who seek peace from the radical pressures that have done such harm in Lebanon. Syria is trying to lead a radical effort to dominate the region through terrorism and intimidation aimed in particular at America's friends. One such friend we continue to urge to negotiate with Israel is King Hussein of Jordan. Today, Jordan is crucial to the peace process, and for that very reason, Jordan, like Israel, is confronted by Syria and faces military threats and terrorist attacks. Since the security of Jordan is crucial to the security of the entire region, it is in America's strategic interest, and I believe it is in Israel's strategic interest, for us to help meet Jordan's legitimate needs for defense against the growing power of Syria and Iran. Now, such assistance to Jordan does not threaten Israel, but enhances the prospects for Mide's peace by reducing the dangers of the radical threat. This is an historic moment in the Middle East. Syria must decide whether to allow Lebanon to retain control over its own destiny or condemn it to occupation. Syria forced the Lebanese government to renounce the May 17th agreement with Israel precisely because it was a good agreement. Now those who have chosen this course will have to find other ways to secure the withdrawal of Israeli forces. Arab governments and the Palestinian Arabs must decide whether to reach peace with Israel through direct negotiations. And if Arab negotiators step forward, Israel must decide if she will take the risks necessary to attain the real security that comes only with genuine peace. I have no doubt that given that choice, Israelis will once again have the courage to choose peace. I'm convinced that the initiative that I presented on September 1st, 1982 remains the best option for all the parties. It is squarely based on the Camp David framework and UN Security Council Resolution 242. It is time for the Arab world to negotiate directly with Israel and to recognize Israel's right to exist. Now we hope that the government of Israel will understand that continued settlement activity between the West Bank and Gaza will make the peace process more difficult. Peace can only come about through the give and take of direct negotiations. These negotiations will deal with many issues, including the status of Jerusalem, voting rights, land use, and security. If there's to be any hope for these negotiations, however, we must preserve our credibility as a fair-minded broker seeking a comprehensive solution. Only the United States can advance this process, and we must not undermine our role. And permit me to reaffirm a long-standing American commitment. So long as the PLO refuses to recognize Israel's right to exist and to accept Security Council Resolutions 242 and 338, the United States will neither recognize nor negotiate with the PLO. Only two weeks ago, terrorists planted hand grenades outside a store on a crowded street in Jerusalem. When they exploded, 21 shoppers and passers-by were injured, some seriously. Yasser Arafat, on behalf of the PLO, praised the attack on innocent civilians. He had the gall to call it a military operation. But terrorism, whether by government or individuals, is repulsive and peaceful coexistence can never come from indiscriminate violence. If I could leave you with one thought today, it would be this. Even though in the Middle East and elsewhere, the world seems hostile to democratic ideals, it's the free men and women on this earth who are making history. Here in the United States, we've only seen the beginning of what a free and a brave people can do. Today, America is leading a revolution even more sweeping than the industrial revolution of a century ago. It's a revolution ranging from tiny microchips to voyages into the vast dark spaces of space, from home computers that can put the great music, film, and literature at a family's fingertips to new medical breakthroughs that can add years to our lives, even helping the lame to walk in the blind to see. In Israel, free men and women are every day demonstrating the power of courage and faith. Back in 1948, when Israel was founded, pundits claimed the new country could never survive. Well, today, no one questions that Israel is a land of stability and democracy in a region of tyranny and unrest. So this Sunday, as Jews the world over observe Purim, they'll certainly celebrate not only the ancient deliveries of Jews from the wicked, but a modern joy as well. The miracle of the State of Israel. Permit me to join you and all Jews, and I'm now going to demonstrate my own courage, your fervent and triumphant affirmation. Yes, Israel. Thank you and God bless you.


This research quantifies the relationship between TC-induced direct economic loss, DEL, and maximum wind speed, asset value, and gross domestic product, GDP, per capita using a regression model with TC records from 2000 to 2015 for China's mainland area. The study finds that a doubling of the maximum wind speed increases DELs by 225%, while a doubling of asset value exposed to TCS increases DELs by 79%. A doubling in GDP per capita could decrease DELs by 54%. The research emphasizes how human dimensions contribute to TC risk and can be used to estimate TC risk under climate change and future socio-economic development in the context of China. This article was authored by Ming Chi Yi, Jidong Wu, Wenhui Liu, and others.


I can do it very quickly, save us a lot of time. Yes. End of statement, because you have to understand something.  And I expect that when you got up this morning, you spent the day talking with people, listening to things on the radio or the television, maybe reading newspaper. Maybe you kind of shared an interest with someone, whether it be politics or maybe you had an idea for a new business. Or maybe you tried to figure out how you're going to take care of your health. Or get prosperous. Or maybe find something special to do with your life. Maybe you wanted to really get into your spirituality. You were set up to find meaning in every one of those cases that makes you a perfect target. You're all set to join a cult. Because you have to understand something. Joining a cult isn't what actually occurs. You're recruited. And the way you're recruited, very simple. You have a relationship with someone. And you have a desire to do something with your life. That's all it takes for you to be recruited. So anyone you know who says, oh, I'm never going to be recruited into a cult, not me, I know too much. I already got it all figured out. They're actually the best recruits. And they're the ones who fall for it first. Now, I want to get quite specific here. Because whenever people read about cults or hear about cults, it's always a scare word. A lot of people like to say, that group over there is a cult. Not my group. I'm not involved with that. I could never become involved with that. Sorry it doesn't work that way. When Jews for Judaism goes out and talks about cults, we're talking about something very specific. Because really, if you look at the definitions of cults, you're talking about something that has been abused. The press constantly likes to refer to any group that's a little different as a cult. When people are trying to set their enemies up, they always say, oh, that's a cultic group. That's why we're very careful when we talk about groups. There is a very specific understanding.  And you know what? Most of them involve very basic things. You would be able to say that the boy scouts or the girl scouts are a cult. You have that in Canada, right? Something like that. The scouts are a cult. Sorry. You could say that the Marines or your army is a cult. Because they would fit certain definitions. In Webster's, it says a religion that is considered or regarded as unorthodox or spurious. Okay. Or a system for the cure of a disease based on dogmas or tenants or principles set forth by someone who is ready to put it forth to the exclusion of any scientific evidence. That's a cult. Or showing a greater obsessive devotion or dedication to some person, idea or thing. Well, how many people do we know that don't get enamored of some idea or person or thing? Ah, that's a cult. We go a little step further. And the reason I do this, is to understand what we're talking about. When we use the word cult, we're talking about something that's destructive. And I want you to also know that when you talk about a cult, you aren't just talking about that one individual who gets involved. You're talking about the family, the friends, and the general community as well. Because a cult doesn't just pick off one person when it does recruitment. So when we talk about cults, we talk about the totalist type. Very specific words. Actually, the definition that we use is taken from the American Psychiatric Association. They did a task force, because you're talking about deceptive and indirect techniques of persuasion and control. That's the key here. Any group that isn't forthcoming with you, anytime you ask a question, there's a problem. Red flags should shoot up immediately. If you ask a question, they say, oh, don't worry, that's not important right now. We'll get to that later. And they don't get to it when you think later is, and you can't convince them that you really want to do it. You've got a red flag. You really have to pay attention. But if that group or movement exhibits a great devotion or excessive obsession to that person, idea or thing, and this is the key, it employs unethically manipulative, and that means the deceptive and indirect. That's the key. Techniques of persuasion and control, then we have a problem here. But you know what, people don't realize this until they see it in the news. Cults aren't a problem. Unless, of course, they just killed 900 people down in South America, then it's a problem.  It's not the way it works anymore. The experts in the field talk about the fact that in North America alone, there are probably about 5,000 economic, political, and religious cults operating. And they have approximately two and a half million people as members, but it's affecting about 20 million people. This is just North America. The cult problem is so prevalent that the chances of a family member joining a cult are greater than that of a family member catching chickenpox.  You all have family and friends you know about catching chickenpox. It's more likely you're going to lose someone to a cult than you are them catching chickenpox. And that was Dr. Paul Martin said this a short while ago. He runs a rehabilitation center for ex-cult members. He knows what he's talking about. The problem is that all of these dangers are growing. The fact that we don't see it is because when we all got worried about it back in the 70s, it was a big thing because we saw it on the streets. We saw it on the college campuses. We were concerned that our youth were getting caught up in it, but it changed. The cults have changed. They've gone corporate.  I can help your business. We can help your business. Come to this seminar. Come to that seminar. A lot of those are front groups. Or you decided, you know what, I want to really get into my own business. I want to set something up for me. I don't think multi-level marketing might be my tap-tee. I'm going to get into some sales. A lot of those groups are cult fronts. Or maybe, you know what, I'm having some problems in my relationships. I think I'm going to join some sort of a psychotherapy group, a group awareness therapy. I'm going to go out and I'm going to really get into things and try to understand my feelings and what it means to be me and how I should relate to other people. Some of those groups are very helpful.  Or maybe you've decided, you know what, spirituality I'm dealing with now, shows kind of nice, but I'm not really getting much out of it. I think I want to go into some mysticism. I think I want to really get into some ancient texts and really deal with what it means.  I want to get healthy and I have this promise. I understand from all this wonderful stuff I'm reading that all I have to do is get into ancient texts and I'm going to be okay. Or maybe you've decided, you know what, that moon out there, it comes and goes and there's some sort of a cycle working here. And there's a group that kind of meets out in the woods and looks at the moon.  Some of these groups are very legitimate. Many of them are not. But the thing is, people are getting caught up in this every single day. And as I said in the first place, why do we call it a cult and what's the effect it's having on us in this community and what can we do about it? Very simple. We call it a cult because they are not direct when you ask the question. Many of these groups require you to sign up for their seminars before you can get any information about what's going to go on at the seminar. When you show up, you'll know what's going on. But you have to pre-register and give them a lot of information first, along with, of course, your major cash contribution. Others say, you know, come take this series of courses at X number of dollars and by the end of it, you will find prosperity. But they don't tell you what the steps are going to be, what the program is going to be, what the qualifications of the people are. What we're saying is, be a good spiritual consumer.  Because when you don't, you run into troubles. And we're talking all across the board. In the last four or five months alone, a number of cult cases that come through my office are tremendous. My family's even worse. A family had a child who was having difficulties. Got them to a proper psychiatrist. Trying to deal with a drug problem and some psychological problems went along with it. The psychiatrist felt they needed a resident treatment program. And you know what? Opened up the file, looked through a couple of things that had been sent to her and said, this is the one. It's a good program. These wonderful awards given to it, et cetera, et cetera. We think that that's where your son should go. They pulled a few strings. They got him in. No problem. After the fact, we get a call. This is within the first couple of days that the boy's there. Why? Because this happened to have been an observant child and they wouldn't give him kosher or vegetarian food, at least, at the place. He was getting a hassle. So he called home. He found a way to get to a phone, which was interesting, but I'm not going to go into that. And he made a call and said, something's wrong here. So the psychiatrist had enough sense to call us. We recognized, unfortunately, the group immediately, the place that had been sent. And it was a front for a cult, a very large cult. And I'll leave it at that. But the point is that the psychiatrist didn't know. She had been solicited. Just as the psychiatrist all across the country, all across North America are solicited because this is one of the larger residential treatment programs. They have some success, but the bigger success that they have is moving the people right into the cult. The statistics are very, very somber. And the way that people get involved are very straightforward. They're searching for meaning. They're looking for something special to do with their life. Or they're looking to get rich quick. Or they're looking to improve the business they have. Or the practice they have. In each of those cases, people get sucked in because they shut down their critical thinking skills. One of the most important things that we have is a brain. And the minute we say, you know what? This looks good. I'm just going to buy it. How many of you watch Infomercials just for the fun of it? How many of you bought something from an infomercial? Hmm, okay. You don't want to admit it. This is all marketing. We've talked about intermarriage. We've talked about missionaries. They also, and the missionaries capitalize on many of the techniques. The missionary groups, though, tend to be cultic. They are not cults, per se. Some of them do develop into cults, individual groupings. But there is a difference between the missionaries and the cults that we deal with. And Jews for Judaism has been dealing with it. We are on the front lines in the Jewish community and also non-Jewish community. We work interfaith on the cult problem because it's a societal problem. And it's a big one. We've developed certain programs in Canada. You have cultivating, cultivating. It's a worldwide program. Rabbi Spobak runs that specifically for training seminars because education is one of our key things. And as I said before, critical thinking, which, as Jews, we know, we've never been asked to check our brains at the door. Asking questions is part and parcel of what we are. We are so clitsped if we argue with God. So we shouldn't ask a few questions of the group that we're joining. Come on. The key here is getting the education and realizing when you're in the situation. And if you feel that your questions aren't being answered in the situation, doing something about it. We go around to colleges and do a program called Cults 101 where we go through not only the definition but the history and then we go through the techniques and what are the actual characteristics, such as that charismatic leadership and mind control techniques and those things that occur within a group, the us versus them, right? We're okay. They're the ones who aren't. That occur within these closed groups. It's something that's very important because people have stopped thinking. Charlie Chan used to say, a parachute is like a mine, right? It only works when it's open. But a philosopher in the last century said, an open mind is wonderful, but you can't afford to allow your brains to fall out. Both the cults and the missionaries capitalize on this. The fact that we don't necessarily ask questions, the fact that we have certain responses that we do automatically, especially to someone in authority, an authority just is the authority that we grant them, whether it be someone who comes to speak and you have that question and you heard the lecture and they went into a Q&A and you just never asked a question of them. That happens in many situations. You just said that people don't like to admit it when they've made a mistake. But once we're honest with ourselves, then we can feel comfortable in asking the questions of somebody else. Juice for Judaism is part of both its CPR program and JTTV. And as part of its curricular programs that go forth for teachers, educating the educators is very important. Because as you see, there are just a few of us. But once we can educate the educators, they can educate the communities. And that's very important. So that's how we spend our time in proactive education. And then of course the counseling. The counseling to individuals and the support for families and communities in many cases. Because these groups don't just come in and take up residence and go after just that one family down the block. Once they get that one family, that one family is set to start up with everyone else in the neighborhood. So, as Mark said earlier, you're the eyes and ears out there. If you see a group that's starting to act in a specific way or members of your community are starting to go into a certain path that you know what, there are some questions on. And they don't quite have the answers. So they don't quite understand what the group's about. Or, you know, the group has some questionable practices. Get out there and ask the questions. Call us up. We'll be happy to help you ask the questions. But the thing is to ask them and to get answers. Because that's how you respond to a cult that's in your midst. Through information and exposure. Through dealing with it in a straightforward fashion. Not by saying, you know what, it's a free country. If that's what they want to hold by, let them hold by it. Because if they're straightforward about it, absolutely. Everyone's entitled to their opinion. Everyone's entitled to their way of life. But when it's done through deception, it's not fair to anybody. It's not fair to the individual, the family, or the community. And that's where we're here to help and we welcome your help in dealing with it. Thank you.