Good afternoon, everyone. My name is John Hagan. I'm the Executive Director of the US ETDA, and it's my pleasure today to introduce the, and to facilitate and moderate the next panel discussion, the survey of our indoctrits, tips on compliance, privacy, and institutional data resources. Today we have Pat Green and Peter Inaudi from RTI International. And so this plenary panel presentation will explore the survey of our indoctrits. Oh, can you guys mute? I hear an echo. Peter and Pat. Can you mute your audio? Pat? Okay, the echo's gone, so we're good. This plenary panel discussion will explore the survey of our indoctrits or SED. We will begin with an overview of the survey and its uses, best practices to motivate respondents, and improved access to survey data using new data tools that can be used for analysis and planning. So Patricia Green is one of the RTI International Senior Experts in Education Research. She brings an academic background in sociology to her leadership roles on many of our major surveys and studies for the National Science Foundation, the US Department of Education, Foundations, and other major private and public sector clients. Currently she directs two surveys for the National Science Foundation, the survey of our indoctrits, which is a census of all students receiving research doctorates in the United States, and the survey of graduate students and postdoctorates in science and engineering, which provides data on trends in graduate enrollment and financial support. She's also designed and directed studies across all levels of schooling in the US and China, including the administration of international assessments in US elementary and secondary schools. So Peter Inaudi is a Director of Education and Research at RTI International, with more than 20 years of work experience in corporate and academic settings. He has expertise in survey methodology, data analysis, quality management, and systems design. Since joining RTI in 2006, he's led data delivery and dissemination tasks for multiple large federal surveys, and he was the Project Director for the National Science Foundation's early career doctorate survey. Currently, he's leading several methodological tasks for the survey of her indoctrits, including exploring the use of data analytic techniques to develop a taxonomy of research topics from dissertation titles and associated metadata. That sounds really cool. He receives a bachelor's degree from mathematics at Harvard University and a master's degree in sociology from the University of North Carolina Chapel Hill. So without further ado, I'm going to turn it over to Peter to run the slideshow and go ahead guys. And Peter, go ahead and start your screen sharing. Yes, please. Oh, and Peter, you would need to unmute if you need to talk as well too. Pat, can you hear? Yes, we can hear you, Pat. You're breaking up a little bit, but let's see if we can go forth. Peter, we're having a lot of audio glitches. I wonder if Pat needs to do the same thing, get out and come back in. It seems like it did improve it. I don't know if you can begin her presentation for her. I can try. Hold on a second. Hey, Pat, we're having big audio problems.  Okay. Audience, please bear with us. So I should mention, and I should have mentioned this earlier in the session, if you have questions for the panelists, please enter them in the Q&A tab, and then I'll call upon folks along the way. And at the end of the session, if you'd like to ask your question live, that's great. If not, I'll be happy to read it to our panelists and the audience from the Q&A tab. If you'd like to chat amongst yourselves, you can certainly use the chat tab. Okay. So Pat is going to try logging in with her Mac as opposed to her other computer. So what I'm going to do is I'm going to start and hopefully I haven't practiced her section, but maybe I think it'll be okay. Welcome together. Thank you. And hopefully she can do it. When she logs in, she can take over. Okay, great. Thank you. So our agenda for this plenary is to give a brief overview of the survey of earned doctorates, talk about institution and student participation, new changes or what we've done for privacy and confidentiality and some new and exciting things we're doing on that front, and then talk about data uses by institutions and others, and then how to access your data. So that's part of that last section. So in terms of the overview, the survey of earned doctorates is a census of all research, of all research doctorates in the United States, okay, who are getting the, there are the people who are graduating this year within a year cycle that are getting their degree. It should be a census of all people with, with research doctorates. So it's not professional doctorates. It's not practitioner level doctorate degrees, but rather all PhDs, EDDs, you know, SCDs and other types of doctorate degrees. It is, yeah, oh, I can hear you perfectly. Yes, it sounds wonderful. Okay, go ahead and take over.  Okay, so I think you were just talking about research doctorates. The professional doctorates are not included in the study. It really is oriented toward research. So next slide. Okay, so who sponsors the SCD? There are four sponsors. NSF is the lead sponsor, and they are the ones that are responsible for actually conducting the survey. NIH, the Department of Education and National Endowment for the Humanities also contribute to the study. And both Peter and I work for RTI International and we're under contract to NSF to actually conduct the study. All phases from, you know, developing the instruments to collecting all the data to analyzing and reporting. Next. Okay, what is the purpose of the SCD? It is to collect information on the doctoral recipients' educational history, demographic characteristics, and post-graduation plans. You know, this study has been going since I think it was first done in 1958. So it really is possible to examine trends in many areas, both the representation of women minorities and foreign nationals, emergence of new fields of study, and post-doctoral employment opportunities. We examine fields of study every other year to see if there are new fields that should be included in the taxonomy. And just aside, not this year, we changed. And we're now using the SIP codes, course and instructional program codes, rather than the SCD field of study codes, although we'll still be reporting by those codes. This dataset is used by many people, both at the institution level, often at the state level and certainly at the national level, to determine if there needs to be changes in the way money is distributed across institutions and graduate programs and to benchmark, you know, what is really happening. Next slide. Okay, so there are two ways that doctoral recipients can complete the SCD, either online via a web instrument, which is like almost everyone, and via caddy computer-assisted telephone interviewing. If students haven't responded like when they're on campus and have been asked to do it and they don't respond to our emails and other contacts, the sort of last resort is to go to telephone interviews, and that's an abbreviated interview. We no longer use the paper questionnaire at all. Zero paper questionnaires this year. Last year, we had maybe 10. And so we're very pleased with that. Next. Okay, if you want more information, this is the SCD information website. The address is up in the top corner and also these slides are all online on the website with the presentation, so you can consult that if you want to find this website. Next. So now I'm going to talk a little bit about institution and student participation. You might have noticed that I changed compliance in the title to participation because really this study, you know, it's not, there are no federal regulations really regarding it. We have no sticks, you know, it really is because people want to do this study and contribute to it. So next slide. Okay, the role of institutions, I have to thank all of you. I mean the institutional participation is really what allows the study to happen. First, most institutions make the initial contact with students when they file for graduation. And then secondly, institutions confirm which people actually did graduate with each graduation ceremony. So, you know, this is a critical activity and we really couldn't do the study without that. Next. Institutions also have a role in non-response follow-up. When we get the graduation list from the universities, we match them with the completes that we have on file, people who completed the web instrument, and then we post on the, there's an administrative tool, institutional coordinator, administrative tool, I can't. We post all the people on the graduation list and whether or not they have completed the survey. Some institutions prompt non-responding students, other ones, we ask them, the school to provide their grad address roster so that we can begin prompting. And then near the end of data collection, we ask institutions to provide critical items for non-responding students. Next. Okay, so you can see there are three times that we go to institutions to ask for information. And each time, fewer and fewer institutions and students are on these lists. Ideally, if people really responded right away, the only thing we would ask for is that initial graduation list because everyone would complete the survey. But we do go back and ask for address rosters for the non-respondence. And then at the very end, the missing information rosters for the total non-respondence. The data elements on that request are shown on this slide. It's very basic information that analysts need to really analyze the SED and also it's used as a sampling frame for the survey of doctor recipients. Next. Okay, SED non-respondent follow-up is also done by RTI. RTI prompts non-respondence via email providing them a link to the survey. If they don't respond to our emails, then we send hard copy mailings with links. And then finally, we have telephone interviewers who contact non-respondence. And at that time, they can choose either to complete a short caddy survey that really basically asks only critical items or if they prefer the interviewer can automatically send them a link to the survey again and they go on the web. And I would say it's about 50-50, you know. Some of them haven't responded because we had bad email addresses or bad snail mail addresses. And once we get a phone, they are willing to participate. Next. So we've been very successful with this methodology. Approximately 92% of doctor recipients complete the survey cycle each year. And I think, you know, people as they file to get their Ph.D., they're usually pretty darn happy. You know, it's like the end of a very long road and they're very proud of themselves and most of them are willing to share their experiences and respond to the survey. And I have to thank all the institutions because they work hard to have their students complete the survey. Next. So here are actually the response rates by school. So we have about 570 schools in the survey. It varies each year because, you know, schools can be just one school at a university or could be, you know, other universities report for the entire all the schools there. And the 60% of all schools have student response rates that are 90% or higher, which is fabulous. And another 21% have response rates that are over 70, but less than 90. So only 19% of schools have response rates that are less than 70%. And we do identify, actually we identify all the schools with less than 80% response rates and often contact them and try to figure out what we could do to help or, you know, share with them ideas for what they could do. We also traditionally until the pandemic do several site visits a year to schools that have a sudden drop in their response rates to see if we can figure out, you know, how we can help them move those response rates up. Next. Okay. Here are some of the best practices that we found. Steps to achieve high response rates. The most important one is if you have a graduation checklist, make sure to put the link to the SED on that graduation checklist. If you don't have a graduation checklist, if you can distribute the web survey link three to four months prior to graduation, you know, that would be great. And then finally, another option is to link the SED survey to your institution's exit survey, if you have one, that way you can sort of deduplicate the questions and reduce the burden on the students, make it faster for them to respond.  You know, we can send them every quarter or however often you want so that you can merge that data with your own exit survey data on a timely basis. Next. Okay. And more steps to improve response rates, I think I'm going over. Monitoring student participation and providing graduation lists as soon as possible. Sometimes there are a few schools that wait until like September or October to send us their spring graduation lists. And so then when we see the non-respondents, there's really not a lot we can do. Some of them have moved far away. And, you know, for sure their, you know, snail mail address has changed. The phone had addresses changed. Maybe an email address is good, but it makes it much more difficult. So being timely with the grad lists really does help us. And we also now offer an API and application programming interface. Instead of sending copies of the certificate of completion emails that students get after they finish the SUD, you know, we send those to schools as well as to the students. You don't have to organize all of those or put them any place. This automatically pings us and we send you those names directly to your database. So it can hopefully reduce your burden. Next. So institutional participation is critical. I think I've already talked about the first several bullets, but I would like to hear from you about ways of making SED data of more use to institutions because we really do believe that if institutions use the data that they'll be more willing to participate. And second, are there tools that we could provide you to help you directly prompt non-responding graduates? Anyway, any ideas that you have as well as any questions I open to hearing from you. So, John. Great. Thank you so much, Payette. Oh, sorry. This was a last minute change. Please go ahead. I need one minute to talk about privacy and confidentiality. Please. Michael. Next. So we really do take privacy and confidentiality very serious because SED is an unusual survey in that it's a census, not a sample. And everyone earning a research doctorate in the U.S. is included. So that means this target population is highly visible. I mean, their CVs are easily online. So you could find somebody who, you know, got their PhD from the University of Minnesota in, you know, 1982, who's a female and whose field of study was in physics. And with that, you know, there aren't too many of those people. And so they are highly identifiable. And so that is why this data is protected. So these concerns require we review any data that we release goes, you know, through a disclosure review, and we take that very seriously. But NSF, go to the next slide, is also very concerned that people have new ways to access and use the data. So in a few minutes, Peter is going to be talking to you about some of those new methods. And we are trying to make it easier and easier for people to do this. So over to you, John. Great. Thank you, Pat. That was very informative. So we have a question from Deborah Charlesworth. Deborah, if you'd like to ask that live, go ahead and unmute your video and audio. Oh, wait a second for you to acclimate. Great. Hi, Deborah. Yeah, I was just wondering, I hadn't heard about the API yet. And I work with my colleague who collects the SCD data, but I'm sort of the technology person in our graduate school. Does that integrate with the banner databases? Do you know? Banners are pretty common database for universities. I think you're muted, Patricia. Right. It is. I am not sure about that, but I can get back to you with that answer. Great. I'd like that. Thank you. Good question here. So I had a question, which I think derived from your website here, over the life of the SCD, the survey has consistently attained response rates exceeding 90% from doctoral recipients, as you had mentioned. While the study is voluntary, we found that most of the graduates find reasons for completing the survey compelling. What advice can you give institutions to compel students to complete the survey? For example, what key items, a handful of items that are really compelling to the student to actually make them interested to participate or what's atop several reasons? In the communications like brochures we do for students, one of the big things is the use of the data. We point them to some of the products coming out. And the fact that decisions about future graduate programs are going to be made on SCD data. And it's the way of getting their voice and experience heard. And as I said before, a lot of people, the fact that it is the National Science Foundation that's funding this, I mean, I think they do see the scientific integrity of the data collection and so are confident that their data is going to be protected. Peter, do you have any ideas on that? Yeah, I think in this day and age, one thing you might want to talk about is diversity and inequality. Because what one of the things that the SCD does is it's the only reliable source for the number of minorities that are receiving doctorates across the nation, research doctorates across the nation in different fields by field. And so this is, if we have their responses, it really helps universities understand the availability of doctorates by race and ethnicity and citizenship and helps with planning. And disability status too. Yeah, yeah. Great, great. Yeah, I think there are many valid reasons both at the student level for why they would want their field to be enhanced, their institution to be enhanced by the analytics from this and certainly the institutions in terms of future planning, I think it's invaluable information. So a lot of that I think is up to the front lines of people to, you know, people to effectively communicate that and pass on the information from the survey-earned doctorates about these compelling reasons. So we have a question from Roxanne Trees. Roxanne, if you'd like to ask your question live, go ahead and unmute your audio and video. And if not, I'll be happy to read the questions aloud. Okay, so I'll go ahead and read Roxanne's question. We require students to complete the SED prior to graduation, but I'm waiting for the day that a student challenges this requirement. Any advice? Could a student only complete part of the survey? Yes, a student can skip any questions they want. That's on the opening screen. There's an informed consent that they can read through that says skip any question that they want. So that's certainly a possibility. We actually have had people that have completed the survey, but then afterwards regret having done it and they call and ask us their information. And we will do that. You know, that only happens once in a while. That's now the common occurrence. So and, you know, this varies by institution, institution, what their regulations are about like assessment activities. A lot of times, like in the college catalog, which is sort of the contract with the students, there is mention of participating in assessment activities or depending on how evaluations, whatever, which does give them some leeway. You know, I've never really heard of anything really becoming a legal challenge. I know that every once in a while, it, you know, it does go to the university general council and they tweak the language that maybe is used on the dissertation checklist. But so far, you know, there are a couple universities who want students to opt in, which is it's a whole lot of work for them. And, you know, it does definitely reduce the response rates. But again, I think it is just examining the language and we can send you if you what we think is good language to use. That's very helpful. It's interesting too that I think we've gone from, you know, years ago, it used to be much more considered a compliance issue. And now, as you mentioned earlier in reflecting the change of the title, it's more to compel people if they're informed and they know the reasons for why the survey is done and how that can improve their situation in their field or whatever at their institution. People are much more likely to participate. And so, you know, I think gone are the days or most of the schools say, you know, thou shalt, you must and but, you know, just say, here's the part of the normal process that we do and, you know, here's the link to it. And then people have additional guided information if, you know, they have questions or not quite sure about it. But I think these days compliance is less of an issues is more about compelling people to, you know, convince them, you know, it's the right thing to do. And I don't think at many institutions it's an issue. Janice Robinson had just one brief comment here at Brigham Young University. And we made the SED a required item to be cleared before we approve the graduation we approve for graduation. It works great. The email notice that it is completed comes to our staff. And so it's easy to clear. So Janice, thanks for your comment there. So I guess we should probably jump into the Peter's portion of the presentation. We're going till two o'clock here and we'll entertain some more questions along the way here or as well. Thank you. Thank you. Oh, Peter, if you wanted to share a movie, the icon next to the screen share, I think is it looks like a YouTube icon in the middle. I think that's actually if it's a YouTube video, you could use that. It's not a YouTube video. Unfortunately, I embedded it in the program. All right, we'll cross our fingers if you want to. Yeah, we'll cross our fingers. And if it doesn't work, I'll just describe it and we can send links to those that are interested to the tool. So thank you. So my name is Peter. And now I work with Pat for a long time on different surveys. And what I'm talking to you today about is, you know, sort of how the SCD data are used. And they're used by various entities, so researchers, federal and state agencies, as well as our universities, I would say our universities are probably the biggest users. But they're really used to evaluate the pipeline of doctors coming into the program into the workforce. They're used to evaluate to do program evaluation to make decisions about whether the financial resources going into different fields are appropriate and or the mechanisms that people use to fund their graduate students are sufficient. They are used to inform government policy as well. This is really focused on institutional uses. And we actually went out and talked to several of our respondents over the last year, talking about how do they use the SCD data. And we got the three, well, the two main uses are for program planning and institutional reporting. So generic institutional reporting about, you know, what are the demographics of our graduating doctorate holders, what are their postgraduate plans, what were their funding sources, how long did it take for them to to finish their degree, things like that, the level of debt. In terms of planning and peer analysis, though, you know, the SCD has a lot of really great potential. A lot of people, as Pat mentioned, request their own data. And so you can use that data to do all sorts of nice things. I'm actually going to go down one slide briefly to show you like this is a graph that one of the our respondents used in their annual report that looks at the funding and the change in funding for graduate students or for doctoral students between 2015 and 2019.  So you could say, what is our university, what is the funding, what are the funding mechanisms we use compared to our peers. And so they do, respondents do are all of you and and other respondents do use the data to benchmark their programs by field and Carnegie classification. The other thing that sometimes is used is to look at, well, who's doing interdisciplinary research, how many people are doing interdisciplinary research, and what fields are they in. And then how does that compare to how other schools and other doctoral recipients report whether or not they do interdisciplinary research. And a final use is actually using these data as potentially sources for your exit surveys or potentially if you wanted to change your taxonomies over time. We're doing, we're constantly working on taxonomies and I apologize for the phone in the background. So looking at how NSF or as Pat said, we just started to use the SIP code, which is the Department of Ed's SIP classifications for field field of study. All right. So how do you, how do people access the data? And I'll say that there are many ways to access the data. Perhaps the best starting point is to go to the regularly produced reports. So there are a series of annual reports that are produced by NSF as well as the National Science Board. You can also request your institutions data and then there are several data tools that I'm going to show you briefly that you can use to look across institutions. And then finally, the last way is to actually ask for a data request or analysis through the project officer or through our website for specific data if you can't get it through these other sources. So the primary mechanism I would point you to is, we call it the digest, but it's an annual report. If you go to this website that's up at the top, NSF.gov statistics SED, it'll take you to this, this slide and there are two main sections. There's the reports and the data tables. The report is a, you know, it's a summary of the key topics that the analysts at NCSES have decided are the most interesting in this particular year. So it varies year by year. It has texts associated with all of the graphics. The figures are interactive so you can move your cursor over them and it'll tell you the data points. You can also on many of the figures take out a line if you're only interested in U.S. citizens as an example. And it was, and the initial slide talks about U.S. citizens and foreign nationals. It covers a lot of different things. Typically it's meant to be an overview to touch on the main ideas that are of interest to the analysts at NCSES. There's also 77 data tables that are produced annually. They cover most of the data that we collect, financial support, educational history of doctoral recipients, fields and demographic characteristics of doctoral recipients, postgraduate commitments and plans. That's one that is often of interest to universities and other stakeholders. Peter, you only have about five minutes left. Okay, thank you. So then we also produce a series of annual tables, supplemental tables that are available by request because they do not suppress any of the data in the tables. But they show by detailed field, sex, race, ethnicity, counts of doctoral recipients, as well as the baccalaureate origins of those doctoral recipients. And actually, the baccalaureate origins is one of the most requested tables. I would just like to point out two congratulately mandated reports that, again, are excellent sources of data about the state of science and engineering and about sort of the equity issues. And while these are cross survey reports, the SED is utilized in both of these every year. And so that's part of what we do is we support the authors of these reports as well. Every institution that has at least 25 doctorates over the last five years will get a institutional profile. This is sent to the coordinator as well as the dean of the graduate school. And as of last year, the VP of research. If you've not seen this report, it's a really interesting report. It shows your school and it does a peer analysis where the peers are the other institutions in your Carnegie classification. It's a six or eight page depending on the size of your school, the number of doctorate recipients you have documents. And it shows, it basically does a breakdown of many of the items that we collect and compares you to your Carnegie classifications and for many it's also for the national average. In terms of institutional data files, you can request historic data for your institution from 1920 to the most recently published year of data. This is something that you can do straight from the website. Pat also talked about getting your preliminary data. You can get that and you can link it to your exit surveys as an example as a way to add context, add additional data to your exit surveys. All these data are provided through secure transcription and with Excel and ASCII formats as well as SAS and SPSS code to read in and apply variables. For all of these data use agreements are required, but there are two tools that you can use. One, this is the NCSS data tool. You can get to by going to ncssdata.nsf.gov. Again, this goes across all their surveys, but this isn't just an example showing 2019, the distribution of men and women across race ethnicity. But you can see on the left hand side the variables that are available here. Unfortunately, it's a fairly small set of variables. So part of the reason for doing that is really because of the potential for disclosure for this population. So most of the things that you might be interested in looking at aren't available here. Mostly it's just demographics and counts. So NCSS has agreed to work on a new file called the RDS, a new system, and it's really intended to help you do analysis of a much broader set of variables. No institution level data can be done here, but you can do a broader set of analyses. It's based on a sample. And so it provides weighted estimates, a variety of statistics. It's very intuitive and fast. I was going to show you a video, but I'm not going to do it. That sort of walks you through this. It is in beta testing. And so it is something that if you're interested in doing, please let Pat or I know, or the survey know, we'll send you the instructions for getting on to the beta testing group. We're really excited about it. And so we hope you actually take us up on this. And then, oh yeah, so customer data request. So this is the last thing. So if you can't get the data that you need through these data tools or through the reports that are out there, you can write up a request and things that are typically done through these types of requests other than getting your own data include like 30 year trend analyses or there's something called the institutional yield, which is, you know, you're looking at bachelor's degrees, bachelor institutions to see of these people. So you've seen lists probably of which bachelor level schools produce the most doctor recipients, right? So but that really privileges the large schools. So one of the things that NCSS has done is they've come up with a way of using the iPad's data to say sort of what percentage of graduates of a school go on to a doctorate degree. That's the type of thing that really can't be done through the data that's available. But we could help you do that. So that's the overview. If you have any questions. Great. Thank you, Peter. So I just wanted to add a comment here at least in the time when I was handling a survey of our doctorates seemed to me in our institution, there were two lines of reporting one for the front lines person who handles, you know, contacting the students to survey and so forth and sending the graduation lists. And then there was another level West Virginia University, they had a decentralized office of graduate affairs. And those folks actually had access to the institution reporting and data and the communications back and forth from the folks that survey for doctorates. And I did not necessarily see those communications, but I think it would be a good idea for the front line front lines people, whether you're in the libraries or the graduate school to talk to the folks who could be in your graduate school or in office of institutional analysis and planning, who receive those communications and receive the reporting for that. So that they were all on the same page and aware, particularly the higher up administrative folks in your grad school, or planning office, that these resources are available. And sometimes a quick email, you know, will pave the way for that conversation and, you know, can open up, you know, a lot of other things avenues as well. Yeah, at that point, if you if you haven't seen the institutional profile for your institution, you might want to contact the graduate school dean or you could contact us directly. And I think we could probably send it directly to you. But Pat, do you think that's true? I'm not 100% because they do sign something or yeah said not to be released beyond the institution. But we could. And now our database, we don't just track the coordinator and the graduate dean, there are multiple people involved in this. And so those multiple people are in our database. And I'm sure that we could probably send it to any of any of those people. So if you want to just ask us and we'll figure your something out. Great, great. Do you all have a link publicly available link to that video? Or is it because it's in beta testing, you have to just request by email or whatever your process is. We don't have a link right now, but we're thinking about putting one up. Okay, that's cool. Yeah, let us know. And we'll be happy to, you know, even post it from our website. I think that'll be very valuable resource for folks. We can send a link to people for the their actual artist so they can go in and play with it. Great, great. Okay, it's very helpful. Just looking through the Q&A tab here, Sally Evans had a follow up to Janice Robinson's comment. Sally says, Janice, we do the same at George Mason University. I get the notification that the student has completed the survey, and I will not process their submission materials until I receive it. So interesting various levels of compliance, carrots and sticks and so forth. But sometimes it's a trick is just a trick the student into thinking that it's really required when it's technically not and then given reasons for, you know, compelling reasons why if they beg the question further. Janice, yes, we have a PowerPoint actually posted in our presentation. This PowerPoint presentation is in our presentation's web page. So you can take a look at that immediately. Can you give some practical examples of how institutions have used their SED data to analyze degree equity? Yes, actually, the custom requests that we receive, two-thirds of them really address equity issues. They're asking for data on women, minorities, you know, citizens, non-citizens, and often by field and either by like sort of fine fields and often over a long period of time to see how those distributions have changed. So I think also a lot of affirmative action officers use SED data to gauge the sort of population of people eligible to become assistant professors in a given area, you know, if there are 100 graduates in a year, PhDs in a certain field, what percentage of them are white underrepresented minorities or women so they can see what their recruiting pool really should look like and compare it to what they actually got. Yes, great. So that comes full circle from, you know, passing the students on and then they graduate to the recruiting cycle of the new incoming fold and, you know, so you can see how this all ties together. I think that's a very useful and practical example. Stacey Wallace at the University of Florida says we require the survey burn doctor. It's for a degree award. However, students can choose to fill in on what they want, which would be fine. But then we are notified to complete the missing information report. And then she follows up. It is not our policy to release student information without their consent, hence the 99% response rate rather than 100%. The other institutions provide this data on behalf of their students. Yes. And typically those missing information rosters, we only send, we only request them for people who are total non respondents for people who just didn't answer a question, skipped over it. We consider that question to be refused and we won't ask for it. It used to be years back that they did. There was a that we asked it for everybody, but that's no longer the case. It is for the total non-responsive. You know, we could never locate or they never gave us anything. Okay, great. I have another question here. Does FERPA apply to the SED? Family Education. Yeah, no, I know what it is. Yeah, because the data is also sponsored by the U.S. Department of Education. It is allowed. I mean, there's no FERPA. You know, FERPA definitely allows SED that institutions can release data directly to the SED. Okay, very good. Oh, here's a good one. What is an SED microdata and how can one obtain it? Well, the only way you can get microdata is through a restricted use license. Right now, like all that, we're developing the data analysis tools that will hopefully let people do the analysis they want. And, you know, they'll get more sophisticated over time, but releasing actual microdata files is not part of the planning, just because there would be too big of a risk of disclosure. I see. And so specifically microdata then that drills down at the fine-tooth level of, I guess it doesn't, the generalized data tables don't include those nuances or how is it differentiated? Well, the ARDAS does. I mean, it's based on microdata. Microdata is loaded in there and then you can manipulate it there. And so you can produce your table. Someday you can probably be able to produce regression and trend analysis. That's not there yet.  And that it's just, again, since this population is so identifiable, it would be very hard to actually make a public use microdata set unless it only contained a very limited number of variables. And that's very useful. Yeah. So that's mainly an internal use kind of analytic. Okay. Very good. So our time is up, gang. I would like very much to thank Patricia Green and Peter Inaudi for joining us. It's been very informative. And let's keep up this dialogue. Years ago, we've had representatives from the survey of earned doctorates and it's been almost a decade. So we're delighted to have you join us. And we're happy to help participate in creating this national data set. Yeah. Well, thank you so much. Thank you for having us. You people are very important to us. So, you know, if you need something, just let us know. We shall. Thank you. All right. Take care all. Everyone join us in five minutes for the next set of breakout sessions. And thanks again. Thank you.


Ukrainian drones attack Russian strategic bombers base. Two drones attacked Russia's Saratov region, where they were reportedly shot down and fell on the Engels' two military airfields. That's according to the Astra Telegram channel, referring to its sources on the ground. The consequences of the attack are yet to be clarified, the report adds. Earlier, Russia's Ministry of Defence reported that an UAV had been downed over Saratov region, offering no details about the incident. As reported earlier, the military airfield in Engels, which is home to Russia's strategic bombers, was also attacked in late 2022. It is known that the TU-95 and TU-160 bombers launching missiles at Ukraine take off from the said base. According to unverified reports, Russia's defence officials confirmed three servicemen had been killed in that attack, after which some of the warplanes were relocated to other airfields. As of 2021, the Engels' airbase is the only one in Russia where TU-160 strategic bombers are based. Since February 2022, the base has been used for large-scale airstrikes against Ukraine. The Soviet-era Engels-2 airbase, named after the communist philosopher Friedrich Engels, is home to Russia's 121st Heavy Bomber Aviation Regiment, which includes the TU-95 and TU-160 strategic bomber squadrons, whose missiles have devastated Ukraine's energy infrastructure.


Most proofs in mathematics can be built around the conditional if A, then B. We say that A is the antecedent, and B is the consequent. So, for example, if this is Tuesday, then this must be Belgium. In the statement, this is Tuesday is the antecedent, this must be Belgium is the consequent. In a perfect world we'd always use the if, then construction for conditionals. We don't live in that world. And we often need to rewrite statements as conditionals. It helps to view the conditional if A, then B as follows. If you have A, then you have B. So, for example, let's rewrite as a conditional the square of an odd number is odd. Now our statement is claiming something about the square of an odd number. And it appears that we have the odd number. And so consequently, well actually antecedently, we have the square of an odd number. And the claim is that we also have an odd number. And this suggests that our simple statements are a number is the square of an odd number, and a number is odd. So rewriting this as a conditional gives us if you have the square of an odd number, then you have an odd number. Now while this is a perfectly good way to write the conditional, we might try to clean up the grammar a little bit. Well, we could start with the square of an odd number, for example 49, in practice we're more likely to start with an odd number and square it. And so our antecedent can be that we have an odd number and that we're squaring it. So rewriting this a little bit more simply, if a number is odd, then its square is odd. Another place where we might have a hidden conditional is in statements like all and every. Consider a statement like all x are y. This says that if you have x, then you have y. So this corresponds to the conditional if x, then y, or cleaning up the grammar a little bit, if something is x, then that thing is y. So let's try to rewrite as a conditional all prime numbers are odd. So again let's identify our simple statements. We have a prime number, a number is prime, and then we have that the number is odd. So remember all x are y is the conditional if x, then y, and so this gets rewritten as a conditional if the number is prime, then the number is odd. Another common phrasing is no x is y. It seems that we have no x, but it's useful to remember we want to make positive statements, and so it's easier to suppose that we have x. If we do, then since no x is y, then we know we don't have y. And so this gives us the conditional if x, then not y. So we could try to rewrite as a conditional no multiple of 100 is prime. And so our statements, a number is a multiple of 100, and the number is prime. So again remember the statement no x is y corresponds to the conditional if x, then not y, and so the corresponding conditional if a number is a multiple of 100, then it is not prime. Sometimes we have paired conditionals if a, then b, and also if b, then a. This pair is called a biconditional, and we could write it this way, but we won't. We could read this as a if and only if b, but we'd rather not. While we commonly do read it as a if and only if b, here's a useful idea to keep in mind. How you speak influences how you think. You should always speak of biconditionals as two conditionals if a, then b, and also if b, then a. So we might rewrite the statement a matrix has eigenvalues if and only if it is square. So let's identify the simple statements. A matrix has eigenvalues, a matrix is square. Since this is an if and only if construction, then as two conditionals, we have if a matrix has eigenvalues, then it is square. And then switching the antecedent and consequent, if a matrix is square, then it has eigenvalues. Given any conditional if a, then b, there are three related conditionals. The converse, where we switch the antecedent and consequent, if b, then a. There's also the inverse, if not a, then not b. And finally, the contrapositive, if not b, then not a. So for example, write the converse, inverse, and contrapositive, all prime numbers are even. So a good strategy, if possible, rewrite statements as conditionals. So let's take a look at our statements, and we can break it down into these simple statements. A number is prime, a number is even. And so our statement corresponds to the conditional, if a number is prime, then it is even. So this gives us the following. The converse is where we switch the antecedent and the consequent. So if a number is even, then it is prime. The inverse is what happens when we not both of them. So if a number is not prime, then it is not even. And the contrapositive is sort of a combination of the converse and the inverse. We swap antecedent and consequent, and we not both of them. If a number is not even, then it is not prime. And finally, one of the most important occurrences of conditionals. Every definition is a biconditional. Wait, that's an every statement, so we should be able to rewrite it as a conditional. We'll leave that as an exercise. And this does lead to a very good strategy for mathematical proof. Always rewrite definitions as two conditionals. For example, we might define an even number as follows. An even number is a number that is a product of two and an integer. So first, let's identify these simple statements. A number is even. A number is the product of two and an integer. Since it's a definition, it's also a biconditional. And so we have two conditional statements where we have our antecedent and consequent from these two. And so the two conditionals are, if a number is even, then it is the product of two and an integer. And also, if a number is the product of two and an integer, then it is even.


Live from Las Vegas, Nevada, it's the Cube at IBM Interconnect 2015 brought to you by headline sponsor IBM. Ariana Grato here and I'm at IBM Interconnect in Las Vegas, Nevada and you're watching the Cube on the ground. I'm here with Tim Moran and he is one of the VIP influencers here at IBM Interconnect. So can you tell me a little bit about yourself Tim and what you're doing and how you became a VIP influencer? I guess I just stumbled into it really. I'm a software developer by trade and I did all that coding stuff for ten years before I realized I wanted to get out from behind the computer for a little while. So I'm now the director of digital sales for one of them for Tribute Media Group and it's just a completely different world. So I was invited to this basically through Twitter. I was really excited about the Watson Analytics Package that was released around October last year and I had tweeted a few articles and then IBM had taken some note and the conversations really went well because I liked how they were so progressive and I liked how they were a company that really values the voice of people that aren't, you know, they're paid customers and they're not paid sponsors or advertisers, that kind of a thing, so. Yeah, definitely. So you mentioned Tribute Media, is that correct? Can you tell me more about that, what you do with them? Sure. So Tribute Media is a company that owns 59-ish television stations and with those television stations comes a lot of digital opportunities. So across about 75% of the country we have markets that will basically take these news and information websites and, you know, give the local audiences information that affects them in their daily lives. So through those platforms they produce upwards of two to three hundred million page views on a monthly basis, produced by multiple tens of millions of unique visitors. So, you know, it's a voice and what we do with that voice is we work to help advertisers and help sponsors to, you know, talk about themselves and talk about how they can help our audiences without being intrusive to our audiences. So it needs to make sense from both angles because, you know, if the advertiser voice is too loud and too intrusive then that's going to really upset our just natural news viewers and we just don't want that to happen. So we really see social media integrating on television in general. What are some ways that you see this transition happening and where do you see social media interacting with television and entertainment world in general? It's such a big topic. Social media, it's giving the audiences a voice, you know. So anybody who might be, you know, watching Shark Tank, for example, we had the sharks here yesterday afternoon and, you know, they can comment and they can engage in social conversation along with either locally produced programming or network programming in a way that they weren't able to previously. So, you know, they can engage in this conversation and then, depending on how loud that conversation is, potentially affect the brand, really. Because if we're listening and we should be listening then we need to understand what they're liking and what they're not liking and pivot according to that. So that's an interesting topic. How do you see social media helping to create a brand and send a message out in general? I think everybody's actually a brand on social media.  It's just fact. You know, we all have our own online persona. So social media, in terms of big brand, is helping to connect them directly to consumers or customers in ways that were previously not possible. And actually, I think it's good because in a modern, agile environment, consumer environment, whatever the case might be. These companies need to be listening because there are so many disruptions happening across every industry just about on a daily basis that if you're the entrenched player and you're not pivoting and adjusting and growing with what your core clientele wants, then there's somebody else probably waiting next to you that's going to do it. Yeah. What are some of the biggest distractions on social media? Dog and cat videos. Yeah, that's the worst. Yeah, I just wasted my time now. There's still a lot of spam in social media. You have to see through the noise. There are people that are self-promoting, just to be self-promotional, if you will.  And then there's negative social noise that we're all aware of. But then again, too, that negative social noise can also figure into feedback for a product. Like for example, we were looking at the IBM Social Analytics package earlier today and what the IBM Social Analytics package does in nearly real time with maybe just about a 20-minute or so lag in data, depending on how quickly the data collection happens. These big brands, whether they be grocery store chains, which was the sample that we saw today, like for example grocery store chains, they could see what the public sentiment is across any number of social platforms, whether it be Twitter, Facebook, but then don't forget social platforms are also review websites, couponing websites, travel websites and things like that. So it aggregates all of that data. It uses some natural language processing to understand what the sentiment is that's being put forth by the public for those brands. And then it really helps you to drill down into the different areas. Like you can drill right down in the Twitter and see literally what people are saying and they color code red for not good and blue for, I'm sorry, green for good. And hopefully that can help you to adjust whatever people don't like. Sounds like a great product. It is. Yeah, it's cool. So we really see this hashtag revolution kind of creating a conversation, getting rid of a lot of the noise. And I see hashtags everywhere along with join the conversation. How do you think hashtags are benefiting or a beneficial thing for Twitter and Facebook? And can you talk a little bit about that subject? Sure. I think it's a great way to anchor this conversation. So hashtags, for example, they're tying this conference together. You're either going to tweet hashtag IBM interconnect or you're going to tweet a new way to work or any number of other hashtags. So then people that are here and they're looking for that social conversation, they can use that as the anchor through which they can listen to people that they don't know. They're not their friends, but they can see this in their social streams. But then it also helps, I think, people that aren't here. Maybe they couldn't make the trip to be able to listen in from wherever they are in the world. So I think hashtags are great. And I think that until there's a better way to come up with something that's a unique anchor and identifier for individual topics in a social setting, hashtags got to be it. Yeah, and also it's a great way to get the backend data on a topic and everything like that. Until my mom goes and wants to figure out what a hashtag is. And she's like, what's that little pound sign right there? And how do I do that? Yeah, it's a new thing. We're all getting on board as well. A new fangled thing. Yeah. So what are some trends, some other trends that you're seeing in social media in 2015? There's going to be more of it. The platforms are changing. Instagram is huge right now for interaction. Facebook is always extremely dominant. I mean, what's new in 2015 is just that there's so many new voices, whether it be the Snapchats of the world or the next thing that the kids these days are grabbing onto that we won't find out about until it's not cool anymore. There's always something new. And it's my opinion that as a business or a company or whatever the case might be, it's so easy to get lost in trying to be all things to all people that you really need to focus on doing a few things really well. Because if you dilute your attention to too many different areas, you're not going to do anything well. Awesome. Well, great advice. Any last minute tips for people that are getting that are new to social media and just trying to build their followers and build their brand? There's a lot of great social media people here this week. And I always try to listen to other people that are smarter than me, which is just about everybody. So I like to learn from people. And one of the main takeaways that I picked up on is be yourself and be authentic and use your own voice. Don't try to be something that you're not, because people can see through that really quickly. And whatever you find passionate in your daily life, put that on social media. Because chances are there are people that might think that your point of view is really cool and really interesting and might think that there's some benefit to it. Awesome. Well, thank you so much. It was great talking to you, Tim. And I'm Ariana Grato, and you're watching theCUBE.


Hi, welcome to APS 1060 Seconds. So in this episode I'll be focusing on doing referencing for the books. So how do you do it? Simple, just start with names followed by year, followed by the title of the book, followed by the place of publication, and then the publisher's name. So make sure you pay attention to the name because you have to put the initial as well. And then the title of the book, we don't capitalize every single word, we only capitalize the first letter of the first word, followed by the place of publication. Now if you're using books published by publishers in the US, you will have to include the city and also the state. Now for the state you have to use the abbreviation, which means you have to use like for example New York state, you don't put New York, you have to put N-O-Y. Other parts of the world you can use city, state, or even state country depending on how you want to include that example Beijing, China.