Can the Biomedical Research Cycle be a Model for Political Science?

In sciences such as biomedicine, researchers and journal editors are well aware that progress in answering difficult questions generally requires movement through a research cycle: Research on a topic or problem progresses from pure description, through correlational analyses and natural experiments, to phased randomized controlled trials (RCTs). In biomedical research all of these research activities are valued and find publication outlets in major journals. In political science, however, a growing emphasis on valid causal inference has led to the suppression of work early in the research cycle. The result of a potentially myopic emphasis on just one aspect of the cycle reduces incentives for discovery of new types of political phenomena, and more careful, efficient, transparent, and ethical research practices. Political science should recognize the significance of the research cycle and develop distinct criteria to evaluate work at each of its stages.

A research cycle in a scientific discipline is constituted by researchers working at various stages of inquiry, from more tentative and exploratory investigations to the testing of more definitive and wellsupported claims. As a particular research area matures, scientists are less frequently surprised by new phenomena because core processes are well understood. They are more likely to focus their efforts on making precise estimates of causal effects, often through randomized experiments. And indeed, such a pattern is evident in biomedical research. In fact, descriptive and correlational work is often published in the major biomedical research journals, although different criteria are used to assess their significance than are used to assess experimental research.
In this essay, I consider the value of this model for political science. My motivation is a sense that political scientists may be paying disproportionate attention to studies that focus on the precise estimation of causal effects to the exclusion of other types of complementary research-so much so that the range of questions to which we can eventually apply sophisticated strategies for causal inference may become severely limited, curtailing our collective contributions to useful knowledge. Put another way, might we be able to answer a greater number of causal questions, with respect to a richer set of subject areas, if we created more intellectual space (particularly in leading, peer-reviewed journals) for high quality scholarship that does not make strictly causal claims-or that draws more tentative conclusions about causal relationships? Would scholars be more likely to accurately describe the nature of their contributions if they were not under pressure to report their findings as "causally well identified?" Specifically, I highlight the need for the type of research cycles and division of labor 1 one sees in other scientific fields, including the biomedical sciences. The notion of a research cycle that I have in mind is one that is constituted as a scholarly conversation through peerreviewed publications, and includes a mix of inductive and deductive theorizing and observation. It explicitly recognizes differences in the state of research within a substantively-delimited cycle, such that we might expect a move from more tentative to more definitive claims about causal relationships. It is a cycle because we rarely expect research to "end," but merely to generate new observations and surprises that will spur new inquiries. Within a discipline that takes the notion of research cycles seriously, the criteria for what would constitute a contribution to knowledge depends on where in the research cycle an author was attempting to publish. More exploratory research could be recognized as "cutting edge" if it were breaking open new questions through identification of novel patterns or processes. Large-scale randomized controlled trials (RCTs) would be more appropriate as a cycle matures, and could provide definitive evidence about more narrowly-defined questions about specific causal relationships.
My point is not to eschew interest in causal questions or causal relationships, or to challenge the potential value of experimental research in political science. On the contrary, I raise the analogy to biomedical science as a science thatgiven its immediate practical search for knowledge to improve human well-being, and recognition of the harms associated with faulty inference-is seriously concerned with establishing clear cause-and-effect relationships. But as I will illustrate, causal analysis is just one part of the division of research-oriented labor.
In the remainder of this essay, I begin by describing the manifestation of the research cycle in the publication of biomedical research, and highlight the extent to which an analogous cycle seems far more limited in political science, at least within leading publication outlets. 2 Subsequently, I propose a framework for developing such a research cycle in the discipline, detailing some standards of excellence for evaluation and publication.

The Biomedical Research Cycle
Biomedical research and political science differ along many key dimensions such that one might question the utility of using the former as a model for the latter. A great deal of biomedical research is rooted in a clear and focused mandate to try to develop best practices and new technologies for improving the health of humans, and much research drives product and protocol development. By contrast, a much smaller share of political science research is intended for "practical" ends, as most scholars search simply for deeper understanding of the sociopolitical world in which we live, and generally have more modest expectations about how research might be used for practical purposes.
In terms of occupation, many biomedical researchers have responsibilities as clinicians, which complement and inform their research, whereas only a limited number of political scientists simultaneously work in the political or policy arena-though increasingly, many collaborate with "implementing partners." In turn, the resources associated with biomedical research are exponentially larger than what is available for social science. While the findings from much biomedical research are embargoed until published (in a timely manner) by a journal, most political science research has been discussed and widely circulated long before it is published in journal form (and very rarely in a timely manner).
Nonetheless, I believe that the biomedical research cycle offers insights for social scientists that are worth considering, particularly for those who seek to make greater progress in gathering evidence that can contribute to knowledge accumulation and in some cases, practical knowledge that might have policy-related or other relevance. Of course, I recognize that not all political science research is advanced with such aims, nor with great optimism about knowledge accumulation, and the intended lessons of this essay simply do not apply to such work.
While the leading scholarly journals of any scientific discipline are not democratic reflections of the interests and priorities of all who participate in the field, they are intended to be the most important outlets for scholarly research. Publication in such journals is rewarded through likely impact on the field and individual professional promotion, and thus, what is published in such journals helps shape the research agenda of many in the field. So it makes sense to focus one's attention on what is published in such journals.
Arguably, the most important scientific journals of clinical medicine are the New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), The British Medical Journal (BMJ), and The Lancet. These outlets provide a lens onto what is considered the most substantively significant research in that field.
For example, consider the July 30, 2015 issue of the NEJM. Not surprisingly, it includes an article on a large scale RCT: the authors report the findings from a study of the effect of hypothermia on organ donors. And the article demonstrates exactly why randomized studies are so valued: We would be extremely unsatisfied with a study that simply reported a correlation between hypothermia and health outcomes following transplants because we would always wonder, what was it that caused some doctors or hospitals to implement this protocol and not others? Deliberate randomization does a great job of addressing potential confounders (selection effects and omitted variables that might affect health outcomes), and at the moment, we lack a better strategy. The article reports that an interim analysis revealed that the differences in health outcomes were so profound that the experiment was "discontinued early owing to overwhelming efficacy." Treatment was associated with much better health outcomes and given the research design, it is quite reasonable for us to infer that those outcomes were caused by the fact that they received treatment.
Political scientists could understandably envy the clarity and significance of such results. Just imagine running a field experiment on foreign aid and development and finding that a deliberative discussion treatment led by village elders helped to produce a 30 percent decrease in funds leakage, and the World Bank and USAID insisted that the experiment be terminated early because of the demonstrated efficacy of the intervention! Indeed, this is the type of solid scientific evidence that many modern social scientists aspire to produce, and I believe it fuels the enthusiasm around impact evaluation research.
For many of us who read only the big headline stories about new medical breakthroughs and who speak frequently about treatment and control groups, it would be tempting to imagine that the leading biomedical journals are themselves outlets for just this singular (experimental) type of research, and if we want to accumulate knowledge that this is all we should be doing. But in fact, if one reads on, one quickly sees that many other types of contributions reach the peak of this scientific community.
For example, in the same issue of the aforementioned article, we find purely descriptive articles. One epidemiological study (an important sub-field that routinely acknowledges its limits to make strict causal claims) reports on the incidence of pneumonia in two American cities; another reports on the characteristics of a previously undocumented cancer variant. In neither case do the authors advance any real causal claims, but they do provide a rich set of analyses of important outcomes of interest, using their scientific expertise to accurately describe these phenomena.
Also of note, the journal reports not simply "definitive" RCTs, but early-stage research findings. As is well known, biomedical researchers classify "phase 1" studies as proof of concept exercises in which a new drug or regimen is tested in a very small group of people, and "phase 2" studies as experiments conducted with larger groups of people to test efficacy and to further evaluate safety before risking the cost and expense to many more people. "Phase 3" trialslarge-sample experiments on patients-are conducted only once prior research has demonstrated safety and efficacy. Because of stringent ethical rules around biomedical experimental research, large-scale RCTs are frequently not possible without clearing preliminary hurdles. But the results of earlier studies are still considered important scientific contributions in their own right: The aforementioned NEJM issue included a phase 1 study of a new tumor treatment with 41 patients; and a randomized, double-blind study of 57 patients in a phase 2 trial of a drug to reduce triglyceride levels.
Finally, there was a very short article with an image of strawberry tongue in a child that provides a clinical observation. That article was just one paragraph long, and highlighted that this observable symptom was used to make a clinical diagnosis of Kawasaki's disease.
What makes this disparate set of articles evidence that a research cycle is at work? And what is the relevance for political science? In this one issue of a leading medical journal, we learn about a range of very different discoveries from problem identification all the way to the test of an intervention that would modify the outcome of interest in a predicted manner. Each is novel, but with different levels of uncertainty about the nature of patterns and causal relationships. At one level, we read of the most basic description-presentation of a visual image to aid in a diagnosis or classification, to some correlations, to some tentative theories about the effects of a new experimental protocol, to a late-stage RCT. Each is deemed a sufficiently important advance in thinking about substantively important problems. While no one would claim that a very brief case report would provide any deep answers to broader scientific questions of interest, as Ankeny argues, clinical studies published in journals have provided an important foundation for the development of diagnostic (conceptual) categories, and the formulation of key research questions and hypotheses. 3 The most famous example is the 1981 publication of case reports that turned out to be observations of what is now known as the acquired immunodeficiency syndrome or AIDS, and indeed, case reports comprise an important share of the aforementioned journals. 4 Moreover, when reading the late-stage experimental study, we see that it makes reference to a retrospective study. And the very differentiation of phased trials implies a step-wise, but cumulative path toward discovery.
Turning to other clinical journals, such as JAMA, BMJ, or The Lancet, a similar pattern is clear. And this applies also to other high-impact multi-disciplinary journals such as Nature and Science. To be certain, not all biomedical research journals are as eclectic in the types of research published, and of course I have not established here the actual influence of scholarship on particular research efforts within a scientific community. Nonetheless, what is clear is that leading scientific outlets clearly publish across the research cycle, and causal research is frequently strongly rooted in prior published studies documenting important discoveries.
And while the focus of this essay is empirical research, it is worth highlighting that within the leading biomedical journals, normatively-oriented scholars frequently play an important role in various steps of such cycles, by commenting on the implications of particular sets of research findings, or by highlighting the need to focus more on particular questions. For example, following the publication of research demonstrating the effectiveness of HPV vaccines, and the subsequent FDA approval of the vaccine, an ethicist from a school of public health published a brief analysis of the ethics and politics of compulsory HPV vaccination in the NEJM. This article sheds light on a range of important considerations that will strongly mediate how scientific discovery actually affects human health and well-being, but in ways that were surely not explicitly discussed in most or all of the earlier stages of the research cycle. Normative analyses routinely appear in leading medical journals through editorials and "perspectives" pieces, which help to address the ethical dimensions of research practice as well as clinicaland policy-related developments. In short, normative work is tightly linked to the empirical research cycle.
Before turning to political science, I don't want to leave the impression that the biomedical research community is of one voice on issues of causal inference. I think it is true that biomedical researchers are generally extremely hesitant to assign causal attribution to any observational study. Rather than dismissing such work, findings are published as "associational" relationships. And in turn, policymakers and clinicians are made aware of such findings, but are repeatedly reminded to apply such findings with great caution as healthy skepticism about causation remains. That said, within certain circles of the biomedical research community, one can find similar types of methodological debates as those we engage in political science. For example, one pair of nephrologists has recently written of their frustrations with a research paradigm that does not allow for causal attribution except in the context of an RCT. 5 They argue for greater appreciation of techniques such as an instrumental variable approach!

The (Near) Absence of a Research Cycle in Contemporary Political Science
In practice, political science research already proceeds as a mix of inductive and deductive research. That is, scholars (often in their everyday clothing as civilian observers of the political world, sometimes as consultants or research partners, or as part of the early stages of their research) come to observe new phenomena that disrupt their view of the status quo, sometimes against prior theoretical expectations. In turn, scholars describe what happened, come to theorize about the causes or consequences of such phenomena, often through observation of patterns studied formally or informally, develop causal propositions, and provide evidence testing those propositions with various types of data and analyses.
And yet, what is very distinct in political science from the biomedical model I described above is that most of those steps are very rarely publicly recorded as distinct scientific enterprises. In fact, increasingly, only the last set of studiesthose that test causal relationships, especially using evidence from research designs that explicitly avoid threats to causal inference from confounders, and designed to accurately detect null relationships 6 -are the ones that get published in top political science journals. 7 Many of the other steps are described in a cursory manner and barely find their way into the appendices of published work. In other words, it appears that increasingly, the only types of contributions are those associated with the "final" stages of the research cycle.
For example, if one looks at the eight empiricallyoriented articles of the May 2015 issue of the political science flagship journal, the American Political Science Review, all eight sought to provide a fully worked-out theory; most were explicitly testing causal models. Only one was an experiment, while six used statistical analyses to analyze large datasets; and one used case studies. But in virtually all of these articles, the authors largely say or imply that they are providing the best answer to a causal question with causal evidence. My point here is not that they were all quantitative or all experimental, because there was actually a bit of diversity on those dimensions.
The vast majority of political science articles at virtually all of the top journals and the papers presented by ambitious graduate students in search of academic jobs are increasingly of a single type: claim to provide a new theory, specify some hypotheses, test it with analysis of a large dataset, frequently associated with an experimental or quasi-experimental research design, and on occasion, explore with a few case studies. A great deal of this work is excellent and in many ways, has provided much more reliable knowledge than what was published in prior generations. More and more political scientists have turned toward design-based experimental and quasiexperimental research, and the bar for what should be trusted as causal evidence has certainly been raised. As a discipline, we have developed a heightened appreciation for the range of confounders that limit our ability to infer causal relationships even when presented with strong statistical associations. And in turn, more applied researchers have focused on implementing "wellidentified" designs, lest they be challenged for overclaiming the fit between evidence and theory. Excitement over a range of new strategies for making causal inferences has implied greater attention to such work in leading political science journals and in the profession more generally. Clearly, these are largely positive developments.
But alongside this trend, Gerring documents the virtual disappearance of descriptive studies from the leading political science publication outlet and indeed, part of the problem is that scholars are not particularly interested in carrying out "mere" description. 8 Moreover, the unspoken presumption that the best work ought to be confirmatory or a test of an ex ante specified hypothesis, rules out the honest publication of findings of surprise patterns. While increasing calls for the public registration of pre-analysis plans are aimed to keep professionals "honest" by limiting post-hoc findings being reported as if they were confirmatory, such efforts may inadvertently devalue the potential importance of strong and surprising inductive, accidental, or post hoc findings that shed light on big questions.
Moreover, the explicitly normative portion of the discipline-what is generally referred to within departments and the discipline as "Political Theory"-with some notable exceptions, largely operates and publishes in isolation from its more empirically-oriented counterparts. While the topics of democracy, violence, public goods provision, identity formation, and the like do largely overlap, true integration within research cycles is largely absent. One rarely finds theorists citing or commenting on the latest empirical research, and one almost never finds empirical researchers discussing more contemporary normative research.
And finally, the research that seems to be disappearing most quickly from the heights of the discipline are those studies that fall in between pure description and very strong causal inference. What biomedical researchers would describe as correlational studies, such as retrospective cohort studies, are like kryptonite to aspiring young scholars who have a good sense of how such work will be judged-irrespective of the potential substantive or theoretical importance of some such studies. We provide very little space for "tentative" or "suggestive" findings, insisting that research ought to be definitive, or at the end of the research cycle.
In many ways, I share the view that the focus on improving the quality of causal inferences marks an important and positive development in the discipline for both quantitative and qualitative research. We should not go back to the times of interpreting any old significant regression coefficient as evidence of a causal effect. But it is also worth taking a step back to consider what it might mean for disciplinary practice and output if the only studies that are highly valued are the ones that can unambiguously demonstrate random assignment to treatment, allowing for more certain identification of causal effects. What are the implications for the types of questions that might (not) get asked? What does this imply about the efficient allocation of resources, and transparency in research? Are there lessons to be learned from the biomedical paradigm described above?

Costs: The Crowding Out of Discovery-Premature Experimentation
If we are ultimately interested in causal questions and causal evidence, shouldn't we focus our attention on research that identifies causal effects? If as a discipline, we lack a large body of definitive scientific findings, shouldn't we play "catch up" by gatekeeping out the types of more tentative and ambiguous research that simply leads to endless debate about model specification and the like?
In fact, I believe that there are several important costs in terms of the potential discoveries that are not incentivized because they are not appreciated, and the potential misallocation of our human and financial resources toward experimental and quasi-experimental research, not all of which is as promising as it could be. What we might call "late-stage" RCTs are (generally) extremely expensive in multiple ways: They often involve substantial burdens on human subjects in terms of time for participation or enumeration, they can be very expensive to administer from a data collection standpoint, and if there are ethical implications, these tend to be multiplied on a large scale, all because experimental analyses require analytic power, which for most social science experiments (which tend to have relatively small treatment effects) implies large sample sizes.
In the biomedical sciences, owing to the very clear threat to human life and well-being of ill-conceived treatments and protocols, phased research is generally required for research with human subjects. As discussed earlier, early stage studies tend to be smaller in scale, and look more holistically at possible secondary and unanticipated interactive effects. For example, an adverse outcome within a subset of treated subjects would demand a retrospective analysis of differences, and an inductive, post hoc analysis of the predictors of heterogeneous treatment effects. Such exploratory study can be usefully carried out within the context of a smaller-scale experiment such that the findings, if deemed relevant, can be implemented in the design of subsequent, larger-scale studies.
But political scientists generally lack the equivalent opportunities to publish phase I or phase II trials. At the very least, we lack a shared understanding of the role that such work might play in a larger research cycle. Nonetheless, most ambitious field experiments ought to begin with some degree of piloting and qualitative research, including, for example, more open-ended focus group discussions and interviews with subjects. 9 Owing to costs and uncertainties, such pilot experiments are, by definition, not at a scale that allow sufficient statistical power to reach definitive answers to causal questions. The question is, should such studies form part of the "official" research cycle, in the sense of being published? Or should they remain part of the internal analytic support that largely remains hidden until the "full" study is completed? I advocate the former. At the moment, political scientists might exercise the option of writing a blog entry about their findings, but this clearly winds up being a temporary, insiders' outlet, and particularly for young scholars, provides little professional reward. The lack of a peerreviewed outlet reflects the low value such findings are currently ascribed.
Absent any obvious outlet to publish such studies in political science, most political scientists will find little incentive to conduct such work or to take it as seriously as they should. Rather, they are more likely to "go big or go home," in pursuit of results that limbo their way under the conventional p 5.05 level of statistical significance.
Even before conducting early-stage experimental research, good scientific practice would demand that we at least try to establish plausible connections between variables with existing or non-obtrusive data. And yet, in the leading political science journals, it is increasingly rare to find an observational analysis that simply reports important and robust associations unless the author has identified some "as-if random" natural experiment and can use some type of naturally-occurring discontinuity to infer causation. Now, of course, we would rather find an interesting natural experiment if the costs in terms of external validity are not too great. But sometimes, this is not possible, especially for analyses of large-scale, macrolevel processes. And why are retrospective observational studies not still valuable if scholars are honest about what they can infer, demonstrating they have made the best attempts to answer their research questions with available evidence (or all evidence that could be reasonably gathered)? Shouldn't predictive patterns provide some initial confidence that a causal relationship may exist? Correlation does not mean causation . . . but it certainly can be suggestive of an important piece of evidence in support of a causal relationship. If we consider again the biomedical model, the first studies that found an association between smoking and lung cancer were hardly definitive, and we still would not run a randomized study to test the direct causal relationship. But the significance of the finding cannot be overestimated, particularly as scientists have concluded with mechanistic evidence (and without experimentation on human subjects), that smoking causes cancer.
And yet, for young social scientists-the ones most likely to be making new and creative discoveries, and perhaps the least well-positioned to be raising vast sums of money for large-scale experiments-increasingly "causal identification strategy" is the only name of the game. And if they are not implementing proper experiments, they are seeking out causal research projects through "natural experiments." 10 That is, they search for the perfect "plausibly exogenous" instrumental variables such as rainfall or other arbitrary cutpoints and decision-making rules. And de rigeur, they are expected to proclaim that their particular strategy is "novel," and/or a rare "exploitation" of an untapped inferential resource.
To be sure, many such studies are exceptionally creative and valuable. And the sometimes quite clever identification of naturally-occurring experiments is a feat that deserves proper accolades and professional rewards . . . But if the proverbial tail is wagging the dog-that is, if researchers wind up looking for outcomes to study because they finally stumbled upon an exogenous source of variation of "something" that ought to be consequential-well, that seems not to be the basis for a promising or coherent research agenda. There may be undue temptations for false discovery-i.e., "I've found something exogenous, now let me try to find some plausible outcome that it can predict," in which case we may wind up with the same types of spurious associations that experimentalists have been trying to avoid. (I discuss the potential use of pre-analysis plans later in the essay.) Moreover, I think that many will agree that way too many recent social science papers are making overly-heroic claims that particular choices or events are plausible instruments and that they meet the necessary claims to make causal inferences. 11 I suspect that if our vision of good science explicitly allowed for "tentative," "suggestive," or "predictive" findings, we would see less over-claiming about the strength of causal evidence.
The increasing focus of talents, energies, and professional rewards on causal research per se poses several additional costs.
First, it likely obscures timely documentation of potentially important new descriptive discoveries, at least by political scientists, with the skills and insights they could bring to such research. Such descriptive analysis ought to be both an end in itself, and also a gateway to other types of observational, and experimental studies. 12 Along these lines, the discipline has turned away from a legacy of description at potentially great cost. Of course, it is not possible to account for the studies that might have been written and published had they been properly incentivized, but at the very least, we can say that much has happened in the political world in recent years . . . and political scientists have documented very, very little of it, at least in our leading journals! Perceptions of disciplinary norms and expectations weigh heavily and are self-reinforcing. For example, today, if I had a graduate student who was working in rural Zimbabwe and identified some new form of interest articulation, or deliberation that we had never seen before-let's say that they had developed a pattern of decision-making in which they decided that all of the children in the village got to decide how to manage the local budget-I am fairly sure that the only way in which that graduate student could get that observation published in a top journal would be to figure out some way to observe tons of variation in the manifestation of that institution, and then to develop a theory of its causes or consequences, and then, test that theory by identifying natural randomness in the causal variable, or to run an experiment in which the institution itself was randomly assigned. And if that graduate student, who found this extremely interesting new aspect of political life that we had never seen before could not do all of these other things, I would need to, in good conscience with respect to that student's professional prospects, advise dropping this project immediately. Maybe the student could publish in some obscure area-studies journal, but definitely not in a political science journal.
In a similar manner, if a political scientist had been able to rapidly conduct a survey of social and political attitudes of Cubans just after the thaw in U.S.-Cuban relations, it strikes me that we would want to document such attitudes, to do it with the best social science skills available, even if the research had no ambitions of being able to detect specific causal effects. Whether Cubans favored political reform or not-the answer, particularly the distribution of responses, would be intrinsically interesting-and that piece of research could generate deeper inquiry about the causes or consequences of such sentiment. But in the near-term it would be a truly significant contribution to knowledge, simply as a piece of descriptive inference.
The point is not that political scientists should be reporting the news. They should be using their conceptual, analytical, and measurement skills to describe patterns and phenomena about contemporary and historical political life that would otherwise go unrecognized.
At the moment, however, apart from making valuable contributions to blogs such as the extremely popular and successful Monkey Cage, scholars are not incentivized to use their sophisticated tools to describe what is going on because again, we do not reward those contributions with our central currency: publication in peer-reviewed journals. Moreover, non peer-reviewed blogs are not intended for in-depth scholarly studies, and they do not provide an opportunity for disclosure of research methodologies, uncertainty of estimates, etc.
By contrast, in the biomedical sciences, when a new set of life-threatening or otherwise critical symptoms present themselves, particularly in a patterned manner, one can be certain that such discoveries will be reported in a top journal with the expectation that future research will be needed to understand the causes and consequences of such discovery, and to develop interventions to prevent or to treat those symptoms. As discussed earlier, the New England Journal of Medicine published an article describing a strawberry tongue, in effect communicating, "Hey, this is important, take a look. More later." I believe this state of affairs dis-incentivizes novel discovery, and incentivizes work within a narrow band of research in which processes and measures are already well understood. It is true that much of "normal science" involves small and marginal revisions and even just replications of prior studies. Such work deserves an important place in the discipline. But there also needs to be a place for examination of previously unexamined phenomena even if the causal connections are not fully worked out. In recent years, many graduate studentsincluding those who have been extremely well trained in the best methods of causal inference-have confided in me that they feel "paralyzed" by the emphasis on causal identification and close out certain types of research questions very quickly because they don't believe that they will eventually be able to estimate a causal effect in the manner that they perceive the discipline now expects.

Toward a Framework for a Research Cycle
If the concerns about the need for a publication and professional opportunity-incentivized research cycle in political science are valid, what is to be done? Importantly, I think we need to distinguish and to label the different types of contributions scholars might make, and to establish standards of excellence for each. (Although in the discussion provided earlier I identify an important place for normative research in the cycle, I do not include here a discussion of standards for such pieces. Normative contributions might be made at any stage of the cycle.) Not all journals will want to publish pieces from all stages; and the specific contributions of any piece are likely to be unique and subject to scholarly tastes and concerns. Nonetheless, authors, reviewers, editors, and readers should identify quite explicitly where in the research cycle any given study is likely to fit, and thus, how to evaluate the nature of the contribution. Our expectation should not be that every paper would tackle every concern within a substantive research agenda, but that it will take its proper place within a larger division of labor.
I follow Gerring's "criterial approach" and an appreciation of tradeoffs in research as a framework for making distinctions between types of studies (refer to table 1), but with a focus on the research cycle. 13 A key tenet of good social science research is to avoid "over-claiming." That is, do not attempt to draw conclusions that your data cannot support. But if we are going to provide a framework for honest research, we need a greater diversity of the types of claims that we might make, and associated standards for excellence and importance. What is critical about the notion of a research cycle is that we ought to value new contributions based on what has come previously within that substantive area of research. This, of course, places a particular burden on scholars and reviewers to be cognizant of what has and has not been learned in an area of research, and to properly frame contributions with respect to such background. While this might seem obvious, I think it is a point worth emphasizing in order to guard against the simple application of a single set of standards (i.e., what is the strength of the causal identification strategy?) to all scholarly work.
I will describe several broad types of studies and contrast them in terms of the nature of the claims they make and how they might be evaluated based on the novelty of the descriptive or causal theories associated with the claims, the strength of association or effect size, the additional credibility associated with a publicly registered pre-analysis plan, and other considerations for evaluation. In all cases, high-quality measurement of constructs is a prerequisite for excellence: if constructs are not properly measured, no results can be considered trustworthy.
In each case, our criteria for a "significant" study should be to disrupt some aspect of prior knowledge. Critically, however, not all studies can or should contribute along every dimension.
A descriptive study in political science ought to use the best-available conceptual, measurement, and sampling tools to depict a phenomenon of interest. What are citizens' political attitudes? How have certain institutions evolved over time? In order to be considered important, such studies generally need to focus on a subject that is Value of exante public registration of propositions (i.e., preanalysis plan)?

Observational Descriptive
To describe novel or unexpected phenomena, including variation within a population.

Associational/ predictive
To demonstrate a novel and robust pattern potentially consistent with a new or existing theoretical proposition.

Natural experiment
To estimate a specific, predicted causal effect, using a naturallyoccurring, but plausibly randomlyassigned treatment.

Less important Important Critical Limited
Experimental Early-stage experiment To assess the plausibility of a specific causal effect and other possible (adverse) effects, using investigator randomization as identification strategy.

Late-stage experiment
To estimate a specific, predicted causal effect, using investigator randomization as identification strategy.

Least important Least important Critical Critical
truly novel or that disrupts conventional wisdom about a particular state of affairs: for example, documenting either a new type of institution or set of political attitudes or behaviors, describing some aspect of political life in the wake of an important historical moment, or showing that a particular way of understanding some existing phenomenon is no longer correct, given superior data or measurement techniques, which in turn might cast some existing causal theories in doubt. These are akin to the biomedical case studies or studies that simply describe the prevalence of a particular disease in different locations, perhaps reporting on associations, with no claims of estimates of causal relationships. The field of epidemiology provides critical insights for the biomedical sciences more generally by offering careful description of the pathogenesis of disease. In a similar manner, political scientists could and should be making important and methodologically sophisticated contributions by describing the prevalence and variance of key political phenomena. And with the advent of "big data," I expect that many such contributions will be advanced along these lines. In their seminal work, King, Keohane, and Verba discuss descriptive inference at length, but that part of the methodological treatise is routinely ignored. 14 An important descriptive study must demonstrate an outcome or pattern of interest that was not previously observed or expected, and such findings should open up new areas of inquiry within a research cycle. Descriptive studies may be retrospective (tapping existing observational data) or prospective (for example, planned surveys). Fundamentally, these studies must be judged in terms of whether they demonstrate something that is truly new and if they are carefully measured or implemented. Beyond description, analysis of observational data of naturally-occurring phenomena can be used to detect patterns and the strength of relationships among variables. Within such studies, political scientists will make claims about the extent to which relationships might be interpreted as truly causal, providing not simply statistical or qualitative assessments of uncertainty in the strength of relationships, but additional discussions of the credibility of the research design, and the ability to address rival explanations. All studies, of course, face the "fundamental problem of causal inference," which is that we cannot know for sure what the counterfactual outcome would have been if particular units had received different values on the explanatory or treatment variable. 15 Most nonexperimental studies exhibit a set of hallmark limitations in this regard: we do not know for certain the process by which treatments were assigned and if the selection criteria were potentially biased in a manner that is correlated with the outcome of interest. Thus, the onus on retrospective studies trying to advance causal claims is to show that a wide range of other rival explanations are not driving the results. In turn, much scholarly attention focuses on the credibility of causal inference depending on the "identification strategy" or "identifying assumptions." Indeed, some research designs do, in practice, appear to provide more credible estimates of causal effects because they have found a way of leveraging some phenomenon that is "plausibly" random. For other studies, more questions remain at the conclusion of the study concerning whether the key treatment or explanatory variable was truly exogenously assigned, and given that uncertainty, it is difficult to conclude that any estimated relationship reflects a causal process. In table 1, I distinguish those studies that can credibly claim to be leveraging a true natural experiment from those that do not, labeling the latter "associational/predictive" studies.
And here is the fundamental rub: if we cannot be convinced that the treatment variable is truly exogenousif we are always left wondering whether some omitted variable has confounded the results-can we really believe that the research output is significant and worthy of publication at a top journal or is the basis for professional recognition?
My answer is that strength of causal identification strategy should be considered as just one criterion among several. And again, this is where I think the notion of a research cycle sheds important light on how to evaluate a contribution. In the early stages of a research cycle, we might heavily weight the extent to which the estimated relationship between variables represents a novel and theoretically innovative association, and the extent to which the demonstrated strength of that relationship is substantively significant. Such associations might be demonstrated through careful model-based statistical analyses or (comparative) case studies.
By contrast, in the latter stages of a cycle, particularly if a strong predictive pattern has already been empirically demonstrated, we should hold studies to standards that more credibly detect causal relationships with less tolerance for potential confounding. Specifically, here we should expect research that does a better job of approximating a "natural experiment," and we would expect to see, for example, regression discontinuity designs, effective use of instrumental variables, or difference-in-differences designs, which might more directly address the threat of confounders to causal inference as compared with a more straightforward regression or matching approach to analysis. 16 In an analogous manner, qualitative research at this stage in the research cycle would need to reach a very high bar of addressing potential confounders with explicit evidence. To the extent that researchers develop strong and credible causal research designs for testing wellmotivated causal claims, we should be less concerned with the extent to which effect sizes are small or large as a criteria for publication or for professional merit more generally. We will need to depend on scholars to adequately frame the nature of the contribution and for expert evaluators to assess the particular contribution relative to prior work.
Moreover, at the earlier stages of the cycle, the correlational study or its qualitative analog ought to be theory-motivating. In turn, if observed correlations are weak, or if the case study research finds no clear pattern or logic, the contribution is ambiguous and almost certainly not worthy of publication. On the other hand, at the latter stages of a research cycle, when expectations about a theory are greater, a research design that more credibly isolates the effect of X on Y ought to contribute to knowledge irrespective of the actual findings. The better the test (i.e., the less likely the research design is to report a null result when a causal relationship actually exists), the less we should be concerned about the specific results as a criterion of scholarly review. Of course, substantively large findings will always be more likely to gain more attention, all else being equal, but that is a separate issue from scientific merit.
Finally, there are experimental studies in which assignment to treatment is randomized by the investigator. Building on the biomedical paradigm, I propose that political scientists would be well served to distinguish between early-stage and late-stage experiments: Early-stage experiments should be designed explicitly as way stations for larger-scale, costlier experiments, particularly when little experimental research has been previously conducted in this research area. While social scientists are currently not expected to adhere to phased research standards akin to clinical trials, in many circumstances there would be great value to such practice. Because early-stage studies are, almost by definition, underpowered (there are not enough subjects or observations to confidently "fail to reject the null hypothesis"), the criteria for publication or contribution to knowledge should not be the magnitude or statistical significance of estimated effects. Rather, an article reporting on an earlystage experiment ought to provide deeper insights into the fit between treatment and real-world or theoretical constructs, to discuss ethical implications of the experiment, to highlight qualitatively observed processes that link (or impede) the relationship between treatment and outcome, and offer the specifics of an innovative experimental protocol. The criteria for publishing articles that document such studies is the extent to which the analyst provides strong evidence to motivate or to discourage large-scale experiments with the same or a related protocol. Through description of preliminary results, description of focus-group or other interviews, and detailing of other observations, such articles can more definitively assess the promise of carrying out potentially difficult and costly research, even if the estimates of causal effects are more tentative.
By contrast, late-stage experiments should be judged to a much greater extent in terms of the extent to which they provide unambiguous tests of the effects of X on Y. By definition, they should not be underpowered, which makes them uniquely suited for drawing conclusions about null relationships. But beyond that, experiments can be judged on the extent to which they are implemented in a manner that fully addresses potential confounders in as efficient a manner as possible. Articles reporting on large-scale, late-stage experiments should not be judged primarily on theoretical innovation or novelty of association: such novelty ought to be established in less costly ways, earlier in a research cycle. Instead, late-stage experiments ought to be clean and definitive tests of wellmotivated hypotheses. If social scientists (and funders) were to take the notion of a research cycle seriously, they would not carry out expensive or potentially unethical experiments in the absence of one of the earlier studies providing strong suggestive evidence of the merits of the hypothesis under examination.

What Role for Registration / Pre-Analysis Plans?
A welcome trend that has already been imported from the biomedical sciences to the social sciences is the practice of public pre-registration of design protocols and analysis plans prior to the fielding of experiments. In the biomedical sciences, this has been an important corrective to the burying of null results and post-hoc finding of "positive" results obtained from "creative" re-analysis of data.
Although a full discussion of the merits of pre-analysis plans is beyond the scope of this essay, it is worth reflecting on their potential role within the context of a research cycle. The goal of pre-analysis plans is to keep scholars "honest" and to avoid "p-hacking"-the search for results that accord with conventional thresholds for statistical significance through some combination of variables in a post hoc manner, after predicted findings were not attained. This is a worthy goal, and for a great deal of research, I fully support the use of such plans. 17 Not only should such planning and public registration deter false discovery, but it ought to provide a public tool for justifying prospective research in the first place. As I argue in table 1, for late-stage RCTs, such registration is critical and it is difficult to imagine a strong counter-argument against their use for that type of research. Even for earlystage RCTs, scholars ought to pre-register their research designs and pre-analysis plans, but our criteria for the significance of the contribution of a paper should not be as closely tied to those plans as would be the case with a latestage RCT.
A more difficult question concerns the value of preregistration of retrospective and non-experimental studies. On the one hand, for observational research taking place late in a research cycle, pre-registration may indeed provide great value. If scholars publicly registered that they were going to investigate a particular set of archives in a particular way, and predicted a set of patterns with a pre-specified analysis, of course, it would be very convincing and impressive to find those patterns observed in analyses conducted after the data were collected (assuming, of course, a logical theory, sensible data collection, and sound analysis). All else equal, such a study would be more credible than one that did not pre-register hypotheses and analytic strategies.
But again, if we take the idea of pre-registration too far, particularly if we develop norms in which scholars perceive that their "hands are tied" to report only those analyses that have been pre-specified, we will surely crowd out the important inductive work (some call it fishing) upon which scientific discovery depends. 18 Let me return to the (biomedical) example of HIV and AIDS. On the one hand, in the later stages of understanding this disease, science, and frankly, humanity, clearly benefits from scientific practice that insists on pre-registration of trials around the efficacy of drug treatment. We would not want the practitioner community to be confused about what actually works because the only studies available to them were the ones that demonstrated positive results. Drug trial registries help to solve this problem.
On the other hand, let's consider the process of discovery around the important question of what causes the transmission of HIV? This research clearly involved lots of inductive pattern-detection, particularly in the early stages of the epidemic. I recognize that early recognition of the association between sexual orientation and AIDS symptoms generated some awful inductive theories (famously, the Rev. Jerry Falwell declared AIDS was a punishment from God), but also was a necessary pre-requisite for valid scientific discovery of the pathways for transmission. It is difficult to reasonably imagine that such relationships could have been predicted ex ante, or for that matter, hypotheses about the protective benefits of circumcision, but these have proven to be unimaginably consequential discoveries for curbing the epidemic. If gatekeepers in the biomedical community had restricted such knowledge because the research designs were not "causally well identified" or a pre-analysis plan was not on file, one can only imagine how many more lives would have been lost to the epidemic.
Recognizing that registration of studies is not a prerequisite for all forms of important research in the biomedical sciences, political science should avoid being overly restrictive and we should not necessarily value a study more than another on the sole criteria that one was pre-registered. To be more precise, the value of preregistration depends on the type of study and place in the research cycle. In fact, because social and political phenomena are surely much less predictable and mutate more rapidly than bio-physical phenomena, I would argue that much less of our research ought to be constrained in this manner. Specifically, as I outline in table 1, I find only limited value for registration of studies other than prospective RCT's. Where scholars are able to pre-specify research plans with some confidence they should by all means do so. At the extreme, of course, purposive research is better practice than "barefoot empiricism." But particularly at the early stages of a research cycle, we should not expect that scholars will know exactly what they are looking for before they have looked. (That said, they should not claim ex post that they knew what they were looking for when their findings were actually a surprise.) Problem-oriented research starts with puzzles about outcomes, and the search for plausible predictors of those outcomes is necessarily inductive. It is not always easy to judge whether findings from such studies are trivial or spurious or the advancement of real knowledge, but if other scientific programs are any guide, we should not restrict such inquiry wholesale.
For retrospective studies that advance a causal identification strategy involving a "natural experiment," public pre-registration plans could be a useful disciplining device, but their use should not give readers false confidence in the results should they be consistent with predictions. By definition, a retrospective study implies that the events of interest have already occurred, and it is often difficult to imagine that a researcher proposing to study causal relationships in a particular context will not have some prior knowledge of patterns in the data. As such, the finding of consistency between actual results and preregistered hypotheses may not be as powerful as they appear. At the very least, pre-registration of analysis plans for observational data ought to welcome discussion of what has already been observed and analyzed. knowledge if we built stronger scientific foundations with a greater diversity of research techniques and allowance for recognition of different types of claims.
How could research cycles, as described earlier, play a greater role in the discipline? The most important agents in this regard should be the editors and editorial boards of our leading scholarly journals. First, editors could more explicitly recognize a larger range of research contributions within their journals and label them as such, perhaps incorporating some of the language I have used here. Second, they could provide guidelines for reviewers concerning the appropriate criteria to use when reviewing articles with particular aims. Third, we must figure out ways to incentivize a more rapid timeline from submission to publication. It simply will not be possible to use scholarly journals as serious anchors for the accumulation of knowledge if it continues to take well over a year, sometimes longer, between submission and publication for successful pieces.
And beyond the journals, academic departments will need to make clear how they value different contributions in the research cycle as a basis for promotion and tenure. If younger scholars knew that they could advance their careers with different types of contributions, they would be more likely to focus on a wider set of concerns than an almost single-minded focus on strategies for causal identification. In fact, some of the self-monitoring that occurs within academic conferences and workshops might shift to dissuasion from premature experimentation on the grounds I have described.
To be clear, my point here is not that political science should try to look just like biomedicine. Rather, I think that there are some surprising lessons to learn that are worth considering. Academic disciplines evolve according to tastes and norms, and some appreciation of how other disciplines operate may widen our scholarly palates. At the moment, it certainly feels as if we could do a lot better in leveraging the collective research talents that exist throughout the discipline to answer serious questions about the political world. 6 That is, avoiding "Type II" errors, the false failure to reject the null hypothesis. 7 Perspectives on Politics is somewhat unique in this respect, because it publishes a wide range of studies, including research articles that are not strictly concerned with estimates of causal effects. 8 Gerring 2012a. Top journals do sometimes publish purely descriptive articles, but these works almost always make a significant methodological contribution as well as a substantive one. 9 Glennerster 2013; Paluck 2010. 10 Dunning 2012 provides a thoughtful treatment of how more inductive field research can establish a foundation for recognizing "as-if" randomly assigned treatments in natural settings. 11 For example, the exclusion restriction is rarely plausibly met in political science applications of instrumental variable analysis. 12 Gerring 2012a. 13 Gerring 2012b. 14 King, Keohane, and Verba 1994. 15 Rubin 1974 For example, see Angrist and Pischke 2014. 17 See, for example, Humphreys, Sanchez de la Sierra, and van der Windt 2013, and discussion and guidelines at the Evidence in Governance and Politics (EGAP) website, http://egap.org/content/registration. 18 Thoughtful advocates of pre-analysis plan registers have explained that we ought to simply make distinctions between analyses that were pre-registered and those that were not, but to feel free to report both.


extfieldsofstudy: Psychology Medicine

text:
Culturally diverse attitudes and beliefs of students majoring in speech-language pathology

Purpose
Academic education in speech-language pathology should prepare students to provide professional services that mirror current knowledge, skills, and scope of practice in a pluralistic society. This study seeks to examine the impact of speech-language pathology (SLP) students prior multicultural experiences and previous formal education on attitudes and beliefs toward language diversity.

Methods
A survey to investigate SLP students attitudes toward language diversity was applied. After the research study and instructions to complete the consent form questionnaire was presented by a research assistant, an announcement was given by a graduate student who speaks English as a second language with an accent. The participants then completed a questionnaire containing questions related to attitudes about the presentation of the announcement in particular and toward language diversity in general.

Results
Responses suggested a relationship between self-reported cultural bias and ability to concentrate on speech with an accent, and the extent of interaction with individuals from a cultural and linguistic diverse (CLD) background. Additional outcomes revealed that cultural bias may be predicted by factors related to amount of CLD exposure.

Conclusion
Results of this study indicated critical areas that need to be considered when developing curricula in speech-language pathology programs. The results will be useful in determining procedures applicable in larger investigations, and encourage future research on attitudes and beliefs toward aspects of cultural diversity.

INTRODUCTION
When working with multicultural populations, speech-language pathologists (SLPs) should provide services that are culturally appropriate. Increasing culturally competent research and education in the speech-language pathology (SLP) field are important steps to prepare professionals for serving consumers within an ethical environment, infused by shared trust and respect (1) . The United States Census Bureau (2) provides clear evidence that diversity is steadily growing. There are more U.S. residents speaking more languages and dialects of the English language than ever before, and an increasing number of adults learning English as a second language (ESL) (3) . It stands to reason that the ideal SLP education philosophy should consider multicultural issues.
In order to meet the communication needs of a growing multicultural population in increasingly globalized communities, SLPs are challenged to provide culturally competent services. This may not be a simple task, with approximately 90% of ASHA affiliated SLPs identifying with a Caucasian background, as estimated by the demographic profile of certificate holders from the American Speech-Language-Hearing Association (ASHA) (4) . These professionals may lack the understanding of other cultures and may be uncertain about designing assessments for individuals from a diverse background. They must receive adequate education, with multicultural awareness, in order to build knowledge of a variety of culturally appropriate techniques (5,6) . Hence, culturally competent SLPs should be aware of their own values and biases and be sensible to the values and needs of individuals with diverse backgrounds (7) . SLPs must develop a thoughtful and sensitive attitude about cultural background because there are no established solutions for dealing with every situation involving a diverse caseload (8) .
According to ASHA's system of accreditation for college and university graduate programs, academic education in communication disorders should be specifically designed to prepare students for entry into professional practice and provide curriculum that reflects current knowledge, skills, technology, scope of practice, and the diversity of society (9) . However, faculty are often unsure how to develop and implement multicultural competence instruction (10) . Moreover, rather than leading students to become aware of their own stereotypes and beliefs, typical multicultural training and literature tend to focus on the differences of the clients such as diversity of origins and cultural backgrounds (8) . Therefore, students may be trained to acknowledge differences, but they may lack the necessary personal insight to change their own behaviors to correspond with the treatment needs of clients from diverse cultures. Incongruities between the commitment to multicultural training and the execution of producing culturally competent SLPs should be addressed (11) .
It has been suggested that accessibility to cultural and linguistic diversity (CLD) may impact attitudes and beliefs of pre-professional clinicians toward diversity, particularly in light of a previous similar investigation involving students majoring in rehabilitation. The aforementioned study's outcomes unveiled concerns associated with general needs of increasing education and awareness toward cultural diversity in higher education (12) .
Additionally, Kritikos conducted an investigation involving SLPs, concluding that lack of education regarding language diversity resulted in low efficacy in providing services to CLD individuals (13) . Furthermore, Preis revealed that studies involving SLP students suggested that CLD coursework may impact their attitudes and behavior surrounding diversity (14) .
A pluralistic education should encourage future professionals to adjust to the client's worldview (15) . As such, examination of diversity self-perceptions is fundamental in multicultural education, ultimately having an impact on the development of clinical skills (15,16) . Thus, in addition to implementation of preferred models of cultural competence such as infusion, in which multicultural concepts and issues are treated as integral components throughout the entire training program (11) , self-assessment of attitudes and beliefs toward diversity should be a starting point in multicultural education (17,18) . Furthermore, increased exposure to multicultural coursework may result in higher levels of multicultural competency.
This study seeks to examine the relationship between aspects that represent multicultural experiences and attitudes toward language diversity in SLP students, specifically examining attitudes and beliefs of SLP students toward the accents of nonnative speakers of Standard American English (SAE). This strategy for collecting data may also generate reflections and considerations regarding multicultural awareness in respondents.

Communication with an accent
Language, more than any other aspect of human communication, defines origins (19) . An accent refers to a phonetic trait from a person's original language that is carried over a second language, and a dialect refers to a language variation associated with a particular region, social class, or ethnicity (20) . Both concepts are applicable to all languages and all speakers (21,22) .
Research in language acquisition demonstrates that most people retain their original accents when acquiring a second language after childhood (23,24) . It is natural that the native language's phonological system has an impact on ESL (24) . Historically, the U.S. has a wide range of influx of people from several linguistic and cultural backgrounds (25) . It is reasonable to expect that many of these individuals' use of English will be characterized by an accent. As such, accents should be treated as a fundamental part of cultural diversity.
Moreover, accent is a subjective concept. An individual does not have to be from another country to have an accent. Individuals from different regions of the same country can sound very dissimilar. Interestingly, people are frequently convinced that they are the only ones without an accent (26) . Nonetheless, all individuals speak with an accent and/or a dialect, depending on the person's origin and current location (20) . Speech sound production, its transmission, and perceptual features of listeners vary according to several elements, such as socio-economic status, education, geographic origin and cultural background (27,28) . Further, phonetic characteristics encompass both the speech produced and its perception by the listeners making communication a pluralistic product (22) .

Bias
Despite general self and collective unawareness, people tend to act and react based on preconceived notions and attitudes (18) . The inability to consciously recognize personal bias may characterize an "implicit bias" which can shape interactions with others (29) . The dilemma surrounding implicit bias stems from the apparent seeming inevitability of stereotyping which plays a helpful role in creating associations for quick retrieval as part of our everyday cognitive functions, but has a negative effect when these inevitable associations do not reflect our true values, appreciations or goals (29) .
Clinicians should avoid developing stereotypes and keep the individual at the forefront of any clinical encounter. This includes developing the competence to accept and appreciate others' cultural viewpoint (17,29) . Efforts to increase cultural awareness among future SLPs should prioritize self-examination, as an attempt to identify biases from their own culture and how this can have an impact in working with clients. Therefore, clinicians should conduct an examination of their personal attitudes and beliefs regarding diversity (17,29) .
American Speech-Language-Hearing Association (ASHA) position on accent ASHA's Code of Ethics states that SLPs and audiologists must deliver services equally to accented and non-accented clients. ASHA affiliates are urged to encourage an understanding of linguistic differences among consumers and the general population. Additionally, according to ASHA, SLPs and audiologists are responsible to educate clients, parents, and other professionals about CLD acceptance (21) . That same nondiscriminatory behavior is expected in recruiting students for higher education programs and hiring SLPs and audiologists (19,20) . Students and professionals in communication sciences and disorders who speak with an accent or a dialect can effectively provide speech, language, and audiological services to persons with communication disorders, provided that they have the expected level of knowledge about normal and disordered communication (20,21) .
Furthermore, nonnative speakers who want to modify their speech production in order to accommodate speech-related tasks and needs, may choose to receive accent reduction services. Accent modification assistance is included under the scope of practice in SLP, under elective services targeting communication performance enhancement (30) .

Multicultural education in communication disorders
Attitudes about racial issues have an impact on multicultural development in clinicians (16) . Therefore, multicultural competence should be added to the ability to develop attitudes, beliefs, knowledge, and skills in working with individuals from diverse backgrounds. An increase in exposure to CLD should lead to higher levels of multicultural proficiency.
Communication disorders students should be aware about the impact of culture on communication. Moreover, they should acquire multicultural competence in order to work with clients from diverse cultures. Although multicultural content typically has been included in curricula using infusion and embedding approaches (9) , courses on multicultural issues may tend to address broad principles of language and social structure regarding cultural group variations (11) . Nevertheless, it is essential to address personal issues such as bias, prejudice and discrimination as well. Increasing awareness regarding these topics could have a positive impact on perception, and ultimately on treatment of future clients (15) .

Rationale
A need to reduce cultural bias in speech-language clinical management has been described in the literature (17,18) . A study conducted prior to this investigation using a population of pre-professionals in rehabilitation revealed that a greater exposure to individuals with CLD backgrounds may increase multicultural awareness in university students, which can in turn impact quality of professional services delivered in the future (12) .
While the literature hints that increased CLD experiences tend to reduce cultural bias, there is a need to validate this concept in order to advance evidence based practices in the field. Furthermore, understanding the specific role of accented speech in cultural background should lead future clinicians to provide culturally relevant services. Outcomes from this introductory study shall enlighten critical aspects of attitudes and beliefs of pre-clinicians towards CLD, such as self-reported cultural bias. Results ought to prompt reflections for further investigation, as well as considerations in developing curricula for SLP programs.

Participants
The participants in this investigation were 20 undergraduate university students from a large university located in the U.S. Midwest, with a current enrollment of approximately 20,000 students. The participants were majoring in speech-language pathology; 17 female and 3 male, ranging in age from 18 to 35 years, all of whom were native English speakers, identifying themselves as primarily from a Caucasian background. Following appropriate approval by the Institutional Review Board (Southern Illinois University Carbondale Human Subjects Committee, Protocol Number 15091), the researcher contacted and explained this investigation to instructors. After the instructors' endorsement, their students were invited to participate. The legitimate population associated with the inferential process is an abstract population similar to those included in the sample.

Questionnaire
The questionnaire implemented in this study was based on an instrument designed for a previous study applied to university students including demographic data in addition to questions related to attitudes about the announcement in particular and toward language diversity in general (12) . Items included on a pilot version of the questionnaire were originated from past research (13,14) , and revised several times based on feedback from University faculty.
The instrument included 50 questions regarding feelings concerning diversity, level of regular interaction with diverse populations, amount of information about diversity, and extent of CLD education. Participants were asked to indicate how true each statement was in their opinion by circling a number using a Likert-type scale. Association with level of agreement or disagreement with the statements posed were conducted using numbers from one to five. Additionally, participants were asked to estimate weekly interaction time with people who are from diverse backgrounds, as well as weekly time spent acquiring information about diversity. Furthermore, questions involved the extent of knowledge about aspects of diversity acquired both at university and non-university levels (see Appendix A). Demographic information included age and gender, directly indicated on scan sheets.

Procedure
The research study and instructions for completing the consent form and questionnaire were presented by a research assistant. After obtaining a signed consent form to join the study from each participant, an oral announcement of 200 words, describing a research related event on campus of interest of the participants, was given by a graduate student who speaks English as a second language (ESL) with a nonnative accent. Immediately after listening to the announcement, participants were asked to record responses in scan sheets. The responses were then electronically captured by optical scanning equipment.

Design
Correlations between specific concepts were applied to the data in order to inspect key aspects of the study. Multiple regressions were also used in this investigation, as a tentative approach to extend examination of the data. An a-priori power analysis indicated that with alpha coefficient and power set at .05 and.80, the present study needed 20-40 participants to detect a small effect.

Variables
In this study, items from the questionnaire were used in bidirectional correlations as well as a dependent and independent variables (i.e., criterion and predictors), with emphasis in identifying factors that may suggest an association with self-reported cultural bias. Correlations of self-reported CLD bias and all other variables were tested. Furthermore, statements such as "I usually have difficulty paying attention to what ESL speakers are saying" and "I would rather have a native language speaker giving the announcement, were applied as predictors of item number 12, which stated "I have cultural biases." Additional instances involved regularity of interaction and information about individuals from diverse characteristics such as age, gender, language, nationality, physical abilities, mental abilities, race, ethnicity, religion, and sexual orientation as predictors of self-reported cultural bias (see Appendix).

Data analysis
Statistical analyses were computed using SPSS software version 20.0 (SPSS Inc., Chicago. IL). Frequency distributions were also applied to examine the data. Because a change in the level of significance will affect the risk for both Type I and Type II error, the traditional alpha level of .05 was selected for the current study.
Variables in this study were analyzed using correlation statistical techniques. Prior to testing the hypotheses of the present study, the assumptions of correlation and multiple regression (i.e., normality, homoscedasticity, linearity, and independence of observations) were assessed. Results of the skewness and kurtosis analyses revealed that the data were normally distributed. Next, the assumption of homoscedasticity was tested using a visual analysis of bivariate scatter plots of residuals for all predictor and dependent variables. The visual analysis revealed that the variances of the error terms were normally distributed and constant, thus indicating the assumption of homoscedasticity was met.
The assumption of linearity was tested by using a visual analysis of scatter plots of the variables of interest, demonstrating a linear relationship. Lastly, it was inferred that the assumption of independence of observations was met because each participant completed their own questionnaire individually.

Multicultural exposure
Frequency counts signaled limited exposure of participants to people of various cultures: 80% of the participants reported having minimal contact (i.e., less than one hour per week or no contact at all) with ESL speakers, approximately 50% of the participants reported having minimal contact with people from diverse race and ethnicity, and 60% of the participants reported having minimal contact with people of diverse nationalities. Furthermore, 75% of the participants reported having minimal access to information about ESL speakers; approximately 60% of the participants reported having minimal weekly access to information about people of diverse nationality, race, and ethnicity. Possible limited CLD proficiency was suggested by participants' indication (i.e., agree/strongly agree) of making a special effort to interpret what the speaker was saying (85%), and hardly thinking about multicultural issues (30%). Only 15% of the participants acknowledged awareness of their own cultural bias (see Tables 1, 2, Figures 1, 2 and 3).
Regarding prior academic information, 55% of the participants reported no previous university level coursework on second language acquisition or laws covering services provided to bilingual clients. Additionally, approximately 30% of the respondents reported no university coursework regarding differential assessment of bilingual vs. monolingual individuals or assessment tools for bilingual individuals, and 60% had accomplished no coursework on how to use a language interpreter (see Tables 1, 2, Figures 4, 5).

Cultural bias
In this study Pearson's correlations were used to test relationships of self-reported cultural bias (i.e., I have cultural bias) and several variables (i.e., attitudes towards language diversity, level of CLD interaction and information, amount of university and non-university CLD coursework). Statistically significant positive correlations were detected between cultural bias and effort in interpreting CLD speech, r = .495, p = .013, as well as between cultural bias and difficulty paying attention to CLD speech, r = .454, p = .022. This indicates that higher levels of cultural bias were related to higher levels of effort in interpreting CLD speech as well as difficulty paying attention to CLD speech. Positive correlations were also demonstrated by the relationship between cultural bias and (a) level of interaction with low SES, r = .646, p = .001, as well as in (b) level of interaction with high SES, r = .465, p = .019 (see Table 3). None of the other correlations were significant.
Stepwise multiple regression analysis was used as a tentative approach to further examine the strength of correlation between self-reported cultural bias and CLD attitudes and beliefs. Results from a combination model including all 11 statements from section A of the questionnaire (see Appendix A) indicated a significant relationship among these factors, F = 5.835, p = .027, R 2 = .245 (see Table 4). Making a special effort in interpreting CLD speech was found to have the strongest beta coefficient, Minutes/Week: 1=0, 2=>60, 3=60,-420, 4=421-840, 5=<840. Categories: age, gender, language, nationality, physical abilities, mental abilities, race, ethnicity, religion, LSE, HSE  indicating that this variable was the strongest contributor to the overall R 2 followed by having difficulty paying attention to CLD. Furthermore, cultural bias was predicted by level of interaction with individuals from diverse characteristics such as age, gender, language, nationality, physical and mental abilities, race, ethnicity, religion, sexual orientation, and socioeconomic status (see Appendix A). These combined variables statistically significantly predicted cultural bias, F = 12.914, p = .002, R 2 = .418; the strongest contributors to the overall R 2 in this model were associated with interactions with diverse SES. A similar approach applied to level of information about individuals from diverse characteristic did not reach statistical significance (see Table 4). Additional analyses were performed using self-reported bias as criterion and responses to attitudes' statements (i. e., questionnaire, section A) organized by "positive" and "negative" (i.e., 1, 6, 7, 8, 10 and 2, 3, 4, 5, 9 respectively) as predictors. A t-test revealed no differences between means of positive and negative statements (see Table 5). Lack of statistically significant decreases for some of the comparisons may be explained by the small sample size used in this stud as large sample sizes.

DISCUSSION
The main objective of this study was to examine procedures and aspects involved in a broad investigation regarding attitudes and beliefs regarding language diversity, by specifically examining attitudes and beliefs of SLP students toward the accented speech of individuals who are nonnative Standard American English (SAE) speakers. Understanding the specific role of accented speech in cultural background should lead future clinicians to provide culturally competent services.
Outcomes from this introductory study shall enlighten critical aspects of attitudes and beliefs of pre-clinicians towards CLD, such as self-reported cultural bias. Results should prompt reflections for further investigation, as well as considerations in developing curricula for SLP programs.
Results of this work indicated critical areas that need to be further investigated so that a more comprehensive knowledge base can be established to include priority items for consideration when developing multicultural curricula in SLP programs.
The results suggest that exposure to CLD may impact attitudes and beliefs of SLP students toward diversity. This pilot study revealed correlations between students' effort in interpreting CLD speech as well ability to pay attention to CLD speech with self-reported cultural bias. This may be associated with the finding that a large of the participants reported having minimal contact with those who are ESL speakers. Research in several disciplines has suggested that even minimal exposure to varied cultural groups may be associated with more positive attitudes, suggesting that educational activities allowing increased exposure to varied cultural groups may be a beneficial addition to SLP curricula. More than half of the participants reported having minimal access to information about ESL speakers and access to information about people of diverse nationality. Increasing exposure through the inclusion of guest speakers, through media supports and community activities are feasible additions to increase cultural competency. These outcomes are concurrent with previous findings associated with needs of increasing awareness towards cultural diversity in higher education, as well as efficacy of services delivered (13,14) .

CONCLUSION
Results of this study have a variety of implications and uses. While the recommendations for increased CLD training is a logical and intuitive conclusion for practice, fostering awareness of implicit bias among students, given its dwelling unconscious nature, is much more complicated than fostering awareness of its explicit counterpart, which is consciously endorsed.
Data reported may help in determining procedures applicable in larger investigations regarding the effect of attitudes and beliefs toward language diversity in SLP. These results may also be considered when developing curricula in SLP programs. Finally, this study could inspire future research about attitudes and beliefs toward aspects of diversity.