Quality of Qualitative Research

What is "good" science? What are "good" social sciences? What is "good" qualitative social research? What are the criteria and standards for such evaluations?

As far as the official discourses are concerned, the discussions about the foregoing questions have traditionally occurred in the context of theory of knowledge and the philosophy of science. In the early 20th century the idea of a "unitary science" was popular in Europe. It postulated and tried to work out common methodological principles and goodness criteria for all empirical sciences. But no such common basis for the specific requirements of the different disciplines was found. The traces of this tradition had a great impact on the social sciences in the "Western hemisphere"—especially since World War II. In contrast to the methods and methodologies for knowledge production in the natural sciences, hermeneutic and "qualitative" approaches were (and still are) considered less "scientific" and acceptable in the social sciences (with certain differences across disciplines and nations).

With the decreasing importance of the concepts of the philosophy of science (prototypically: Critical Rationalism) and with a stronger focus on "social relations" in science and scientific communities (e.g. in the KUHNian tradition), a new trend in the philosophy of science could be observed. Apart from goodness criteria grounded in logic and epistemology, the cognitive, social, economic, cultural, medial and historical characteristics of scientific research came into focus to ensure certainty of knowledge. It is not that these aspects did not play a role before. The prospects of practical application of knowledge in business and warfare have always strongly stimulated science, including the social sciences. Now these aspects have become part of the official discourse and object of detailed and realistic description and analysis. This has consequences for the politics of science.

In the 1970s, getting research funded by private industries was frowned upon in the West-German social sciences. Nowadays such funds are a widely accepted as proof of the quality of scientific work (projects, scientists). What scientific arguments justify this change in attitude?

The social standards for evaluating scientific knowledge production have changed considerably. Apart from a closer tie between research and "business", the publishing industry plays an important role in this context. The salient issues range from inner-scientific aspects of publishing (ranking of journals, frequency of citation, etc.) or outer-scientific resonance (in the mass media: press, television, etc.). Are these instances of and criteria for selection grounded in a scientifically justified rationality? How can one legitimize them from a scientific point of view?

The traditional methodological goodness criteria of the 20th century have come under pressure from at least two different sides in the recent past:

  • The inner-scientific claims for (principally) attainable certainty of knowledge have weakened. Many received "rock-solid" ideas about the quality of scientific research have been put in perspective by recent work in the sociology of science, work that concerned the role and meaning of instruments for knowledge production, language, social contexts, discourses, epistemological subjects, etc. for our scientific knowledge. The sciences no longer aim for the "one truth", but attempt to determine the (legitimate) number of truths—or have even given up all claims for truth.
  • Scientific work is to a greater extent guided and controlled by economic and administrative interests, evaluations and corresponding institutional procedures.

Obviously, there have been major changes over the past 25 years. What principals underlie the actually practized guidelines for the evaluation and assessment of the (qualitative) social sciences? What principles should be followed? Can social scientists contribute to this question? Why is there such a (strange) widespread silence concerning these essential and basic questions? Are there no science-internal standpoints regarding the justification of standards? Or do the protagonists—competing for the allocation of resources—prefer to follow dubious (and unofficially ridiculed) administrative criteria in order not to risk their chances to "win"?

In textbooks, the positions on the quality of qualitative research in the social sciences are spread across a broad spectrum. They range from an in-principal adoption of classical canonical standards of "quantitative" research (with certain modifications and adjustments) to demands for entirely new standards. So far, however, no agreement has been reached about a new, "alternative" catalogue of criteria. New and creative ideas about scientific knowledge production have emerged following the weakening of claims about the certainty of scientific knowledge (the recent self-reflexive debate in anthropology is a good example). At the same time, this critical approach to the certainty of knowledge weakens the "political" position of qualitative research against the ("quantitative") mainstream, and disadvantages the former in the competition for material resources to do research.

This charged field about the quality of research—constituted by diverse arguments from the theory of knowledge, scientific methodology, the sociology of science and the politics of science—define the topics for our FQS debate on "Quality of Qualitative Research". Currently, the discussions about these topics seem to have become unfashionable. But this situation itself seems to be part of the problem. There exists considerable uncertainty, which arises from a sense of epistemological arbitrariness and politically charged claims about the validity of knowledge production and practical, useful and accountable solutions to existing problems.

We hope that many authors and readers will be willing to deal with these questions in a scholarly way that is characterized by self-consciousness, autonomy and reflexivity.

Debate "Quality of Qualitative Research": Franz Breuer, Carlos Kölbl, Jo Reichertz, Wolff-Michael Roth