A review of the characteristics of 108 author-level bibliometric indicators
---
paper_title: Quality, quantity, and impact in academic publication

paper_content:

Publication records of 85 social-personality psychologists were tracked from the time of their doctoral studies until 10 years post-PhD. Associations between publication quantity (number of articles), quality (mean journal impact factor and article influence score), and impact (citations, h-index, g-index, webpage visits) were examined. Publication quantity and quality were only modestly related, and there was evidence of a quality-quantity trade-off. Impact was more strongly associated with quantity than quality. Authors whose records weighed quality over quantity tended to be associated with more prestigious institutions, but had lesser impact. Quantity- and quality-favoring publication strategies may have important implications for the shape and success of scientific careers. Copyright © 2009 John Wiley & Sons, Ltd.
---
paper_title: Assessing basic research : Some partial indicators of scientific progress in radio astronomy

paper_content:

As the costs of certain types of scientific research have escalated and as growth rates in overall national science budgets have declined, so the need for an explicit science policy has grown more urgent. In order to establish priorities between research groups competing for scarce funds, one of the most important pieces of information needed by science policy-makers is an assessment of those groups' recent scientific performance. This paper suggests a method for evaluating that performance. ::: ::: After reviewing the literature on scientific assessment, we argue that, while there are no simple measures of the contributions to scientific knowledge made by scientists, there are a number of ‘partial indicators’ — that is, variables determined partly by the magnitude of the particular contributions, and partly by ‘other factors’. If the partial indicators are to yield reliable results, then the influence of these ‘other factors’ must be minimised. This is the aim of the method of ‘converging partial indicators’ proposed in this paper. We argue that the method overcomes many of the problems encountered in previous work on scientific assessment by incorporating the following elements: (1) the indicators are applied to research groups rather than individual scientists; (2) the indicators based on citations are seen as reflecting the impact, rather than the quality or importance, of the research work; (3) a range of indicators are employed, each of which focusses on different aspects of a group's performance; (4) the indicators are applied to matched groups, comparing ‘like’ with ‘like’ as far as possible; (5) because of the imperfect or partial nature of the indicators, only in those cases where they yield convergent results can it be assumed that the influence of the ‘other factors’ has been kept relatively small (i.e. the matching of the groups has been largely successful), and that the indicators therefore provide a reasonably reliable estimate of the contribution to scientific progress made by different research groups. ::: ::: In an empirical study of four radio astronomy observatories, the method of converging partial indicators is tested, and several of the indicators (publications per researcher, citations per paper, numbers of highly cited papers, and peer evaluation) are found to give fairly consistent results. The results are of relevance to two questions: (a) can basic research be assessed? (b) more specifically, can significant differences in the research performance of radio astronomy centres be identified? We would maintain that the evidence presented in this paper is sufficient to justify a positive answer to both these questions, and hence to show that the method of converging partial indicators can yield information useful to science policy-makers.
---
paper_title: The w-index: A significant improvement of the h-index

paper_content:

I propose a new measure, the w-index, as a particularly simple and useful way to assess the integrated impact of a researcher's work, especially his or her excellent papers. The w-index can be defined as follows: If w of a researcher's papers have at least 10w citations each and the other papers have fewer than 10(w+1) citations, his/her w-index is w. It is a significant improvement of the h-index.
---
paper_title: Towards objectivity in research evaluation using bibliometric indicators – A protocol for incorporating complexity

paper_content:

Abstract Publications are thought to be an integrative indicator best suited to measure the multifaceted nature of scientific performance. Therefore, indicators based on the publication record (citation analysis) are the primary tool for rapid evaluation of scientific performance. Nevertheless, it has to be questioned whether the indicators really do measure what they are intended to measure because people adjust to the indicator value system by optimizing their indicator rather than their performance. Thus, no matter how sophisticated an indicator may be, it will never be proof against manipulation. A literature review identifies the most critical problems of citation analysis: database-related problems, inflated citation records, bias in citation rates and crediting of multi-author papers. We present a step-by-step protocol to address these problems. By applying this protocol, reviewers can avoid most of the pitfalls associated with the pure numbers of indicators and achieve a fast but fair evaluation of a scientist's performance. We as ecologists should accept complexity not only in our research but also in our research evaluation and should encourage scientists of other disciplines to do so as well.
---
paper_title: The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence

paper_content:

Abstract Evaluation studies of scientific performance conducted during the past years more and more focus on the identification of research of the 'highest quality', 'top' research, or 'scientific excellence'. This shift in focus has lead to the development of new bibliometric methodologies and indicators. Technically, it meant a shift from bibliometric impact scores based on average values such as the average impact of all papers published by some unit to be evaluated towards indicators reflecting the topof the citation distribution, such as the number of 'highly cited' or 'top' articles. In this study we present a comparative analysis of a number of standard and new indicators of research performance or 'scientific excellence', using techniques applied in studies conducted by CWTS in recent years. It will be shown that each type of indicator reflects a particular dimension of the general concept of research performance. Consequently, the application of one single indicator only may provide an incomplet...
---
paper_title: h-Index : A review focused in its variants , computation and standardization for different scientific fields

paper_content:

The h-index and some related bibliometric indices have received a lot of attention from the scientific community in the last few years due to some of their good properties (easiness of computation, balance between quantity of publications and their impact and so on). Many different indicators have been developed in order to extend and overcome the drawbacks of the original Hirsch proposal. In this contribution we present a comprehensive review on the h-index and related indicators field. From the initial h-index proposal we study their main advantages, drawbacks and the main applications that we can find in the literature. A description of many of the h-related indices that have been developed along with their main characteristics and some of the works that analyze and compare them are presented. We also review the most up to date standardization studies that allow a fair comparison by means of the h-index among scientists from different research areas and finally, some works that analyze the computation of the h-index and related indices by using different citation databases (ISI Citation Indexes, Google Scholar and Scopus) are introduced.
---
paper_title: Scaling the h-index for different scientific ISI fields

paper_content:

We propose a simple way to put in a common scale the h values of researchers working in different scientific ISI fields, so that the foreseeable misuse of this index for inter-areas comparison might be prevented, or at least, alleviated.
---
paper_title: UK Research Assessment Exercises: Informed judgments on research quality or quantity?

paper_content:

A longitudinal analysis of UK science covering almost 20 years revealed in the years prior to a Research Assessment Exercise (RAE 1992, 1996 and 2001) three distinct bibliometric patterns, that can be interpreted in terms of scientists’ responses to the principal evaluation criteria applied in a RAE. When in the RAE 1992 total publications counts were requested, UK scientists substantially increased their article production. When a shift in evaluation criteria in the RAE 1996 was announced from ‘quantity’ to ‘quality’, UK authors gradually increased their number of papers in journals with a relatively high citation impact. And during 1997–2000, institutions raised their number of active research staff by stimulating their staff members to collaborate more intensively, or at least to co-author more intensively, although their joint paper productivity did not. This finding suggests that, along the way towards the RAE 2001, evaluated units in a sense shifted back from ‘quality’ to ‘quantity’. The analysis also observed a slight upward trend in overall UK citation impact, corroborating conclusions from an earlier study. The implications of the findings for the use of citation analysis in the RAE are briefly discussed.
---
paper_title: The Four Literatures of Social Science

paper_content:

This chapter reviews bibliometric studies of the social sciences and humanities. SSCI bibliometrics will work reasonably well in economics and psychology, whose literatures share many characteristics with science, and less well in sociology, characterised by a typical social science literature. The premise of the chapter is that quantitative evaluation of research output faces severe methodological difficulties in fields whose literature differs in nature from scientific literature. Bibliometric evaluations are based on international journal literature indexed in the SSCI, but social scientists also publish books, write for national journals and for the non-scholarly press. These literatures form distinct, yet partially overlapping worlds, each serving a different purpose. For example, national journals communicate with a local scholarly community, and the non-scholarly press represents research in interaction with contexts of application. Each literature is more trans-disciplinary than its scientific counterpart, which itself poses methodological challenges. The nature and role of each of the literatures will be explored here, and the chapter will argue that by ignoring the three other literatures of social science bibliometric evaluation produces a distorted picture of social science fields.
---
paper_title: Is it possible to compare researchers with different scientific interests?

paper_content:

The number h of papers with at least h citations has been proposed to evaluate individuals scientific research production. This index is robust in several ways but yet strongly dependent on the research field. We propose a complementary index ) ( 2 T a I N h h = , with ) (T a N being the total number of authors in the considered h papers. A researcher with index hI has hI papers with at least hI citation if he/she had published alone. We have obtained the rank plots of h and h I for four Brazilian scientific communities. In contrast with the h-index, the h I index rank plots collapse into a single curve allowing comparison among different research areas.
---
paper_title: Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators

paper_content:

The purpose of this article is to come up with a valid categorization and to examine the performance and properties of a wide range of h-type indices presented recently in the relevant literature. By exploratory factor analysis (EFA) we study the relationship between the h-index, its variants, and some standard bibliometric indicators of 26 physicists compiled from the Science Citation Index in the Web of Science.
---
paper_title: Lost in publication: how measurement harms science

paper_content:

Measurement of scientific productivity is difficult. The measures used (impact factor of the journal, citations to the paper being measured) are crude. But these measures are now so univer- sally adopted that they determine most things that matter: tenure or unemployment, a postdoctoral grant or none, success or failure. As a result, scientists have been forced to downgrade their primary aim from making discoveries to publishing as many papers as possible — and trying to work them into high impact factor journals. Consequently, scientific behaviour has become distorted and the utility, quality and objectivity of articles has deteriorated. Changes to the way scientists are assessed are urgently needed, and I suggest some here.
---
paper_title: How good is research really?

paper_content:

Bibliometrics increasingly determine the allocation of jobs and funding in science. Bibliometricians must therefore develop and adopt reliable measures of quality that truly reflect a scientist's contribution to his or her field.
---
paper_title: The h-index and its alternatives: An application to the 100 most prolific economists

paper_content:

The h-index is a recent but already quite popular way of measuring research quality and quantity. However, it discounts highly-cited papers. The g-index corrects for this, but it is sensitivity to the number of never-cited papers. Besides, h- or g-index-based rankings have a large number of ties. Therefore, this paper introduces two new indices, and tests their performance for the 100 most prolific economists. A researcher has a t-number (f-number) of t (f) if t (f) is the largest number for which it holds that she has t (f) publications for which the geometric (harmonic) average number of citations is at least t (f). The new indices overcome the shortcomings of the old indices.
---
paper_title: New developments related to the Hirsch index

paper_content:

It is shown that the h-index on the one hand, and the A- and g-indices on the other, measure different things. The A-index, however, seems overly sensitive to one extremely highly cited article. For this reason it would seem that the g-index is the more useful of the two. As to the h- and the g-index: they do measure different aspects of a scientist’s publication list. Certainly the h-index does not tell the full story, and, although a more sensitive indicator than the h-index, neither does the g-index. Taken together, g and h present a concise picture of a scientist’s achievements in terms of publications and citations.
---
paper_title: Quantifying Scholarly Impact: IQp Versus the Hirsch h

paper_content:

Hirsch's (2005) h index of scholarly output has generated substantial interest and wide acceptance because of its apparent ability to quantify scholarly impact simply and accurately. We show that the excitement surrounding h is premature for three reasons: h stagnates with increasing scientific age; it is highly dependent on publication quantity; and it is highly dependent on field-specific citation rates. Thus, it is not useful for comparing scholars across disciplines. We propose the scholarly “index of quality and productivity” (IQp) as an alternative to h. The new index takes into account a scholar's total impact and also corrects for field-specific citation rates, scholarly productivity, and scientific age. The IQp accurately predicts group membership on a common metric, as tested on a sample of 80 scholars from three populations: (a) Nobel winners in physics (n = 10), chemistry (n = 10), medicine (n = 10), and economics (n = 10), and towering psychologists (n = 10); and scholars who have made more modest contributions to science including randomly selected (b) fellows (n = 15) and (c) members (n = 15) of the Society of Industrial and Organizational Psychology. The IQp also correlates better with expert ratings of greatness than does the h index. © 2008 Wiley Periodicals, Inc.
---
paper_title: On measuring the relation between social science research activity and research publication

paper_content:

Using data from the ESRC Research Activity and Publications Information Database (RAPID), this is a report of an investigation that differs from ‘traditional bibliometrics’. With the aid of a data model that prompts an initial focus on the research project rather than those research publications that are easy to see, evidence was uncovered on the varied type of publication used to disseminate social science findings, and on the considerable time intervals involved between funded research activity and the date of publication. The former suggests shortcomings in the performance indicators generated by bibliometric methods; the latter has important implications for the research evaluation practices of funding bodies. Copyright , Beech Tree Publishing.
---
paper_title: Meeting the Micro-Level Challenges : Bibliometrics at the Individual Level

paper_content:

The aim of this paper is to demonstrate a method for bibliometric evaluation of individuals, i.e. research staff currently employed within a university department or other knowledge organisations w ...
---
paper_title: Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature

paper_content:

Interdisciplinary scientific research (IDR) extends and challenges the study of science on a number of fronts, including creating output science and engineering (S&E) indicators. This literature review began with a narrow search for quantitative measures of the output of IDR that could contribute to indicators, but the authors expanded the scope of the review as it became clear that differing definitions, assessment tools, evaluation processes, and measures all shed light on different aspects of IDR. Key among these broader aspects is (a) the importance of incorporating the concept of knowledge integration, and (b) recognizing that integration can occur within a single mind as well as among a team. Existing output measures alone cannot adequately capture this process. Among the quantitative measures considered, bibliometrics (co-authorships, co-inventors, collaborations, references, citations and co-citations) are the most developed, but leave considerable gaps in understanding of the social dynamics that lead to knowledge integration. Emerging measures in network dynamics (particularly betweenness centrality and diversity), and entropy are promising as indicators, but their use requires sophisticated interpretations. Combinations of quantitative measures and qualitative assessments being applied within evaluation studies appear to reveal IDR processes but carry burdens of expense, intrusion, and lack of reproducibility year-upon-year. This review is a first step toward providing a more holistic view of measuring IDR, although research and development is needed before metrics can adequately reflect the actual phenomenon of IDR.
---
paper_title: An index to quantify an individual's scientific research output

paper_content:

I propose the index $h$, defined as the number of papers with citation number higher or equal to $h$, as a useful index to characterize the scientific output of a researcher.
---
paper_title: Meeting the Micro-Level Challenges : Bibliometrics at the Individual Level

paper_content:

The aim of this paper is to demonstrate a method for bibliometric evaluation of individuals, i.e. research staff currently employed within a university department or other knowledge organisations w ...
---
paper_title: Assessing basic research : Some partial indicators of scientific progress in radio astronomy

paper_content:

As the costs of certain types of scientific research have escalated and as growth rates in overall national science budgets have declined, so the need for an explicit science policy has grown more urgent. In order to establish priorities between research groups competing for scarce funds, one of the most important pieces of information needed by science policy-makers is an assessment of those groups' recent scientific performance. This paper suggests a method for evaluating that performance. ::: ::: After reviewing the literature on scientific assessment, we argue that, while there are no simple measures of the contributions to scientific knowledge made by scientists, there are a number of ‘partial indicators’ — that is, variables determined partly by the magnitude of the particular contributions, and partly by ‘other factors’. If the partial indicators are to yield reliable results, then the influence of these ‘other factors’ must be minimised. This is the aim of the method of ‘converging partial indicators’ proposed in this paper. We argue that the method overcomes many of the problems encountered in previous work on scientific assessment by incorporating the following elements: (1) the indicators are applied to research groups rather than individual scientists; (2) the indicators based on citations are seen as reflecting the impact, rather than the quality or importance, of the research work; (3) a range of indicators are employed, each of which focusses on different aspects of a group's performance; (4) the indicators are applied to matched groups, comparing ‘like’ with ‘like’ as far as possible; (5) because of the imperfect or partial nature of the indicators, only in those cases where they yield convergent results can it be assumed that the influence of the ‘other factors’ has been kept relatively small (i.e. the matching of the groups has been largely successful), and that the indicators therefore provide a reasonably reliable estimate of the contribution to scientific progress made by different research groups. ::: ::: In an empirical study of four radio astronomy observatories, the method of converging partial indicators is tested, and several of the indicators (publications per researcher, citations per paper, numbers of highly cited papers, and peer evaluation) are found to give fairly consistent results. The results are of relevance to two questions: (a) can basic research be assessed? (b) more specifically, can significant differences in the research performance of radio astronomy centres be identified? We would maintain that the evidence presented in this paper is sufficient to justify a positive answer to both these questions, and hence to show that the method of converging partial indicators can yield information useful to science policy-makers.
---
paper_title: Assessing basic research : Some partial indicators of scientific progress in radio astronomy

paper_content:

As the costs of certain types of scientific research have escalated and as growth rates in overall national science budgets have declined, so the need for an explicit science policy has grown more urgent. In order to establish priorities between research groups competing for scarce funds, one of the most important pieces of information needed by science policy-makers is an assessment of those groups' recent scientific performance. This paper suggests a method for evaluating that performance. ::: ::: After reviewing the literature on scientific assessment, we argue that, while there are no simple measures of the contributions to scientific knowledge made by scientists, there are a number of ‘partial indicators’ — that is, variables determined partly by the magnitude of the particular contributions, and partly by ‘other factors’. If the partial indicators are to yield reliable results, then the influence of these ‘other factors’ must be minimised. This is the aim of the method of ‘converging partial indicators’ proposed in this paper. We argue that the method overcomes many of the problems encountered in previous work on scientific assessment by incorporating the following elements: (1) the indicators are applied to research groups rather than individual scientists; (2) the indicators based on citations are seen as reflecting the impact, rather than the quality or importance, of the research work; (3) a range of indicators are employed, each of which focusses on different aspects of a group's performance; (4) the indicators are applied to matched groups, comparing ‘like’ with ‘like’ as far as possible; (5) because of the imperfect or partial nature of the indicators, only in those cases where they yield convergent results can it be assumed that the influence of the ‘other factors’ has been kept relatively small (i.e. the matching of the groups has been largely successful), and that the indicators therefore provide a reasonably reliable estimate of the contribution to scientific progress made by different research groups. ::: ::: In an empirical study of four radio astronomy observatories, the method of converging partial indicators is tested, and several of the indicators (publications per researcher, citations per paper, numbers of highly cited papers, and peer evaluation) are found to give fairly consistent results. The results are of relevance to two questions: (a) can basic research be assessed? (b) more specifically, can significant differences in the research performance of radio astronomy centres be identified? We would maintain that the evidence presented in this paper is sufficient to justify a positive answer to both these questions, and hence to show that the method of converging partial indicators can yield information useful to science policy-makers.
---
paper_title: Google Scholar duped and deduped – the aura of “robometrics”

paper_content:

Purpose – This purpose of this paper is to discuss some of the problems that exist with Google Scholar, particularly regarding content spam and citation spam.Design/methodology/approach – The paper provides an analysis of how Google Scholar has been duped by real but manipulated documents and reference lists, as well as by fake documents and references. Details of research regarding the duping of Google Scholar is presented and a possible solution is offered.Findings – Researchers showed how easy it was to dupe Google Scholar. In one case, the researchers added invisible words to the first page of one of their conference papers (using the well‐known white letter on white screen/paper technique), and modified the content and bibliography of some of their already published papers, then posted them on the web to see if Google Scholar would bite, i.e. would improve their rank position, and increase the number of citations that the targeted papers received, and the number of papers published by the authors. Go...
---
paper_title: On the h-index - A mathematical approach to a new measure of publication activity and citation impact

paper_content:

L'A. analyse les proprietes basiques de l'index-h, indicateur developpe par J. E. Hirsch, sur la base d'un modele de distribution de probabilites largement utilise en bibliometrie, a savoir les distributions Pareto. L'index-h, fonde sur le nombre de citations recues, mesure l'activite de publication et l'impact en citations. C'est un indicateur utile avec d'interessantes proprietes mathematiques, mais qui ne saurait se substituer aux indicateurs bibliometriques courants plus sophistiques.
---
paper_title: Is scientific literature subject to a sell-by-date? A general methodology to analyze the durability of scientific documents

paper_content:

The study of the citation histories and ageing of documents are topics that have been addressed from several perspectives, especially in the analysis of documents with delayed recognition or sleeping beauties. However, there is no general methodology that can be extensively applied for different time periods and/or research fields. In this paper a new methodology for the general analysis of the ageing and durability of scientific papers is presented. This methodology classifies documents into three general types: Delayed documents, which receive the main part of their citations later than normal documents; Flash in the pans, which receive citations immediately after their publication but they are not cited in the long term; and Normal documents, documents with a typical distribution of citations over time. These three types of durability have been analyzed considering the whole population of documents in the Web of Science with at least 5 external citations (i.e. not considering self-citations). Several patterns related to the three types of durability have been found and the potential for further research of the developed methodology is discussed.
---
paper_title: A bibliometric analysis of NOAA’s Office of Ocean Exploration and Research

paper_content:

Bibliometric analysis techniques are increasingly being used to analyze and evaluate scientific research produced by institutions and grant funding agencies. This article uses bibliometric methods to analyze journal articles funded by NOAA’s Office of Ocean Exploration and Research (OER), an extramural grant-funding agency focused on the scientific exploration of the world’s oceans. OER-supported articles in this analysis were identified through grant reports, personal communication, and acknowledgement of OER support or grant numbers. The articles identified were analyzed to determine the number of publications and citations received per year, subject, and institution. The productivity and citation impact of institutions in the US receiving OER grant funding were mapped geographically. Word co-occurrence and bibliographic coupling networks were created and visualized to identify the research topics of OER-supported articles. Finally, article citation counts were evaluated by means of percentile ranks. This article demonstrates that bibliometric analysis can be useful for summarizing and evaluating the research performance of a grant funding agency.
---
paper_title: The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level

paper_content:

The relationship of the h-index with other bibliometric indicators at the micro level is analysed for Spanish CSIC scientists in Natural Resources, using publications downloaded from the Web of Science (1994–2004). Different activity and impact indicators were obtained to describe the research performance of scientists in different dimensions, being the h-index located through factor analysis in a quantitative dimension highly correlated with the absolute number of publications and citations. The need to include the remaining dimensions in the analysis of research performance of scientists and the risks of relying only on the h-index are stressed. The hypothesis that the achievement of some highly visible but intermediate-productive authors might be underestimated when compared with other scientists by means of the h-index is tested.
---
paper_title: The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence

paper_content:

Abstract Evaluation studies of scientific performance conducted during the past years more and more focus on the identification of research of the 'highest quality', 'top' research, or 'scientific excellence'. This shift in focus has lead to the development of new bibliometric methodologies and indicators. Technically, it meant a shift from bibliometric impact scores based on average values such as the average impact of all papers published by some unit to be evaluated towards indicators reflecting the topof the citation distribution, such as the number of 'highly cited' or 'top' articles. In this study we present a comparative analysis of a number of standard and new indicators of research performance or 'scientific excellence', using techniques applied in studies conducted by CWTS in recent years. It will be shown that each type of indicator reflects a particular dimension of the general concept of research performance. Consequently, the application of one single indicator only may provide an incomplet...
---
paper_title: Scaling the h-index for different scientific ISI fields

paper_content:

We propose a simple way to put in a common scale the h values of researchers working in different scientific ISI fields, so that the foreseeable misuse of this index for inter-areas comparison might be prevented, or at least, alleviated.
---
paper_title: How good is research really?

paper_content:

Bibliometrics increasingly determine the allocation of jobs and funding in science. Bibliometricians must therefore develop and adopt reliable measures of quality that truly reflect a scientist's contribution to his or her field.
---
paper_title: Quantifying Scholarly Impact: IQp Versus the Hirsch h

paper_content:

Hirsch's (2005) h index of scholarly output has generated substantial interest and wide acceptance because of its apparent ability to quantify scholarly impact simply and accurately. We show that the excitement surrounding h is premature for three reasons: h stagnates with increasing scientific age; it is highly dependent on publication quantity; and it is highly dependent on field-specific citation rates. Thus, it is not useful for comparing scholars across disciplines. We propose the scholarly “index of quality and productivity” (IQp) as an alternative to h. The new index takes into account a scholar's total impact and also corrects for field-specific citation rates, scholarly productivity, and scientific age. The IQp accurately predicts group membership on a common metric, as tested on a sample of 80 scholars from three populations: (a) Nobel winners in physics (n = 10), chemistry (n = 10), medicine (n = 10), and economics (n = 10), and towering psychologists (n = 10); and scholars who have made more modest contributions to science including randomly selected (b) fellows (n = 15) and (c) members (n = 15) of the Society of Industrial and Organizational Psychology. The IQp also correlates better with expert ratings of greatness than does the h index. © 2008 Wiley Periodicals, Inc.
---
paper_title: The publication-citation matrix and its derived quantities

paper_content:

We give an overview of the main data of a publication-citation matrix. We show how impact factors are defined, and, in particular, point out the difference between the synchronous and the diachronous impact factor. The advantages and disadvantages of using both as tools in research evaluation are discussed.
---
paper_title: On the calculation of percentile-based bibliometric indicators

paper_content:

A percentile-based bibliometric indicator is an indicator that values publications based on their position within the citation distribution of their field. The most straightforward percentile-based indicator is the proportion of frequently cited publications, for instance, the proportion of publications that belong to the top 10% most frequently cited of their field. Recently, more complex percentile-based indicators have been proposed. A difficulty in the calculation of percentile-based indicators is caused by the discrete nature of citation distributions combined with the presence of many publications with the same number of citations. We introduce an approach to calculating percentile-based indicators that deals with this difficulty in a more satisfactory way than earlier approaches suggested in the literature. We show in a formal mathematical framework that our approach leads to indicators that do not suffer from biases in favor of or against particular fields of science.
---
paper_title: Discovering author impact: A PageRank perspective

paper_content:

This article provides an alternative perspective for measuring author impact by applying PageRank algorithm to a coauthorship network. A weighted PageRank algorithm considering citation and coauthorship network topology is proposed. We test this algorithm under different damping factors by evaluating author impact in the informetrics research community. In addition, we also compare this weighted PageRank with the h-index, citation, and program committee (PC) membership of the International Society for Scientometrics and Informetrics (ISSI) conferences. Findings show that this weighted PageRank algorithm provides reliable results in measuring author impact.
---
paper_title: Quality, quantity, and impact in academic publication

paper_content:

Publication records of 85 social-personality psychologists were tracked from the time of their doctoral studies until 10 years post-PhD. Associations between publication quantity (number of articles), quality (mean journal impact factor and article influence score), and impact (citations, h-index, g-index, webpage visits) were examined. Publication quantity and quality were only modestly related, and there was evidence of a quality-quantity trade-off. Impact was more strongly associated with quantity than quality. Authors whose records weighed quality over quantity tended to be associated with more prestigious institutions, but had lesser impact. Quantity- and quality-favoring publication strategies may have important implications for the shape and success of scientific careers. Copyright © 2009 John Wiley & Sons, Ltd.
---
paper_title: Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators

paper_content:

The purpose of this article is to come up with a valid categorization and to examine the performance and properties of a wide range of h-type indices presented recently in the relevant literature. By exploratory factor analysis (EFA) we study the relationship between the h-index, its variants, and some standard bibliometric indicators of 26 physicists compiled from the Science Citation Index in the Web of Science.
---
paper_title: A RATIONAL, SUCCESSIVE G-INDEX APPLIED TO ECONOMICS DEPARTMENTS IN IRELAND

paper_content:

A rational, successive g-index is proposed, and applied to economics departments in Ireland. The successive g-index has greater discriminatory power than the successive h-index, and the rational index performs better still. The rational, successive g-index is also more robust to differences in department size.
---
paper_title: Development of bibliometric indicators for utility of research to users in society: Measurement of external knowledge transfer via publications in trade journals

paper_content:

The development of a set of bibliometric tools to contribute to the assessment and monitoring of utility of university and non-university research institutes to society is described. Trade publications were weighted according to the utility of the journals for relevant nonscientific user groups. Furthermore, one indicator addresses the extent to which a general or a specific type of audience is addressed. Results are shown for one university and one university department. In general, validation interviews show that the indicator provide a good first estimation of the potential effectivity of the knowledge transfer efforts by means of publications in trade journals to practice and policy bodies.
---
paper_title: An index to quantify an individual's scientific research output

paper_content:

I propose the index $h$, defined as the number of papers with citation number higher or equal to $h$, as a useful index to characterize the scientific output of a researcher.
---
paper_title: Theory and practise of the g-index

paper_content:

The g-index is introduced as an improvement of the h-index of Hirsch to measure the global citation performance of a set of articles. If this set is ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received (together) at least g 2 citations. We prove the unique existence of g for any set of articles and we have that g ≥ h. The general Lotkaian theory of the g-index is presented and we show that g = (α-1 / α-2) α-1/α T 1/α where a> 2 is the Lotkaian exponent and where T denotes the total number of sources. We then present the g-index of the (still active) Price medallists for their complete careers up to 1972 and compare it with the h-index. It is shown that the g-index inherits all the good properties of the h-index and, in addition, better takes into account the citation scores of the top articles. This yields a better distinction between and order of the scientists from the point of view of visibility.
---
paper_title: h-Index : A review focused in its variants , computation and standardization for different scientific fields

paper_content:

The h-index and some related bibliometric indices have received a lot of attention from the scientific community in the last few years due to some of their good properties (easiness of computation, balance between quantity of publications and their impact and so on). Many different indicators have been developed in order to extend and overcome the drawbacks of the original Hirsch proposal. In this contribution we present a comprehensive review on the h-index and related indicators field. From the initial h-index proposal we study their main advantages, drawbacks and the main applications that we can find in the literature. A description of many of the h-related indices that have been developed along with their main characteristics and some of the works that analyze and compare them are presented. We also review the most up to date standardization studies that allow a fair comparison by means of the h-index among scientists from different research areas and finally, some works that analyze the computation of the h-index and related indices by using different citation databases (ISI Citation Indexes, Google Scholar and Scopus) are introduced.
---
paper_title: Scaling the h-index for different scientific ISI fields

paper_content:

We propose a simple way to put in a common scale the h values of researchers working in different scientific ISI fields, so that the foreseeable misuse of this index for inter-areas comparison might be prevented, or at least, alleviated.
---
paper_title: Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators

paper_content:

The purpose of this article is to come up with a valid categorization and to examine the performance and properties of a wide range of h-type indices presented recently in the relevant literature. By exploratory factor analysis (EFA) we study the relationship between the h-index, its variants, and some standard bibliometric indicators of 26 physicists compiled from the Science Citation Index in the Web of Science.
---
paper_title: A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants

paper_content:

This paper presents the first meta-analysis of studies that computed correlations between the h index and variants of the h index (such as the g index; in total 37 different variants) that have been proposed and discussed in the literature. A high correlation between the h index and its variants would indicate that the h index variants hardly provide added information to the h index. This meta-analysis included 135 correlation coefficients from 32 studies. The studies were based on a total sample size of N=9005; on average, each study had a sample size of n=257. The results of a three-level cross-classified mixed-effects meta-analysis show a high correlation between the h index and its variants: Depending on the model, the mean correlation coefficient varies between .8 and .9. This means that there is redundancy between most of the h index variants and the h index. There is a statistically significant study-to-study variation of the correlation coefficients in the information they yield. The lowest correlation coefficients with the h index are found for the h index variants MII and m index. Hence, these h index variants make a non-redundant contribution to the h index.
---
paper_title: Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine

paper_content:

In this study, we examined empirical results on the h index and its most important variants in order to determine whether the variants developed are associated with an incremental contribution for evaluation purposes. The results of a factor analysis using bibliographic data on postdoctoral researchers in biomedicine indicate that regarding the h index and its variants, we are dealing with two types of indices that load on one factor each. One type describes the most productive core of a scientist's output and gives the number of papers in that core. The other type of indices describes the impact of the papers in the core. Because an index for evaluative purposes is a useful yardstick for comparison among scientists if the index corresponds strongly with peer assessments, we calculated a logistic regression analysis with the two factors resulting from the factor analysis as independent variables and peer assessment of the postdoctoral researchers as the dependent variable. The results of the regression analysis show that peer assessments can be predicted better using the factor ‘impact of the productive core’ than using the factor ‘quantity of the productive core.’ © 2008 Wiley Periodicals, Inc.
---
paper_title: The e-Index, Complementing the h-Index for Excess Citations

paper_content:

BACKGROUND ::: The h-index has already been used by major citation databases to evaluate the academic performance of individual scientists. Although effective and simple, the h-index suffers from some drawbacks that limit its use in accurately and fairly comparing the scientific output of different researchers. These drawbacks include information loss and low resolution: the former refers to the fact that in addition to h(2) citations for papers in the h-core, excess citations are completely ignored, whereas the latter means that it is common for a group of researchers to have an identical h-index. ::: ::: ::: METHODOLOGY/PRINCIPAL FINDINGS ::: To solve these problems, I here propose the e-index, where e(2) represents the ignored excess citations, in addition to the h(2) citations for h-core papers. Citation information can be completely depicted by using the h-index together with the e-index, which are independent of each other. Some other h-type indices, such as a and R, are h-dependent, have information redundancy with h, and therefore, when used together with h, mask the real differences in excess citations of different researchers. ::: ::: ::: CONCLUSIONS/SIGNIFICANCE ::: Although simple, the e-index is a necessary h-index complement, especially for evaluating highly cited scientists or for precisely comparing the scientific output of a group of scientists having an identical h-index.
---
paper_title: The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level

paper_content:

The relationship of the h-index with other bibliometric indicators at the micro level is analysed for Spanish CSIC scientists in Natural Resources, using publications downloaded from the Web of Science (1994–2004). Different activity and impact indicators were obtained to describe the research performance of scientists in different dimensions, being the h-index located through factor analysis in a quantitative dimension highly correlated with the absolute number of publications and citations. The need to include the remaining dimensions in the analysis of research performance of scientists and the risks of relying only on the h-index are stressed. The hypothesis that the achievement of some highly visible but intermediate-productive authors might be underestimated when compared with other scientists by means of the h-index is tested.
---
paper_title: On the robustness of the h-index

paper_content:

The h-index (Hirsch, 2005) is robust, remaining relatively unaffected by errors in the long tails of the citations-rank distribution, such as typographic errors that short-change frequently cited articles and create bogus additional records. This robustness, and the ease with which h-indices can be verified, support the use of a Hirsch-type index over alternatives such as the journal impact factor. These merits of the h-index apply both to individuals and to journals.
---
paper_title: h-Index : A review focused in its variants , computation and standardization for different scientific fields

paper_content:

The h-index and some related bibliometric indices have received a lot of attention from the scientific community in the last few years due to some of their good properties (easiness of computation, balance between quantity of publications and their impact and so on). Many different indicators have been developed in order to extend and overcome the drawbacks of the original Hirsch proposal. In this contribution we present a comprehensive review on the h-index and related indicators field. From the initial h-index proposal we study their main advantages, drawbacks and the main applications that we can find in the literature. A description of many of the h-related indices that have been developed along with their main characteristics and some of the works that analyze and compare them are presented. We also review the most up to date standardization studies that allow a fair comparison by means of the h-index among scientists from different research areas and finally, some works that analyze the computation of the h-index and related indices by using different citation databases (ISI Citation Indexes, Google Scholar and Scopus) are introduced.
---
paper_title: Generalized Hirsch h-index for disclosing latent facts in citation networks

paper_content:

What is the value of a scientist and its impact upon the scientific thinking? How can we measure the prestige of a journal or a conference? The evaluation of the scientific work of a scientist and the estimation of the quality of a journal or conference has long attracted significant interest, due to the benefits by obtaining an unbiased and fair criterion. Although it appears to be simple, defining a quality metric is not an easy task. To overcome the disadvantages of the present metrics used for ranking scientists and journals, J. E. Hirsch proposed a pioneering metric, the now famous h-index. In this article we demonstrate several inefficiencies of this index and develop a pair of generalizations and effective variants of it to deal with scientist ranking and publication forum ranking. The new citation indices are able to disclose trendsetters in scientific research, as well as researchers that constantly shape their field with their influential work, no matter how old they are. We exhibit the effectiveness and the benefits of the new indices to unfold the full potential of the h-index, with extensive experimental results obtained from the DBLP, a widely known on-line digital library.
---
paper_title: h-index sequence and h-index matrix: Constructions and applications

paper_content:

The calculation of Hirsch's h-index is a detail-ignoring way, therefore, single h-index could not reflect the difference of time spans for scientists to accumulate their papers and citations. In this study the h-index sequence and the h-index matrix are constructed, which complement the absent details of single h-index, reveal different increasing manner and the increasing mechanism of the h-index, and make the scientists at different scientific age comparable.
---
paper_title: A made-to-measure indicator for cross-disciplinary bibliometric ranking of researchers performance

paper_content:

This paper presents and discusses a new bibliometric indicator of research performance, designed with the fundamental concern of enabling cross-disciplinary comparisons. The indicator, called x-index, compares a researcher's output to a reference set of research output from top researchers, identified in the journals where the researcher has published. It reflects publication quantity and quality, uses a moderately sized data set, and works with a more refined definition of scientific fields. x-index was developed to rank researchers in a scientific excellence award in the Faculty of Engineering of the University of Porto. The data set collected for the 2009 edition of the award is used to study the indicator's features and design choices, and provides the basis for a discussion of its advantages and limitations.
---
paper_title: Universality of citation distributions: towards an objective measure of scientific impact

paper_content:

We study the distributions of citations received by a single publication within several disciplines, spanning broad areas of science. We show that the probability that an article is cited c times has large variations between different disciplines, but all distributions are rescaled on a universal curve when the relative indicator cf = c/c0 is considered, where c0 is the average number of citations per article for the discipline. In addition we show that the same universal behavior occurs when citation distributions of articles published in the same field, but in different years, are compared. These findings provide a strong validation of cf as an unbiased indicator for citation performance across disciplines and years. Based on this indicator, we introduce a generalization of the h index suitable for comparing scientists working in different fields.
---
paper_title: The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level

paper_content:

The relationship of the h-index with other bibliometric indicators at the micro level is analysed for Spanish CSIC scientists in Natural Resources, using publications downloaded from the Web of Science (1994–2004). Different activity and impact indicators were obtained to describe the research performance of scientists in different dimensions, being the h-index located through factor analysis in a quantitative dimension highly correlated with the absolute number of publications and citations. The need to include the remaining dimensions in the analysis of research performance of scientists and the risks of relying only on the h-index are stressed. The hypothesis that the achievement of some highly visible but intermediate-productive authors might be underestimated when compared with other scientists by means of the h-index is tested.
---
paper_title: The pure h-index: calculating an author’s h- index by taking co-authors into account

paper_content:

Abstract The article introduces a new Hirsch-type index for a scientist. This so-called pure h-index, denoted by hP, takes the actual number of coauthors, and the scientist’s relative position in the byline into account. The transformation from h to hP can also be applied to the R-index, leading to the pure R-index, denoted as RP. This index takes the number of collaborators, possibly the rank in the byline and the actual number of citations into account.
---
paper_title: Is it possible to compare researchers with different scientific interests?

paper_content:

The number h of papers with at least h citations has been proposed to evaluate individuals scientific research production. This index is robust in several ways but yet strongly dependent on the research field. We propose a complementary index ) ( 2 T a I N h h = , with ) (T a N being the total number of authors in the considered h papers. A researcher with index hI has hI papers with at least hI citation if he/she had published alone. We have obtained the rank plots of h and h I for four Brazilian scientific communities. In contrast with the h-index, the h I index rank plots collapse into a single curve allowing comparison among different research areas.
---
paper_title: The w-index: A significant improvement of the h-index

paper_content:

I propose a new measure, the w-index, as a particularly simple and useful way to assess the integrated impact of a researcher's work, especially his or her excellent papers. The w-index can be defined as follows: If w of a researcher's papers have at least 10w citations each and the other papers have fewer than 10(w+1) citations, his/her w-index is w. It is a significant improvement of the h-index.
---
paper_title: Theory and practise of the g-index

paper_content:

The g-index is introduced as an improvement of the h-index of Hirsch to measure the global citation performance of a set of articles. If this set is ranked in decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received (together) at least g 2 citations. We prove the unique existence of g for any set of articles and we have that g ≥ h. The general Lotkaian theory of the g-index is presented and we show that g = (α-1 / α-2) α-1/α T 1/α where a> 2 is the Lotkaian exponent and where T denotes the total number of sources. We then present the g-index of the (still active) Price medallists for their complete careers up to 1972 and compare it with the h-index. It is shown that the g-index inherits all the good properties of the h-index and, in addition, better takes into account the citation scores of the top articles. This yields a better distinction between and order of the scientists from the point of view of visibility.
---
paper_title: h-Index : A review focused in its variants , computation and standardization for different scientific fields

paper_content:

The h-index and some related bibliometric indices have received a lot of attention from the scientific community in the last few years due to some of their good properties (easiness of computation, balance between quantity of publications and their impact and so on). Many different indicators have been developed in order to extend and overcome the drawbacks of the original Hirsch proposal. In this contribution we present a comprehensive review on the h-index and related indicators field. From the initial h-index proposal we study their main advantages, drawbacks and the main applications that we can find in the literature. A description of many of the h-related indices that have been developed along with their main characteristics and some of the works that analyze and compare them are presented. We also review the most up to date standardization studies that allow a fair comparison by means of the h-index among scientists from different research areas and finally, some works that analyze the computation of the h-index and related indices by using different citation databases (ISI Citation Indexes, Google Scholar and Scopus) are introduced.
---
paper_title: The h-index and its alternatives: An application to the 100 most prolific economists

paper_content:

The h-index is a recent but already quite popular way of measuring research quality and quantity. However, it discounts highly-cited papers. The g-index corrects for this, but it is sensitivity to the number of never-cited papers. Besides, h- or g-index-based rankings have a large number of ties. Therefore, this paper introduces two new indices, and tests their performance for the 100 most prolific economists. A researcher has a t-number (f-number) of t (f) if t (f) is the largest number for which it holds that she has t (f) publications for which the geometric (harmonic) average number of citations is at least t (f). The new indices overcome the shortcomings of the old indices.
---
paper_title: The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level

paper_content:

The relationship of the h-index with other bibliometric indicators at the micro level is analysed for Spanish CSIC scientists in Natural Resources, using publications downloaded from the Web of Science (1994–2004). Different activity and impact indicators were obtained to describe the research performance of scientists in different dimensions, being the h-index located through factor analysis in a quantitative dimension highly correlated with the absolute number of publications and citations. The need to include the remaining dimensions in the analysis of research performance of scientists and the risks of relying only on the h-index are stressed. The hypothesis that the achievement of some highly visible but intermediate-productive authors might be underestimated when compared with other scientists by means of the h-index is tested.
---
paper_title: Harmonic publication and citation counting: sharing authorship credit equitably – not equally, geometrically or arithmetically

paper_content:

Bibliometric counting methods need to be validated against perceived notions of authorship credit allocation, and standardized by rejecting methods with poor fit or questionable ethical implications. Harmonic counting meets these concerns by exhibiting a robust fit to previously published empirical data from medicine, psychology and chemistry, and by complying with three basic ethical criteria for the equitable sharing of authorship credit. Harmonic counting can also incorporate additional byline information about equal contribution, or the elevated status of a corresponding last author. By contrast, several previously proposed counting schemes from the bibliometric literature including arithmetic, geometric and fractional counting, do not fit the empirical data as well and do not consistently meet the ethical criteria. In conclusion, harmonic counting would seem to provide unrivalled accuracy, fairness and flexibility to the long overdue task of standardizing bibliometric allocation of publication and citation credit.
---
paper_title: Usage impact factor: The effects of sample characteristics on usage-based impact metrics

paper_content:

There exist ample demonstrations that indicators of scholarly impact analogous to the citation-based ISI Impact Factor can be derived from usage data; however, so far, usage can practically be recorded only at the level of distinct information services. This leads to community-specific assessments of scholarly impact that are difficult to generalize to the global scholarly community. In contrast, the ISI Impact Factor is based on citation data and thereby represents the global community of scholarly authors. The objective of this study is to examine the effects of community characteristics on assessments of scholarly impact from usage. We define a journal Usage Impact Factor that mimics the definition of the Thomson Scientific ISI Impact Factor. Usage Impact Factor rankings are calculated on the basis of a large-scale usage dataset recorded by the linking servers of the California State University system from 2003 to 2005. The resulting journal rankings are then compared to the Thomson Scientific ISI Impact Factor that is used as a reference indicator of general impact. Our results indicate that the particular scientific and demographic characteristics of a discipline have a strong effect on resulting usage-based assessments of scholarly impact. In particular, we observed that as the number of graduate students and faculty increases in a particular discipline, Usage Impact Factor rankings will converge more strongly with the ISI Impact Factor. © 2008 Wiley Periodicals, Inc.
---
paper_title: Unethical practices in authorship of scientific papers

paper_content:

Over the past few decades, there has been an increase in the number of multi-author papers within scientific journals. This increase, in combination with the pressure to publish within academia, has precipitated various unethical authorship practices within biomedical research. These include dilution of authorship responsibility, ‘guest’, ‘pressured’ and ‘ghost’ authorship, and obfuscation of authorship credit within by-lines. Other authorship irregularities include divided and duplicate publication. This article discusses these problems and why the International Committee of Medical Journal Editors guidelines are failing to control them.
---
paper_title: Harmonic publication and citation counting: sharing authorship credit equitably – not equally, geometrically or arithmetically

paper_content:

Bibliometric counting methods need to be validated against perceived notions of authorship credit allocation, and standardized by rejecting methods with poor fit or questionable ethical implications. Harmonic counting meets these concerns by exhibiting a robust fit to previously published empirical data from medicine, psychology and chemistry, and by complying with three basic ethical criteria for the equitable sharing of authorship credit. Harmonic counting can also incorporate additional byline information about equal contribution, or the elevated status of a corresponding last author. By contrast, several previously proposed counting schemes from the bibliometric literature including arithmetic, geometric and fractional counting, do not fit the empirical data as well and do not consistently meet the ethical criteria. In conclusion, harmonic counting would seem to provide unrivalled accuracy, fairness and flexibility to the long overdue task of standardizing bibliometric allocation of publication and citation credit.
---
paper_title: Discovering author impact: A PageRank perspective

paper_content:

This article provides an alternative perspective for measuring author impact by applying PageRank algorithm to a coauthorship network. A weighted PageRank algorithm considering citation and coauthorship network topology is proposed. We test this algorithm under different damping factors by evaluating author impact in the informetrics research community. In addition, we also compare this weighted PageRank with the h-index, citation, and program committee (PC) membership of the International Society for Scientometrics and Informetrics (ISSI) conferences. Findings show that this weighted PageRank algorithm provides reliable results in measuring author impact.
---

Title: A Review of the Characteristics of 108 Author-Level Bibliometric Indicators

Section 1: Introduction
Description 1: Introduce the importance of bibliometric indicators for assessing researchers' reputations and the explosion in their use.

Section 2: Methodology
Description 2: Explain the selection of indicators, sources of supplementary information, and the criterion for analysis.

Section 3: Categories of Publication Indicators
Description 3: Describe the typology of publication and effect indicators used in the study.

Section 4: Judgement of Complexity
Description 4: Detail the complexity assessment criteria used for evaluating each indicator.

Section 5: Results
Description 5: Provide an overview of the identified indicators, their complexity scores, and the detailed results of the complexity analysis for each indicator.

Section 6: Overview of the Identified Indicators
Description 6: Summarize the identified indicators, including their complexity scores and the number of indicators in each category.

Section 7: Summary of Complexity Scores
Description 7: Discuss the overall complexity scores and their implications for end-user usability.

Section 8: Publication Count
Description 8: Present and discuss indicators related to the count of publications.

Section 9: Qualifying Output as Journal Impact
Description 9: Explore indicators that measure journal impact at the author-level.

Section 10: Effect of Output
Description 10: Detail indicators measuring the effect of output through citations and field normalization.

Section 11: Indicators that Rank the Publications in an Individual Portfolio
Description 11: Discuss indicators that rank the publications in an individual's portfolio, including h-dependent and h-independent indicators.

Section 12: Indicators of Impact Over Time
Description 12: Examine indicators assessing the impact of a researcher’s work over time, normalized to the portfolio and field.

Section 13: Discussion
Description 13: Reflect on challenges and implications of using multiple indicators, and the necessity of combining them for comprehensive assessment.

Section 14: Methodological Considerations
Description 14: Discuss the limitations of the study, methodological choices, and areas for future research.

Section 15: Conclusions
Description 15: Summarize the main findings, emphasizing the inadequacy of single indicators and recommending combinations of simple indicators for practical use.