Article Text

Download PDFPDF

Value assessment frameworks: who is valuing the care in healthcare?
  1. Jonathan Anthony Michaels
  1. Health Economics and Decision Science, University of Sheffield School of Health and Related Research, Sheffield, UK
  1. Correspondence to Professor Jonathan Anthony Michaels, Health Economics and Decision Science, University of Sheffield School of Health and Related Research, Sheffield S1 4DA, UK; j.michaels{at}


Many healthcare agencies are producing evidence-based guidance and policy that may determine the availability of particular healthcare products and procedures, effectively rationing aspects of healthcare. They claim legitimacy for their decisions through reference to evidence-based scientific method and the implementation of just decision-making procedures, often citing the criteria of ‘accountability for reasonableness’; publicity, relevance, challenge and revision, and regulation. Central to most decision methods are estimates of gains in quality-adjusted life-years (QALY), a measure that combines the length and quality of survival. However, all agree that the QALY alone is not a sufficient measure of all relevant aspects of potential healthcare benefits, and a number of value assessment frameworks have been suggested. I argue that the practical implementation of these procedures has the potential to lead to a distorted assessment of value. Undue weight may be ascribed to certain attributes, particularly those that favour commercial or political interests, while other attributes that are highly valued by society, particularly those related to care processes, may be omitted or undervalued. This may be compounded by a lack of transparency to relevant stakeholders, resulting in an inability for them to participate in, or challenge, the decisions. The makes it likely that costly new technologies, for which inflated prices can be justified by the current value frameworks, are displacing aspects of healthcare that are highly valued by society.

  • allocation of health care resources
  • decision-making
  • distributive justice
  • health economics
  • quality/value of life/personhood

Statistics from

Request Permissions



The rapid development of costly new healthcare technologies has resulted in escalating healthcare costs and the need for complex decisions regarding the funding and provision of such technologies. Claims of legitimacy for such decisions are often founded on the ‘evidence-based’ methods that underpin them. However, even where the best available evidence informs the predicted consequences of a decision, value judgements are necessary in balancing the competing risks, benefits, economic consequences and surrounding uncertainty. In a previous paper, I have explored some of the potential epistemic injustices that are inherent in the generation and interpretation of the underlying evidence.1 In this paper, I explore the necessary value judgements, identify some of the explicit or implicit principles that are at play and consider the practical implementation of different value assessment frameworks.

My personal experience largely stems from my clinical experience of working for the National Health Service (NHS) in the UK, and my involvement with the processes of the National Institute for Health and Care Excellence (NICE), which informs policy and guidance for the NHS.i Although issues of resource limitation and opportunity costs may be more overt in a publicly funded healthcare system, the need to value and balance the positive and negative outcomes of healthcare is universal, and many different healthcare systems have bodies that produce guidance based on principles of cost-effectiveness or comparative effectiveness.ii


Detailed scientific methods to support evidence-based guidance and policy are well documented. Even with the best scientific evidence, decision making requires value judgements about the relative importance of qualitatively different outcomes and attributes of healthcare in widely differing conditions and treatments.

Utilitarian principles demand that the greatest benefit is obtained from available resources, which requires that the benefits (and risks) of healthcare in differing domains are quantified and aggregated in an acceptable way. Egalitarian considerations may focus on equity of access to, or provision of, services, or may be concerned with addressing health inequalities, objectives that may conflict with each other and the desire to maximise overall benefit. Libertarian concerns put value on autonomy and participation in healthcare decisions. Taken together, these create competing drivers that must be balanced in the decision-making process.

Rather than explicitly defining sets of criteria and weightings for particular attributes and outcomes, agencies may focus on achieving legitimacy for their decisions through the processes by which they are made. NICE published principles for ‘Social Value Judgements’ in 2005, which were revised in 2008.2 These were developed with input from a Citizens Council, established by NICE to provide ‘a public perspective on overarching moral and ethical issues’iii, along with a public survey and a literature review, with further consultation, surveys and workshops to inform the revised version.

NICE states that it fulfils its obligation to distributive justice through a set of procedural features and refers to the principles of ‘accountability for reasonableness’ (A4R) that require publicity, relevance, challenge and revision, and regulation.3 The implementation of these principles by NICE, and many other bodies, is through deliberative processes in which advisory committees consider all the scientific evidence and publish reasoned decisions that are open to processes of consultation, challenge and revision. This avoids the need for rigid predetermination of the values ascribed to specific attributes and features of the technologies under consideration. However, the devil is in the detail of implementation, particularly in relation to determining which criteria the committee consider ‘relevant’, the relative weights assigned to these, and how transparency and challenge are achieved in relation to all possible stakeholders.


Daniels and Sabin’s description of the relevance condition states:


The usual model of decision making gives the advisory committee the responsibility for assigning values and weights to the various attributes of specific interventions. One notable exception is the use of the quality-adjusted life-year (QALY) as the preferred metric for the quantification of healthcare benefit. This combines the quality and the length of survival, using estimates of the ‘utility’ for particular health states that are based on societal valuation of generic health-related quality of life (HRQoL) instruments, such as the EuroQol 5-Dimension (EQ-5D).4

Many agencies take a primarily utilitarian approach, calculating the incremental benefit of healthcare options in terms of QALYs and calculating an incremental cost-effectiveness ratio (ICER), which may be compared with an acceptable threshold for ‘willingness to pay’ (WTP). In practice, other factors frequently play into the decision through adjustment of the acceptable WTP threshold.

All agencies accept that the QALY alone is insufficient to capture everything that is important in healthcare. In recent years, a number of value assessment frameworks have proposed additional elements to be included in such evaluations.5–9 Some agencies have explicitly considered additional value elements such as innovation, rarity, burden of disease and end-of-life treatments, assigning additional value to some through varying the acceptable WTP threshold,9–13 while rejecting others.7 14

The identification of additional value elements raises several questions;

  1. Which are relevant, and which should be excluded from consideration?

  2. Whose values and preferences should be applied in considering the relevant attributes?

  3. How should they be incorporated into decision-making processes?

  4. How might they be considered in relation to opportunity costs and potentially displaced aspects of healthcare.

Potential value elements

I suggest that the potential value elements fall into three main groups. The first relate to the way that individual health outcomes are measured (table 1). These include the choice of dimensions that go into HRQoL measures and their relative weights. There is evidence that the current generic HRQoL measures, used to generate QALYs,15 may undervalue particular conditions, such as sensory impairment and mental health.16 17 Other value elements that fall into this category reflect the way in which benefits are distributed between individuals or over time, patterns of risk and the redistribution of value based on the severity or the nature of the condition or its treatment.

Table 1

Value elements that relate to the evaluation of health outcomes

Some of these additional elements may risk double counting of benefits or may be in direct conflict with other elements. For example, ‘cure’ by its nature creates greater benefit than palliation and is likely to be closely related to ‘hope’, while ‘real-option value’, the opportunity to benefit from future developments, adds value to non-curative treatment (see table 1). Additional value may be claimed for conditions with a high burden of disease and end-of-life treatments, on the basis that the public consider that greater weight should be assigned to similar benefits in those with greater burden of disease or near the end of their life.18 However, since the methods used to derive the utility weights that are used to calculate QALYs rely on societal trade-offs between the chance or duration of survival and different health scenarios, these preferences may already be captured in the calculated QALY benefit.

There is also an issue of perspective. For example, both ‘hope’ and ‘insurance value’ relate, at least in part, to attitudes to risk.7 An individual may pay more than the average prize money for a lottery ticket in the hope of winning (risk seeking) or may pay more than the average claim pay-out expected from an insurance policy that covers a high impact event (risk averse). Although similar considerations may apply to risk-seeking or risk-averse attitudes to healthcare, it is not clear that these are relevant to insurance based or publicly funded healthcare systems that spread the risk across a large portfolio of conditions and treatments.

A second set of potential value elements relates to the wider implications of healthcare, rather than the risks and benefits for individual patients. These include equity considerations, wider societal impacts and additional value that might be attributed to treatments for rare conditions or innovative technologies (table 2).

Table 2

Value elements related to wider impacts of healthcare

NICE considered wider societal impact as part of a consultation on value-based pricing.18 They highlighted some potential discriminatory problems that it raises and suggested a ‘wider societal shortfall’ approach using average values to overcome this, but subsequently dropped the proposal following consultation.14 Equity considerations highlight the tension between providing equal and fair access to healthcare and targeted measures that aim to reduce health inequalities.19 Innovation and scientific spillover have been suggested, but it is not clear that a scientifically innovative product should be assigned value over that which is captured in health benefits. Although healthcare developments are often incremental there are difficulties in evaluating the potential future value of, as yet unknown, future developments. Furthermore, the implication of assigning value to a development that might underpin future products is that value should also be reduced to account for prior stages in development. For example, should all gene therapies be discounted to allow for the publicly funded human genome project on which they are founded?20 The current COVID-19 pandemic demonstrates the huge potential health and economic implications of contagious diseases, but in the absence of a specific epidemic, it is difficult to obtain credible estimates of the risks, costs and other implications of an, as yet unknown, contagious condition.

A final group of potential attributes that may warrant value are healthcare processes, as distinct from outcomes (table 3).

Table 3

Value elements relating to the process of care

Health technology assessment nominally includes all healthcare activities and care processes but, in practice, tends to focus on drugs, devices and diagnostic or therapeutic procedures, particularly new developments with significant cost implications. Considerable healthcare resources are devoted to service provision for chronic care and people with self-limiting conditions, which are less likely to be subject to such assessments.21 22 Expenditure is rarely ring-fenced, so value that is attributed to care processes must compete directly with the resources assigned to achieving health outcomes. Furthermore, the increasing elderly and frail population with complex health needs is blurring the boundary between social care and healthcare.23 24 In the UK, NICE has taken on responsibility for social care guidance and the close relationship and interdependencies between social care and healthcare raises questions about the need for a common basis for evaluation.25

Values attributable to aspects of care processes have received far less attention in the literature than health outcomes, but it is clear that there are many features of the process of care that are valued by society.26–28 These include more abstract concepts such as dignity, respect, compassion and autonomy, and more concrete issues such as choice over the location and timing of services, continuity of care, participation in decisions and the invasiveness of treatments.

NICE’s Citizens Council concluded that ‘…there are elements of care that provide huge benefit but cost nothing to provide, compassion for example, but that can get forgotten in the continuous discussion about resources and cost of care.’ 29 They identified other process attributes of importance including dignity, respect and individual choice that should be taken into account in considering equity and efficiency.30 Far from ‘costing nothing’, I suggest that compassionate care requires staffing levels that are adequate and consistent, staff with the appropriate levels of experience, training, appropriate provision for clinical supervision and oversight, and professional accountability.31 Dignity requires sufficient facilities to provide privacy for patients receiving end-of-life care and to avoid patients being treated on trolleys in corridors, while awaiting an available bed. Patient choice requires time for professionals to adequately present the pros and cons of all treatment options and enough spare capacity in the system to accommodate flexibility. All these aspects are costly to provide and are elements of healthcare provision that may be displaced by the financial pressures that result from the approval of new and costly technologies.32

This brief review demonstrates the large number of potential value elements that may be relevant to healthcare decisions. The value that society assigns to modern healthcare, as a public good, extends far beyond the maximisation of quality-adjusted survival, or any other measure of health processes and outcomes. Society may value healthcare activities that provide little or no health benefit, or may even risk harms, when measured in such terms, to attain other perceived benefits. Although such areas are often a cause for controversy, many healthcare systems will fund cosmetic procedures, contraception, infertility treatment, gender reassignment and, in some jurisdictions, abortion and assisted dying. These examples demonstrate the increasing remit of healthcare services in meeting societal objectives that are not confined to providing health benefits, but may encompass lifestyle preferences and respect for autonomy and self-determination.

Whose values?

A number of constituencies may have differing views on relevance and priority. There may be executive or political priorities set by government agencies or other authorities, views from expert advisory bodies, societal perspectives derived from the population in question, or the values of individual patients.

The widely accepted view with regard to prioritising dimensions of health outcomes, is that the most appropriate values are those of a societal sample from the general population of the community concerned.33 This has been extended to other aspects of value, and research has attempted to value preferences in other areas.34 Most agencies consider that societal preferences should guide such decisions; as NICE puts it these are ‘social value judgements [that] relate to society rather than science’.2

However, NICE’s original document on social value judgements has now been superseded, which raised concerns about a change in direction.35 Despite the Citizens Council concluding that rarity should not be given any special treatment2 and empirical evidence that society does not consider rarity to deserve special consideration,34 NICE produced guidance that prioritises ‘highly specialised technologies’.10 This allows some technologies to be approved at a threshold that potentially displaces at least ten QALYs for one gained from the new technology. NICE is not unique in this. Rarity is an attribute that has received considerable attention from several agencies and often attracts a premium,12 13 despite concerns that this lacks face validity.36

Decision making

Every decision or recommendation requires explicit or implicit choices about the value attributes that are relevant, and the weight attached to each. The process for determining these is closely related to the question of whose values are considered relevant. Much evidence-based guidance provides flexibility that may allow individual patients to participate in such decisions, incorporating their personal preferences. However, meaningful participation requires that they must be fully informed about the impact of those choices on the aspects of care and outcomes about which they may have such preferences. Geographical variation in practice might suggest that, where there is such flexibility, it is more often the clinicians’ rather than patients’ preferences that govern treatment decisions.37

There are a number of methods for obtaining empirical societal valuations through techniques such as discrete choice experiments,38 time-trade-off,4 contingent valuation39 and standard gamble.40 Such methods have been used to quantify societal preferences for HRQoL and many other aspects of healthcare, including location,41 42 process utilities,43 44 waiting times45 46 and other characteristics of care.38 47 However, in practice, attributes other than QALYs are considered by advisory committees through a deliberative process within a policy framework. Thus, the values assigned to specific attributes are rarely transparent and reflect those of policymakers and committee members rather than wider society.

The lack of transparency is compounded by the most common method of decision making, which relies on defining a WTP threshold for the acceptable ICER, which may be varied to account for additional value elements. This creates anomalies and distortions, as very different situations may result in similar ratios. Furthermore, if a more costly healthcare activity has other valuable attributes, but produces no QALY gain, such as the compassionate care of an unconscious patient, increased patient choice, or some of the other examples given above, these can never be considered cost-effective, however, high the threshold. A possible alternative to varying the WTP threshold is to use net costs or benefits (see box 1, for a worked example).

Box 1

Illustration of decision making based on willingness-to-pay thresholds or net benefit

Many agencies make decisions on the basis of comparing the incremental cost-effectiveness ratio (ICER) to a threshold range, with the acceptable threshold being varied to take account of additional elements of value. As the ICER is a ratio, this runs the risk of reducing the transparency of the decision, perhaps best illustrated by an example based on the National Institute for Health and Care Excellence (NICE) methodology.55

Consider the case of an intervention that has the benefit of a less invasive or more convenient process, such as outpatient rather than inpatient treatment, or oral rather than intravenous administration. If an advisory committee wishes to recognise this in its decision, it may vary the willingness to pay threshold within the range specified in the NICE methods guidance. For a high-cost intervention, such as regenerative medicines,72 where an ICER of £30 000 per quality-adjusted life-years (QALYs) might be based on an incremental cost of £300 000 per patient for a 10 QALY benefit, the effect of pushing the threshold from £20 000 to £30 000 per QALY is effectively to value this extra benefit at £100 000 per patient. In contrast, for an intervention which has low average cost and benefit, such as is the case for many preventative treatments or screening procedures, the average incremental cost per patient may be £30 for an incremental benefit of 0.001 QALY, so the same change in threshold will value the additional benefit of less invasive treatment at only £10 per patient.

An alternative approach would be to use a baseline threshold, currently £20 000 per QALY, to calculate net monetary/health cost or benefit per patient. This would then allow a transparent consideration of the additional cost, or reduced health that is justified by any additional considerations. In the above example the first new, less invasive procedure would have a net monetary cost of £100 000, while the new preventative treatment has a net cost of £10. It might be understandable that a committee would consider it worth a few additional pounds per patient for a less invasive procedure, but not several thousand pounds, a difference that would not be apparent when considering this in terms of ICERs and thresholds.

There have been attempts to make such decisions more transparent through formal processes, such as multicriteria decision analysis or augmented cost-effectiveness analysis.8 48–50 Although such methods may improve transparency, they also rely on the values and preferences of committee members, which may not be an adequate proxy for societal preferences. Committees will tend to be largely composed of those with a specific interest in the technologies under consideration and may not formally represent the views of those whose healthcare may be displaced by decisions with significant resource implications.

Furthermore, such methods assume that values assigned to different attributes may vary between decision problems. This may be acceptable for a decision that considers an exhaustive set of possible uses for defined resources, but it cannot account for the opportunity costs of unidentified and unknown activities that may be displaced. Although such decisions are open to consultation, the consultees on policy documents and specific guidance are largely those with a vested commercial, professional or personal interest in the new technologies, with those having greater commercial resources able to mount the strongest challenges.51 It is unlikely that those who may be disadvantaged by reduced, withdrawn or delayed services that might result from such decisions will even be aware of the potential effects, let alone be in a position to contribute meaningfully to the consultation.

Opportunity costs

For most bodies carrying out such appraisals, decisions are primarily based on estimated cost-effectiveness, compared with an explicit or implicit WTP threshold.52 The main rationale for such thresholds relates to ‘opportunity costs’, limiting the resources that can be allocated to an intervention to prevent it from displacing greater benefit elsewhere (described as ‘supply-side’ thresholds). For this purpose, benefit is commonly defined in terms of QALYs. There is no universally agreed mechanism for setting such thresholds, and there is considerable international variation.53 The little empirical evidence that exists regarding displaced activity, suggests that thresholds may currently be set too high, resulting in a net loss of health in QALY terms.54

Consideration of additional value elements creates difficulties in dealing with opportunity costs. In practice, where additional elements are currently taken into account, such as in guidance relating to rarity, end-of-life or other factors,10–13 55 this has been through elevation of the threshold, without any corresponding reduction in other areas, or consideration of the attribute in relation to displaced activity.

There are several potential ways to address the issue of opportunity costs. The first option is to ignore them completely. Where healthcare budgets are not fixed, some argue that the threshold should represent what society is able or willing to pay for healthcare (‘demand-side’ thresholds).52 The evidence suggests that such estimates tend to be higher than those based on potentially displaced activity and are thus likely to result in increasing healthcare expenditure.56 Healthcare systems that do not have a clearly fixed budget may accept increasing costs to cover new technologies that are considered cost-effective against such a threshold.

The implication of such a policy is that healthcare expenditure is likely to rise. The overall increase in the resources devoted to healthcare will raise the supply side threshold, until the point is reached at which it matches the demand-side threshold, when opportunity costs will come into play. There will always be opportunity costs, but these may fall outside the healthcare system, as an increasing proportion of personal or collective income is assigned to healthcare and displaces other opportunities for use of those resources. Under these circumstances, there is no theoretical need to alter the threshold to account for additional value elements, but the failure to do so will result in a more rapid inflation of healthcare costs, and accelerate the point at which opportunity costs become relevant.

Where opportunity costs are considered, any threshold could, in theory, be adjusted to allow for the prevalence of additional value elements in potentially displaced activity. This might be achieved by combining or weighting other attributes to create a ‘compound’ measure of benefit, for which a threshold could be sought, or by establishing a set of per-patient or per-service net benefit thresholds, at which certain attributes would be considered acceptable. However, even evaluating the QALY value of displaced activity is difficult and, with the lack of clearly agreed attributes and metrics, this would be a complex task.

A second approach is to consider disinvestment decisions using identical criteria to investment decisions, creating a level playing field, in which the criteria used to judge both investment and disinvestment are aligned.57 However, it is unusual for cost pressures to result in savings through disinvestment in a particular technology.58 More frequently, these are achieved through reducing service levels, delaying or limiting access, or dilution of services.32

A third possibility is to consider new technologies within a ring-fenced development budget, similar to the current arrangements for the Cancer Drugs Fund (CDF) in the UK.59 The CDF was introduced in 2011 with a fund of £200M for new cancer drugs and rapidly outgrew its budget, with a total spend of £466M in 2015/2016, amidst criticism that it was a poor use of resources.60 NICE was given responsibility for managing the CDF in 2016, and the new arrangements gave NICE the potential to recommend drugs for interim funding, subject to the pharmaceutical companies agreeing financial controls, which prevent overspend by applying a rebate where demand outstrips the available resources.59

Such arrangements provide a practical solution that allows multiple criteria to be considered in deciding between competing technologies and is a way in which it might be acceptable to use different weighting for particular attributes, without needing to consider the impact on unidentified and potentially displaced aspects of care. However, such arrangements risk creating different thresholds for new and established technologies or creating different thresholds from year to year, depending on the current competition for resources.


It is widely accepted that the QALY alone is not a sufficient measure of value for quantifying the benefit of healthcare. However, it is not clear that the additional elements that are currently considered align with societal preferences, elements relating to care processes appear to be missing, and little attempt has been made to account for opportunity costs. I suggest that a set of conditions are required to satisfy the relevance criteria of A4R:

  1. All relevant attributes are considered: it is not sufficient for those which are considered, to be relevant, if additional relevant criteria are omitted.

  2. Attributes that are not relevant and potential overlaps are excluded.

  3. Attributes are weighted, formally or informally, according to the values and preferences of the appropriate constituency. Generally, this will be societal or patient values.

  4. Attributes and weights should remain consistent between decisions in different circumstances that compete for the same pool of resources.

  5. Thresholds should recognise the opportunity cost of potentially displaced activity, including the prevalence of all relevant attributes in this activity.

If society is to distribute limited healthcare resources in a morally justifiable fashion, then it seems appropriate that all technologies and caring processes are judged against a consistent set of criteria that reflect societal preferences. To focus on the QALY, or any other measure that purely reflects health outcomes, without considering the importance that the public attributes to aspects such as processes of care, self-determination and equity, risks displacing highly valued aspects of the benefits that health services provide and unjustly disadvantaging certain patient groups.

Ethics statements

Patient consent for publication



  • Contributors All aspects of the work were carried out by the sole author.


  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Strictly, the NICE remit does not extend to the whole of the UK, with some responsibilities for healthcare devolved to Wales, Scotland and Northern Ireland. Thus, the constituency varies for different aspects of NICE guidance.

  • Examples of such bodies include; Institute for Clinical and Economic Review in the US (, CADTH in Canada ( and the Australian Medical Services Advisory Committee (

  • Since this article was written, NICE has disbanded the Citizens Council, removed its reports from their website and developed a new policy for public engagement (see:

Other content recommended for you