Criteria catalog for sustainable software


This document is the outcome of the first of five work packages in the German Federal Environment Agency's UFOPLAN project “Sustainable software design—Development and application of criteria for resource-efficient software products with consideration of existing methods.”

The goal of the project is to develop a method for evaluating the environmental impacts of software products. This method is intended to support both the procurement of software products with consideration of environmental criteria and the development of resource-efficient software. In particular, the method is supposed to enable a comparison of two given software products with similar functionality in terms of their impacts on natural resources. Based on the formulation of ambitious minimum standards, the method will also help to define criteria for awarding an environmental or quality label to sustainable software products.

Thus, the project makes a contribution to expanding the focus of “Green IT” beyond the hardware level to include the software level. Since software products are immaterial goods, one problem arising is to capture the indirect material impacts of these products in conceptual and methodological terms.

A product's environmental impacts generally occur through the use of natural resources1 during the life cycle of the product. We take on this life-cycle perspective in relation to software products as well (see figure 1). We take into account that the hardware needed to operate a software product must be produced, supplied with electricity, and disposed of at the end of its useful life. Thus, every software product is responsible for a quantifiable fraction of the life cycle of all the hardware products required for its operation (programmable devices of any kind, peripheral devices, and storage media).

Figure 1: Life cycles of hardware and software (horizontal dimension) and the resource use induced by the life cycles (vertical dimension)

Because it takes a life-cycle perspective, this approach can be expanded to include the social aspects of producing the raw materials as well as the working conditions in hardware production and disposal; our focus is on the environmental aspects.

At the software level, we intentionally limit our perspective to the use phase in the following. The goal of the criteria defined here is to evaluate a software product on the basis of characteristics that are observable in the use phase, be it by the users themselves or by persons conducting special tests. In principle, it would also be possible to take the production phase of software into account by broadening the approach. However, evaluating the process of software development seems less important to us than influencing it, among other things by making recommendations addressed to those responsible for software development. A software development guide will be prepared in a later phase of this project.

In general, we examine standard application software in this project, in other words, neither system software nor special application software for a small number of users. In the latter case, the resources required for development would surely have to be included.

As a matter of principle, especially the evaluation of widespread software products requires not just a snapshot, but consideration of the use of the software product (including several versions) over longer periods of time. Only from this perspective does the question as to software-induced purchasing of hardware become relevant, for example.

Expressed in abstract terms, our analysis focuses on two flows caused by a software product:

  • the flow of hardware through the organization using it (new hardware to waste),
  • the flow of energy through the hardware (electricity to waste heat).

If a software product causes significantly lower hardware and energy flows than competing products with similar functionality, then it can be considered "sustainable."2

The resource consumption induced by the flow of hardware can be estimated by applying life cycle assessment (LCA) methods. Life cycle inventories for production and disposal of the most important hardware components exist for this purpose, and we take them as given without entering a detailed discussion. Energy flow can also be evaluated with LCA methods; the various methods for generating electricity have been examined sufficiently; therefore, we also take this data as given.

That is why it is sufficient to use the criteria developed in this project to address the impact of software on the required hardware capacities. If one imagines a chain of impacts from software characteristics to natural resource use, then we analyze exclusively the section of the chain of impacts from the software to the hardware products and their electricity consumption3, because it is the only part that is specific to our object of investigation.

Practicable criteria are necessary to be able to assess the sustainability of software with reference to the hardware and energy flows it induces. These criteria can then be applied, e.g., to inform those responsible for software development or software procurement or to award an environmental label.

The set of criteria proposed here focuses on environmental impacts resulting from the operation of a software product. This does not rule out that the awarding of environmental labels also includes further criteria regarding the process of software development (e.g., compliance with ILO4 standards when outsourcing programming work), the functionality of the software (e.g., accessibility, or exclusion of particular categories such as violent games), or other aspects. It seems important to us, however, to treat the impacts of software characteristics on natural resource consumption as a clearly defined object of research from the outset and not to confound it with other questions.

Studies and criteria are available for many of these neighboring questions and can be used to complement our set of criteria.

A tree (a hierarchy) of criteria and indicators is described on the following pages. The leaves of the tree are indicators serving to operationalize the relevant higher-level criterion. This document's table of contents provides a complete overview of the criteria.

An evaluation model whose structure we will lay out in this project will later be applied to combine (arithmetically or logically) the indicators to criteria and then to combine the criteria to higher-level criteria. The actual evaluation by means of weighting, normalization functions, or designating mandatory criteria remains up to the German Federal Environment Agency or other organizations applying our criteria and may change over time.

Weighting of indicators and criteria may vary depending on the class of software in an existing evaluation model. In particular, indicators or criteria can be assigned a weight of zero if they are not applicable or irrelevant to certain classes of software. We have developed a classification of software products that is adapted to the application of our criteria (see also Appendix A).

  • local application
  • application with remote data storage
  • application with remote processing
  • remote service

These four classes are relevant to our approach as they attempt to encompass not only the resource consumption induced by local execution of the software, but also the resource consumption induced remotely, from network infrastructure to dedicated servers to the cloud. Otherwise, the approach would be useless because the criteria could be fulfilled by shifting environmental impacts elsewhere. A detailed impact model describing the various impact paths from software characteristics via hardware capacities to natural resources can be found in Appendix B.

In the following main section of this document, each criterion is characterized by

  • a one- to three-digit number, depending on its level,
  • a designation (heading),
  • a question explaining the criterion,
  • a comment following the question, as appropriate.

Each criterion in the lowest position in the hierarchy is operationalized by indicators identified with a lower-case letter.

This tree of criteria and indicators is based on a comprehensive literature research on criteria for evaluating software analyzing more than 130 such criteria from more than 60 sources. A draft of the set of criteria was discussed with experts from the scientific community, public agencies, and industry at a stakeholder workshop on March 11th, 2016. The participants' feedback during and after the workshop was taken into consideration during the revision of the document.

Some of the following criteria and indicators refer to a "reference system", a "standard configuration", or a "standard usage scenario"; these and other key concepts are defined in the glossary.


Prof. Dr. Lorenz M. Hilty
Yuliyan Maksimov, M.Sc.

Research Group Informatics and Sustainability
Institute for Informatics, University of Zurich
Binzmühlestrasse 14, CH-8050 Zurich

Prof. Dr. Stefan Naumann
Andreas Filler, M.Sc.
Achim Guldner, M.Sc.
Eva Kern, M.Sc.

Institute for Software Systems
Trier University of Applied Sciences, Environemental Campus Birkenfeld
Post Box 13 80, D-55761 Birkenfeld

Dipl.-Ing. Jens Gröger
Dr. Andreas Köhler

Öko-Institut e.V.
Post Box 17 71, D-79017 Freiburg

By order of the German Federal Environmental Agency

Term Description
Energy efficiency Generally, the amount of “useful work” divided by the amount of energy it requires. In the context of this document, “useful work” is operationalized as the successful execution of standard usage scenarios.
Hardware The material goods required to run programs or to store or transport data.
Hardware capacity Quantifiable characteristic of a hardware system which represents its performance limit on a given dimension of performance (e.g., working memory capacity, computing power, bandwidth).
Hardware system Delimitable unit of hardware that performs defined functions.
Indicator An empirically determinable quantity that provides insight into a matter that cannot be measured directly. The indicators proposed in this document have different levels of measurement. In some cases, researchers will have to settle for an ordinal scale (e.g., “insufficient”, “sufficient”, “good”, “very good”, or even merely “fulfilled”, “not fulfilled”) to avoid giving the false impression of non-existent precision arising from a cardinal scale.
Reference system A hardware system that is defined as generally customary in terms of its most important capacities (e.g., working memory, processor performance) during a defined period of time (e.g., one year). The purpose of the reference system is to be able to express indicators such as “minimum local memory” in relation to a reference value (currently “customary” memory).
Resource In the context of this document, a natural resource, in particular a raw material, a form of energy, or also the capacity of an environmental medium to absorb emissions. To differentiate natural resources from technical ones, especially hardware resources, the more precise term “hardware capacities” is used here for the latter. Since using hardware capacities always results in using natural resources, this distinction (which ultimately amounts to a definitionally difficult differentiation between the ecosphere and the technosphere) is not of decisive importance here.
Resource efficiency Generally, the amount of “useful work” divided by the amount of resources it requires. In the context of this document, “useful work” is operationalized as the successful execution of standard usage scenarios.
Software Programs and data in digital form.
Software product A delimitable unit of programs and data for which a license is available.
Standard configuration A set of conditions, defined as a reference, under which a given software product is run; it includes the parameter settings selected during installation or operation, the system software provided, potentially additional software products required for operation, as well as the reference system at the hardware level.
Standard usage scenario A usage scenario that is used for testing a software product and is supposed to be as representative as possible for the customary use case.
Usage pattern Abstracted form of a sequence of interactions with a given software product.
Usage scenario Description of a usage pattern which is generally machine executable.

[1] Definitions of "resource" and other key terms are provided in the glossary. In this document, we reserve the term "resource" for natural resources and mostly avoid the technical term “hardware resource” by describing hardware resources directly in terms of capacities, i.e., quantifiable aspects of their performance.

[2] The functionality of a software product, and thus its utility, will not be evaluated here. The goal is exclusively to estimate and evaluate the amount of resource use it induces. A given amount of useful work can be related to the amount of resource use induced to determine efficiency.

[3] In some cases, it may be necessary to broaden this perspective and take the flow of consumables such as paper or toner through the hardware into account, analogously to the flow of energy. Whether this is the case for a given software product and which consumables are relevant can be decided based on a first screening.

[4] International Labour Organization

back-to-top nach oben