# sqlite

Package sqlite is a cgo-free port of SQLite. Although you could see mattn's driver (`github.com/mattn/go-sqlite3`) in go.mod file, we import it for tests only.

SQLite is an in-process implementation of a self-contained, serverless,
zero-configuration, transactional SQL database engine.

## Thanks

This project is sponsored by Schleibinger Geräte Teubert u. Greim GmbH by
allowing one of the maintainers to work on it also in office hours.

## Installation

    $ go get modernc.org/sqlite

## Documentation

[pkg.go.dev/modernc.org/sqlite](https://pkg.go.dev/modernc.org/sqlite)

## Builders

[modern-c.appspot.com/-/builder/?importpath=modernc.org%2fsqlite](https://modern-c.appspot.com/-/builder/?importpath=modernc.org%2fsqlite)

## Speedtest1

Numbers for the pure Go version were produced by

     ~/src/modernc.org/sqlite/speedtest1$ go build && ./speedtest1

Numbers for the pure C version were produced by

     ~/src/modernc.org/sqlite/testdata/sqlite-src-3410200/test$ gcc speedtest1.c ../../sqlite-amalgamation-3410200/sqlite3.c -lpthread -ldl && ./a.out

The results are from Go version 1.20.4 and GCC version 10.2.1 on a
Linux/amd64 machine, CPU: AMD Ryzen 9 3900X 12-Core Processor × 24, 128GB
RAM. Shown are the best of 3 runs.

     Go											C

     -- Speedtest1 for SQLite 3.41.2 2023-03-22 11:56:21 0d1fc92f94cb6b76bffe3ec34d69	-- Speedtest1 for SQLite 3.41.2 2023-03-22 11:56:21 0d1fc92f94cb6b76bffe3ec34d69
      100 - 50000 INSERTs into table with no index......................    0.071s            100 - 50000 INSERTs into table with no index......................    0.077s
      110 - 50000 ordered INSERTS with one index/PK.....................    0.114s            110 - 50000 ordered INSERTS with one index/PK.....................    0.082s
      120 - 50000 unordered INSERTS with one index/PK...................    0.137s            120 - 50000 unordered INSERTS with one index/PK...................    0.099s
      130 - 25 SELECTS, numeric BETWEEN, unindexed......................    0.083s            130 - 25 SELECTS, numeric BETWEEN, unindexed......................    0.091s
      140 - 10 SELECTS, LIKE, unindexed.................................    0.210s            140 - 10 SELECTS, LIKE, unindexed.................................    0.120s
      142 - 10 SELECTS w/ORDER BY, unindexed............................    0.276s            142 - 10 SELECTS w/ORDER BY, unindexed............................    0.182s
      145 - 10 SELECTS w/ORDER BY and LIMIT, unindexed..................    0.183s            145 - 10 SELECTS w/ORDER BY and LIMIT, unindexed..................    0.099s
      150 - CREATE INDEX five times.....................................    0.172s            150 - CREATE INDEX five times.....................................    0.127s
      160 - 10000 SELECTS, numeric BETWEEN, indexed.....................    0.080s            160 - 10000 SELECTS, numeric BETWEEN, indexed.....................    0.078s
      161 - 10000 SELECTS, numeric BETWEEN, PK..........................    0.080s            161 - 10000 SELECTS, numeric BETWEEN, PK..........................    0.078s
      170 - 10000 SELECTS, text BETWEEN, indexed........................    0.187s            170 - 10000 SELECTS, text BETWEEN, indexed........................    0.169s
      180 - 50000 INSERTS with three indexes............................    0.196s            180 - 50000 INSERTS with three indexes............................    0.154s
      190 - DELETE and REFILL one table.................................    0.200s            190 - DELETE and REFILL one table.................................    0.155s
      200 - VACUUM......................................................    0.180s            200 - VACUUM......................................................    0.142s
      210 - ALTER TABLE ADD COLUMN, and query...........................    0.004s            210 - ALTER TABLE ADD COLUMN, and query...........................    0.005s
      230 - 10000 UPDATES, numeric BETWEEN, indexed.....................    0.093s            230 - 10000 UPDATES, numeric BETWEEN, indexed.....................    0.080s
      240 - 50000 UPDATES of individual rows............................    0.153s            240 - 50000 UPDATES of individual rows............................    0.137s
      250 - One big UPDATE of the whole 50000-row table.................    0.024s            250 - One big UPDATE of the whole 50000-row table.................    0.019s
      260 - Query added column after filling............................    0.004s            260 - Query added column after filling............................    0.005s
      270 - 10000 DELETEs, numeric BETWEEN, indexed.....................    0.278s            270 - 10000 DELETEs, numeric BETWEEN, indexed.....................    0.263s
      280 - 50000 DELETEs of individual rows............................    0.188s            280 - 50000 DELETEs of individual rows............................    0.180s
      290 - Refill two 50000-row tables using REPLACE...................    0.411s            290 - Refill two 50000-row tables using REPLACE...................    0.359s
      300 - Refill a 50000-row table using (b&1)==(a&1).................    0.175s            300 - Refill a 50000-row table using (b&1)==(a&1).................    0.151s
      310 - 10000 four-ways joins.......................................    0.427s            310 - 10000 four-ways joins.......................................    0.365s
      320 - subquery in result set......................................    0.440s            320 - subquery in result set......................................    0.521s
      400 - 70000 REPLACE ops on an IPK.................................    0.125s            400 - 70000 REPLACE ops on an IPK.................................    0.106s
      410 - 70000 SELECTS on an IPK.....................................    0.081s            410 - 70000 SELECTS on an IPK.....................................    0.078s
      500 - 70000 REPLACE on TEXT PK....................................    0.174s            500 - 70000 REPLACE on TEXT PK....................................    0.116s
      510 - 70000 SELECTS on a TEXT PK..................................    0.153s            510 - 70000 SELECTS on a TEXT PK..................................    0.117s
      520 - 70000 SELECT DISTINCT.......................................    0.083s            520 - 70000 SELECT DISTINCT.......................................    0.067s
      980 - PRAGMA integrity_check......................................    0.436s            980 - PRAGMA integrity_check......................................    0.377s
      990 - ANALYZE.....................................................    0.107s            990 - ANALYZE.....................................................    0.038s
            TOTAL.......................................................    5.525s                  TOTAL.......................................................    4.637s

This particular test executes 16.1% faster in the C version.

## Troubleshooting

* Q: **How can I write to a database concurrently without getting the `database is locked` error (or `SQLITE_BUSY`)?**
     * A: You can't. The C sqlite implementation does not allow concurrent writes, and this libary does not modify that behaviour. You can, however, use [DB.SetMaxOpenConns(1)](https://pkg.go.dev/database/sql#DB.SetMaxOpenConns) so that only 1 connection is ever used by the `DB`, allowing concurrent access to DB without making the writes concurrent. More information on issues [#65](https://gitlab.com/cznic/sqlite/-/issues/65) and [#106](https://gitlab.com/cznic/sqlite/-/issues/106).


# Contributing Guidelines

Welcome to Kubernetes. We are excited about the prospect of you joining our [community](https://git.k8s.io/community)! The Kubernetes community abides by the CNCF [code of conduct](code-of-conduct.md). Here is an excerpt:

_As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities._

## Criteria for adding code here

This library adapts the stdlib `encoding/json` decoder to be compatible with
Kubernetes JSON decoding, and is not expected to actively add new features.

It may be updated with changes from the stdlib `encoding/json` decoder.

Any code that is added must:
* Have full unit test and benchmark coverage
* Be backward compatible with the existing exposed go API
* Have zero external dependencies
* Preserve existing benchmark performance
* Preserve compatibility with existing decoding behavior of `UnmarshalCaseSensitivePreserveInts()` or `UnmarshalStrict()`
* Avoid use of `unsafe`

## Getting Started

We have full documentation on how to get started contributing here:

<!---
If your repo has certain guidelines for contribution, put them here ahead of the general k8s resources
-->

- [Contributor License Agreement](https://git.k8s.io/community/CLA.md) Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests
- [Kubernetes Contributor Guide](https://git.k8s.io/community/contributors/guide) - Main contributor documentation, or you can just jump directly to the [contributing section](https://git.k8s.io/community/contributors/guide#contributing)
- [Contributor Cheat Sheet](https://git.k8s.io/community/contributors/guide/contributor-cheatsheet) - Common resources for existing developers

## Community, discussion, contribution, and support

You can reach the maintainers of this project via the
[sig-api-machinery mailing list / channels](https://github.com/kubernetes/community/tree/master/sig-api-machinery#contact).

## Mentorship

- [Mentoring Initiatives](https://git.k8s.io/community/mentoring) - We have a diverse set of mentorship programs available that are always looking for volunteers!

Perforator is a cluster-wide continuous profiling tool designed for large data centers


# sigs.k8s.io/json

[![Go Reference](https://pkg.go.dev/badge/sigs.k8s.io/json.svg)](https://pkg.go.dev/sigs.k8s.io/json)

## Introduction

This library is a subproject of [sig-api-machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery#json).
It provides case-sensitive, integer-preserving JSON unmarshaling functions based on `encoding/json` `Unmarshal()`.

## Compatibility

The `UnmarshalCaseSensitivePreserveInts()` function behaves like `encoding/json#Unmarshal()` with the following differences:

- JSON object keys are treated case-sensitively.
  Object keys must exactly match json tag names (for tagged struct fields)
  or struct field names (for untagged struct fields).
- JSON integers are unmarshaled into `interface{}` fields as an `int64` instead of a
  `float64` when possible, falling back to `float64` on any parse or overflow error.
- Syntax errors do not return an `encoding/json` `*SyntaxError` error.
  Instead, they return an error which can be passed to `SyntaxErrorOffset()` to obtain an offset.

## Additional capabilities

The `UnmarshalStrict()` function decodes identically to `UnmarshalCaseSensitivePreserveInts()`,
and also returns non-fatal strict errors encountered while decoding:

- Duplicate fields encountered
- Unknown fields encountered

### Community, discussion, contribution, and support

You can reach the maintainers of this project via the
[sig-api-machinery mailing list / channels](https://github.com/kubernetes/community/tree/master/sig-api-machinery#contact).

### Code of conduct

Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](code-of-conduct.md).

[owners]: https://git.k8s.io/community/contributors/guide/owners.md
[Creative Commons 4.0]: https://git.k8s.io/website/LICENSE


# Contributing Guidelines

Welcome to Kubernetes. We are excited about the prospect of you joining our [community](https://github.com/kubernetes/community)! The Kubernetes community abides by the CNCF [code of conduct](code-of-conduct.md). Here is an excerpt:

_As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities._

## Getting Started

We have full documentation on how to get started contributing here:

<!---
If your repo has certain guidelines for contribution, put them here ahead of the general k8s resources
-->

- [Contributor License Agreement](https://git.k8s.io/community/CLA.md) Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests
- [Kubernetes Contributor Guide](http://git.k8s.io/community/contributors/guide) - Main contributor documentation, or you can just jump directly to the [contributing section](http://git.k8s.io/community/contributors/guide#contributing)
- [Contributor Cheat Sheet](https://git.k8s.io/community/contributors/guide/contributor-cheatsheet.md) - Common resources for existing developers

## Mentorship

- [Mentoring Initiatives](https://git.k8s.io/community/mentoring) - We have a diverse set of mentorship programs available that are always looking for volunteers!

<!---
Custom Information - if you're copying this template for the first time you can add custom content here, for example:

## Contact Information

- [Slack channel](https://kubernetes.slack.com/messages/kubernetes-users) - Replace `kubernetes-users` with your slack channel string, this will send users directly to your channel.
- [Mailing list](URL)

-->


# YAML marshaling and unmarshaling support for Go

[![Build Status](https://travis-ci.org/kubernetes-sigs/yaml.svg)](https://travis-ci.org/kubernetes-sigs/yaml)

kubernetes-sigs/yaml is a permanent fork of [ghodss/yaml](https://github.com/ghodss/yaml).

## Introduction

A wrapper around [go-yaml](https://github.com/go-yaml/yaml) designed to enable a better way of handling YAML when marshaling to and from structs.

In short, this library first converts YAML to JSON using go-yaml and then uses `json.Marshal` and `json.Unmarshal` to convert to or from the struct. This means that it effectively reuses the JSON struct tags as well as the custom JSON methods `MarshalJSON` and `UnmarshalJSON` unlike go-yaml. For a detailed overview of the rationale behind this method, [see this blog post](http://web.archive.org/web/20190603050330/http://ghodss.com/2014/the-right-way-to-handle-yaml-in-golang/).

## Compatibility

This package uses [go-yaml](https://github.com/go-yaml/yaml) and therefore supports [everything go-yaml supports](https://github.com/go-yaml/yaml#compatibility).

## Caveats

**Caveat #1:** When using `yaml.Marshal` and `yaml.Unmarshal`, binary data should NOT be preceded with the `!!binary` YAML tag. If you do, go-yaml will convert the binary data from base64 to native binary data, which is not compatible with JSON. You can still use binary in your YAML files though - just store them without the `!!binary` tag and decode the base64 in your code (e.g. in the custom JSON methods `MarshalJSON` and `UnmarshalJSON`). This also has the benefit that your YAML and your JSON binary data will be decoded exactly the same way. As an example:

```
BAD:
	exampleKey: !!binary gIGC

GOOD:
	exampleKey: gIGC
... and decode the base64 data in your code.
```

**Caveat #2:** When using `YAMLToJSON` directly, maps with keys that are maps will result in an error since this is not supported by JSON. This error will occur in `Unmarshal` as well since you can't unmarshal map keys anyways since struct fields can't be keys.

## Installation and usage

To install, run:

```
$ go get sigs.k8s.io/yaml
```

And import using:

```
import "sigs.k8s.io/yaml"
```

Usage is very similar to the JSON library:

```go
package main

import (
	"fmt"

	"sigs.k8s.io/yaml"
)

type Person struct {
	Name string `json:"name"` // Affects YAML field names too.
	Age  int    `json:"age"`
}

func main() {
	// Marshal a Person struct to YAML.
	p := Person{"John", 30}
	y, err := yaml.Marshal(p)
	if err != nil {
		fmt.Printf("err: %v\n", err)
		return
	}
	fmt.Println(string(y))
	/* Output:
	age: 30
	name: John
	*/

	// Unmarshal the YAML back into a Person struct.
	var p2 Person
	err = yaml.Unmarshal(y, &p2)
	if err != nil {
		fmt.Printf("err: %v\n", err)
		return
	}
	fmt.Println(p2)
	/* Output:
	{John 30}
	*/
}
```

`yaml.YAMLToJSON` and `yaml.JSONToYAML` methods are also available:

```go
package main

import (
	"fmt"

	"sigs.k8s.io/yaml"
)

func main() {
	j := []byte(`{"name": "John", "age": 30}`)
	y, err := yaml.JSONToYAML(j)
	if err != nil {
		fmt.Printf("err: %v\n", err)
		return
	}
	fmt.Println(string(y))
	/* Output:
	age: 30
	name: John
	*/
	j2, err := yaml.YAMLToJSON(y)
	if err != nil {
		fmt.Printf("err: %v\n", err)
		return
	}
	fmt.Println(string(j2))
	/* Output:
	{"age":30,"name":"John"}
	*/
}
```


.. image:: https://img.shields.io/pypi/v/backports.functools_lru_cache.svg
   :target: https://pypi.org/project/backports.functools_lru_cache

.. image:: https://img.shields.io/pypi/pyversions/backports.functools_lru_cache.svg

.. image:: https://github.com/jaraco/backports.functools_lru_cache/workflows/tests/badge.svg
   :target: https://github.com/jaraco/backports.functools_lru_cache/actions?query=workflow%3A%22tests%22
   :alt: tests

.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json
    :target: https://github.com/astral-sh/ruff
    :alt: Ruff

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/psf/black
   :alt: Code style: Black

.. image:: https://readthedocs.org/projects/backportsfunctools_lru_cache/badge/?version=latest
   :target: https://backportsfunctools_lru_cache.readthedocs.io/en/latest/?badge=latest

.. image:: https://img.shields.io/badge/skeleton-2023-informational
   :target: https://blog.jaraco.com/skeleton

.. image:: https://tidelift.com/badges/package/pypi/backports.functools_lru_cache
   :target: https://tidelift.com/subscription/pkg/pypi-backports.functools_lru_cache?utm_source=pypi-backports.functools_lru_cache&utm_medium=readme

Backport of functools.lru_cache from Python 3.3 as published at `ActiveState
<http://code.activestate.com/recipes/578078/>`_.

Usage
=====

Consider using this technique for importing the 'lru_cache' function::

    try:
        from functools import lru_cache
    except ImportError:
        from backports.functools_lru_cache import lru_cache

For Enterprise
==============

Available as part of the Tidelift Subscription.

This project and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use.

`Learn more <https://tidelift.com/subscription/pkg/pypi-backports.functools_lru_cache?utm_source=pypi-backports.functools_lru_cache&utm_medium=referral&utm_campaign=github>`_.


backports.shutil_get_terminal_size
==================================

A backport of the `get_terminal_size`_ function from Python 3.3's shutil.

Unlike the original version it is written in pure Python rather than C,
so it might be a tiny bit slower.

.. _get_terminal_size: https://docs.python.org/3/library/shutil.html#shutil.get_terminal_size

Example usage
-------------

    >>> from backports.shutil_get_terminal_size import get_terminal_size
    >>> get_terminal_size()
    terminal_size(columns=105, lines=33)


.. image:: https://img.shields.io/pypi/v/configparser.svg
   :target: https://pypi.org/project/configparser

.. image:: https://img.shields.io/pypi/pyversions/configparser.svg

.. image:: https://img.shields.io/travis/jaraco/configparser/master.svg
   :target: https://travis-ci.org/jaraco/configparser

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/ambv/black
   :alt: Code style: Black

.. .. image:: https://img.shields.io/appveyor/ci/jaraco/configparser/master.svg
..    :target: https://ci.appveyor.com/project/jaraco/configparser/branch/master

.. image:: https://readthedocs.org/projects/configparser/badge/?version=latest
   :target: https://configparser.readthedocs.io/en/latest/?badge=latest

.. image:: https://tidelift.com/badges/package/pypi/configparser
   :target: https://tidelift.com/subscription/pkg/pypi-configparser?utm_source=pypi-configparser&utm_medium=readme

The ancient ``ConfigParser`` module available in the standard library 2.x has
seen a major update in Python 3.2. This is a backport of those changes so that
they can be used directly in Python 2.6 - 3.5.

To use the ``configparser`` backport instead of the built-in version on both
Python 2 and Python 3, simply import it explicitly as a backport::

  from backports import configparser

If you'd like to use the backport on Python 2 and the built-in version on
Python 3, use that invocation instead::

  import configparser

For detailed documentation consult the vanilla version at
http://docs.python.org/3/library/configparser.html.

Why you'll love ``configparser``
--------------------------------

Whereas almost completely compatible with its older brother, ``configparser``
sports a bunch of interesting new features:

* full mapping protocol access (`more info
  <http://docs.python.org/3/library/configparser.html#mapping-protocol-access>`_)::

    >>> parser = ConfigParser()
    >>> parser.read_string("""
    [DEFAULT]
    location = upper left
    visible = yes
    editable = no
    color = blue

    [main]
    title = Main Menu
    color = green

    [options]
    title = Options
    """)
    >>> parser['main']['color']
    'green'
    >>> parser['main']['editable']
    'no'
    >>> section = parser['options']
    >>> section['title']
    'Options'
    >>> section['title'] = 'Options (editable: %(editable)s)'
    >>> section['title']
    'Options (editable: no)'

* there's now one default ``ConfigParser`` class, which basically is the old
  ``SafeConfigParser`` with a bunch of tweaks which make it more predictable for
  users. Don't need interpolation? Simply use
  ``ConfigParser(interpolation=None)``, no need to use a distinct
  ``RawConfigParser`` anymore.

* the parser is highly `customizable upon instantiation
  <http://docs.python.org/3/library/configparser.html#customizing-parser-behaviour>`__
  supporting things like changing option delimiters, comment characters, the
  name of the DEFAULT section, the interpolation syntax, etc.

* you can easily create your own interpolation syntax but there are two powerful
  implementations built-in (`more info
  <http://docs.python.org/3/library/configparser.html#interpolation-of-values>`__):

  * the classic ``%(string-like)s`` syntax (called ``BasicInterpolation``)

  * a new ``${buildout:like}`` syntax (called ``ExtendedInterpolation``)

* fallback values may be specified in getters (`more info
  <http://docs.python.org/3/library/configparser.html#fallback-values>`__)::

    >>> config.get('closet', 'monster',
    ...            fallback='No such things as monsters')
    'No such things as monsters'

* ``ConfigParser`` objects can now read data directly `from strings
  <http://docs.python.org/3/library/configparser.html#configparser.ConfigParser.read_string>`__
  and `from dictionaries
  <http://docs.python.org/3/library/configparser.html#configparser.ConfigParser.read_dict>`__.
  That means importing configuration from JSON or specifying default values for
  the whole configuration (multiple sections) is now a single line of code. Same
  goes for copying data from another ``ConfigParser`` instance, thanks to its
  mapping protocol support.

* many smaller tweaks, updates and fixes

A few words about Unicode
-------------------------

``configparser`` comes from Python 3 and as such it works well with Unicode.
The library is generally cleaned up in terms of internal data storage and
reading/writing files.  There are a couple of incompatibilities with the old
``ConfigParser`` due to that. However, the work required to migrate is well
worth it as it shows the issues that would likely come up during migration of
your project to Python 3.

The design assumes that Unicode strings are used whenever possible [1]_.  That
gives you the certainty that what's stored in a configuration object is text.
Once your configuration is read, the rest of your application doesn't have to
deal with encoding issues. All you have is text [2]_. The only two phases when
you should explicitly state encoding is when you either read from an external
source (e.g. a file) or write back.

Versioning
----------

This project uses `semver <https://semver.org/spec/v2.0.0.html>`_ to
communicate the impact of various releases while periodically syncing
with the upstream implementation in CPython.
`The changelog <https://github.com/jaraco/configparser/blob/master/CHANGES.rst>`_
serves as a reference indicating which versions incorporate
which upstream functionality.

Prior to the ``4.0.0`` release, `another scheme
<https://github.com/jaraco/configparser/blob/3.8.1/README.rst#versioning>`_
was used to associate the CPython and backports releases.

Maintenance
-----------

This backport was originally authored by Łukasz Langa, the current vanilla
``configparser`` maintainer for CPython and is currently maintained by
Jason R. Coombs:

* `configparser repository <https://github.com/jaraco/configparser>`_

* `configparser issue tracker <https://github.com/jaraco/configparser/issues>`_

Security Contact
----------------

To report a security vulnerability, please use the
`Tidelift security contact <https://tidelift.com/security>`_.
Tidelift will coordinate the fix and disclosure.

Conversion Process
------------------

This section is technical and should bother you only if you are wondering how
this backport is produced. If the implementation details of this backport are
not important for you, feel free to ignore the following content.

``configparser`` is converted using `python-future
<http://python-future.org>`_. The project takes the following
branching approach:

* the ``3.x`` branch holds unchanged files synchronized from the upstream
  CPython repository. The synchronization is currently done by manually copying
  the required files and stating from which CPython changeset they come from.

* the ``master`` branch holds a version of the ``3.x`` code with some tweaks
  that make it independent from libraries and constructions unavailable on 2.x.
  Code on this branch still *must* work on the corresponding Python 3.x but
  will also work on Python 2.6 and 2.7 (including PyPy).  You can check this
  running the supplied unit tests with ``tox``.

The process works like this:

1. In the ``3.x`` branch, run ``pip-run -- sync-upstream.py``, which
   downloads the latest stable release of Python and copies the relevant
   files from there into their new locations here and then commits those
   changes with a nice reference to the relevant upstream commit hash.

2. I check for new names in ``__all__`` and update imports in
   ``configparser.py`` accordingly. I run the tests on Python 3. Commit.

3. I merge the new commit to ``master``. I run ``tox``. Commit.

4. If there are necessary changes, I do them now (on ``master``). Note that
   the changes should be written in the syntax subset supported by Python
   2.6.

5. I run ``tox``. If it works, I update the docs and release the new version.
   Otherwise, I go back to point 3. I might use ``pasteurize`` to suggest me
   required changes but usually I do them manually to keep resulting code in
   a nicer form.

Footnotes
---------

.. [1] To somewhat ease migration, passing bytestrings is still supported but
       they are converted to Unicode for internal storage anyway. This means
       that for the vast majority of strings used in configuration files, it
       won't matter if you pass them as bytestrings or Unicode. However, if you
       pass a bytestring that cannot be converted to Unicode using the naive
       ASCII codec, a ``UnicodeDecodeError`` will be raised. This is purposeful
       and helps you manage proper encoding for all content you store in
       memory, read from various sources and write back.

.. [2] Life gets much easier when you understand that you basically manage
       **text** in your application.  You don't care about bytes but about
       letters.  In that regard the concept of content encoding is meaningless.
       The only time when you deal with raw bytes is when you write the data to
       a file.  Then you have to specify how your text should be encoded.  On
       the other end, to get meaningful text from a file, the application
       reading it has to know which encoding was used during its creation.  But
       once the bytes are read and properly decoded, all you have is text.  This
       is especially powerful when you start interacting with multiple data
       sources.  Even if each of them uses a different encoding, inside your
       application data is held in abstract text form.  You can program your
       business logic without worrying about which data came from which source.
       You can freely exchange the data you store between sources.  Only
       reading/writing files requires encoding your text to bytes.


+++++++++++++
Fault handler
+++++++++++++

.. image:: https://img.shields.io/pypi/v/faulthandler.svg
   :alt: Latest release on the Python Cheeseshop (PyPI)
   :target: https://pypi.python.org/pypi/faulthandler

.. image:: https://travis-ci.org/vstinner/faulthandler.svg?branch=master
   :alt: Build status of faulthandler on Travis CI
   :target: https://travis-ci.org/vstinner/faulthandler

.. image:: http://unmaintained.tech/badge.svg
   :target: http://unmaintained.tech/
   :alt: No Maintenance Intended

Fault handler for SIGSEGV, SIGFPE, SIGABRT, SIGBUS and SIGILL signals: display
the Python traceback and restore the previous handler. Allocate an alternate
stack for this handler, if sigaltstack() is available, to be able to allocate
memory on the stack, even on stack overflow (not available on Windows).

Import the module and call faulthandler.enable() to enable the fault handler.

Alternatively you can set the PYTHONFAULTHANDLER environment variable to a
non-empty value.

The fault handler is called on catastrophic cases and so it can only use
signal-safe functions (eg. it doesn't allocate memory on the heap). That's why
the traceback is limited: it only supports ASCII encoding (use the
backslashreplace error handler for non-ASCII characters) and limits each string
to 100 characters, doesn't print the source code in the traceback (only the
filename, the function name and the line number), is limited to 100 frames and
100 threads.

By default, the Python traceback is written to the standard error stream. Start
your graphical applications in a terminal and run your server in foreground to
see the traceback, or pass a file to faulthandler.enable().

faulthandler is implemented in C using signal handlers to be able to dump a
traceback on a crash or when Python is blocked (eg. deadlock).

This module is the backport for CPython 2.7. faulthandler is part of CPython
standard library since CPython 3.3: `faulthandler
<http://docs.python.org/dev/library/faulthandler.html>`_. For PyPy,
faulthandler is builtin since PyPy 5.5: use ``pypy -X faulthandler``.

Website:
https://faulthandler.readthedocs.io/

faulthandler 3.2 is the last version released by Victor Stinner. I maintained
it for 10 years in my free time for the great pleasure of Python 2 users, but
Python 2 is no longer supported upstream since 2020-01-01. Each faulthandler
release requires me to start my Windows VM, install Python 2.7 in 32-bit and
64-bit, install an old C compiler just for Python 2.7, and type manually some
commands to upload Windows binaries. Moreover, I have to fix some issues on
Travis CI and many small boring tasks. The maintenance is far from being free.
In 10 years, I got zero "thank you" (and 0€), only bug reports :-)


scandir, a better directory iterator and faster os.walk()
=========================================================

.. image:: https://img.shields.io/pypi/v/scandir.svg
   :target: https://pypi.python.org/pypi/scandir
   :alt: scandir on PyPI (Python Package Index)

.. image:: https://travis-ci.org/benhoyt/scandir.svg?branch=master
   :target: https://travis-ci.org/benhoyt/scandir
   :alt: Travis CI tests (Linux)

.. image:: https://ci.appveyor.com/api/projects/status/github/benhoyt/scandir?branch=master&svg=true
   :target: https://ci.appveyor.com/project/benhoyt/scandir
   :alt: Appveyor tests (Windows)

``scandir()`` is a directory iteration function like ``os.listdir()``,
except that instead of returning a list of bare filenames, it yields
``DirEntry`` objects that include file type and stat information along
with the name. Using ``scandir()`` increases the speed of ``os.walk()``
by 2-20 times (depending on the platform and file system) by avoiding
unnecessary calls to ``os.stat()`` in most cases.

Now included in a Python near you!
----------------------------------

``scandir`` has been included in the Python 3.5 standard library as
``os.scandir()``, and the related performance improvements to
``os.walk()`` have also been included. So if you're lucky enough to be
using Python 3.5 (release date September 13, 2015) you get the benefit
immediately, otherwise just
`download this module from PyPI <https://pypi.python.org/pypi/scandir>`_,
install it with ``pip install scandir``, and then do something like
this in your code:

.. code-block:: python

    # Use the built-in version of scandir/walk if possible, otherwise
    # use the scandir module version
    try:
        from os import scandir, walk
    except ImportError:
        from scandir import scandir, walk

`PEP 471 <https://www.python.org/dev/peps/pep-0471/>`_, which is the
PEP that proposes including ``scandir`` in the Python standard library,
was `accepted <https://mail.python.org/pipermail/python-dev/2014-July/135561.html>`_
in July 2014 by Victor Stinner, the BDFL-delegate for the PEP.

This ``scandir`` module is intended to work on Python 2.7+ and Python
3.4+ (and it has been tested on those versions).

Background
----------

Python's built-in ``os.walk()`` is significantly slower than it needs to be,
because -- in addition to calling ``listdir()`` on each directory -- it calls
``stat()`` on each file to determine whether the filename is a directory or not.
But both ``FindFirstFile`` / ``FindNextFile`` on Windows and ``readdir`` on Linux/OS
X already tell you whether the files returned are directories or not, so
no further ``stat`` system calls are needed. In short, you can reduce the number
of system calls from about 2N to N, where N is the total number of files and
directories in the tree.

In practice, removing all those extra system calls makes ``os.walk()`` about
**7-50 times as fast on Windows, and about 3-10 times as fast on Linux and Mac OS
X.** So we're not talking about micro-optimizations. See more benchmarks
in the "Benchmarks" section below.

Somewhat relatedly, many people have also asked for a version of
``os.listdir()`` that yields filenames as it iterates instead of returning them
as one big list. This improves memory efficiency for iterating very large
directories.

So as well as a faster ``walk()``, scandir adds a new ``scandir()`` function.
They're pretty easy to use, but see "The API" below for the full docs.

Benchmarks
----------

Below are results showing how many times as fast ``scandir.walk()`` is than
``os.walk()`` on various systems, found by running ``benchmark.py`` with no
arguments:

====================   ==============   =============
System version         Python version   Times as fast
====================   ==============   =============
Windows 7 64-bit       2.7.7 64-bit     10.4
Windows 7 64-bit SSD   2.7.7 64-bit     10.3
Windows 7 64-bit NFS   2.7.6 64-bit     36.8
Windows 7 64-bit SSD   3.4.1 64-bit     9.9
Windows 7 64-bit SSD   3.5.0 64-bit     9.5
Ubuntu 14.04 64-bit    2.7.6 64-bit     5.8
Mac OS X 10.9.3        2.7.5 64-bit     3.8
====================   ==============   =============

All of the above tests were done using the fast C version of scandir
(source code in ``_scandir.c``).

Note that the gains are less than the above on smaller directories and greater
on larger directories. This is why ``benchmark.py`` creates a test directory
tree with a standardized size.

The API
-------

walk()
~~~~~~

The API for ``scandir.walk()`` is exactly the same as ``os.walk()``, so just
`read the Python docs <https://docs.python.org/3.5/library/os.html#os.walk>`_.

scandir()
~~~~~~~~~

The full docs for ``scandir()`` and the ``DirEntry`` objects it yields are
available in the `Python documentation here <https://docs.python.org/3.5/library/os.html#os.scandir>`_.
But below is a brief summary as well.

    scandir(path='.') -> iterator of DirEntry objects for given path

Like ``listdir``, ``scandir`` calls the operating system's directory
iteration system calls to get the names of the files in the given
``path``, but it's different from ``listdir`` in two ways:

* Instead of returning bare filename strings, it returns lightweight
  ``DirEntry`` objects that hold the filename string and provide
  simple methods that allow access to the additional data the
  operating system may have returned.

* It returns a generator instead of a list, so that ``scandir`` acts
  as a true iterator instead of returning the full list immediately.

``scandir()`` yields a ``DirEntry`` object for each file and
sub-directory in ``path``. Just like ``listdir``, the ``'.'``
and ``'..'`` pseudo-directories are skipped, and the entries are
yielded in system-dependent order. Each ``DirEntry`` object has the
following attributes and methods:

* ``name``: the entry's filename, relative to the scandir ``path``
  argument (corresponds to the return values of ``os.listdir``)

* ``path``: the entry's full path name (not necessarily an absolute
  path) -- the equivalent of ``os.path.join(scandir_path, entry.name)``

* ``is_dir(*, follow_symlinks=True)``: similar to
  ``pathlib.Path.is_dir()``, but the return value is cached on the
  ``DirEntry`` object; doesn't require a system call in most cases;
  don't follow symbolic links if ``follow_symlinks`` is False

* ``is_file(*, follow_symlinks=True)``: similar to
  ``pathlib.Path.is_file()``, but the return value is cached on the
  ``DirEntry`` object; doesn't require a system call in most cases;
  don't follow symbolic links if ``follow_symlinks`` is False

* ``is_symlink()``: similar to ``pathlib.Path.is_symlink()``, but the
  return value is cached on the ``DirEntry`` object; doesn't require a
  system call in most cases

* ``stat(*, follow_symlinks=True)``: like ``os.stat()``, but the
  return value is cached on the ``DirEntry`` object; does not require a
  system call on Windows (except for symlinks); don't follow symbolic links
  (like ``os.lstat()``) if ``follow_symlinks`` is False

* ``inode()``: return the inode number of the entry; the return value
  is cached on the ``DirEntry`` object

Here's a very simple example of ``scandir()`` showing use of the
``DirEntry.name`` attribute and the ``DirEntry.is_dir()`` method:

.. code-block:: python

    def subdirs(path):
        """Yield directory names not starting with '.' under given path."""
        for entry in os.scandir(path):
            if not entry.name.startswith('.') and entry.is_dir():
                yield entry.name

This ``subdirs()`` function will be significantly faster with scandir
than ``os.listdir()`` and ``os.path.isdir()`` on both Windows and POSIX
systems, especially on medium-sized or large directories.

Further reading
---------------

* `The Python docs for scandir <https://docs.python.org/3.5/library/os.html#os.scandir>`_
* `PEP 471 <https://www.python.org/dev/peps/pep-0471/>`_, the
  (now-accepted) Python Enhancement Proposal that proposed adding
  ``scandir`` to the standard library -- a lot of details here,
  including rejected ideas and previous discussion

Flames, comments, bug reports
-----------------------------

Please send flames, comments, and questions about scandir to Ben Hoyt:

http://benhoyt.com/

File bug reports for the version in the Python 3.5 standard library
`here <https://docs.python.org/3.5/bugs.html>`_, or file bug reports
or feature requests for this module at the GitHub project page:

https://github.com/benhoyt/scandir


subprocess32
------------
[![PyPI version](https://badge.fury.io/py/subprocess32.svg)](https://badge.fury.io/py/subprocess32)
[![POSIX Build Status](https://travis-ci.org/google/python-subprocess32.svg?branch=master)](https://travis-ci.org/google/python-subprocess32)
[![Windows Build Status](https://ci.appveyor.com/api/projects/status/53apbb2jk1uslj0m?svg=true
)](https://ci.appveyor.com/project/gpshead/python-subprocess32)

This is a backport of the Python 3 subprocess module for use on Python 2.
This code has not been tested on Windows or other non-POSIX platforms.

subprocess32 includes many important reliability bug fixes relevant on
POSIX platforms.  The most important of which is a C extension module
used internally to handle the code path between fork() and exec().
This module is reliable when an application is using threads.

Refer to the
[Python 3.5 subprocess documentation](https://docs.python.org/3.5/library/subprocess.html)
for usage information.

* Timeout support backported from Python 3.3 is included.
* The run() API from Python 3.5 was backported in subprocess32 3.5.0.
* Otherwise features are frozen at the 3.2 level.

Usage
-----

The recommend pattern for cross platform code is to use the following:

```python
if os.name == 'posix' and sys.version_info[0] < 3:
    import subprocess32 as subprocess
else:
    import subprocess
```

Or if you fully control your POSIX Python 2.7 installation, this can serve
as a replacement for its subprocess module.  Users will thank you by not
filing concurrency bugs.

Got Bugs?
---------

Try to reproduce them on the latest Python 3.x itself and file bug
reports on [bugs.python.org](https://bugs.python.org/).
Add gregory.p.smith to the Nosy list.

If you have reason to believe the issue is specifically with this backport
and not a problem in Python 3 itself, use the github issue tracker.

-- Gregory P. Smith  _greg@krypto.org_


========================================
How to Build the LLVM* OpenMP* Libraries
========================================
This repository requires `CMake <http://www.cmake.org/>`_ v2.8.0 or later.  LLVM
and Clang need a more recent version which also applies for in-tree builds.  For
more information than available in this document please see
`LLVM's CMake documentation <https://llvm.org/docs/CMake.html>`_ and the
`official documentation <https://cmake.org/cmake/help/v2.8.0/cmake.html>`_.

.. contents::
   :local:

How to Call CMake Initially, then Repeatedly
============================================
- When calling CMake for the first time, all needed compiler options must be
  specified on the command line.  After this initial call to CMake, the compiler
  definitions must not be included for further calls to CMake.  Other options
  can be specified on the command line multiple times including all definitions
  in the build options section below.
- Example of configuring, building, reconfiguring, rebuilding:

  .. code-block:: console

    $ mkdir build
    $ cd build
    $ cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ..  # Initial configuration
    $ make
    ...
    $ make clean
    $ cmake -DCMAKE_BUILD_TYPE=Debug ..                               # Second configuration
    $ make
    ...
    $ rm -rf *
    $ cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ ..        # Third configuration
    $ make

- Notice in the example how the compiler definitions are only specified for an
  empty build directory, but other build options are used at any time.
- The file ``CMakeCache.txt`` which is created after the first call to CMake is
  a configuration file which holds all values for the build options.  These
  values can be changed using a text editor to modify ``CMakeCache.txt`` as
  opposed to using definitions on the command line.
- To have CMake create a particular type of build generator file simply include
  the ``-G <Generator name>`` option:

  .. code-block:: console

    $ cmake -G "Unix Makefiles" ...

  You can see a list of generators CMake supports by executing the cmake command
  with no arguments.

Instructions to Build
=====================
.. code-block:: console

 $ cd openmp_top_level/ [ this directory with libomptarget/, runtime/, etc. ]
 $ mkdir build
 $ cd build

 [ Unix* Libraries ]
 $ cmake -DCMAKE_C_COMPILER=<C Compiler> -DCMAKE_CXX_COMPILER=<C++ Compiler> ..

 [ Windows* Libraries ]
 $ cmake -G <Generator Type> -DCMAKE_C_COMPILER=<C Compiler> -DCMAKE_CXX_COMPILER=<C++ Compiler> -DCMAKE_ASM_MASM_COMPILER=[ml | ml64] -DCMAKE_BUILD_TYPE=Release ..

 $ make
 $ make install

CMake Options
=============
Builds with CMake can be customized by means of options as already seen above.
One possibility is to pass them via the command line:

.. code-block:: console

  $ cmake -DOPTION=<value> path/to/source

.. note:: The first value listed is the respective default for that option.

Generic Options
---------------
For full documentation consult the CMake manual or execute
``cmake --help-variable VARIABLE_NAME`` to get information about a specific
variable.

**CMAKE_BUILD_TYPE** = ``Release|Debug|RelWithDebInfo``
  Build type can be ``Release``, ``Debug``, or ``RelWithDebInfo`` which chooses
  the optimization level and presence of debugging symbols.

**CMAKE_C_COMPILER** = <C compiler name>
  Specify the C compiler.

**CMAKE_CXX_COMPILER** = <C++ compiler name>
  Specify the C++ compiler.

**CMAKE_Fortran_COMPILER** = <Fortran compiler name>
  Specify the Fortran compiler. This option is only needed when
  **LIBOMP_FORTRAN_MODULES** is ``ON`` (see below).  So typically, a Fortran
  compiler is not needed during the build.

**CMAKE_ASM_MASM_COMPILER** = ``ml|ml64``
  This option is only relevant for Windows*.

Options for all Libraries
-------------------------

**OPENMP_ENABLE_WERROR** = ``OFF|ON``
  Treat warnings as errors and fail, if a compiler warning is triggered.

**OPENMP_LIBDIR_SUFFIX** = ``""``
  Extra suffix to append to the directory where libraries are to be installed.

**OPENMP_TEST_C_COMPILER** = ``${CMAKE_C_COMPILER}``
  Compiler to use for testing. Defaults to the compiler that was also used for
  building.

**OPENMP_TEST_CXX_COMPILER** = ``${CMAKE_CXX_COMPILER}``
  Compiler to use for testing. Defaults to the compiler that was also used for
  building.

**OPENMP_LLVM_TOOLS_DIR** = ``/path/to/built/llvm/tools``
  Additional path to search for LLVM tools needed by tests.

**OPENMP_LLVM_LIT_EXECUTABLE** = ``/path/to/llvm-lit``
  Specify full path to ``llvm-lit`` executable for running tests.  The default
  is to search the ``PATH`` and the directory in **OPENMP_LLVM_TOOLS_DIR**.

**OPENMP_FILECHECK_EXECUTABLE** = ``/path/to/FileCheck``
  Specify full path to ``FileCheck`` executable for running tests.  The default
  is to search the ``PATH`` and the directory in **OPENMP_LLVM_TOOLS_DIR**.

**OPENMP_NOT_EXECUTABLE** = ``/path/to/not``
  Specify full path to ``not`` executable for running tests.  The default
  is to search the ``PATH`` and the directory in **OPENMP_LLVM_TOOLS_DIR**.

Options for ``libomp``
----------------------

**LIBOMP_ARCH** = ``aarch64|arm|i386|mic|mips|mips64|ppc64|ppc64le|x86_64|riscv64``
  The default value for this option is chosen based on probing the compiler for
  architecture macros (e.g., is ``__x86_64__`` predefined by compiler?).

**LIBOMP_MIC_ARCH** = ``knc|knf``
  Intel(R) Many Integrated Core Architecture (Intel(R) MIC Architecture) to
  build for.  This value is ignored if **LIBOMP_ARCH** does not equal ``mic``.

**LIBOMP_LIB_TYPE** = ``normal|profile|stubs``
  Library type can be ``normal``, ``profile``, or ``stubs``.

**LIBOMP_USE_VERSION_SYMBOLS** = ``ON|OFF``
  Use versioned symbols for building the library.  This option only makes sense
  for ELF based libraries where version symbols are supported (Linux*, some BSD*
  variants).  It is ``OFF`` by default for Windows* and macOS*, but ``ON`` for
  other Unix based operating systems.

**LIBOMP_ENABLE_SHARED** = ``ON|OFF``
  Build a shared library.  If this option is ``OFF``, static OpenMP libraries
  will be built instead of dynamic ones.

  .. note::

    Static libraries are not supported on Windows*.

**LIBOMP_FORTRAN_MODULES** = ``OFF|ON``
  Create the Fortran modules (requires Fortran compiler).

macOS* Fat Libraries
""""""""""""""""""""
On macOS* machines, it is possible to build universal (or fat) libraries which
include both i386 and x86_64 architecture objects in a single archive.

.. code-block:: console

  $ cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_OSX_ARCHITECTURES='i386;x86_64' ..
  $ make

There is also an option **LIBOMP_OSX_ARCHITECTURES** which can be set in case
this is an LLVM source tree build. It will only apply for the ``libomp`` library
avoids having the entire LLVM/Clang build produce universal binaries.

Optional Features
"""""""""""""""""

**LIBOMP_USE_ADAPTIVE_LOCKS** = ``ON|OFF``
  Include adaptive locks, based on Intel(R) Transactional Synchronization
  Extensions (Intel(R) TSX).  This feature is x86 specific and turned ``ON``
  by default for IA-32 architecture and Intel(R) 64 architecture.

**LIBOMP_USE_INTERNODE_ALIGNMENT** = ``OFF|ON``
  Align certain data structures on 4096-byte.  This option is useful on
  multi-node systems where a small ``CACHE_LINE`` setting leads to false sharing.

**LIBOMP_OMPT_SUPPORT** = ``ON|OFF``
  Include support for the OpenMP Tools Interface (OMPT).
  This option is supported and ``ON`` by default for x86, x86_64, AArch64,
  PPC64 and RISCV64 on Linux* and macOS*.
  This option is ``OFF`` if this feature is not supported for the platform.

**LIBOMP_OMPT_OPTIONAL** = ``ON|OFF``
  Include support for optional OMPT functionality.  This option is ignored if
  **LIBOMP_OMPT_SUPPORT** is ``OFF``.

**LIBOMP_STATS** = ``OFF|ON``
  Include stats-gathering code.

**LIBOMP_USE_DEBUGGER** = ``OFF|ON``
  Include the friendly debugger interface.

**LIBOMP_USE_HWLOC** = ``OFF|ON``
  Use `OpenMPI's hwloc library <https://www.open-mpi.org/projects/hwloc/>`_ for
  topology detection and affinity.

**LIBOMP_HWLOC_INSTALL_DIR** = ``/path/to/hwloc/install/dir``
  Specify install location of hwloc.  The configuration system will look for
  ``hwloc.h`` in ``${LIBOMP_HWLOC_INSTALL_DIR}/include`` and the library in
  ``${LIBOMP_HWLOC_INSTALL_DIR}/lib``.  The default is ``/usr/local``.
  This option is only used if **LIBOMP_USE_HWLOC** is ``ON``.

Additional Compiler Flags
"""""""""""""""""""""""""

These flags are **appended**, they do not overwrite any of the preset flags.

**LIBOMP_CPPFLAGS** = <space-separated flags>
  Additional C preprocessor flags.

**LIBOMP_CXXFLAGS** = <space-separated flags>
  Additional C++ compiler flags.

**LIBOMP_ASMFLAGS** = <space-separated flags>
  Additional assembler flags.

**LIBOMP_LDFLAGS** = <space-separated flags>
  Additional linker flags.

**LIBOMP_LIBFLAGS** = <space-separated flags>
  Additional libraries to link.

**LIBOMP_FFLAGS** = <space-separated flags>
  Additional Fortran compiler flags.

Options for ``libomptarget``
----------------------------

An installed LLVM package is a prerequisite for building ``libomptarget``
library. So ``libomptarget`` may only be built in two cases:

- As a project of a regular LLVM build via **LLVM_ENABLE_PROJECTS**,
  **LLVM_EXTERNAL_PROJECTS**, or **LLVM_ENABLE_RUNTIMES** or
- as a standalone project build that uses a pre-installed LLVM package.
  In this mode one has to make sure that the default CMake
  ``find_package(LLVM)`` call `succeeds <https://cmake.org/cmake/help/latest/command/find_package.html#search-procedure>`_.

**LIBOMPTARGET_OPENMP_HEADER_FOLDER** = ``""``
  Path of the folder that contains ``omp.h``.  This is required for testing
  out-of-tree builds.

**LIBOMPTARGET_OPENMP_HOST_RTL_FOLDER** = ``""``
  Path of the folder that contains ``libomp.so``, and ``libLLVMSupport.so``
  when profiling is enabled.  This is required for testing.

Options for ``NVPTX device RTL``
--------------------------------

**LIBOMPTARGET_NVPTX_ENABLE_BCLIB** = ``ON|OFF``
  Enable CUDA LLVM bitcode offloading device RTL. This is used for link time
  optimization of the OMP runtime and application code. This option is enabled
  by default if the build system determines that `CMAKE_C_COMPILER` is able to
  compile and link the library.

**LIBOMPTARGET_NVPTX_CUDA_COMPILER** = ``""``
  Location of a CUDA compiler capable of emitting LLVM bitcode. Currently only
  the Clang compiler is supported. This is only used when building the CUDA LLVM
  bitcode offloading device RTL. If unspecified, either the Clang from the build
  itself is used (i.e. an in-tree build with LLVM_ENABLE_PROJECTS including
  clang), or the Clang compiler that the build uses as C compiler
  (CMAKE_C_COMPILER; only if it is Clang). The latter is common for a
  stage2-build or when using -DLLVM_ENABLE_RUNTIMES=openmp.

**LIBOMPTARGET_NVPTX_BC_LINKER** = ``""``
  Location of a linker capable of linking LLVM bitcode objects. This is only
  used when building the CUDA LLVM bitcode offloading device RTL. If
  unspecified, either the llvm-link in that same directory as
  LIBOMPTARGET_NVPTX_CUDA_COMPILER is used, or the llvm-link from the
  same build (available in an in-tree build).

**LIBOMPTARGET_NVPTX_ALTERNATE_HOST_COMPILER** = ``""``
  Host compiler to use with NVCC. This compiler is not going to be used to
  produce any binary. Instead, this is used to overcome the input compiler
  checks done by NVCC. E.g. if using a default host compiler that is not
  compatible with NVCC, this option can be use to pass to NVCC a valid compiler
  to avoid the error.

 **LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES** = ``35``
  List of CUDA compute capabilities that should be supported by the NVPTX
  device RTL. E.g. for compute capabilities 6.0 and 7.0, the option "60;70"
  should be used. Compute capability 3.5 is the minimum required.

 **LIBOMPTARGET_NVPTX_DEBUG** = ``OFF|ON``
  Enable printing of debug messages from the NVPTX device RTL.

**LIBOMPTARGET_LIT_ARGS** = ``""``
  Arguments given to lit. ``make check-libomptarget`` and
  ``make check-libomptarget-*`` are affected. For example, use
  ``LIBOMPTARGET_LIT_ARGS="-j4"`` to force ``lit`` to start only four parallel
  jobs instead of by default the number of threads in the system.

Example Usages of CMake
=======================

Typical Invocations
-------------------

.. code-block:: console

  $ cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ..
  $ cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ ..
  $ cmake -DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc ..

Advanced Builds with Various Options
------------------------------------

- Build the i386 Linux* library using GCC*

  .. code-block:: console

    $ cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DLIBOMP_ARCH=i386 ..

- Build the x86_64 debug Mac library using Clang*

  .. code-block:: console

    $ cmake -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLIBOMP_ARCH=x86_64 -DCMAKE_BUILD_TYPE=Debug ..

- Build the library (architecture determined by probing compiler) using the
  Intel(R) C Compiler and the Intel(R) C++ Compiler.  Also, create Fortran
  modules with the Intel(R) Fortran Compiler.

  .. code-block:: console

    $ cmake -DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc -DCMAKE_Fortran_COMPILER=ifort -DLIBOMP_FORTRAN_MODULES=on ..

- Have CMake find the C/C++ compiler and specify additional flags for the
  preprocessor and C++ compiler.

  .. code-blocks:: console

    $ cmake -DLIBOMP_CPPFLAGS='-DNEW_FEATURE=1 -DOLD_FEATURE=0' -DLIBOMP_CXXFLAGS='--one-specific-flag --two-specific-flag' ..

- Build the stubs library

  .. code-blocks:: console

    $ cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DLIBOMP_LIB_TYPE=stubs ..

**Footnotes**

.. [*] Other names and brands may be claimed as the property of others.


# Abseil in gRPC

This document explains how to use Abseil throughout gRPC. Note that this isn't
supposed to explain general usage of Abseil.

## The version of Abseil

gRPC intends to use the LTS versions of Abseil only because it simplifies
dependency management. Abseil is being distributed via package distribution
systems such as vcpkg and cocoapods. If gRPC depends on the certain version
that aren't registered, gRPC in that system cannot get the right version of
Abseil when being built, resulting in a build failure.
Therefore, gRPC will use the LTS version only, preferably the latest one.

## Libraries that are not ready to use

Most of Abseil libraries are okay to use but there are some exceptions
because they're not going well yet on some of our test machinaries or
platforms it supports. The following is a list of targets that are NOT
ready to use.

- `y_absl/synchronization:*`: Blocked by b/186685878.
- `y_absl/random`: [WIP](https://github.com/grpc/grpc/pull/23346).

## Implemetation only

You can use Abseil in gRPC Core and gRPC C++. But you cannot use it in
the public interface of gRPC C++ because i) it doesn't gurantee no breaking
API changes like gRPC C++ does and ii) it may make users change their build
system to address Abseil.


# Third-party libraries

gRPC depends on several third-party libraries, their source code is available
(usually as a git submodule) in this directory.

## Guidelines on updating submodules

- IMPORTANT: whenever possible, try to only update to a stable release of a library (= not to master / random commit). Depending on unreleased revisions
  makes gRPC installation harder for users, as it forces them to always build the dependency from source and prevents them from using more
  convenient installation channels (linux packages, package managers etc.)

- bazel BUILD uses a different dependency model - whenever updating a submodule, also update the revision in `grpc_deps.bzl` so that bazel and
  non-bazel builds stay in sync (this is actually enforced by a sanity check in some cases)

## Considerations when adding a new third-party dependency

- gRPC C++ needs to stay buildable/installable even if the submodules are not present (e.g. the tar.gz archive with gRPC doesn't contain the submodules),
  assuming that the dependencies are already installed. This is a requirement for being able to provide a reasonable install process (e.g. using cmake)
  and to support package managers for gRPC C++.

- Adding a new dependency is a lot of work (both for us and for the users).
  We currently support multiple build systems (BAZEL, cmake, make, ...) so adding a new dependency usually requires updates in multiple build systems
  (often not trivial). The installation process also needs to continue to work (we do have distrib tests to test many of the possible installation scenarios,
  but they are not perfect). Adding a new dependency also usually affects the installation instructions that need to be updated.
  Also keep in mind that adding a new dependency can be quite disruptive
  for the users and community - it means that all users will need to update their projects accordingly (for C++ projects often non-trivial) and
  the community-provided C++ packages (e.g. vcpkg) will need to be updated as well.

## Checklist for adding a new third-party dependency

**READ THIS BEFORE YOU ADD A NEW DEPENDENCY**

- [ ] Make sure you understand the hidden costs of adding a dependency (see section above) and that you understand the     complexities of updating the build files. Maintenance of the build files isn't for free, so expect to be involved in maintenance tasks, cleanup and support (e.g resolving user bugs) of the build files in the future.

- [ ] Once your change is ready, start an [adhoc run of artifact - packages - distribtests flow](https://fusion.corp.google.com/projectanalysis/summary/KOKORO/prod%3Agrpc%2Fcore%2Fexperimental%2Fgrpc_build_artifacts_multiplatform) and make sure everything passes (for technical reasons, not all the distribtests can run on each PR automatically).

- [ ] Check the impact of the new dependency on the size of our distribution packages (compare BEFORE and AFTER) and post the comparison on your PR (it should not be approved without checking the impact sizes of packages first). The package sizes AFTER can be obtained from the adhoc package build from bullet point above.

## Instructions for updating dependencies

Usually the process is

1. update the submodule to selected commit (see guidance above)
2. update the dependency in `grpc_deps.bzl` to the same commit
3. update `tools/run_tests/sanity/check_submodules.sh` to make the sanity test pass
4. (when needed) run `tools/buildgen/generate_projects.sh` to regenerate the generated files
5. populate the bazel download mirror by running `bazel/update_mirror.sh`

Updating some dependencies requires extra care.

### Updating third_party/abseil-cpp

- Two additional steps should be done before running `generate_projects.sh` above.
  - Running `src/abseil-cpp/preprocessed_builds.yaml.gen.py`.
  - Updating `abseil_version =` scripts in `templates/gRPC-C++.podspec.template` and
    `templates/gRPC-Core.podspec.template`.
- You can see an example of previous [upgrade](https://github.com/grpc/grpc/pull/24270).

### Updating third_party/boringssl-with-bazel

NOTE: updating the boringssl dependency is now part of the internal grpc release tooling (see [go/grpc-release](http://go/grpc-release)).
Prefer using the release tooling when possible. The instructions below are provided as a reference and aren't guaranteed to be up-to-date.

- Update the `third_party/boringssl-with-bazel` submodule to the latest [`master-with-bazel`](https://github.com/google/boringssl/tree/master-with-bazel) branch
```
git submodule update --init      # just to start in a clean state
cd third_party/boringssl-with-bazel
git fetch origin   # fetch what's new in the boringssl repository
git checkout origin/master-with-bazel  # checkout the current state of master-with-bazel branch in the boringssl repo
# Note the latest commit SHA on master-with-bazel branch
cd ../..   # go back to grpc repo root
git status   #  will show that there are new commits in third_party/boringssl-with-bazel
git add  third_party/boringssl-with-bazel     # we actually want to update the changes to the submodule
git commit -m "update submodule boringssl-with-bazel with origin/master-with-bazel"   # commit
```

- Update boringssl dependency in `bazel/grpc_deps.bzl` to the same commit SHA as master-with-bazel branch
    - Update `http_archive(name = "boringssl",` section by updating the sha in `strip_prefix` and `urls` fields.
    - Also, set `sha256` field to "" as the existing value is not valid. This will be added later once we know what that value is.

- Update `tools/run_tests/sanity/check_submodules.sh` with the same commit

- Commit these changes `git commit -m "update boringssl dependency to master-with-bazel commit SHA"`

- Run `tools/buildgen/generate_projects.sh` to regenerate the generated files
    - Because `sha256` in `bazel/grpc_deps.bzl` was left empty, you will get a DEBUG msg like this one:
      ```
      Rule 'boringssl' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "SHA value"
      ```
    - Commit the regenrated files `git commit -m "regenerate files"`
    - Update `bazel/grpc_deps.bzl` with the SHA value shown in the above debug msg. Commit again `git commit -m "Updated sha256"`

- Run `tools/distrib/generate_boringssl_prefix_header.sh`
    - Commit again `git commit -m "generate boringssl prefix headers"`

- Increment the boringssl podspec version number in
  `templates/src/objective-c/BoringSSL-GRPC.podspec.template` and `templates/gRPC-Core.podspec.template`.
  [example](https://github.com/grpc/grpc/pull/21527/commits/9d4411842f02f167209887f1f3d2b9ab5d14931a)
    - Commit again `git commit -m "Increment podspec version"`

- Run `tools/buildgen/generate_projects.sh` (yes, again)
    - Commit again `git commit -m "Second regeneration"`

- Create a PR with all the above commits.

- Run `bazel/update_mirror.sh` to update GCS mirror.

### Updating third_party/protobuf

Updating the protobuf dependency is now part of the internal release process (see [go/grpc-release](http://go/grpc-release)).

### Updating third_party/envoy-api

Apart from the above steps, please perform the following two steps to generate the Python `xds-protos` package:

1. Bump the version in the `tools/distrib/python/xds_protos/setup.py`;
2. Run `tools/distrib/python/xds_protos/build_validate_upload.sh` to upload the built wheel.

### Updating third_party/upb

Since upb is vendored in the gRPC repo, you cannot use submodule to update it. Please follow the steps below.

1. Update third_party/upb directory by running
   - `export GRPC_ROOT=~/git/grpc`
   - `wget https://github.com/protocolbuffers/upb/archive/refs/heads/main.zip`
   - `rm -rf $GRPC_ROOT/third_party/upb`
   - `unzip main.zip -d $GRPC_ROOT/third_party`
   - `mv $GRPC_ROOT/third_party/upb-main $GRPC_ROOT/third_party/upb`
2. Update the dependency in `grpc_deps.bzl` to the same commit
3. Populate the bazel download mirror by running `bazel/update_mirror.sh`
4. Update `src/upb/gen_build_yaml.py` for newly added or removed upb files
   - Running `bazel query "deps(upb) union deps(json) union deps(textformat)"`
     under third_party/upb would give some idea on what needs to be included.
5. Run `tools/buildgen/generate_projects.sh` to regenerate the generated files
6. Run `tools/codegen/core/gen_upb_api.sh` to regenerate upb files.

### Updating third_party/xxhash

TODO(https://github.com/Cyan4973/xxHash/issues/548): revisit LICENSE
instructions if upstream splits library and CLI.

The upstream xxhash repository contains a bunch of files that we don't want, so
we employ a rather manual update flow to pick up just the bits we care about:

```
git remote add xxhash https://github.com/Cyan4973/xxHash.git
git fetch xxhash
git show xxhash/dev:xxhash.h > third_party/xxhash/xxhash.h
git show xxhash/dev:LICENSE | sed -nE '/^-+$/q;p' > third_party/xxhash/LICENSE
```


## Общие соображения

В библиотеку добавлены реализации символов, появившихся в `libc.so.6` Ubuntu
14.04 с момента Ubuntu 12.04.

Если какой-либо объектный файл ссылается на один из таких символов, то при
компоновке этот символ будет взят из нашей библиотеки. В противном случае,
компоновщик сослался бы на такой символ из `libc.so.6`, добавив к исполняемому
файлу зависимость от новой версии динамической библиотеки (`GLIBC_2.16`,
`GLIBC_2.17` или `GLIBC_2.18`). Такой исполняемый файл не может быть запущен на
Ubuntu 12.04, даже если ни один из новых символов не используется в runtime.

На Ubuntu 14.04 или более новых, в случае если процесс загружает `libc.so.6`, мы
будем в runtime иметь две реализации символов, нашу и libc. Но все добавленные
функции не имеют какого-либо состояния и никакими деталями внутренней реализации
не связаны с другими. Нет разницы, какую из реализаций использовать, и даже
попеременное использование различных реализаций в одном и том же контексте не
должно приводить к некорректной работе.

В какой-то момент этот слой совместимости будет отключен: https://st.yandex-team.ru/DEVTOOLS-7436

### Разделяемая реализация

Была идея оформить новые реализации так, чтобы в случае их наличия и в
загруженной libc динамический компоновщик выбирал для всех ссылок одну
реализацию.

По всей видимости, для этого требуется собирать исполняемый файл как PIE. Только
в этом случае для наших реализаций будут сгенерированы дополнительные PLT
прослойки и вызовы наших реализаций будут проходить через них.

Но в этом случае вообще все вызовы будут происходить таким образом и это
повлияет на производительность. Ухудшения производительности, наверное, можно
избежать, явно указав, какие символы в исполняемом файле должны быть публичными,
но сейчас нет способа сделать это в одном месте, учитывая, что в некоторых
случаях этот список должен дополняться.

## `getauxval` и `secure_getenv`

Функция `getauxval` требует загрузки и хранения «Auxiliary Vector» (см.
[здесь](https://refspecs.linuxfoundation.org/LSB_1.3.0/IA64/spec/auxiliaryvector.html)).

Эти данные доступны в момент запуска процесса. libc из новых Ubuntu сохраняет
эти данные и предоставляет реализацию `getauxval`. В этом случае наша реализация
перенаправляет вызовы `getauxval` в libc, получив при старте исполняемого файла
соответствующий указатель.

Если реализация libc недоступна (на старых Ubuntu или если libc не загружена),
то эти данные можно получить, прочитав `/proc/self/auxv`. Это также делается
один раз при старте.

В обоих случаях, статически инициализируется синглтон `NUbuntuCompat::TGlibc`,
который производит эти действия в конструкторе.

`secure_getenv` использует одно из значений `getauxval`, поэтому к нему всё это
также относится.

Каждый метод новой libc реализован в отдельном объектом файле. `TGlibc` также
находится в отдельном файле, и ссылки на неё стоят только в местах использования.
Если при компоновке не понадобились ни `getauxval`, ни `secure_getenv`, то
объектный файл с `TGlibc` тоже не будет выбран компоновщиком, и в этом случае
никакой лишней статической инициализации выполняться не будет.

## Патч libc.so

Чтобы иметь возможность использовать `getauxval` (точнее его реализацию
`__getauxval`) из новой libc, если таковая уже используется процессом,
библиотека совместимости объявляет этот символ у себя как внешний и слабый. При
загрузке динамический компоновщик устанавливает значение этого символа из libc
или `nullptr`, если его там нет. Наша реализация `getauxval` проверяет
доступность реализации libc, просто сравнивая указатель с `nullptr`.

В Аркадии также есть код, который подобным образом работает с символом
`__cxa_thread_atexit_impl`.

Однако, если компоновать такую программу с использованием новой libc, то к таким
символам и самой программе будет приписано требование соответствующей (новой)
версии libc. Чтобы этого не произошло, при сборке с этой библиотекой
совместимости используется патченный вариант `libc.so.6`, где у таких символов
удалена версия.

Также, файлы, проверяющие, что доступна реализация из libc, должны быть собраны
как PIC. В противном случае вместо значения, заполненного динамическим
компоновщиком, компилятор ещё на стадии компиляции использует `nullptr` и
проверка никогда не срабатывает.

## Упоминания

Идея о возможности добавить слой совместимости была взята из ClickHouse.
* [https://clickhouse.tech/](https://clickhouse.tech/)
* [https://wiki.yandex-team.ru/clickhouse/](https://wiki.yandex-team.ru/clickhouse/)


Protocol Buffers - Google's data interchange format
===================================================

Copyright 2008 Google Inc.

https://developers.google.com/protocol-buffers/

CMake Installation
-----------------------

To compile or install protobuf from source using CMake, see
[cmake/README.md](../cmake/README.md).

C++ Protobuf - Unix
-----------------------

To build protobuf from source, the following tools are needed:

  * bazel
  * git
  * g++

On Ubuntu/Debian, for example, you can install them with:

    sudo apt-get install g++ git bazel

On other platforms, please use the corresponding package managing tool to
install them before proceeding.  See https://bazel.build/install for further
instructions on installing Bazel, or to build from source using CMake, see
[cmake/README.md](../cmake/README.md).

To get the source, download the release .tar.gz or .zip package in the
release page:

    https://github.com/protocolbuffers/protobuf/releases/latest

For example: if you only need C++, download `protobuf-cpp-[VERSION].tar.gz`; if
you need C++ and Java, download `protobuf-java-[VERSION].tar.gz` (every package
contains C++ source already); if you need C++ and multiple other languages,
download `protobuf-all-[VERSION].tar.gz`.

You can also get the source by "git clone" our git repository. Make sure you
have also cloned the submodules and generated the configure script (skip this
if you are using a release .tar.gz or .zip package):

    git clone https://github.com/protocolbuffers/protobuf.git
    cd protobuf
    git submodule update --init --recursive

To build the C++ Protocol Buffer runtime and the Protocol Buffer compiler
(protoc) execute the following:

    bazel build :protoc :protobuf

The compiler can then be installed, for example on Linux:

    cp bazel-bin/protoc /usr/local/bin

For more usage information on Bazel, please refer to http://bazel.build.

**Compiling dependent packages**

To compile a package that uses Protocol Buffers, you need to setup a Bazel
WORKSPACE that's hooked up to the protobuf repository and loads its
dependencies.  For an example, see [WORKSPACE](../examples/WORKSPACE).

**Note for Mac users**

For a Mac system, Unix tools are not available by default. You will first need
to install Xcode from the Mac AppStore and then run the following command from
a terminal:

    sudo xcode-select --install

To install Unix tools, you can install "port" following the instructions at
https://www.macports.org . This will reside in /opt/local/bin/port for most
Mac installations.

    sudo /opt/local/bin/port install bazel

Alternative for Homebrew users:

    brew install bazel

Then follow the Unix instructions above.

C++ Protobuf - Windows
--------------------------

If you only need the protoc binary, you can download it from the release
page:

    https://github.com/protocolbuffers/protobuf/releases/latest

In the downloads section, download the zip file protoc-$VERSION-win32.zip.
It contains the protoc binary as well as public proto files of protobuf
library.

Protobuf and its dependencies can be installed directly by using `vcpkg`:

    >vcpkg install protobuf protobuf:x64-windows

If zlib support is desired, you'll also need to install the zlib feature:

    >vcpkg install protobuf[zlib] protobuf[zlib]:x64-windows

See https://github.com/Microsoft/vcpkg for more information.

To build from source using Microsoft Visual C++, see [cmake/README.md](../cmake/README.md).

To build from source using Cygwin or MinGW, follow the Unix installation
instructions, above.

Binary Compatibility Warning
----------------------------

Due to the nature of C++, it is unlikely that any two versions of the
Protocol Buffers C++ runtime libraries will have compatible ABIs.
That is, if you linked an executable against an older version of
libprotobuf, it is unlikely to work with a newer version without
re-compiling.  This problem, when it occurs, will normally be detected
immediately on startup of your app.  Still, you may want to consider
using static linkage.  You can configure this in your `cc_binary` Bazel rules
by specifying:

    linkstatic=True

Usage
-----

The complete documentation for Protocol Buffers is available via the
web at:

https://protobuf.dev/


Protocol Buffers - Google's data interchange format
===================================================

Copyright 2008 Google Inc.

https://developers.google.com/protocol-buffers/

C++ Installation - Unix
-----------------------

To build protobuf from source, the following tools are needed:

  * autoconf
  * automake
  * libtool
  * make
  * g++
  * unzip

On Ubuntu/Debian, you can install them with:

    sudo apt-get install autoconf automake libtool curl make g++ unzip

On other platforms, please use the corresponding package managing tool to
install them before proceeding.

To get the source, download one of the release .tar.gz or .zip packages in the
release page:

    https://github.com/protocolbuffers/protobuf/releases/latest

For example: if you only need C++, download `protobuf-cpp-[VERSION].tar.gz`; if
you need C++ and Java, download `protobuf-java-[VERSION].tar.gz` (every package
contains C++ source already); if you need C++ and multiple other languages,
download `protobuf-all-[VERSION].tar.gz`.

You can also get the source by "git clone" our git repository. Make sure you
have also cloned the submodules and generated the configure script (skip this
if you are using a release .tar.gz or .zip package):

    git clone https://github.com/protocolbuffers/protobuf.git
    cd protobuf
    git submodule update --init --recursive
    ./autogen.sh

To build and install the C++ Protocol Buffer runtime and the Protocol
Buffer compiler (protoc) execute the following:

     ./configure
     make
     make check
     sudo make install
     sudo ldconfig # refresh shared library cache.

If "make check" fails, you can still install, but it is likely that
some features of this library will not work correctly on your system.
Proceed at your own risk.

For advanced usage information on configure and make, please refer to the
autoconf documentation:

    http://www.gnu.org/software/autoconf/manual/autoconf.html#Running-configure-Scripts

**Hint on install location**

By default, the package will be installed to /usr/local.  However,
on many platforms, /usr/local/lib is not part of LD_LIBRARY_PATH.
You can add it, but it may be easier to just install to /usr
instead.  To do this, invoke configure as follows:

    ./configure --prefix=/usr

If you already built the package with a different prefix, make sure
to run "make clean" before building again.

**Compiling dependent packages**

To compile a package that uses Protocol Buffers, you need to pass
various flags to your compiler and linker.  As of version 2.2.0,
Protocol Buffers integrates with pkg-config to manage this.  If you
have pkg-config installed, then you can invoke it to get a list of
flags like so:

    pkg-config --cflags protobuf         # print compiler flags
    pkg-config --libs protobuf           # print linker flags
    pkg-config --cflags --libs protobuf  # print both

For example:

    c++ my_program.cc my_proto.pb.cc `pkg-config --cflags --libs protobuf`

Note that packages written prior to the 2.2.0 release of Protocol
Buffers may not yet integrate with pkg-config to get flags, and may
not pass the correct set of flags to correctly link against
libprotobuf.  If the package in question uses autoconf, you can
often fix the problem by invoking its configure script like:

    configure CXXFLAGS="$(pkg-config --cflags protobuf)" \
              LIBS="$(pkg-config --libs protobuf)"

This will force it to use the correct flags.

If you are writing an autoconf-based package that uses Protocol
Buffers, you should probably use the PKG_CHECK_MODULES macro in your
configure script like:

    PKG_CHECK_MODULES([protobuf], [protobuf])

See the pkg-config man page for more info.

If you only want protobuf-lite, substitute "protobuf-lite" in place
of "protobuf" in these examples.

**Note for Mac users**

For a Mac system, Unix tools are not available by default. You will first need
to install Xcode from the Mac AppStore and then run the following command from
a terminal:

    sudo xcode-select --install

To install Unix tools, you can install "port" following the instructions at
https://www.macports.org . This will reside in /opt/local/bin/port for most
Mac installations.

    sudo /opt/local/bin/port install autoconf automake libtool

Then follow the Unix instructions above.

**Note for cross-compiling**

The makefiles normally invoke the protoc executable that they just
built in order to build tests.  When cross-compiling, the protoc
executable may not be executable on the host machine.  In this case,
you must build a copy of protoc for the host machine first, then use
the --with-protoc option to tell configure to use it instead.  For
example:

    ./configure --with-protoc=protoc

This will use the installed protoc (found in your $PATH) instead of
trying to execute the one built during the build process.  You can
also use an executable that hasn't been installed.  For example, if
you built the protobuf package for your host machine in ../host,
you might do:

    ./configure --with-protoc=../host/src/protoc

Either way, you must make sure that the protoc executable you use
has the same version as the protobuf source code you are trying to
use it with.

**Note for Solaris users**

Solaris 10 x86 has a bug that will make linking fail, complaining
about libstdc++.la being invalid.  We have included a work-around
in this package.  To use the work-around, run configure as follows:

    ./configure LDFLAGS=-L$PWD/src/solaris

See src/solaris/libstdc++.la for more info on this bug.

**Note for HP C++ Tru64 users**

To compile invoke configure as follows:

    ./configure CXXFLAGS="-O -std ansi -ieee -D__USE_STD_IOSTREAM"

Also, you will need to use gmake instead of make.

**Note for AIX users**

Compile using the IBM xlC C++ compiler as follows:

    ./configure CXX=xlC

Also, you will need to use GNU `make` (`gmake`) instead of AIX `make`.

C++ Installation - Windows
--------------------------

If you only need the protoc binary, you can download it from the release
page:

    https://github.com/protocolbuffers/protobuf/releases/latest

In the downloads section, download the zip file protoc-$VERSION-win32.zip.
It contains the protoc binary as well as public proto files of protobuf
library.

Protobuf and its dependencies can be installed directly by using `vcpkg`:

    >vcpkg install protobuf protobuf:x64-windows

If zlib support is desired, you'll also need to install the zlib feature:

    >vcpkg install protobuf[zlib] protobuf[zlib]:x64-windows

See https://github.com/Microsoft/vcpkg for more information.

To build from source using Microsoft Visual C++, see [cmake/README.md](../cmake/README.md).

To build from source using Cygwin or MinGW, follow the Unix installation
instructions, above.

Binary Compatibility Warning
----------------------------

Due to the nature of C++, it is unlikely that any two versions of the
Protocol Buffers C++ runtime libraries will have compatible ABIs.
That is, if you linked an executable against an older version of
libprotobuf, it is unlikely to work with a newer version without
re-compiling.  This problem, when it occurs, will normally be detected
immediately on startup of your app.  Still, you may want to consider
using static linkage.  You can configure this package to install
static libraries only using:

    ./configure --disable-shared

Usage
-----

The complete documentation for Protocol Buffers is available via the
web at:

https://developers.google.com/protocol-buffers/


Zstandard library files
================================

The __lib__ directory is split into several sub-directories,
in order to make it easier to select or exclude features.

#### Building

`Makefile` script is provided, supporting [Makefile conventions](https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html#Makefile-Conventions),
including commands variables, staged install, directory variables and standard targets.
- `make` : generates both static and dynamic libraries
- `make install` : install libraries and headers in target system directories

`libzstd` default scope is pretty large, including compression, decompression, dictionary builder,
and support for decoding legacy formats >= v0.5.0.
The scope can be reduced on demand (see paragraph _modular build_).

#### Multithreading support

When building with `make`, by default the dynamic library is multithreaded and static library is single-threaded (for compatibility reasons).

Enabling multithreading requires 2 conditions :
- set build macro `ZSTD_MULTITHREAD` (`-DZSTD_MULTITHREAD` for `gcc`)
- for POSIX systems : compile with pthread (`-pthread` compilation flag for `gcc`)

For convenience, we provide a build target to generate multi and single threaded libraries:
- Force enable multithreading on both dynamic and static libraries by appending `-mt` to the target, e.g. `make lib-mt`.
  Note that the `.pc` generated on calling `make lib-mt` will already include the require Libs and Cflags.
- Force disable multithreading on both dynamic and static libraries by appending `-nomt` to the target, e.g. `make lib-nomt`.
- By default, as mentioned before, dynamic library is multithreaded, and static library is single-threaded, e.g. `make lib`.

When linking a POSIX program with a multithreaded version of `libzstd`,
note that it's necessary to invoke the `-pthread` flag during link stage.

The `.pc` generated from `make install` or `make install-pc` always assume a single-threaded static library
is compiled. To correctly generate a `.pc` for the multi-threaded static library, set `MT=1` as ENV variable.

Multithreading capabilities are exposed
via the [advanced API defined in `lib/zstd.h`](https://github.com/facebook/zstd/blob/v1.4.3/lib/zstd.h#L351).

#### API

Zstandard's stable API is exposed within [lib/zstd.h](zstd.h).

#### Advanced API

Optional advanced features are exposed via :

- `lib/zstd_errors.h` : translates `size_t` function results
                        into a `ZSTD_ErrorCode`, for accurate error handling.

- `ZSTD_STATIC_LINKING_ONLY` : if this macro is defined _before_ including `zstd.h`,
                          it unlocks access to the experimental API,
                          exposed in the second part of `zstd.h`.
                          All definitions in the experimental APIs are unstable,
                          they may still change in the future, or even be removed.
                          As a consequence, experimental definitions shall ___never be used with dynamic library___ !
                          Only static linking is allowed.

#### Modular build

It's possible to compile only a limited set of features within `libzstd`.
The file structure is designed to make this selection manually achievable for any build system :

- Directory `lib/common` is always required, for all variants.

- Compression source code lies in `lib/compress`

- Decompression source code lies in `lib/decompress`

- It's possible to include only `compress` or only `decompress`, they don't depend on each other.

- `lib/dictBuilder` : makes it possible to generate dictionaries from a set of samples.
        The API is exposed in `lib/dictBuilder/zdict.h`.
        This module depends on both `lib/common` and `lib/compress` .

- `lib/legacy` : makes it possible to decompress legacy zstd formats, starting from `v0.1.0`.
        This module depends on `lib/common` and `lib/decompress`.
        To enable this feature, define `ZSTD_LEGACY_SUPPORT` during compilation.
        Specifying a number limits versions supported to that version onward.
        For example, `ZSTD_LEGACY_SUPPORT=2` means : "support legacy formats >= v0.2.0".
        Conversely, `ZSTD_LEGACY_SUPPORT=0` means "do __not__ support legacy formats".
        By default, this build macro is set as `ZSTD_LEGACY_SUPPORT=5`.
        Decoding supported legacy format is a transparent capability triggered within decompression functions.
        It's also allowed to invoke legacy API directly, exposed in `lib/legacy/zstd_legacy.h`.
        Each version does also provide its own set of advanced API.
        For example, advanced API for version `v0.4` is exposed in `lib/legacy/zstd_v04.h` .

- While invoking `make libzstd`, it's possible to define build macros
        `ZSTD_LIB_COMPRESSION`, `ZSTD_LIB_DECOMPRESSION`, `ZSTD_LIB_DICTBUILDER`,
        and `ZSTD_LIB_DEPRECATED` as `0` to forgo compilation of the
        corresponding features. This will also disable compilation of all
        dependencies (e.g. `ZSTD_LIB_COMPRESSION=0` will also disable
        dictBuilder).

- There are a number of options that can help minimize the binary size of
  `libzstd`.

  The first step is to select the components needed (using the above-described
  `ZSTD_LIB_COMPRESSION` etc.).

  The next step is to set `ZSTD_LIB_MINIFY` to `1` when invoking `make`. This
  disables various optional components and changes the compilation flags to
  prioritize space-saving.

  Detailed options: Zstandard's code and build environment is set up by default
  to optimize above all else for performance. In pursuit of this goal, Zstandard
  makes significant trade-offs in code size. For example, Zstandard often has
  more than one implementation of a particular component, with each
  implementation optimized for different scenarios. For example, the Huffman
  decoder has complementary implementations that decode the stream one symbol at
  a time or two symbols at a time. Zstd normally includes both (and dispatches
  between them at runtime), but by defining `HUF_FORCE_DECOMPRESS_X1` or
  `HUF_FORCE_DECOMPRESS_X2`, you can force the use of one or the other, avoiding
  compilation of the other. Similarly, `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT`
  and `ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG` force the compilation and use of
  only one or the other of two decompression implementations. The smallest
  binary is achieved by using `HUF_FORCE_DECOMPRESS_X1` and
  `ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT` (implied by `ZSTD_LIB_MINIFY`).

  On the compressor side, Zstd's compression levels map to several internal
  strategies. In environments where the higher compression levels aren't used,
  it is possible to exclude all but the fastest strategy with
  `ZSTD_LIB_EXCLUDE_COMPRESSORS_DFAST_AND_UP=1`. (Note that this will change
  the behavior of the default compression level.) Or if you want to retain the
  default compressor as well, you can set
  `ZSTD_LIB_EXCLUDE_COMPRESSORS_GREEDY_AND_UP=1`, at the cost of an additional
  ~20KB or so.

  For squeezing the last ounce of size out, you can also define
  `ZSTD_NO_INLINE`, which disables inlining, and `ZSTD_STRIP_ERROR_STRINGS`,
  which removes the error messages that are otherwise returned by
  `ZSTD_getErrorName` (implied by `ZSTD_LIB_MINIFY`).

  Finally, when integrating into your application, make sure you're doing link-
  time optimization and unused symbol garbage collection (via some combination of,
  e.g., `-flto`, `-ffat-lto-objects`, `-fuse-linker-plugin`,
  `-ffunction-sections`, `-fdata-sections`, `-fmerge-all-constants`,
  `-Wl,--gc-sections`, `-Wl,-z,norelro`, and an archiver that understands
  the compiler's intermediate representation, e.g., `AR=gcc-ar`). Consult your
  compiler's documentation.

- While invoking `make libzstd`, the build macro `ZSTD_LEGACY_MULTITHREADED_API=1`
  will expose the deprecated `ZSTDMT` API exposed by `zstdmt_compress.h` in
  the shared library, which is now hidden by default.

- The build macro `STATIC_BMI2` can be set to 1 to force usage of `bmi2` instructions.
  It is generally not necessary to set this build macro,
  because `STATIC_BMI2` will be automatically set to 1
  on detecting the presence of the corresponding instruction set in the compilation target.
  It's nonetheless available as an optional manual toggle for better control,
  and can also be used to forcefully disable `bmi2` instructions by setting it to 0.

- The build macro `DYNAMIC_BMI2` can be set to 1 or 0 in order to generate binaries
  which can detect at runtime the presence of BMI2 instructions, and use them only if present.
  These instructions contribute to better performance, notably on the decoder side.
  By default, this feature is automatically enabled on detecting
  the right instruction set (x64) and compiler (clang or gcc >= 5).
  It's obviously disabled for different cpus,
  or when BMI2 instruction set is _required_ by the compiler command line
  (in this case, only the BMI2 code path is generated).
  Setting this macro will either force to generate the BMI2 dispatcher (1)
  or prevent it (0). It overrides automatic detection.

- The build macro `ZSTD_NO_UNUSED_FUNCTIONS` can be defined to hide the definitions of functions
  that zstd does not use. Not all unused functions are hidden, but they can be if needed.
  Currently, this macro will hide function definitions in FSE and HUF that use an excessive
  amount of stack space.

- The build macro `ZSTD_NO_INTRINSICS` can be defined to disable all explicit intrinsics.
  Compiler builtins are still used.

- The build macro `ZSTD_DECODER_INTERNAL_BUFFER` can be set to control
  the amount of extra memory used during decompression to store literals.
  This defaults to 64kB.  Reducing this value reduces the memory footprint of
  `ZSTD_DCtx` decompression contexts,
  but might also result in a small decompression speed cost.

- The C compiler macros `ZSTDLIB_VISIBLE`, `ZSTDERRORLIB_VISIBLE` and `ZDICTLIB_VISIBLE`
  can be overridden to control the visibility of zstd's API. Additionally,
  `ZSTDLIB_STATIC_API` and `ZDICTLIB_STATIC_API` can be overridden to control the visibility
  of zstd's static API. Specifically, it can be set to `ZSTDLIB_HIDDEN` to hide the symbols
  from the shared library. These macros default to `ZSTDLIB_VISIBILITY`,
  `ZSTDERRORLIB_VSIBILITY`, and `ZDICTLIB_VISIBILITY` if unset, for backwards compatibility
  with the old macro names.

- The C compiler macro `HUF_DISABLE_FAST_DECODE` disables the newer Huffman fast C
  and assembly decoding loops. You may want to use this macro if these loops are
  slower on your platform.

#### Windows : using MinGW+MSYS to create DLL

DLL can be created using MinGW+MSYS with the `make libzstd` command.
This command creates `dll\libzstd.dll` and the import library `dll\libzstd.lib`.
The import library is only required with Visual C++.
The header file `zstd.h` and the dynamic library `dll\libzstd.dll` are required to
compile a project using gcc/MinGW.
The dynamic library has to be added to linking options.
It means that if a project that uses ZSTD consists of a single `test-dll.c`
file it should be linked with `dll\libzstd.dll`. For example:
```
    gcc $(CFLAGS) -Iinclude/ test-dll.c -o test-dll dll\libzstd.dll
```
The compiled executable will require ZSTD DLL which is available at `dll\libzstd.dll`.

#### Advanced Build options

The build system requires a hash function in order to
separate object files created with different compilation flags.
By default, it tries to use `md5sum` or equivalent.
The hash function can be manually switched by setting the `HASH` variable.
For example : `make HASH=xxhsum`
The hash function needs to generate at least 64-bit using hexadecimal format.
When no hash function is found,
the Makefile just generates all object files into the same default directory,
irrespective of compilation flags.
This functionality only matters if `libzstd` is compiled multiple times
with different build flags.

The build directory, where object files are stored
can also be manually controlled using variable `BUILD_DIR`,
for example `make BUILD_DIR=objectDir/v1`.
In which case, the hash function doesn't matter.

#### Deprecated API

Obsolete API on their way out are stored in directory `lib/deprecated`.
At this stage, it contains older streaming prototypes, in `lib/deprecated/zbuff.h`.
These prototypes will be removed in some future version.
Consider migrating code towards supported streaming API exposed in `zstd.h`.

#### Miscellaneous

The other files are not source code. There are :

 - `BUCK` : support for `buck` build system (https://buckbuild.com/)
 - `Makefile` : `make` script to build and install zstd library (static and dynamic)
 - `README.md` : this file
 - `dll/` : resources directory for Windows compilation
 - `libzstd.pc.in` : script for `pkg-config` (used in `make install`)