Configuring Datastore Indexes with index.yaml

You can use Cloud Datastore for storing data for your applications that run in the flexible environment. Datastore uses indexes for every query your application makes. These indexes are updated whenever an entity changes, so the results can be returned quickly when the app makes a query. To do this, the datastore needs to know in advance which queries the application will make. You specify which indexes your app needs in a index.yaml configuration file. You can use the datastore emulator to generate the file automatically as you test your app, or write the file yourself. The index.yaml file must be uploaded when you deploy your app.

About index.yaml

Every datastore query made by an application needs a corresponding index. Indexes for simple queries, such as queries over a single property, are created automatically. Indexes for complex queries must be defined in a configuration file named index.yaml. This file is uploaded with the application to create indexes in the datastore.

The following is an example of an index.yaml file:


- kind: Cat
  ancestor: no
  - name: name
  - name: age
    direction: desc

- kind: Cat
  - name: name
    direction: asc
  - name: whiskers
    direction: desc

- kind: Store
  ancestor: yes
  - name: business
    direction: asc
  - name: owner
    direction: asc

The syntax of index.yaml is the YAML format. For more information about this syntax, see the YAML website for more information.

Index definitions

index.yaml has a single list element called indexes. Each element in the list represents an index for the application.

An index element can have the following elements:

The kind of the entity for the query. This should be the kind argument given to datastore.NewKey when creating the entity. This element is required.

A list of properties to include as columns of the index, in the order to be sorted: properties used in equality filters first, followed by the property used in inequality filters, then the sort orders and their directions.

Each element in this list has the following elements:

The datastore name of the property.
The direction to sort, either asc for ascending or desc for descending. This is only required for properties used in sort orders of the query, and must match the direction used by the query. The default is asc.
yes if the query has an ancestor clause (Query.Ancestor). The default is no.

Creating index files

You can create an index file manually, using a text editor and following the file layout described above. A more efficient approach is to automatically generate the file as you test your app locally. You can combine the two methods.

When you are testing in your local environment, you can use the gcloud emulator command to start a service that emulates Cloud Datastore before you run your app:

gcloud beta emulators datastore start --data-dir DATA-DIR

Use the --data-dir flag to specify the directory where the auto-generated index.yaml file will appear.

As you test your app, each time you generate a Datastore query, the emulator adds a generated index definition to index.yaml. All the automatically generated indexes will appear in the file below the following line:


All index definitions above this line are considered to be under manual control, and are not updated by the development web server. The web server will only make changes below the line, and will only do so if the complete index.yaml file does not describe an index that accounts for a query executed by the application. To take control of an automatic index definition, move it above this line.

The emulator may update existing definitions below this line as the application makes queries. If the app generates every query it will make while you test it, then the generated entries in index.yaml will be complete. You might need to edit the file manually to delete indexes that are not used in production, or to define indexes that were not created while testing.

Deploying the index file

Before you deploy your application, you should use the gcloud command to deploy the index.yaml file:

gcloud app deploy index.yaml

Monitor your resources on the go


Send feedback about...

App Engine flexible environment for Go