Skip to content

Instantly share code, notes, and snippets.

@olliefr
Last active November 18, 2022 02:46
Show Gist options
  • Save olliefr/4180347478cca1de40533635d628749a to your computer and use it in GitHub Desktop.
Save olliefr/4180347478cca1de40533635d628749a to your computer and use it in GitHub Desktop.
Datastore (Google Cloud)

Datastore is weird!

It is a legacy service tied to App Engine. Once enabled, App Engine API cannot be disabled on a Cloud project.

Cloud projects can use either Datastore or Firestore but not both. Once set, it cannot be changed. Since all legacy App Engine apps use Datastore, a different project must be used for Firestore usage.

This Codelab gives some historical context on App Engine, Datastore, and Firestore.

For Python client for Datastore API there are at least two options:

There is a Terraform module for deploying Datastore (as Firestore in Datastore mode). It appears to be using App Engine related Terraform resources as it seems that there are no Terraform resources directly addressing Datastore or Firestore creation in Terraform Google provider.

The App Engine Terraform Google provider resource appears to be using App Engine Admin API client library. The client library makes calls to App Engine Admin API, probaby via gRPC.

I cannot find what the default value for the databaseType field is.

The Firestore in Datastore mode documentation explains how to change the database mode.

The call to create a Datastore appears to have been deprecated, so not using gcloud datastore databases create ....

Will create a Firestore and then change the database mode.

Attempting to create a Firestore fails because App Engine Admin API has not been enabled yet.

gcloud firestore databases create --region=europe-west2 --project=$GOOGLE_CLOUD_PROJECT

Enabling the App Engine Admin API.

gcloud services enable appengine.googleapis.com --project=$GOOGLE_CLOUD_PROJECT

Repeating an attempt to create a Firestore fails again, now with the error that an App Engine application must be created first. Creating the App Engine application.

gcloud app create --region=europe-west2 --project=$GOOGLE_CLOUD_PROJECT

Apparently, creating an App Engine for the project is "irreversible", and the region cannot be changed. LOL at those legacy services!

The App Engine application has now been created. Not going to use gcloud app deploy ..., as the message returned by gcloud suggests as I have no application to deploy. All I want is Datastore.

Now attempting to create a Firestore yet again... success this time.

Changing the database mode using the method described in the documentation.

gcloud alpha firestore databases update --type=datastore-mode --project=$GOOGLE_CLOUD_PROJECT

Noted that gcloud datastore databases ... does not have an option to list or describe database instances. It appears that the whole gcloud datastore group of commands is legacy.

Further to this, gcloud firestore databases also doesn't have the option to list or describe database instances.

As of now, the database instances can only be listed by an alpha command.

gcloud alpha firestore databases describe --project=$GOOGLE_CLOUD_PROJECT

There, I can see that the database has indeed been created, and is of type: DATASTORE_MODE.

Generating MongoDB object ID: mongodb-objectid-generator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment