Simple utility script for generating and managing code samples for GAPIC client libraries.
FYI:Β this can also be used as a general purpose tool for generating GAPIC client libraries.
Here Be Dragons π
This is my personal swiss army knife for performing sample generation operations.
It may change at any time. It's all held together with scotch tape and rubber bands.
# Download samplegen script (latest)
curl -O https://gist.githubusercontent.com/beccasaurus/570ae7bb4fb819c87ef029bafd61f28e/raw/samplegen
# Clone required repositories (clones repos into your workspace)
samplegen --clone
# Print out all of the samples for the Language API (right now there should be none)
samplegen videointelligence --print
Note: run
./samplegen [command]
if you're running the script directly versus adding to your PATH
If you want to add a sample and generate it...
# Edit the GAPIC config file for the api/library you'd like to add samples to
samplegen speech v1p1beta1 --edit --gapic-config
# Generate the library with samples
samplegen speech v1p1beta1 --print
If you want to edit the .snip
template file for generated samples...
# Edit the .snip for a given language
samplegen --edit -l php
# Print out the sample after changed were made to the template file
# You must --rebuild which builds gapic-generator to use the updated .snip
samplegen speech --print --rebuild
Generates Code Samples for GAPIC-generated client libraries
https://github.com/googleapis/gapic-generator
Usage:
samplegen --help
samplegen --clone
samplegen speech --print
samplegen speech v1 --print
samplegen speech v1 --edit
samplegen speech v1 --gapic-generator ~/my-local-gapic-generator
--googleapis ~/my-local-googleapis
--output ~/mysamples/
--region-tag speech_transcribe_file
--language python
Options:
-a, --artman Reserved (for passing additional info/flags to arman)
Passed the provided arguments to `artman` invocation
Not currently useful
-b, --bash Rather than generating the library, open the bash
prompt in the provided Docker image (with mounts)
-c, --config Path to persisted configuration YAML file
The following are searched, in order:
- Explicitly provided --config argument
- SAMPLEGEN_CONFIG environment variable
- samplegen.yml in local directory
- [workspace]/config.yml
--clone Clones googleapis & gapic-generator repositories
into your workspace, e.g. ~/.samplegen by default
-d, --docker Path to docker command
Default: docker
--dry-run Performs no code generation, prints commands instead
Example: speech v1 --dry-run
-e, --edit Opens gapic config and language .snip in your $EDITOR
Pass --gapic-config or --snip to open only one file
--enable-dev_samples Edits GAPIC files to enable sample generation in
unstable languages. Run with --rebuild to apply
--gapic-config Print full path to api_gapic.yaml in googleapis
Example: language --gapic-config
--gapic-generator Directory to gapic-generator project
Default: [workspace]/gapic-generator
--googleapis Directory to googleapis project
Default: [workspace]/googleapis
-i, --image Docker image to execute `artman`
Default: googleapis/artman
--info Prints configuration values and version
-l, --language Programming language to genererate samples for
Default: python
--no-gen Sample generation is not run but other operations are
whereas --dry-run does not execute any operations
-o, --output Directory to save sample code files
Default: .
-p, --print Print samples rather than saving them
Example: --print (pygmentize used when available)
Files will also be saved if explicit --output provided
--pull Pull latest googleapis, gapic-generator, and googleapis/artman
--rebuild Rebuilds gapic-generator before running
Executes: ./gradlew fatJar (for .snip changes)
-r, --region-tag Filter subset of region tags to save
Example: -r sentiment_text
-s, --save Persist passed options to config file
Example: --googleapis ../api --save
--snip Print full path to .snip sample template in gapic-generator
Example: --language python --snip
--version Prints the version number of this script
-w, --workspace Root directory of typical directory structure
Default: ~/.samplegen
Environment Variable: SAMPLEGEN_WORKSPACE
Um, okay. Fine then. π€·πΌ
Here is a 2-line script that works today for generating any APIs in googleapis/ and will save output to your current directory
Pick an API by finding an artman_[api].yaml
file in https://github.com/googleapis/googleapis/tree/master/google
The following example is for cloudkms:
docker pull googleapis/artman
docker run --rm --workdir /googleapis --volume `pwd`:/googleapis/artman-genfiles \
--env RUNNING_IN_ARTMAN_DOCKER=True googleapis/artman /bin/bash -c \
"artman --local --config /googleapis/google/cloud/kms/artman_cloudkms.yaml generate python_gapic"
Here are unabridged instructions, Ms "Do It Yourself" ...
# NOTE: The following instructions are not specific to Code Samples.
# These instructions can be used to generate client libraries.
# Dependencies: docker, java
# (Optional) Clone the 2 repositories you'll need
# [gapic-generator] β The GAPIC generator (Java project)
git clone https://github.com/googleapis/gapic-generator.git
# [googleapis] βΒ Repository containing necessary artifacts for generating
# client libraries, e.g. .proto files, and [api]_gapic.yaml
# configuration files (where samples are authored as YAML),
# and artman_[api].yaml files which are simple files containing
# paths/options which `artman` used to invoke GAPIC generator.
#
# Why use artman? Because we have a published Docker image pre-built
# with BAGILLIONS of dependencies needed to successfully run the generator.
# ... yes "bagillions" ... I stand by this assessment ;)
git clone https://github.com/googleapis/googleapis.git
# Cloning these repos locally is optional!
# The docker image we'll be using pulls the latest HEAD versions of googleapis and gapic-generator by default
# But I'll show how to use local directories because these instructions are for setting up a DEVELOPMENT environment
# The commands will need your repository paths, so I'll put them into variables:
GOOGLEAPIS_PATH="`pwd`/googleapis"
GAPIC_GENERATOR_PATH="`pwd`/googleapis"
# Now choose an existing, published API from googleapis β
# We will use its existing artman_[api].yaml config file to generate that API.
# You can find this file alongside the API's .proto, gapic config, etc.
# Let's pick an API... um... KMS! Let's search for the artman file...
cd $GOOGLEAPIS_PATH
ARTMAN_CONFIG_PATH=`git ls-files | grep artman_ | grep kms | grep .yaml`
# ARTMAN_CONFIG_PATH should equal google/cloud/kms/artman_cloudkms.yaml (relative to googleapis)
#
# Feel free to take a look at the artman config, it's very small:
# https://github.com/googleapis/googleapis/blob/master/google/cloud/kms/artman_cloudkms.yaml
# Now let's generate a KMS client library!
# Pull the docker image we'll be using...
# (you can think of this as the GAPIC generator's official Docker image)
# Source Dockerfile: https://github.com/googleapis/artman/blob/master/Dockerfile
# ^--- work a quick skim/read βΒ shows how to install dependencies to run GAPIC generator
DOCKER_IMAGE=googleapis/artman
docker pull $DOCKER_IMAGE
# Now you're ready!
# Pick a language βΒ we'll just generate 1 in this tutorial
LANGUAGE=python
# We'll be MOUNTING your local googleapis and gapic-generator repos
# and invoking artman passing our artman config file for KMS
# Let's create the artman command first... noting that googleapis will be mounted at /googleapis
ARTMAN_COMMAND="artman --local --config /googleapis/$ARTMAN_CONFIG_PATH generate $LANGUAGE_gapic"
# Sweet, let's create the main docker command (we'll kick off the ^ ARTMAN_COMMAND against the container we run)
DOCKER_COMMAND="docker run --rm
--volume $GAPIC_GENERATOR_PATH:/toolkit
--volume $GOOGLEAPIS_PATH:/googleapis
--workdir /googleapis
--env RUNNING_IN_ARTMAN_DOCKER=True
$DOCKER_IMAGE"
# Sweet! Let's run it y'll! π€
FULL_DOCKER_COMMAND="$DOCKER_COMMAND /bin/bash -c \"$ARTMAN_COMMAND\""
$FULL_DOCKER_COMMAND
# If this worked, you should see: "Code generated: /googleapis/artman-genfiles/python/kms-v1"
# Take a look at all of the generated files :)
find $GOOGLEAPIS/artman-genfiles/
# π MAKING EDITS / CHANGES
# If you want to add/change code samples, you'll want to edit the API's _gapic.yaml "GAPIC config"
# The GAPIC config is used to configure TONS of things for library generation βΒ *including samples*
# Following with the KMS theme...
GAPIC_CONFIG_PATH=`git ls-files | grep gapic | grep kms | grep .yaml`
# Change something!
vi $GAPIC_CONFIG_PATH
# Here's an example of adding a code sample...
# We'll make a sample which lists KMS key rings under the ListKeyRings rpc method
# https://cloud.google.com/kms/docs/reference/rpc/google.cloud.kms.v1#google.cloud.kms.v1.KeyManagementService.ListKeyRings
# Add the following to the yaml config under the `- name: ListKeyRings` config section for the ListKeyRings rpc API method
methods:
- name: ListKeyRings
samples:
standalone:
- calling_forms: ".*"
value_sets: list_key_rings
region_tag: list_key_rings
sample_value_sets:
- id: list_key_rings
title: "List Key Rings"
description: "List Key Rings"
parameters:
defaults:
- parent: "Full path to location, e.g. projects/MY-GCP-PROJECT/locations/global"
attributes:
- parameter: parent
sample_argument: true
on_success:
- loop:
variable: key
collection: $resp.key_rings
body:
print:
- "Key ring name: %s"
- key.name
# Now that you've added a sample, run the generator.
# There will be a new ./samples/ directory in the folder of the generated client library.
$FULL_DOCKER_COMMAND
find $GOOGLEAPIS_PATH/artman-genfiles/python/kms-v1/samples/google/cloud/kms_v1/gapic/list_key_rings/
# => artman-genfiles/python/kms-v1/samples/google/cloud/kms_v1/gapic/list_key_rings/list_key_rings_request_paged_list_key_rings.py
# => artman-genfiles/python/kms-v1/samples/google/cloud/kms_v1/gapic/list_key_rings/list_key_rings_request_paged_all_list_key_rings.py
# Sweet, right?
# Now let's edit the template file for generated Python samples
# 1. Edit the template and change something
cd $GAPIC_GENERATOR_PATH
vi src/main/resources/com/google/api/codegen/py/standalone_sample.snip
# 2. Recompile gapic-generator (required)
./gradlew fatJar
# 3. Run generation again & you should see the changes!
cd $GOOGLEAPIS_PATH
$FULL_DOCKER_COMMAND