Skip to content

Instantly share code, notes, and snippets.

@baflo

baflo/1 Introduction.md

Last active October 8, 2017 10:29
Show Gist options
  • Save baflo/4264417e4a5557e086e540f3d7672291 to your computer and use it in GitHub Desktop.
Save baflo/4264417e4a5557e086e540f3d7672291 to your computer and use it in GitHub Desktop.
Download ZIP
From Zero to multilingual Actions on Google
Raw
1 Introduction.md

From Zero to multilingual Actions on Google

Actions on Google is Google's platform to provide apps for the Google Assistant and Google Home. Technically speaking, the Actions on Google platform is just a broker connecting services with the Google Assistant system, i.e. invocations by users arrive at Actions on Google, which uses information from the Action Package (or API.AI project, ref) to route the invocation request to the correct service.

There are a couple of ways to create services for Actions on Google. One can use Templates available on the Actions on Google platform, or define a request/response pattern on API.AI, or completely build and self-host the applications using the Actions SDK.

They all have in common that they use the Actions on Google console to manage, test and deploy different projects that shall be published to Google Assistant. Some of those tasks may be done using the gactions CLI, but all data appears there eventually.

Templates are used directly from the Actions console and can be clicked together completely in browser. Plus, its content is configured in Google Docs sheets.

API.AI is used to create an agent service, that receives requests (from Actions on Google) and resolves tem to the correct intents. An intent may be triggered by many different requests ("user says"), but results in exactly one action (e.g. "input.welcome", "input.goodbye", see here). However, each intent/action can have multiple responses or a so-called fulfillment. A fulfillment is an Actions SDK powered service accessed by a webhook. It may be hosted anywhere. API.AI can also be used to connect to other servies using the same agent, i.e. Slack, Cortana, Twitter and even helps to clone the project to Alexa.

Actions SDK is a toolkit helping to write own fulfillments. These may be hosted as serverless function at Google Cloud Platform (or indirectly at Google's Firebase) and are accessed directly by the Actions on Google service or indirectly through API.AI.

In the following, we'll build a multilingual app using API.AI. First, we'll build and test an app using API.AI's request/response pattern only. After that, we'll serve some intents with a webhooks (build using Actions SDK).

Raw
2 API.AI Agent.md

Configure an API.AI agent

Now we build an agent can receive requests and tries to map them to an intent. After these steps, there will be a couple of static request/response pairs for multiple languages. Eventually, the agent is connected to the Actions on Google platform to receive requests from Google Assistant.

Create agent

  • Goto API.AI
  • Create a new agent
    • Select sample data, e.g. EasterEgg (which has no templates, i.e. bit easier to understand in the beginning)
    • Give it a name and description
    • Select English (en) as default language and timezone
    • Save!

Translate intents

  • In the top left corner, next to [en] (or whatever was selected as default language), click on the plus
    • Add new language
    • Save!
  • Go to Intents
    • Select an intent (still being in your default language)
    • Note what the intent is about
    • Now select new language in top left corner
    • Enter some texts in "User says" and "Responses" for the second language
      • There should currently be only static requests/responses without templates
    • Save! Continue translating other intents.

Integrate agent with Actions on Google

  • Go to Integrations
  • Select Google Assistant
  • optional: use API v2
  • Activate and click Update Draft!
Raw
3 Actions on Google.md

Test and connect with Google Assistant

API.AI has automatically created a new project in the Actions Console.

Test agent with "Actions on Google" simulator

  • If you're still in API.AI, you should see a button "Visit Console". Either
  • Go to Simulator
    • Select your default language
    • "Start testing"
    • Select the correct version (there should only be one)
      • If something was changed in API.AI agent, you must update the simulator by
        1. Update draft in API.AI console -> Integrations -> Google Assistant
        2. Select new version in Actions console -> Simulator -> "Change Version"
    • Start talking in simulator
      • Until now, the app must be started by using invocation name "my test app"
    • Test in all configured languages

Change invocation name

  • Go to Actions console -> Overview
  • Add App information
  • Setup assistant name and pronounciation for all languages
  • For now, ignore other fields.
  • Save and go to Simulator.
  • Try to invoke by new name.
    • PLUS: it should work with any device that's connected with the same Google account, now.
Raw
4 Templates on API.AI.md

Create an echo service using templates

In this section, first templates are used to create dynamic responses while still being in API.AI. These steps don't require hosting own functions/webhooks/fulfillments.

Create new intent for command "echo"

  • Start API.AI console
  • Go to intents sections and create intent
  • Call is sth like easteregg.echo
  • In the "User says" section, click on the quotes next to the input field. It should now show an @.
  • Enter echo @sys.any:content
    • This is a template that expects a leading command echo
    • The rest of the request string (@sys.any) is written into the parameter content
  • If you click Save now, it automatically creates a parameter called content in the Action section
    • @sys.any strips off some symbols. We want the original, so we add another parameter in the Action section:
    • Enter paramerter name = original_content, entity = @sys.any, value = $content.original
  • Finally, in the Response section:
    • Add a response as follow You said: "$original_content"
    • Don't forget to enter a response for your second language.
  • Save intent, go to Integrations, update draft for Google Assistant
  • Go to Actions on Google console and test:
    • Go to simulator
    • Update version
    • Start conversation
    • Enter echo Hi
Raw
99-INFOS.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment