Building your first Google Assistant Skill

Just earlier this year, Google introduced two new tools for developing conversational interfaces and Actions on the Google Assistant Platform: The Actions Builder and the Actions SDK. The Actions Builder is a powerful web-based IDE that will make the development process for your Google Assistant Action easy and efficient by providing a graphical user interface to the Actions SDK. Actions Builder and SDK work very well in conjunction together.

In this example, our goal will be to develop a Google Assistant Action that talks to the Twitter API and reads out the latest Tweet of a specific user. For this, you’ll need a Google account and a Twitter developer account.

Setting up your project

Requirements

• Node.js
• gactions CLI:
  The gactions CLI will be useful to sync our project between the Actions Builder and our local development environment. It also allows you to deploy the latest changes to the desired channel. I recommend installing the CLI via homebrew, although it's also available as a binary via https://developers.google.com/assistant/conversational/quickstart#install_the_gactions_command-line_tool
  Install via homebrew: brew install gactions
After successful installation, run gactions login to authenticate the CLI with your Google account.

Creating an Actions project

To create your project, head to the Actions Console: console.actions.google.com

Since we’re building a custom app for which we don’t need any samples, just select “custom” and “blank” in the following screens. This will create an empty project and gives us all the freedom we need.

console.actions.google.com

Pulling the Actions project to your local environment

We can now use the gactions CLI to ‘clone’ our project to our local machine for easier workflows. For this, we’ll need the project ID, which you can find via your project settings.

Create an empty directory in your desired location and pull your project:

mkdir my-first-action

gactions pull --project-id my-first-action

… and we’re ready to go!

Structure of a Google Assistant Action

Before actually building our Tweets Action, let’s have a look at how a user would normally interact with a skill on the Google Assistant platform, no matter the device.

https://www.youtube.com/watch?v=V6s5vp4l4WE

Invocation

To start an Assistant Action, say the wake phrase “Hey Google” or “Ok Google” followed by an invocation. An invocation is what lets Assistant know that the user wants to start a conversation with your Action. Combined, this might result in something like “Hey Google, talk to CNN.”. Invocations are unique and can be reserved.

In your Develop tab in the Actions Builder, you’ll find a tab called “Invocation”. This tab houses the most important settings, such as your Action’s name, and the first actions after the invocation took place.

Scenes

The purpose of Scenes is to help you organize your conversation into building blocks. Imagine an Action that reads you your daily routine: Scenes might be Greeting, Calendar, Weather, News, Music. You can transition freely between scenes and even carry over data.

Intents

A user intent represents an action the user wants to take. Google Assistant uses NLU (Natural Language Understanding) to map user input to an intent. For this to work well, you can provide multiple training phrases for intents. This helps the Assistant to better understand what the user wants to do.

Prompts

And finally, Prompts are how the Assistant responds. This can be as simple as a spoken answer, but can also include complex cards and media responses for devices with displays.

Mapping out our Action

For a start, we are going to modify the greeting that should take place after the first invocation of our Action. For this, navigate to Invocation → Main invocation and click on the block that says “Invoke … Your Action”. Since we only want to respond with a simple spoken sentence for now, select “Send prompts”. You’ll see a small text editor, where you can simply edit the text you want the Assistant to say in either YAML or JSON format.

After the greeting took place, we’ll transition into our first scene. To create a scene, simply open the dropdown below “Transition”, enter a name like “Read Tweet” and add it. You will now also see the newly created scene in the menu on the left.

To implement the logic to read out a Tweet, switch to the “Read Tweet” scene. Since we’ll need the Twitter API to get our answer, a simple Prompt won’t do the trick this time. For this to work, we’ll use a webhook which should be called after entering the Scene. Select the new Scene on the left, in the “Transition to” block, select the “On enter” row and add a webhook. For the name, choose something simple — like “twitter”.

console.actions.google.com

Now finally, to create the webhook that contains all the logic behind our action, select “Webhook” in the menu on the left and make sure you see the default template for Cloud Functions.

Creating our Cloud Function

Since it’s more convenient to edit the actual code from a desktop IDE, we’ll now sync the basic conversation tree we created in the Actions Builder to our local filesystem. Just run gactions pull --clean and you should be up-to-date again :)

Now, let’s open the project folder in a text editor or IDE, and navigate to webhooks/ActionsOnGoogleFulfillment. Here you’ll find the basic structure of a node project, but for now, only the index.js file is going to be interesting. This is where all the magic happens.

As a first step, remove the sample code and rename your handler to the name you gave the webhook before. Since I named mine “twitter”, it looks like this for me:

const { conversation } = require('@assistant/conversation');
const functions = require('firebase-functions');
const app = conversation();
app.handle('twitter', conv => {
  });
exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);

We will need to use the networking library needle later, now is a good time to install that with npm i needle.

Working with the Twitter API

In this example, we’ll be working with the Twitter API v2. To get access, sign up for a Twitter developer account on developer.twitter.com. Once you have a developer account, create a new app and note down the Bearer token, which we’ll need to authenticate.

In the index.js, we’ll add the following constants:

//imports the networking library needle which we'll use to handle API requests
const needle = require('needle');
//your Twitter App Bearer token
const token = 'TOKEN';
//Twitter API v2 URL
const apiUrl = 'https://api.twitter.com/2/tweets/search/recent';

Getting the most recent Tweets of a specific user

Let’s write a simple asynchronous function that gets us the latest tweets.

async function getTweets(user) {
    /*here we can configure query parameters, but since we'll only be using the Tweet's body, we only need to add the username and include retweets.*/
    const params = {
        'query': `from:${user} -is:retweet`
    };
    //this is the actual API request
    const res = await needle('get', apiUrl, params, { headers: {
        "authorization": `Bearer ${token}`
    }});
    //If the request was successful, return the response body
    if(res.body) {
        return res.body;
    } else {
        throw new Error ('Unsuccessful request');
    };
};

Getting the Assistant to speak out the tweet

In our twitter handler, we can now use the function we just wrote to pass the Tweet body to our conversation and get our Action to read the Tweet to the user:

return getTweets('realdonaldtrump').then(response => {
/*By default, Twitter's response will include the 10 most recent Tweets. Since we only need the most recent one, and only the body, we'll select that from the response using response.data[0].text */
    conv.add(JSON.stringify(response.data[0].text));
  }).catch(e => {
    conv.close('Oops, I could not find any Tweets.');
  });

Finishing touches

We’ve successfully built an action that connects to the Twitter API and reads the most recent tweet of a specified user. To round it all off, let’s add a nice ending to our Action.

A simple

conv.close('If you would like to hear this again, simply say Hey Google, talk to MY ACTION');

will do the trick!

The full index.js file now looks like this:

const { conversation } = require('@assistant/conversation');
const functions = require('firebase-functions');
const app = conversation();
const needle = require('needle');
const token = 'XXXX';
const apiUrl = 'https://api.twitter.com/2/tweets/search/recent';
app.handle('twitter', conv => {
  return getTweets('realdonaldtrump').then(response => {
    conv.add(JSON.stringify(response.data[0].text));
    conv.close('If you would like to hear this again, simply say Hey Google, talk to MY ACTION');
  }).catch(e => {
    conv.close('Oops, I could not find any Tweets.');
  });
  async function getTweets(user) {
    const params = {
      'query': `from:${user} -is:retweet`
    };
    const res = await needle('get', apiUrl, params, {
      headers: {
        "authorization": `Bearer ${token}`
      }
    });
    if (res.body) {
      return res.body;
    } else {
      throw new Error ('Unsuccessful request');
    };
  };
});
exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);

Let’s test this out! 🎉

gactions deploy preview