Video Playback with the Google Assistant on Android TV

Posted by Benjamin Baxter, Developer Programs Engineer

How to integrate the Google Assistant in a TV app

Earlier this year, we announced that the Google Assistant will be coming to Android TV and it has arrived. The Google Assistant on Android TV will allow users to discover, launch and control media content, control smart devices like light bulbs, and much more. Your Assistant also understands that you're interacting on a TV, so you'll get the best experience possible while watching your favorite movies and TV shows.

The Google Assistant has a built-in capability to understand commands like "Watch The Incredibles", and media controls, like pause, fast forward, etc. This article will walk through how to integrate the Google Assistant into your application.

There are no new APIs needed to integrate with the Google Assistant. You just need to follow the pattern that the Google Assistant expects from your app. If you want to experiment and play with the APIs and the Assistant, you can download this sample from github.

Discovery

The Google Assistant has made some changes to improve finding information on Android TV.

There are a few ways to expose your content to users through the Google Assisant.

Server side integration. (Requires registration and onboarding)

You need to provide your content catalog to Google. This data is ingested and available to the Google Assistant outside of your app.

This is not specific for Google Assistant. It will also enable other Google services such as search and discovery on Google Search, Google Play, Google Home App, and Android TV.

Client side integration. (Available to all apps)

If your app is already searchable, then you only need to handle the EXTRA_START_PLAYBACK flag, which we go into more detail later. Content will auto-play if the app name is explicitly specified in the search results or if the user is already in your app.

Once your app is searchable, you can test by asking the Assistant or, if you are in a loud area, test quietly by running the following adb command:

adb shell am start -a "android.search.action.GLOBAL_SEARCH" --es query \"The Incredibles\"

Each app that responds to the search query will have a row displaying their search results. Notice that YouTube and the sample app, Assistant Playback, each receive their own rows for content that match the search query.

For specific searches such as "Play Big Buck Bunny", the Assistant will present a card with a button for each app that exactly matched the search query. In the screenshot below, you can see the sample app, Assistant Playback, shows up as an option to watch Big Buck Bunny.

There are times when the Google Assistant will launch an app directly to start playing content. An example of when this occurs is when content is exclusive to the app; "Play the Netflix original House of Cards".

Launching

When the user selects a video from search results, an intent is sent to your app. The priority order for the intent actions are as follows:

Intent specified in the cursor returned from the search (SUGGEST_COLUMN_INTENT_ACTION).
Intent specific in the searchable.xml file with the searchSuggestIntentAction value.
Defaults to ACTION_VIEW.

In addition, the Assistant will also pass an extra to signal if playback should begin immediately. You app should be able to handle the intent and expect a boolean extra called EXTRA_START_PLAYBACK.

import static android.support.v4.content.IntentCompat.EXTRA_START_PLAYBACK;

public class SearchableActivity extends Activity {

   @Override
   protected void onCreate(@Nullable Bundle savedInstanceState) {
       super.onCreate(savedInstanceState);
       if (getIntent() != null) {
           // Retrieve video from getIntent().getData().

           boolean startPlayback = getIntent().getBooleanExtra(EXTRA_START_PLAYBACK, false);
           Log.d(TAG, "Should start playback? " + (startPlayback ? "yes" : "no"));

           if (startPlayback) {
               // Start playback.
               startActivity(...);
           } else {
               // Show details for movie.
               startActivity(...);
           }
       }
       finish();
   }
}

You can test this by modifying and running the following adb command. If your app has a custom action, then replace android.intent.action.VIEW with the custom action. Replace the value of the -d argument with the URI you return from the Assistant's query.

adb shell 'am start -a android.intent.action.VIEW --ez
android.intent.extra.START_PLAYBACK true -d <URI> -f 0x14000000'

The -f argument is the logical OR value from FLAG_ACTIVITY_NEW_TASK | FLAG_ACTIVITY_CLEAR_TOP. This will force your activity to be freshly launched.

For example, in the sample app, you can run the following command to launch playback of "Big Buck Bunny" as if the assistant had launched it.

adb shell 'am start -a android.intent.action.VIEW --ez
android.intent.extra.START_PLAYBACK true -d 
content://com.example.android.assistantplayback/video/2 -n
com.example.android.assistantplayback/.SearchableActivity -f 0x14000000'

The URI above is defined by the value of android:searchSuggestIntentData in searchable.xml (content://com.example.android.assistantplayback/video/) in addition to the value of SearchManager.SUGGEST_COLUMN_INTENT_DATA_ID (2) returned from the query.

Note that intents may be cached by the Google Assistant up to 7 days. Your app could receive a request to play content that is no longer available. The intent handler should be designed to be stateless and not rely on any previously knowledge to handle the deep link. Your app should gracefully handle this situation. One solution would be to show an error message and let the user land on your main activity or another relevant activity.

Playback

If your app implements MediaSession correctly, then your app should work right away with no changes.

By Blender Foundation. License under Creative Commons Attribution 3.0 license.

The Google Assistant assumes that your app handles transport controls. The Assistant uses the TransportControls to send media commands to your app's MediaSession. Video apps must support the following controls wherever possible:

Play/Pause/Stop
Previous/Next
Rewind/Fast Forward (implemented with seekTo())

You can easily get a hook for these controls by implementing a MediaSession.Callback. If you play videos using PlaybackTransportControlGlue, then all your callback needs to do it sync the glue and the MediaSession. Otherwise use this callback to sync your player.

public class MyMediaSessionCallback extends MediaSessionCompat.Callback {

   private final PlaybackTransportControlGlue<?> mGlue;

   public MediaSessionCallback(PlaybackTransportControlGlue<?> glue) {
       mGlue = glue;
   }

   @Override
   public void onPlay() {
       Log.d(TAG, "MediaSessionCallback: onPlay()");
       mGlue.play();
       updateMediaSessionState(...);
   }

   @Override
   public void onPause() {
       Log.d(TAG, "MediaSessionCallback: onPause()");
       mGlue.pause();
       updateMediaSessionState(...);
   }

   @Override
   public void onSeekTo(long position) {
       Log.d(TAG, "MediaSessionCallback: onSeekTo()");
       mGlue.seekTo(position);
       updateMediaSessionState(...);
   }

   @Override
   public void onStop() {
       Log.d(TAG, "MediaSessionCallback: onStop()");
       // Handle differently based on your use case.
   }

   @Override
   public void onSkipToNext() {
       Log.d(TAG, "MediaSessionCallback: onSkipToNext()");
       playAndUpdateMediaSession(...);
   }

   @Override
   public void onSkipToPrevious() {
       Log.d(TAG, "MediaSessionCallback: onSkipToPrevious()");
       playAndUpdateMediaSession(...);
   }
}