Android Speech Recognition for Unity3D

Adding voice commands to your Android games has never been easier!

This plugin allows you to integrate the powerful Android Speech Recognition technology in your Unity3D project.

You can choose to listen to the events of the RecognizerIntent directly or you can use the SpeechDictionary class from this plugin to listen to a limited number of commands.

The plugin allows you to listen continuously (evaluating speech is started a short time after a loud noise), or you can choose to start and stop listening manually. The latter can be done from code or by defining a ‘touch to listen’ rectangle.

The plugin supports Unity 4.3+ (Free). Android 2.3+ is supported when the device is connected to the internet. Devices running Android 4.1+ have the possibility to install languages for offline speech recognition.

Demo APK:
A build of the demo application can be downloaded here.
Its a simple 3×3 grid where you can move the cross around, using the following commands: go up, go down, go left, go right. The word go is optional if you are using touch to detect.

Manual:

Insert the SpeechRecognition prefab into your first scene and adapt the settings to handle your needs.

These are all the options you have on SpeechRecognition prefab, they can also be changed in code:

  • Max Results: The maximum number of possible results you want to receive. Offline speech recognition will only give the best result.
  • Preferred Language: The recognizer will try to recognize words from this language. More information about the format can be found at http://developer.android.com/reference/java/util/Locale.html
  • Enable Debug Log: Enable this if you want to enable Debug.Log messages for certain errors.
  • Enable Event Log: Enable this if you want to enable Debug.Log messages for each of the events.
  • Enable <certain event>: Enable the events you want to listen to directly. When using the SpeechDictionary you don’t have to enable any of these events. More information about the events can be found at http://developer.android.com/reference/android/speech/RecognitionListener.html
  • Auto Restart: This can only be enabled when not using touch to listen. When enabled the app will listen to the background noise. When the background noise is loud enough it will restart the SpeechRecognizer to actually listen to speech. Note that it can take 1-2 seconds before the plugin is ready to listen to speech. This functionality is useful when for example, the user has to speak a certain word before giving the actual command.
  • Auto Restart On Resume: This is only used when ‘Auto Restart’ is enabled. When enabled and the app comes back to the foreground the plugin will automatically start listening again.
  • Auto Restart Amp Threshold: The threshold used to automatically restart listening to speech. A higher value means that the background sound must be louder to trigger the start of speech recognition.
  • Disable Screen Lock On Results: This will disable the screen lock for ‘Screen Lock Timeout’ seconds, after the plugin has received the results from the latest speech command. This is useful if the user will not touch the screen for a while because he is only using voice commands. Do not enable this option when you are manually disabling and enabling the screen lock.

The SpeechDictionary can be edited in the Inspector tab, or can be created from Code. Each item in the SpeechDictionary has a name and multiple valid speech commands. The plugin has an example on how to use it. You can look in the SpeechDictionary.cs file to see how you can define a SpeechDictionary from code.

To connect the plugin with your code you should call the static methods from the SpeechRecognition class.

You can change the following settings on the SpeechRecognition.instance object:

public int maxResults = 5;
public string preferredLanguage = “en-US”;
public bool enableDebugLog = false;
public bool enableEventLog = false;
public bool enableOnBeginningOfSpeech = false;
public bool enableOnBufferReceived = false;
public bool enableOnEndOfSpeech = false;
public bool enableOnEvent = false;
public bool enableOnPartialResults = false;
public bool enableOnReadyForSpeech = false;
public bool enableOnRmsChanged = false;
public bool autoRestart = false;
public bool autoRestartOnResume = false;
public float autoRestartAmpThreshold = 0.5f;
public bool disableScreenLockOnResult = false;
public float screenLockTimeout = 10f;

The next static functions are available on the SpeechRecognition class:

public static void AddSpeechRecognitionListeren(ISpeechRecognitionListener listener)
public static void RemoveSpeechRecognitionListeren(ISpeechRecognitionListener listener)
public static void AddSpeechRecognitionListerenJs(ISpeechRecognitionListenerJs listener)
public static void RemoveSpeechRecognitionListerenJs(ISpeechRecognitionListenerJs listener)
public static void SetTouchToListenZone(Rect rect, bool isUISpace){
public static void SetTouchToListenEnabled(bool enabled): watch out: enabling touchToListen will disable the autoRestart functionality
public static bool GetTouchToListenEnabled()
public static bool IsRecognitionAvailable()
public static SpeechDictionary GetSpeechDictionary()
public static bool CommandRecognized(string commandName)
public static void StartListening()
public static void StopListening()

When you are already using a custom AndroidManifest.xml file, make sure that the next permissions are in your manifest.

<uses-permission android:name=”android.permission.WRITE_EXTERNAL_STORAGE” />
<uses-permission android:name=”android.permission.RECORD_AUDIO” />
<uses-permission android:name=”android.permission.INTERNET” />

2 thoughts on “Android Speech Recognition for Unity3D”

  1. john grant said:

    hi where can i download this plugin?

  2. Hi John,

    Thanks for showing interest in this plugin.

    I tried a submission for the Unity Asset Store a few months ago. But something must have gone wrong with the submission.

    I just re-submitted the plugin, if all goes well it should be on the Asset store in a few weeks. It will cost $10.

    If you need it earlier you can contact me on speechrecognizerforunity@gmail.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s