Nuance Translator SDK

Nuance Translator SDK is a platform for adding sophisticated speech recognition and speech synthesis (text-to-speech) capabilities to your mobile applications.

NuanceMessagingActivity made use of Nuance Translator SDK to provide ASR and TTS functionality to the messaging UI.

This guide covers the API's that can be used to add ASR/TTS functionality to NuanceMessagingActivity.

SDK also comes with WakeUp Word funtionality.

Use below bool properties to enable and configure the translator sdk features.

Name type default description
translator bool <bool name="translator">false</bool>
Use this property to turn on translator functionality.
continuousListening bool <bool name="continuousListening">false</bool>
Lets you configure the way speech recognition starts listening, configuring to true makes the sdk starts listen after every agent message is played.
wakeupWord bool <bool name="wakeupWord">false</bool> Use the property to turn on wake up word.
playTTSOnRestore bool <bool name="playTTSOnRestore">false</bool> Lets you configure how the messages are read upon restore.
playTTSOnlyOnASRInput bool <bool name="playTTSOnlyOnASRInput">false</bool> Lets you configure how the messages are read when keyboard is used instead of speech.
playOpener bool <bool name="playTTSOnlyOnASRInput">true</bool> Lets you configure if opener should be played or not.
playASRSuccessChime bool <bool name="playASRSuccessChime">true</bool> Lets you configure if chime sound should play after successfull listening.
playASRNoInputChime bool <bool name="playASRNoInputChime">true</bool> Lets you configure if chime sound should play after a no input error.
playRecordingChime bool <bool name="playRecordingChime">true</bool> Lets you configure if chime shoud should play before starting listening.
  1. Initializing NinaMobileController
  2. NinaServerConfigurationBuilder
  3. Configuring NinaSetting
  4. NinaSynthesisValuesBuilder
  5. SessionValuesBuilder
  6. RecognitionValuesBuilder
  7. WakeUp word
  8. Translator API

NinaMobileController singleton class exposes the method that is used to initalize the Translator SDK.

Public Methods

Use the Builder public methods to set the configuration values that are used by the SDK to do processing.

NinaSetting singleton class can be used to configure translator SDK TTS and ASR functionality.

Public Methods

Use the Builder public methods to update the commands that are used to play audio using text as input.

Public Methods

Use the Builder public methods to update the commands that are used to while establishing a session.

Public Methods

Use the Builder public methods to update the commands that are used to recognize speech using audio as input.


					public class Application extends android.app.Application {

    public NuanMessaging chatInstance;
    private RefWatcher refWatcher;

    @Override
    public void onCreate() {
        super.onCreate();
        refWatcher = LeakCanary.install(this);
        chatInstance = NuanMessaging.getInstance();
        chatInstance.initialize(this, "west", "ceapiClientId", "ceapiClientSecret", "bestbrands");
        initializeNinaMobileController();
    }

    private void initializeNinaMobileController() {
        NinaServerConfiguration ninaServerConfiguration = new NinaServerConfiguration.NinaServerConfigurationBuilder()
                .setApplicationKey("companyName_appName")
                .setVerificationCode("5796d9e101d5355f5dbf95a3681f6ca5317fd9e96e5ffacac5e1305361a4eb1c")
                .setGateWayScheme("wss")
                .setGateWayAddress("webapi-demo.nuance.mobi")
                .setGateWayPort(443)
                .setGateWayPath("webapi-platform/websocket")
                .build();
        NinaMobileController.getInstance().initialize(getApplicationContext(), ninaServerConfiguration);

        RecognitionValues recognitionValues = new RecognitionValues.RecognitionValuesBuilder()
                .setStartOfSpeechVoicedFrames(0)
                .setStartOfSpeechHistoryFrames(0)
                .build();
        NinaMobileController.getInstance().getNinaSetting().setRecognitionValues(recognitionValues);
        NinaMobileController.getInstance().shouldDebugLog(true);
    }
}


			
Changing default translator action tone
Styling Translator Animation

Properties for changing the appearance of Translator animated icon holder.

Name type default description
msg_mic_permission string <![CDATA[Need Mic permission, go to Setting -> Apps -> <i>Your App</i> -> permission and provide mic permission]]></td> Sets the message that gets diplayed when micro phone permission is denied.
translatorLogBackground color #9000 Sets the translator icon connection status label background color
translatorTextSize dimen 12sp Sets the translator icon connection status label text size.

You can override the below default style classes to customize Translator animated icon Fragment.


			//Translator Container style
<style name="TranslatorContainerDefault.TranslatorContainer></style>



			//Below style class is applied to translator place holder image view, TranslatorPlaceHolder is displayed while initializing translator SDK and if permission id denied
<style name="TranslatorPlaceHolderDefault.TranslatorPlaceHolder"></style>

//Below style class is applied to the translator animatod icon holder
<style name="AnimatedIconDefault.AnimatedIconDefaultHolder"/></style>

//Below style class is applied to translator state label view like connecting and no internet state
<style name="TranslatorStateDefault.TranslatorState"></style >

//procesing text style
<style name="TranslatorProcessingContainerDefault.TranslatorProcessingContainer"></style >



			
Persona Animation

Persona Animation for various states are configured through a JSON file, Application can override the JSON config and customize according to App needs.

Refer the below PDF to understand the high level items in the json config. Example JSON config and images can be found in Example11.


	 //Example to change the translator persona layout to center
	 <style name="TranslatorContainerDefault.TranslatorContainer">
        <item name="android:layout_centerVertical">true</item>
        <item name="android:layout_centerHorizontal">true</item>
        <item name="android:layout_height">75dp</item>
        <item name="android:layout_width">80dp</item>
        <item name="android:layout_alignParentLeft">false</item>
        <item name="android:layout_alignParentTop">true</item>
        <item name="android:layout_gravity">center_horizontal</item>
        <item name="android:layout_above">@null</item>
    </style>

    <style name="MessagingInputContainerDefault.MessagingInputContainer">
        <item name="android:layout_height">75dp</item>
        <item name="android:layout_alignParentBottom">true</item>
        <item name="android:layout_toRightOf">@null</item>
    </style>

    <style name="TranslatorProcessingContainerDefault.TranslatorProcessingContainer">
        <item name="android:layout_below">@+id/translator_container</item>
        <item name="android:layout_toRightOf">@null</item>
        <item name="android:padding">0dp</item>
    </style>

    <style name="MessagingFooterDefault.MessagingFooter">
        <item name="android:layout_height">150dp
    </style>

Configuring WakeUp Word

WakeUp Word functionality lets customer to wake up translator engine and immediately starts listening.

To configure the wake up word phrase overide the default String Array resource.

Translator API

SDK exposes API which allows Application to programatically control the state of Translator .

Following API methods from NinaMobileController.getInstance().getObserver() lets application to registar listeners for various Translator events.

Following API methods from NinaMobileController.getInstance() lets application to trigger Translator functionality.