recognize user voice

2019-05-23

let's see how to add the feature to recognize user voice and transform to text by react-native-voice on RN(React Native)

Outline

if you need to recognize user voice and transform to text(STT, Speech To Text), you can use react-native-voice. in this blog, we will see how to add the feature to recognize user voice and transform to text by using react-native-voice.

Installation

install react-native-voice library by executing below command for STT(Speech To Text) feature which recognize user voice and transform to text.

npm install --save react-native-voice

after installing, execute below command to link react-native-voice to RN(React Native) project.

react-native link react-native-voice

Configure Permission

we need to configure the permission to use STT(Speech To Text) feature which recognize user voice and transform to text.

iOS

add below code to ios/app_name/Info.plist file in RN(React Native) project folder to set the permission on iOS.

<dict>
    ...
    <key>NSMicrophoneUsageDescription</key>
    <string>Description of why you require the use of the microphone</string>
    <key>NSSpeechRecognitionUsageDescription</key>
    <string>Description of why you require the use of the speech recognition</string>
    ...
</dict>

Android(Unidentified)

add below contnet to android/app/src/AndroidManifest.xml file in RN(React Native) project folder to set the permission on Android. (Warning: I didn’t check this is OK on Android for the permission yet.)

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="package_name">
    ...
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW"/>
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    ...
</manifest>

How to use

in here, I will use RN(React Native) code basically applied typescript and styled-components. if you want to know how to apply typescript and styled-components to RN(React Native), see my previous blogs.

below is how to use react-native-voice about STT(Speech To Text) feature which recognize user voice and transform to text.

...
export default class App extends React.Component<Props, State> {
  constructor(props: Props) {
    super(props);
    ...
    Voice.onSpeechStart = this._onSpeechStart;
    Voice.onSpeechEnd = this._onSpeechEnd;
    Voice.onSpeechResults = this._onSpeechResults;
    Voice.onSpeechError = this._onSpeechError;
  }
  ...
  componentWillUnmount() {
    Voice.destroy().then(Voice.removeAllListeners);
  }
  ...
  private _onSpeechStart = event => {
    console.log('onSpeechStart');
    this.setState({
      voice: '',
    });
  };
  private _onSpeechEnd = event => {
    console.log('onSpeechEnd');
    ...
  };
  private _onSpeechResults = event => {
    console.log('onSpeechResults');
    this.setState({
      voice: event.value[0],
    });
  };
  private _onSpeechError = event => {
    console.log('_onSpeechError');
    ...
  };

  private _onRecordVoice = () => {
    const { isRecord } = this.state;
    if (isRecord) {
      Voice.stop();
    } else {
      Voice.start('en-US');
    }
    this.setState({
      isRecord: !isRecord,
    });
  };
  ...
}

let’s see above source deeply.

Voice.onSpeechStart = this._onSpeechStart;
Voice.onSpeechEnd = this._onSpeechEnd;
Voice.onSpeechResults = this._onSpeechResults;
Voice.onSpeechError = this._onSpeechError;

connect react-native-voice event functions in constructor.

componentWillUnmount() {
  Voice.destroy().then(Voice.removeAllListeners);
}

when the component which use react-native-voice is unmounted, make react-native-voice also removed.

private _onRecordVoice = () => {
  const { isRecord } = this.state;
  if (isRecord) {
    Voice.stop();
  } else {
    Voice.start('en-US');
  }
  this.setState({
    isRecord: !isRecord,
  });
};

you should bind a specific event to recognize or stop the user voice.

...
private _onSpeechResults = event => {
  console.log('onSpeechResults');
  this.setState({
    voice: event.value[0],
  });
};

if react-native-voice is executed by Voice.start('en-US');, user voice which is recognized by the mic is continuously changed through Voice.onSpeechResults = this._onSpeechResults;.

you can see full source code via below link.

Configure Language

if you want to use STT(Speech To Text) feature which recognize user voice and transform text, you should set what user use language.

Voice.start('ja-JP');

below list is language-country code to configure the language.

ar-SA
cs-CZ
da-DK
de-DE
el-GR
en-AU
en-GB
en-IE
en-US
en-ZA
es-ES
es-MX
fi-FI
fr-CA
fr-FR
he-IL
hi-IN
hu-HU
id-ID
it-IT
ja-JP
ko-KR
nl-BE
nl-NL
no-NO
pl-PL
pt-BR
pt-PT
ro-RO
ru-RU
sk-SK
sv-SE
th-TH
tr-TR
zh-CN
zh-HK
zh-TW

Reference

Buy me a coffeeBuy me a coffee
Posts