Adding Audio

Because the LumenVox Speech Engine is hardware independent, the client application has great flexibility when collecting audio data. Once the audio is acquired, your client application should ensure the data is in a supported audio format, either by collecting it in this format initially or else converting it somehow.

The audio sent to the Speech Engine must be header-less, otherwise known as "raw" audio. For example, the standard Windows .wav files have a header which needs to be removed.

Audio data is stored in a voice channel. Each speech port has 64 different voice channels. This allows 64 different audio data samples to be stored in a speech port at once, although most applications will only need 2: one for the main answer, and one holding the results of a confirmation yes/no question.

Audio may be entered at once, as a batch decode, or it may be streamed in. Batch decodes are generally used for applications where you have audio files saved to disk, while streaming decodes are used when your application is collecting audio and sending it directly to the Engine (most telephony applications will use streaming).

Batched Audio

To get your audio into the port all you have to do is collect your audio into a buffer and call LoadVoiceChannel, supplying the handle of the port, the number of the voice channel you wish to use, a pointer to the audio data, the length of the audio, and the correct sound format.

C Code

  1. void LoadAudio(HPORT hport, void* audio, int audiolength)
  2. {
  3. LV_SRE_LoadVoiceChannel(hport, 1, audio, audiolength, PCM_16KHZ);
  4. }

C++ Code

  1. void LoadAudio(LVSpeechPort &myport, void* audio, int audiolength)
  2. {
  3. myport.LoadVoiceChannel(1, audio, audiolength, PCM_16KHZ);
  4. }

Streaming

In order to stream audio into the speech server, there are several parameters to set, as the Speech Engine must do voice activity detection (VAD), correctly identifying the beginning and end of speech. See Recommended Engine Settings.

If you are having problems with barge in or with the Engine chopping off words at the end of utterances, it is probably because of the way the streaming parameters are set. Please review the Recommended Engine Settings, Sensitivity Settings, and the SetStreamParameter pages.

The code below will set up streaming and set the stream parameters to the most commonly used settings.

C Code

  1. LV_SRE_StreamSetParameter(hport,STREAM_PARM_DETECT_BARGE_IN,1);
  2. LV_SRE_StreamSetParameter(hport,STREAM_PARM_DETECT_END_OF_SPEECH,1);
  3. LV_SRE_StreamSetParameter(hport,STREAM_PARM_AUTO_DECODE,1);
  4. LV_SRE_StreamSetParameter(hport,STREAM_PARM_VOICE_CHANNEL, 1);
  5. LV_SRE_StreamSetParameter(hport,STREAM_PARM_GRAMMAR_SET, LV_ACTIVE_GRAMMAR_SET);

C++ Code

  1. // The port gets opened and initialized.
  2. LVSpeechPort Port;
  3. Port.CreateClient();
  4. // ...
  5.  
  6. // let the port detect beginning and end of speech,
  7. // and handle the speech decoding automatically
  8. port.StreamSetParameter(STREAM_PARM_DETECT_BARGE_IN,1);
  9. port.StreamSetParameter(STREAM_PARM_DETECT_END_OF_SPEECH,1);
  10. port.StreamSetParameter(STREAM_PARM_AUTO_DECODE,1);
  11.  
  12. //pick a voice channel to record audio and send responses to.
  13. port.StreamSetParameter(STREAM_PARM_VOICE_CHANNEL, 1);
  14.  
  15. // If you wish to use your activated SRGS grammars, the grammar set
  16. // must be LV_ACTIVE_GRAMMAR_SET
  17. port.StreamSetParameter(STREAM_PARM_GRAMMAR_SET, LV_ACTIVE_GRAMMAR_SET);

The rest of this example will be in C++. Suppose we have an interface that intermittently provides audio to us. For simplicity, assume it always sends audio in u-Law 8KHz:

  1. typedef bool (*AudioStreamCallback)(char* audio_chunk,
  2. int audio_length,
  3. void* user_data)
  4. class AudioStreamer
  5. {
  6. public:
  7. // non-blocking function.  Sends audio through the callback function
  8. // at regular intervals on a separate thread.  It will stop sending
  9. // audio if the callback returns "false".
  10. void StartStream(AudioStreamCallback cb, void* user_data);
  11.  
  12. // The audio thread will stop sending audio through the callback if
  13. // StopStream is called.  When StopStream returns, the audio thread
  14. // is no longer sending.
  15. void StopStream( );
  16.  
  17. //constructors, destructors, hardware hooks, etc.
  18. //...
  19. };

The speech port also has a callback mechanism for letting the user know what state of processing it is in.

  1. typedef void (*StreamStateChangeFn)(long new_state,
  2. unsigned long total_bytes,
  3. unsigned long recorded_bytes,
  4. void* user_data);

We can connect our speech port and the audio streamer together by way of their callbacks.

  1. struct SimpleRecognizer
  2. {
  3. LVSpeechPort port;
  4. AudioStreamer audio;
  5. };
  6.  
  7. bool AudioCB(char* audio_chunk, int audio_length, void* user_data)
  8. {
  9. SimpleRecognizer* self = (SimpleRecognizer*)user_data;
  10. self->port.StreamSendData(audio_chunk,audio_length);
  11. return true;
  12. }
  13.  
  14. static void PortCB(long new_state, unsigned long total_bytes,
  15. unsigned long recorded_bytes, void* user_data)
  16. {
  17. SimpleRecognizer* self = (SimpleRecognizer*)user_data;
  18. switch (new_state)
  19. {
  20. case STREAM_STATUS_READY:
  21. self->audio.StartStream(AudioCB,self);
  22. break;
  23. case STREAM_STATUS_STOPPED:
  24. case STREAM_STATUS_END_SPEECH:
  25. self->audio.StopStream();
  26. // retrieve answers: we will define this later
  27. break;
  28. case STREAM_STATUS_BARGE_IN:
  29. // stop playing prompt
  30. break;
  31. }
  32. }

Now all that has to happen is to plug the PortCB function into the port.

  1. SimpleRecognizer reco;
  2.  
  3. //initialize the speech port and the audio streamer
  4. //...
  5. //start the stream.
  6. reco.port.StreamSetStateChangeCallBack(PortCB,&reco);
  7. reco.port.StreamSetParameter(STREAM_PARM_SOUND_FORMAT,ULAW_8KHZ);
  8.  
  9. //StreamStart will put the port into the STREAM_STATUS_READY state, which
  10. //will trigger the audio streamer to start sending audio to the port.
  11. reco.port.StreamStart();
© 2012 LumenVox LLC. All rights reserved.