[Scummvm-devel] Questions about sound streaming

[D G Turner]

NEC,
  Please see my responses below...

On 06/11/14 22:35, N. E. C. wrote:
> Hi,
> 
> I am implementing a new engine (still in the experimental stage at
> this point), and I'm having trouble with sound.
> 
> The engine contains a movie player. After decoding and displaying the
> final video frame, I want the player to enter an Ending state where it
> waits for the audio to finish, and then enter the Stopped state when
> the audio is done. However, I there are some things I'm not sure
> about.

To be clear, are you implementing a new format for VideoDecoder API?
https://github.com/scummvm/scummvm/blob/master/video/video_decoder.h

This already provides most of the standard container formats i.e. AVI,
QT etc. and you should check to ensure that you are not making life
hard on yourself by duplicating existing code that you can just use.

The container formats then decode the contents based on the Image
decoder codecs here:
https://github.com/scummvm/scummvm/tree/master/image/codecs

Again, we have decoders for a number of current formats, so you should
check to ensure that you are not again duplicating work and making
things hard on yourself.

Anyway, will answer the rest of your questions assuming that this
is a custom format and not using this.

> 
> Here is an overview of my code.
> 
> When the movie starts, I make a QueuingAudioStream and tell the mixer
> to play it. There are no queued audio buffers yet.
> 
> _audioStream = Audio::makeQueuingAudioStream(22050, false);
> _mixer->playStream(Audio::Mixer::kPlainSoundType, &_audioHandle, _audioStream);
> 
> QUESTION 1: Would it be better to queue sound BEFORE calling playStream?

AFAIK you can do this in either way without any issue. You should check
the code and comments in the audio subsystem code, especially:
https://github.com/scummvm/scummvm/blob/master/audio/mixer.h

> QUESTION 2: The sound is premixed, so it is impossible to control the
> music, speech and SFX volumes separately. Is it appropriate to use
> kPlainSoundType in this situation?

Again, the code is usually the best example so you check a similar
engine in the code to see what that does:
https://github.com/scummvm/scummvm/blob/master/audio/mixer.h#L56

Personally, if this is a cutscene movie, I think we normally use the
speech volume, but unsure.

> QUESTION 3: The volume controls in the global menu seem to have no
> effect. Am I doing something wrong?

I would suspect that you have not implemented syncSoundSettings() ?
See
http://wiki.scummvm.org/index.php/Advanced_Engine_Features#Global_options_dialog_support

> 
> When a sound packet is found, I copy the data into a buffer allocated
> with malloc, and tell the audio stream to queue it.
> 
> // _buf was allocated with malloc and loaded with 8-bit unsigned PCM
> _audioStream->queueBuffer(_buf, _totalSize, DisposeAfterUse::YES,
> Audio::FLAG_UNSIGNED);
> _buf = nullptr; // Audio buffer will be freed by the audio system.

That looks correct yes... as per:
https://github.com/scummvm/scummvm/blob/master/audio/audiostream.h#L329

> After the final sound packet is queued, I tell the stream to finish.
> 
> _audioStream->finish();
> _audioStream = nullptr; // Discard.

Ah... that does not look correct.

_audioStream was allocated by a new in the factory method
Audio::makeQueuingAudioStream():
https://github.com/scummvm/scummvm/blob/master/audio/audiostream.h#L348

The caller has the responsibility to call delete when they are finished
with it... or it will leak.

So IMHO that should be:
  _audioStream->finish();
  delete _audioStream;
  _audioStream = nullptr; // For safety in case anything tries to use
after delete

You can avoid having to do this manually using the DisposeAfterUse::YES
option to _mixer->playStream():
https://github.com/scummvm/scummvm/blob/master/audio/mixer.h#L114

> 
> QUESTION 4: Will the mixer free the audio stream?
> 
> When the player is in the Ending state, it checks whether the audio
> handle is active to know when to enter the Stopped state.
> 
> if (!_mixer->isSoundHandleActive(_audioHandle)) {
>     _audioHandle = Audio::SoundHandle(); // Discard
>     _state = kStopped;
> }

I think I addressed this in the previous question.

I don't think overwriting the _audioHandle in that way is a good idea as
that also looks like a way to create a memory leak...

> 
> QUESTION 5: Is this the correct way to tell whether a sound stream has
> finished playing?

The mixer API is the definitive reference here:
https://github.com/scummvm/scummvm/blob/master/audio/mixer.h#L187

> 
> Thanks for the help!
> 
> 
> P.S. I was about to ask why my player was hanging in the Ending state,
> but I think found the problem while typing this email. Fun, huh?