Android7.1 Offload模式下的音频数据抽取过程

这是一个典型的生产者和消费者的模式
1. 消费者获取数据：

流程代码：

opt Loop
AudioTrack->+MediaPlayerService:mCbf(EVENT_MORE_DATA,...)
Note left of AudioTrack:AudioTrackThread:processAudioBuffer
Note left of AudioTrack:AudioTrack::EVENT_MORE_DATA
    Note left of MediaPlayerService:AudioOutput::CallbackWrapper
    MediaPlayerService->+NuPlayerRenderer: (*me->mCallback)(...,CB_EVENT_FILL_BUFFER)
    Note left of NuPlayerRenderer:AudioSinkCallback
    Note left of NuPlayerRenderer:获取mAudioQueue队列中的音频数据
    opt fillAudioBuffer  
    NuPlayerRenderer->NuPlayerDecoderPassThrough:mNotifyConsumed->post()
    NuPlayerDecoderPassThrough-->NuPlayerDecoderPassThrough:kWhatBufferConsumed
    NuPlayerDecoderPassThrough-->-NuPlayerRenderer:
    end
    opt onBufferConsumed
    NuPlayerDecoderPassThrough->NuPlayerDecoderPassThrough:onRequestInputBuffers
    end
    Note right of NuPlayerDecoderPassThrough:拿走数据后，删除队列元素，使得生产者持续生产数据

    
MediaPlayerService-->-AudioTrack:
end

2. 生产者生产数据

流程代码：

opt doRequestBuffers
opt fetchInputData
    DecoderPassThrough->GenericSource:dequeueAccessUnit
        GenericSource->GenericSource:postReadBuffer
        GenericSource->GenericSource:onReadBuffer
        opt readBuffer
        GenericSource->MP3Source(MP3Extractor):read 
        MP3Source(MP3Extractor)->FileSource:readAt
        FileSource->FileSource: :: read
        end
end

opt onInputBufferFetched
    DecoderPassThrough->NuPlayerRenderer:queueBuffer
        NuPlayerRenderer-->NuPlayerRenderer:kWhatQueueBuffer
    NuPlayerRenderer-->DecoderPassThrough:
end

opt onQueueBuffer
    note left of NuPlayerRenderer:音频数据入列
    NuPlayerRenderer->NuPlayerRenderer:mAudioQueue.push_back(entry);
end
end

3. 完整的代码流程如下：
针对offload模式的AudioTrack，在构建的时候会new一个抽取音频数据的线程如下：

AudioTrack.cpp (frameworks\av\media\libmedia)

AudioTrack::set
    if (cbf != NULL) {
        mAudioTrackThread = new AudioTrackThread(*this, threadCanCallJava);
        mAudioTrackThread->run("AudioTrack", ANDROID_PRIORITY_AUDIO, 0 /*stack*/);
        // thread begins in paused state, and will not reference us until start()
    }

如上会创建一个AudioTrackThread的线程，这个线程里面会调用threadLoop进入循环

threadLoop：
	mReceiver.processAudioBuffer();

processAudioBuffer这个函数是关键

AudioTrack::processAudioBuffer:
	//从AudioFinger获取共享内存的地址，之后我们把音频数据拷贝到这个地址上，随后通知 AudioFinger去播放
	obtainBuffer(&audioBuffer, requested, NULL, &nonContig);
	//向Nuplayer播放器索要音频数据
	mCbf(EVENT_MORE_DATA, mUserData, &audioBuffer);

mCbf是在AudioTrack构造的时候(AudioSink->AudioPort :: Open)传递进来的，对应下面：

MediaPlayerService.cpp (frameworks\av\media\libmediaplayerservice)

MediaPlayerService::AudioOutput::CallbackWrapper:
	case AudioTrack::EVENT_MORE_DATA: {
		size_t actualSize = (*me->mCallback)(
	                me, buffer->raw, buffer->size, me->mCallbackCookie,
	                CB_EVENT_FILL_BUFFER);

NuPlayerRenderer.cpp (frameworks\av\media\libmediaplayerservice\nuplayer)

NuPlayer::Renderer::AudioSinkCallback:
	NuPlayer::Renderer *me = (NuPlayer::Renderer *)cookie;
	switch (event) {
	        case MediaPlayerBase::AudioSink::CB_EVENT_FILL_BUFFER:
	        {	
	        	//从NuPlayer的Render要数据去
	            return me->fillAudioBuffer(buffer, size);
	            break;
	        }

ok，我们接着看音频渲染器是如何为AudioTrack填充数据的

NuPlayer::Renderer::fillAudioBuffer:
	...
	while (sizeCopied < size && !mAudioQueue.empty()) {
		//从AudioQueue中抽取数据，然后拷贝到共享内存中
		entry = &*mAudioQueue.begin();
		memcpy((char *)buffer + sizeCopied,
               entry->mBuffer->data() + entry->mOffset,
               copy);
       if (entry->mOffset == entry->mBuffer->size()) {
       		//通知生产者，生产更多的数据
       		entry->mNotifyConsumed->post(); 
       		//从队列中删除这条数据
       		mAudioQueue.erase(mAudioQueue.begin());
       	}
	...

那么问题的关键在于是谁在什么时候将数据插入到mAudioQueue中的，就在NuPlayer播放器Start的时候(instantiateDecoder->(*decoder)->configure(format))

instantiateDecoder
	NuPlayer::DecoderPassThrough::onConfigure
		DecoderBase::onRequestInputBuffers
			DecoderBase::doRequestBuffers(纯虚函数，找实现类)
    			DecoderPassThrough::fetchInputData
    				DecoderPassThrough::dequeueAccessUnit
    					NuPlayer::GenericSource::dequeueAccessUnit
							GenericSource::postReadBuffer
								GenericSource::onReadBuffer
									GenericSource ::readBuffer (GenericSource.cpp)
										MP3Source::read (MP3Extractor.cpp)

如上是整个音频数据的抽取过程 ，当DecoderPassThrough::fetchInputData获取到数据后，我们看下随后的处理

NuPlayerDecoderPassThrough.cpp (frameworks\av\media\libmediaplayerservice\nuplayer)

bool NuPlayer::DecoderPassThrough::doRequestBuffers() {
  status_t err = OK;
  while (!isDoneFetching()) {
      sp<AMessage> msg = new AMessage();

      err = fetchInputData(msg);
      if (err != OK) {
          break;
      }

      onInputBufferFetched(msg);
  }

  return err == -EWOULDBLOCK
          && mSource->feedMoreTSData() == OK;
}

很显然，offload的Decode拿到源数据后就会直接调用onInputBufferFetched方法

NuPlayer::DecoderPassThrough::onInputBufferFetched:
	bool hasBuffer = msg->findBuffer("buffer", &buffer);
	//这里的reply要特别留意，当AudioTrack取走数据后，会通过这条消息让生产者持续生产数据
	sp<AMessage> reply = new AMessage(kWhatBufferConsumed, this);
	reply->setInt32("generation", mBufferGeneration);
	reply->setInt32("size", bufferSize);
	//让buffer入列
	mRenderer->queueBuffer(true /* audio */, buffer, reply);

NuPlayerRenderer.cpp (frameworks\av\media\libmediaplayerservice\nuplayer)

void NuPlayer::Renderer::queueBuffer(
        bool audio,
        const sp<ABuffer> &buffer,
        const sp<AMessage> &notifyConsumed) {
    sp<AMessage> msg = new AMessage(kWhatQueueBuffer, this);
    msg->setInt32("queueGeneration", getQueueGeneration(audio));
    msg->setInt32("audio", static_cast<int32_t>(audio));
    //留意下面这2个地方，很关键，一个是传递buffer，一个传送callback的回调
    msg->setBuffer("buffer", buffer);
    msg->setMessage("notifyConsumed", notifyConsumed);
    msg->post();
}

Looper线程的处理如下：

NuPlayer::Renderer::onMessageReceived:
        case kWhatQueueBuffer:
        {
            onQueueBuffer(msg);
            break;
        }

NuPlayerRenderer.h (frameworks\av\media\libmediaplayerservice\nuplayer)

NuPlayer::Renderer::onQueueBuffer:
	QueueEntry entry;
    entry.mBuffer = buffer;
    entry.mNotifyConsumed = notifyConsumed;
    entry.mOffset = 0;
    entry.mFinalResult = OK;
    entry.mBufferOrdinal = ++mTotalBuffersQueued;
	if (audio) {
        Mutex::Autolock autoLock(mLock);
        //千转白回，音频数据终于入列了
        mAudioQueue.push_back(entry);
        //下面这个仅对非offload(软解)的情况才有意义
        postDrainAudioQueue_l();
    }
    ...

文章的最后，我们再看下这个状态机是如何源源不断的转起来的，前文提到AudioTrack拿到数据后会调用entry->mNotifyConsumed->post(); 这里的mNotifyConsumed就是对应的new AMessage(kWhatBufferConsumed, this);这儿不赘述原因了，上面很清晰的说明了来源，之后代码会Call到

NuPlayerDecoderPassThrough.cpp (frameworks\av\media\libmediaplayerservice\nuplayer)

NuPlayer::DecoderPassThrough::onMessageReceived：
	case kWhatBufferConsumed:
		onBufferConsumed(size);

看下onBufferConsumed

void NuPlayer::DecoderPassThrough::onBufferConsumed(int32_t size) {
    --mPendingBuffersToDrain;
    mCachedBytes -= size;
    ALOGV("onBufferConsumed: #ToDrain = %zu, cachedBytes = %zu",
            mPendingBuffersToDrain, mCachedBytes);
    ();
}

如上，将mCachedBytes 自减，因为系统只会缓存kMaxCachedBytes = 200000这么大的数据，更多的会阻塞。随后调用onRequestInputBuffers再次从MP3Source读取源数据