android 音频数据在AudioFlinger中的处理（出入口）
AudioFlinger::PlaybackThread::threadLoop()
{
…
if (!waitingAsyncCallback()) {
// sleepTime == 0 means we mustwrite to audio hardware
if (sleepTime == 0) {
if (mBytesRemaining) {
ssize_tret = threadLoop_write();
if (ret < 0) {
mBytesRemaining = 0;
} else {
mBytesWritten += ret;
mBytesRemaining -= ret;
}
} else if ((mMixerStatus ==MIXER_DRAIN_TRACK) ||
(mMixerStatus ==MIXER_DRAIN_ALL)) {
threadLoop_drain();
}
…
}
AudioFlinger 是android 多媒体模块Audio模块的两大服务之一。音频相关的数据必须通过它来传递到底层，所以它就会有一个音频数据的处理过程。这里主要就是分析音频数据从编码器出来之后，怎么流向驱动的。
从audoFlinger的代码中很容易发现，数据写到驱动的处理是在函数
AudioFlinger::PlaybackThread::threadLoop_write()中。这个threadLoop_write 函数在AudioFlinger::PlaybackThread::threadLoop()函数中调用，很显然audioFlinger的数据处理都会在这个线程函数中处理。那么数据的出口是这样实现
那么数据的入口在哪里呢？
通过分析，发现PlaybackThread::threadLoop() 函数主要做三件事，prepareTracks_l()， threadLoop_mix(),threadLoop_write().
prepareTracks_l() 函数主要的工作就是检查是否有track 的状态，并做相应的处理，比如track准备好了，就把它添加到队列中区，还做一些处理比如
// XXX: these things DON'T need to be done each time
mAudioMixer->setBufferProvider(name, track);
mAudioMixer->enable(name);
mAudioMixer->setParameter(name, param, AudioMixer::VOLUME0, (void*)vl);
mAudioMixer->setParameter(name, param, AudioMixer::VOLUME1, (void*)vr);
mAudioMixer->setParameter(name, param, AudioMixer::AUXLEVEL, (void*)va);
mAudioMixer->setParameter(
name,
AudioMixer::TRACK,
AudioMixer::FORMAT, (void*)track->format());
mAudioMixer->setParameter(
name,
AudioMixer::TRACK,
AudioMixer::CHANNEL_MASK, (void*)track->channelMask());
设置参数到AudioMixer。这个很关键，会影响后面混音的结果和处理。
prepareTracks_l() 之后就是混音处理了，处理函数就是threadLoop_mix()。混音，顾名思义，就是将几种声音混到一起，实际上就是将同时处于active状态的track进行混音，每一路音频对应一个track实例。目前android 最多支持32路混音，在类AudioMixer中有相应的定义： static const uint32_tMAX_NUM_TRACKS = 32;
threadLoop_write 是数据的出口，prepareTracks_l只是准备tracks，那么很显然，数据的入口也只有threadLoop_mix 了，混音必须要对数据处理，如果没有数据怎么混音，此时将数据拉进来时最好的时机，事实上也是在threadLoop_mix 的处理过程中数据进入到
AudioFlinger服务中。下面来分析threadLoop_mix。
voidAudioFlinger::MixerThread::threadLoop_mix()
{
// obtain the presentation timestamp of the next output buffer
int64_t pts;
status_t status = INVALID_OPERATION;
if (mNormalSink != 0) {
status = mNormalSink->getNextWriteTimestamp(&pts);
}else {
status = mOutputSink->getNextWriteTimestamp(&pts);
}
if (status != NO_ERROR) {
pts = AudioBufferProvider::kInvalidPTS;
}
// mix buffers...
mAudioMixer->process(pts);
mCurrentWriteLength = mixBufferSize;
if ((sleepTime == 0) && (sleepTimeShift > 0)) {
sleepTimeShift--;
}
sleepTime = 0;
standbyTime = systemTime() + standbyDelay;
//TODO: delay standby when effects have a tail
}从函数中发现，
mAudioMixer->process(pts);就是这个函数的核心，从名字就可以看出来，process就是处理的意思。接着，看看
mAudioMixer->process(pts); 究竟做了什么事情。
void AudioMixer::process(int64_t pts)
{
mState.hook(&mState, pts);
}
只有一句代码，hook 是一个函数指针，指向一个函数实体，那么这个实体函数是哪个呢？这个问题先放一边，感觉比较迷茫，先看看其他的。AudioFlinger的数据处理，主要就是集中在函数bool AudioFlinger::PlaybackThread::threadLoop(),
先看看threadloop_write()函数的具体实现
ssize_t AudioFlinger::PlaybackThread::threadLoop_write()
{
….
if (mNormalSink != 0) {
…
ssize_t framesWritten = mNormalSink->write(mMixBuffer + offset,count);
// otherwise use the HAL / AudioStreamOut directly
}else {
// Direct output and offload threads
…
bytesWritten = mOutput->stream->write(mOutput->stream,
mMixBuffer + offset,mBytesRemaining);
…
}
}
以上代码可以看出，数据写到驱动就是通过上面代码来实现，看见这个就是数据在audioFlinger的出口，很明白，在我看来，数据的入口比较隐晦，写数据的buffer 是mMixBuffer，这个buffer的数据是从哪里来的呢？知道这个buffer的数据来源，应该就可以搞清楚AudioFlinger 的数据入口在哪里了。
mMixBuffer 的空间分配在函数PlaybackThread::readOutputParameters()
voidAudioFlinger::PlaybackThread::readOutputParameters()
{
…
mAllocMixBuffer = new int8_t[mNormalFrameCount * mFrameSize + align -1];
mMixBuffer= (int16_t *) ((((size_t)mAllocMixBuffer + align - 1) / align) * align);
memset(mMixBuffer,0, mNormalFrameCount * mFrameSize);
…
}
查询代码发现，关于mMixBuffer 还有一个函数很关键，在头文件thread.h实现的，
int16_t *mixBuffer() const { return mMixBuffer; };
然后查询这个mixBuffer() 函数究竟被谁，在哪调用了，查找发现
AudioFlinger::PlaybackThread::Track::Track(
PlaybackThread *thread,
const sp<Client>& client,
audio_stream_type_t streamType,
uint32_t sampleRate,
audio_format_t format,
audio_channel_mask_t channelMask,
size_t frameCount,
const sp<IMemory>& sharedBuffer,
int sessionId,
IAudioFlinger::track_flags_t flags)
: TrackBase(thread, client, sampleRate, format,channelMask, frameCount, sharedBuffer,
sessionId, true /*isOut*/),
mFillingUpStatus(FS_INVALID),
// mRetryCount initialized later when needed
mSharedBuffer(sharedBuffer),
mStreamType(streamType),
mName(-1), // see note below
<strong><span style="color:#FF0000;">mMainBuffer(thread->mixBuffer())</span></strong>,
mAuxBuffer(NULL),
mAuxEffectId(0), mHasVolumeController(false),
mPresentationCompleteFrames(0),
mFlags(flags),
mFastIndex(-1),
mCachedVolume(1.0),
mIsInvalid(false),
mAudioTrackServerProxy(NULL),
mResumeToStopping(false)
{
}
在创建track对象的时候初始化track的一个对象mMainBuffer，顾名思义，感觉真相越来越近了，然后再看看track的mMainBuffer怎么样被使用起来的，我们之前已经假设数据的入口是在混音(AudioMixer)的过程中,看看AudioMixer 有什么跟track有关的函数或者变量。
看看AudioMixer类的定义
class AudioMixer
{
public:
AudioMixer(size_tframeCount, uint32_t sampleRate,
uint32_tmaxNumTracks = MAX_NUM_TRACKS);
/*virtual*/ ~AudioMixer(); // non-virtualsaves a v-table, restore if sub-classed
// This mixer has a hard-coded upper limit of 32 active track inputs.
// Adding support for > 32 tracks would require more than simplychanging this value.
static const uint32_t MAX_NUM_TRACKS = 32;
// maximum number of channels supported by the mixer
// This mixer has a hard-coded upper limit of 2 channels for output.
// There is support for > 2 channel tracks down-mixed to 2 channeloutput via a down-mix effect.
// Adding support for > 2 channel output would require more thansimply changing this value.
static const uint32_t MAX_NUM_CHANNELS = 2;
// maximum number of channels supported for the content
static const uint32_t MAX_NUM_CHANNELS_TO_DOWNMIX = 8;
static const uint16_t UNITY_GAIN = 0x1000;
enum { // names
// track names (MAX_NUM_TRACKS units)
TRACK0 = 0x1000,
// 0x2000 is unused
// setParameter targets
TRACK = 0x3000,
RESAMPLE = 0x3001,
RAMP_VOLUME = 0x3002, // rampto new volume
VOLUME = 0x3003, // don'tramp
// set Parameter names
// for target TRACK
CHANNEL_MASK = 0x4000,
FORMAT = 0x4001,
MAIN_BUFFER = 0x4002,
AUX_BUFFER = 0x4003,
DOWNMIX_TYPE = 0X4004,
// for target RESAMPLE
SAMPLE_RATE = 0x4100, //Configure sample rate conversion on this track name;
// parameter'value' is the new sample rate in Hz.
// Onlycreates a sample rate converter the first time that
// the tracksample rate is different from the mix sample rate.
// If the newsample rate is the same as the mix sample rate,
// and asample rate converter already exists,
// then the samplerate converter remains present but is a no-op.
RESET = 0x4101, // Resetsample rate converter without changing sample rate.
// Thisclears out the resampler's input buffer.
REMOVE = 0x4102, //Remove the sample rate converter on this track name;
// the trackis restored to the mix sample rate.
// for target RAMP_VOLUME and VOLUME (8 channels max)
VOLUME0 = 0x4200,
VOLUME1 = 0x4201,
AUXLEVEL = 0x4210,
};
// For all APIs with "name": TRACK0 <= name < TRACK0 +MAX_NUM_TRACKS
// Allocate a track name. Returnsnew track name if successful, -1 on failure.
int getTrackName(audio_channel_mask_t channelMask, int sessionId);
// Free an allocated track by name
void deleteTrackName(intname);
// Enable or disable an allocated track by name
void enable(int name);
void disable(int name);
void setParameter(int name,int target, int param, void *value);
void setBufferProvider(intname, AudioBufferProvider* bufferProvider);
void process(int64_t pts);
uint32_t trackNames() const {return mTrackNames; }
size_t getUnreleasedFrames(int name) const;
private:
enum {
NEEDS_CHANNEL_COUNT__MASK =0x00000007,
NEEDS_FORMAT__MASK =0x000000F0,
NEEDS_MUTE__MASK =0x00000100,
NEEDS_RESAMPLE__MASK =0x00001000,
NEEDS_AUX__MASK =0x00010000,
};
enum {
NEEDS_CHANNEL_1 =0x00000000,
NEEDS_CHANNEL_2 =0x00000001,
NEEDS_FORMAT_16 =0x00000010,
NEEDS_MUTE_DISABLED =0x00000000,
NEEDS_MUTE_ENABLED =0x00000100,
NEEDS_RESAMPLE_DISABLED =0x00000000,
NEEDS_RESAMPLE_ENABLED =0x00001000,
NEEDS_AUX_DISABLED = 0x00000000,
NEEDS_AUX_ENABLED =0x00010000,
};
struct state_t;
struct track_t;
class DownmixerBufferProvider;
typedef void (*hook_t)(track_t* t, int32_t* output, size_t numOutFrames,int32_t* temp,
int32_t* aux);
static const int BLOCKSIZE = 16; // 4 cache lines
<span style="color:#FF0000;">struct track_t</span> {
uint32_t needs;
union {
int16_t volume[MAX_NUM_CHANNELS]; // [0]3.12 fixed point
int32_t volumeRL;
};
int32_t prevVolume[MAX_NUM_CHANNELS];
// 16-byte boundary
int32_t volumeInc[MAX_NUM_CHANNELS];
int32_t auxInc;
int32_t prevAuxLevel;
// 16-byte boundary
int16_t auxLevel; //0 <= auxLevel <= MAX_GAIN_INT, but signed for mul performance
uint16_t frameCount;
uint8_t channelCount; // 1 or 2, redundant with (needs &NEEDS_CHANNEL_COUNT__MASK)
uint8_t format; // always 16
uint16_t enabled; // actually bool
audio_channel_mask_t channelMask;
// actual buffer provider used by the track hooks, seeDownmixerBufferProvider below
// for how the Track bufferprovider is wrapped by another one when dowmixing is required
AudioBufferProvider* bufferProvider;
// 16-byte boundary
mutable AudioBufferProvider::Buffer buffer; // 8 bytes
hook_t hook;
const void* in; //current location in buffer
// 16-byte boundary
AudioResampler* resampler;
uint32_t sampleRate;
<span style="color:#FF0000;"><strong>int32_t* mainBuffer;</strong></span>
int32_t* auxBuffer;
// 16-byte boundary
DownmixerBufferProvider* downmixerBufferProvider; // 4 bytes
int32_t sessionId;
int32_t padding[2];
};
// pad to 32-bytes to fill cache line
struct state_t {
uint32_t enabledTracks;
uint32_t needsChanged;
size_t frameCount;
void (*hook)(state_t*state, int64_t pts); // one ofprocess__*, never NULL
int32_t *outputTemp;
int32_t *resampleTemp;
NBLog::Writer* mLog;
int32_t reserved[1];
// FIXME allocate dynamically to save some memory when maxNumTracks <MAX_NUM_TRACKS
track_t tracks[MAX_NUM_TRACKS]; __attribute__((aligned(32)));
};
以上代码看出，类AudioMixer有个类内类track_t，字面上看应该跟track很有关系，track_t 有个成员变量mainBuffer， 这个变量猜想就是数据处理的buffer，看看哪里调用了。
state_t结构也很重要，他是mixer的主要结构，里面有track_t的对象数组。
voidAudioMixer::process__genericResampling(state_t* state, int64_t pts)
{
int32_t* const outTemp = state->outputTemp;
while (e0) {
}
int32_t *out = t1.mainBuffer;
memset(outTemp,0, size);
while (e1) {
} else {
while (outFrames <numFrames) {
t.buffer.frameCount =numFrames - outFrames;
int64_t outputPTS =calculateOutputPTS(t, pts, outFrames);
t.bufferProvider-><span style="color:#FF0000;"><strong>getNextBuffer</strong></span>(&t.buffer,outputPTS);
t.in = t.buffer.raw;
<span style="color:#FF0000;">t.hook(&t,outTemp + outFrames*MAX_NUM_CHANNELS, t.buffer.frameCount, state->resampleTemp, aux);</span>
outFrames += t.buffer.frameCount;
t.bufferProvider->releaseBuffer(&t.buffer);
}
}
}
<span style="color:#FF0000;">ditherAndClamp(out, outTemp, numFrames);</span>
}
}
以上代码精简了，注意红色标记，t.bufferProvider->getNextBuffer(&t.buffer,outputPTS);这里就是获得数据的地方了，取得的数据再处理，应该就是混音处理的，t.hook，后期再分析hook指针指向哪里，处理之后的数据在outTemp所指向的内存当中，ditherAndClamp(out, outTemp, numFrames); 函数就是将数据再处理，至于这个函数是做什么处理，目前我还没有搞清楚，反正最后等到处理后的数据时out指针指向的内存，登登登….int32_t *out = t1.mainBuffer;这个out指针就是track_t结构的成员mainBuffer，这个mainBuffer跟track类的mMainBuffer有什么关系呢？
看看前面的prepareTracks_l() 函数，有一段处理代码
mAudioMixer->setParameter(
name,
AudioMixer::TRACK,
AudioMixer::MAIN_BUFFER, (void*)track->mainBuffer());
track->mainBuffer()，这个函数不陌生吧，前面有提过。
再看AudioMixer.cpp的
void AudioMixer::setParameter (int name,int target, int param, void *value)
{
int valueInt = (int)value;
<strong><span style="color:#FF0000;">int32_t *valueBuf = (int32_t *)value;</span></strong>
switch (target){
case TRACK:
switch (param) {
case MAIN_BUFFER:
if (track.mainBuffer != valueBuf) {
<span style="color:#FF0000;"><strong>track.mainBuffer= valueBuf</strong></span>;
ALOGV("setParameter(TRACK,MAIN_BUFFER, %p)", valueBuf);
invalidateState(1 <<name);
}
break;
…
}
这下对上了吧，这样就知道往底层驱动写的buffer数据从哪里来了。这样音频数据在audioFlinger的入口也就找到了。
简单再说一下这个过程，在prepareTracks_l() 的时候将active的track的mMainBuffer 赋值到AudioMixer中去，这样混音后的数据就会保存到这个buffer，而mMainBuffer 又是从playbackThread中的mMixBuffer 赋值得来。
回到前面看看，数据是怎么从入口得来的，通过查找代码发现，
t.bufferProvider->getNextBuffer(&t.buffer,outputPTS); 的最终实现是
status_tAudioFlinger::PlaybackThread::Track::getNextBuffer
AudioBufferProvider::Buffer* buffer, int64_t pts)
{
ServerProxy::Buffer buf;
size_t desiredFrames = buffer->frameCount;
buf.mFrameCount = desiredFrames;希望获得的帧数
status_t status = mServerProxy->obtainBuffer(&buf); 从共享内存好获取一段可用的数据
buffer->frameCount = buf.mFrameCount; 实际获取数据返回的帧数
buffer->raw = buf.mRaw;
if (buf.mFrameCount == 0) {
mAudioTrackServerProxy->tallyUnderrunFrames(desiredFrames);
}
return status;
}
在创建audioTrack的时候会创建一个共享内存，getNextBuffer就是从共享内存中获取数据，这个共享内存的写入端在audioTrack，读端在AudioFlinger。下一章再详细写一下，关于AudioTrack的数据到AudioFlinger的处理和audioTrack 共享内存的工作方式
写得比较乱，算是自己的一个总结吧，有空再整理整理。。。