RK3568 Android 11 蓝牙BluetoothA2dpSink 获取用于生成频谱的PCM

Android 中的 A2DP Sink

A2DP Sink 在 Android 系统中主要用于 接收 其他蓝牙设备（如手机、平板、电脑等）发送过来的 高质量的立体声音频。简单来说，它让你的 Android 设备可以充当一个 蓝牙音箱 或 耳机 的角色。

核心功能：

• 接收音频流： 通过蓝牙协议接收来自其他设备的音频数据。
• 解码音频： 将接收到的音频数据解码成可播放的音频格式。
• 播放音频： 通过设备的扬声器或耳机输出解码后的音频。

应用场景：

• 无线音箱： 将 Android 设备连接到蓝牙音箱，实现无线音乐播放。
• 车载蓝牙： 将手机连接到车载蓝牙系统，通过车载音响播放音乐。
• 蓝牙耳机： 将 Android 设备连接到蓝牙耳机，进行通话或听音乐。

技术实现：

• BluetoothA2dpSink： Android 提供了 BluetoothA2dpSink 类来实现 A2DP Sink 功能。开发者可以通过这个类来管理 A2DP 连接、控制音频播放等。
• 蓝牙配置文件： A2DP（Advanced Audio Distribution Profile）是一种蓝牙配置文件，专门用于高质量立体声音频的无线传输。

如何获取音频数据并生成音频频谱?

什么是音乐频谱?
音乐频谱是声音频率的分布图。声音是由不同频率的声波组成的，这些声波的振幅（强度）不同，就形成了不同的音色。频谱图就是将这些频率和振幅的关系用图形表示出来。

频谱图的组成

• 横轴： 表示频率，通常以赫兹（Hz）为单位。频率越高，音调越高。
• 纵轴： 表示振幅，也就是声音的强度。振幅越大，声音越响。
• 颜色或灰度： 表示不同频率的振幅大小。颜色越深或灰度越高，表示该频率的振幅越大。

频谱图的种类

• 线性频谱图： 频率轴按线性比例分布，适用于分析整个音频频段。
• 对数频谱图： 频率轴按对数比例分布，更适合显示低频部分的细节，常用于音频分析。
• 时频图： 显示声音频率随时间的变化情况，可以直观地看到声音的动态变化。
  

总的来说

音乐频谱是了解声音的重要工具，它不仅能帮助我们更好地理解声音的本质，还能在音乐创作、音频处理等领域发挥重要作用。

---

在蓝牙音箱的模式下, 如何生成音频频谱?

    在打上RK提供的A2dpSink补丁后, 手机等设备可以通过蓝牙连接播放音乐, RK3568充当蓝牙音箱的角色. 在这种状态下, 系统播放音频并不是采用android上层的MediaPlayer 或 AudioTrack, 所以无法采用常规的方式来生成, 若需要获取播放器的音频频谱, 首先, 需要获得音频的PCM数据.

在蓝牙音箱模式下, 音频的播放器的位置处于android 源码的 system目录下

system/bt/btif/src/btif_avrcp_audio_track.cc

/*
 * Copyright 2015 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

#define LOG_NDEBUG 1
#define LOG_TAG "bt_btif_avrcp_audio_track"

#include "btif_avrcp_audio_track.h"

#include 
#include 
#include 

#include "bt_target.h"
#include "osi/include/log.h"

using namespace android;

typedef struct {
  AAudioStream* stream;
  int bitsPerSample;
  int channelCount;
  float* buffer;
  size_t bufferLength;
} BtifAvrcpAudioTrack;

#if (DUMP_PCM_DATA == TRUE)
FILE* outputPcmSampleFile;
char outputFilename[50] = "/data/misc/bluedroid/output_sample.pcm";
#endif

void* BtifAvrcpAudioTrackCreate(int trackFreq, int bitsPerSample,
                                int channelCount) {
  LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btCreateTrack freq %d bps %d channel %d ",
              __func__, trackFreq, bitsPerSample, channelCount);

  AAudioStreamBuilder* builder;
  AAudioStream* stream;
  aaudio_result_t result = AAudio_createStreamBuilder(&builder);
  AAudioStreamBuilder_setSampleRate(builder, trackFreq);
  AAudioStreamBuilder_setFormat(builder, AAUDIO_FORMAT_PCM_FLOAT);
  AAudioStreamBuilder_setChannelCount(builder, channelCount);
  AAudioStreamBuilder_setSessionId(builder, AAUDIO_SESSION_ID_ALLOCATE);
  AAudioStreamBuilder_setPerformanceMode(builder,
                                         AAUDIO_PERFORMANCE_MODE_LOW_LATENCY);
  result = AAudioStreamBuilder_openStream(builder, &stream);
  CHECK(result == AAUDIO_OK);
  AAudioStreamBuilder_delete(builder);

  BtifAvrcpAudioTrack* trackHolder = new BtifAvrcpAudioTrack;
  CHECK(trackHolder != NULL);
  trackHolder->stream = stream;
  trackHolder->bitsPerSample = bitsPerSample;
  trackHolder->channelCount = channelCount;
  trackHolder->bufferLength =
      trackHolder->channelCount * AAudioStream_getBufferSizeInFrames(stream);
  trackHolder->buffer = new float[trackHolder->bufferLength]();

#if (DUMP_PCM_DATA == TRUE)
  outputPcmSampleFile = fopen(outputFilename, "ab");
#endif
  return (void*)trackHolder;
}

void BtifAvrcpAudioTrackStart(void* handle) {
  if (handle == NULL) {
    LOG_ERROR(LOG_TAG, "%s: handle is null!", __func__);
    return;
  }
  BtifAvrcpAudioTrack* trackHolder = static_cast<BtifAvrcpAudioTrack*>(handle);
  CHECK(trackHolder != NULL);
  CHECK(trackHolder->stream != NULL);
  LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btStartTrack", __func__);
  AAudioStream_requestStart(trackHolder->stream);
}

void BtifAvrcpAudioTrackStop(void* handle) {
  if (handle == NULL) {
    LOG_DEBUG(LOG_TAG, "%s handle is null.", __func__);
    return;
  }
  BtifAvrcpAudioTrack* trackHolder = static_cast<BtifAvrcpAudioTrack*>(handle);
  if (trackHolder != NULL && trackHolder->stream != NULL) {
    LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btStartTrack", __func__);
    AAudioStream_requestStop(trackHolder->stream);
  }
}

void BtifAvrcpAudioTrackDelete(void* handle) {
  if (handle == NULL) {
    LOG_DEBUG(LOG_TAG, "%s handle is null.", __func__);
    return;
  }
  BtifAvrcpAudioTrack* trackHolder = static_cast<BtifAvrcpAudioTrack*>(handle);
  if (trackHolder != NULL && trackHolder->stream != NULL) {
    LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btStartTrack", __func__);
    AAudioStream_close(trackHolder->stream);
    delete trackHolder->buffer;
    delete trackHolder;
  }

#if (DUMP_PCM_DATA == TRUE)
  if (outputPcmSampleFile) {
    fclose(outputPcmSampleFile);
  }
  outputPcmSampleFile = NULL;
#endif
}

void BtifAvrcpAudioTrackPause(void* handle) {
  if (handle == NULL) {
    LOG_DEBUG(LOG_TAG, "%s handle is null.", __func__);
    return;
  }
  BtifAvrcpAudioTrack* trackHolder = static_cast<BtifAvrcpAudioTrack*>(handle);
  if (trackHolder != NULL && trackHolder->stream != NULL) {
    LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btPauseTrack", __func__);
    AAudioStream_requestPause(trackHolder->stream);
    AAudioStream_requestFlush(trackHolder->stream);
  }
}

void BtifAvrcpSetAudioTrackGain(void* handle, float gain) {
  if (handle == NULL) {
    LOG_DEBUG(LOG_TAG, "%s handle is null.", __func__);
    return;
  }
  // Does nothing right now
}

constexpr float kScaleQ15ToFloat = 1.0f / 32768.0f;
constexpr float kScaleQ23ToFloat = 1.0f / 8388608.0f;
constexpr float kScaleQ31ToFloat = 1.0f / 2147483648.0f;

static size_t sampleSizeFor(BtifAvrcpAudioTrack* trackHolder) {
  return trackHolder->bitsPerSample / 8;
}

static size_t transcodeQ15ToFloat(uint8_t* buffer, size_t length,
                                  BtifAvrcpAudioTrack* trackHolder) {
  size_t sampleSize = sampleSizeFor(trackHolder);
  size_t i = 0;
  for (; i <= length / sampleSize; i++) {
    trackHolder->buffer[i] = ((int16_t*)buffer)[i] * kScaleQ15ToFloat;
  }
  return i * sampleSize;
}

static size_t transcodeQ23ToFloat(uint8_t* buffer, size_t length,
                                  BtifAvrcpAudioTrack* trackHolder) {
  size_t sampleSize = sampleSizeFor(trackHolder);
  size_t i = 0;
  for (; i <= length / sampleSize; i++) {
    size_t offset = i * sampleSize;
    int32_t sample = *((int32_t*)(buffer + offset - 1)) & 0x00FFFFFF;
    trackHolder->buffer[i] = sample * kScaleQ23ToFloat;
  }
  return i * sampleSize;
}

static size_t transcodeQ31ToFloat(uint8_t* buffer, size_t length,
                                  BtifAvrcpAudioTrack* trackHolder) {
  size_t sampleSize = sampleSizeFor(trackHolder);
  size_t i = 0;
  for (; i <= length / sampleSize; i++) {
    trackHolder->buffer[i] = ((int32_t*)buffer)[i] * kScaleQ31ToFloat;
  }
  return i * sampleSize;
}

static size_t transcodeToPcmFloat(uint8_t* buffer, size_t length,
                                  BtifAvrcpAudioTrack* trackHolder) {
  switch (trackHolder->bitsPerSample) {
    case 16:
      return transcodeQ15ToFloat(buffer, length, trackHolder);
    case 24:
      return transcodeQ23ToFloat(buffer, length, trackHolder);
    case 32:
      return transcodeQ31ToFloat(buffer, length, trackHolder);
  }
  return -1;
}

constexpr int64_t kTimeoutNanos = 100 * 1000 * 1000;  // 100 ms

int BtifAvrcpAudioTrackWriteData(void* handle, void* audioBuffer,
                                 int bufferLength) {
  BtifAvrcpAudioTrack* trackHolder = static_cast<BtifAvrcpAudioTrack*>(handle);
  CHECK(trackHolder != NULL);
  CHECK(trackHolder->stream != NULL);
  aaudio_result_t retval = -1;
  //return 0;
#if (DUMP_PCM_DATA == TRUE)
  if (outputPcmSampleFile) {
    fwrite((audioBuffer), 1, (size_t)bufferLength, outputPcmSampleFile);
  }
#endif

  size_t sampleSize = sampleSizeFor(trackHolder);
  int transcodedCount = 0;
  do {
    transcodedCount +=
        transcodeToPcmFloat(((uint8_t*)audioBuffer) + transcodedCount,
                            bufferLength - transcodedCount, trackHolder);

    retval = AAudioStream_write(
        trackHolder->stream, trackHolder->buffer,
        transcodedCount / (sampleSize * trackHolder->channelCount),
        kTimeoutNanos);
    LOG_VERBOSE(LOG_TAG, "%s Track.cpp: btWriteData len = %d ret = %d",
                __func__, bufferLength, retval);
  } while (transcodedCount < bufferLength);

  return transcodedCount;
}

BtifAvrcpAudioTrackWriteData 函数中可以把PCM数据取出来用, 可以打开 DUMP_PCM_DATA 把蓝牙音频播放的PCM内容保存到本地文件char outputFilename[50] = "/data/misc/bluedroid/output_sample.pcm";中, 把文件拿出来用工具打包成WAV格式, 测试音频数据的正确性!

拿到PCM数据后, 通过算法, 便可以轻松实现音频频谱功能.

参考

• Android 音频可视化：频谱特效的探索与实践
• android获取和展示音乐的频谱
• [RK3566-Android11] 关于 a2dpsink -蓝牙支持接收播放/无PIN码连接
• Android 音频可视化