Android音视频开发入门（五）
任务目标
使用MediaExtractor和MediaMuxer API解析和封装Mp4文件
MediaExtractor
这两个API相对来说内容很少，具体请参考MediaExtractor官方文档。文档给了一个简单的示例，如下：
MediaExtractor extractor = new MediaExtractor();
extractor.setDataSource(...);
int numTracks = extractor.getTrackCount();
for (int i = 0; i < numTracks; ++i) {
MediaFormat format = extractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
if (weAreInterestedInThisTrack) {
extractor.selectTrack(i);
}
}
ByteBuffer inputBuffer = ByteBuffer.allocate(...)
while (extractor.readSampleData(inputBuffer, ...) >= 0) {
int trackIndex = extractor.getSampleTrackIndex();
long presentationTimeUs = extractor.getSampleTime();
...
extractor.advance();
}
extractor.release();
extractor = null;
简单介绍下方法
- setDataSource(…) 设置输入文件的路径，可以是本地也可以是网络文件，如果是网络文件需要网络权限
- getTrackCount() 获取通道数
- getTrackFormat(…) 获取通道格式
- selectTrack(…) 选中追踪通道
- readSampleData(ByteBuffer byteBuf, int offset) 把指定通道的数据按偏移量读取到ByteBuffer中
- getSampleTime() 获取当前时间戳
- advance() 读取下一帧数据
- release() 释放资源
MediaMuxer
MediaMuxer官方文档内容也比较简单，现在支持Mp4、Webp、3GP输出格式，示例如下：
MediaMuxer muxer = new MediaMuxer("temp.mp4", OutputFormat.MUXER_OUTPUT_MPEG_4);
// More often, the MediaFormat will be retrieved from MediaCodec.getOutputFormat()
// or MediaExtractor.getTrackFormat().
MediaFormat audioFormat = new MediaFormat(...);
MediaFormat videoFormat = new MediaFormat(...);
int audioTrackIndex = muxer.addTrack(audioFormat);
int videoTrackIndex = muxer.addTrack(videoFormat);
ByteBuffer inputBuffer = ByteBuffer.allocate(bufferSize);
boolean finished = false;
BufferInfo bufferInfo = new BufferInfo();
muxer.start();
while(!finished) {
// getInputBuffer() will fill the inputBuffer with one frame of encoded
// sample from either MediaCodec or MediaExtractor, set isAudioSample to
// true when the sample is audio data, set up all the fields of bufferInfo,
// and return true if there are no more samples.
finished = getInputBuffer(inputBuffer, isAudioSample, bufferInfo);
if (!finished) {
int currentTrackIndex = isAudioSample ? audioTrackIndex : videoTrackIndex;
muxer.writeSampleData(currentTrackIndex, inputBuffer, bufferInfo);
}
};
muxer.stop();
muxer.release();
- MediaMuxer(String path, int format) 第一个参数是输出路径，第二个是输出格式，支持的格式上面已说明
- addTrack(MediaFormat format) 添加通道，本文使用Extractor.getTrackFormat(int index)来获取MediaFormat
- start() 开始合成，要在addTrack()之后调用
- writeSampleData(int trackIndex, ByteBuffer byteBuf, MediaCodec.BufferInfo bufferInfo) 把ByteBuffer中的数据写入到文件
- stop() 停止合成文件
- release() 释放资源
抽取视频的示例代码
/**
* 抽取视频
*/
private void extractorVideo() {
int videoTrackIndex = -1;
mExtractor = new MediaExtractor();
try {
mExtractor.setDataSource(inputPath);
//获取通道数
int tractCount = mExtractor.getTrackCount();
for (int i = 0; i < tractCount; i++) {
MediaFormat mediaFormat = mExtractor.getTrackFormat(i);
String mimeType = mediaFormat.getString(MediaFormat.KEY_MIME);
if (mimeType.startsWith("video/")) {
videoTrackIndex = i;
break;
}
}
//设置视频通道
mExtractor.selectTrack(videoTrackIndex);
//初始化MediaMuxer
mMuxer = new MediaMuxer(outVideoPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
//获取MediaFormat
MediaFormat videoFormat = mExtractor.getTrackFormat(videoTrackIndex);
//添加通道给muxer
mMuxer.addTrack(videoFormat);
ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
mMuxer.start();
//获取相邻视频帧的时间间隔
long videoSampleTime = getSampleTime(mExtractor, byteBuffer, videoTrackIndex);
MediaCodec.BufferInfo videoInfo = new MediaCodec.BufferInfo();
while (true) {
int readSize = mExtractor.readSampleData(byteBuffer, 0);
if (readSize < 0) {
break;
}
mExtractor.advance();
videoInfo.offset = 0;
videoInfo.size = readSize;
videoInfo.flags = mExtractor.getSampleFlags();
videoInfo.presentationTimeUs += videoSampleTime;
mMuxer.writeSampleData(videoTrackIndex, byteBuffer, videoInfo);
}
mMuxer.stop();
mMuxer.release();
mExtractor.release();
} catch (IOException e) {
e.printStackTrace();
}
}
我发现有人用帧率计算时间，我也试了下，报错了，可能用帧率计算时间不是通用的；还有抽取视频后发现视频文件不是很完整，但是可以播放，不知道具体什么原因，希望知道的留言告知。 另外试过这篇文章的方式，分离后的视频不能正常播放，不知道是不是我操作不当，但是我直接copy的作者代码没有改动过。
抽取音频的实例代码
音频跟视频的抽取是一样的
/**
* 抽取音频
*/
private void extractorAudio() {
int audioTrackIndex = -1;
mExtractor = new MediaExtractor();
try {
//设置输入源
mExtractor.setDataSource(inputPath);
//获取通道数
int trackCount = mExtractor.getTrackCount();
for (int i = 0; i < trackCount; i++) {
MediaFormat mediaFormat = mExtractor.getTrackFormat(i);
String mime = mediaFormat.getString(MediaFormat.KEY_MIME);
if (mime.startsWith("audio/")) {
audioTrackIndex = i;
break;
}
}
//设置音频通道
mExtractor.selectTrack(audioTrackIndex);
MediaFormat audioFormat = mExtractor.getTrackFormat(audioTrackIndex);
mMuxer = new MediaMuxer(outAudioPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
//添加通道
int writeAudio = mMuxer.addTrack(audioFormat);
ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
mMuxer.start();
long audioSampleTime = getSampleTime(mExtractor, byteBuffer, audioTrackIndex);
MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();
while (true) {
int readSize = mExtractor.readSampleData(byteBuffer, 0);
if (readSize < 0) {
break;
}
mExtractor.advance();
audioInfo.offset = 0;
audioInfo.size = readSize;
audioInfo.flags = mExtractor.getSampleFlags();
audioInfo.presentationTimeUs += audioSampleTime;
mMuxer.writeSampleData(writeAudio, byteBuffer, audioInfo);
}
mMuxer.stop();
mMuxer.release();
mExtractor.release();
} catch (IOException e) {
e.printStackTrace();
}
}
用抽取的音视频重新合成一个MP4文件
/**
* 合成视频音频
*/
public void muxerVideoAudio(View view) {
mMuxerStatus.setText("开始合成");
MediaExtractor videoExtractor = new MediaExtractor();
MediaExtractor audioExtractor = new MediaExtractor();
try {
videoExtractor.setDataSource(outVideoPath);
audioExtractor.setDataSource(outAudioPath);
//获取要追踪的通道
int videoTrackIndex = getTrack(videoExtractor, "video/");
int audioTrackIndex = getTrack(audioExtractor, "audio/");
//选中通道
videoExtractor.selectTrack(videoTrackIndex);
audioExtractor.selectTrack(audioTrackIndex);
MediaCodec.BufferInfo videoInfo = new MediaCodec.BufferInfo();
MediaCodec.BufferInfo audioInfo = new MediaCodec.BufferInfo();
//创建缓冲区
ByteBuffer byteBuffer = ByteBuffer.allocate(500 * 1024);
mMuxer = new MediaMuxer("/storage/emulated/0/merge.mp4", MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
//添加通道
int writeVideoTrack = mMuxer.addTrack(videoExtractor.getTrackFormat(videoTrackIndex));
int writeAudioTrack = mMuxer.addTrack(audioExtractor.getTrackFormat(audioTrackIndex));
mMuxer.start();
long videoSampleTime = getSampleTime(videoExtractor, byteBuffer, videoTrackIndex);
//顺序写入数据
while (true) {
int videoSize = videoExtractor.readSampleData(byteBuffer, 0);
if (videoSize < 0) {
break;
}
videoInfo.offset = 0;
videoInfo.size = videoSize;
videoInfo.flags = videoExtractor.getSampleFlags();
videoInfo.presentationTimeUs += videoSampleTime;
mMuxer.writeSampleData(writeVideoTrack, byteBuffer, videoInfo);
videoExtractor.advance();
}
long audioSampleTime = getSampleTime(audioExtractor, byteBuffer, audioTrackIndex);
while (true) {
int audioSize = audioExtractor.readSampleData(byteBuffer, 0);
if (audioSize < 0) {
break;
}
audioInfo.offset = 0;
audioInfo.size = audioSize;
audioInfo.flags = audioExtractor.getSampleFlags();
audioInfo.presentationTimeUs += audioSampleTime;
mMuxer.writeSampleData(writeAudioTrack, byteBuffer, audioInfo);
audioExtractor.advance();
}
SystemClock.sleep(500);
mMuxer.stop();
mMuxer.release();
videoExtractor.release();
audioExtractor.release();
mMuxerStatus.setText("合成完毕");
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* 获取要追踪的信道
* @param extractor {@link MediaExtractor}
* @return track
*/
private int getTrack(MediaExtractor extractor, String mimeType) {
int trackCount = extractor.getTrackCount();
int trackIndex = -1;
for (int i = 0; i < trackCount; i++) {
MediaFormat format = extractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
if (mime.startsWith(mimeType)) {
trackIndex = i;
break;
}
}
return trackIndex;
}
//获取每一帧的时间差
private long getSampleTime(MediaExtractor extractor, ByteBuffer byteBuffer, int trackIndex) {
extractor.readSampleData(byteBuffer, 0);
//跳过I帧，要P帧（视频是由个别I帧和很多P帧组成）h264编码中有IBP帧 I为关键帧。
if (extractor.getSampleFlags() == MediaExtractor.SAMPLE_FLAG_SYNC) {
extractor.advance();//跳过
}
extractor.readSampleData(byteBuffer, 0);
//得到第一个P帧的时间戳
long firstPTS = extractor.getSampleTime();
//下一帧
extractor.advance();
extractor.readSampleData(byteBuffer, 0);
long secondPTS = extractor.getSampleTime();
long sampleTime = Math.abs(secondPTS - firstPTS);
// 重新切换此信道，不然上面跳过了3帧,造成前面的帧数模糊
extractor.unselectTrack(trackIndex);
extractor.selectTrack(trackIndex);
return sampleTime;
}
音视频合成的时候有一点需要注意while循环里有这么一句audioExtractor.advance()，开始我放到if语句后面，运行报错了。可能有一点小问题就会引起java.lang.IllegalStateException: Failed to stop the muxer。报错了不要急，可能就是你的代码跟别人的有一点顺序不太一样。
最后代码已上传GitHub
参考
- Android音视频开发-入门(四):使用 MediaExtractor 和 MediaMuxer API 解析和封装 mp4 文件
- Android 音视频开发(五)：使用 MediaExtractor 和 MediaMuxer API 解析和封装 mp4 文件