音视频系列
什么是音视频的分离和合成
分离就是将视频1的声音和图像分别取出来
合成就是将视频1的图像和非视频1的声音组合成一个新的视频
如何进行音视频的分离和合成
安卓提供了两个API来帮助我们完成这个操作
MediaExtractor用于分离视频
MediaMuxer用于合成视频
下面我就来介绍一下这两个API的使用
MediaExtractor
分离音频
1.设置音频源
MediaExtractor audioExtractor = new MediaExtractor();
audioExtractor.setDataSource(audioPath);
2.获取源文件中轨道的数量，并遍历找到我们需要的音频轨
for (int i = 0; i < audioExtractor.getTrackCount(); i++) {
MediaFormat format = audioExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) {
srcATrackIndex = i;
audioTrackIndex = muxer.addTrack(format);
break;
}
}
分离视频
1.设置视频源
MediaExtractor videoExtractor = new MediaExtractor();
videoExtractor.setDataSource(videoPath);
2.获取视频源文件中的轨道数，并遍历找到我们所需要的视频轨
for (int i = 0; i < videoExtractor.getTrackCount(); i++) {
MediaFormat format = videoExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) {
srcVTrackIndex = i;
videoTrackIndex = muxer.addTrack(format);
break;
}
}
音频和视频额的分离方法一模一样，区别在于MediaFormat类型的不同，MediaFormat封装了媒体的数据格式信息
MediaMuxer
如何合成视频？
1.设置合成后视频的路径和格式
MediaMuxer muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
2.将MediaExtractor分离出来的音轨和视轨添加到自己的轨道中
audioTrackIndex = muxer.addTrack(format);
videoTrackIndex = muxer.addTrack(format);
3.添加完所有轨道后start
muxer.start();
4.采集音频源的音轨样本
audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上
if (audioTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(100 * 1024);
while (true) {
int sampleSize = audioExtractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
//没有可获取的样本，退出循环
break;
}
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = audioExtractor.getSampleTime();
muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道
audioExtractor.advance(); //进入下一个样本
}
}
5.采集视频源的视频轨样本
videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上
if (videoTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(100 * 1024);
while (true) {
int sampleSize = videoExtractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
//没有可获取的样本，退出循环
break;
}
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = videoExtractor.getSampleTime();
muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道
videoExtractor.advance(); //进入下一个样本
}
}
6.停止并释放资源
muxer.stop();
muxer.release();
完整代码
public class MediaUtil {
private static final String TAG = "MediaUtil";
/**
* @param audioPath 音频文件路劲
* @param videoPath 视频文件路径
* @param outPath 合成之后的保存路径
*/
public static void combineVideo(String audioPath, String videoPath, String outPath) {
MediaMuxer muxer = null;
MediaExtractor audioExtractor = null;
MediaExtractor videoExtractor = null;
try {
muxer = new MediaMuxer(outPath, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
//找到音频文件的音频轨
audioExtractor = new MediaExtractor();
audioExtractor.setDataSource(audioPath);
int srcATrackIndex = -1; //音频源的音频轨
int audioTrackIndex = -1; //音频轨添加到muxer后返回的新的轨道
//在此循环，目的是找到我们需要的音频轨
for (int i = 0; i < audioExtractor.getTrackCount(); i++) {
MediaFormat format = audioExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("audio/")) {
srcATrackIndex = i;
audioTrackIndex = muxer.addTrack(format);
break;
}
}
Log.d(TAG,"音频轨源索引："+srcATrackIndex+" 音频轨新索引:"+audioTrackIndex);
//找到视频文件的视频轨
videoExtractor = new MediaExtractor();
videoExtractor.setDataSource(videoPath);
int srcVTrackIndex = -1; //视频源的视频轨
int videoTrackIndex = -1; //视频轨添加到muxer后返回的新的轨道
for (int i = 0; i < videoExtractor.getTrackCount(); i++) {
MediaFormat format = videoExtractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")) {
srcVTrackIndex = i;
videoTrackIndex = muxer.addTrack(format);
break;
}
}
Log.d(TAG,"视频轨源索引："+srcVTrackIndex+" 视频频轨新索引:"+videoTrackIndex);
//添加完所有轨道后start
muxer.start();
Log.d(TAG,"开始合成视频...");
//封装音频track
audioExtractor.selectTrack(srcATrackIndex); //移动到音频轨上
if (audioTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(100 * 1024);
while (true) {
int sampleSize = audioExtractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
//没有可获取的样本，退出循环
break;
}
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = audioExtractor.getSampleTime();
muxer.writeSampleData(audioTrackIndex, buffer, info);//将样本写入新的轨道
audioExtractor.advance(); //进入下一个样本
}
}
Log.d(TAG,"音频轨样本采集完成");
//封装视频track
videoExtractor.selectTrack(srcVTrackIndex); //移动到视频轨上
if (videoTrackIndex != -1) {
MediaCodec.BufferInfo info = new MediaCodec.BufferInfo();
info.presentationTimeUs = 0;
ByteBuffer buffer = ByteBuffer.allocate(100 * 1024);
while (true) {
int sampleSize = videoExtractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
//没有可获取的样本，退出循环
break;
}
info.offset = 0;
info.size = sampleSize;
info.flags = MediaCodec.BUFFER_FLAG_SYNC_FRAME;
info.presentationTimeUs = videoExtractor.getSampleTime();
muxer.writeSampleData(videoTrackIndex, buffer, info);//将样本写入新的轨道
videoExtractor.advance(); //进入下一个样本
}
}
muxer.stop();
Log.d(TAG,"视频轨样本采集完成");
Log.d(TAG,"视频合成完毕："+outPath);
} catch (IOException e) {
e.printStackTrace();
Log.d(TAG,"合成出错:"+e.getMessage());
} finally {
//释放资源
if(audioExtractor!=null){
audioExtractor.release();
}
if(videoExtractor!=null){
videoExtractor.release();
}
if(muxer!=null){
muxer.release();
}
}
}
}
踩坑总结
问题1
MPEG4Writer: Unsupported mime 'audio/mpeg'
当音频文件是mp3时会报这个错误 ,需要的格式是AAC，m4a
stackoverflow
我将两个源文件都替换了mp4格式后不再报错了
问题2
WVMExtractor: Failed to open libwvm.so: dlopen failed: library "libwvm.so" not found
小米5 6.0.1机型上出现的问题，意思是缺少so库
由于我就是这款手机，也没测过其他手机有木有这个so库
后续我会用其他手机在测试这个问题
代码地址
MediaUtil类的封装放置在libplayer下的util包下
方法调用放置在app/demo/media/track下