javacv: 图片合成音视频

需求是根据本地图片生成15s的视频 同时插入本地音频

Github
https://github.com/bytedeco/javacv
javacv下载链接
https://www.softpedia.com/get/Programming/Other-Programming-Files/JavaCV.shtml

1. 需要注意：参考github 添加的话要下载很多jar包，包含全平台的，所以我这边是采取的自己去下载Android相关的jar包：javacv下载链接

dependencies {
   implementation group: 'org.bytedeco', name: 'javacv-platform', version: '1.5.8'
 }

2.链接打开 如图下载压缩包：

image.png

解压会看到很多jar包，经过摸索发现这几个是必须的.

image.png

so库只用了armeabi-v7a，可以满足需求了，具体想了解so库相关的知识可以参考这篇文章 https://www.jianshu.com/p/9b8438de044e

3. 编译报错 提示有重复，build.gradle -> android目录下加上

 packagingOptions {
        exclude 'META-INF/native-image/android-arm/jnijavacpp/reflect-config.json'
        exclude 'META-INF/native-image/android-arm/jnijavacpp/jni-config.json'
    }

  /**
     * javacv
     */
    var finishOut:File? =  null
    var audioOut:File? =  null
    private fun testJavacv() {

        val rootPath: String = getExternalFilesDir(null).toString()

        val pathStringBuffer: StringBuffer = StringBuffer(rootPath).append(File.separator)
        val photoPath = pathStringBuffer.toString()
        val fileName = "${System.currentTimeMillis()}.mp4"
        var output = File(photoPath)
        if (!output.exists()) {
            output.mkdirs()
        }
        output = File(photoPath, fileName)
        finishOut = File(photoPath, "aaa" + fileName)
        audioOut = File(photoPath, "bbb" + fileName)
        createMp4(output.path, 800, 800)
    }

    @SuppressLint("UseCompatLoadingForDrawables")
    private fun createMp4(mp4SavePath: String, width: Int, height: Int) {
        val imgs =
            arrayOf(R.drawable.img_guide2_01, R.drawable.img_guide2_02, R.drawable.img_guide2_03)

        //视频宽高最好是按照常见的视频的宽高  16：9  或者 9：16
        val recorder = FFmpegFrameRecorder(mp4SavePath, width, height)
        //设置视频编码层模式
        recorder.videoCodec = 27
        //设置视频为1帧每秒
        recorder.frameRate = 1.0
        //设置视频图像数据格式
        recorder.pixelFormat = 0
        recorder.format = "mp4"
        try {
            recorder.start()
            val converter = AndroidFrameConverter()
            for (i in 1..15) {
                val res = resources
                val d = res.getDrawable(imgs[i / 5], null) as BitmapDrawable
                //一秒是1帧 所以要记录1次 图片多的话可以设置一秒多帧
                recorder.record(converter.convert(d.bitmap))
            }
        } catch (e: Exception) {
            e.printStackTrace()
        } finally {
            //最后一定要结束并释放资源
            recorder.stop()
            recorder.release()
        }
        val audioStr = "audio.mp3"
        val inputStream = assets.open(audioStr)

        val fos: FileOutputStream = FileOutputStream(audioOut)
        val buffer = ByteArray(1024)
        var byteCount = 0
        while (inputStream.read(buffer).also { byteCount = it } != -1) { //循环从输入流读取 buffer字节
            fos.write(buffer, 0, byteCount) //将读取的输入流写入到输出流
        }
        fos.flush() //刷新缓冲区
        inputStream.close()
        fos.close()
        if (Javac.mergeAudioAndVideo(mp4SavePath, audioOut?.path, finishOut?.path)){
            binding.video.setVideoPath(finishOut?.path)
            binding.video.start()

            binding.video.setOnCompletionListener {
                val duration = binding.video.duration
                Toast.makeText(this, "时长：$duration", Toast.LENGTH_SHORT).show()
                it.start()
            }
        }
//            mergeAudioAndVideo(mp4SavePath, audioOut?.path, finishOut?.path)
    }

对于加音频这块单独写的一个类

public class Javac {
    public static boolean mergeAudioAndVideo(String videoPath, String audioPath, String outPut) throws Exception {
        boolean isCreated = true;
        File file = new File(videoPath);
        if (!file.exists()) {
            return false;
        }
        FrameRecorder recorder = null;
        FrameGrabber grabber1 = null;
        FrameGrabber grabber2 = null;
        try {
            //抓取视频帧
            grabber1 = new FFmpegFrameGrabber(videoPath);
            //抓取音频帧
            grabber2 = new FFmpegFrameGrabber(audioPath);
            grabber1.start();
            grabber2.start();
            //创建录制
            recorder = new FFmpegFrameRecorder(outPut,
                    grabber1.getImageWidth(), grabber1.getImageHeight(),
                    grabber2.getAudioChannels());

            recorder.setFormat("mp4");
            //设置视频编码层模式
            recorder.setVideoCodec(27);
            //设置视频为25帧每秒
            //设置视频图像数据格式
            recorder.setPixelFormat(0);
            recorder.setFrameRate(grabber1.getFrameRate());
            recorder.setSampleRate(grabber2.getSampleRate());
            recorder.start();

            Frame frame1;
            Frame frame2 ;
            //先录入视频
            while ((frame1 = grabber1.grabFrame()) != null ){
                recorder.record(frame1);
            }
            //然后录入音频
            while ((frame2 = grabber2.grabFrame()) != null) {
                recorder.record(frame2);
            }
            grabber1.stop();
            grabber2.stop();
            recorder.stop();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                if (recorder != null) {
                    recorder.release();
                }
                if (grabber1 != null) {
                    grabber1.release();
                }
                if (grabber2 != null) {
                    grabber2.release();
                }
            } catch (FrameRecorder.Exception e) {
                e.printStackTrace();
            }
        }
        return isCreated;

    }
}

audio.mp3 是运营提供的一个15s的音频文件 防止assets下即可

5.以上是完整的测试代码，可以先复制 体验一下Javacv图片生成音视频，涉及具体需要自己相应调整吧，另外我这边也有一个问题待解决，so库太大了，感觉也不能再精简了，有知道怎么精简一下的大佬 欢迎指教一下小弟，不胜感激！

image.png