这段时间对rtsp/rtmp等视频直播技术比较感兴趣,想自己做一个用android摄像头采集图像推流到电脑端,于是学习android camera2、MediaCodec、ffmpeg等用法,对比了rtp、rtsp、rtmp等各种推流技术,最终决定在Android端采集音视频数据并编码后,采用live555库做个轻量级服务端,电脑端采用vlc或者opencv都可以轻松打开手机端视频。这就必须要将live555移植到android平台。网上介绍此类移植的文章不少,但大多数都是记录了整个过程中某一两个重点步骤或是坑,使用的环境与我的也有很大不同,我是android新手,对于ndk更是小白一个,照着别人的步骤做,自然是趟了不少坑,但好歹经过自己一天的折腾,竟然都爬出来了,最后编译成功的那一刻自己都不相信了。这个过程必须要记录下来,总结经验,还是要多看程序的输出提示,多动脑子,多尝试几种方案,最后一定能成功。
编译环境:Linuxmint 20.04 + android studio 4.0 + ndk 21.1,ndk是android studio勾选后自动下载安装的,不是单独下载的。
$ANDROID_HOME=~/Android/Sdk
$ANDROID_NDK_HOME=~/Android/Sdk/ndk-bundle (还有一个是sdk/ndk//21.1.6363665,两个目录下的内容好像差不多,不知道为什么要弄两个。后面的编译等过程都是使用的ndk-bundle目录)
一、第一步,自然是下载live555库源码,下载地址:http://www.live555.com/liveMedia/public/live555-latest.tar.gz,下载后解码并将live555目录名更改为jni,因为ndk-build约定要从jni开始
二、第二步,编写Android.mk和Application.mk,放置在jni目录下
这一步遇到过很多坑,感谢https://blog.csdn.net/baby313/article/details/7289489和https://blog.csdn.net/leonpengweicn/article/details/27648229这两篇文章给的提示,但最后的Android.mk还是改动不小。期间遇到了缺少文件问题、需要openssl的问题、undefined referenced stderr等等各种问题,一一化解。其中最大的坑是openssl问题,接下来重点描述这个坑。
Android.mk
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := live555
LOCAL_ARM_MODE := arm
LOCAL_PRELINK_MODULE := false
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES :=\
$(LOCAL_PATH)/groupsock/GroupEId.cpp \
$(LOCAL_PATH)/groupsock/IOHandlers.cpp \
$(LOCAL_PATH)/groupsock/NetInterface.cpp \
$(LOCAL_PATH)/groupsock/NetAddress.cpp \
$(LOCAL_PATH)/groupsock/GroupsockHelper.cpp \
$(LOCAL_PATH)/groupsock/Groupsock.cpp \
$(LOCAL_PATH)/groupsock/inet.c \
$(LOCAL_PATH)/BasicUsageEnvironment/DelayQueue.cpp \
$(LOCAL_PATH)/BasicUsageEnvironment/BasicHashTable.cpp \
$(LOCAL_PATH)/BasicUsageEnvironment/BasicUsageEnvironment.cpp \
$(LOCAL_PATH)/BasicUsageEnvironment/BasicUsageEnvironment0.cpp \
$(LOCAL_PATH)/BasicUsageEnvironment/BasicTaskScheduler.cpp \
$(LOCAL_PATH)/BasicUsageEnvironment/BasicTaskScheduler0.cpp \
$(LOCAL_PATH)/UsageEnvironment/HashTable.cpp \
$(LOCAL_PATH)/UsageEnvironment/strDup.cpp \
$(LOCAL_PATH)/UsageEnvironment/UsageEnvironment.cpp \
$(LOCAL_PATH)/liveMedia/AC3AudioFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/AC3AudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/AC3AudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/AC3AudioStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/ADTSAudioFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/ADTSAudioFileSource.cpp \
$(LOCAL_PATH)/liveMedia/ADTSAudioStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioFileSink.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioFileSource.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/AMRAudioSource.cpp \
$(LOCAL_PATH)/liveMedia/AudioInputDevice.cpp \
$(LOCAL_PATH)/liveMedia/AudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/AVIFileSink.cpp \
$(LOCAL_PATH)/liveMedia/Base64.cpp \
$(LOCAL_PATH)/liveMedia/BasicUDPSink.cpp \
$(LOCAL_PATH)/liveMedia/BasicUDPSource.cpp \
$(LOCAL_PATH)/liveMedia/BitVector.cpp \
$(LOCAL_PATH)/liveMedia/ByteStreamFileSource.cpp \
$(LOCAL_PATH)/liveMedia/ByteStreamMemoryBufferSource.cpp \
$(LOCAL_PATH)/liveMedia/ByteStreamMultiFileSource.cpp \
$(LOCAL_PATH)/liveMedia/DeviceSource.cpp \
$(LOCAL_PATH)/liveMedia/DigestAuthentication.cpp \
$(LOCAL_PATH)/liveMedia/DVVideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/DVVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/DVVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/DVVideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/EBMLNumber.cpp \
$(LOCAL_PATH)/liveMedia/FileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/FileSink.cpp \
$(LOCAL_PATH)/liveMedia/FramedFileSource.cpp \
$(LOCAL_PATH)/liveMedia/FramedFilter.cpp \
$(LOCAL_PATH)/liveMedia/FramedSource.cpp \
$(LOCAL_PATH)/liveMedia/GenericMediaServer.cpp \
$(LOCAL_PATH)/liveMedia/GSMAudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/H261VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/H263plusVideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/H263plusVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/H263plusVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/H263plusVideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/H263plusVideoStreamParser.cpp \
$(LOCAL_PATH)/liveMedia/H264or5VideoFileSink.cpp \
$(LOCAL_PATH)/liveMedia/H264or5VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/H264or5VideoStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/H264or5VideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoFileSink.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/H264VideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoFileSink.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/H265VideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/HLSSegmenter.cpp \
$(LOCAL_PATH)/liveMedia/HMAC_SHA1.cpp \
$(LOCAL_PATH)/liveMedia/InputFile.cpp \
$(LOCAL_PATH)/liveMedia/JPEG2000VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/JPEG2000VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/JPEGVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/JPEGVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/JPEGVideoSource.cpp \
$(LOCAL_PATH)/liveMedia/Locale.cpp \
$(LOCAL_PATH)/liveMedia/MatroskaDemuxedTrack.cpp \
$(LOCAL_PATH)/liveMedia/MatroskaFile.cpp \
$(LOCAL_PATH)/liveMedia/MatroskaFileParser.cpp \
$(LOCAL_PATH)/liveMedia/MatroskaFileServerDemux.cpp \
$(LOCAL_PATH)/liveMedia/MatroskaFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/Media.cpp \
$(LOCAL_PATH)/liveMedia/MediaSession.cpp \
$(LOCAL_PATH)/liveMedia/MediaSink.cpp \
$(LOCAL_PATH)/liveMedia/MediaSource.cpp \
$(LOCAL_PATH)/liveMedia/MIKEY.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADU.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADUdescriptor.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADUinterleaving.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADURTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADURTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MP3ADUTranscoder.cpp \
$(LOCAL_PATH)/liveMedia/MP3AudioFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MP3AudioMatroskaFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MP3FileSource.cpp \
$(LOCAL_PATH)/liveMedia/MP3Internals.cpp \
$(LOCAL_PATH)/liveMedia/MP3InternalsHuffman.cpp \
$(LOCAL_PATH)/liveMedia/MP3InternalsHuffmanTable.cpp \
$(LOCAL_PATH)/liveMedia/MP3StreamState.cpp \
$(LOCAL_PATH)/liveMedia/MP3Transcoder.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2AudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2AudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2AudioStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2Demux.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2DemuxedElementaryStream.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2DemuxedServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2FileServerDemux.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2VideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2VideoStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEG1or2VideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2IndexFromTransportStream.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamAccumulator.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamDemux.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamDemuxedTrack.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamFromESSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamFromPESSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamIndexFile.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamMultiplexor.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamParser.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamParser_PAT.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamParser_PMT.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamParser_STREAM.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportStreamTrickModeFilter.cpp \
$(LOCAL_PATH)/liveMedia/MPEG2TransportUDPServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4ESVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4ESVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4GenericRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4GenericRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4LATMAudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4LATMAudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4VideoFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4VideoStreamDiscreteFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEG4VideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEGVideoStreamFramer.cpp \
$(LOCAL_PATH)/liveMedia/MPEGVideoStreamParser.cpp \
$(LOCAL_PATH)/liveMedia/MultiFramedRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/MultiFramedRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/OggDemuxedTrack.cpp \
$(LOCAL_PATH)/liveMedia/OggFile.cpp \
$(LOCAL_PATH)/liveMedia/OggFileParser.cpp \
$(LOCAL_PATH)/liveMedia/OggFileServerDemux.cpp \
$(LOCAL_PATH)/liveMedia/OggFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/OggFileSink.cpp \
$(LOCAL_PATH)/liveMedia/OnDemandServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/ourMD5.cpp \
$(LOCAL_PATH)/liveMedia/OutputFile.cpp \
$(LOCAL_PATH)/liveMedia/PassiveServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/ProxyServerMediaSession.cpp \
$(LOCAL_PATH)/liveMedia/QCELPAudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/QuickTimeFileSink.cpp \
$(LOCAL_PATH)/liveMedia/QuickTimeGenericRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/RawVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/RawVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/RTCP.cpp \
$(LOCAL_PATH)/liveMedia/rtcp_from_spec.c \
$(LOCAL_PATH)/liveMedia/RTPInterface.cpp \
$(LOCAL_PATH)/liveMedia/RTPSink.cpp \
$(LOCAL_PATH)/liveMedia/RTPSource.cpp \
$(LOCAL_PATH)/liveMedia/RTSPClient.cpp \
$(LOCAL_PATH)/liveMedia/RTSPCommon.cpp \
$(LOCAL_PATH)/liveMedia/RTSPRegisterSender.cpp \
$(LOCAL_PATH)/liveMedia/RTSPServer.cpp \
$(LOCAL_PATH)/liveMedia/RTSPServerRegister.cpp \
$(LOCAL_PATH)/liveMedia/ServerMediaSession.cpp \
$(LOCAL_PATH)/liveMedia/SimpleRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/SimpleRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/SIPClient.cpp \
$(LOCAL_PATH)/liveMedia/SRTPCryptographicContext.cpp \
$(LOCAL_PATH)/liveMedia/StreamParser.cpp \
$(LOCAL_PATH)/liveMedia/StreamReplicator.cpp \
$(LOCAL_PATH)/liveMedia/T140TextRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/TextRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/TheoraVideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/TheoraVideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/TLSState.cpp \
$(LOCAL_PATH)/liveMedia/uLawAudioFilter.cpp \
$(LOCAL_PATH)/liveMedia/VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/VorbisAudioRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/VorbisAudioRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/VP8VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/VP8VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/VP9VideoRTPSink.cpp \
$(LOCAL_PATH)/liveMedia/VP9VideoRTPSource.cpp \
$(LOCAL_PATH)/liveMedia/WAVAudioFileServerMediaSubsession.cpp \
$(LOCAL_PATH)/liveMedia/WAVAudioFileSource.cpp
LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/liveMedia/include \
$(LOCAL_PATH)/liveMedia \
$(LOCAL_PATH)/groupsock/include \
$(LOCAL_PATH)/groupsock \
$(LOCAL_PATH)/BasicUsageEnvironment/include \
$(LOCAL_PATH)/BasicUsageEnvironment \
$(LOCAL_PATH)/UsageEnvironment/include \
$(LOCAL_PATH)/UsageEnvironment \
$(LOCAL_PATH)/../openssl-1.1.1g/include
LOCAL_CPPFLAGS += -DSOCKLEN_T=socklen_t -DBSD=1 -DANDROID -fexceptions -DXLOCALE_NOT_USED -DNULL=0 -DNO_SSTREAM -UIP_ADD_SOURCE_MEMBERSHIP -fPIC
#LOCAL_SHARED_LIBRARIES := libutils libdl libstlport_shared
LOCAL_LDLIBS := -llog -lz -lm -lssl -lcrypto
#LOCAL_STATIC_LIBRARIES := $(LOCAL_PATH)/../openssl-1.1.1g/ssl $(LOCAL_PATH)/../openssl-1.1.1g/crypto
include $(BUILD_SHARED_LIBRARY)
Application.mk
APP_ABI := armeabi-v7a
APP_PLATFORM := android-29
APP_STL := c++_static
重点是openssl这个坑。网上很少有说明live555如何编译openssl的文章,只有自己摸索。刚开始使用ndk-build 编译live555,出现缺少openssl/ssl.h头文件错误,于是想都没想就直接 sudo apt install libssl-dev,编译依然报这个错误,查找/usr/include/openssl/目录下确实有ssl.h头文件,难道ndk不认识linux的默认路径,管他的,手动将/usr/include/路径添加到Android.mk下的LOCAL_C_INCLUDES下,结果错的更离谱,出现了invalid output constraint '=c' in asm_错误,想了想应该是armeaib-v7a与x86的架构不同引起的,应该要从源码重新交叉编译openssl才行。
于是下载了openssl-1.1.1g.tar.gz,解压到与jni同级的目录下,开始折腾openssl。好在Openssl的文档非常完整,有编译android平台的介绍文档NOTES.ANDROID。对照开始第一次编译,进入openssl-1.1.1g目录,执行:
./Configure android-arm64 -D__ANDROID_API__=29
make
正常编译成功,生成了libssl.so,libssl.a,libcrypto.so,libcrypto.a等库文件 。为了让 ndk 能找到openssl源码和编译后的库文件,在Android.mk LOCAL_C_INCLUDES变量中中添加如下内容
LOCAL_C_INCLUDES := $(LOC_PATH)/../openssl-1.1.1g/include \
LOCAL_LDLIBS := LOCAL_LDLIBS := -llog -lz -lm -l../openssl-1.1.1g/crypto -l../openssl-1.1.1g/ssl
继续在jni目录执行~/Android/Sdk/ndk-bundle/ndk-build,错误提示ssl和crypto是两个目录,不能链接库文件。进入openssl目录,将ssl和crypto两个目录改名,再次运行ndk-build,提示要通过LOCAL_STATIC_LIBRARIES 变量引入这两个库,于是又更改Android.mk,添加如下行:
LOCAL_STATIC_LIBRARIES := $(LOCAL_PATH)/../openssl-1.1.1g/ssl $(LOCAL_PATH)/../openssl-1.1.1g/crypto
再次ndk-build,一大堆undefined referenced,明显是没有找到openssl的这两个库。一顿疯狂度娘也没找到办法,仔细分析ndk-build的输出信息,发现ndk-build实际是引用$ANDROID_HOME/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin目录下的aarch64-linux-android22-clang等文件执行的交叉编译,其引用的ndk自带的liblog.a,libdl.a,libm.a等库也对应位于$ANDROID_HOME/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/arm-linux-androideabi目录下,于是直接将libssl.a,libcrypto.a两个文件拷贝到该目录,再次编译,果然没有报unreferenced错误,但是提示架构错误,应该是我编译openssl的时候,配置的是android-arm64架构,而ndk使用的是android-linux-armeabi架构,于是重新编译Openssl,配置如下
./Configure android-arm -D__ANDROID_API__=29 && make
生成的库再次拷贝到sysroot/usr/lib/arm-linux-androideabi目录下。再次运行ndk-build,没有架构错误,但是又出现了 undefined refernece stderr等错误。小白一个,再次度娘,百度上面很多误导的文章要注意甄别,比如加上#undef stderr等操作,都是不对的。解决方法很简单,在Application.mk中添加 APP_STL := c++_static 就行了。
再次ndk-build,终于成功了,生成了live555.so文件。
接下来就是摸索在android中如何使用live555了。