Android平台GB28181设备接入模块开发填坑指南
技术背景
为什么要开发Android平台GB28181设备接入模块？这个问题不再赘述，在做Android平台GB28181客户端的时候，媒体数据这块，我们已经有了很好的积累，因为在此之前，我们就开发了非常成熟的RTMP推送、轻量级RTSP服务、录像模块、针对音视频的对接处理单元。这让我们在做Android平台GB28181设备接入模块的时候，可以有更多的精力在信令交互和国标平台对接。
好多开发者会觉得，GB28181设备接入模块有啥好做的？不就找个开源的SIP信令，视频编码ps打包下投递到国标平台就好了吗？
事实上，当回头看看开发的功能时，就会觉得，一两个月的东西，仅就可以作为项目交付或demo使用，并不会有多大的商业价值，因为需要解决的问题实在太多了。
- [视频格式]H.264/H.265(Android H.265硬编码)；
- [音频格式]G.711 A律、AAC；
- [音量调节]Android平台采集端支持实时音量调节；
- [H.264硬编码]支持H.264特定机型硬编码；
- [H.265硬编码]支持H.265特定机型硬编码；
- [软硬编码参数配置]支持gop间隔、帧率、bit-rate设置；
- [软编码参数配置]支持软编码profile、软编码速度、可变码率设置；
- 支持纯视频、音视频PS打包传输；
- 支持RTP OVER UDP和RTP OVER TCP被动模式（TCP媒体流传输客户端）；
- 支持信令通道网络传输协议TCP/UDP设置；
- 支持注册、注销，支持注册刷新及注册有效期设置；
- 支持设备目录查询应答；
- 支持心跳机制，支持心跳间隔、心跳检测次数设置；
- 支持移动设备位置(MobilePosition)订阅和通知；
- 支持语音广播；
- 支持语音对讲；
- 支持历史视音频文件检索；
- 支持历史视音频文件下载；
- 支持历史视音频文件回放；
- 支持云台控制和预置位查询；
- [实时水印]支持动态文字水印、png水印；
- [镜像]Android平台支持前置摄像头实时镜像功能；
- [实时静音]支持实时静音/取消静音；
- [实时快照]支持实时快照；
- [降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测；
- [外部编码前视频数据对接]支持YUV数据对接；
- [外部编码前音频数据对接]支持PCM对接；
- [外部编码后视频数据对接]支持外部H.264数据对接；
- [外部编码后音频数据对接]外部AAC数据对接；
- [扩展录像功能]支持和录像模块组合使用，录像相关功能。
技术实现
[视频格式]H.264/H.265(Android H.265硬编码)
目前GB28181-2022已经明确表示支持H.265，GB28181设备接入这块，如果需要有好的画质，编码算法这块，一定需要做好，Android端除了低分辨率软编外，超过1280*720，一般建议硬编码。
/**
* Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)
*
* @param kbps: the kbps of different resolution.
*
* @return {0} if successful
*/
public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);
/**
* Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)
*
* @param kbps: the kbps of different resolution.
*
* @return {0} if successful
*/
public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);硬编码参数设置
/*
* 设置视频硬编码码率控制模式
* @param hw_bitrate_mode: -1表示使用默认值, 不设置也会使用默认值, 0:CQ, 1:VBR, 2:CBR, 3:CBR_FD, 请参考:android.media.MediaCodecInfo.EncoderCapabilities
* 注意硬编码和手机硬件有关，多数手机只支持部分码率模式, 另外硬编码设备差异很大，不同设备同一码率控制模式效果可能不一样
* @return {0} if successful
*/
public native int SetVideoHWEncoderBitrateMode(long handle, int hw_bitrate_mode);
/*
* 设置视频硬编码复杂度, 安卓5.0及以上支持
* @param hw_complexity: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getComplexityRange() 和 android.media.MediaFormat.KEY_COMPLEXITY
* 注意硬编码和手机硬件有关，部分手机可能不支持此设置
* @return {0} if successful
*/
public native int SetVideoHWEncoderComplexity(long handle, int hw_complexity);
/*
* 设置视频硬编码质量, 安卓9及以上支持, 仅当硬编码器码率控制模式(BitrateMode)是CQ(constant-quality mode)时才有效
* @param hw_quality: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getQualityRange() 和 android.media.MediaFormat.KEY_QUALITY
* 注意硬编码和手机硬件有关，部分手机可能不支持此设置
* @return {0} if successful
*/
public native int SetVideoHWEncoderQuality(long handle, int hw_quality);
/*
* 设置H.264硬编码Profile, 安卓7及以上支持
* @param hw_avc_profile: 0表示使用默认值, 0x01: Baseline, 0x02: Main, 0x08: High, 0x10000: ConstrainedBaseline, 0x80000: ConstrainedHigh;
* 注意: ConstrainedBaseline 和 ConstrainedHigh 可能多数设备不支持,
* H.264推荐使用 High 或者 ConstrainedHigh, 如果您使用的手机硬解码解不了，那还是设置Baseline
* 如果设置的Profile硬编码器不支持，应编码器会使用默认值
* 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
* @return {0} if successful
*/
public native int SetAVCHWEncoderProfile(long handle, int hw_avc_profile);
/*
* 设置H.264硬编码Level, 这个只有在设置了Profile的情况下才有效, 安卓7及以上支持
* @param hw_avc_level: 0表示使用默认值, 0x100: Level3, 0x200: Level3.1, 0x400: Level3.2,
* 0x800: Level4, 0x1000: Level4.1, 0x2000: Level4.2,
* 0x4000: Level5, 0x8000: Level5.1, 0x10000: Level5.2,
* 0x20000: Level6, 0x40000: Level6.1, 0x80000: Level6.2,
* 如果设置的level太高硬编码器不支持，SDK内部会做相应调整
* 注意: 640*480@25fps最小支持的是Level3, 720p最小支持的是Level3.1, 1080p最小支持的是Level4
* 具体参考:android.media.MediaCodecInfo.CodecProfileLevel
* @return {0} if successful
*/
public native int SetAVCHWEncoderLevel(long handle, int hw_avc_level);
/*
* 设置视频硬编码最大码率, 安卓没有相关文档说明, 所以不建议设置,
* @param hw_max_bitrate: 每秒最大码率, 单位bps
* @return {0} if successful
*/
public native int SetVideoHWEncoderMaxBitrate(long handle, long hw_max_bitrate);[音频格式]G.711 A律、AAC
/**
* Set audio encoder type(设置音频编码类型)
*
* @param type: if with 1:AAC, if with 2: SPEEX, if with 3: PCMA
*
* @return {0} if successful
*/
public native int SmartPublisherSetAudioCodecType(long handle, int type);
/**
* Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效
*
* @param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0
*
* @return {0} if successful
*/
public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);[音量调节]Android平台采集端支持实时音量调节
/**
* 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量
*
* @param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量
*
* @param volume: 音量，默认是1.0，范围是[0.0, 5.0], 设置成0静音, 1音量不变
*
* @return {0} if successful
*/
public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);[软硬编码参数配置]支持gop间隔、帧率、bit-rate设置，支持软编码profile、软编码速度、可变码率设置
/**
* Set software encode vbr mode(软编码可变码率).
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr
*
* video_quality: vbr video quality, range with (1,50), default 23
*
* vbr_max_kbitrate: vbr max encode bit-rate(kbps)
*
* @return {0} if successful
*/
public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);
/**
* Set gop interval(设置I帧间隔)
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* gopInterval: encode I frame interval, the value always > 0
*
* @return {0} if successful
*/
public native int SmartPublisherSetGopInterval(long handle, int gopInterval);
/**
* Set software encode video bit-rate(设置视频软编码bit-rate)
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* avgBitRate: average encode bit-rate(kbps)
*
* maxBitRate: max encode bit-rate(kbps)
*
* @return {0} if successful
*/
public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);
/**
* Set fps(设置帧率)
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* fps: the fps of video, range with (1,25).
*
* @return {0} if successful
*/
public native int SmartPublisherSetFPS(long handle, int fps);
/**
* Set software video encoder profile(设置视频编码profile).
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* profile: the software video encoder profile, range with (1,3).
*
* 1: baseline profile
* 2: main profile
* 3: high profile
*
* @return {0} if successful
*/
public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);
/**
* Set software video encoder speed(设置视频软编码编码速度)
*
* please set before SmartPublisherStart while after SmartPublisherOpen.
*
* @param speed: range with(1, 6), the default speed is 6.
*
* if with 1, CPU is lowest.
* if with 6, CPU is highest.
*
* @return {0} if successful
*/
public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);信令通道网络传输协议TCP/UDP设置
gb28181_agent_.setTransportProtocol(gb28181_sip_trans_protocol_==0?"UDP":"TCP");支持注册、注销，支持注册刷新及注册有效期设置
private int gb28181_reg_expired_ = 3600; // 注册有效期时间最小3600秒
// GB28181配置
gb28181_agent_.config(gb28181_reg_expired_, gb28181_heartbeat_interval_, gb28181_heartbeat_count_);
@Override
public void ntsRegisterOK(String dateString) {
Log.i(TAG, "ntsRegisterOK Date: " + (dateString!= null? dateString : ""));
}
@Override
public void ntsRegisterTimeout() {
Log.e(TAG, "ntsRegisterTimeout");
}
@Override
public void ntsRegisterTransportError(String errorInfo) {
Log.e(TAG, "ntsRegisterTransportError error:" + (errorInfo != null?errorInfo :""));
}支持心跳机制，支持心跳间隔、心跳检测次数设置
private int gb28181_heartbeat_interval_ = 20; // 心跳间隔GB28181默认是60, 目前调整到20秒
private int gb28181_heartbeat_count_ = 3; // 心跳间隔3次失败，表示和服务器断开了
@Override
public void ntsOnHeartBeatException(int exceptionCount, String lastExceptionInfo) {
Log.e(TAG, "ntsOnHeartBeatException heart beat timeout count reached, count:" + exceptionCount+
", exception info:" + (lastExceptionInfo!=null?lastExceptionInfo:""));
// 停止信令, 然后重启
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "gb28281_heart_beart_timeout");
record_executor_.cancel_tasks();
stopRecordDownloads(true);
stopPlaybacks(true);
stopAudioPlayer();
destoryRTPReceiver();
gb_broadcast_source_id_ = null;
gb_broadcast_target_id_ = null;
btnGB28181AudioBroadcast.setText("GB28181语音广播");
btnGB28181AudioBroadcast.setEnabled(false);
stopGB28181Stream();
destoryRTPSender();
if (gb28181_agent_ != null) {
gb28181_agent_.terminateAllAudioBroadcasts(true);
gb28181_agent_.terminateAllPlays(true);
Log.i(TAG, "gb28281_heart_beart_timeout sip stop");
gb28181_agent_.stop();
String local_ip_addr = IPAddrUtils.getIpAddress(context_);
if (local_ip_addr != null && !local_ip_addr.isEmpty() ) {
Log.i(TAG, "gb28281_heart_beart_timeout get local ip addr: " + local_ip_addr);
gb28181_agent_.setLocalAddress(local_ip_addr);
}
record_executor_.cancel_tasks();
initPlaybacks(null);
initRecordDownloads(null);
Log.i(TAG, "gb28281_heart_beart_timeout sip start");
gb28181_agent_.start();
}
}
},0);
}支持移动设备位置(MobilePosition)订阅和通知
com.gb.ntsignalling.Device gb_device = new com.gb.ntsignalling.Device("34020000001380000001", "安卓测试设备", Build.MANUFACTURER, Build.MODEL,
"宇宙","火星1","火星", true);
if (mLongitude != null && mLatitude != null) {
com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();
device_pos.setTime(mLocationTime);
device_pos.setLongitude(mLongitude);
device_pos.setLatitude(mLatitude);
gb_device.setPosition(device_pos);
gb_device.setSupportMobilePosition(true); // 设置支持移动位置上报
}
@Override
public void ntsOnDevicePositionRequest(String deviceId, int interval) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
getLocation(context_);
Log.v(TAG, "ntsOnDevicePositionRequest, deviceId:" + this.device_id_ + ", Longitude:" + mLongitude
+ ", Latitude:" + mLatitude + ", Time:" + mLocationTime);
if (mLongitude != null && mLatitude != null) {
com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();
device_pos.setTime(mLocationTime);
device_pos.setLongitude(mLongitude);
device_pos.setLatitude(mLatitude);
if (gb28181_agent_ != null ) {
gb28181_agent_.updateDevicePosition(device_id_, device_pos);
}
}
}
private String device_id_;
private int interval_;
public Runnable set(String device_id, int interval) {
this.device_id_ = device_id;
this.interval_ = interval;
return this;
}
}.set(deviceId, interval),0);
}支持语音广播和语音对讲
@Override
public void ntsOnAudioBroadcast(String commandFromUserName, String commandFromUserNameAtDomain, String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnAudioBroadcastPlay, fromFromUserName:" + command_from_user_name_
+ " FromUserNameAtDomain:" + command_from_user_name_at_domain_
+ " sourceID:" + source_id_ + ", targetID:" + target_id_);
stopAudioPlayer();
destoryRTPReceiver();
if (gb28181_agent_ != null ) {
String local_ip_addr = IPAddrUtils.getIpAddress(context_);
boolean is_tcp = true; // 考虑到跨网段, 默认用TCP传输rtp包
rtp_receiver_handle_ = lib_player_.CreateRTPReceiver(0);
if (rtp_receiver_handle_ != 0 ) {
lib_player_.SetRTPReceiverTransportProtocol(rtp_receiver_handle_, is_tcp?1:0);
lib_player_.SetRTPReceiverIPAddressType(rtp_receiver_handle_, 0);
if (0 == lib_player_.CreateRTPReceiverSession(rtp_receiver_handle_, 0) ) {
int local_port = lib_player_.GetRTPReceiverLocalPort(rtp_receiver_handle_);
boolean ret = gb28181_agent_.inviteAudioBroadcast(command_from_user_name_,command_from_user_name_at_domain_,
source_id_, target_id_, "IP4", local_ip_addr, local_port, is_tcp?"TCP/RTP/AVP":"RTP/AVP");
if (!ret ) {
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
else {
btnGB28181AudioBroadcast.setText("GB28181语音广播呼叫中");
}
} else {
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
}
}
}
private String command_from_user_name_;
private String command_from_user_name_at_domain_;
private String source_id_;
private String target_id_;
public Runnable set(String command_from_user_name, String command_from_user_name_at_domain, String source_id, String target_id) {
this.command_from_user_name_ = command_from_user_name;
this.command_from_user_name_at_domain_ = command_from_user_name_at_domain;
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(commandFromUserName, commandFromUserNameAtDomain, sourceID, targetID),0);
}
@Override
public void ntsOnInviteAudioBroadcastException(String sourceID, String targetID, String errorInfo) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnInviteAudioBroadcastException, sourceID:" + source_id_ + ", targetID:" + target_id_);
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}
@Override
public void ntsOnInviteAudioBroadcastTimeout(String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnInviteAudioBroadcastTimeout, sourceID:" + source_id_ + ", targetID:" + target_id_);
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}支持历史视音频文件检索、支持历史视音频文件下载和回放
/**
* Author: daniusdk.com
*/
package com.gb.ntsignalling;
public interface GBSIPAgent {
void addDownloadListener(GBSIPAgentDownloadListener downloadListener);
void removeDownloadListener(GBSIPAgentDownloadListener removeListener);
/*
*响应Invite Download 200 OK
*/
boolean respondDownloadInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
/*
*响应Invite Download 其他状态码
*/
boolean respondDownloadInvite(int statusCode, long id, String deviceId, String startTime, String stopTime);
/*
* 媒体流发送者在文件下载结束后发Message消息通知SIP服务器回文件已发送完成
* notifyType 必须是"121“
*/
boolean notifyDownloadMediaStatus(long id, String deviceId, String startTime, String stopTime, String notifyType);
/*
*终止Download会话
*/
void terminateDownload(long id, String deviceId, String startTime, String stopTime, boolean isSendBYE);
/*
*终止所有Download会话
*/
void terminateAllDownloads(boolean isSendBYE);
}
package com.gb.ntsignalling;
public interface GBSIPAgent {
void addPlaybackListener(GBSIPAgentPlaybackListener playbackListener);
void removePlaybackListener(GBSIPAgentPlaybackListener playbackListener);
/*
*响应Invite Playback 200 OK
*/
boolean respondPlaybackInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);
/*
*响应Invite Playback 其他状态码
*/
boolean respondPlaybackInvite(int statusCode, long id, String deviceId);
/*
* 媒体流发送者在回放结束后发Message消息通知SIP服务器回放文件已发送完成
* notifyType 必须是"121"
*/
boolean notifyPlaybackMediaStatus(long id, String deviceId, String notifyType);
/*
*终止Playback会话
*/
void terminatePlayback(long id, String deviceId, boolean isSendBYE);
/*
*终止所有Playback会话
*/
void terminateAllPlaybacks(boolean isSendBYE);
}
/**
* 信令Playback Listener
*/
package com.gb.ntsignalling;
public interface GBSIPAgentPlaybackListener {
/*
*收到s=Playback的历史回放Invite
*/
void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sessionDescription);
/*
*发送Playback invite response 异常
*/
void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo);
/*
* 收到CANCEL Playback INVITE请求
*/
void ntsOnCancelPlayback(long id, String deviceId);
/*
* 收到Ack
*/
void ntsOnAckPlayback(long id, String deviceId);
/*
* 播放命令
*/
void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId);
/*
* 暂停命令
*/
void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId);
/*
* 快进/慢进命令
*/
void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale);
/*
* 随机拖动命令
*/
void ntsOnPlaybackMANSRTSPSeekCommand(long id, String deviceId, double position_sec);
/*
* 停止命令
*/
void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String deviceId);
/*
* 收到Bye
*/
void ntsOnByePlayback(long id, String deviceId);
/*
* 不是在收到BYE Message情况下， 终止Playback
*/
void ntsOnTerminatePlayback(long id, String deviceId);
/*
* Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
收到这个, 请做相关清理处理
*/
void ntsOnPlaybackDialogTerminated(long id, String deviceId);
}
/**
* 部分JNI接口, rtp ps 打包发送等代码C++实现
*/
public class SmartPublisherJniV2 {
/**
* Open publisher(启动推送实例)
*
* @param ctx: get by this.getApplicationContext()
*
* @param audio_opt:
* if 0: 不推送音频
* if 1: 推送编码前音频(PCM)
* if 2: 推送编码后音频(aac/pcma/pcmu/speex).
*
* @param video_opt:
* if 0: 不推送视频
* if 1: 推送编码前视频(NV12/I420/RGBA8888等格式)
* if 2: 推送编码后视频(AVC/HEVC)
* if 3: 层叠加模式
*
* This function must be called firstly.
*
* @return the handle of publisher instance
*/
public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt, int width, int height);
/**
* 设置流类型
* @param type: 0:表示 live 流, 1:表示 on-demand 流, SDK默认为0(live流)
* 注意: 流类型设置当前仅对GB28181媒体流有效
* @return {0} if successful
*/
public native int SetStreamType(long handle, int type);
/**
* 投递视频 on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
*
* @param codec_id: 编码id, 当前支持H264和H265, 1:H264, 2:H265
*
* @param packet: 视频数据, 包格式请参考H264/H265 Annex B Byte stream format, 例如:
* 0x00000001 nal_unit 0x00000001 ...
* H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
* H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....
*
* @param offset: 偏移量
* @param size: packet size
* @param pts_us: 时间戳, 单位微秒
* @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
* @param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧
* @param codec_specific_data: 可选参数，可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps
* ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps
* @param codec_specific_data_size: codec_specific_data size
* @param width: 图像宽, 可传0
* @param height: 图像高, 可传0
*
* @return {0} if successful
*/
public native int PostVideoOnDemandPacketByteBuffer(long handle, int codec_id,
ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity, int is_key,
byte[] codec_specific_data, int codec_specific_data_size,
int width, int height);
/**
* 投递音频on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer
*
* @param codec_id: 编码id, 当前支持PCMA和AAC, 65536:PCMA, 65538:AAC
* @param packet: 音频数据
* @param offset：packet偏移量
* @param size: packet size
* @param pts_us: 时间戳, 单位微秒
* @param is_pts_discontinuity: 是否时间戳间断，0:未间断，1:间断
* @param codec_specific_data: 如果是AAC的话，需要传 Audio Specific Configuration
* @param codec_specific_data_size: codec_specific_data size
* @param sample_rate: 采样率
* @param channels: 通道数
*
* @return {0} if successful
*/
public native int PostAudioOnDemandPacketByteBuffer(long handle, int codec_id,
ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity,
byte[] codec_specific_data, int codec_specific_data_size,
int sample_rate, int channels);
/**
* on demand source完成seek后, 请调用
* @return {0} if successful
*/
public native int OnSeekProcessed(long handle);
/**
* 启动 GB28181 媒体流
*
* @return {0} if successful
*/
public native int StartGB28181MediaStream(long handle);
/**
* 停止 GB28181 媒体流
*
* @return {0} if successful
*/
public native int StopGB28181MediaStream(long handle);
/**
* 关闭推送实例，结束时必须调用close接口释放资源
*
* @return {0} if successful
*/
public native int SmartPublisherClose(long handle);
}
/**
* Listener部分实现代码
*/
public class PlaybackListenerImpl implements com.gb.ntsignalling.GBSIPAgentPlaybackListener {
/*
*收到s=Playback的文件下载Invite
*/
@Override
public void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sdp) {
if (!post_task(new PlaybackListenerImpl.OnInviteTask(this.context_, this.is_exit_, this.senders_map_, deviceId, sdp, id))) {
Log.e(TAG, "ntsOnInvitePlayback post_task failed, " + RecordSender.make_print_tuple(id, deviceId, sdp.getTime().getStartTime(), sdp.getTime().getStopTime()));
// 这里不发488, 等待事务超时也可以的
GBSIPAgent agent = this.context_.get_agent();
if (agent != null)
agent.respondPlaybackInvite(488, id, deviceId);
}
}
/*
*发送Playback invite response 异常
*/
@Override
public void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo) {
Log.i(TAG, "ntsOnPlaybackInviteResponseException, status_code:" + statusCode + ", "
+ RecordSender.make_print_tuple(id, deviceId) + ", error_info:" + errorInfo);
RecordSender sender = senders_map_.remove(id);
if (null == sender)
return;
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 收到CANCEL Playback INVITE请求
*/
@Override
public void ntsOnCancelPlayback(long id, String deviceId) {
Log.i(TAG, "ntsOnCancelPlayback, " + RecordSender.make_print_tuple(id, deviceId));
RecordSender sender = senders_map_.remove(id);
if (null == sender)
return;
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 收到Ack
*/
@Override
public void ntsOnAckPlayback(long id, String deviceId) {
Log.i(TAG, "ntsOnAckPlayback, "+ RecordSender.make_print_tuple(id, deviceId));
RecordSender sender = senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnAckPlayback get sender is null, " + RecordSender.make_print_tuple(id, deviceId));
GBSIPAgent agent = this.context_.get_agent();
if (agent != null)
agent.terminatePlayback(id, deviceId, false);
return;
}
PlaybackListenerImpl.StartTask task = new PlaybackListenerImpl.StartTask(sender, this.senders_map_);
if (!post_task(task))
task.run();
}
/*
* 收到Bye
*/
@Override
public void ntsOnByePlayback(long id, String deviceId) {
Log.i(TAG, "ntsOnByePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
RecordSender sender = this.senders_map_.remove(id);
if (null == sender)
return;
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 播放命令
*/
@Override
public void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId) {
RecordSender sender = this.senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPPlayCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
return;
}
sender.post_play_command();
Log.i(TAG, "ntsOnPlaybackMANSRTSPPlayCommand " + RecordSender.make_print_tuple(id, deviceId));
}
/*
* 暂停命令
*/
@Override
public void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId) {
RecordSender sender = this.senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPPauseCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));
return;
}
sender.post_pause_command();
Log.i(TAG, "ntsOnPlaybackMANSRTSPPauseCommand " + RecordSender.make_print_tuple(id, deviceId));
}
/*
* 快进/慢进命令
*/
@Override
public void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale) {
if (scale < 0.01) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand invalid scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));
return;
}
RecordSender sender = this.senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand can not get sender, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));
return;
}
sender.post_scale_command(scale);
Log.i(TAG, "ntsOnPlaybackMANSRTSPScaleCommand, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));
}
/*
* 随机拖动命令
*/
@Override
public void ntsOnPlaybackMANSRTSPSeekCommand(long id, String device_id, double position_sec) {
if (position_sec < 0.0) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand invalid seek pos:" + position_sec + ", " + RecordSender.make_print_tuple(id, device_id));
return;
}
RecordSender sender = this.senders_map_.get(id);
if (null == sender) {
Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand can not get sender " + RecordSender.make_print_tuple(id, device_id));
return;
}
long offset_ms = sender.get_file_start_time_offset_ms();
position_sec += (offset_ms/1000.0);
sender.post_seek_command(position_sec);
Log.i(TAG, "ntsOnPlaybackMANSRTSPSeekCommand seek pos:" + RecordSender.out_point_3(position_sec) + "s, " + RecordSender.make_print_tuple(id, device_id));
}
/*
* 停止命令
*/
@Override
public void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String device_id) {
CallTerminatePlaybackTask call_terminate_task = new CallTerminatePlaybackTask(this.context_, id, device_id, true);
post_task(call_terminate_task);
RecordSender sender = this.senders_map_.remove(id);
if (null == sender) {
Log.w(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand can not remove sender " + RecordSender.make_print_tuple(id, device_id));
return;
}
Log.i(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand " + RecordSender.make_print_tuple(id, device_id));
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* 不是在收到BYE Message情况下， 终止Playback
*/
@Override
public void ntsOnTerminatePlayback(long id, String deviceId) {
Log.i(TAG, "ntsOnTerminatePlayback, "+ RecordSender.make_print_tuple(id, deviceId));
RecordSender sender = this.senders_map_.remove(id);
if (null == sender)
return;
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
/*
* Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发
收到这个, 请做相关清理处理
*/
@Override
public void ntsOnPlaybackDialogTerminated(long id, String deviceId) {
Log.i(TAG, "ntsOnPlaybackDialogTerminated, "+ RecordSender.make_print_tuple(id, deviceId));
RecordSender sender = this.senders_map_.remove(id);
if (null == sender)
return;
PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);
if (!post_task(task))
task.run();
}
}支持云台控制和预置位查询
@Override
public void ntsOnDevicePresetQueryCommand(String fromUserName, String fromUserNameAtDomain, String sn, String deviceId) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "DaniuSDK ntsOnDevicePresetQueryCommand from_user_name:" + from_user_name_ + ", sn:" + sn_ + ", device_id:" + device_id_);
List preset_list = new LinkedList<>();
preset_list.add(new com.gb.ntsignalling.PresetItem("1", "Android PreSet1"));
preset_list.add(new com.gb.ntsignalling.PresetItem("2", "Android PreSet2"));
if (gb28181_agent_ != null )
gb28181_agent_.respondDevicePresetQueryCommand(this.from_user_name_, this.from_user_name_at_domain_, this.sn_, this.device_id_, preset_list);
}
private String from_user_name_;
private String from_user_name_at_domain_;
private String sn_;
private String device_id_;
public Runnable set(String from_user_name, String from_user_name_at_domain,String sn, String device_id) {
this.from_user_name_ = from_user_name;
this.from_user_name_at_domain_ = from_user_name_at_domain;
this.sn_ = sn;
this.device_id_ = device_id;
return this;
}
}.set(fromUserName, fromUserNameAtDomain, sn, deviceId),0);
}
@Override
public void ntsOnDeviceControlPTZCmd(String deviceId, String typeValue) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "DaniuSDK ntsOnDeviceControlPTZCmd device_id:" + device_id_ + " PTZType:" + ptz_type_);
if (null == ptz_type_)
return;
ptz_type_ = ptz_type_.trim();
if (ptz_type_.length() != 16)
return;
int instruction = hexStringToInt(ptz_type_.substring(6, 8));
int combination_code2 = hexStringToInt(ptz_type_.substring(12, 14));
//Android平台GB28181设备接入端，针对性的解析处理即可，这里不再赘述
private String device_id_;
private String ptz_type_;
public Runnable set(String device_id, String ptz_type) {
this.device_id_ = device_id;
this.ptz_type_ = ptz_type;
return this;
}
}.set(deviceId, typeValue),0);
} [实时水印]支持动态文字水印、png水印
watermarkSelctor = (Spinner) findViewById(R.id.watermarkSelctor);
final String[] watermarks = new String[]{"图片水印", "全部水印", "文字水印", "不加水印"};
ArrayAdapter adapterWatermark = new ArrayAdapter(this,
android.R.layout.simple_spinner_item, watermarks);
adapterWatermark.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
watermarkSelctor.setAdapter(adapterWatermark);
watermarkSelctor.setSelection(3,true);
watemarkType = 3; //默认不加水印
watermarkSelctor.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
@Override
public void onItemSelected(AdapterView parent, View view,
int position, long id) {
watemarkType = position;
Log.i(TAG, "[水印类型]Currently choosing: " + watermarks[position] + ", watemarkType: " + watemarkType);
if (layer_post_thread_ != null) {
layer_post_thread_.enableText(isHasTextWatermark());
layer_post_thread_.enablePicture(isHasPictureWatermark());
}
}
@Override
public void onNothingSelected(AdapterView parent) {
}
}); [实时快照]支持实时快照
class ButtonCaptureImageListener implements View.OnClickListener {
@SuppressLint("SimpleDateFormat")
public void onClick(View v) {
if(isPushingRtmp || isRecording || isRTSPPublisherRunning || isGB28181StreamRunning)
{
String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
String imageFileName = "dn_" + timeStamp; //创建以时间命名的文件名称
String imagePath = imageSavePath + "/" + imageFileName + ".png";
Log.i(TAG, "imagePath:" + imagePath);
libPublisher.SmartPublisherSaveCurImage(publisherHandle, imagePath);
}
else
{
Log.e(TAG, "快照失败，请确保在推送、录像或内置RTSP服务发布状态..");
}
}
}[降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测
boolean is_noise_suppression = true;
libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1 : 0);
boolean is_agc = false;
libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);
int echo_cancel_delay = 0;
libPublisher.SmartPublisherSetEchoCancellation(publisherHandle, 1, echo_cancel_delay);外部编码前后视频数据对接
- 编码前数据（目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型），其中，Android平台前后摄像头数据，或者屏幕数据，或者Unity拿到的数据，均属编码前数据；
- 编码后数据（如无人机等264/HEVC数据，或者本地解析的MP4音视频数据）；
[扩展录像功能]支持和录像SDK组合使用，录像相关功能
class ButtonStartRecorderListener implements View.OnClickListener {
public void onClick(View v) {
if (isRecording) {
stopRecorder();
if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
ConfigControlEnable(true);
}
btnStartRecorder.setText("实时录像");
btnPauseRecorder.setText("暂停录像");
btnPauseRecorder.setEnabled(false);
isPauseRecording = true;
return;
}
Log.i(TAG, "onClick start recorder..");
if (libPublisher == null)
return;
if (!isPushingRtmp && !isRTSPPublisherRunning&& !isGB28181StreamRunning) {
InitAndSetConfig();
}
ConfigRecorderParam();
int startRet = libPublisher.SmartPublisherStartRecorder(publisherHandle);
if (startRet != 0) {
if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
if (publisherHandle != 0) {
long handle = publisherHandle;
publisherHandle = 0;
libPublisher.SmartPublisherClose(handle);
}
}
Log.e(TAG, "Failed to start recorder.");
return;
}
if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {
CheckInitAudioRecorder();
ConfigControlEnable(false);
}
startLayerPostThread();
btnStartRecorder.setText("停止录像");
isRecording = true;
btnPauseRecorder.setEnabled(true);
isPauseRecording = true;
}
}
class ButtonPauseRecorderListener implements View.OnClickListener {
public void onClick(View v) {
if (isRecording) {
if(isPauseRecording)
{
int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 1);
if (ret == 0)
{
isPauseRecording = false;
btnPauseRecorder.setText("恢复录像");
}
else if(ret == 3)
{
Log.e(TAG, "Pause recorder failed, please re-try again..");
}
else
{
Log.e(TAG, "Pause recorder failed..");
}
}
else
{
int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 0);
if (ret == 0)
{
isPauseRecording = true;
btnPauseRecorder.setText("暂停录像");
}
else if(ret == 3)
{
Log.e(TAG, "Resume recorder failed, please re-try again..");
}
else
{
Log.e(TAG, "Resume recorder failed..");
}
}
}
}
}总结
Android平台GB28181设备接入侧模块，如果需要做的更好，上述提到的技术层面的问题解决了还不够，还需要针对各类国标平台适配对接，只有这样，才能更好的为执法记录仪、智能安全帽、智能监控、智慧零售、智慧教育、远程办公、明厨亮灶、智慧交通、智慧工地、雪亮工程、平安乡村、生产运输、车载终端等场景服务。