在之前的blog,我们以Android平台国标接入终端为例,分别介绍了一些常规的功能,比如REGISTER、CATALOG、INVITE、Keepalive、SUBSCRIBE、NOTIFY等常规操作,今天主要介绍下语音广播和语音对讲这部分。
GB28181平台广播和对讲这块,重要性不言而喻,没有广播的接入终端,数据只是单向流入,加入后,指挥中心和终端之间的联系更紧密,实时双向沟通更方便,适用的行业范围也更广泛。
关于语音广播和对讲,感兴趣的开发者可直接参阅GBT 28181-2016.pdf相关技术规范里面的9.12章节,以下是部分精选介绍:
a) 1:SIP服务器向语音流接收者发送语音广播通知消息,消息中通过 To头域标明作为目的地址 的语音流接收者ID,消息采用 Message方法携带。
举例说明:
MESSAGE sip:[email protected]:12070 SIP/2.0
Via: SIP/2.0/UDP 192.168.2.154:15060;rport;branch=z9hG4bK311226558
From: ;tag=280226558
To:
Call-ID: 172226558
CSeq: 207 MESSAGE
Content-Type: Application/MANSCDP+xml
Max-Forwards: 70
Content-Length: 206
Broadcast
461226558
34020000002000000001
34020000001380000001
b) 2:语音流接收者收到语音广播通知消息后,向SIP服务器发送200OK 响应。
SIP/2.0 200 OK
CSeq: 207 MESSAGE
Call-ID: 172226558
From: ;tag=280226558
To:
Via: SIP/2.0/UDP 192.168.2.154:15060;rport=15060;branch=z9hG4bK311226558;received=192.168.2.154
Content-Length: 0
c) 3:语音流接收者向SIP服务器发送语音广播应答消息,消息中通过 To头域标明作为目的地 址的SIP服务器ID,消息采用 Message方法携带。
MESSAGE sip:34020000002000000001@3402000000 SIP/2.0
Call-ID: [email protected]
CSeq: 337044229 MESSAGE
From: ;tag=93882333
To:
Via: SIP/2.0/UDP 192.168.2.212:12070;rport;branch=z9hG4bK-363733-79fd88c45667975e5ebaf18f84b91a8e
Max-Forwards: 70
User-Agent: NT GB28181 User Agent V1.2(daniusdk.com)
Content-Type: Application/MANSCDP+xml
Content-Length: 180
Broadcast
461226558
34020000001380000001
OK
d) 4:SIP服务器收到语音广播应答消息后,向语音流接收者发送200OK 响应。
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.168.2.212:12070;rport=12070;received=192.168.2.212;branch=z9hG4bK-363733-79fd88c45667975e5ebaf18f84b91a8e
From: ;tag=93882333
To: ;tag=355226593
CSeq: 337044229 MESSAGE
Call-ID: [email protected]
Content-Length: 0
e) 5:语音流接收者向SIP服务器发送Invite消息,消息中通过 To头域标明作为目的地址的语音 流发送者ID,消息头域中携带Subject字段,表明请求的语音流发送者ID、发送方媒体流序列 号、语音流接收者ID、接收方媒体流序列号等参数,SDP消息体中s字段为“Play”代表实时点 播,m 字段中媒体参数标识为“audio”表示请求语音媒体流。
INVITE sip:34020000002000000001@3402000000 SIP/2.0
Call-ID: [email protected]
CSeq: 44264 INVITE
From: ;tag=32ecf22a
To:
Via: SIP/2.0/UDP 192.168.2.212:12070;rport;branch=z9hG4bK-363733-15283c8a0ea0a1e9dbf295ce2359dbe7
Max-Forwards: 70
Contact:
Subject: 34020000002000000001:0200006727,34020000001380000001:0
User-Agent: NT GB28181 User Agent V1.2(daniusdk.com)
Content-Type: APPLICATION/SDP
Content-Length: 221
v=0
o=34020000002000000001 0 0 IN IP4 192.168.2.212
s=Play
c=IN IP4 192.168.2.212
t=0 0
m=audio 25002 TCP/RTP/AVP 8
a=setup:active
a=connection:new
a=recvonly
a=rtpmap:8 PCMA/8000
y=0200006727
f=v/a/1/8/1
f) 6:SIP服务器收到Invite请求后,通过三方呼叫控制建立媒体服务器和语音流发送者之间的媒体连接。向媒体服务器发送Invite消息,此消息不携带SDP消息体。
g) 7:媒体服务器收到SIP服务器的Invite请求后,回复200OK 响应,携带SDP消息体,消息体 中描述了媒体服务器接收媒体流的IP、端口、媒体格式等内容。
h) 8:SIP服务器收到媒体服务器返回的200OK 响应后,向语音流发送者发送Invite请求,消息 中通过 To头域标明作为目的地址的语音流发送者ID,消息头域中携带 Subject字段,表明请 求的语音流发送者ID、发送方媒体流序列号、语音流接收者ID、接收方媒体流序列号等参数, 请求中携带消息7中媒体服务器回复的200OK 响应消息体,s字段为“Play”代表实时点播, m 字段中媒体参数标识为“audio”表示请求语音媒体流,增加y字段描述SSRC值,f字段描述 媒体参数。
SIP/2.0 200 OK
Via: SIP/2.0/UDP 192.168.2.212:12070;rport=12070;received=192.168.2.212;branch=z9hG4bK-363733-15283c8a0ea0a1e9dbf295ce2359dbe7
From: ;tag=32ecf22a
To: ;tag=954226632
CSeq: 44264 INVITE
Call-ID: [email protected]
Contact:
Content-Length: 222
Content-Type: APPLICATION/SDP
v=0
o=34020000002000000001 0 0 IN IP4 192.168.2.154
s=Play
c=IN IP4 192.168.2.154
t=0 0
m=audio 30005 TCP/RTP/AVP 8
a=sendonly
a=rtpmap:8 PCMA/8000
a=setup:passive
a=connection:new
y=0200006727
f=v/a/1/8/1
i) 9:语音流发送者收到SIP服务器的Invite请求后,回复200OK 响应,携带SDP消息体,消息 体中描述了媒体流发送者发送媒体流的IP、端口、媒体格式、SSRC 字段等内容,s字段为 “Play”代表实时点播,m 字段中媒体参数标识为“audio”表示请求语音媒体流。
j) 10:SIP服务器收到语音流发送者返回的200OK 响应后,向媒体服务器发送 ACK 请求,请求 中携带消息9中语音流发送者回复的200OK 响应消息体,完成与媒体服务器的Invite会话 建立过程。
k) 11:SIP服务器收到语音流发送者返回的200OK 响应后,向语音流发送者发送 ACK 请求,请 求中不携带消息体,完成与语音流发送者的Invite会话建立过程。
l) 12:完成三方呼叫控制后,SIP服务器通过 B2BUA 代理方式建立语音流接收者和媒体服务器 之间的媒体连接。在消息5中增加SSRC值,转发给媒体服务器。
m)13:媒体服务器收到Invite请求,回复200OK 响应,携带SDP消息体,消息体中描述了媒体服 务器发送媒体流的IP、端口、媒体格式、SSRC值等内容,s字段为“Play”代表实时点播,m 字段 中媒体参数标识为“audio”表示请求语音媒体流。
n) 14:SIP服务器将消息13转发给语音流接收者。
o) 15:语音流接收者收到200OK 响应后,回复 ACK 消息,完成与SIP服务器的Invite会话建立 过程。
p) 16:SIP服务器将消息15转发给媒体服务器,完成与媒体服务器的Invite会话建立过程。
q) 17:SIP服务器向语音流接收者发送 BYE消息,断开消息5、14、15建立的Invite会话。
r) 18:语音流接收者收到 BYE消息后回复200OK 响应,会话断开。
s) 19:SIP服务器向媒体服务器发送 BYE 消息,断开消息 12、13、16 建立的同媒体服务器的 Invite会话。
t) 20:媒体服务器收到 BYE消息后回复200OK 响应,会话断开。
u) 21:SIP服务器向媒体服务器发送 BYE消息,断开消息6、7、10建立的同媒体服务器的Invite 会话。
v) 22:媒体服务器收到 BYE消息后回复200OK 响应,会话断开。
w)23:SIP服务器向语音流发送者发送 BYE 消息,断开消息8、9、11建立的同语音流发送者的 Invite会话。
x) 24:语音流发送者收到 BYE消息后回复200OK 响应,会话断开。
注:语音广播通知消息除上述流程中通过SIP服务器发出外,也可由语音流发送者发出,消息中通过 To头域标明 作为目的地址的语音流接收者ID,经SIP服务器中转后发往语音流接收者;语音流接收者处理后发送应答消 息,消息中通过 To头域标明作为目的地址的语音流发送者ID,经SIP服务器中转后回复给语音流发送者。后续呼叫流程与上述流程相同。
语音对讲功能实现中心用户与前端用户之间的一对一语音对讲功能。 语音对讲功能由下述两个独立的流程组合实现:
a) 通过9.2的实时视音频点播功能,中心用户获得前端设备的实时视音频媒体流;
b) 通过9.12的语音广播功能,中心用户向前端对讲设备发送实时音频媒体流,语音流的封装格 式见 C.2.4音频流的 RTP封装定义。
C.2.4 音频流的 RTP封装
语音比特流宜采用标准的 RTP协议进行打包,这里只摘录G.711A律的:
在一个 RTP包中,音频载荷数据应为整数个音频编码帧,且时间长度在20ms~180ms之间。
音频载荷数据的 RTP封装参数如下:
a) G.711的主要参数 G.711A律语音编码 RTP包的负载类型(PayloadType)的参数规定如下(见IETFRFC3551— 2003中的表4): 1)负载类型(PT):8; 2) 编码名称(encodingname):PCMA; 3) 时钟频率(clockrate):8kHz; 4) 通道数:1; 5) SDP描述中“m”字段的“media”项:audio。
语音广播接收这块,由于有之前的RTMP和RTSP播放器积累,直接在player端做相应扩展即可,当收到广播后,GB28181语音广播按钮使能。
相关接口设计如下:
/*++++++++++++++++++RTP Receiver++++++++++++++++++++++*/
//GitHub: https://github.com/daniulive/SmarterStreaming
//WebSite: https://daniusdk.com
/*
* 创建RTP Receiver
*
* @param reserve:保留参数传0
*
* @return RTP Receiver 句柄,0表示失败
*/
public native long CreateRTPReceiver(int reserve);
/**
*设置 RTP Receiver传输协议
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param transport_protocol, 0:UDP, 1:TCP, 默认是UDP
*
* @return {0} if successful
*/
public native int SetRTPReceiverTransportProtocol(long rtp_receiver_handle, int transport_protocol);
/**
*设置 RTP Receiver IP地址类型
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param ip_address_type, 0:IPV4, 1:IPV6, 默认是IPV4
*
* @return {0} if successful
*/
public native int SetRTPReceiverIPAddressType(long rtp_receiver_handle, int ip_address_type);
/**
*设置 RTP Receiver RTP Socket本地端口
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param port, 必须是偶数,设置0的话SDK会自动分配, 默认值是0
*
* @return {0} if successful
*/
public native int SetRTPReceiverLocalPort(long rtp_receiver_handle, int port);
/**
*设置 RTP Receiver SSRC
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param ssrc, 如果设置的话,这个字符串要能转换成uint32类型, 否则设置失败
*
* @return {0} if successful
*/
public native int SetRTPReceiverSSRC(long rtp_receiver_handle, String ssrc);
/**
*创建 RTP Receiver 会话
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param reserve, 保留值,目前传0
*
* @return {0} if successful
*/
public native int CreateRTPReceiverSession(long rtp_receiver_handle, int reserve);
/**
*获取 RTP Receiver RTP Socket本地端口
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @return 失败返回0, 成功的话返回响应的端口, 请在CreateRTPReceiverSession返回成功之后调用
*/
public native int GetRTPReceiverLocalPort(long rtp_receiver_handle);
/**
*设置 RTP Receiver Payload 相关信息
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @param payload_type, 请参考 RFC 3551
*
* @param encoding_name, 编码名, 请参考 RFC 3551, 如果payload_type不是动态的, 可能传null就好
*
* @param media_type, 媒体类型, 请参考 RFC 3551, 1 是视频, 2是音频
*
* @param clock_rate, 请参考 RFC 3551
*
* @return {0} if successful
*/
public native int SetRTPReceiverPayloadType(long rtp_receiver_handle, int payload_type, String encoding_name, int media_type, int clock_rate);
/**
*设置 RTP Receiver 音频采样率
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param sampling_rate, 音频采样率
*
* @return {0} if successful
*/
public native int SetRTPReceiverAudioSamplingRate(long rtp_receiver_handle, int sampling_rate);
/**
*设置 RTP Receiver 音频通道数
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param channels, 音频通道数
*
* @return {0} if successful
*/
public native int SetRTPReceiverAudioChannels(long rtp_receiver_handle, int channels);
/**
*设置 RTP Receiver 远端地址
*
* @param rtp_receiver_handle, CreateRTPReceiver
* @param address, IP地址
* @param port, 端口
*
* @return {0} if successful
*/
public native int SetRTPReceiverRemoteAddress(long rtp_receiver_handle, String address, int port);
/**
*初始化 RTP Receiver
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @return {0} if successful
*/
public native int InitRTPReceiver(long rtp_receiver_handle);
/**
*UnInit RTP Receiver
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @return {0} if successful
*/
public native int UnInitRTPReceiver(long rtp_receiver_handle);
/**
*Destory RTP Receiver Session
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @return {0} if successful
*/
public native int DestoryRTPReceiverSession(long rtp_receiver_handle);
/**
*Destory RTP Receiver
*
* @param rtp_receiver_handle, CreateRTPReceiver
*
* @return {0} if successful
*/
public native int DestoryRTPReceiver(long rtp_receiver_handle);
/*++++++++++++++++++RTP Receiver++++++++++++++++++++++*/
相关调用代码:
class ButtonGB28181AudioBroadcastListener implements OnClickListener {
public void onClick(View v) {
if (gb_broadcast_source_id_ != null && gb_broadcast_target_id_ != null) {
if (gb28181_agent_ != null ) {
if (gb28181_agent_.byeAudioBroadcast(gb_broadcast_source_id_, gb_broadcast_target_id_) ) {
gb_broadcast_source_id_ = null;
gb_broadcast_target_id_ = null;
btnGB28181AudioBroadcast.setText("GB28181语音广播");
btnGB28181AudioBroadcast.setEnabled(false);
}
}
}
stopAudioPlayer();
destoryRTPReceiver();
}
}
@Override
public void ntsOnNotifyBroadcastCommand(String fromUserName, String fromUserNameAtDomain, String sn, String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnNotifyBroadcastCommand, fromUserName:"+ from_user_name_ + ", fromUserNameAtDomain:"+ from_user_name_at_domain_
+ ", SN:" + sn_ + ", sourceID:" + source_id_ + ", targetID:" + target_id_);
if (gb28181_agent_ != null ) {
gb28181_agent_.respondBroadcastCommand(from_user_name_, from_user_name_at_domain_,sn_,source_id_, target_id_, true);
btnGB28181AudioBroadcast.setText("收到GB28181语音广播通知");
}
}
private String from_user_name_;
private String from_user_name_at_domain_;
private String sn_;
private String source_id_;
private String target_id_;
public Runnable set(String from_user_name, String from_user_name_at_domain, String sn, String source_id, String target_id) {
this.from_user_name_ = from_user_name;
this.from_user_name_at_domain_ = from_user_name_at_domain;
this.sn_ = sn;
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(fromUserName, fromUserNameAtDomain, sn, sourceID, targetID),0);
}
@Override
public void ntsOnAudioBroadcast(String commandFromUserName, String commandFromUserNameAtDomain, String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnAudioBroadcastPlay, fromFromUserName:" + command_from_user_name_
+ " FromUserNameAtDomain:" + command_from_user_name_at_domain_
+ " sourceID:" + source_id_ + ", targetID:" + target_id_);
stopAudioPlayer();
destoryRTPReceiver();
if (gb28181_agent_ != null ) {
String local_ip_addr = IPAddrUtils.getIpAddress(context_);
boolean is_tcp = true; // 考虑到跨网段, 默认用TCP传输rtp包
rtp_receiver_handle_ = lib_player_.CreateRTPReceiver(0);
if (rtp_receiver_handle_ != 0 ) {
lib_player_.SetRTPReceiverTransportProtocol(rtp_receiver_handle_, is_tcp?1:0);
lib_player_.SetRTPReceiverIPAddressType(rtp_receiver_handle_, 0);
if (0 == lib_player_.CreateRTPReceiverSession(rtp_receiver_handle_, 0) ) {
int local_port = lib_player_.GetRTPReceiverLocalPort(rtp_receiver_handle_);
boolean ret = gb28181_agent_.inviteAudioBroadcast(command_from_user_name_,command_from_user_name_at_domain_,
source_id_, target_id_, "IP4", local_ip_addr, local_port, is_tcp?"TCP/RTP/AVP":"RTP/AVP");
if (!ret ) {
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
else {
btnGB28181AudioBroadcast.setText("GB28181语音广播呼叫中");
}
} else {
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
}
}
}
private String command_from_user_name_;
private String command_from_user_name_at_domain_;
private String source_id_;
private String target_id_;
public Runnable set(String command_from_user_name, String command_from_user_name_at_domain, String source_id, String target_id) {
this.command_from_user_name_ = command_from_user_name;
this.command_from_user_name_at_domain_ = command_from_user_name_at_domain;
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(commandFromUserName, commandFromUserNameAtDomain, sourceID, targetID),0);
}
@Override
public void ntsOnInviteAudioBroadcastException(String sourceID, String targetID, String errorInfo) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnInviteAudioBroadcastException, sourceID:" + source_id_ + ", targetID:" + target_id_);
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}
@Override
public void ntsOnInviteAudioBroadcastTimeout(String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnInviteAudioBroadcastTimeout, sourceID:" + source_id_ + ", targetID:" + target_id_);
destoryRTPReceiver();
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}
@Override
public void ntsOnInviteAudioBroadcastResponse(String sourceID, String targetID, int statusCode, PlaySessionDescription sessionDescription) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnInviteAudioBroadcastResponse, statusCode:" + status_code_ +" sourceID:" + source_id_ + ", targetID:" + target_id_);
boolean is_need_destory_rtp = true;
if (gb28181_agent_ != null ) {
boolean is_need_bye = 200==status_code_;
if (200 == status_code_ && session_description_ != null && rtp_receiver_handle_ != 0 ) {
MediaSessionDescription audio_des = session_description_.getAudioDescription();
SDPRtpMapAttribute audio_attr = null;
if (audio_des != null && audio_des.getRtpMapAttributes() != null && !audio_des.getRtpMapAttributes().isEmpty() )
audio_attr = audio_des.getRtpMapAttributes().get(0);
if ( audio_des != null && audio_attr != null ) {
lib_player_.SetRTPReceiverSSRC(rtp_receiver_handle_, audio_des.getSSRC());
lib_player_.SetRTPReceiverPayloadType(rtp_receiver_handle_, audio_attr.getPayloadType(),
audio_attr.getEncodingName(), 2, audio_attr.getClockRate());
lib_player_.SetRTPReceiverRemoteAddress(rtp_receiver_handle_, audio_des.getAddress(), audio_des.getPort());
lib_player_.InitRTPReceiver(rtp_receiver_handle_);
if (startAudioPlay()) {
is_need_bye = false;
is_need_destory_rtp = false;
gb_broadcast_source_id_ = source_id_;
gb_broadcast_target_id_ = target_id_;
btnGB28181AudioBroadcast.setText("终止GB28181语音广播");
btnGB28181AudioBroadcast.setEnabled(true);
}
}
} else {
btnGB28181AudioBroadcast.setText("GB28181语音广播");
}
if (is_need_bye)
gb28181_agent_.byeAudioBroadcast(source_id_, target_id_);
}
if (is_need_destory_rtp)
destoryRTPReceiver();
}
private String source_id_;
private String target_id_;
private int status_code_;
private PlaySessionDescription session_description_;
public Runnable set(String source_id, String target_id, int status_code, PlaySessionDescription session_description) {
this.source_id_ = source_id;
this.target_id_ = target_id;
this.status_code_ = status_code;
this.session_description_ = session_description;
return this;
}
}.set(sourceID, targetID, statusCode, sessionDescription),0);
}
@Override
public void ntsOnByeAudioBroadcast(String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnByeAudioBroadcast sourceID:" + source_id_ + " targetID:" + target_id_);
gb_broadcast_source_id_ = null;
gb_broadcast_target_id_ = null;
btnGB28181AudioBroadcast.setText("GB28181语音广播");
btnGB28181AudioBroadcast.setEnabled(false);
stopAudioPlayer();
destoryRTPReceiver();
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}
@Override
public void ntsOnTerminateAudioBroadcast(String sourceID, String targetID) {
handler_.postDelayed(new Runnable() {
@Override
public void run() {
Log.i(TAG, "ntsOnTerminateAudioBroadcast sourceID:" + source_id_ + " targetID:" + target_id_);
gb_broadcast_source_id_ = null;
gb_broadcast_target_id_ = null;
btnGB28181AudioBroadcast.setText("GB28181语音广播");
btnGB28181AudioBroadcast.setEnabled(false);
stopAudioPlayer();
destoryRTPReceiver();
}
private String source_id_;
private String target_id_;
public Runnable set(String source_id, String target_id) {
this.source_id_ = source_id;
this.target_id_ = target_id;
return this;
}
}.set(sourceID, targetID),0);
}
至此、Android平台GB28181接入终端,如位置订阅、语音广播和语音对讲这块已经全面覆盖,加上之前的技术积累,看了下,已覆盖了以下部分:
特别是语音广播和语音对讲这块,是GB28181终端接入模块的一个核心扩展功能,在智能门禁、工业与物联网、监控等行业,用途非常广泛,技术实现这块,不要忽略的技术点还有降噪和回音消除这块,由于之前我们有技术积累,可以直接复用,对新入手的开发者来说,也提出了新的挑战,感兴趣的开发者,可以酌情参考。