1、worker
2、router
3、producer
4、consumer
5、transport
1、支持IPv6
2、ICE/DRLS/RTP/RTCP over UDP and TCP
3、支持simulcast和svc
4、支持拥塞控制
5、带宽控制
6、支持:STCP
7、多流使用同一个ICE+DTLS传输通道
8、性能高
├── AudioLevelObserver.js #音频音量
├── Channel.js #用于和c++部分信令通信
├── Consumer.js #消费媒体数据
├── DataConsumer.js
├── DataProducer.js
├── DirectTransport.js
├── EnhancedEventEmitter.js #事件处理
├── Logger.js#日志
├── PayloadChannel.js
├── PipeTransport.js#router之间数据转发
├── PlainTransport.js#普通传输通道
├── Producer.js#生产媒体数据
├── Router.js#代表一个房间(路由)
├── RtpObserver.js#rtp的观察者
├── RtpParameters.js
├── SctpParameters.js
├── SrtpParameters.js
├── Transport.js#传输通道基类
├── WebRtcTransport.js#webrtc的传输通道
├── Worker.js#c++工作进程,节点
├── errors.js#错误信息
├── index.js#库索引
├── ortc.js
├── scalabilityModes.js
├── supportedRtpCapabilities.js#支持的RTP能力
├── types.js
└── utils.js #工具函数
1、起到管理作用
2、生产json传给C++层
例子:
先祭出mediasoup的架构图,可看ffmpeg/gstreamer可作为推流或接受流的工具,媒体流被推至mediasoup后可由router分发到其他客户端。
由于mediasoup是SFU模型的流媒体服务器,实际上它很适合做媒体数据转发工作。
脚本中自带说明:
关闭https校验,不然443端口会校验。
ffmpeg.sh 中修改HTTPIE_COMMAND="http --verify=no --check-status"
执行脚本
SERVER_URL=https://192.168.1.110:4443 ROOM_ID=akkb2kdu MEDIA_FILE=./../app/resources/videos/video-audio-stereo.mp4 ./ffmpeg.sh
flutter run -d web-server
MediaSoup通信流程
MediaSoup通信流程 | ProcessOn免费在线作图,在线流程图,在线思维导图 |
vlc是可以同时播放多路流媒体的,我是同时播放了视频264音频g711rtp流
想只用rtp播放音视频流是没法做到同步的,必须结合rtcp
播放多路流媒体的时候只要把sdp文件结合一下就OK了
我是用dm365发送音视频流,pc vlc接收,程序中添加的rtp头,没使用jrtplib
————————————————
版权声明:本文为CSDN博主「zhangjikuan」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:RTP发送音视频流vlc播放_zhangjikuan的博客-CSDN博客
1、构成
app应用:提供客户端所需要的应用代码
broadcasters:用于广播使用,用于推流的模块。单向传输,只有去或者只有回
server端:信令服务和媒体流服务,两者通过管道通信。细分为下面几部分:
--->config.js:配置文件/js文件,通过js获取一些基本信息。将配置信息交给servere.js使用
--->server.js:从config.js中去获取基本信息,获取信息之后去启动基本服务,比如wensocket服务、信令服务
--->lib:server.js使用的库文件,细分为以下几部分
------->Room.js:所有的真正的信令处理逻辑都是在这里实现,还描述了房间相关信息
------->interactiveClient.js:运行时内部信息查询客户端,与客户端交互(debug使用)
------->interactiveServer.js:运行时内部信息查询服务端,与服务端交换(debug使用)
mediasoup C++:C++部分,用于处理流媒体传输,包括lib与worker两部分
--->lib:一些js文件组成,主要用于对mediasoup的管理工作
--->worker:C++核心代码
cert:证书及密钥
DEBUG=${DEBUG:='*mediasoup* *INFO* *WARN* *ERROR*'} INTERACTIVE=${INTERACTIVE:='true'} node server.js
process.env.DEBUG: *mediasoup* *INFO* *WARN* *ERROR*
config.js:{...}
mediasoup-demo-server:INFO running 8 mediasoup Workers... +0ms
mediasoup createWorker() +0ms
mediasoup:Worker constructor() +0ms
mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
mediasoup:Channel constructor() +0ms
mediasoup:PayloadChannel constructor() +0ms
[opening Readline Command Console...]
type help to print available commands
cmd> mediasoup:Worker worker process running [pid:1414] +28ms
mediasoup createWorker() +29ms
mediasoup:Worker constructor() +1ms
mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
mediasoup:Channel constructor() +28ms
mediasoup:PayloadChannel constructor() +28ms
mediasoup:Worker worker process running [pid:1415] +7ms
mediasoup createWorker() +7ms
mediasoup:Worker constructor() +0ms
mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
mediasoup:Channel constructor() +7ms
mediasoup:PayloadChannel constructor() +7ms
mediasoup:Worker worker process running [pid:1416] +7ms
mediasoup createWorker() +7ms
mediasoup:Worker constructor() +0ms
mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
mediasoup:Channel constructor() +7ms
mediasoup:PayloadChannel constructor() +7ms
mediasoup:Worker worker process running [pid:1417] +7ms
mediasoup createWorker() +7ms
mediasoup:Worker constructor() +0ms
mediasoup-demo server端的启动过程
::run();
async function runMediasoupWorkers()
export async function createWorker(
const worker = new Worker(
{
logLevel,
logTags,
rtcMinPort,
rtcMaxPort,
dtlsCertificateFile,
dtlsPrivateKeyFile,
appData
});
spawn //使用指定的命令行参数创建新进程
export PATH_TO_LIBWEBRTC_SOURCES="/Users/uianster/learn/webrtc/webrtc_ios/src"
export PATH_TO_LIBWEBRTC_BINARY="/Users/uianster/learn/webrtc/webrtc_ios/src/out/mac/obj"
export PATH_TO_OPENSSL_HEADERS="/usr/local/openssl/include"
安装oenssl
Mac上安装OpenSSL - 知乎
export OPENSSL_ROOT_DIR="/usr/local/openssl"
mediasoup流程图_wzw88486969的博客-CSDN博客_mediasoup流程
设备:MacBook Air (M1, 2020)
系统:macOS Big Bur
nodejs版本:
$ git clone https://github.com.cnpmjs.org/versatica/mediasoup-demo.git
#先下载server所需要的nodejs依赖包,依赖都在package.json中指定,但是由于相互依赖其他包的原因,实际上下载到
#当前目录下node_modules中(比package.json中指定的的包不少)
$ cd mediasoup-demo
$ git checkout v3
$ cd server
#这里使用yarn来安装nojs所需要的包,使用npm install满的要死还各种问题
$ yarn install
#由于墙的关系mediasoup下的速度较慢,可以使用yarn指定git路径进行下载
#因此使用进行加速下载
github.com.cnpmjs.org替换github.com后可实现加速下载
#指定下载mediasoup
yarn add https://github.com.cnpmjs.org/versatica/mediasoup.git
#再跑一遍,避免有的依赖包没有加上
yarn install
#如下则依赖安装安好
yarn install v1.22.10
[1/4] Resolving packages...
success Already up-to-date.
✨ Done in 0.15s.
到app中:yarn start
ubuntu@VM-0-4-ubuntu:~/learn/mediasoup-demo/app$ yarn
00h00m00s 0/0: : ERROR: There are no scenarios; must have at least one.
javascript - Yarn ERROR: There are no scenarios; must have at least one - Stack Overflow
ubuntu@VM-0-4-ubuntu:~/learn/mediasoup-demo/app$ yarn
yarn install v1.22.5
info No lockfile found.
[1/4] Resolving packages...
error An unexpected error occurred: "https://github.com/versatica/mediasoup-client: read ECONNRESET".
info If you think this is a bug, please open a bug report with the information provided in "/home/ubuntu/learn/mediasoup-demo/app/yarn-error.log".
info Visit yarn install | Yarn for documentation about this command.
单独下载
yarn add https://github.com.cnpmjs.org/versatica/mediasoup-client
error [email protected]: The engine "node" is incompatible with this module. Expected version ">=12". Got "10.19.0"
yarn config set ignore-engines true
/**
* IMPORTANT (PLEASE READ THIS):
*
* This is not the "configuration file" of mediasoup. This is the configuration
* file of the mediasoup-demo app. mediasoup itself is a server-side library, it
* does not read any "configuration file". Instead it exposes an API. This demo
* application just reads settings from this file (once copied to config.js) and
* calls the mediasoup API with those settings when appropriate.
*/
const os = require('os');
module.exports =
{
// Listening hostname (just for `gulp live` task).
domain : process.env.DOMAIN || 'localhost',
// Signaling settings (protoo WebSocket server and HTTP API server).
https :
{
listenIp : '0.0.0.0', //设置为0
// NOTE: Don't change listenPort (client app assumes 4443).
listenPort : process.env.PROTOO_LISTEN_PORT || 4443,
// NOTE: Set your own valid certificate files.
tls :
{
cert : process.env.HTTPS_CERT_FULLCHAIN || `${__dirname}/certs/1_cuican.tech_bundle.crt`,
key : process.env.HTTPS_CERT_PRIVKEY || `${__dirname}/certs/2_cuican.tech.key`
}
},
// mediasoup settings.
mediasoup :
{
// Number of mediasoup workers to launch.
numWorkers : Object.keys(os.cpus()).length,
// mediasoup WorkerSettings.
// See https://mediasoup.org/documentation/v3/mediasoup/api/#WorkerSettings
workerSettings :
{
logLevel : 'warn',
logTags :
[
'info',
'ice',
'dtls',
'rtp',
'srtp',
'rtcp',
'rtx',
'bwe',
'score',
'simulcast',
'svc',
'sctp'
],
rtcMinPort : process.env.MEDIASOUP_MIN_PORT || 40000,
rtcMaxPort : process.env.MEDIASOUP_MAX_PORT || 49999
},
// mediasoup Router options.
// See https://mediasoup.org/documentation/v3/mediasoup/api/#RouterOptions
routerOptions :
{
mediaCodecs :
[
{
kind : 'audio',
mimeType : 'audio/opus',
clockRate : 48000,
channels : 2
},
{
kind : 'video',
mimeType : 'video/VP8',
clockRate : 90000,
parameters :
{
'x-google-start-bitrate' : 1000
}
},
{
kind : 'video',
mimeType : 'video/VP9',
clockRate : 90000,
parameters :
{
'profile-id' : 2,
'x-google-start-bitrate' : 1000
}
},
{
kind : 'video',
mimeType : 'video/h264',
clockRate : 90000,
parameters :
{
'packetization-mode' : 1,
'profile-level-id' : '4d0032',
'level-asymmetry-allowed' : 1,
'x-google-start-bitrate' : 1000
}
},
{
kind : 'video',
mimeType : 'video/h264',
clockRate : 90000,
parameters :
{
'packetization-mode' : 1,
'profile-level-id' : '42e01f',
'level-asymmetry-allowed' : 1,
'x-google-start-bitrate' : 1000
}
}
]
},
// mediasoup WebRtcTransport options for WebRTC endpoints (mediasoup-client,
// libmediasoupclient).
// See https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtcTransportOptions
webRtcTransportOptions :
{
listenIps :
[
{
ip : process.env.MEDIASOUP_LISTEN_IP || '172.17.0.4',//内网
announcedIp : process.env.MEDIASOUP_ANNOUNCED_IP|| '121.5.142.187'//外网
}
],
initialAvailableOutgoingBitrate : 1000000,
minimumAvailableOutgoingBitrate : 600000,
maxSctpMessageSize : 262144,
// Additional options that are not part of WebRtcTransportOptions.
maxIncomingBitrate : 1500000
},
// mediasoup PlainTransport options for legacy RTP endpoints (FFmpeg,
// GStreamer).
// See https://mediasoup.org/documentation/v3/mediasoup/api/#PlainTransportOptions
plainTransportOptions :
{
listenIp :
{
ip : process.env.MEDIASOUP_LISTEN_IP || '1.2.3.4',
announcedIp : process.env.MEDIASOUP_ANNOUNCED_IP
},
maxSctpMessageSize : 262144
}
}
};
服务器:腾讯云 1核2G
期间会出现问题,能百度到的就不是问题,没必要记录了。
最后app启动后会弹出页面,但是是内网的需要手动该成服务器公网ip
特别说明:最好使用chrome进行测试,但是第一次加载非常满需要耐心。
https://blog.csdn.net/frodocheng/article/details/107118826?spm=1001.2101.3001.6650.1&utm_medium=distribute.pc_relevant.none-task-blog-2~default~CTRLIST~default-1.no_search_link&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2~default~CTRLIST~default-1.no_search_link(MediaSoup MediaSoup-demo v3 版本部署测试)
https://blog.csdn.net/u014338577/article/details/109682735?utm_medium=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~default-4.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~default-4.control