WebRTC服务器Mediasoup学习与实践

一、mediasoup基本概念:

基础架构


1、worker
2、router
3、producer
4、consumer
5、transport

整体结构

WebRTC服务器Mediasoup学习与实践_第1张图片

特性

1、支持IPv6
2、ICE/DRLS/RTP/RTCP over UDP and TCP
3、支持simulcast和svc
4、支持拥塞控制
5、带宽控制
6、支持:STCP
7、多流使用同一个ICE+DTLS传输通道
8、性能高

二、mediasoup库JS部分

├── AudioLevelObserver.js #音频音量
├── Channel.js            #用于和c++部分信令通信
├── Consumer.js								#消费媒体数据
├── DataConsumer.js       
├── DataProducer.js
├── DirectTransport.js
├── EnhancedEventEmitter.js  #事件处理
├── Logger.js#日志
├── PayloadChannel.js
├── PipeTransport.js#router之间数据转发
├── PlainTransport.js#普通传输通道
├── Producer.js#生产媒体数据
├── Router.js#代表一个房间(路由)
├── RtpObserver.js#rtp的观察者
├── RtpParameters.js
├── SctpParameters.js
├── SrtpParameters.js
├── Transport.js#传输通道基类
├── WebRtcTransport.js#webrtc的传输通道
├── Worker.js#c++工作进程,节点
├── errors.js#错误信息
├── index.js#库索引
├── ortc.js
├── scalabilityModes.js
├── supportedRtpCapabilities.js#支持的RTP能力
├── types.js
└── utils.js #工具函数

WebRTC服务器Mediasoup学习与实践_第2张图片

medisoup的JS的作用

1、起到管理作用
2、生产json传给C++层
例子:

讨论mediasoup是否支持ffmpeg推流

先祭出mediasoup的架构图,可看ffmpeg/gstreamer可作为推流或接受流的工具,媒体流被推至mediasoup后可由router分发到其他客户端。
由于mediasoup是SFU模型的流媒体服务器,实际上它很适合做媒体数据转发工作。

先撸mediasoup官方提供的ffmpeg.sh

脚本中自带说明:

关闭https校验,不然443端口会校验。
ffmpeg.sh 中修改HTTPIE_COMMAND="http --verify=no --check-status"

执行脚本
SERVER_URL=https://192.168.1.110:4443 ROOM_ID=akkb2kdu MEDIA_FILE=./../app/resources/videos/video-audio-stereo.mp4 ./ffmpeg.sh

flutter run -d web-server

MediaSoup通信流程
MediaSoup通信流程 | ProcessOn免费在线作图,在线流程图,在线思维导图 |

ffmpng vlc 推流 ——》webrtc播放

vlc是可以同时播放多路流媒体的,我是同时播放了视频264音频g711rtp流

想只用rtp播放音视频流是没法做到同步的,必须结合rtcp

播放多路流媒体的时候只要把sdp文件结合一下就OK了

我是用dm365发送音视频流,pc vlc接收,程序中添加的rtp头,没使用jrtplib
————————————————
版权声明:本文为CSDN博主「zhangjikuan」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:RTP发送音视频流vlc播放_zhangjikuan的博客-CSDN博客

1、构成

WebRTC服务器Mediasoup学习与实践_第3张图片

app应用:提供客户端所需要的应用代码

broadcasters:用于广播使用,用于推流的模块。单向传输,只有去或者只有回

server端:信令服务和媒体流服务,两者通过管道通信。细分为下面几部分:

--->config.js:配置文件/js文件,通过js获取一些基本信息。将配置信息交给servere.js使用

--->server.js:从config.js中去获取基本信息,获取信息之后去启动基本服务,比如wensocket服务、信令服务

--->lib:server.js使用的库文件,细分为以下几部分

------->Room.js:所有的真正的信令处理逻辑都是在这里实现,还描述了房间相关信息

------->interactiveClient.js:运行时内部信息查询客户端,与客户端交互(debug使用)

------->interactiveServer.js:运行时内部信息查询服务端,与服务端交换(debug使用)

mediasoup C++:C++部分,用于处理流媒体传输,包括lib与worker两部分

--->lib:一些js文件组成,主要用于对mediasoup的管理工作

--->worker:C++核心代码
cert:证书及密钥

DEBUG=${DEBUG:='*mediasoup* *INFO* *WARN* *ERROR*'} INTERACTIVE=${INTERACTIVE:='true'} node server.js
process.env.DEBUG: *mediasoup* *INFO* *WARN* *ERROR*
config.js:{...}
  mediasoup-demo-server:INFO running 8 mediasoup Workers... +0ms
  mediasoup createWorker() +0ms
  mediasoup:Worker constructor() +0ms
      mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
  mediasoup:Channel constructor() +0ms
  mediasoup:PayloadChannel constructor() +0ms

[opening Readline Command Console...]
type help to print available commands
cmd>   mediasoup:Worker worker process running [pid:1414] +28ms
  mediasoup createWorker() +29ms
  mediasoup:Worker constructor() +1ms
  mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
  mediasoup:Channel constructor() +28ms
  mediasoup:PayloadChannel constructor() +28ms
  mediasoup:Worker worker process running [pid:1415] +7ms
  mediasoup createWorker() +7ms
  mediasoup:Worker constructor() +0ms
  mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
  mediasoup:Channel constructor() +7ms
  mediasoup:PayloadChannel constructor() +7ms
  mediasoup:Worker worker process running [pid:1416] +7ms
  mediasoup createWorker() +7ms
  mediasoup:Worker constructor() +0ms
  mediasoup:Worker spawning worker process: /Users/uianster/Nutstore Files/.symlinks/坚果云/learn/mediasoup-demo/server/node_modules/mediasoup/worker/out/Release/mediasoup-worker --logLevel=warn --logTag=info --logTag=ice --logTag=dtls --logTag=rtp --logTag=srtp --logTag=rtcp --logTag=rtx --logTag=bwe --logTag=score --logTag=simulcast --logTag=svc --logTag=sctp --rtcMinPort=40000 --rtcMaxPort=49999 +0ms
  mediasoup:Channel constructor() +7ms
  mediasoup:PayloadChannel constructor() +7ms
  mediasoup:Worker worker process running [pid:1417] +7ms
  mediasoup createWorker() +7ms
  mediasoup:Worker constructor() +0ms

mediasoup-demo server端的启动过程

::run();
async function runMediasoupWorkers()

export async function createWorker(

const worker = new Worker(
{
logLevel,
logTags,
rtcMinPort,
rtcMaxPort,
dtlsCertificateFile,
dtlsPrivateKeyFile,
appData
});

spawn //使用指定的命令行参数创建新进程

如何使用wireshark抓包

export PATH_TO_LIBWEBRTC_SOURCES="/Users/uianster/learn/webrtc/webrtc_ios/src"

export PATH_TO_LIBWEBRTC_BINARY="/Users/uianster/learn/webrtc/webrtc_ios/src/out/mac/obj"

export PATH_TO_OPENSSL_HEADERS="/usr/local/openssl/include"

安装oenssl
Mac上安装OpenSSL - 知乎

需要指定一下oepngl的root路径,不然还是找不到

export OPENSSL_ROOT_DIR="/usr/local/openssl"

mediasoup流程图_wzw88486969的博客-CSDN博客_mediasoup流程

本地环境

设备:MacBook Air (M1, 2020)
系统:macOS Big Bur

克隆官方demo

nodejs版本:
$ git clone https://github.com.cnpmjs.org/versatica/mediasoup-demo.git
#先下载server所需要的nodejs依赖包,依赖都在package.json中指定,但是由于相互依赖其他包的原因,实际上下载到
#当前目录下node_modules中(比package.json中指定的的包不少)
$ cd mediasoup-demo
$ git checkout v3
$ cd server
#这里使用yarn来安装nojs所需要的包,使用npm install满的要死还各种问题
$ yarn install
#由于墙的关系mediasoup下的速度较慢,可以使用yarn指定git路径进行下载
#因此使用进行加速下载
github.com.cnpmjs.org替换github.com后可实现加速下载
#指定下载mediasoup
yarn add https://github.com.cnpmjs.org/versatica/mediasoup.git

#再跑一遍,避免有的依赖包没有加上
yarn install
#如下则依赖安装安好

yarn install v1.22.10
[1/4]   Resolving packages...
success Already up-to-date.
✨  Done in 0.15s.

sever运行步骤

  1. 修改配置文件
  2. 将config.simple.js 修改为config.js
  3. IP改为局域网本机器IP(别改漏了)
  4. 生成密钥----参考
    5.yarn start运行sever

app运行

到app中:yarn start

常见错误

yarn错误版本

ubuntu@VM-0-4-ubuntu:~/learn/mediasoup-demo/app$ yarn
00h00m00s 0/0: : ERROR: There are no scenarios; must have at least one.
javascript - Yarn ERROR: There are no scenarios; must have at least one - Stack Overflow

部分依赖被墙了

ubuntu@VM-0-4-ubuntu:~/learn/mediasoup-demo/app$ yarn
yarn install v1.22.5
info No lockfile found.
[1/4] Resolving packages...
error An unexpected error occurred: "https://github.com/versatica/mediasoup-client: read ECONNRESET".
info If you think this is a bug, please open a bug report with the information provided in "/home/ubuntu/learn/mediasoup-demo/app/yarn-error.log".
info Visit yarn install | Yarn for documentation about this command.
单独下载
yarn add https://github.com.cnpmjs.org/versatica/mediasoup-client

兼容性忽略

error [email protected]: The engine "node" is incompatible with this module. Expected version ">=12". Got "10.19.0"
yarn config set ignore-engines true

mediasoup公网部署

/**
 * IMPORTANT (PLEASE READ THIS):
 *
 * This is not the "configuration file" of mediasoup. This is the configuration
 * file of the mediasoup-demo app. mediasoup itself is a server-side library, it
 * does not read any "configuration file". Instead it exposes an API. This demo
 * application just reads settings from this file (once copied to config.js) and
 * calls the mediasoup API with those settings when appropriate.
 */

const os = require('os');

module.exports =
{
    // Listening hostname (just for `gulp live` task).
    domain : process.env.DOMAIN || 'localhost',
    // Signaling settings (protoo WebSocket server and HTTP API server).
    https  :
    {
        listenIp   : '0.0.0.0', //设置为0
        // NOTE: Don't change listenPort (client app assumes 4443).
        listenPort : process.env.PROTOO_LISTEN_PORT || 4443,
        // NOTE: Set your own valid certificate files.
        tls        :
        {
            cert : process.env.HTTPS_CERT_FULLCHAIN || `${__dirname}/certs/1_cuican.tech_bundle.crt`,
            key  : process.env.HTTPS_CERT_PRIVKEY || `${__dirname}/certs/2_cuican.tech.key`
        }
    },
    // mediasoup settings.
    mediasoup :
    {
        // Number of mediasoup workers to launch.
        numWorkers     : Object.keys(os.cpus()).length,
        // mediasoup WorkerSettings.
        // See https://mediasoup.org/documentation/v3/mediasoup/api/#WorkerSettings
        workerSettings :
        {
            logLevel : 'warn',
            logTags  :
            [
                'info',
                'ice',
                'dtls',
                'rtp',
                'srtp',
                'rtcp',
                'rtx',
                'bwe',
                'score',
                'simulcast',
                'svc',
                'sctp'
            ],
            rtcMinPort : process.env.MEDIASOUP_MIN_PORT || 40000,
            rtcMaxPort : process.env.MEDIASOUP_MAX_PORT || 49999
        },
        // mediasoup Router options.
        // See https://mediasoup.org/documentation/v3/mediasoup/api/#RouterOptions
        routerOptions :
        {
            mediaCodecs :
            [
                {
                    kind      : 'audio',
                    mimeType  : 'audio/opus',
                    clockRate : 48000,
                    channels  : 2
                },
                {
                    kind       : 'video',
                    mimeType   : 'video/VP8',
                    clockRate  : 90000,
                    parameters :
                    {
                        'x-google-start-bitrate' : 1000
                    }
                },
                {
                    kind       : 'video',
                    mimeType   : 'video/VP9',
                    clockRate  : 90000,
                    parameters :
                    {
                        'profile-id'             : 2,
                        'x-google-start-bitrate' : 1000
                    }
                },
                {
                    kind       : 'video',
                    mimeType   : 'video/h264',
                    clockRate  : 90000,
                    parameters :
                    {
                        'packetization-mode'      : 1,
                        'profile-level-id'        : '4d0032',
                        'level-asymmetry-allowed' : 1,
                        'x-google-start-bitrate'  : 1000
                    }
                },
                {
                    kind       : 'video',
                    mimeType   : 'video/h264',
                    clockRate  : 90000,
                    parameters :
                    {
                        'packetization-mode'      : 1,
                        'profile-level-id'        : '42e01f',
                        'level-asymmetry-allowed' : 1,
                        'x-google-start-bitrate'  : 1000
                    }
                }
            ]
        },
        // mediasoup WebRtcTransport options for WebRTC endpoints (mediasoup-client,
        // libmediasoupclient).
        // See https://mediasoup.org/documentation/v3/mediasoup/api/#WebRtcTransportOptions
        webRtcTransportOptions :
        {
            listenIps :
            [
                {
                    ip          : process.env.MEDIASOUP_LISTEN_IP || '172.17.0.4',//内网
                    announcedIp : process.env.MEDIASOUP_ANNOUNCED_IP|| '121.5.142.187'//外网
                }
            ],
            initialAvailableOutgoingBitrate : 1000000,
            minimumAvailableOutgoingBitrate : 600000,
            maxSctpMessageSize              : 262144,
            // Additional options that are not part of WebRtcTransportOptions.
            maxIncomingBitrate              : 1500000
        },
        // mediasoup PlainTransport options for legacy RTP endpoints (FFmpeg,
        // GStreamer).
        // See https://mediasoup.org/documentation/v3/mediasoup/api/#PlainTransportOptions
        plainTransportOptions :
        {
            listenIp :
            {
                ip          : process.env.MEDIASOUP_LISTEN_IP || '1.2.3.4',
                announcedIp : process.env.MEDIASOUP_ANNOUNCED_IP
            },
            maxSctpMessageSize : 262144
        }
    }
};

服务器:腾讯云 1核2G

期间会出现问题,能百度到的就不是问题,没必要记录了。
最后app启动后会弹出页面,但是是内网的需要手动该成服务器公网ip

特别说明:最好使用chrome进行测试,但是第一次加载非常满需要耐心。

https://blog.csdn.net/frodocheng/article/details/107118826?spm=1001.2101.3001.6650.1&utm_medium=distribute.pc_relevant.none-task-blog-2~default~CTRLIST~default-1.no_search_link&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2~default~CTRLIST~default-1.no_search_link(MediaSoup MediaSoup-demo v3 版本部署测试)

https://blog.csdn.net/u014338577/article/details/109682735?utm_medium=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~default-4.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2~default~BlogCommendFromBaidu~default-4.control

你可能感兴趣的:(流媒体知识,ffmpeg)