译：构建音视频直播应用需要考虑的12件事
问题背景：
近期看到一篇音视频技术周刊的一篇文章《12 Things to Consider When Building a Live Streaming App》，感觉写的还不错，是Red5官网挂出来的一篇博客，对设计、构建一个流媒体平台系统有一定的参考意义。这里简单翻译一下，以飨读者。其中Red5是一个采用Java开发的Flash流媒体服务器，与之对标的有Nginx-Rtmp、SRS和FMS等。
打造一个流媒体平台除了像一般后端服务要求那样，比如性能、扩展性、可维护性以及可测试性，还有其它音视频技术方面的考虑。作者大体从12个方面阐述了需要考虑的因素，同时我对每段意思做了言简意赅的概括，英语比较low，所以原文还是贴出来，防止有时因为翻译不准误人子弟。
Building a live streaming application requires many moving pieces. Not only is the process of live streaming complicated, but the number of companies offering to help “simplify” the process can have the opposite effect.
Let’s take a look at some of the key features you should consider in choosing a platform to build your live streaming application on.
译：
构建一个实时流媒体应用需要很多活动组件，直播过程不仅仅复杂而且许多公司为了能简化这个过程，但是实际却起到了相反的效果。
现在让我们看看你在选择平台构建你的直播应用时需要考虑的关键特性。
了解更多关于音视频&IOT，欢迎关注公众号：智媒黑板报
1) Latency
Having the lowest possible latency is a keystone to live experiences.
Live auctions, drone guidance, event broadcasts, even basic conversations all need real-time latency. Any sort of delay between the broadcaster and subscriber will have negative consequences. High latency results in staggered conversations, spoilers and a general sense that the live event is not actually live.
In order to achieve real-time latency, you must use protocols that have been developed specifically for low latency. For browser-based apps WebRTC is really your only choice to get near real-time latency. WebRTC when handled correctly consistently achieves under 500 milliseconds of latency. That is fast enough to be nearly imperceptible.
Other efforts to lower browser based video latency including Low Latency HLS, CMAF, Apple Low Latency HLS and WebSockets and Media Source Extensions have all fallen short of delivering actual real-time latency.
For more on how WebRTC provides such low latency, check out this blog post.
If it’s a native mobile app you are building you have a few more options. RTSP is a very good protocol for mobile because the underlying video transport is RTP or its secure cousin SRTP, which also happens to be exactly what WebRTC uses. RTSP also happens to be a much simpler protocol than WebRTC better geared towards a client server model. Thus RTSP typically allows for more connections on your server endpoint making your server infrastructure scale with lower costs.
While the rest of this post talks mostly about streaming solutions based on WebRTC, the same exact principles apply to RTSP based video streaming apps.
译：
对于直播体验最关键的就是要实现低延时。实时拍卖，无人机导航，事件广播，甚至最基本的多人对话场景都需要低延时。广播公司和和订阅者任何形式的延时都会产生多多少少的负面影响，对于多人对话，电视直播以及现场事件场景，高延时都会带来不在现场的那种感觉。
为了实现低延时，你需要使用专门为低延时开发的流媒体传输协议。基于浏览器的应用程序，WebRTC几乎是实现低延时通信的唯一选择。如果能正确的使用WebRTC，那你可以实现低于500ms的延迟，这种速度快得让你几乎觉察不到。
像其它为了实现基于浏览器低延迟通信的协议，比如低延时HLS、CMAF和WebSocket，其实都没有真正实现实时通信。
如果你要创建一个移动APP应用，那你的选择显得就比较多。RTSP就是一种非常好的移动端低延时实现协议，因为该协议底层用到的是RTP或者跟它相近的SRTP，这种协议也被WebRTC所使用。对于客户端-服务端模型，RTSP恰好是比WebRTC更的简单的协议，同时RTSP通常允许服务器端可以有更多的链接，这样可以花费很小的成本实现扩展。
这篇文章其余部分主要讨论基于WebRTC的流媒体方案，当然讨论的原则也适用于基于RTSP流媒体方案的应用程序。
注：
音视频系统根据延时敏感性可以分为三挡：第一挡要求音视频延时一般低于800ms,视频会议，远程教育都是这类代表。这类一般用的音视频分发协议基于UDP，基于TCP很难做到这么低的延时。其中WebRTC方案和RTP协议是这类的代表。第二挡要求延时在1-5s,这类应用延时容忍度稍微高点，基于TCP的分发协议诸如RTMP、HLS和Dash都是可以的。最后一档就是不对延时有啥要求，一般的点播应用。这种像优酷爱奇艺和B站都是这种。对延时的敏感性决定了不同应用背后的音视频技术栈都不一样，设计流媒体直播应用需要考虑的因素也不一样。
2) Scalability
Equally important to having a low latency stream is the ability to send that out to as many people as needed. Importantly, it needs to maintain that low latency.
Some streaming platforms like Vonage Video API (formerly TokBox) switch to higher latency protocols after they meet a certain threshold, meaning that those additional viewers won’t have the same experience as those that joined earlier. While that allows them to achieve scalability, it comes with the tradeoff of high latency and a poor user experience.
However, some platforms based on traditional CDN infrastructure use HTTP based high latency protocols by default. That means none of their users get low latency at all.
In order to provide the best user experience, the challenge lies in scaling out to large audiences while maintaining real-time latency for the best user experience.
One way to solve this is by leveraging cloud networks to spin up new compute instances to create a cluster of server nodes that deliver content without the caching mechanisms that CDNs rely on. Using this approach the platform can scale to millions of concurrent users while maintaining sub 500 milliseconds of latency. This WebRTC based delivery design creates an opportunity to broadcast large sports events, concerts or other events making a much more interactive experience possible.
译：
和低延时同样重要的的是能够将音视频尽可能多的及时分发需要它的用户，同时还要保持低延时。
一些流媒体平台，比如Vonage Video API(以前的 ToKBox)，在并发量达到设定的阈值后，会自动将分发协议切换成高延时的协议，这意味着那些超过阈值设定值后来加入的用户将不会用户和加入之前用户的体验。虽然这允许他们实现了伸缩性，但是带来的是高延迟和糟糕的用户体验。
然而基于传统CDN基础设施平台虽然拥有很高的并发能力，但是他们默认基于HTTP的高延迟协议，这意味着他们的用户也根本没有获得低延迟。
为了提供最佳的用户体验，这个挑战在于向大量用户提供流媒体服务时还能保持最佳的实时低延迟。
解决这个问题的一种方法是利用云计算启动新的计算实例，创建一个服务器集群。这些节点不使用CDN缓存机制来交付内容。使用这种方法可以将平台的并发量扩展到百万级别，同时保证了延时不低于500ms。这种基于WebRTC 的交付设计可以为大型体育赛事、音乐会以及其它体育活动创造大量机会使得很多交互式体验成为可能。
注：流媒体平台单单只关注低延时还不行，还必须关注高并发。只有做到大量并发情况下系统依然能保障低延时才有意义。基于传统的CDN缓存方案，提升并发量靠谱但是不能保证低延时，这里可用的方案比如WebRTC还是充满了希望。
3) Efficiency
Your live streaming app needs to work efficiently so your users are not waiting around for the back end infrastructure to setup everything appropriately. Scaling out is great but without streamlining the CPU consumption of the live streams your network could be bloated with many unnecessary servers.
There also needs to be stable connection logic to ensure that clients and broadcasters are connected is a way that works well. Additional features such as relays can help optimize streaming performance as well.
Furthermore, efficient coding will result in compact streams which will streamline data delivery. This enables more streams per server and reduces the need to run hordes of servers to support large events with lots of connections.
译：
你的实时流媒体应用程序需要高效工作，这样你的用户就不需要做太多和后端基础设施不匹配带来的工作。即使扩展性很好，但是如果不减少实时流的CPU消耗，你的网络最终会变得异常臃肿。
除此之外，你的流媒体系统还需要稳定的链接逻辑以确保客户端和广播良好工作，另外像一些转分发技术也可以提升流媒体平台的稳定性。
除此之外，高效率的编码也很关键，这样会以更快的速度实现流媒体数据的在线交付。这样做还可以增加每台服务器传输流的数量，也意味着可以减少因为需要大量链接而运行的服务器。
注：
实现一个高效率的流媒体系统应该从三方面优化，一个CPU的消耗，其次利用一些转分发技术提升链接的稳定性，最后还要采用高效率的编码方案。
其实低延时、扩展性和成本效率本来就是三角制约关系，一般我们要根据特殊的应用场景，进行取舍来完成流媒体系统的开发。
4) Portability
Thriving within the tech sector requires agility. Beyond responding to evolving customer expectations, there is a need to address changes within the structure of your application.
Platforms that you used to build your application can change from internal company decisions to drop features, an external merger that changes pricing, or outright going out of business. The ability to proactively respond to these changes can make or break an application, as being locked-in to a particular platform can have dire consequences.
A hosting agnostic solution with a flexible API is preferable in order to maintain flexibility. That way multiple platforms will be supported to avoid being locked-in to a single provider. AWS, GCP, Azure, and DigitalOcean are just some of the few providers with a variety of benefits and disadvantages. An API that allows you to plug into any back-end, allows you to setup your application on different systems without having to overhaul the entire infrastructure.
译：
科技行业的繁荣需要灵活性，除了不断要响应客户不断变化的需求和期望，还需要处理应用程序的结构。
用于构建你应用程序的第三方平台常常会发生不可预知的变化，这些变化可能因为商业决策，外部合并或者彻底停业引起。如果响应这些变化，我们的应用程序结构也会发生变化，所以我们要尽力避免被单一的平台锁定，因为这会带来可怕的结果。
为了保持灵活性，最好使用具有灵活性的跟特定第三方无关的API，避免被单一第三方平台锁定。AWS、GCP、Azure和DigitalOcean是少数考虑了上述问题的服务提供商，他们提供的API可以允许你在不同的系统和平台上集成，同时不需要你应用程序做过多的适配。
注：
为了快速响应客户的变化需求，提高流媒体系统的灵活性主要还是在你的程序结构设计和依赖的第三方服务上，不仅不能让自己的平台和特定的一家服务供应商绑定，还要集成那些水平比较高的第三方平台。
5) ABR
Not all users will have perfect, 5G connectivity. Even those in highly connected areas can have periods of instability or throttled bandwidth.
In order to ensure the smooth delivery of video to every customer possible, responsive features such as Adaptive Bitrate (ABR) are very important. ABR allows the client to request a lower bitrate that is more appropriate to the connectivity they are experiencing at that moment.
The approach to implement ABR is quite different when dealing with real-time protocols like WebRTC. You can’t rely on simply requesting a new manifest and pulling new segment files as is done with MPEG DASH and HLS. In order to adjust on the fly with WebRTC the system needs to be able to shift to the different stream variants via information provided over a control protocol. REMB messages sent over RTCP allow the edge node to deliver just the right size stream for every network situation.
译：
并不是所有用户将来都能拥有完美的5G链接，即使高度链接的地区，也会存在不稳定和带宽受限的时候。
为了确保视频数据能够顺利地交付给用户，诸如自适应码率ABR响应特性变得非常重要。ABR技术允许客户端根据他们的连接网络情况，可以传输更低码率的视频。
在处理像WebRTC这样的低延时通信协议时，实现ABR技术是很不同的。不能像使用MPEG DASH和HLS那样简单地请求新的播放清单和提取新的文件片段即可，为了能够实时调整WebRTC，流媒体系统需要通过控制协议交换相关流信息，比如通过RTCP的REMB消息允许每个边缘节点为每个网络情况提供正确大小的流。
注：自适应码率是实时流媒体系统必须考虑的事情，不同的客户端网络情况应该提供匹配网络情况的码率，这样即能保证视频的质量又能兼顾相应的流量和带宽，给每个用户提供足够优秀的体验。ABR技术的核心是要根据传输流媒体的控制协议比如RTCP交换客户端和服务端的网络信息进行判断当前网络情况，进而确定应该分发的视频码率。
6) Modern Protocols
Keeping up to date with the latest technology is quite obviously important to a tech platform. As the phasing out of Flash has shown, well established protocols that were once dominant, can go away.
WebRTC has emerged to take the place of Flash (and RTMP) as a widely supported and effective browser based protocol. Capable of transporting live streaming video in near real-time, WebRTC is suited for the modern demands placed upon live streams.
Based on web-standards, WebRTC also enjoys the support of Apple, Google, Microsoft, Mozilla, and Opera. That support ensures that the WebRTC standard remains up to date and functional for the foreseeable future.
That being said, you also need…
译：
对于一个流媒体技术平台而言，跟上最新的技术非常重要，正如Flash的逐步淘汰一样，那些曾经稳如磐石的成熟流媒体协议也会随着时间而可能流逝。
WebRTC已经取代了曾经Flash-RTMP方案，现在已经变成一种得到所有浏览器广泛支持的协议。因为WebRTC能够接近实时的传输流媒体视频数据，这点满足了现代实时流媒体的要求。
基于web标准，WebRTC还得到了苹果、谷歌、微软、Mozilla和Opera的支持。这种支持确保了WebRTC标准在可预见的未来仍然是最新的和有效的。
注：铁打的营盘流水的兵，流媒体系统要紧跟最新技术，特别对于最新的流媒体传输协议。WebRTC的成功就是顺应了时代的要求进而得到了几乎所有Web浏览器和大厂的支持。以前基于TCP的的RTMP、RTSP、HLS等技术都会随着时代的变迁而消退，最近像QUIC、SRT、RTP一些基于UDP的流媒体传输协议就得到了长足的发展。
7) Support for Legacy Systems
Even though Flash is going away, RTMP (the capable workhorse of live streaming) will still be around. As such, you will need to have RTMP to support older browsers/devices. Look for solutions with failover support to RTMP and HLS to ensure that your streams will reach all your users.
However, support for legacy protocols is not the only thing you need for compatibility.
译:
虽然Flash技术逐渐消亡了，但是作为流媒体曾经主力的RTMP依然存在，因此我们的流媒体系统需要支持RTMP来实现对旧浏览器和设备的支持。寻找这些支持HLS和RTMP的兼容性方案目的就是让你的视频流能够确保到达所有的用户手中。
同时，兼容老旧系统并不是流媒体系统兼容性的唯一，需要兼容的方面还很多，
注：
评价一个好用的流媒体系统重要一点还在于兼容性，能兼容老的客户端和老的接入设备，不仅能兼容老协议还能兼容各种平台抑或兼容更多的编码方式都很重要。兼容性也是一个优秀流媒体平台需要考虑的因素。
8) Multi-Platform Support
Your video streaming server will need to work across a variety of different devices and browsers. You wouldn’t want to alienate any of your potential customers by not supporting their device of choice. Android users are painfully aware of this whenever they hear about a cool new app that’s only available in the Apple Store.
WebRTC can be run directly in the browser without a plugin. As mentioned earlier, it has the support of the major tech players, so it can be run in Chrome, Firefox, Safari, Edge, and Opera. More importantly, you can use mobile as well as desktop browsers.
Furthermore, you should also have the flexibility of adding a native app with a mobile SDK. The ability to stream in mobile browsers or in an app, allows you to support mobile devices even if you don’t have the resources to devote to customizing a native app.
Beyond just mobile devices, you might want to support IoT devices as well. Drones, IP cameras, and VR headsets can all benefit from live video streaming. These flexible options ensure that you can stream to all the devices your customers are using.
译：
你的视频流服务器需要跨各种不同的设备和浏览器工作。你肯定不想因为不支持你客户选择的设备而选择疏远他们。安卓用户每当听到一个很酷的应用只在IOS系统商店可以使用时感到非常的痛苦。
WebRTC技术不需要任何插件就可以直接在浏览器中跑起来运行。正如前面说的那样，它已经得到主流技术播放器的支持，所以可以运行在几乎任何一款浏览器，比如Chrome、Firefox、Safari和Edge。同时你可以使用桌面浏览器和移动端浏览器。
同时你的流媒体系统应该提供移动端SDK支持移动端能力。哪怕你没有资源去专门定制和开发移动端APP，那你至少需要通过SDK方式让移动端的浏览器和应用程序具备流媒体能力。
除了移动设备，你估计也需要支持物联网设备，像无人机，VR设备和IP摄像头都应该从视频直播中获益，这些灵活的选项确保了了你的流媒体能力可以到达您的客户使用的所有设备。
注：
一个完善的流媒体不仅仅包括服务端能力的支持，同时需要考虑移动端、IOT等业务的支持，这样做的目的就是为了提供完整的端到端能力，可以跨平台支持任何的设备和浏览器，这样你的流媒体能力将无孔不入，满足所有的客户需求。
9) Technical Support
This may be a little counterintuitive as an argument could be made that a well built platform won’t need support. However, most people would agree that even the most well built products will need the occasional technical support.
The more flexibility and extensible features a platform offers the more having good support becomes a necessity. With customizable platforms it is essential that you receive guidance in making the most out of all the available features. Chat channels, online ticketing systems, and advanced support contracts can all be leveraged to make sure that all your needs are met.
译：
技术支持这事有点可能违反你的直觉因为一个良好的平台按道理是不需要技术支持的。然而大部分人也同意即使是再好的产品也需要提供偶尔的技术支持。
平台的灵活性和扩展性越好就让技术支持这件事变得越必要。对于定制化的平台，要是能充分利用所有特性那么提供一点技术支持更是必不可少。提供技术支持的方式可以通过在线聊天，在线合同和票务的形式进行支持。
注：
一个良好的技术平台要有一些跟客户沟通的渠道，即使再牛逼的产品都必不可少。一来能让客户充分挖掘你平台的产品特性，同时也能提供一定反馈回来指导和优化平台。实际中你可以建立一个社区或者建立一个在线聊天系统等都可以做好这件事。
10) Customizable
“One size fits all” hardly ever does. Just because it worked for someone else, doesn’t mean it will work for you.
Those that just bought a new phone are all too familiar with this as they struggle to eliminate the bloatware pre-installed (sometimes permanently) on their new device. If you don’t need something why should you be forced to have it?
Conversely, there is the issue of needing something and not being able to get it. Whether it’s a security configuration or special feature, you need a solution that lets you build what you want with no restrictions.
译：
一刀切或者一套标准很难奏效，因为你的平台适应一部分人意味着不一定另外的人。
对于买了新手机的人非常熟悉一点，就是新设备上常常会预安装一堆东西，这些东西非常难删除。如果有些用户不需要这些，你为啥让他必须拥有这些？
相反地，有一个问题是需要某样东西但是你却得不到它。无论是安全配置还是特殊功能，流媒体平台都要提供一种解决方案，让用户不受任何限制地构建自己想要的东西。
11) Forward Thinking Platform
Obviously, a well functioning platform is a very high priority. However, another major consideration should be how well it will continue to perform. As the recent downfall of Flash has shown, the technologies that drive live streaming can be subject to change.
There are always new standards in development. Streamlined protocols and more efficient codecs are all part of the marketplace.
A good video streaming platform doesn’t just create a product and stop. They make adjustments, respond to customer feedback and generally work to improve everything.
译：
显然，一个运行良好的平台非常重要，然而，它如何持续长时间的运行也非常重要。正如Falsh最近衰落显示的那样，驱动流媒体直播的技术是随时可能发生变化的。
在开发过程中总会出现新的标准，同时精简的协议和高效的编码器也是市场不可或缺的一部分。
一个好的视频流媒体平台不会因为创造一个产品就停止脚步，应该根据客户的反馈做出调整，以便努力的改善平台。
注：
打造一个好的流媒体平台需要关注市场中出现的新协议，新标准和高效的编解码方案。只有不断跟随潮流，才能让自己的系统更优秀。
12) Well Tested and Proven
There are no amount of features that will make up for a poor performing streaming platform. Extensive testing and consistent performance are important to your application.
译：没有多少功能可以弥补一个性能不佳的流媒体平台。广泛的测试和一致的性能对您的应用程序非常重要。
往期文章回顾:
基于HLS-TS&RTMP-FLV的微信小程序点直播方案
一图看懂音视频核心技术栈（框架、工具和场景））
国产开源流媒体SRS4.0对视频监控GB28181的支持
从方块效应&呼吸效应看编码量化参数对流控的作用
家庭消费类摄像头选择攻略和隐私保护小建议
音视频封装小总结（PS TS 和FLV）
SDP在RTSP、国标GB28181、WebRTC中的实践
视频监控摄像头的互联网化实践思路
在HTML5上开发音视频应用的五种思路
周末活动回顾：视频质量主观评价、实时RTC和AV1
音视频封装：MP4结构概述和分析工具
音视频解封装：MP4核心Box详解及H264&AAC打包方案
音视频基础知识-时间戳的理解
音视频封装格式：AAC音频基础和ADTS打包方案详解
从人类的第一次直播聊聊视频监控行业
音视频压缩：H264码流层次结构和NALU详解
音视频传输：RTP协议详解和H.264打包方案
音视频常见问题分析和解决：延时和抖动
个人转载内容至朋友圈和群聊天，无需特别申请版权许可。
引用转载该订阅号文章，注明文章来源即可。
记得右下角点“在看”，还可以关注该订阅号，防止遗漏推送哦
今天就说这么多，祝您工作顺利！
如果有疑问，你可以在公众号后台发消息咨询我。