FreeSwitch模块加载流程和视频音频处理源代码分析

熟悉的过程:

switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_INFO, "Activate Eventing Engine.\n");

将日志输出到文件,并 SetConsoleTextAttribute来设置输出log的颜色,便于区分

fs提供多种方式来输出日志文件

switch_log_printf ( SWITCH_CHANNEL_LOG   ,   SWITCH_LOG_CONSOLE ,   "Successfully Loaded [%s]\n" ,   module_interface ->   module_name );

switch_log_printf ( SWITCH_CHANNEL_LOG   ,   SWITCH_LOG_DEBUG ,   "Successfully Loaded [%s]\n" ,   module ->   filename );

switch_log_printf(SWITCH_CHANNEL_LOG , SWITCH_LOG_DEBUG, "Call switch_core_media_read_frame() Function!\n");

说明一下switch_log_printf的使用方法

SetConsoleTextAttribute设置输出文本的颜

freeswitch在启动时的加载过程

switch_event_init     初始化事件

switch_nat.c进行nat检测

switch_core_sqldb.c 数据库的初始化和检测

网络相关的设置

switch_loadable_module.c加载需要的模块(比如:编解码模块,API Function )

加载模块Timer

switch_loadable_module.c:1401 Successfully Loaded [CORE_SOFTTIMER_MODULE]

switch_loadable_module.c:238 Adding Timer 'soft'

加载Logger中的Console和logFile文件

switch_loadable_module.c:1401 Successfully Loaded [mod_console]

switch_loadable_module.c:1401 Successfully Loaded [mod_logfile]

加载通信方式(以Event_Socket方式通信)

switch_loadable_module.c:1401 Successfully Loaded [mod_event_socket]

初始化消息线程

 mod_sofia.c:5176 Starting initial message thread

2015-04-14 10:12:04.535099 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_sofia]

加载对应配置文件后,加载sofia模块,成功.

需要介绍一下sofia模块的作用

加载Endpoint模块

2015-04-14 10:12:04.535151 [NOTICE] switch_loadable_module.c:147 Adding Endpoint 'sofia'
2015-04-14 10:12:04.535240 [NOTICE] switch_loadable_module.c:147 Adding Endpoint 'rtp'

介绍Endpoint**

Endpoint是什么冬冬?

Endpoint是freeswitch的终结的地方,我理解的是和外界通信的界限.出了Endpoint的地方就脱离了freeswitch的控制了.

正在开启数据库线程

2015-04-14 10:12:03.027548 [INFO] switch_core_sqldb.c:1449 sofia:internal-ipv6 Starting SQL thread.
2015-04-14 10:12:03.030162 [INFO] switch_core_sqldb.c:1449 sofia:external Starting SQL thread.
2015-04-14 10:12:03.031207 [NOTICE] sofia_reg.c:3165 Added gateway 'example.com' to profile 'external'
2015-04-14 10:12:03.032423 [INFO] switch_core_sqldb.c:1449 sofia:internal Starting SQL thread.

加载命令行模块

switch_loadable_module.c:1401 Successfully Loaded [mod_commands]

加载mod_conference模块

2015-04-14 10:12:04.594931 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_conference]
2015-04-14 10:12:04.594955 [NOTICE] switch_loadable_module.c:259 Adding Application 'conference'
2015-04-14 10:12:04.595006 [NOTICE] switch_loadable_module.c:259 Adding Application 'conference_set_auto_outcall'
2015-04-14 10:12:04.595050 [NOTICE] switch_loadable_module.c:305 Adding API Function 'conference'
2015-04-14 10:12:04.595079 [NOTICE] switch_loadable_module.c:417 Adding Chat interface 'conf'

加载DB数据库

2015-04-14 10:12:04.627689 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_db]
2015-04-14 10:12:04.627736 [NOTICE] switch_loadable_module.c:259 Adding Application 'db'
2015-04-14 10:12:04.627779 [NOTICE] switch_loadable_module.c:259 Adding Application 'group'
2015-04-14 10:12:04.627864 [NOTICE] switch_loadable_module.c:305 Adding API Function 'db'
2015-04-14 10:12:04.627894 [NOTICE] switch_loadable_module.c:305 Adding API Function 'group'
2015-04-14 10:12:04.627920 [NOTICE] switch_loadable_module.c:490 Adding Limit interface 'db'

2015-04-14 10:12:05.208136 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_fifo]

2015-04-14 10:12:05.213144 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_hash]

定位到源文件

相当于添加一个api接口hash到系统核心,当有事件发生时,则调用hash_remote_function()函数

加载语音信箱模块

2015-04-14 10:12:05.252991 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_voicemail]

2015-04-14 10:12:05.256017 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_esf]

2015-04-14 10:12:05.257130 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_fsv]

2015-04-14 10:12:05.258452 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_cluechoo]

2015-04-14 10:12:05.261723 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_valet_parking]

2015-04-14 10:12:05.263445 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_httapi]
2015-04-14 10:12:05.263464 [NOTICE] switch_loadable_module.c:259 Adding Application 'httapi'
2015-04-14 10:12:05.263529 [NOTICE] switch_loadable_module.c:305 Adding API Function 'httapi'
2015-04-14 10:12:05.263576 [NOTICE] switch_loadable_module.c:332 Adding File Format 'http'
2015-04-14 10:12:05.263618 [NOTICE] switch_loadable_module.c:332 Adding File Format 'https'

加载拨号计划(比如SIP和asterisk)

2015-04-14 10:12:05.268003 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_dialplan_xml]
2015-04-14 10:12:05.268066 [NOTICE] switch_loadable_module.c:217 Adding Dialplan 'XML'
2015-04-14 10:12:05.268799 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_dialplan_asterisk]
2015-04-14 10:12:05.268824 [NOTICE] switch_loadable_module.c:147 Adding Endpoint 'SIP'
2015-04-14 10:12:05.268902 [NOTICE] switch_loadable_module.c:147 Adding Endpoint 'IAX2'
2015-04-14 10:12:05.268953 [NOTICE] switch_loadable_module.c:217 Adding Dialplan 'asterisk'
2015-04-14 10:12:05.268996 [NOTICE] switch_loadable_module.c:259 Adding Application 'Dial'
2015-04-14 10:12:05.269040 [NOTICE] switch_loadable_module.c:259 Adding Application 'Goto'
2015-04-14 10:12:05.269084 [NOTICE] switch_loadable_module.c:259 Adding Application 'AvoidingDeadlock'

2015-04-14 10:12:05.344929 [CONSOLE] switch_loadable_module.c:1401 Successfully Loaded [mod_spandsp]

开源项目组织结构

核心: 核心代码都经过精心的编码和严格的测试，最大限度地保持了系统整体的稳定。外围模块只能通过 API 调用核心的功能。 模块是可以动态加载（以及卸载）的，在实际应用中可以只加载用到的模块。外围模块通过核心提供的 Public API 与核心进行通信，而核心则通过回调机制执行外围模块中的代码。 FreeSWITCH 使用线程模型来处理并发请求，每个连接都在单独的线程中进行处理。

主要核心内容:

1/项目模块的初始化和加载过程

mofia.c是个核心的源文件, 项目源代码整体上采用的是核心+模块的方式

switch_loadable_module_load_module_ex()函数调用

switch_loadable_module_load_file将共享库加载到进程的地址空间中

switch_loadable_module_process 将共享库提供的各个接口,加入到全局的哈希表中

2/读取(写)音频视频流的过程,以读为例mod_sofia.c

系统在加载mod_sofia模块的时候,会注册一些回调函数给核心core

注册的回调函数有

switch_io_routines_t   sofia_io_routines   = {

               /*.outgoing_channel */   sofia_outgoing_channel   ,

               /*.read_frame */   sofia_read_frame   ,

               /*.write_frame */   sofia_write_frame   ,

               /*.kill_channel */   sofia_kill_channel   ,

               /*.send_dtmf */   sofia_send_dtmf   ,

               /*.receive_message */   sofia_receive_message   ,

               /*.receive_event */   sofia_receive_event   ,

               /*.state_change */   NULL   ,

               /*.read_video_frame */   sofia_read_video_frame   ,

               /*.write_video_frame */   sofia_write_video_frame   ,

               /*.state_run*/   NULL   ,

               /*.get_jb*/   sofia_get_jb

};

当发生指定的fs事件时,核心core就会调用相应的回调函数.

fs在读取音频数据和视频数据的时候是分离的,过程如下:

第一步:sofia_read_frame()函数中,处理音频数据,继续调用switch_core_media_read_frame

static   switch_status_t   sofia_read_frame (   switch_core_session_t   * session   ,   switch_frame_t   ** frame ,   switch_io_flag_t   flags ,   int   stream_id   )

{

               private_object_t   * tech_pvt   =   switch_core_session_get_private (   session );

               switch_channel_t   * channel   =   switch_core_session_get_channel (   session );

               uint32_t   sanity   = 1000;

               switch_status_t   status   =   SWITCH_STATUS_FALSE ;

               switch_assert ( tech_pvt   !=   NULL );

             //监测sofia模块运行,如果失败则挂断并返回错误

               if   (! sofia_test_pflag   ( tech_pvt ->   profile ,   PFLAG_RUNNING   )) {

                           switch_channel_hangup ( tech_pvt   -> channel ,   SWITCH_CAUSE_NORMAL_CLEARING   );

                           return   SWITCH_STATUS_FALSE   ;

            }

             //监测 TFLAG_HUP标志

               if   ( sofia_test_flag   ( tech_pvt ,   TFLAG_HUP )) {

                           return   SWITCH_STATUS_FALSE   ;

            }

               while   (!( switch_core_media_ready   ( tech_pvt ->   session ,   SWITCH_MEDIA_TYPE_AUDIO ) && ! switch_channel_test_flag   ( channel ,   CF_REQ_MEDIA ))) {

                           switch_ivr_parse_all_messages ( tech_pvt   -> session );

                           if   (-- sanity   &&   switch_channel_up (   channel )) {

                                       switch_yield (10000);

                        }   else   {

                                       switch_channel_hangup ( tech_pvt   -> channel ,   SWITCH_CAUSE_RECOVERY_ON_TIMER_EXPIRE   );

                                       return   SWITCH_STATUS_GENERR   ;

                        }

            }

             //加锁,读取桢数据

               sofia_set_flag_locked ( tech_pvt   ,   TFLAG_READING );

             

             //在读取过程中,本端或者对端挂断

               if   ( sofia_test_flag   ( tech_pvt ,   TFLAG_HUP ) ||   sofia_test_flag   ( tech_pvt ,   TFLAG_BYE )) {

                           return   SWITCH_STATUS_FALSE   ;

            }

             //调用switch_core_media.c中的函数来读取桢数据

               status   =   switch_core_media_read_frame   ( session ,   frame ,   flags   ,   stream_id ,   SWITCH_MEDIA_TYPE_AUDIO );

             //读取完毕,解锁

               sofia_clear_flag_locked ( tech_pvt   ,   TFLAG_READING );

               return   status   ;

}

第二步:在switch_core_media_read_frame函数 中

//获取媒体的类型

//SWITCH_MEDIA_TYPE_AUDIO为音频,SWITCH_MEDIA_TYPE_VIDEO为视频数据

engine   = & smh   -> engines [   type ];

//check the read_codec is ready,if no,return SWITCH_STATUS_FALSE

//监测用于读的解码库是否就绪

if   (! engine   -> read_codec .   implementation   || ! switch_core_codec_ready (& engine   -> read_codec )) {

                return   SWITCH_STATUS_FALSE   ;

}

然后继续调用 switch_rtp_zerocopy_read_frame()函数从rtp中读取桢数据(主要读取桢数据的地方,两个客户端通信主要是RTP协议)

调用并读取桢数据成功则创建事件,并对事件各参数赋值

/* Fast PASS! if proxy mode !*/

if   ( switch_test_flag   ((& engine ->   read_frame ),   SFF_PROXY_PACKET   )) {

            *   frame   = & engine   -> read_frame ;

             return   SWITCH_STATUS_SUCCESS   ;

}

如果是代理模式,则读取桢数据后直接返回,不做任何处理,只起到代理的功能.

第三步:在switch_rtp_zerocopy_read_frame()函数中调用rtp_common_read()函数

将读取到的数据赋值给 frame的各个参数

目前只分析到 rtp_common_read()函数,下面的就是从底层中读取数据了

sofia模块使用 sofia_read_video_frame()函数来读取视频数据

在函数内部调用 switch_core_media_read_frame 函数

现在再来看,mod_sofia模块中对音频和视频的读写处理

和上面的源代码是差不多的.

分析switch_core_session.c中的会话

switch_core_session_request_uuid()分析