GStreamer iOS教程2 —— 运行pipeline
1. 目标
在Basic和Playback的教程中（注：另外两个教程），GStreamer可以和GLib的主循环完美的集成起来，这样能用一种简单的方法同时监控pipeline的操作和UI的处理。而在不支持GLib的iOS和Android平台上就必须小心对于pipeline的操作——不能阻塞住UI。
这份教程讲述了：
- 如何把GStreamer的相关处理代码放到其他的线程（DispatchQueue），保持UI仍然保留在主线程（MainDispatchQueue）
- 在ObjC的UI代码中如何和GStreamer的C代码通信
2. 介绍
当由UI界面的时候，如果应用等待GStreamer的回传消息然后进行UI的处理是非常悲催的。通常的做法是（用GTK＋toolkit做例子哈）让GMainLoop（GLib）来处理收到的event，无论是UI的还是GStreamer发出的。
悲催的是这个方法不适合其他非基于GLib的图形系统（比如iOS的GocoaTouch框架），我们的方法是在另一个线程里面简单的调用GMainLoop，确保不会阻塞UI主线程。
这个教程还会指出几个ObjC和C互相调用时需要注意的几个地方。
下面的代码使用的pipeline仅仅使用了audiotestsrc和autoaudiosink两个element。UI上包含两个按钮，用来设置pipeline的状态PLAYING/ PAUSED。还有一个Label用来显示一些信息（错误信息或状态改变）。
3. UI
这个界面底下包含了一个工具条，工具条上放了Play和Pause两个按钮。工具条上方就是一个Label，用来显示GStreamer的信息。本次教程就包含这么多内容，后面的教程会在这个基础上逐渐增加内容。
ViewController.m
#import "ViewController.h"
#import "GStreamerBackend.h"
#import <UIKit/UIKit.h>
@interface ViewController () {
GStreamerBackend *gst_backend;
}
@end
@implementation ViewController
/*
* Methods from UIViewController
*/
- (void)viewDidLoad
{
[super viewDidLoad];
play_button.enabled = FALSE;
pause_button.enabled = FALSE;
gst_backend = [[GStreamerBackend alloc] init:self];
}
- (void)didReceiveMemoryWarning
{
[super didReceiveMemoryWarning];
// Dispose of any resources that can be recreated.
}
/* Called when the Play button is pressed */
-(IBAction) play:(id)sender
{
[gst_backend play];
}
/* Called when the Pause button is pressed */
-(IBAction) pause:(id)sender
{
[gst_backend pause];
}
/*
* Methods from GstreamerBackendDelegate
*/
-(void) gstreamerInitialized
{
dispatch_async(dispatch_get_main_queue(), ^{
play_button.enabled = TRUE;
pause_button.enabled = TRUE;
message_label.text = @"Ready";
});
}
-(void) gstreamerSetUIMessage:(NSString *)message
{
dispatch_async(dispatch_get_main_queue(), ^{
message_label.text = message;
});
}
@end
在这个类里面包含了一个GStreamerBackend的实例，
@interface ViewController () {
GStreamerBackend *gst_backend;
}
在viewDidLoad方法里面创建并调用了自定义的init方法
- (void)viewDidLoad
{
[super viewDidLoad];
play_button.enabled = FALSE;
pause_button.enabled = FALSE;
gst_backend = [[GStreamerBackend alloc] init:self];
}
这个自定义的init方法必须传入一个对象作为UI的delegate（本例是使用了self）
在viewDidLoad的时候，Play/Pause两个按钮都是不能使用的，直到GStreamerBackend通知初始化结束为止。
/* Called when the Play button is pressed */
-(IBAction) play:(id)sender
{
[gst_backend play];
}
/* Called when the Pause button is pressed */
-(IBAction) pause:(id)sender
{
[gst_backend pause];
} 在用户按下Play/Pause按钮时，上面的方法会被调用。我们看到仅仅就是简单的调用了GStreamerBackend里面对应的方法。
-(void) gstreamerInitialized
{
dispatch_async(dispatch_get_main_queue(), ^{
play_button.enabled = TRUE;
pause_button.enabled = TRUE;
message_label.text = @"Ready";
});
}
gstreamerInitialized方法是定义在GStreamerBackendDelegate协议里面的，用来标识后台已经准备好，可以接受命令了。在这个例子中我们把Play/Pause按钮激活，并显示“Ready”信息。这个方法不是在UI线程里面运行的，所以要用dispatch_async()方法把UI的内容封起来。
-(void) gstreamerSetUIMessage:(NSString *)message
{
dispatch_async(dispatch_get_main_queue(), ^{
message_label.text = message;
});
}
gstreamerSetUIMessage:方法同样是定义在GStreamerBackendDelegate协议里面的，当后台有消息通知UI的时候会调用这个方法。在这个例子里面消息会拷贝到界面的Label控件上，当然，也同样要用dispatch_async()方法来封装。
4. GStreamer后端
GStreamerBackend类处理了所有和GStreamer相关的内容，并给应用提供了一个简单的接口。这个接口并不需要实现所有的GStreamer的细节，当需要引起UI的变化的时候，调用GSreamerBackendDelegate协议来解决。
GStreamerBackend.m
#import "GStreamerBackend.h"
#include <gst/gst.h>
GST_DEBUG_CATEGORY_STATIC (debug_category);
#define GST_CAT_DEFAULT debug_category
@interface GStreamerBackend()
-(void)setUIMessage:(gchar*) message;
-(void)app_function;
-(void)check_initialization_complete;
@end
@implementation GStreamerBackend {
id ui_delegate; /* Class that we use to interact with the user interface */
GstElement *pipeline; /* The running pipeline */
GMainContext *context; /* GLib context used to run the main loop */
GMainLoop *main_loop; /* GLib main loop */
gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
}
/*
* Interface methods
*/
-(id) init:(id) uiDelegate
{
if (self = [super init])
{
self->ui_delegate = uiDelegate;
GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
/* Start the bus monitoring task */
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
[self app_function];
});
}
return self;
}
-(void) dealloc
{
if (pipeline) {
GST_DEBUG("Setting the pipeline to NULL");
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);
pipeline = NULL;
}
}
-(void) play
{
if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to playing"];
}
}
-(void) pause
{
if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to paused"];
}
}
/*
* Private methods
*/
/* Change the message on the UI through the UI delegate */
-(void)setUIMessage:(gchar*) message
{
NSString *string = [NSString stringWithUTF8String:message];
if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
{
[ui_delegate gstreamerSetUIMessage:string];
}
}
/* Retrieve errors from the bus and show them on the UI */
static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
{
GError *err;
gchar *debug_info;
gchar *message_string;
gst_message_parse_error (msg, &err, &debug_info);
message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
g_clear_error (&err);
g_free (debug_info);
[self setUIMessage:message_string];
g_free (message_string);
gst_element_set_state (self->pipeline, GST_STATE_NULL);
}
/* Notify UI about pipeline state changes */
static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
{
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
/* Only pay attention to messages coming from the pipeline, not its children */
if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
[self setUIMessage:message];
g_free (message);
}
}
/* Check if all conditions are met to report GStreamer as initialized.
* These conditions will change depending on the application */
-(void) check_initialization_complete
{
if (!initialized && main_loop) {
GST_DEBUG ("Initialization complete, notifying application.");
if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
{
[ui_delegate gstreamerInitialized];
}
initialized = TRUE;
}
}
/* Main method for the bus monitoring code */
-(void) app_function
{
GstBus *bus;
GSource *bus_source;
GError *error = NULL;
GST_DEBUG ("Creating pipeline");
/* Create our own GLib Main Context and make it the default one */
context = g_main_context_new ();
g_main_context_push_thread_default(context);
/* Build pipeline */
pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
if (error) {
gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
g_clear_error (&error);
[self setUIMessage:message];
g_free (message);
return;
}
/* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
bus = gst_element_get_bus (pipeline);
bus_source = gst_bus_create_watch (bus);
g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
g_source_attach (bus_source, context);
g_source_unref (bus_source);
g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
gst_object_unref (bus);
/* Create a GLib Main Loop and set it to run */
GST_DEBUG ("Entering main loop...");
main_loop = g_main_loop_new (context, FALSE);
[self check_initialization_complete];
g_main_loop_run (main_loop);
GST_DEBUG ("Exited main loop");
g_main_loop_unref (main_loop);
main_loop = NULL;
/* Free resources */
g_main_context_pop_thread_default(context);
g_main_context_unref (context);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
return;
}
@end
其中，接口方法是：
-(id) init:(id) uiDelegate
{
if (self = [super init])
{
self->ui_delegate = uiDelegate;
GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "iOS tutorial 2");
gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
/* Start the bus monitoring task */
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
[self app_function];
});
}
return self;
} 这个init方法通过调用［super init］来生成实例，保存delegate的对象用来做UI互动，接着调用app_function并运行在一个独立的并发的线程里面，app_function会一直监听GStreamer总线，看看有没有应用需要处理的消息或者警告发生。
init:方法同样注册了一个新的GStreamer调试类别并设置了吐出的信息的等级，我们就可以在Xcode里面看到打印信息了。
-(void) dealloc
{
if (pipeline) {
GST_DEBUG("Setting the pipeline to NULL");
gst_element_set_state(pipeline, GST_STATE_NULL);
gst_object_unref(pipeline);
pipeline = NULL;
}
} dealloc方法把pipeline置成NULL状态并释放它。
-(void) play
{
if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to playing"];
}
}
-(void) pause
{
if(gst_element_set_state(pipeline, GST_STATE_PAUSED) == GST_STATE_CHANGE_FAILURE) {
[self setUIMessage:"Failed to set pipeline to paused"];
}
} play/pause方法仅仅简单的设置pipeline状态的该变，并在出错的时候通知UI
下面是文件中几个私有方法：
-(void)setUIMessage:(gchar*) message
{
NSString *string = [NSString stringWithUTF8String:message];
if(ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerSetUIMessage:)])
{
[ui_delegate gstreamerSetUIMessage:string];
}
} setUIMessage:方法是把GStreamer使用的C的字符串转变成NSString＊字符串，然后调用GStreamerBackendProtocal协议里面的gstreamerSetUIMessage:方法来在屏幕上显示出来。
因为这个方法是optional的，所以需要用respondsToSelector来判一下是否存在。
/* Retrieve errors from the bus and show them on the UI */
static void error_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
{
GError *err;
gchar *debug_info;
gchar *message_string;
gst_message_parse_error (msg, &err, &debug_info);
message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
g_clear_error (&err);
g_free (debug_info);
[self setUIMessage:message_string];
g_free (message_string);
gst_element_set_state (self->pipeline, GST_STATE_NULL);
}
/* Notify UI about pipeline state changes */
static void state_changed_cb (GstBus *bus, GstMessage *msg, GStreamerBackend *self)
{
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
/* Only pay attention to messages coming from the pipeline, not its children */
if (GST_MESSAGE_SRC (msg) == GST_OBJECT (self->pipeline)) {
gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
[self setUIMessage:message];
g_free (message);
}
} error_cb()和state_changed_cb()是注册的两个回调，分别在GStreamer出错和状态变化的时候被调用。这两个回调的目的是当事件发生时能通知到用户。
这两个回调函数在Base的教程中出现了多次，实现起来也除了下面2点之外基本一致：一个是消息使用私有方法setUIMessage:来传递到UI；第二个是要调用setUIMessage:就需要一个GStreamerBackend的实例，通过callback的userdata来传递，这个在下面讨论app_function里回调的注册时可以看到
-(void) check_initialization_complete
{
if (!initialized && main_loop) {
GST_DEBUG ("Initialization complete, notifying application.");
if (ui_delegate && [ui_delegate respondsToSelector:@selector(gstreamerInitialized)])
{
[ui_delegate gstreamerInitialized];
}
initialized = TRUE;
}
}
check_initialization_complete()方法确认满足所有的条件之后通知UI后台GStreamer准备完成。在这个教程里面这个条件非常简单，仅仅是主循环存在并且没有通知过UI。后续的教程这里会更加复杂。
绝大部分的GStreamer的行为都是在app_function里面实现的，这些代码和android的教程几乎一致。
/* Create our own GLib Main Context and make it the default one */
context = g_main_context_new ();
g_main_context_push_thread_default(context);
这里第一次创建了一个GLib的上下文，使用g_main_context_new()，然后用g_main_context_push_thread_default()来创建了一个线程
/* Build pipeline */
pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
if (error) {
gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
g_clear_error (&error);
[self setUIMessage:message];
g_free (message);
return;
}
这里用gst_arse_launch()方法很轻易的创建了一个pipeline。在这个教程里面仅仅audiotestsrc和autoaudiosink两个element需要完成适配。
/* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
bus = gst_element_get_bus (pipeline);
bus_source = gst_bus_create_watch (bus);
g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
g_source_attach (bus_source, context);
g_source_unref (bus_source);
g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, (__bridge void *)self);
g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, (__bridge void *)self);
gst_object_unref (bus);
这几行创建了一个总线信号的监视器，并和设置了需要监视的信号，这些和Basic教程里面的做法也是一致的。这个监视器我们这里是一步一步创建的，并非调用gst_bus_add_signal_watch()来创建，这样可以看清如何使用GLib的一些内容。这里需要指出的是使用了__bridge来把一个ObjC的对象指针转换成C语言里面的指针。
这里我们不需要太担心对象的所有权的转移问题，因为在回调里面userdata会把这个指针带回来，重新转换成GStreamerBackend的对象指针
/* Create a GLib Main Loop and set it to run */
GST_DEBUG ("Entering main loop...");
main_loop = g_main_loop_new (context, FALSE);
[self check_initialization_complete];
g_main_loop_run (main_loop);
GST_DEBUG ("Exited main loop");
g_main_loop_unref (main_loop);
main_loop = NULL;
最后，主循环创建并开始运行，在进入主循环之前，我们调用了check_initialization_complete()方法。主循环会一直运行，直到退出为止。
这篇教程有点长了，主要是需要讲清楚一系列的基础内容。在这个基础之上，后面的会比较短一些，并且会只关注新的内容。
5. 结论
这篇教程主要讲述了：
- 使用DispatchQueue如何让GStreamer的代码单独运行在子线程中
- 如何在ObjC的UI代码和GStreamer的C代码中传递对象
在这篇教程里面的方法，象check_initialization_complete()和app_function()，init:，play;，pause:，gstreamerInitialized:和setUIMessage:等接口后续会简单修改一下继续使用，所以最好熟悉一下。