Gstreamer基本概念深入解析

  我们先看看gstreamer结构体中的一些对象的继承关系图：

 

 

     GObject
         +----GstElement（包含GstBus,GstState, List of pad、GstClock等）
             +----GstBin （元件的容器，包含GstClock，子bus等）
                  +----GstPipeline（包含stream_time等）

 

另外几个类关系：

 GObject
        +---- GstPad

 GObject
        +---- GstBus

GstObject
   +---- GstPluginFeature
      +---- GstElementFactory（factory that the element was created from，_GstElementClass保存有该结构的一个指针）

 我们看一个pipeline的解释和示例图：

Pipeline
~~~~~~~~

A pipeline is a special bin subclass that provides the following features to its
children:

 - Select and manage a global clock for all its children.
 - Manage running_time based on the selected clock. Running_time is the elapsed
   time the pipeline spent in the PLAYING state and is used for
   synchronisation.
 - Manage latency in the pipeline.
 - Provide means for elements to comunicate with the application by the GstBus.
 - Manage the global state of the elements such as Errors and end-of-stream.

Normally the application creates one pipeline that will manage all the elements
in the application.

 

   +-----------------------------------------------------------+
   |    ----------> downstream ------------------->            |
   |                                                           |
   | pipeline                                                  |
   | +---------+   +----------+   +-----------+   +----------+ |
   | | filesrc |   | oggdemux |   | vorbisdec |   | alsasink | |
   | |        src-sink       src-sink        src-sink        | |
   | +---------+   +----------+   +-----------+   +----------+ |
   |                                                           |
   |    <---------< upstream <-------------------<             |
   +-----------------------------------------------------------+

The application does not have to manage any of the complexities of the
actual dataflow/decoding/conversions/synchronisation etc. but only calls high
level functions on the pipeline object such as PLAY/PAUSE/STOP.

GStreamer supports two possible types of dataflow, the push and pull model. In the
push model, an upstream element sends data to a downstream element by calling a
method on a sinkpad. In the pull model, a downstream element requests data from
an upstream element by calling a method on a source pad.

The most common dataflow is the push model. The pull model can be used in specific
circumstances by demuxer elements. The pull model can also be used by low latency
audio applications.

The data passed between pads is encapsulated in Buffers. The buffer contains a
pointer to the actual data and also metadata describing the data.

 

       可见，pipeline的主要布局是elements,  Bins 和pads.把上述几个对象继承关系牢记脑海中。PADs是连接器。pipeline的各对象之间的通信通过缓存buffers、事件event、查询query、消息message机制实现，  调度通过时钟和队列实现。

     

      下面我们开始深入解剖。本篇文章，专门为边看代码，边学gstreamer的初级开发者了解基本概念而写。

 

1. 元件(Elements)

 

    元件(element)是GStreamer中最重要的概念。它是整个gstreamer框架中最重要的一个对象，它定义了pipeline的结构。你可以通过创建一系列的元件(Elements),并把它们连接起来,从而让数据流在这个被连接的各个元件(Elements)之间传输。每个元件(Elements)都有一个特殊的函数接口,对于有些元件(Elements)的函数接口它们是用于能够读取文件的数据,解码文件数据的。而有些元件(Elements)的函数接口只是输出相应的数据到具体的设备上(例如,声卡设备)。你可以将若干个元件(Elements)连接在一起,从而创建一个管道(pipeline)来完成一个特殊的任务,例如,媒体播放或者录音。GStreamer已经默认安装了很多有用的元件(Elements),通过使用这些元件(Elements)你能够构建一个具有多种功能的应用程序。

     每个元件包含有多个gstPad(衬垫)。这些gstpad随后用于元件（element）之间的连接和通信，并最终构成一个可以传递和处理数据的管道pipeline.几乎所有的decoder,encoder,demuxer,video or audio output都实际上是一个GstElement。（ 注：面向对象里的is-a 关系）

gstelement实例的结构如下所示

struct _GstElement
{
  GstObject  object;、

  /*< public >*/ /* with LOCK */
  GStaticRecMutex      *state_lock;               //用于串行化执行gst_element_set_state()

  /* element state */
  GCond                *state_cond;                  //用于状态变化完成时发射通知信号
  guint32  state_cookie;                           //用于检测gst_element_set_state()和gst_element_get_state()的并发执行
  GstState              current_state;             //元件当前所处状态   
  GstState              next_state;                //元件的下一个状态。如果正处于正确的状态，该值可以为GST_STATE_VOID_PENDING
  GstState              pending_state;      //元件的应趋向的最终状态。如果正处于正确的状态，该值可以为GST_STATE_VOID_PENDING
  GstStateChangeReturn  last_return;  //元件状态改变时，最后的那个返回值

  GstBus        *bus;            //包含的GstBus,该总线由父元素或应用程序提供。Gstpipeline有自己的bus

  /* allocated clock */
  GstClock        *clock;    //时钟，通常由顶级的gstpipeline管道提供
  GstClockTimeDiff base_time; /* NULL/READY: 0 - PAUSED: current time - PLAYING: difference to clock*///element状态刚变为playing时的时钟时间，播放状态下时钟时间减去base_time可以获得流时间（即播放时间，运行时间）

  /* element pads, these lists can only be iteated while holding
   * the LOCK or checking the cookie after each LOCK. */
  guint16               numpads;//number of pads of the element, includes both source and sink pads.
  GList                *pads; //List of pad,衬垫列表（每个元件都包含有pad）, 一个衬垫(Pads)可以被看作是一个元件(element)插座或者端口,元件(element)之间的链接就是依靠着衬垫(Pads)
  guint16               numsrcpads;
  GList                *srcpads;
  guint16               numsinkpads;
  GList                *sinkpads;
  guint32               pads_cookie;

  /*< private >*/
  union {
    struct {
      /* state set by application */
      GstState              target_state;
    } ABI;
    /* adding + 0 to mark ABI change to be undone later */
    gpointer _gst_reserved[GST_PADDING + 0];
  } abidata;
};

       添加和删除pad的方法在gstelement的类结构体中有定义，这里从略。读者可从网上下载代码阅读。

 

 

2. 箱柜(Bins)和管道(pipelines)

 

 

    箱柜(Bins)是一个可装载一系列元件(element)的容器。管道(pipelines)是箱柜(Bins)的一个特殊的子类型,管道(pipelines)可以操作包含在它自身内部的所有元件(element)。gstbin的实例结构体定义如下：
   A bin is an element subclass and acts as a container for other elements so that multiple
elements can be combined into one element.

    A bin can have its own source and sinkpads by ghostpadding one or more of its children's
pads to itself.

struct _GstBin {
  GstElement  element;   

  /*< public >*/ /* with LOCK */
  /* our children, subclass are supposed to update these
   * fields to reflect their state with _iterate_*() */
  gint   numchildren; //the number of children in this bin
  GList  *children;//the list of children in this bin

  guint32  children_cookie;//updated whenever @children changes

  GstBus        *child_bus;// internal bus for handling child messages
  GList         *messages;//queued and cached messages

  gboolean  polling;//the bin is currently calculating its state
  gboolean       state_dirty;//the bin needs to recalculate its state (deprecated)

  gboolean       clock_dirty;//the bin needs to select a new clock
  GstClock *provided_clock;//the last clock selected
  GstElement    *clock_provider;//the  element that provided @provided_clock

  /*< private >*/
  GstBinPrivate *priv;/

  gpointer _gst_reserved[GST_PADDING - 1];
};

     我们再来分析下GstPipeline的实例结构：

struct _GstPipeline {
  GstBin   bin;

  /*< public >*/ /* with LOCK */
  GstClock      *fixed_clock;  //The fixed clock of the pipeline, used when  GST_PIPELINE_FLAG_FIXED_CLOCK is set.

 

  GstClockTime   stream_time; //The stream time of the pipeline. A better name for this property would be the running_time, the total time spent in the  PLAYING state without being flushed. (deprecated, use the start_time  on GstElement).注意到gstelement实例结构里有一个base_time.它们是什么关系呢？当前时间与base_time相减可以得到running-time(运行时间)。在播放速率一致的情况下，这个运行时间与stream_time是相等的！ element可以提供一个时钟，管道pipeline可以选择一个时钟，所有的接收器sinks都要与之同步！

  GstClockTime   delay;//Extra delay added to base_time to compensate for computing delays when setting elements to PLAYING

 

  /*< private >*/
  GstPipelinePrivate *priv;

  gpointer _gst_reserved[GST_PADDING-1];

};

       因为箱柜(Bins)本身又是元件(element)的子集,所以你能够象操作普通元件(element)一样的操作一个箱柜(Bins),通过这种方法可以降低你的应用程序的复杂度。你可以改变一个箱柜(Bins)的状态来改变箱柜(Bins)内部所有元件(element)的状态。箱柜(Bins)可以发送总线消息(bus messages)给它的子集元件(element)(这些消息包括:错误消息(error messages),标签消息(tag messages),EOS消息(EOS messages))。

  struct _GstPipelineClass {
  GstBinClass parent_class;

  /*< private >*/
  gpointer _gst_reserved[GST_PADDING];
};

 

    管道(pipeline)是高级的箱柜(Bins)。当你设定管道的暂停或者播放状态的时候,数据流将开始流动,并且媒体数据处理也开始处理。一旦开始,管道将在一个单独的线程中运行,直到被停止或者数据流播放完毕。

 

 

3.衬垫(Pads)

 

 

     衬垫在GStreamer中被用于多个元件的链接,从而让数据流能在这样的链接中流动。一个衬垫(Pads)可以被看作是一个元件(element)插座或者端口,元件(element)之间的链接就是依靠着衬垫(Pads)。衬垫(Pads)有处理特殊数据的能力:一个衬垫(Pads)能够限制数据流类型的通过。链接成功的条件是：只有在两个衬垫(Pads)允许通过的数据类型一致的时候才被建立。数据类型的设定使用了一个叫做caps negotiation的方法。数据类型被为一个GstCaps变量所描述。

struct _GstCaps {
  GType type;

  /*< public >*/ /* with COW */
  /* refcounting */
  gint           refcount;

  /*< public >*/ /* read only */
  GstCapsFlags flags;

  /*< private >*/
  GPtrArray *structs;//

/*注释，方便理解-----------struct _GPtrArray
{
  gpointer *pdata;
  guint     len;
};*/

  /*< private >*/
  gpointer _gst_reserved[GST_PADDING];
};

 

     下面的这个比喻可能对你理解衬垫(Pads)有所帮助。一个衬垫(Pads)很象一个物理设备上的插头。例如一个家庭影院系统。一个家庭影院系统由一个功放(amplifier),一个DVD机,还有一个无声的视频投影组成。我们需要连接DVD机到功放(amplifier),因为两个设备都有音频插口;我们还需要连接投影机到DVD机上,因为两个设备都有视频处理插口。但我们很难将投影机与功放(amplifier)连接起来,因为他们之间处理的是不同的插口。GStreamer衬垫(Pads)的作用跟家庭影院系统中的插口是一样的。

      对于大部分情况,所有的数据流都是在链接好的元素之间流动。数据向元件(element)以外流出可以通过一个或者多个 source衬垫(Pads),元件(element)接受数据是通过一个或者多个sink衬垫(Pads)来完成的。Source元件(element)和sink元件(element)分别有且仅有一个 sink 衬垫(Pads)或者source衬垫(Pads)。数据在这里代表的是缓冲区(buffers) (GstBuffer对象描述了数据的缓冲区(buffers)的信息)和事件(events) (GstEvent对象描述了数据的事件(events)信息)。

 

/**
 * GstPad:
 * @element_private: private data owned by the parent element
 * @padtemplate: padtemplate for this pad
 * @direction: the direction of the pad, cannot change after creating
 *             the pad.
 * @stream_rec_lock: recursive stream lock of the pad, used to protect
 *                   the data used in streaming.
 * @task: task for this pad if the pad is actively driving dataflow.
 * @preroll_lock: lock used when prerolling
 * @preroll_cond: conf to signal preroll
 * @block_cond: conditional to signal pad block
 * @block_callback: callback for the pad block if any
 * @block_data: user data for @block_callback
 * @caps: the current caps of the pad
 * @getcapsfunc: function to get caps of the pad
 * @setcapsfunc: function to set caps on the pad
 * @acceptcapsfunc: function to check if pad can accept caps
 * @fixatecapsfunc: function to fixate caps
 * @activatefunc: pad activation function
 * @activatepushfunc: function to activate/deactivate pad in push mode
 * @activatepullfunc: function to activate/deactivate pad in pull mode
 * @linkfunc: function called when pad is linked
 * @unlinkfunc: function called when pad is unlinked
 * @peer: the pad this pad is linked to
 * @sched_private: private storage for the scheduler
 * @chainfunc: function to chain data to pad
 * @checkgetrangefunc: function to check if pad can operate in pull mode
 * @getrangefunc: function to get a range of data from a pad
 * @eventfunc: function to send an event to a pad
 * @mode: current activation mode of the pad
 * @querytypefunc: get list of supported queries
 * @queryfunc: perform a query on the pad
 * @intlinkfunc: get the internal links of this pad
 * @bufferallocfunc: function to allocate a buffer for this pad
 * @do_buffer_signals: counter counting installed buffer signals
 * @do_event_signals: counter counting installed event signals
 * @iterintlinkfunc: get the internal links iterator of this pad
 *
 * The #GstPad structure. Use the functions to update the variables.
 */