【Media Foundation】简单实例 - 使用Media Session来播放文件
/*
* blackboy [email protected]
* QQ群： 135202158
* 转载请注明作者及出处
*/
参考MSDN官方的页面：http://msdn.microsoft.com/en-us/library/ms703190(v=vs.85)
本文详细演示了如何使用Media Foundation中的Media Session对象来播放媒体文件。也就是不自己编写/自定义任何的Media Foundation组件，一切都是用现成的，以及让Media Foundation“自动完成”的（如Topology的解析）。Media Foundation的API会根据文件的路径或URL智能创建合适的media source组件，并会智能地在media source和音视频渲染器(renderer)之间添加合适的解码器等等。Topology中的数据流等任务由Media Session来处理。
这是最简单的开发任务。然而，如果要实现使用自定义的meida source或media transform组件这样的任务，可能不能使用Media Session。
预备知识
在阅读本主题之前，你需要熟悉以下MF概念：
- Media Session
- Source Resolver
- Topologies
- Media Event Generators
- Presentation Descriptors
注意：此主题不描述如何播放被DRM保护的文件。关于MMF中DRM的相关信息，见 How to Play Protected Media Files。
其实不太了解以上概念也没关系，通过这个小例子的动手实践，我们会对一些基本概念有个基本了解。
概述
以下对象用来和Media Session播放多媒体文件：
- media source对象用来解析多媒体文件或其他媒体数据源。media source为文件中的每个音频或视频流创建一个steam对象。 Decoders把编码后的多媒体数据转换为非压缩视频和音频
- Source Resolver从URL创建一个media source
- EVR将视频渲染到屏幕上
- SAR将音频渲染至扬声器或其他音频输出设备
- Topology定义从media source至EVR和SAR的数据流
- Media Session控制数据流，并发送状态数据到应用程序。下图展示了这个过程
step by step实例
大概了解一下概念，我们可以来进行实践了。我们主要将完成以下任务：
- Media Foundation平台的初始化与关闭
- 创建media session
- 根据文件路径，（智能）创建（合适的）media source
- 创建topology，添加media source、EVR/SAR（renderer）节点，并将其连接，此时的topology是一个partial topology
- 将刚创建的topology关联到media session，内部的topology loader会给partial topology“智能地”加入所需的解码器等节点，使其成为一个complete topology
- 获取和处理来自media session的事件
- 用media session来控制播放，但不要直接操作media source
- 程序结束，释放资源
1. 创建程序
我使用visual studio 2012创建了一个基于对话框的MFC项目。含有可缩放的边框、 最小化框。
再创建一个菜单，把对话框的菜单属性设为此菜单。
添加全局的播放核心类对象
Core* g_pCore = NULL;
在stdafx.h中添加要用到的头文件和类模板。头文件后面的注释说明了为什么需要它
#include <mfapi.h> // MFStartup, mfplat.lib
#include <mfidl.h> // MFCreateMediaSession, mf.lib
#include <evr.h> // IMFVideoDisplayControl, strmiids.lib
#include <shlwapi.h> // QITABENT, shlwapi.lib
#include <mferror.h> // MF_E_ALREADY_INITIALIZED
template <class T> void SafeRelease(T **ppT)
{
if (*ppT)
{
(*ppT)->Release();
*ppT = NULL;
}
}
#include "Core.h" // Core类头文件
配置项目属性，此项目需要链接以下Lib:
mfplat.lib; mf.lib; mfuuid.lib; strmiids.lib; shlwapi.lib
2. 创建播放核心类Core
首先定义播放事件标识和播放状态的枚举
const UINT WM_APP_PLAYER_EVENT = WM_APP + 1;
enum PlayerState
{
Closed = 0, // No session.
Ready, // Session was created, ready to open a file.
OpenPending, // Session is opening a file.
Started, // Session is playing a file.
Paused, // Session is paused.
Stopped, // Session is stopped (ready to play).
Closing // Application has closed the session,
// but is waiting for MESessionClosed.
};
此类被设计为单例模式（singleton），为了处理事件方便，它继承自IMFAsyncCallback接口，此接口又继承自IUnknown接口。因此我们要为Core类实现这些接口的所有必需方法。
类声明：
class Core : public IMFAsyncCallback
{
protected:
Core(HWND hVideo);
virtual ~Core(void);
public:
static HRESULT CreateInstance(HWND hVideo, Core** ppCore);
// IUnknown方法
STDMETHODIMP QueryInterface(REFIID iid, void** ppv);
STDMETHODIMP_(ULONG) AddRef();
STDMETHODIMP_(ULONG) Release();
// IMFAsyncCallback方法
STDMETHODIMP GetParameters(DWORD*, DWORD*)
{ return E_NOTIMPL;}
STDMETHODIMP Invoke(IMFAsyncResult* pAsyncResult);
HRESULT Initialize();
HRESULT OpenFile(PCWSTR sURL);
HRESULT HandleEvent(UINT_PTR pEvent);
PlayerState GetState() const { return m_state; }
BOOL HasVideo() const { return (m_pVideoControl != NULL); }
HRESULT StartPlay();
HRESULT Play();
HRESULT Pause();
HRESULT Stop();
HRESULT Shutdown();
HRESULT Repaint();
HRESULT ResizeVideo(WORD width, WORD height);
HRESULT OnTopologyStatus(IMFMediaEvent*);
HRESULT OnPresentationEnded(IMFMediaEvent*);
private:
long m_nRefCount;
PlayerState m_state;
IMFMediaSession* m_pMediaSession;
IMFMediaSource* m_pMediaSource;
IMFVideoDisplayControl* m_pVideoControl;
HWND m_hwndVideo;
HANDLE m_hCloseEvent;
};
构造函数、接口以及初始化等函数的实现， Event句柄m_hCloseEvent用来设置播放关闭时的标志，m_hwndVideo就是播放用的视频窗口，我设为主对话框的客户窗口，m_pVideoControl是一个IMFVideoDisplayControl接口指针，用来完成播放窗口相关的控制，如调整尺寸和重绘：
Core::Core(HWND hVideo) :
m_pMediaSession(NULL),
m_pMediaSource(NULL),
m_pVideoControl(NULL),
m_hwndVideo(hVideo),
m_hCloseEvent(NULL),
m_state(Closed),
m_nRefCount(1)
{
}
Core::~Core(void)
{
if(m_pMediaSession)
{
Shutdown();
}
}
HRESULT Core::QueryInterface(REFIID riid, void** ppv)
{
static const QITAB qit[] =
{
QITABENT(Core, IMFAsyncCallback),
{ 0 }
};
return QISearch(this, qit, riid, ppv);
}
ULONG Core::AddRef()
{
return InterlockedIncrement(&m_nRefCount);
}
ULONG Core::Release()
{
ULONG uCount = InterlockedDecrement(&m_nRefCount);
if(uCount == 0)
delete this;
return uCount;
}
HRESULT Core::CreateInstance(HWND hVideo, Core** ppCore)
{
if(hVideo == NULL) return E_UNEXPECTED;
if(ppCore == NULL) return E_POINTER;
Core* pCore = new Core(hVideo);
if(pCore == NULL) return E_OUTOFMEMORY;
HRESULT hr = pCore->Initialize();
if(SUCCEEDED(hr))
{
*ppCore = pCore;
(*ppCore)->AddRef();
}
SafeRelease(&pCore);
return hr;
}
HRESULT Core::Initialize()
{
if(m_hCloseEvent)
return MF_E_ALREADY_INITIALIZED;
HRESULT hr = MFStartup(MF_VERSION);
if(FAILED(hr)) return hr;
m_hCloseEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
if(m_hCloseEvent == NULL)
hr = HRESULT_FROM_WIN32(GetLastError());
return hr;
}
3. 打开文件时的处理
这是一个核心的部分，代码较多，为了简单，我把所有代码都写到一个函数里去了，主要是为了理解方便。
主要完成了以下工作：
- 创建media session
- 开始从media session取得事件，开启播放相关事件的流动
- 根据文件路径创建合适的media source
- 创建partial topology
- 为media source的各个选定(selected)流创建source节点和renderer节点，将它们添加到partial topology并相互连接
- 将partial topology与media session关联，将partial topology解析成完整可用的topology
HRESULT Core::OpenFile(PCWSTR sURL)
{
assert(m_state == Closed || m_state == Stopped);
// 创建Media Session
HRESULT hr = MFCreateMediaSession(NULL, &m_pMediaSession);
if(FAILED(hr)) return hr;
m_state = Ready;
// 开始从Media Session取得事件
hr = m_pMediaSession->BeginGetEvent((IMFAsyncCallback*)this, NULL);
if(FAILED(hr)) return hr;
// 创建Media Source
IMFSourceResolver* pSourceResolver = NULL;
IUnknown* pUnknown = NULL;
IMFTopology* pTopology = NULL;
IMFPresentationDescriptor* pPD = NULL;
MF_OBJECT_TYPE objType = MF_OBJECT_INVALID;
SafeRelease(&m_pMediaSource);
hr = MFCreateSourceResolver(&pSourceResolver);
if(FAILED(hr)) goto over;
// 为简单，弄成同步方法
hr = pSourceResolver->CreateObjectFromURL(
sURL,
MF_RESOLUTION_MEDIASOURCE,
NULL,
&objType,
&pUnknown);
if(FAILED(hr)) goto over;
hr = pUnknown->QueryInterface(IID_PPV_ARGS(&m_pMediaSource));
if(FAILED(hr)) goto over;
// 创建Topology
assert(m_pMediaSession != NULL);
assert(m_pMediaSource != NULL);
DWORD cStreams = 0;
hr = MFCreateTopology(&pTopology);
if(FAILED(hr)) goto over;
// 创建PresentationDescriptor
hr = m_pMediaSource->CreatePresentationDescriptor(&pPD);
if(FAILED(hr)) goto over;
// 获取source中的stream数目
hr = pPD->GetStreamDescriptorCount(&cStreams);
if(FAILED(hr)) goto over;
assert(pTopology != NULL);
// 为每个stream创建topology节点，并将其加入到topology
for(DWORD i = 0; i<cStreams; i++ )
{
IMFStreamDescriptor* pSD = NULL;
IMFTopologyNode* pSourceNode = NULL;
IMFTopologyNode* pOutputNode = NULL;
BOOL bSelected = FALSE;
hr= pPD->GetStreamDescriptorByIndex(i, &bSelected, &pSD);
if(FAILED(hr)) goto over2;
if(bSelected)
{
// source node
hr = MFCreateTopologyNode(MF_TOPOLOGY_SOURCESTREAM_NODE, &pSourceNode);
if(FAILED(hr)) goto over2;
hr = pSourceNode->SetUnknown(MF_TOPONODE_SOURCE, m_pMediaSource);
if(FAILED(hr)) goto over2;
hr = pSourceNode->SetUnknown(MF_TOPONODE_PRESENTATION_DESCRIPTOR, pPD);
if(FAILED(hr)) goto over2;
hr = pSourceNode->SetUnknown(MF_TOPONODE_STREAM_DESCRIPTOR, pSD);
if(FAILED(hr)) goto over2;
// output node
IMFMediaTypeHandler* pHandler = NULL;
IMFActivate* pRendererActivate = NULL;
GUID guidMajorType = GUID_NULL;
DWORD id = 0;
pSD->GetStreamIdentifier(&id); // 忽略错误
hr = pSD->GetMediaTypeHandler(&pHandler);
if(FAILED(hr)) goto over3;
hr = pHandler->GetMajorType(&guidMajorType);
if(FAILED(hr)) goto over3;
hr = MFCreateTopologyNode(MF_TOPOLOGY_OUTPUT_NODE, &pOutputNode);
if(FAILED(hr)) goto over3;
if(MFMediaType_Audio == guidMajorType)
hr = MFCreateAudioRendererActivate(&pRendererActivate);
else if(MFMediaType_Video == guidMajorType)
hr = MFCreateVideoRendererActivate(m_hwndVideo, &pRendererActivate);
else
hr = E_FAIL;
if(FAILED(hr)) goto over3;
hr = pOutputNode->SetObject(pRendererActivate);
if(FAILED(hr)) goto over3;
// 把source节点和输出节点添加到topology，并连接它们
hr = pTopology->AddNode(pSourceNode);
if(FAILED(hr)) goto over3;
hr = pTopology->AddNode(pOutputNode);
if(FAILED(hr)) goto over3;
hr = pSourceNode->ConnectOutput(0, pOutputNode, 0);
over3:
SafeRelease(&pRendererActivate);
SafeRelease(&pHandler);
goto over2;
}
over2:
SafeRelease(&pSD);
SafeRelease(&pSourceNode);
SafeRelease(&pOutputNode);
}
hr = m_pMediaSession->SetTopology(0, pTopology);
if(FAILED(hr)) goto over;
m_state = OpenPending;
over:
if(FAILED(hr)) m_state = Closed;
SafeRelease(&pPD);
SafeRelease(&pTopology);
SafeRelease(&pSourceResolver);
SafeRelease(&pUnknown);
return hr;
}
主对话框需要提供一个文件/打开菜单，用来打开文件，它的响应如下。由于IMFSourceResolver::CreateObjectFromURL方法只支持LPCWSTR类型的文件路径字符串参数，所以当我们以多字节配置build程序时，需要添加代码，把打开文件对话框取得的多字节文件路径字符串转换成宽字符串。
void CBlackPlayerDlg::OnOpen()
{
HRESULT hr = S_OK;
TCHAR path[MAX_PATH];
path[0] = _T('\0');
OPENFILENAME ofn;
::ZeroMemory(&ofn, sizeof(ofn));
ofn.lStructSize = sizeof(ofn);
ofn.hwndOwner = this->GetSafeHwnd();
ofn.lpstrFilter = _T("Media Files\0*.asf;*.avi;*.mp3;")
_T("*.mp4;*.wav;*.wma;*.wmv\0All files\0*.*\0");
ofn.lpstrFile = path;
ofn.nMaxFile = MAX_PATH;
ofn.Flags = OFN_FILEMUSTEXIST;
ofn.hInstance = AfxGetInstanceHandle();
if(::GetOpenFileName(&ofn))
{
#ifdef UNICODE
hr = g_pCore->OpenFile(ofn.lpstrFile);
#else
size_t cLen = 0, cChars = 0;
cLen = _tcslen(ofn.lpstrFile);
WCHAR wstr[MAX_PATH*2] = {L'\0'};
::MultiByteToWideChar(CP_ACP, 0, ofn.lpstrFile, -1, (LPWSTR)wstr, MAX_PATH);
hr = g_pCore->OpenFile(wstr);
#endif
if(SUCCEEDED(hr))
{
UpdateUI(g_pCore->GetState());
}
else
{
AfxMessageBox(_T("无法打开文件！"));
}
}
}
4. 处理media session事件
第3步代码的开头我们已经使用BeginGetEvent方法开始获取Media Session的事件，这是一个异步方法，当下一个事件发生时，media session会调用IMFAsyncCallback::Invoke方法。注意此方法是在worker线程调用的，不是主程序所在线程，所以这方法必须线程安全。
HRESULT Core::Invoke(IMFAsyncResult* pResult)
{
MediaEventType meType = MEUnknown;
IMFMediaEvent* pEvent = NULL;
// 从事件队列中获取事件
HRESULT hr = m_pMediaSession->EndGetEvent(pResult, &pEvent);
if(FAILED(hr)) goto over;
// 取得事件的类型
hr = pEvent->GetType(&meType);
if(FAILED(hr)) goto over;
if(meType == MESessionClosed)
{
::SetEvent(m_hCloseEvent);
}
else
{
hr = m_pMediaSession->BeginGetEvent(this, NULL);
if(FAILED(hr)) goto over;
}
if(m_state != Closing)
{
pEvent->AddRef();
::PostMessage(m_hwndVideo, WM_APP_PLAYER_EVENT, (WPARAM)pEvent, (LPARAM)0);
}
over:
SafeRelease(&pEvent);
return S_OK;
}
Invoke方法中，我们先用EndGetEvent取得事件，接着用PostMessage将此事件发送给主程序窗口，其实主窗口还是把它送回给了Core的HandleEvent方法来进行实际处理。还要再次调用BeginGetEvent方法来异步获取下一事件。所以主对话框类要添加WM_APP_PLAYER_EVENT事件的处理程序：
afx_msg LRESULT CBlackPlayerDlg::OnPlayerEvent(WPARAM wParam, LPARAM lParam)
{
HRESULT hr = S_OK;
hr = g_pCore->HandleEvent(wParam);
if(FAILED(hr))
AfxMessageBox(_T("事件处理发生错误！"));
UpdateUI(g_pCore->GetState());
return 0;
}
以下是Core类的HandleEvent，还可以根据需要添加更多的事件处理：
HRESULT Core::HandleEvent(UINT_PTR pEventPtr)
{
HRESULT hrStatus = S_OK;
HRESULT hr = E_FAIL;
IMFMediaEvent* pEvent = NULL;
IUnknown* pUnk = (IUnknown*)pEventPtr;
if(pUnk == NULL) return E_POINTER;
hr = pUnk->QueryInterface(IID_PPV_ARGS(&pEvent));
if(FAILED(hr)) goto over;
MediaEventType meType = MEUnknown;
hr = pEvent->GetType(&meType);
if(FAILED(hr)) goto over;
hr = pEvent->GetStatus(&hrStatus);
if(FAILED(hr)) goto over;
if(FAILED(hrStatus))
{
hr = hrStatus;
goto over;
}
switch(meType)
{
case MESessionTopologyStatus:
hr = OnTopologyStatus(pEvent);
break;
case MEEndOfPresentation:
hr = OnPresentationEnded(pEvent);
break;
default:
break;
}
over:
SafeRelease(&pUnk);
SafeRelease(&pEvent);
return hr;
}
相关的具体事件的处理方法 ：
HRESULT Core::OnTopologyStatus(IMFMediaEvent* pEvent)
{
MF_TOPOSTATUS status;
HRESULT hr = pEvent->GetUINT32(MF_EVENT_TOPOLOGY_STATUS, (UINT32*)&status);
if(SUCCEEDED(hr) && (status == MF_TOPOSTATUS_READY))
{
SafeRelease(&m_pVideoControl);
// 如果source没有视频stream，此方法将失败
(void)MFGetService(m_pMediaSession, MR_VIDEO_RENDER_SERVICE,
IID_PPV_ARGS(&m_pVideoControl));
hr = StartPlay();
}
return hr;
}
HRESULT Core::OnPresentationEnded(IMFMediaEvent* pEvent)
{
m_state = Stopped;
return S_OK;
}
5. 控制播放
构建好完整的topology、设置好事件处理之后，可以真正地开始播放了，以下是播放的一些相关代码：
// 从当前位置开始播放
HRESULT Core::StartPlay()
{
PROPVARIANT varStart;
PropVariantInit(&varStart);
varStart.vt = VT_EMPTY;
// start是异步方法
HRESULT hr = m_pMediaSession->Start(&GUID_NULL, &varStart);
if(SUCCEEDED(hr))
m_state = Started;
PropVariantClear(&varStart);
return hr;
}
HRESULT Core::Play()
{
if(m_state != Paused && m_state != Stopped)
return INET_E_INVALID_REQUEST;
if(m_pMediaSession == NULL || m_pMediaSource == NULL)
return E_UNEXPECTED;
return StartPlay();
}
HRESULT Core::Pause()
{
if(m_state != Started)
return INET_E_INVALID_REQUEST;
if(m_pMediaSession == NULL || m_pMediaSource == NULL)
return E_UNEXPECTED;
HRESULT hr = m_pMediaSession->Pause();
if(SUCCEEDED(hr))
m_state = Paused;
return hr;
}
HRESULT Core::Stop()
{
if(m_state != Started && m_state != Paused)
return INET_E_INVALID_REQUEST;
if(m_pMediaSession == NULL)
return E_UNEXPECTED;
HRESULT hr = m_pMediaSession->Stop();
if(SUCCEEDED(hr))
m_state = Stopped;
return hr;
}
视频render(EVR)在我们提供的视频窗口绘制视频图象，这发生在一个工作线程，一般不需要管它。但如果播放暂停或停止时，当视频窗口接收到WM_PAINT消息时，我们必须通知EVR，方法是调用IMFVideoDisplayControl::RepaintVideo方法：
HRESULT Core::Repaint()
{
if(m_pVideoControl)
return m_pVideoControl->RepaintVideo();
else
return S_OK;
}同时，我们还需要修改主对话框的OnPaint事件处理函数：
void CBlackPlayerDlg::OnPaint()
{
if (IsIconic())
{
// 略。。。。
}
else
{
if(g_pCore && g_pCore->HasVideo())
g_pCore->Repaint();
CDialogEx::OnPaint();
}
}
我们还需要让用户可以调整视频窗口大小，这通过IMFVideoDisplayControl::SetVideoPostition方法来完成：
HRESULT Core::ResizeVideo(WORD width, WORD height)
{
if(m_pVideoControl)
{
RECT rc = {0, 0, width, height};
return m_pVideoControl->SetVideoPosition(NULL, &rc);
}
else
return S_OK;
}类似与重绘，我们也要相应修改主对话框类，为其添加WM_SIZE的消息处理程序：
void CBlackPlayerDlg::OnSize(UINT nType, int cx, int cy)
{
CDialogEx::OnSize(nType, cx, cy);
if(g_pCore != NULL && g_pCore->HasVideo())
g_pCore->ResizeVideo(cx, cy);
}
6. 结束后的清理
播放完成后我们需要做一些清理工作，比如析构函数中的ShutDown方法：
HRESULT Core::Shutdown()
{
HRESULT hr = S_OK;
SafeRelease(&m_pVideoControl);
if(m_pMediaSession)
{
DWORD dwRet = 0;
m_state = Closing;
hr = m_pMediaSession->Close();
if(FAILED(hr)) goto over;
dwRet = WaitForSingleObject(m_hCloseEvent, 3000);
}
// 以下shutdown都是同步方法，无事件
if(m_pMediaSource)
m_pMediaSource->Shutdown();
if(m_pMediaSession)
m_pMediaSession->Shutdown();
SafeRelease(&m_pMediaSource);
SafeRelease(&m_pMediaSession);
m_state = Closed;
MFShutdown();
if(m_hCloseEvent)
{
::CloseHandle(m_hCloseEvent);
m_hCloseEvent = NULL;
}
over:
return hr;
}
我在对话框类的析构函数里添加：
CBlackPlayerDlg::~CBlackPlayerDlg()
{
if(g_pCore != NULL)
{
g_pCore->Shutdown();
g_pCore->Release();
}
}
7. 杂项
如果要用键盘的空格键控制视频的暂停/重新播放，主对话框类需要重载PreTranslateMessage函数：
// 必须直接重载此函数，直接处理WM_CHAR或WM_KEYUP消息不行
BOOL CBlackPlayerDlg::PreTranslateMessage(MSG* pMsg)
{
// TODO: 在此添加专用代码和/或调用基类
if(g_pCore != NULL && pMsg->message == WM_KEYUP)
{
if(pMsg->wParam == VK_SPACE)
{
if(g_pCore->GetState() == Started)
g_pCore->Pause();
else if(g_pCore->GetState() == Paused)
g_pCore->Play();
}
return TRUE;
}
return CDialogEx::PreTranslateMessage(pMsg);
}
如果要添加个控制菜单，里面有暂停、停止，它们的响应如下：
void CBlackPlayerDlg::OnPause()
{
if(g_pCore != NULL)
{
if(g_pCore->GetState() == Started)
g_pCore->Pause();
}
}
void CBlackPlayerDlg::OnStop()
{
if(g_pCore != NULL)
{
if(g_pCore->GetState() == Started || g_pCore->GetState() == Paused)
g_pCore->Stop();
}
}
