实现HTTP协议Get、Post和文件上传功能——设计和模块

        之前写过一遍《使用WinHttp接口实现HTTP协议Get、Post和文件上传功能》，其中谈到了如何使用WinHttp接口实现Http的Get、Post和文件上传功能。后来发现蛮多人关注该技术的实现，于是我决定重新设计框架结构，梳理这块的知识并提供可供测试的方案。同时，该系列也将作为《VC开发Windows客户端软件之旅》中"网络"模块的一部分。（转载请指明出于breaksoftware的csdn博客）

        本系列不再将技术限定于WinHttp接口，还引入curllib库。同时为了公正且方便测试代码的正确性，我们将引入成熟的技术方案进行测试。

测试环境

        使用Python搭建一个Http服务器，用于检测Get和Post请求。

        使用hfs(http file server)搭建Http服务器，用于检测文件上传功能。

        使用curl作为发送Get、Post和文件上传的工具。

        hfs和curl比较方便获取，我们只要在官网上下载可用的二进制文件即可。

     hfs配置

      如上图，给该服务器新增一个真实目录（real floder)。然后设置该目录的权限为anyone。

     Python配置和脚本

        我们选用webpy库作为框架（python使用2.7）。其安装方法详见http://webpy.org/install.zh-cn。然后我们提供如下脚本打印输入数据

import web

urls = (
    '/get','get_class',
    '/post','post_class',
)

# deal get request
class get_class:
    def GET(self):
        request_data = web.input()
        print request_data
        return "hello,world!"
    
#deal post request
class post_class:   
    def POST(self):
        request_data = web.input()
        print request_data
        data = web.data()
        print data
        return "hello,world!"
        
if __name__ == '__main__':
    app = web.application(urls, globals())
    app.run()

     发送测试数据

     上传文件到hfs服务

curl -F "action=upload" -F "filename=@C:/vcredist_x86.log" http://192.168.191.1:8000/12

 

       这样curl便将文件上传到服务器了。

      发送Get请求

curl "http://127.0.0.1:8080/get?k1=v1"

        python打印

      发送Post请求

curl -d "key1=value1&key2=value2" "http://127.0.0.1:8080/post?k1=v1"

        python打印

key1=value1&key2=value2
127.0.0.1:44834 - - [26/May/2015 19:56:22] "HTTP/1.1 POST /post" - 200 OK

        通过如上的请求和信息，我们将对照我们之后编写的代码，以检测我们代码的有效性。

框架设计

        “下载” 这两个字将是我设计该框架的来源。“下”是将数据从某端传输过来；“载”是将传过来的数据在本地保存。怎么“下”，如何“载”是可以选择的。于是这两种行为我分别定义了两个虚类——ToolsInterface::IHttpRequest和ToolsInterface::IMemFileOperation。“下”，我们可以采用WinHttp接口或者Curl接口。“载”，我们可以选择保存在内存中，还是保存到磁盘上。然后这两个独立的行为，我是用一个粘合类——ToolsInterface::CSetOperation让这两种行为产生关联，同时解耦。

        上图就是整个的框架图。它完整反映了各个类的关系。相较于上个版本，我废弃掉了Upload类，而将其融合在Post协议类中。之后我们将结合该图进行扩展和讲解。

        首先我们看下“下”的接口函数定义

 

	class IHttpRequest{
	public:
		IHttpRequest(){};
		virtual ~IHttpRequest(){};
	public:
		virtual void Start() = 0;
		virtual void Stop() = 0;
		virtual bool IsSuc() = 0;

		virtual void SetUrl(const std::string& strUrl) = 0;
		virtual void SetAgent(const std::string& strAgent) = 0;
		virtual void AppendHeader(const ListStr& listHeaders) = 0;

		virtual void SetTimeOut(int nResolveTimeout,
			int nConnectTimeout,
			int nSendTimeout,
			int nReceiveTimeout) = 0;
        
        virtual void SetFinishEvent(HANDLE hFinish) = 0;

		void SetProcessCallBack(LPCallback fProcess){m_CallBack = fProcess;};
	protected:
		ToolsInterface::LPCallback m_CallBack;
	};

       该类的函数命名比较直接，从其命名应该可以猜测出其中的意思。这个接口较上个版本的变化是：废弃了SetParams方法。因为Get协议不需要设置参数，它的参数可以直接在URL里携带过来。

       我们再看下“载”所提供的接口定义

	class IMemFileOperation {
	public:
		IMemFileOperation(){};
		virtual ~IMemFileOperation(){};
	public:
		// void clearerr ( FILE * stream )
		// Resets both the error and the eof indicators of the stream.
		/* 
		When a i/o function fails either because of an error or because the end of the file has been reached, 
		one of these internal indicators may be set for the stream. 
		The state of these indicators is cleared by a call to this function, 
		or by a call to any of: rewind, fseek, fsetpos and freopen.
		*/
		virtual void MFClearErr() {};

		// int fclose ( FILE * stream );
		// Closes the file associated with the stream and disassociates it.
		/*
		All internal buffers associated with the stream are disassociated from it and flushed: 
		the content of any unwritten output buffer is written and the content of any unread input buffer is discarded.
		Even if the call fails, the stream passed as parameter will no longer be associated with the file nor its buffers.
		*/
		virtual bool MFClose() = 0;

		// int feof ( FILE * stream );
		// Checks whether the end-of-File indicator associated with stream is set, returning a value different from zero if it is.
		/*
		This indicator is generally set by a previous operation on the stream that attempted to read at or past the end-of-file.
		Notice that stream's internal position indicator may point to the end-of-file for the next operation, 
		but still, the end-of-file indicator may not be set until an operation attempts to read at that point.
		This indicator is cleared by a call to clearerr, rewind, fseek, fsetpos or freopen. 
		Although if the position indicator is not repositioned by such a call, the next i/o operation is likely to set the indicator again.
		*/
		virtual int MFEof() = 0;

		// int ferror ( FILE * stream );
		// Checks if the error indicator associated with stream is set, returning a value different from zero if it is.
		/*
		This indicator is generally set by a previous operation on the stream that failed, and is cleared by a call to clearerr, rewind or freopen.
		*/
		virtual int MFError() {return 0;};

		// int fflush ( FILE * stream );
		// Flush stream
		/*
		If the given stream was open for writing (or if it was open for updating and the last i/o operation was an output operation) any unwritten data in its output buffer is written to the file.
		If stream is a null pointer, all such streams are flushed.
		In all other cases, the behavior depends on the specific library implementation.
		In some implementations, flushing a stream open for reading causes its input buffer to be cleared (but this is not portable expected behavior).
		The stream remains open after this call.
		When a file is closed, either because of a call to fclose or because the program terminates, all the buffers associated with it are automatically flushed.
		*/
		virtual int MFFlush() {return 0;};

		// int fgetc ( FILE * stream );
		// Get character from stream
		/*
		Returns the character currently pointed by the internal file position indicator of the specified stream. 
		The internal file position indicator is then advanced to the next character.
		If the stream is at the end-of-file when called, the function returns EOF and sets the end-of-file indicator for the stream (feof).
		If a read error occurs, the function returns EOF and sets the error indicator for the stream (ferror).
		fgetc and getc are equivalent, except that getc may be implemented as a macro in some libraries.
		*/
		virtual int MFGetc() = 0;

		// int fgetpos ( FILE * stream, fpos_t * pos );
		// Retrieves the current position in the stream.
		/*
		The function fills the fpos_t object pointed by pos with the information needed from the stream's position indicator to restore the stream to its current position (and multibyte state, if wide-oriented) with a call to fsetpos.
		The ftell function can be used to retrieve the current position in the stream as an integer value.
		*/
		virtual int MFGetPos( fpos_t * pos ) = 0;

		// char * fgets ( char * str, int num, FILE * stream );
		// Get string from stream
		/*
		Reads characters from stream and stores them as a C string into str until (num-1) characters have been read or either a newline or the end-of-file is reached, whichever happens first.
		A newline character makes fgets stop reading, but it is considered a valid character by the function and included in the string copied to str.
		A terminating null character is automatically appended after the characters copied to str.
		Notice that fgets is quite different from gets: not only fgets accepts a stream argument, 
		but also allows to specify the maximum size of str and includes in the string any ending newline character.
		*/
		virtual char * MFGets( char * str, int num ) = 0;

		// FILE * fopen ( const char * filename, const char * mode );
		// Open file
		/*
		Opens the file whose name is specified in the parameter filename and associates it with a stream that can be identified in future operations by the FILE pointer returned.
		The operations that are allowed on the stream and how these are performed are defined by the mode parameter.
		The returned stream is fully buffered by default if it is known to not refer to an interactive device (see setbuf).
		The returned pointer can be disassociated from the file by calling fclose or freopen. All opened files are automatically closed on normal program termination.
		The running environment supports at least FOPEN_MAX files open simultaneously.
		*/
		virtual bool MFOpen() = 0;

		// int fputc ( int character, FILE * stream );
		// Writes a character to the stream and advances the position indicator.
		/*
		The character is written at the position indicated by the internal position indicator of the stream, which is then automatically advanced by one.
		*/
		virtual int MFPutc( int character ) = 0;

		// int fputs ( const char * str, FILE * stream );
		// Writes the C string pointed by str to the stream.
		/*
		The function begins copying from the address specified (str) until it reaches the terminating null character ('\0'). 
		This terminating null-character is not copied to the stream.
		Notice that fputs not only differs from puts in that the destination stream can be specified, 
		but also fputs does not write additional characters, while puts appends a newline character at the end automatically.
		*/
		virtual int MFPuts( const char * str ) = 0;

		// size_t fread ( void * ptr, size_t size, size_t count, FILE * stream );
		// Read block of data from stream
		/*
		Reads an array of count elements, each one with a size of size bytes, from the stream and stores them in the block of memory specified by ptr.
		The position indicator of the stream is advanced by the total amount of bytes read.
		The total amount of bytes read if successful is (size*count).
		*/
		virtual size_t MFRead(void * ptr, size_t size, size_t count) = 0;

		// int fseek ( FILE * stream, long int offset, int origin );
		// Sets the position indicator associated with the stream to a new position.
		/*
		For streams open in binary mode, the new position is defined by adding offset to a reference position specified by origin.
		For streams open in text mode, offset shall either be zero or a value returned by a previous call to ftell, and origin shall necessarily be SEEK_SET.
		If the function is called with other values for these arguments, support depends on the particular system and library implementation (non-portable).
		The end-of-file internal indicator of the stream is cleared after a successful call to this function, and all effects from previous calls to ungetc on this stream are dropped.
		On streams open for update (read+write), a call to fseek allows to switch between reading and writing.
		*/
		virtual int MFSeek( long offset, int origin ) = 0;

		// int fsetpos ( FILE * stream, const fpos_t * pos );
		// Restores the current position in the stream to pos.
		/*
		The internal file position indicator associated with stream is set to the position represented by pos, 
		which is a pointer to an fpos_t object whose value shall have been previously obtained by a call to fgetpos.
		The end-of-file internal indicator of the stream is cleared after a successful call to this function, 
		and all effects from previous calls to ungetc on this stream are dropped.
		On streams open for update (read+write), a call to fsetpos allows to switch between reading and writing.
		A similar function, fseek, can be used to set arbitrary positions on streams open in binary mode.
		*/
		virtual int MFSetpos( const fpos_t * pos ) = 0;

		// long int ftell ( FILE * stream );
		// Returns the current value of the position indicator of the stream.
		/*
		For binary streams, this is the number of bytes from the beginning of the file.
		For text streams, the numerical value may not be meaningful but can still be used to restore the position to the same position later using fseek 
		(if there are characters put back using ungetc still pending of being read, the behavior is undefined).
		*/		
		virtual long MFTell() = 0;

		// size_t fwrite ( const void * ptr, size_t size, size_t count, FILE * stream );
		// Write block of data to stream
		/*
		Writes an array of count elements, each one with a size of size bytes, from the block of memory pointed by ptr to the current position in the stream.
		The position indicator of the stream is advanced by the total number of bytes written.
		Internally, the function interprets the block pointed by ptr as if it was an array of (size*count) elements of type unsigned char, 
		and writes them sequentially to stream as if fputc was called for each byte.
		*/
		virtual size_t MFWrite( const void * ptr, size_t size, size_t count ) = 0;
		
		// void rewind ( FILE * stream );
		// Sets the position indicator associated with stream to the beginning of the file.
		/*
		The end-of-file and error internal indicators associated to the stream are cleared after a successful call to this function, 
		and all effects from previous calls to ungetc on this stream are dropped.
		On streams open for update (read+write), a call to rewind allows to switch between reading and writing.
		*/
		virtual void MFRewind() = 0;

		/* UnSuport
		int putc ( int character, FILE * stream ); use MFPutc
		int getc ( FILE * stream ); use MFGetc
		FILE * freopen ( const char * filename, const char * mode, FILE * stream );
		void setbuf ( FILE * stream, char * buffer );
		int setvbuf ( FILE * stream, char * buffer, int mode, size_t size ); 
		int fprintf ( FILE * stream, const char * format, ... );
		int fscanf ( FILE * stream, const char * format, ... );
		int ungetc ( int character, FILE * stream );
		int vfprintf ( FILE * stream, const char * format, va_list arg );
		int vfscanf ( FILE * stream, const char * format, va_list arg ); 
		*/
	};

        这套接口是参考文件操作的方式定义的。因为不管是内存还是磁盘，它们都是保存数据的一种媒介。文件作为磁盘媒介保存的一种方式，它提供了完整的操作方法定义。所以我们可以认为文件操作涵盖了内存操作。于是我们借用文件操作方法去定义操作接口。这套接口的设计将大大简化我们之后发送Post参数或者上传文件的功能的编写，其巨大的魔力将在和CURL库结合使用之后得到展现。
        最后我们再看下粘合类

	class CSetOperation
	{
	public:
		CSetOperation(){m_pFMOp = NULL;};
		virtual ~CSetOperation(){};
	public:
        	void Open() {
            		if (m_pFMOp) {
                		m_pFMOp->MFOpen();
            		}
        	}
        	void Close() {
            		if (m_pFMOp) {
                		m_pFMOp->MFClose();
            		}
        	}
        	size_t Write( const void * ptr, size_t size, size_t count ) {
            		if (m_pFMOp) {
                		return m_pFMOp->MFWrite(ptr, size, count);
            		}
            	return 0;
        	}
		void SetOperation(IMemFileOperation* pFMop){m_pFMOp = pFMop;};
	private:
		IMemFileOperation* m_pFMOp;
	};

        各个实现了IHttpRequest接口的类通过继承该类获得“载”的能力。其实现也很简单——只是简单的转发，通过这层class，我们将“下”和“载”进行了解耦。

实例

        我们以使用WinHttp接口和内存保存方式的Get请求为例

	HttpRequestFM::CHttpTransByPost* p = new HttpRequestFM::CHttpTransByPost();
	ToolsInterface::IMemFileOperation* pMemOp = new MemFileOperation::CMemOperation();
	p->SetOperation(pMemOp);
	p->SetProcessCallBack(ProcssCallback);
	p->SetUrl(BIGFILEURL);
	p->Start();

	delete pMemOp;
	pMemOp = NULL;
	delete p;
	p = NULL;

        虽然我们是以内存形式保存数据，但是我们依然可以像使用文件的方式去访问它。如在delete pMemop之前插入如下代码

        FILE* file = fopen("F:/11.rar","wb+");
        pMemOp->MFOpen();
        while(!pMemOp->MFEof()) {
            char c = (char)pMemOp->MFGetc();
            fwrite(&c,1,1, file);
        }
        fclose(file);
        pMemOp->MFClose();

        FILE* file1 = fopen("F:/111.rar","wb+");
        pMemOp->MFOpen();
        while(!pMemOp->MFEof()) {
            char buffer[1024] = {0};
            size_t size = pMemOp->MFRead(buffer, 1, 1024);
            fwrite(buffer,1,size, file1);
        }
        fclose(file1);
        pMemOp->MFClose();

        因为本系列是讲解使用不同方式实现Http各种请求的，所以“载”的具体实现不做讨论。我将在之后的工程源码中给出。