Usage of Protocol Buffer

Usage of Protocol Buffer

1        Introduction

　　Protocol buffers supports a language-neutral, platform-neutral,extensible way of serializing structured data for use in communicationsprotocols, data storage, and more.

　　Protocol buffers are a flexible, efficient, automated mechanism forserializing structured data – think XML, but smaller, faster, and simpler.However, protocol buffer will save data in binary mode, it’s not readable. XMLis readable. XML is a standard of W3C,protocol buffer is not a international standard.

　　Note: If you don’t care if the data is readable,you can use protocol buffer. The biggest merit is that you don’t need to writethe parser code and test it.

　　NOTE: If you want to start protocol bufferquickly, you only need to read part: language guide 3.1-3.3 and part 4:C++Tutorials.

2        What you need to do when usingprotocol buffer?

1)     Download protocol buffer andinstall it;
2)     Write the specification file todefine your structured information in .proto files.
3)     Compile the .proto filesinto your language, C++, Java or Python.
4)     Using the generated classes inyour code.

3        Language Guide (Writing .protocfile)

　　This guide describes how to use theprotocol buffer language to structure your protocol buffer data, including .proto filesyntax and how to generate data access classes from your .protofiles.

3.1      Defining A Message Type

　　In protocol buffer, message type is like aclass or structure. It is the basic component of protoc file. You can defineyour message as follow:

message name{

       Messagevar1;   # field1

       Messagevar2;   # field2

       ……

}

Message Variable Format:

Field Rule Field Type var_name = Unique numbered Tag [default= 10];  //Comments

Field Rule defines how todeal with this field in protocol buffer.

    * required: a well-formed message must have exactlyone of this field.

    * optional: a well-formed message can have zero or oneof this field (but not more than one).

　 * repeated: this field can be repeated any number oftimes (including zero) in a well-formed message. The order of the repeatedvalues will be preserved.

Note: Required Is Forever. If you use required for one field, youdon’t have the change to modify the field any more. Thus pay more attention onrequired, using optional and repeated instead except that you are sure that thefield can’t be change any more.

Field Type defines the variable type. Field type is language-neutral, platform-neutral and is will be map to relatedactual variable type when you compile protoc file. 

.proto Type	Notes	C++ Type	Java Type
double		double	double
float		float	float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long
uint32	Uses variable-length encoding.	uint32	int[1]
uint64	Uses variable-length encoding.	uint64	long[1]
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 228.	uint32	int[1]
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 256.	uint64	long[1]
sfixed32	Always four bytes.	int32	int
sfixed64	Always eight bytes.	int64	long
bool		bool	boolean
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String
bytes	May contain any arbitrary sequence of bytes.	string	ByteString

Unique Numbered Tag:Each field in the message definition has a uniquenumbered tag. These tags are used to identify your fields in the messagebinary format, and should not be changed once your message type is in use.

Note:

1)     Tags with values in the range 1through 15 take one byte to encode. Tags in therange 16 through 2047 take two bytes. So youshould reserve the tags 1 through 15 for very frequently occurring messageelements and leave some room for frequently occurring elements that might beadded in the future.

2)     Numbers19000 though 19999 are reserved for the Protocol Buffersimplementation, not use them.

Protocol buffer alsosupport Enum, Nested message, Package, import and Service (RPC--Remote Procedure Call). We will learn them from example.

3.2      Example

package EXAM;        // define a package, like namespace
import "myproject/other_protos.proto";   // import other protoc file, like include

message SearchRequest {              // define a message
  required string query = 1;             // define message fields
  optional int32 page_number = 2;
  optional int32 result_per_page = 3 [default = 10];

  enum Corpus {                    // define an enum a
    UNIVERSAL = 0;
    WEB = 1;
    IMAGES = 2;
    LOCAL = 3;
    NEWS = 4;
    PRODUCTS = 5;
    VIDEO = 6;
  }

optional Corpus corpus = 4 [default = UNIVERSAL];   // use an option value

message Result {                // nested message
  required string url = 1;
  optional string title = 2;
  repeated string snippets = 3;
}

repeated Result result = 5;
}

message other{           // define multi messages in same file
       required string url = 1;
       .......
} 

3.3      Generating Your Classes

To generate the Java, Python, or C++ codeyou need to work with the message types defined in a .proto file,you need to run the protocol buffer compiler protoc on the .proto.Example:

　　protoc --proto_path=IMPORT_PATH --cpp_out=DST_DIR --java_out=DST_DIR --python_out=DST_DIR path/to/file.proto

This command will compile protoc file intoC++/Java/Python code.

Using “protoc –help” for detail.

3.4      Extensions

　　Extensions let you declare that a range offield numbers, which are available for third-partyextensions, in a message. Other peoplecan then declare new fields for your message type with those numeric tags in their own .proto files without having to edit the original file. Let's look at an example:

message Foo {

  // ...

  extensions 100 to 199;

}

　　Other users can now add new fields to Foo in theirown .proto files that import your .proto, using tagswithin your specified range – for example:

extend Foo {

  optional int32 bar = 126;

}

　　However, the way you access extensionfields in your application code is slightly different to accessing regularfields – your generated data access code has special accessors for working withextensions. So, for example, here's how you set the value of bar in C++:

Foo foo;

foo.SetExtension(bar, 15);

3.5      Rules for Updating A Message Type

• Don't change the numeric tagsfor any existing fields.

• Any new fields that you addshould be optional or repeated.

• Non-required fields can be removed,as long as the tag number is not used again in your updated message type.

• A non-required field can beconverted to an extension and vice versa, as long as the type and number staythe same.

• int32, uint32, int64, uint64,and bool are all compatible – this means you can change a field from one ofthese types to another without breaking forwards- or backwards-compatibility.

• sint32 and sint64 arecompatible with each other but are not compatible with the other integer types.

• string and bytes are compatibleas long as the bytes are valid UTF-8.

• Embedded messages arecompatible with bytes if the bytes contain an encoded version of the message.

• fixed32 is compatible withsfixed32, and fixed64 with sfixed64.

3.6      FAQ

Ø        Can I change or the generated code?

Don’t to that! Protocol buffer classes are basically dumb data holders (likestructs in C++). If you want to add richer behavior to a generated class, thebest way to do this is to wrap the generated protocol buffer class in anapplication-specific class.You should never add behavior to thegenerated classes by inheriting from them.This will break internal mechanisms and is not good object-oriented practiceanyway.

Ø        How to write multi messages into one file (buffer)?

If you want to write multiple messages to a single file or stream,it is up to you to keep track of where one message ends and the next begins. Inother word, you need to write/read the message serialization string to/fromfile correctly.

Ø        Can protocol buffer deal with Large Data Sets?

Protocol Buffers are not designed to handle large messages. As ageneral rule of thumb, if you are dealing in messages larger than a megabyteeach, it may be time to consider an alternate strategy.

Ø        How to optimize protocol generated code?

1) Using option, Such as “option optimize_for= LITE_RUNTIME/CODE_SIZE;”. 2) Reuse message objects when possible.

4        C++ Tutorials

Common API

Standard Message Methods

　　Each message class also contains a numberof other methods that let you check or manipulate the entire message,including:

    * boolIsInitialized() const;: checks if allthe required fields have been set.

    * string DebugString() const;: returns ahuman-readable representation of the message, particularly useful fordebugging.

    * void CopyFrom(const Person& from);: overwritesthe message with the given message's values.

　 * void Clear();: clears all theelements back to the empty state.

Parsing and Serialization

　　Each protocol buffer class has methods forwriting and reading messages of your chosen type using the protocol bufferbinary format. These include:

* bool SerializeToString(string* output) const;: serializes the message and stores the bytes in the given string.Note that the bytes are binary, not text; we only use the string class as aconvenient container.

* bool ParseFromString(const string& data);: parses a message from the given string.

* bool SerializeToOstream(ostream* output) const;: writes the message to the given C++ ostream.

* bool ParseFromIstream(istream* input);: parses a message from the given C++ istream.

Example

　　The example code isincluded in the source code package, under the "examples" directory.Study the example!!!

Refer to http://code.google.com/intl/zh-CN/apis/protocolbuffers/docs/cpptutorial.html

NOTE: For each field, get_field() andset_field() interfaces will be generated. However,these interfaces only support lowercase field regardless of Field or field isused in protoc file.

More detail, refer to: http://code.google.com/intl/zh-CN/apis/protocolbuffers/docs/overview.html

----------------------------------------------------

兄弟的公司：立即购--手机购物，诚信网购

欢迎转载，请注明作者和出处。