By defining the protocol buffer message types in
.proto files, we can specify how our information to be serialized and how they are structured. Here is a basic example of a
.proto file that defines a message containing information about a person:
Each message type has one or more uniquely numbered fields, and each field has a name and a value type, where values types can be numbers (integer or floating point), booleans, strings, raw bytes, or even other protocol buffer message types, allowing us to structure our data hierarchically. The field can be
Once we have the message defined, we can run the protocal buffer compiler for our chosen language on the
.proto file to generate data access classes
Person. We can then use this class in our application to populate, serialize, and retrieve
Person protocol buffer messages.
In Java we can write codes to utilize the
Person class like this.
And similar codes in C++.
With Protocol Buffer, we can add new fields to our message formats without breaking backwards-compatibility; old binaries simply ignore the new field when parsing. Therefore, we can extend our protocol without worrying about breaking existing code.
- Define message formats in a
- Use the protocol buffer compiler.
- Use the Java/C++/Python protocol buffer API to write and read messages.
The definitions in a
.proto file are simple: we add a message for each data structure we want to serialize, then specify a name and a type for each field in the message. Let’s use the an address book application as an example. To define our messages, we start with the
.proto file starts with a package declaration
package tutorial, which helps to prevent name conflicting. It will by default used as the Java package unless we explicitly specify the
java_package as shown above. It is recommended to always specify a package to avoid name collisions in Protocol Buffer name spaces even in non-Java languages.
java_outer_classname option defines the class name which should contain all of the classes in this file. If we don’t give a
java_outer_classname explicitly, it will be generated by converting the file name to camel case. For example, “my_proto.proto” would, by default, use “MyProto” as the outer class name.
Next, we have the
message definitions. A message is just an aggregate containing a set of typed fields. Several basic types are available:
A more complicated structure is supported. As shown above,
Person message contains
PhoneNumber messages, while the
AddressBook message contains
Person messages. We can even define message types nested inside other messages – as we can see, the
PhoneNumber type is defined inside
Person. Enum is also supported– here we have a phone number that can be one of MOBILE, HOME, or WORK.
" = 1",
" = 2" markers on each element identify the unique “tag” that field uses in the binary encoding. Tag numbers 1-15 require one less byte to encode than higher numbers, so as an optimization we can decide to use those tags for the commonly used or repeated elements, leaving tags 16 and higher for less-commonly used optional elements. Each element in a repeated field requires re-encoding the tag number, so repeated fields are particularly good candidates for this optimization.
Each field must be annotated with one of the following modifiers:
- required: a value for the field must be provided, otherwise the message will be considered “uninitialized”. Trying to build an uninitialized message will throw a
RuntimeException. Parsing an uninitialized message will throw an
- optional: the field may or may not be set. If an optional field value isn’t set, a default value is used. For simple types, we can specify our own default value, as we’ve done for the phone number type in the example. Otherwise, a system default is used: zero for numeric types, the empty string for strings, false for bools.
- repeated: the field may be repeated any number of times (including zero). The order of the repeated values will be preserved in the protocol buffer.
We’ll find a complete guide to writing
.proto files – including all the possible field types – in the Protocol Buffer Language Guide.
Now that we have a
.proto, the next thing we need to do is generate the classes we’ll need to read and write
AddressBook messages. To do this, we need to run the protocol buffer compiler protoc on our
- If we haven’t installed the compiler, download the package.
- Now run the compiler, specifying the source directory (where our application’s source code lives with the current directory as default), the destination directory (where we want the generated code to go; often the same as the source directory), and the path to our
.proto.1protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto
Assuming we want Java classes, we use the
--java_out option here. Similar options are provided for other supported languages.
This generates com/example/tutorial/AddressBookProtos.java in our specified destination directory.
If we look in
AddressBookProtos.java, we can see that it defines a class called
AddressBookProtos, nested within which is a class for each message we specified in
addressbook.proto. Each class has its own Builder class that we use to create instances of that class.
Here are some of the accessors for the
Person.Builder has the same getters plus setters:
As we can see, there are simple JavaBeans-style getters and setters for each field.
Here is an example about how to create an instance of
To persist the data, we can simply run the following codes.
Once persisted, we can read the data as such.
For example in C++, we can write codes like:
Then later on we can read the message back like:
Check out this post for more information:)