Google Protocol Buffer Tutorial

Introduction

Protocol Buffer, also known as Protobuf, is Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. It has been widely used in many applications, such as TensorFlow and gRPC. We have actually been using it without being aware, thus it has become a black box to most of the users.

In this blog post, I am going to give a short tutorial on how to use Protocol Buffer in C++.

Installation

Protobuf compiler and libraries could be installed via apt on Ubuntu. However, to use the latest version of protobuf, we install from source code.

$ cd /tmp
$ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.8.0/protobuf-all-3.8.0.tar.gz
$ tar -xf protobuf-all-3.8.0.tar.gz
$ cd protobuf-3.8.0

I chose to install the all version so it would be a little bit slow. To accelerate the installation process., we take advantage of multi-threads.

$ ./configure
$ make -j16
$ make check -j16 # Make sure the installation would not have problems
$ sudo make install
$ sudo ldconfig # Refresh shared library cache.

Protocol Buffer libraries would be installed to /usr/local/lib by default.

To verify protobuf has been installed successfully.

$ protoc --version
libprotoc 3.8.0

Tutorial

To learn Protocol Buffer, the most intuitive way is to go to the Google Protocol Buffer official website and start its official tutorial. However, the official tutorials on the Google protobuf website are still using programming standard Proto2 while the latest programming standard is Proto3 which was released in 2016. The official samples are using Proto3 and CMake, however, its CMake is not standard and many of its macros were not documented on the official CMake website. Therefore, I re-implemented Google’s official Protobuf C++ example using Proto3 and CMake with the best practice in a way that users are most likely to run into.

The source code could be found on my GitHub: Protocol Buffer Examples.

The protobuf proto file, C++ cc file, and CMake CMakeLists.txt files should be self-explanatory since I have put a lot of comments in the code to make it human-readable and easy to understand.

Basic Protocol

To summarize the basic protocol of programming using protobuf:

1. Define the structure of the data in a proto file.
2. Implement the C++ programs which serialize/de-serialize the structured data. The programs will use the header and source code files of the structured data generated by the protobuf compiler. Alternatively, the program will use the header file generated by the protobuf compiler and the static library of the structured data generated by the C++ compiler.
3. Write CMake compile system to generate the header and source code files of the structured data, static library of the structured data, and the executable files.

References

• Protocol Buffer Official Website
• Protocol Buffer C++ Tutorial
• Protocol Buffer Examples