Skip to content

RPC serialization formats comparison

Remote Procedure Call (RPC) systems often require specific serialization formats to efficiently encode data for transmission or storage, particularly in cross-language environments. Common formats include [[Protobuf]] (Protocol Buffers), [[Thrift]], and Avro^[600-developer-big-data-java-serializable.md].

Apache Thrift

Thrift is a cross-language RPC framework developed by Facebook^[600-developer-big-data-java-serializable.md]. It provides a modular architecture that separates the data serialization format from the transport layer and the server processing model.

Protocols (Formats)

Thrift supports multiple protocols to define how data is encoded^[600-developer-big-data-java-serializable.md]: * TBinaryProtocol: Uses a binary format for encoding. * TCompactProtocol: A more compact binary format. * TJsonProtocol: Uses JSON for encoding. * TSimpleJsonProtocol: A simplified JSON format (does not write out field metadata). * TDebugProtocol: A human-readable text format intended for debugging.

Transports

The transport layer defines how data is moved between nodes^[600-developer-big-data-java-serializable.md]: * TSocket: Standard socket blocking I/O. * TFramedTransport: Uses framing (sending data chunks with size prefixes). * TFileTransport: Writes to a file. * TMemoryTransport: Uses an in-memory buffer.

Server Models

Thrift offers various server models to handle requests^[600-developer-big-data-java-serializable.md]: * TSimpleServer: A simple single-threaded server. * TThreadPoolServer: Uses a thread pool to handle requests. * TNonblockingServer: Uses non-blocking I/O. * THsHaServer: Half-Sync/Half-Async server.

Protocol Buffers (Protobuf)

Protocol Buffers is a serialization format developed by Google^[600-developer-big-data-java-serializable.md]. It is designed to be smaller and faster than traditional text-based formats like XML or JSON, making it suitable for high-performance RPC systems like gRPC^[600-developer-big-data-java-serializable.md].

Other Formats

  • Avro: A schema-based serialization format often used in big data contexts (e.g., within Hadoop)[^[600-developer-big-data-java-serializable.md]].
  • XML: Historically used in SOAP-based web services^[600-developer-big-data-java-serializable.md].
  • JSON: The standard format for RESTful web services^[600-developer-big-data-java-serializable.md].

Sources

^[600-developer-big-data-java-serializable.md]