ProtoBuf

Implements encoding and decoding classes to/from bytes using Proto2[https://developers.google.com/protocol-buffers/docs/proto] specification. It is typically used by constructing an application-specific instance, with configured specific behaviour and, if necessary, registered custom serializers (in SerializersModule provided by serializersModule constructor parameter).

Correspondence between Protobuf message definitions and Kotlin classes

Given a ProtoBuf definition with one required field, one optional field and one optional field with a custom default value:

message MyMessage {
    required int32 first = 1;
    optional int32 second = 2;
    optional int32 third = 3 [default = 42];
}

The corresponding Serializable class should match the ProtoBuf definition and should use the same default values:

@Serializable
data class MyMessage(val first: Int, val second: Int = 0, val third: Int = 42)

By default, protobuf fields ids are being assigned to Kotlin properties in incremental order, i.e. the first property in the class has id 1, the second has id 2, and so forth. If you need a more stable order (e.g. to avoid breaking changes when reordering properties), provide custom ids using ProtoNumber annotation.

By default, all integer numbers are encoded using varint[https://developers.google.com/protocol-buffers/docs/encoding#varints] encoding. This behaviour can be changed via ProtoType annotation.

Known caveats and limitations

Lists are represented as repeated fields. Because format spec says that if the list is empty, there are no elements in the stream with such tag, you must explicitly mark any field of list type with default = emptyList(). Same for maps. There's no special support for oneof protobuf fields. However, this implementation supports standard kotlinx.serialization's polymorphic and sealed serializers, using their default form (message of serialName: string and other embedded message with actual content).

Proto3 support

This implementation does not support repeated packed fields, so you won't be able to deserialize Proto3 lists. However, other messages could be decoded. You have to remember that since fields in Proto3 messages by default are implicitly optional, corresponding Kotlin properties have to be nullable with default value null.

Usage example

// Serialize to ProtoBuf bytes. Default values are omitted.
val encoded = ProtoBuf.encodeToByteArray(MyMessage(15)) // [0x08, 0x0f]

// Deserialize ProtoBuf bytes will use default values of the MyMessage class
val decoded = ProtoBuf.decodeFromByteArray<MyMessage>(encoded) // MyMessage(first=15, second=0, third=42)

// Serialize to ProtoBuf hex string with all values
val encoded2 = ProtoBuf { encodeDefaults = true }.encodeToHexString(MyMessage(15)) // "080f1000182a"

// Deserialize from ProtoBuf hex string
val decoded2 = ProtoBuf.decodeFromHexString<MyMessage>(encoded2) // MyMessage(first=15, second=0, third=42)

Check existence of optional fields

Null values can be used as the default value for optional fields to implement more complex use-cases that rely on checking if a field was set or not.

@Serializable
data class MyMessage(val first: Int, private val _second: Int? = null, private val _third: Int? = null) {

    val second: Int
        get() = _second ?: 0

    val third: Int
        get() = _third ?: 42

    fun hasSecond() = _second != null

    fun hasThird() = _third != null
}

// Serialize to ProtoBuf bytes, removing all default (null) values
val encoded = ProtoBuf.encodeToByteArray(MyMessage(15)) // [0x08, 0x0f]

// Deserialize ProtoBuf bytes
val decoded = ProtoBuf.decodeFromByteArray<MyMessage>(encoded) // MyMessage(first = 15, _second = null, _third = null)
decoded.hasSecond()     // false
decoded.second          // 0
decoded.hasThird()      // false
decoded.third           // 42

// Serialize to ProtoBuf bytes
val encoded2 = ProtoBuf.encodeToByteArray(MyMessage(15, 0, 0)) // [0x08, 0x0f, 0x10, 0x00, 0x18, 0x00]

// Deserialize ProtoBuf bytes
val decoded2 = ProtoBuf.decodeFromByteArray<MyMessage>(encoded2) // MyMessage(first=15, _second=0, _third=0)
decoded.hasSecond()     // true
decoded.second          // 0
decoded.hasThird()      // true
decoded.third           // 0